Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics
Iyad Ait Hou, Rebecca Hwa · 2026
Read Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics by Iyad Ait Hou, Rebecca Hwa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00443.txt.
Description
Ait Hou and Hwa probe a confound in neuron interpretability. When one neuron fires for both lender and riverside, standard metrics blame superposition, the squeezing of two unrelated ideas into one unit. The authors show much of the overlap comes from shared word forms like bank, not compressed concepts. Across models from 110 million to 70 billion parameters, the lexical-only case beats the semantic-only case. The confound reaches sparse autoencoders and downstream tasks, and filtering it out sharpens word-sense disambiguation and knowledge edits.
Catalog record
| CREATOR | Iyad Ait Hou, Rebecca Hwa |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46415 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.00443