Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics

Iyad Ait Hou, Rebecca Hwa · 2026

Read Polysemanticity or Polysemy? Lexical Identity Confounds Superposition Metrics by Iyad Ait Hou, Rebecca Hwa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-00443.txt.

Description

Ait Hou and Hwa probe a confound in neuron interpretability. When one neuron fires for both lender and riverside, standard metrics blame superposition, the squeezing of two unrelated ideas into one unit. The authors show much of the overlap comes from shared word forms like bank, not compressed concepts. Across models from 110 million to 70 billion parameters, the lexical-only case beats the semantic-only case. The confound reaches sparse autoencoders and downstream tasks, and filtering it out sharpens word-sense disambiguation and knowledge edits.

Catalog record

CREATORIyad Ait Hou, Rebecca Hwa
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46415
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.00443

Browse the full library · About the Tower of Babel