Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features

Jordan F. McCann · 2026

Read Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features by Jordan F. McCann (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12874.txt.

Introduction

McCann spots a flaw in how tools that split language-model activity into readable features get labelled. The flaw, called descriptive collision, is that many distinct features share one plain-language label. Checking 722 hand-labelled features, he finds each label repeats across 3.07 features on average, while one label, 'plural nouns', covers 101 features. He shows current scoring misses the flaw and offers two new scores that punish labels which fail to tell features apart.

Catalog record

CREATORJordan F. McCann
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE26798
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.12874

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features Jordan F. McCann Abstract Sparse autoencoders (SAEs) are now standard tools for decomposing language model activations into interpretable features, and automated interpretability pipelines routinely assign each feature a short natural-language explanation. Existing critiques of this practice focus on polysemanticity—one feature with many meanings—or on whether explanations predict activations. We identify a complementary, structurally distinct problem we call descriptive collision : many distinct SAE features admit the same explanation. Reanalyzing the largest publicly-available dataset of human-annotated SAE features [ 21 ] , comprising 722 annotated features across Gemma 2 2B and Pythia 70M, we find that the mean annotation string is reused across 3.07 features; 82.1% of features share their annotation with at least one other feature; and the single most common annotation string (“plural nouns”) labels 101 distinct features spanning 18 layers and four model components. Information-theoretically, the average annotation resolves only 70% of feature identity. We formalize a property called discrimination , prove that current detection-style auto-interpretability scoring is invariant to collision, and propose two complementary corrective metrics—collision-adjusted detection and discrimination scoring—that explicitly penalize explanations that fail to distinguish a feature from its neighbours. The collision problem is independent of, and additive with, previously identified failure modes of auto-interpretability; ignoring it inflates reported feature interpretability by a quantity equal to roughly one-third of the bits required to identify a feature. 1 Introduction Sparse autoencoders [ 4 , 7 , 8 , 11 , 17 , 26 ] have become the dominant tool for decomposing neural network activations into a large dictionary of putatively monosemantic features. Because modern SAEs routinely produce 10^{5} – 10^{7} features per model, manual inspection is infeasible, and auto-interpretability pipelines—in which a language model is prompted with a feature’s top-activating examples and asked to generate a short natural-language description—have become the workhorse of practical feature labelling [ 3 , 18 , 24 ] . Auto-interp explanations are then scored by checking whether the explanation predicts when the feature fires on held-out text [ 3 , 22 , 24 ] . This evaluation paradigm has accumulated a productive critical literature. Huang et al. [ 13 ] showed that high explanation scores do not necessarily predict intervention effects. Ma et al. [ 19 ] formalized a recall bias in detection-style scoring and proposed precision-aware counterexamples. Paulo and Belrose [ 22 ] introduced scoring methods that bypass natural-language explanations entirely. Heap et al. [ 12 ] showed that auto-interpretability metrics yield similar scores for SAEs trained on randomly-initialized transformers as for those trained on real ones. Korznikov et al. [ 15 ] showed that random baselines match trained SAEs on interpretability scoring. Han et al. [ 10 ] argued that single-explanation pipelines miss polysemanticity within individual features. All of these critiques share an implicit framing: they take the feature as fixed and ask whether the explanation is adequate. This paper points out a complementary problem that goes the other direction. If an explanation is adequate—if it does describe what the feature responds to—we still cannot conclude that the explanation identifies the feature, because many other features may admit the same explanation. We call this the descriptive collision problem. It is the dual of polysemanticity: where polysemanticity describes a one-feature-to-many-meanings relationship, collision describes a many-features-to-one-explanation relationship. Both undermine the implicit contract of auto-interpretability, but they require separate empirical and methodological treatment. Contributions. 1. We define descriptive collision formally and distinguish it from polysemanticity, recall bias, and the random-baseline problem. 2. Reanalyzing the human-annotated feature corpus released with Marks et al. [ 21 ] , we provide direct empirical evidence that collision is pervasive: a single annotation string labels 101 distinct Pythia 70M and Gemma 2 2B features spanning 18 layers and four model components. 3. We prove a simple proposition: detection-style auto-interpretability scores are invariant under arbitrary substitution of any collision-equivalent feature, so high scores cannot certify identification. 4. We propose discrimination scoring and collision-adjusted detection, two complementary corrective measures, give pseudocode, and discuss their computational cost. 5. We argue, with reference to Goodhart’s law [ 9 , 25 ] , that collision is a generic failure mode of any explanation-scoring scheme whose target space is much smaller than the feature space being explained. Scope and caveats. This is a methodological critique. We do not retrain SAEs, generate new auto-interp explanations, or evaluate explanation quality with an LLM judge in this work; producing such large-scale evidence is left to follow-up work with model-API access. Our positive empirical claims are restricted to publicly-released human-annotated feature labels, which serve as a clean, low-noise instance of the more general phenomenon. We argue throughout that the same problem must, by construction, arise more severely for machine-generated labels, since automated explainers operate in an even narrower descriptive register than expert humans. 2 Related Work Auto-interpretability and its critiques. The methodology was introduced for individual GPT-2 neurons by Bills et al. [ 3 ] and extended to SAE features by Bricken et al. [ 4 ] , Templeton et al. [ 26 ] , and others. Paulo et al. [ 24 ] systematized scoring approaches—detection, fuzzing, simulation, and intervention—and released open-source tooling. Critiques have proliferated in three families. Baseline critiques [ 12 , 14 , 15 ] show that random or degenerate baselines achieve scores comparable to real SAEs, and that SAEs often underperform simpler probing baselines on downstream tasks. Precision critiques [ 19 ] attack the recall bias of detection scoring and propose semantically-matched negatives. Causal critiques [ 13 , 20 ] note that explanations that predict activations need not predict the effect of intervening on the feature. Our contribution is orthogonal to all three: collision affects the mapping from explanations to features rather than the mapping from features to scores. Polysemanticity, feature instability, and resolution. Leask et al. [ 16 ] and Paulo and Belrose [ 23 ] document that nominally identical SAE training pipelines yield different feature sets, suggesting that the SAE feature is not a canonical object. Chanin et al. [ 6 ] and Bussmann et al. [ 5 ] study feature splitting and absorption, in which features fragment or fuse across SAE widths. Han et al. [ 10 ] addresses polysemanticity by maintaining multiple parallel explanations per feature. Collision is structurally distinct from all these: it is a property of the label space, not of the feature dictionary or its training dynamics. Even a perfectly identified, atomic, monosemantic feature can collide descriptively with many others. Information-theoretic perspectives. Ayonrinde et al. [ 2 ] argued for evaluating SAEs by description length under an MDL principle. Our notion of discrimination can be read as a particular operationalization of the same intuition—explanations are good only insofar as they distinguish features from each other—but applied to the evaluation of natural-language labels rather than to SAE training. 3 Descriptive Collision: Definitions Let \mathcal{F}=\{f_{1},\ldots,f_{N}\} be a set of SAE features and let E:\mathcal{F}\to\Sigma^{*} be an explanation function that assigns each feature a finite string over some vocabulary \Sigma . Auto-interpretability assumes that E is informative about f in the sense that an observer who reads E(f) learns something useful about what f represents. Definition 1 (Collision cluster) . For an explanation function E , the

Read the complete text