Measuring Sparse Autoencoder Feature Sensitivity

Claire Tian, Katherine Tian, Nathan Hu · 2025

Read Measuring Sparse Autoencoder Feature Sensitivity by Claire Tian, Katherine Tian, Nathan Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23717.txt.

Introduction

A sparse autoencoder feature that fires on neat examples may still fail on near-twins of those examples. Claire Tian and 2 co-authors measure that reliability, which they call feature sensitivity. Instead of writing descriptions of each feature, they ask language models to generate fresh text with the same semantic traits as a feature's activating examples. Then they check whether the feature fires on the new text.

Many features that look interpretable scored poorly on sensitivity. Human checks confirmed the generated texts really did resemble the originals, so the failures belong to the features. Across seven autoencoder variants, average sensitivity fell as width grew, adding a new axis for judging these tools.

Catalog record

CREATORClaire Tian, Katherine Tian, Nathan Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23717

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measuring Sparse Autoencoder Feature Sensitivity Claire Tian, Katherine Tian, Nathan Hu Abstract Sparse Autoencoder (SAE) features have become essential tools for mechanistic interpretability research. SAE features are typically characterized by examining their activating examples, which are often “monosemantic" and align with human interpretable concepts. However, these examples don’t reveal feature sensitivity : how reliably a feature activates on texts similar to its activating examples. In this work, we develop a scalable method to evaluate feature sensitivity. Our approach avoids the need to generate natural language descriptions for features; instead we use language models to generate text with the same semantic properties as a feature’s activating examples. We then test whether the feature activates on these generated texts. We demonstrate that sensitivity measures a new facet of feature quality and find that many interpretable features have poor sensitivity. Human evaluation confirms that when features fail to activate on our generated text, that text genuinely resembles the original activating examples. Lastly, we study feature sensitivity at the SAE level and observe that average feature sensitivity declines with increasing SAE width across 7 SAE variants. Our work establishes feature sensitivity as a new dimension for evaluating both individual features and SAE architectures. \workshoptitle Mechanistic Interpretability † † footnotetext: Contact: [email protected] , [email protected] , [email protected] † † footnotetext: Code: https://github.com/nathanhu0/sae-sensitivity 1 Introduction Sparse Autoencoders (SAEs) have emerged as a powerful technique to identify meaningful directions in language model activation spaces [ 8 , 37 ] . These learned directions, or SAE features, have proven to be valuable for mechanistic interpretability. Use cases include: surfacing surprising information present in model activations [ 37 , 12 ] , controlling model behavior via activation steering [ 10 , 31 ] , identifying computational circuits within models [ 1 , 28 , 25 ] , and more open-ended exploration of training data [ 29 ] or other datasets [ 30 , 18 ] . A key step in almost all SAE applications is to first characterize each SAE feature. This is commonly done by examining example inputs that activate each feature. These activating examples are often cohesive and correspond to human-interpretable concepts [ 8 , 37 ] , e.g., "harmful requests". However, only examining a feature’s activating examples tells us what a feature does but not what it fails to do. We might hope that a harmful request feature activates on all harmful requests, but we cannot determine this by just examining activating text. Additionally, we need to evaluate feature sensitivity : the probability that a feature activates on texts similar to its activating examples. Ideally, features would have high sensitivity—consistently activating on all relevant inputs rather than arbitrary subsets. Understanding a feature’s sensitivity is crucial for scoping what we can learn from the feature. If a harmful request feature has high sensitivity and activates on all harmful requests, understanding its role can reveal how the model generally processes any harmful input. If, instead, the harmful request feature has poor sensitivity, we are mainly gaining narrower insights into how the model handles the specific input that activates the feature. In this work, we use a generation-based approach to evaluate feature sensitivity at scale. As illustrated in Figure 1 , we use language models to generate text with the same semantic properties as a feature’s activating examples. We then test whether the feature activates on these generated texts. Our generation-based approach is more scalable and efficient than previous dataset filtering methods [ 37 , 38 ] . Additionally, our method avoids the need to first generate a description of the feature’s activating text, removing a potential source of error compared to common automated interpretability evaluations [ 33 , 21 ] . Our main contributions are: • We develop an explanation-free, scalable automated evaluation for SAE feature sensitivity , allowing efficient evaluation of thousands of SAE features. • We demonstrate that sensitivity measures a new facet of feature quality by examining its relationship to standard SAE feature metrics. Notably, we find that many interpretable features have poor sensitivity. • We validate our method through automated and human evaluations , finding that when a feature fails to activate on generated text, that text genuinely resembles activating text examples according to human assessment. • We identify declining feature sensitivity as an additional challenge for SAE scaling . We find that wider SAEs have lower average feature sensitivity in large-scale SAEs (up to 1M features) and across 7 different SAE variants. Figure 1 : Sensitivity evaluation methodology. We extract top activating texts for each SAE feature, use GPT-4.1 to generate similar texts based on these examples, and measure how often the feature activates on the generated texts. Features with high sensitivity reliably activate on semantically similar inputs. 2 Related Work 2.1 Prior Investigations of Feature Sensitivity Investigating feature sensitivity requires obtaining candidate input text and checking for feature activation. Most prior work approaches this by first generating natural language explanations for features, then using those explanations to identify candidate inputs. This includes using explanations to generate new text [ 16 , 19 ] or to filter through existing datasets for relevant passages [ 37 , 38 ] . Alternative approaches avoid natural language explanations entirely. Gao et al. 2024 fit n-grams with wildcards to activating text examples to filter datasets for test inputs. Other work evaluates whether features or groups of features can serve as high-sensitivity classifiers for a set of predefined concepts [ 20 , 26 , 6 ] . Chanin et al. 2024 study feature absorption, a special instance of poor feature sensitivity with a clear cause: when features form hierarchies, sparsity incentivizes parent features (e.g., "math") to fail to activate on inputs when a more specific child feature (e.g., "algebra") activates instead. All these approaches evaluate sensitivity with respect to some intermediate description—whether explanations, n-grams, or concept lists. Our approach evaluates sensitivity without needing to first generate such descriptions. 2.2 SAE Evaluation Earlier work primarily evaluated SAEs by their reconstruction error and the interpretability of individual features [ 4 , 37 ] . Although increasing SAE width improves both reconstruction quality and feature interpretability [ 21 ] , a growing body of research investigates problems that arise when scaling SAEs, including feature splitting [ 4 ] , feature absorption [ 6 ] , and feature composition [ 23 ] . These results highlight that only optimizing for sparsity and reconstruction may not yield natural features. Another line of work evaluates SAE latents by their utility for downstream tasks: sparse probing [ 14 ] , spurious correlation removal [ 28 ] , disentangling model representations [ 17 ] , and unlearning [ 11 ] . Karvonen et al. 2025 introduce SAEBench, a benchmark that aggregates many of these evaluation approaches, along with standard automated interpretability and reconstruction metrics. 2.3 Automated Interpretability The standard auto-interpretability pipeline involves collecting activating text examples for a feature, prompting an LLM to generate natural language descriptions from these examples, and validating these descriptions by testing whether they enable another LLM to predict activations on new text. Bills et al. 2023 first proposed this approach for neurons, and it has since become standard for both neuron explanations [ 7 ] and SAE explanations [ 33 , 37

Read the complete text