Evaluating Sparse Autoencoders for Monosemantic Representation

Moghis Fereidouni, Muhammad Umair Haider, Peizhong Ju, A.B. Siddique · 2025

Read Evaluating Sparse Autoencoders for Monosemantic Representation by Moghis Fereidouni, Muhammad Umair Haider, Peizhong Ju, A.B. Siddique (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15094.txt.

Introduction

Fereidouni and three co-authors test tools that aim to make language models easier to read, in a paper from twenty twenty-five. Inside a model, one unit can fire for many unrelated ideas at once. That mix makes the model hard to explain. Sparse tools spread the signal into more units, each tied to one idea.

The team builds a score for how cleanly units split ideas apart. They test two large models and several sparse tools on five data sets. The sparse tools split ideas more cleanly than the base models. They also allow finer control when a single idea is damped down. Their new damping method hurt overall skill the least while still removing the target idea.

Catalog record

CREATORMoghis Fereidouni, Muhammad Umair Haider, Peizhong Ju, A.B. Siddique
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15094

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating Sparse Autoencoders for Monosemantic Representation Moghis Fereidouni, Muhammad Umair Haider, Peizhong Ju, A.B. Siddique Abstract A key barrier to interpreting large language models is polysemanticity, where neurons activate for multiple unrelated concepts. Sparse autoencoders (SAEs) have been proposed to mitigate this issue by transforming dense activations into sparse, more interpretable features. While prior work suggests that SAEs promote monosemanticity, there has been no quantitative comparison with their base models. This paper provides the first systematic evaluation of SAEs against base models concerning monosemanticity. We introduce a fine-grained concept separability score based on the Jensen–Shannon distance, which captures how distinctly a neuron’s activation distributions vary across concepts. Using Gemma-2-2B and multiple SAE variants across five benchmarks, we show that SAEs reduce polysemanticity and achieve higher concept separability. However, greater sparsity of SAEs does not always yield better separability and often impairs downstream performance. To assess practical utility, we evaluate concept-level interventions using two strategies: full neuron masking and partial suppression. We find that, compared to base models, SAEs enable more precise concept-level control when using partial suppression. Building on this, we propose Attenuation via Posterior Probabilities (APP), a new intervention method that uses concept-conditioned activation distributions for targeted suppression. APP outperforms existing approaches in targeted concept removal. Introduction Large language models (LLMs) have achieved remarkable performance across a wide range of natural language tasks, often matching or surpassing human-level performance ( Luo et al. 2025 ; Achiam et al. 2023 ; Touvron et al. 2023 ; Guo et al. 2025 ; Eslamian and Cheng 2025 ) . Nonetheless, understanding how these models internally represent and manipulate concepts remains a major challenge. A key obstacle is polysemanticity: the phenomenon where individual neurons respond to multiple, semantically distinct concepts rather than encoding single, interpretable features ( Janiak, Mathwin, and Heimersheim 2023 ; Olah, Mordvintsev, and Schubert 2017 ; Nguyen, Yosinski, and Clune 2016 ) . This entanglement complicates the interpretation and analysis of model behavior, posing a significant barrier to building transparent and controllable AI systems ( Sharkey et al. 2025 ; Marshall and Kirchner 2024 ; Bereska and Gavves 2024 ) . Dictionary learning via sparse autoencoders (SAEs) ( Huben et al. 2024 ; Gao et al. 2025 ) has recently emerged as a promising approach to mitigating polysemanticity in neural representations. SAEs aim to transform dense activations of a desired component of the base LLM into sparse features by enforcing sparsity and encouraging each neuron to specialize in distinct, concept-specific features ( Huben et al. 2024 ; Rajamanoharan et al. 2024a ; Rajamanoharan et al. 2024b ; Gao et al. 2025 ) . The goal is to produce monosemantic representations, where individual neurons respond to single, well-defined concepts ( Huben et al. 2024 ; Rajamanoharan et al. 2024a ; Rajamanoharan et al. 2024b ; Gao et al. 2025 ) . The underlying hypothesis is intuitive: if we can force the model to use fewer neurons simultaneously, each active neuron should correspond to a more distinct and interpretable concept. Empirical studies have shown that SAEs can uncover interpretable features across domains such as vision and language, facilitating improved interpretability ( Shu et al. 2025 ; Huben et al. 2024 ; Pach et al. 2025 ) . However, most of the evidence for the interpretability benefits of SAEs is qualitative, relying on case studies or anecdotal neuron visualizations, offering limited insight into their systematic effectiveness ( Kissane et al. 2024 ; Li et al. 2025 ) . While SAEs have higher computational costs and degraded downstream performance compared to their dense counterparts ( Huben et al. 2024 ) , no work has quantitatively compared their internal representations for monosemanticity: a core requirement for enhancing model interpretability. Without such comparison, the practical and theoretical value of sparsity remains poorly understood. In this work, we conduct a systematic investigation into the effectiveness of SAEs in promoting monosemanticity in the internal representations of LLMs. In contrast to prior work that evaluates SAEs in isolation, we compare their representations to those of the corresponding base models, providing the first quantitative assessment of how sparsity affects concept disentanglement. Particularly, we conduct comprehensive evaluations using Gemma-2-2B ( Rivière et al. 2024 ) and various SAEs of different widths and sparsity levels using five benchmark datasets. We begin by quantifying polysemanticity using overlap statistics, measuring the fraction of salient neurons that respond to multiple, semantically distinct concepts. While SAEs exhibit lower polysemanticity than their base models, this overlap-based analysis remains coarse-grained: it treats all neurons that respond to multiple concepts as equally entangled, without considering how their activations vary across those concepts. In practice, a neuron may activate for several concepts, yet do so with clearly distinct activation distributions, suggesting behavior that may still be considered monosemantic. That is, monosemanticity is not solely about binary activation overlap, but rather about the separability of a neuron’s activation distributions across concepts. We formalize this view by introducing a new concept separability score, based on the Jensen–Shannon distance ( Lin 1991 ) . This fine-grained, distribution-aware metric quantifies how well a neuron’s activations separate across different concepts by measuring the distance between their activation distributions. Using this score, we find that SAEs exhibit higher concept separability than their dense counterparts. Interestingly, we also observe that increasing sparsity does not always lead to better separability, and often comes at the cost of degraded downstream performance. To further evaluate the practical utility of monosemantic representations of SAEs, we examine their effectiveness in enabling concept-level model interventions. Specifically, we assess how precisely concept-related behavior can be suppressed in SAEs compared to base models. We evaluate two intervention strategies: full neuron masking, which suppresses all activations of salient neurons associated with a target concept, and partial suppression, which intervenes in activations selectively based on their distributional association with the concept. Across SAEs and the base model, partial suppression outperforms full masking in most cases, achieving more effective suppression of the target concept while better preserving unrelated model behavior. Furthermore, our results show that SAEs support more precise and effective concept removal than their dense counterparts when applying partial suppression methods. These findings reinforce the idea that concept separability, when defined in terms of activation distributions rather than binary activation overlap, offers better model control and interpretability. Additionally, motivated by the varying separability of concept activations across neurons, we introduce Attenuation via Posterior Probabilities (APP), a new intervention method that leverages concept-conditioned activation distributions to selectively suppress target concepts with minimal side effects. Specifically, APP computes the posterior probability that a given activation corresponds to a target concept and attenuates it accordingly. Among all methods evaluated, APP achieves the strongest and most targeted concept removal across both SAEs and base models. In summary, this work makes the following contributions: • We present the first quantitative analysis of monosemantic representations in SAEs relative to their base LLM. • We introduce a concept separability score, based on the Jensen-Shannon distance, a fine-grained, distribution-aware metric that captures how well neuron activations separate for different concepts. • We propose a new intervention method, a minimally invasive concept erasure technique, outperforming existing methods in both precision and side-effect minimization. Preliminaries Neuron. A neuron refers to a component of a hidden state vector

Read the complete text