Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, Zeynep Akata · 2025
Read Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models by Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, Zeynep Akata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-02821.txt.
Introduction
Mateusz Pach and four co-authors carry a tool for opening up language models over to vision-language models. Their 2025 paper uses sparse autoencoders, networks that rebuild a model's inner activity from a wide set of single-meaning features. In language models these features make neurons readable and steerable, both prized for safety work. Whether the same holds for visual neurons was open.
The team built a framework for judging whether single visual neurons are monosemantic, meaning each responds to one clear concept. To keep the judging honest, their benchmark comes from a large study of human perception. Trained on models such as CLIP, the autoencoders sharply raised the single-meaning quality of visual neurons, with sparsity and wide hidden layers mattering most. The team then showed the features can steer a multimodal model's answers directly, with no change to the language model itself.
Catalog record
| CREATOR | Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, Zeynep Akata |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-02821
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models Mateusz Pach, Shyamgopal Karthik, Quentin Bouniot, Serge Belongie, Zeynep Akata Abstract Sparse Autoencoders (SAEs) have recently gained attention as a means to improve the interpretability and steerability of Large Language Models (LLMs), both of which are essential for AI safety. In this work, we extend the application of SAEs to Vision-Language Models (VLMs), such as CLIP, and introduce a comprehensive framework for evaluating monosemanticity at the neuron-level in visual representations. To ensure that our evaluation aligns with human perception, we propose a benchmark derived from a large-scale user study. Our experimental results reveal that SAEs trained on VLMs significantly enhance the monosemanticity of individual neurons, with sparsity and wide latents being the most influential factors. Further, we demonstrate that applying SAE interventions on CLIP’s vision encoder directly steers multimodal LLM outputs (e.g., LLaVA), without any modifications to the underlying language model. These findings emphasize the practicality and efficacy of SAEs as an unsupervised tool for enhancing both interpretability and control of VLMs. Code and benchmark data are available at https://github.com/ExplainableML/sae-for-vlm . 1 Introduction In recent years, Vision-Language Models (VLMs) like CLIP [ 47 ] and SigLIP [ 61 ] have gained widespread adoption, owing to their capacity for simultaneous reasoning over visual and textual modalities. They have found a surge of applications in various modalities, such as in audio [ 14 , 57 ] and medicine [ 62 ] , transferring to new tasks with minimal supervision. Yet our current understanding of VLM internals remains limited, necessitating methods that can systematically probe their representations. Sparse AutoEncoders (SAEs) [ 37 ] are an effective approach to probing the internal representations of such models. They efficiently discover concepts (abstract features shared between data points) through their simple architecture learned as a post-hoc reconstruction task. Although analysis with SAEs is popular for Large Language Models (LLMs) [ 7 , 48 , 23 ] , for VLMs it has been limited to interpretable classification [ 49 , 35 ] , or the discovery of concepts shared across models [ 53 ] . Intuitively, SAEs reconstruct activations via a higher-dimensional space to disentangle distinct concepts from their overlapping representations in neural activations [ 7 ] . Neurons at different layers within deep neural networks are known to be naturally polysemantic [ 41 ] , meaning that they can be strongly activated for multiple unrelated concepts such as cellphones and rulers. One common explanation for this behavior is the superposition hypothesis [ 3 , 17 ] , stating that concepts are encoded as linear combination of neurons. SAEs explicitly attempt to solve this issue by separating the entangled concepts into distinct representations. Despite their widespread use in research, the absence of a metric to evaluate SAEs at the neuron-level still hinders their practicality as an interpretation tool. Discovering neurons encoding human-interpretable concepts requires analyzing them individually, which is even more tedious as layers of both the model and SAE are wide. Figure 1 : Sparse Autoencoder (SAE) in VLM (e.g. CLIP): Top activating images of a neuron in a pretrained VLM layer are polysemantic (left), and those of a neuron in a sparse latent of SAE trained to reconstruct the same layer are monosemantic (right), according to MonoSemanticity score (MS). In this work, we quantitatively evaluate SAEs for VLMs through monosemanticity , defined as the similarity between inputs that strongly activate a neuron. We propose the MonoSemanticity score (MS) for vision tasks, that measures the pairwise similarity of images weighted by the activations for a given neuron. Unlike natural language where individual words require surrounding context (such as complete sentences) to clearly identify their meanings, individual images can directly activate neurons without additional context. We define highly activating images as images that strongly fire a particular neuron. Greater similarity among these images suggests that the neuron is more narrowly focused on a single concept, reflecting higher neuron monosemanticity. Using our MS score, we observe and validate that neurons in SAE are significantly more monosemantic (see Figure 1 , right) than the original neurons (see Figure 1 , left). The neuron of the original VLM typically has a low MS score, it fires for a wide range of objects, from cellphones to rulers . On the other hand, neurons within the SAE are more focused on a single concept, e.g. parrots , obtaining a higher score. This holds even for SAE with the same width as the original layer, implying that the sparse reconstruction objective inherently improves the separability of concepts. We further conduct a large-scale study to quantitatively assess alignment of our proposed MS score with human interpretation of monosemanticity. The results confirm that the difference between scores of two neurons strongly correlates with humans perceiving the higher-scoring neuron as more focused on a single concept. Finally, we illustrate applicability of the monosemanticity of vision SAEs by transferring a CLIP-based SAE onto Multimodal LLMs (MLLMs), e.g. LLaVA [ 36 ] . Intervening on a single monosemantic SAE neuron in the vision encoder while keeping the LLM untouched allows steering the overall MLLM generated output to either insert or suppress the concept encoded in the selected SAE neuron. We summarize our contributions as follows: • We propose the MonoSemanticity score (MS) for SAEs in vision tasks, that computes neuron-wise activation-weighted pairwise similarity of image embeddings. To validate our MS score against human judgment, we conduct a large-scale user study, the results of which can also serve as a benchmark for future research. • We quantitatively compare MS between SAEs, and across their neurons. We find that Matryoshka SAE [ 39 , 9 ] achieves overall superior MS, and that wider and sparser latents lead to better scores. • We leverage the well-separability of concepts in SAE layers to intervene on neuron activations and steer outputs of MLLMs to insert or suppress any discovered concept. Figure 2 : Computation of our MonoSemanticity score (MS). (a) Embeddings and activations are extracted for a set of images (b) to compute the pairwise embedding similarities and pairwise neuron activations. (c) MS is the average of embedding similarities weighted by the neuron activations. 2 Related Work Sparse Autoencoders. Recent studies have repurposed traditional dictionary learning to enhance LLM and VLM interpretability [ 6 , 46 ] . Specifically, there has been success in interpreting and steering LLMs with features learned by SAEs [ 52 , 16 ] . Several enhancements to SAE mechanisms have been introduced, including new activation functions such as Batch TopK [ 8 ] or JumpReLU [ 48 ] , and insights from Matryoshka representation learning [ 34 ] for SAEs [ 39 , 9 ] . We analyze and evaluate neuron-level monosemanticity of SAEs in VLMs and their downstream uses. Vision-Language Models. Since Contrastive Language–Image Pretraining (CLIP) [ 47 , 38 , 61 ] , many models have emerged that align images and text in a shared embedding space [ 56 , 32 ] or generate text conditioned on image inputs [ 36 , 12 , 19 ] . They have achieved strong results on benchmarks [ 59 ] and found many use-cases [ 24 , 5 , 26 , 28 ] . As trust in these models has become