Transcoders Beat Sparse Autoencoders for Interpretability
Gonçalo Paulo, Stepan Shabalin, Nora Belrose · 2025
Read Transcoders Beat Sparse Autoencoders for Interpretability by Gonçalo Paulo, Stepan Shabalin, Nora Belrose (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18823.txt.
Introduction
Gonçalo Paulo, Stepan Shabalin, and Nora Belrose wrote this 2025 paper on reading the inner features of neural networks. Two tools compete for the job. Sparse autoencoders rewrite the activity of a network as a sparse set of named features, then rebuild the activity from them. Transcoders instead learn to predict the output of a component from its input, through a similar sparse bottleneck.
Trained on the same model and data, transcoder features proved clearly easier for humans to interpret. The team also add skip transcoders, which carry a direct affine shortcut around the bottleneck. Those rebuilt the activity with lower error while keeping the features just as readable. The results nudge interpretability work toward the transcoder design.
Catalog record
| CREATOR | Gonçalo Paulo, Stepan Shabalin, Nora Belrose |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18823
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Transcoders Beat Sparse Autoencoders for Interpretability Gonçalo Paulo, Stepan Shabalin, Nora Belrose Abstract Sparse autoencoders (SAEs) extract human-interpretable features from deep neural networks by transforming their activations into a sparse, higher dimensional latent space, and then reconstructing the activations from these latents. Transcoders are similar to SAEs, but they are trained to reconstruct the output of a component of a deep network given its input. In this work, we compare the features found by transcoders and SAEs trained on the same model and data, finding that transcoder features are significantly more interpretable. We also propose skip transcoders , which add an affine skip connection to the transcoder architecture, and show that these achieve lower reconstruction loss with no effect on interpretability. † † affiliationnotice: Equal contribution 1 Introduction Recently, large language models have achieved human-level reasoning performance in many tasks ( Guo et al. 2025 ) . Interpretability aims to improve the safety and reliability of these systems by understanding their internal mechanisms and representations. While early research attempted to produce natural language explanations of individual neurons ( Olah et al. 2020 ; Gurnee et al. 2023 ; Gurnee et al. 2024 ) , it is now widely recognized that most neurons are “polysemantic”, activating in semantically diverse contexts ( Arora et al. 2018 ; Elhage et al. 2022 ) . Sparse autoencoders (SAEs) have emerged as a promising tool for partially overcoming polysemanticity, by decomposing activations into interpretable features ( Bricken et al. 2023a ; Templeton et al. 2024b ; Gao et al. 2024 ) . SAEs are single hidden layer neural networks trained with the objective of reconstructing activations with a sparsity penalty ( Bricken et al. 2023a ; Rajamanoharan et al. 2024 ) , sparsity constraint ( Gao et al. 2024 ; Bussmann et al. 2024 ) , or an information bottleneck ( Ayonrinde et al. 2024 ) . They consist of two parts: an encoder that projects activations into a sparse, high-dimensional latent space, and a decoder that reconstructs the original activations from the latents. Bricken et al. 2023a introduced a technique of evaluating the interpretability of SAEs by simulating them with an LLM-based scorer, similar to what had been done on neurons ( Bills et al. 2023 ) . This approach is commonly called automated interpretability, or autointerp. SAE features perform much better on this benchmark compared to neurons, even when neurons are “sparsified” by selecting only the top- k most active neurons in a layer for analysis ( Paulo et al. 2024 ) . One problem with SAEs is that they focus on compressing intermediate activations rather than modeling the functional behavior of network components (e.g., feedforward modules). Figure 1: Skip transcoders are a Pareto improvement on interpretability vs performance degradation . We compare the increase in cross-entropy loss of 3 different sizes of SAEs and transcoders, 32768 (top right), 65536 (bottom left) and 131072 (bottom right), when patched into the model. For all sizes, skip transcoders are better than transcoders and sparse autoencoders, having both lower increase in model loss and a higher average auto interpretability score. On each quadrant we show 3 models that were trained with a different number of active latents, 32, 64 and 128, except for the 65536 latent model, which only has 32 and 64. The auto interp score is defined as the average fuzzing and detection score of c.a. 500 latents. Transcoders are an alternative to sparse autoencoders, initially proposed in Li et al. 2023 and Templeton et al. 2024a , and first rigorously evaluated by Dunefsky et al. 2024 . Unlike SAEs, transcoders approximate the input-output function of a target component, such as an an MLP, using a sparse bottleneck. Dunefsky et al. 2024 demonstrate that transcoders enable fine-grained circuit analysis by learning input-invariant descriptions of component behavior, complementing automated circuit discovery tools like Conmy et al. 2023 . Figure 2: Interpretability of latents and generalization of explanations. The interpretability scores of both detection and fuzzing are higher for skip transcoders and transcoders when compared to SAEs, with the distribution being wider for SAEs. Dots in the left plot indicate the average score. The accuracy of the explanations on examples sampled from different quantiles of the activation distribution we can observe that The accuracy of explanations remains higher even for lower quantiles, where the activations are smaller, showing that transcoder and skip-transcoder latents are probably representing more monosemantic concepts along the full distribution. Transcoder design faces inherent challenges. While ReLU MLPs carve up the input space into polytopes, 1 1 1 The polytope interpretation can also be applied, in a somewhat modified form, to MLPs with other activation functions ( Balestriero & Baraniuk 2019 ) . with each polytope corresponding to a relatively high-rank linear function ( Black et al. 2022 ) , transcoders’ sparse activations mean that each activation pattern corresponds to a low-rank linear transformation. Furthermore, since Marks et al. 2024 and Bricken et al. 2023b , new benchmarks for sparse feature evaluation have emerged ( Gao et al. 2024 ; Karvonen et al. 2024 ; Juang et al. 2024 ) , motivating a broader evaluation of transcoders across models and tasks. We investigate the tradeoff between reconstruction error and interpretability by comparing SAEs and transcoders and address challenges mentioned above by proposing an architectural improvement—the skip transcoder —which mitigates rank limitations via an affine skip connection. In this work, we: 1. Introduce skip transcoders, which reduce reconstruction error without compromising interpretability. 2. Compare transcoders, skip transcoders, and SAEs across diverse models (up to 2B parameters), showing skip transcoders Pareto-dominate SAEs on reconstruction vs. interpretability tradeoffs. 3. Evaluate transcoders on SAEBench ( Karvonen et al. 2024 ) demonstrating improved quality in both latent-level phenomena like absorption and performance on various tasks through sparse probing. We conclude that interpretability researchers should shift their focus away from sparse autoencoders trained on the outputs of MLPs and toward (skip) transcoders. 2 Methods Skip transcoders add a linear “skip connection” to the transcoder, which we find improves its ability to approximate the original MLP at no cost to interpretability scores. Specifically, the transcoder takes the functional form f(\boldsymbol{x})=\boldsymbol{W}_{2}\mathrm{TopK}(\boldsymbol{W}_{1}\boldsymbol{x}+\boldsymbol{b}_{1})+\boldsymbol{W}_{\mathrm{skip}}\boldsymbol{x}+\boldsymbol{b}_{2} (1) Both \boldsymbol{W}_{2} and \boldsymbol{W}_{\mathrm{skip}} are zero-initialized, and \boldsymbol{b}_{2} is initialized to the empirical mean of the MLP outputs, so that the transcoder is a constant function at the beginning of training. We leave a deeper analysis of the skip connection, perhaps interpreting it using SVD ( Millidge & Black 2022 ) , for future work. 2.1 Training We train a collection of sparse coders: sparse autoencoders (SAE), sparse transcoders (ST), and sparse skip transcoders (SST), on the MLP layers of Pythia 160M ( Biderman et al. 2023 ) . We also train SAEs and SSTs on Llama 3.2 1B and Gemma 2 2B. We train with mean squared error between the output of the sparse coder and the MLP output, with no auxiliary loss terms. Unlike prior work on transcoders, we adopt the state-of-the-art TopK activation function proposed by Gao et al. 2024 , which directly enforces a desired sparsity level on the latent activations without the need to tune an L1 sparsity penalty. We sweep across k values of 32, 64, and 128 in our experiments. For sparse