Low-Rank Adapting Models for Sparse Autoencoders

Matthew Chen, Joshua Engels, Max Tegmark · 2025

Read Low-Rank Adapting Models for Sparse Autoencoders by Matthew Chen, Joshua Engels, Max Tegmark (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-19406.txt.

Introduction

Matthew Chen, Joshua Engels, and Max Tegmark improve a tool that helps explain how language models think, in a research paper from 2025. The tool, called a sparse autoencoder, splits a model's inner signals into a small set of clear parts. Putting those parts back tends to hurt the model's word choices.

Instead of retraining the tool, the team lightly tunes the language model itself around a tool that is already trained. This cuts the loss in word accuracy by a large share in their tests. It also reaches the same quality far faster than training both parts together from scratch.

Catalog record

CREATORMatthew Chen, Joshua Engels, Max Tegmark
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-19406

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Low-Rank Adapting Models for Sparse Autoencoders Matthew Chen, Joshua Engels, Max Tegmark Abstract Sparse autoencoders (SAEs) decompose language model representations into a sparse set of linear latent vectors. Recent works have improved SAEs using language model gradients, but these techniques require many expensive backward passes during training and still cause a significant increase in cross entropy loss when SAE reconstructions are inserted into the model. In this work, we improve on these limitations by taking a fundamentally different approach: we use low-rank adaptation (LoRA) to finetune the language model itself around a previously trained SAE. We analyze our method across SAE sparsity, SAE width, language model size, LoRA rank, and model layer on the Gemma Scope family of SAEs. In these settings, our method reduces the cross entropy loss gap by 30% to 55% when SAEs are inserted during the forward pass. We also find that compared to end-to-end (e2e) SAEs, our approach achieves the same downstream cross entropy loss 3 × \times to 20 × \times faster on Gemma-2-2B and 2 × \times to 10 × \times faster on Llama-3.2-1B . We further show that our technique improves downstream metrics and can adapt multiple SAEs at once without harming general language model capabilities. Our results demonstrate that improving model interpretability is not limited to post-hoc SAE training; Pareto improvements can also be achieved by directly optimizing the model itself. 1 1 1 Code available at https://github.com/matchten/LoRA-Models-for-SAEs † † affiliationnotice: Equal contribution 1 Introduction Figure 1: Cross entropy loss vs. training time over 2B tokens for Gemma-2-2B TopK SAEs with \mathrm{width}=18,432,\mathrm{L_{0}}=64 . We find that our method (TopK + LoRA in the plot) outperforms an e2e SAE and vanilla TopK SAE. Although language models demonstrate profound capabilities in tasks such as in-context learning, mathematics, and coding ( Brown et al., 2020 ; OpenAI, 2024 ; Team et al., 2023 ; Bubeck et al., 2023 ; Anthropic, 2024 ) , the mechanisms behind these behaviors remain largely inscrutable. Mechanistic interpretability (MI) ( Bereska & Gavves, 2024 ) seeks to understand these mechanisms by reverse engineering them into human understandable algorithms. In this work, we focus on better understanding features, the “variables” of model computation ( Olah et al., 2020 ; Mueller et al., 2024 ; Marks et al., 2024 ) . A popular hypothesis in MI is the Linear Representation Hypothesis (LRH) ( Elhage et al., 2022a ; Park et al., 2023 ) , which posits that features are one-dimensional directions in activation space. Although some recent research has called aspects of this hypothesis into question ( Engels et al., 2024a ; Csordás et al., 2024 ; Engels et al., 2024b ) , the LRH has been empirically validated for many language model features “in the wild” ( Nanda et al., 2023 ; Heinzerling & Inui, 2024 ) . Inspired by these successes, sparse autoencoders (SAEs) ( Makhzani & Frey, 2013 ) have recently been applied to decompose language model hidden states into many linear features ( Cunningham et al., 2023 ; Bricken et al., 2023 ) . The latents SAEs learn are significantly more interpretable and monosemantic than the original neuron basis ( Cunningham et al., 2023 ; Bricken et al., 2023 ) . While SAEs find interpretable latents, this comes at a cost: when SAE reconstructions are inserted back into the model and the forward pass is performed, the resulting cross entropy loss ( \mathcal{L}_{\text{SAE}} ) is significantly higher than the loss of the original model ( \mathcal{L}_{\text{BASE}} ). For example, when reconstructions from a TopK SAE are inserted into GPT-4, the resulting \mathcal{L}_{\text{SAE}} is equivalent to the \mathcal{L}_{\text{BASE}} of a model trained with just 10% of the pretraining compute of GPT-4 ( Gao et al., 2024 ) . Thus, previous work has extensively focused on optimizing SAE architectures to find Pareto improvements in the SAE sparsity vs. \mathcal{L}_{\text{SAE}} frontier. This work includes TopK SAEs ( Gao et al., 2024 ) , Gated SAEs ( Rajamanoharan et al., 2024a ) , JumpReLU SAEs ( Rajamanoharan et al., 2024b ) , ProLU SAEs ( Taggart, 2024 ) , Switch SAEs ( Mudide et al., 2024 ) , and e2e SAEs ( Braun et al., 2024 ) . However, an unexplored question is whether language models themselves can be optimized after SAE training to gain an additional Pareto improvement in sparsity vs. \mathcal{L}_{\text{SAE}} . In this work, we answer this question in the affirmative: we use Low-Rank Adapters (LoRA) ( Hu et al., 2021 ) to reduce the KL divergence between the original model’s logits and the model’s logits with an SAE inserted. The resulting model-SAE combination improves in \mathcal{L}_{\text{SAE}} and on a diverse set of downstream SAE metrics. Compared to existing proposals for training more interpretable models with SAEs ( Lai & Heimersheim, 2024 ; Lai & Huang, 2024 ) , we estimate our LoRA method is 10^{7} times faster 2 2 2 Gemma-2-2B was trained with 6T tokens, whereas we use 15M tokens on up to 3% of the parameters; 6T/15M/0.03=1.3\times 10^{7} . Overall, we find that low-rank adapting models is a simple and efficient technique to improve the interpretability vs. performance trade-off. Our contributions include the following: 1. To the best of our knowledge, we are the first to focus on improving the model around an existing SAE. 2. In Section 4.1 , we analyze LoRA SAE training on the Gemma Scope ( Lieberum et al., 2024 ) family of SAEs. Across SAE width, SAE sparsity, language model size, LoRA rank, and language model layer, we find a 30\% to 55\% improvement in \mathcal{L}_{\text{SAE}} –with final values between 0.01 and 0.17 nats–with especially large improvements in low sparsity regimes and larger models. 3. In Section 4.2 , we compare our method to e2e SAEs on training time vs. \mathcal{L}_{\text{SAE}} on Gemma-2-2B ( Team et al., 2024 ) and Llama-3.2-1B ( AI@Meta, 2024 ) . We find that our method achieves the same \mathcal{L}_{\text{SAE}} as e2e SAEs with between 2 \times and 20 \times less compute and 130 \times fewer language model backward passes. 4. In Section 4.3 , we perform LoRA SAE training with multiple SAEs inserted into Llama-3.1-8B ( AI@Meta, 2024 ) and see large decreases in \mathcal{L}_{\text{SAE}} , demonstrating the potential of our technique for helping circuit analysis. 5. In Section 5 , we show quantitative improvements on a diverse set of downstream tasks. 6. In Section 6 , we find that we can achieve much of the benefit of full-parameter LoRA by training an adapter only on the layer after the SAE and that our adapters achieve most improvement on tokens with low \mathcal{L}_{\text{SAE}} . 2 Related Work SAE Architecture Improvements. Early SAEs for language models used a simple linear encoder, ReLU activation with an L_{1} penalty (approximating L_{0} ), and a linear decoder ( Bricken et al., 2023 ; Cunningham et al., 2023 ) . The next generation introduced TopK and BatchTopK SAEs, which enforce sparsity by retaining only the k largest activations ( Gao et al., 2024 ; Bussmann et al., 2024 ) , and GatedSAEs and JumpReluSAEs, which use gating functions and straight-through estimators to approximate direct L_{0} optimization ( Rajamanoharan et al., 2024a ; Rajamanoharan et al., 2024b ) . These

Read the complete text