SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data
David Chanin, Adrià Garriga-Alonso · 2026
Read SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data by David Chanin, Adrià Garriga-Alonso (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-14687.txt.
Introduction
Sparse autoencoders promise to expose the features inside language models, but benchmarks lag behind. David Chanin, with Adrià Garriga-Alonso as co-author, introduces SynthSAEBench, a large-scale synthetic benchmark with known ground-truth features. It adds correlation, hierarchy, and superposition, the traits real features show.
Current LLM benchmarks are too noisy to rank designs, while small synthetic tests lack realism. SynthSAEBench reproduces known LLM phenomena, from the reconstruction-quality split to precision-recall trade-offs. It also exposes a new failure: matching-pursuit encoders can boost reconstruction from superposition noise without learning true features. The toolkit sets a controlled floor for future designs.
Catalog record
| CREATOR | David Chanin, Adrià Garriga-Alonso |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-14687
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data David Chanin, Adrià Garriga-Alonso Abstract Improving Sparse Autoencoders (SAEs) requires benchmarks that can precisely validate architectural innovations. However, current SAE benchmarks on LLMs are often too noisy to differentiate architectural improvements, and current synthetic data experiments are too small-scale and unrealistic to provide meaningful comparisons. We introduce SynthSAEBench , a toolkit for generating large-scale synthetic data with realistic feature characteristics including correlation, hierarchy, and superposition, and a standardized benchmark model, SynthSAEBench-16k , enabling direct comparison of SAE architectures. Our benchmark reproduces several previously observed LLM SAE phenomena, including the disconnect between reconstruction and latent quality metrics, poor SAE probing results, and a precision-recall trade-off mediated by L0. We further use our benchmark to identify a new failure mode: Matching Pursuit SAEs exploit superposition noise to improve reconstruction without learning ground-truth features, suggesting that more expressive encoders can easily overfit. SynthSAEBench complements LLM benchmarks by providing ground-truth features and controlled ablations, enabling researchers to precisely diagnose SAE failure modes and validate architectural improvements before scaling to LLMs. 1 Introduction Large language models (LLMs) achieve remarkable performance but remain opaque, motivating interpretability research into how these models represent knowledge. The Linear Representation Hypothesis (LRH) ( 30 ) posits that concepts (hereafter “features”) are represented as nearly-orthogonal linear directions. Models can represent many more features than dimensions by allowing non-orthogonal directions, a phenomenon known as superposition ( 16 ) . Superposition is efficient but makes interpreting activations difficult, motivating the use of Sparse Autoencoders (SAEs) ( 5 ; 15 ) to recover underlying feature directions via sparse dictionary learning. SynthSAEBench feature characteristics Hierarchy Animal Dog Bird Poodle Husky Eagle Hawk Correlation Superposition Figure 1 : SynthSAEBench provides a large-scale synthetic data model with realistic feature characteristics including correlation, hierarchy, superposition and zipfian firing distributions, scalable to hundreds of thousands of features and realistic hidden dimension sizes. A key challenge in improving SAEs is that we lack ground-truth knowledge of the “true features” in an LLM. LLM benchmarks such as SAEBench ( 24 ) evaluate SAE performance on tasks like sparse probing ( 22 ; 23 ) , concept disentanglement ( 25 ) , and autointerpretability ( 31 ) . However, SAEBench metrics exhibit substantial noise between runs (see Appendix I ), making it difficult to evaluate small architectural improvements. Moreover, without ground-truth access, we cannot diagnose why SAEs score poorly, a critical obstacle given recent work showing that SAEs underperform supervised methods like logistic-regression probes ( 23 ) . On the other extreme, existing toy model experiments typically use fewer than 10 features with unrealistic characteristics like fully independent firings ( 34 ; 20 ; 16 ) . Studies of nontrivial firing statistics, such as feature hierarchy ( 10 ; 14 ; 7 ) or correlations ( 8 ; 9 ) , remain small-scale and use bespoke, non-standardized synthetic models that preclude direct comparison between SAE architectures. To bridge this gap, we extend the SAELens library ( 4 ) with tools for training and evaluating SAEs on large-scale synthetic data exhibiting realistic feature phenomena: correlation, hierarchy, and superposition noise. These models provide ground-truth feature directions and firings, enabling fine-grained evaluation of SAE quality. Our data generation scales to over 10,000 features at realistic hidden dimensions on a single GPU. We further release a standard benchmark model, SynthSAEBench-16k 1 1 1 https://huggingface.co/decoderesearch/synth-sae-bench-16k-v1 , with 16k features for direct comparison of SAE architectures. Our synthetic setup reproduces several previously observed LLM SAE phenomena: (1) Matryoshka SAEs overperform on SAEBench despite poor reconstruction ( 7 ; 24 ) , (2) Matching Pursuit SAEs overperform on reconstruction while scoring poorly on SAEBench ( 11 ) , (3) poor SAE probing performance ( 23 ) , and (4) a precision-recall trade-off in probing mediated by SAE L0 ( 10 ) . Additionally, using SynthSAEBench, we find that Matching Pursuit SAEs exploit superposition noise to improve reconstruction without learning ground-truth features, hinting at why the simple linear encoder of traditional SAEs is so hard to outperform in practice despite known theoretical limitations ( 29 ) . No SAE architecture we evaluate achieves perfect performance on SynthSAEBench-16k, highlighting a clear target for improvement in SAE architectures. >\tau Sample correlated firings 1 1 0 0 0 0 1 z z\odot\mathcal{N}(\mu,\sigma) Sample magnitudes 1.3 0.9 0 0 0 0 3.2 c 1 2 3 4 5 6 7 Enforce hierarchy 1.3 0.9 0 0 0 0 0 c \mathbf{D}^{\top}c+\mathbf{b} Apply feature dictionary -2.1 1.9 -0.1 0.7 a Figure 2 : Overview of process to generate a single training activation, a . 2 Background Sparse autoencoders (SAEs). An SAE decomposes an input activation a\in\mathbb{R}^{D} into a hidden state f consisting of L hidden neurons, called “latents”. An SAE is composed of an encoder W_{\text{enc}}\in\mathbb{R}^{L\times D} , a decoder W_{\text{dec}}\in\mathbb{R}^{D\times L} , a decoder bias b_{\text{dec}}\in\mathbb{R}^{D} , and encoder bias b_{\text{enc}}\in\mathbb{R}^{L} , and a nonlinearity \sigma , typically ReLU or a variant like JumpReLU ( 32 ) , TopK ( 19 ) or BatchTopK (BTK) ( 6 ) . \displaystyle f= \displaystyle\sigma(W_{\text{enc}}(a-b_{\text{dec}})+b_{\text{enc}}) (1) \displaystyle\hat{a}= \displaystyle W_{\text{dec}}f+b_{\text{dec}} (2) The SAE is trained with a reconstruction loss, typically Mean Squared Error (MSE), and a sparsity-inducing loss consisting of a function \mathcal{S} that penalizes non-sparse representation with corresponding sparsity coefficient \lambda . For standard L1 SAEs, \mathcal{S} is the L1 norm of f . For TopK and BatchTopK SAEs, there is no sparsity-inducing loss ( \mathcal{S}=0 ) as the TopK function directly induces sparsity. There is sometimes also an additional auxiliary loss \mathcal{L}_{aux} with coefficient \alpha to ensure all latents fire. Standard L1 SAEs typically do not have an auxiliary loss ( 28 ) . The general SAE loss is \mathcal{L}=\|a-\hat{a}\|_{2}^{2}+\lambda\mathcal{S}+\alpha\mathcal{L}_{\text{aux}}. (3) Matryoshka SAEs. A matryoshka SAE ( 7 ) extends the SAE definition by summing losses created by prefixes of SAE latents. This forces each sub-SAE to reconstruct input activations on its own, and incentivizes the SAE to place more common, general concepts into latents with smaller index number. A matryoshka SAE uses nested prefixes with sizes \mathcal{M}=m_{1},m_{2},...m_{n} where m_{1}<m_{2}<\ldots<m_{n}=L , where L is the number of latents in the full dictionary. Matryoshka SAE loss is: \mathcal{L}=\sum_{m\in\mathcal{M}}\left(\|a-\hat{a}_{m}\|_{2}^{2}+\lambda\mathcal{S}_{m}\right)+\alpha\mathcal{L}_{\text{aux}} (4) Where \hat{a}_{m} is the reconstruction for the SAE using the first m latents, and \mathcal{S}_{m} is the sparsity penalty applied to the first m latents. For TopK and BatchTopK Matryoshka SAEs, there is no sparsity penalty ( \mathcal{S}_{m}=0 ) as the TopK function directly imposes sparsity. Matching Pursuit SAEs. A Matching Pursuit (MP) SAE ( 14 ) acts like a TopK SAE where the k latents are selected in serial rather than in parallel. In an MP-SAE, there is no explicit encoder W_{\text{enc}} or encoder bias b_{\text{enc}} . Instead, at each iteration t\in\{0,\dots,k\} , the latent with the highest projection onto the residual \hat{a}_{t} is selected and projected out. \displaystyle l_{t}=\text{argmax}_{i}W_{\text{dec},i}\cdot\hat{a_{t}} (5) \displaystyle\hat{a}_{t+1}=\hat{a}_{t}-W_{\text{dec},l_{t}}\cdot\hat{a}_{t} (6) Where \hat{a}_{0}=a . The reconstruction loss for the MP-SAE is then calculated as follows: \mathcal{L}=\|\hat{a}_{k}\|_{2}^{2}. (7) The variant of MP-SAEs we use in this paper does not do any early stopping based on \|\hat{a}\|_{2} or based on selecting the same latent multiple times, as early stopping adds more complication to the training process and has not been shown to improve results ( 11 ) .