SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders
Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek · 2026
Read SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders by Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06610.txt.
Introduction
Jakub Stępień, Marcin Mazur, Jacek Tabor, and Przemysław Spurek propose SoftSAE, a sparse autoencoder that chooses how many features to activate for each input. Sparse autoencoders unpack neural activations into understandable features for interpretability research. Fixed top-k versions force the same feature count on every input, though simple inputs need fewer features and complex ones need more.
A differentiable soft top-k operator learns an input-dependent sparsity level, so explanation length tracks the information in each sample. Experiments confirm the model finds meaningful features and picks sensible counts per concept. The work targets language models and vision transformers, where fixed sparsity can add noise or miss structure.
Catalog record
| CREATOR | Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.06610
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoders Jakub Stępień, Marcin Mazur, Jacek Tabor, Przemysław Spurek Abstract Sparse Autoencoders (SAEs) have become an important tool in mechanistic interpretability, helping to analyze internal representations in both Large Language Models (LLMs) and Vision Transformers (ViTs). By decomposing polysemantic activations into sparse sets of monosemantic features, SAEs aim to translate neural network computations into human-understandable concepts. However, common architectures such as TopK SAEs rely on a fixed sparsity level. They enforce the same number of active features ( K K ) across all inputs, ignoring the varying complexity of real-world data. Natural data often lies on manifolds with varying local intrinsic dimensionality, meaning the number of relevant factors can change significantly across samples. This suggests that a fixed sparsity level is not optimal. Simple inputs may require only a few features, while more complex ones need more expressive representations. Using a constant K K can therefore introduce noise in simple cases or miss important structure in more complex ones. To address this issue, we propose SoftSAE, a sparse autoencoder with a Dynamic Top-K selection mechanism. Our method uses a differentiable Soft Top-K operator to learn an input-dependent sparsity level k ^ \hat{k} . This allows the model to adjust the number of active features based on the complexity of each input. As a result, the representation better matches the structure of the data, and the explanation length reflects the amount of information in the input. Experimental results confirm that SoftSAE not only finds meaningful features, but also selects the right number of features for each concept. The source code is available at: https://github.com/St0pien/SoftSAE . Figure 1 : Overview of the SoftSAE framework for adaptive mechanistic interpretability. Unlike traditional SAEs that enforce a fixed sparsity level K regardless of input complexity, SoftSAE uses a Dynamic Sparsity MLP to estimate an input-dependent sparsity \hat{k} . Simple inputs, such as a “flying eagle”, are represented with fewer active features, reducing noise and entanglement. In contrast, semantically rich or cluttered inputs, such as “a happy man in sunglasses showing his lobsters”, induce higher sparsity budgets, enabling the model to capture multiple concepts without information loss. This adaptive mechanism aligns the effective explanation length with the local intrinsic dimensionality of the data manifold. 1 Introduction Mechanistic interpretability seeks to reverse engineer the opaque representations of deep neural networks into human-understandable components. As both Large Language Models and Vision Transformers scale in complexity, understanding their internal reasoning becomes critical for safety and alignment. In recent years, Sparse Autoencoders (SAEs) huben2023sparse ; zhang2024beyond have emerged as the dominant paradigm for this task. By mapping dense, polysemantic activations into a higher-dimensional space under a strict sparsity constraint, SAEs successfully isolate monosemantic features. Early foundational approaches, such as the ReLU SAE bricken2023monosemanticity , have demonstrated that neural networks naturally learn superposition, in which multiple distinct concepts share the same latent neurons. To refine feature extraction and address optimization instabilities, the research community has proposed several structural innovations. The TopK SAE gao2024scaling introduced a hard thresholding mechanism that explicitly activates only the most prominent features, effectively bypassing the tuning issues of continuous penalty terms. This concept was further advanced by BatchTopK bussmann2024batchtopk to ensure better feature utilization and prevent dead neurons across diverse training batches. Concurrent innovations include the JumpReLU SAE rajamanoharan2024jumping , which employs a discontinuous activation function to cleanly separate true signals from noise, and Matryoshka representations clip_matryoshka , which organize learned features into highly scalable hierarchical structures. Despite impressive advancements, all these leading methodologies share a fundamental architectural limitation: they impose a rigid, global capacity constraint across the entire dataset. Natural data exhibits immense variability in complexity. As demonstrated by Local Intrinsic Dimensionality (LID) estimation techniques such as LIDL tempczyk2022lidl , the number of latent degrees of freedom fluctuates significantly across different regions of the data manifold. A simple linguistic conjunction or a visually uniform background requires very few active concepts to be fully described. Conversely, highly specialized domain terminology or dense, cluttered visual scenes demand a rich, multifaceted set of explanatory features. Imposing a uniform global parameter K , representing the number of active features, explicitly disregards this fundamental geometric property of the data. It forces the interpretability model into an unavoidable compromise: it either generates overly complex explanations by hallucinating noisy features for simple inputs, or produces insufficient explanations by dropping critical semantic nuance for complex thoughts. To overcome this rigid bottleneck, we propose SoftSAE (see Figure 1 ), a highly adaptive architecture that dynamically calibrates its capacity to match the exact complexity of each processed input. Our core contribution is the integration of a Dynamic Top-K Selection mechanism powered by the differentiable Soft Top-K operator struski2025lapsum . Instead of relying on a static hyperparameter, SoftSAE optimizes an input-dependent scalar \hat{k} , representing the sparsity level of the processed data, through continuous gradient descent. This method allows the network to learn exactly how many concepts are required to faithfully reconstruct a given representation, thereby aligning the explanation length with the signal’s true information density. The primary contributions of our work are as follows: • We introduce SoftSAE, a novel approach that uses a continuous Soft Top-K formulation to dynamically predict and optimize the exact sparsity level ( \hat{k} ) for each input. • We provide a strong theoretical grounding for adaptive sparsity in mechanistic interpretability by linking explanation capacity directly to the local intrinsic dimensionality of the neural activation space. • By dynamically adjusting \hat{k} , the model achieves a superior balance between reconstruction fidelity and interpretability, uncovering monosemantic features that more faithfully reflect the multiscale complexity of natural data. 2 Related Work As deep learning architectures achieve unprecedented performance across diverse domains, their inherent opacity remains a significant barrier to safe and reliable deployment. The pursuit of illuminating these opaque systems has fostered a rich landscape of research, generally bifurcating into structural and semantic methodologies. In this section, we review the foundational and contemporary literature that contextualizes our proposed dynamic framework. We begin by examining the rapid evolution of mechanistic interpretability and sparse autoencoders, which strive to decode internal network representations at a highly granular level. Following this, we explore concept-based explainability paradigms that map latent geometric spaces to human-comprehensible concepts. Finally, we review recent developments in adaptive capacity and differentiable selection, positioning our methodology as a necessary synthesis of these distinct research trajectories. Mechanistic Interpretability and Sparse Autoencoders The field of mechanistic interpretability aims to reverse engineer the opaque inner workings of complex neural networks into understandable, logical components conmy2023towards ; bereska2024mechanistic . Initial attempts in this domain focused on generating natural-language descriptions of individual neurons hernandez2021natural . However, the highly polysemantic nature of these representations, where a single neuron responds to multiple unrelated concepts, posed significant challenges. To disentangle these overlapping representations, Sparse Autoencoders (SAEs) emerged as a robust solution capable of recovering monosemantic features huben2023sparse ; zhang2024beyond . The foundational ReLU-based architectures bricken2023monosemanticity paved the way for scaling these methods to massive language models templeton2024scaling . Subsequent architectural improvements focused heavily on refining the sparsity constraint to improve both reconstruction fidelity and feature purity. The applied Top-K framework gao2024scaling introduced a strict activation limit to bypass the instability of continuous penalty terms. This was