Do Sparse Autoencoders Capture Concept Manifolds?

Usha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold · 2026

Read Do Sparse Autoencoders Capture Concept Manifolds? by Usha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-28119.txt.

Description

Bhalla's team asks whether sparse autoencoders really capture concepts laid out as manifolds, curved low-dim shapes, rather than as independent directions. Theory shows an autoencoder can capture a manifold as a whole, with a compact atom group spanning it, or locally, by tiling its regions with picked features. In practice they mix the two in a broken regime the authors call dilution. That mix hides manifold structure at the single-concept level and calls for group-based discovery methods.

Catalog record

CREATORUsha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88260
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.28119

Browse the full library · About the Tower of Babel