Do Sparse Autoencoders Capture Concept Manifolds?
Usha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold · 2026
Read Do Sparse Autoencoders Capture Concept Manifolds? by Usha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-28119.txt.
Description
Bhalla's team asks whether sparse autoencoders really capture concepts laid out as manifolds, curved low-dim shapes, rather than as independent directions. Theory shows an autoencoder can capture a manifold as a whole, with a compact atom group spanning it, or locally, by tiling its regions with picked features. In practice they mix the two in a broken regime the authors call dilution. That mix hides manifold structure at the single-concept level and calls for group-based discovery methods.
Catalog record
| CREATOR | Usha Bhalla ⋆ , a {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},a} Thomas Fel ⋆ {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\star}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Can Rager Sheridan Feucht , b {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},b} Tal Haklay , c {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},c} Daniel Wurgaft , d {}^{\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}},d} Siddharth Boppana Matthew Kowal Vasudev Shyam Owen Lewis Thomas McGrath Jack Merullo Atticus Geiger † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} Ekdeep Singh Lubana † {}^{{\color[rgb]{0.7227,0.5938,0.2266}\bm{\dagger}}\raisebox{0.5pt}{\hskip 1.42262pt\includegraphics[height]{arxiv/goodfire_logo_small.png}}} ⋆ Equal contribution † Equal senior contribution a Harvard University b Northeastern University c Technion IIT d Stanford University https://github.com/goodfire-ai/sae-manifold |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88260 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.28119