Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
2025
Read Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12892.txt.
Description
Fel and co-authors steady a popular tool for peering inside vision models in a 2025 paper. Sparse autoencoders split a model's inner chatter into a dictionary of clean concepts. Trouble is, retrain the same tool and the dictionary scrambles, so yesterday's concepts vanish.
Their fix borrows archetypal analysis: each dictionary entry must sit inside the crowd of real data points, anchored like a tent pole. The anchored dictionaries stayed far more alike across runs, and a loosened version still rebuilt the signals as well as the best rivals. New tests grade whether dictionaries recover true class directions and unmix synthetic concepts. The anchored versions won, and surfaced fresh, sensible concepts.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12892