Interpretability Transfer from Language to Vision via Sparse Autoencoders
Alexey Kravets, Da Li, Chuan Li, Da Chen, Vinay P. Namboodiri · 2026
Read Interpretability Transfer from Language to Vision via Sparse Autoencoders by Alexey Kravets, Da Li, Chuan Li, Da Chen, Vinay P. Namboodiri (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24946.txt.
Description
Alexey Kravets leads VISTA, a bridge carrying language interpretability into vision without dedicated vision autoencoders. Labeling visual concepts is hard and fuzzy, so past language gains stalled visually. VISTA steers a visual projector in a LLaVA model to map visual tokens into the language model's labeled textual autoencoder space. Regularized by that autoencoder's reconstruction loss, and the matching rate between top textual concepts and image elements roughly triples. The same lens shows DINOv2 localizing space better than rival encoders, and object removal and replacement tests prove alignment.
Catalog record
| CREATOR | Alexey Kravets, Da Li, Chuan Li, Da Chen, Vinay P. Namboodiri |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67101 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24946