Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders
Aaron J. Li, Suraj Srinivas, Usha Bhalla, Himabindu Lakkaraju · 2025
Read Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoders by Aaron J. Li, Suraj Srinivas, Usha Bhalla, Himabindu Lakkaraju (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-16004.txt.
Description
Aaron Li and co-authors probe a weak point in tools that claim to name the concepts inside a large language model. Their 2025 paper deals with sparse autoencoders, which turn the model's inner activity into a set of human-readable concept signals. Past checks graded these tools on rebuild quality and clarity, but not on firmness.
The authors frame firmness as an attack problem: can a tiny tweak to the input flip the concept readout while the model itself barely stirs? In most of their tests, the answer was yes. Small crafted tweaks fooled the concept labels with ease. They warn that, without cleaning steps, such readouts are too fragile to trust for watching or steering a model.
Catalog record
| CREATOR | Aaron J. Li, Suraj Srinivas, Usha Bhalla, Himabindu Lakkaraju |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-16004