AxBench : Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, Christopher Potts · 2025
Read AxBench : Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders by Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, Christopher Potts (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17148.txt.
Description
Zhengxuan Wu and colleagues benchmarked methods for steering language model behavior in a 2025 paper called AxBench. Fine-grained control matters for safety, and interpretability researchers have proposed steering through internal representations, sparse autoencoders chief among them, as an alternative to prompting and fine-tuning. No shared benchmark compared the contenders until AxBench, which grades both steering success and concept detection across two sizes of an open model family.
Prompting won at steering outright, with fine-tuning second, while simple representation methods like difference-in-means led concept detection. Sparse autoencoders, despite their popularity, were not competitive on either task. A new weakly supervised representation fine-tuning method matched the leaders while keeping interpretability that prompting lacks.
Catalog record
| CREATOR | Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, Christopher Potts |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-17148