On the Non-Identifiability of Steering Vectors in Large Language Models
Sohan Venkatesh Ashish Mahendran Kurapath · 2026
Read On the Non-Identifiability of Steering Vectors in Large Language Models by Sohan Venkatesh Ashish Mahendran Kurapath (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06801.txt.
Description
Sohan Venkatesh and Ashish Mahendran Kurapath challenge a popular way of reading language models from the inside. Activation steering nudges a model's behavior by pushing its inner states along a chosen direction. Those directions are often read as meaningful signs of what the model represents. The paper argues that reading is unfounded, because the directions cannot be pinned down from behavior alone.
With white-box access to a single layer, the authors show that large families of different steering vectors produce the same outward behavior. Tests find that unrelated nudges work nearly as well as the intended directions, across models and traits. That match holds across the practical steering range and varied prompts. The result sets a basic limit on steering-based reading of models, and calls for hard rules about structure beyond mere behavior tests.
Catalog record
| CREATOR | Sohan Venkatesh Ashish Mahendran Kurapath |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06801