Analyze Feature Flow to Enhance Interpretation and Steering in Language Models
Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov · 2025
Read Analyze Feature Flow to Enhance Interpretation and Steering in Language Models by Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03032.txt.
Description
Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, and Daniil Gavrilov build a way to follow single features through a large language model, in work published in 2025. Features are patterns that a sparse autoencoder pulls out of the model's inner activity, layer by layer. Earlier work linked features between layers, and this team extends that mapping across the whole depth of a model.
Their method needs no fresh training data and instead compares features by the angle between them. The result is a flow map showing where each feature persists, changes, or first appears. The team then shows the maps can steer the model, by boosting or muting chosen features to guide the text it writes. The work gives researchers a clearer view of how a model builds its answers.
Catalog record
| CREATOR | Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-03032