Analyze Feature Flow to Enhance Interpretation and Steering in Language Models

Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov · 2025

Read Analyze Feature Flow to Enhance Interpretation and Steering in Language Models by Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03032.txt.

Description

Daniil Laptev, Nikita Balagansky, Yaroslav Aksenov, and Daniil Gavrilov build a way to follow single features through a large language model, in work published in 2025. Features are patterns that a sparse autoencoder pulls out of the model's inner activity, layer by layer. Earlier work linked features between layers, and this team extends that mapping across the whole depth of a model.

Their method needs no fresh training data and instead compares features by the angle between them. The result is a flow map showing where each feature persists, changes, or first appears. The team then shows the maps can steer the model, by boosting or muting chosen features to guide the text it writes. The work gives researchers a clearer view of how a model builds its answers.

Catalog record

CREATORDaniil Laptev, Nikita Balagansky, Yaroslav Aksenov, Daniil Gavrilov
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03032

Browse the full library · About the Tower of Babel