The Cylindrical Representation Hypothesis for Language Model Steering

Lang Gao, Jinghui Zhang, Wei Liu, Fengxian Ji, Chenxi Wang, Zirui Song, Akash Ghosh, Youssef Mohamed, Preslav Nakov, Xiuying Chen · 2026

Read The Cylindrical Representation Hypothesis for Language Model Steering by Lang Gao, Jinghui Zhang, Wei Liu, Fengxian Ji, Chenxi Wang, Zirui Song, Akash Ghosh, Youssef Mohamed, Preslav Nakov, Xiuying Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01844.txt.

Description

Lang Gao and nine co-authors give model steering a new geometry. Steering, nudging a language model's activations toward a concept, works erratically, and the reigning linear picture, which assumes concepts can be made fully orthogonal, predicts a tidiness that practice refuses to show.

Relax the orthogonality demand and a cylinder appears. Each concept gets a central axis, the direction between its absence and presence, which drives generation of the concept. Around it lies a normal plane that sets steering sensitivity: only certain sectors of that plane ease the axis into action, while others suppress or delay it. The plane can be located reliably from difference vectors, but the sensitive sector cannot, and that blind spot, the authors argue, is the principled source of steering's mood swings. Experiments confirm the cylindrical structure in real models and show the reading predicts steering behavior usefully.

Catalog record

CREATORLang Gao, Jinghui Zhang, Wei Liu, Fengxian Ji, Chenxi Wang, Zirui Song, Akash Ghosh, Youssef Mohamed, Preslav Nakov, Xiuying Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01844

Browse the full library · About the Tower of Babel