From Directions to Regions: Decomposing Activations in Language Models via Local Geometry

Or Shafran, Shaked Ronen, Omri Fahn, Shauli Ravfogel, Atticus Geiger, Mor Geva · 2026

Read From Directions to Regions: Decomposing Activations in Language Models via Local Geometry by Or Shafran, Shaked Ronen, Omri Fahn, Shauli Ravfogel, Atticus Geiger, Mor Geva (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02464.txt.

Description

Or Shafran and 5 co-authors argue that concepts inside a language model live in regions, not single directions, in a 2026 arXiv paper. Interpretability methods usually hunt for one global direction per concept, assuming concepts separate cleanly along a line. That misses concepts with curved or multi-dimensional structure.

Their tool, a mixture of factor analyzers, models activation space as a collection of Gaussian regions, each with its own center and local pattern of variation. Trained at large scale on Llama-3.1-8B and Gemma-2-2B, the model captured complex nonlinear structures. On localization and steering benchmarks it beat unsupervised baselines, rivaled supervised methods, and often steered behavior more strongly than sparse autoencoders. The authors propose local geometry as the right unit for discovering and controlling concepts at scale.

Catalog record

CREATOROr Shafran, Shaked Ronen, Omri Fahn, Shauli Ravfogel, Atticus Geiger, Mor Geva
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02464

Browse the full library · About the Tower of Babel