Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning

Xinting Huang, Michael Hahn · 2025

Read Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning by Xinting Huang, Michael Hahn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01916.txt.

Description

Xinting Huang and Michael Hahn showed in 2025 that the representation space inside neural models can be split into interpretable subspaces with unsupervised learning. A model's inner space is huge and mixes many aspects of its input. The question was whether those aspects sit in separate natural subspaces that can be found without labels.

Their method, neighbor distance minimization, learns subspaces that are not aligned to the raw axes. In many cases, the information in a subspace shared one abstract concept across inputs, much like a variable. Tests on known circuits in GPT-2 tied subspaces to circuit variables. The method also scaled to a two billion parameter model, separating context from stored knowledge.

Catalog record

CREATORXinting Huang, Michael Hahn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE122 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01916

Browse the full library · About the Tower of Babel