Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models
Yixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected]) · 2026
Read Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models by Yixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected]) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07539.txt.
Description
How closely do language models mirror the brain's language system, and what changes during training? Yixuan Liu and seven co-authors track representational shape, entropy, curvature, and brain-coding scores through training of Pythia models from 70 million to 1 billion in size.
Layers self-organize into stable low- and high-complexity modules, and the low-complexity module predicts human language network activity better. Alignment arrives fast and stably in time regions but later and more dynamically in frontal areas. Low curvature remains a robust predictor after controlling for training progress, strengthening with scale. Cognitive experts get a geometric story for why bigger models align with brains.
Catalog record
| CREATOR | Yixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected]) |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07539