Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models

Yixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected]) · 2026

Read Training-Driven Representational Geometry Modularization Predicts Brain Alignment in Language Models by Yixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected]) (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07539.txt.

Description

How closely do language models mirror the brain's language system, and what changes during training? Yixuan Liu and seven co-authors track representational shape, entropy, curvature, and brain-coding scores through training of Pythia models from 70 million to 1 billion in size.

Layers self-organize into stable low- and high-complexity modules, and the low-complexity module predicts human language network activity better. Alignment arrives fast and stably in time regions but later and more dynamically in frontal areas. Low curvature remains a robust predictor after controlling for training progress, strengthening with scale. Cognitive experts get a geometric story for why bigger models align with brains.

Catalog record

CREATORYixuan Liu ([email protected]), Zhiyuan Ma, Likai Tang, Runmin Gan,, Xinche Zhang, Jinhao Li, Chao Xie ([email protected]), Sen Song ([email protected])
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-07539

Browse the full library · About the Tower of Babel