Representation Alignment Rests on Linear Structure

Kiril Bangachev, Guy Bresler, Yury Polyanskiy · 2026

Read Representation Alignment Rests on Linear Structure by Kiril Bangachev, Guy Bresler, Yury Polyanskiy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-28870.txt.

Description

Kiril Bangachev, Guy Bresler, and Yury Polyanskiy test the Platonic Representation Hypothesis with a three-part signal, bias, and noise setup. They propose Platonic alignment arises from a shared object-trait link in lines under the Linear Representation Hypothesis. Sparse feature nets often align across modes more strongly than dense ones. Centering and scaling improve cross-model alignment, partly easing hidden-bias gaps. They also show model noise tracks data lack, with a strong link between word rate and matching in language and text vector models.

Catalog record

CREATORKiril Bangachev, Guy Bresler, Yury Polyanskiy
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101587
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.28870

Browse the full library · About the Tower of Babel