Cell-Based Representation of Relational Binding in Language Models

Qin Dai, Benjamin Heinzerling, Kentaro Inui · 2026

Read Cell-Based Representation of Relational Binding in Language Models by Qin Dai, Benjamin Heinzerling, Kentaro Inui (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-19052.txt.

Description

Dai, Heinzerling, and Inui show language models bind entities, relations, and attributes through cells. Each cell in a low dimensional subspace pairs an entity with a relation index, and attributes are read from it. Using multi sentence data, they decode indices from attribute signals by partial least squares. Across domains and two model families, indices decode in lines and form a grid. Context specific codes shift by translation vectors, enabling cross context transfer. Patching the subspace changes predictions and perturbing it harms performance, proving causal reliance.

Catalog record

CREATORQin Dai, Benjamin Heinzerling, Kentaro Inui
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105148
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.19052

Browse the full library · About the Tower of Babel