Analyzing the Inner Workings of Transformers in Compositional Generalization

Ryoma Kumon, Hitomi Yanaka · 2025

Read Analyzing the Inner Workings of Transformers in Compositional Generalization by Ryoma Kumon, Hitomi Yanaka (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15277.txt.

Description

Ryoma Kumon and Hitomi Yanaka wrote a paper from 2025 on how a transformer model handles new mixes of known parts. Human speech can build fresh sentences from old words by rule. Models are often judged on this skill by their answers alone, which hides how they get there. The pair opened the model up to watch its inner work.

They hunted for a smaller web inside the model that carries the skill. They also ran cause tests on how the model uses grammar traits. The model does lean on grammar traits to form the right answer. But the inner web that generalizes best also runs a second routine that does not build by parts at all. That web gained its edge slowly across training, while the plain routine was picked up early. The pair says strong scores can mask a mixed strategy inside.

Catalog record

CREATORRyoma Kumon, Hitomi Yanaka
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-15277

Browse the full library · About the Tower of Babel