Analyzing the Inner Workings of Transformers in Compositional Generalization
Ryoma Kumon, Hitomi Yanaka · 2025
Read Analyzing the Inner Workings of Transformers in Compositional Generalization by Ryoma Kumon, Hitomi Yanaka (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15277.txt.
Description
Ryoma Kumon and Hitomi Yanaka wrote a paper from 2025 on how a transformer model handles new mixes of known parts. Human speech can build fresh sentences from old words by rule. Models are often judged on this skill by their answers alone, which hides how they get there. The pair opened the model up to watch its inner work.
They hunted for a smaller web inside the model that carries the skill. They also ran cause tests on how the model uses grammar traits. The model does lean on grammar traits to form the right answer. But the inner web that generalizes best also runs a second routine that does not build by parts at all. That web gained its edge slowly across training, while the plain routine was picked up early. The pair says strong scores can mask a mixed strategy inside.
Catalog record
| CREATOR | Ryoma Kumon, Hitomi Yanaka |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15277