Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models
Ryoma Kumon, Hitomi Yanaka · 2026
Read Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models by Ryoma Kumon, Hitomi Yanaka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22166.txt.
Description
Kumon and Yanaka ask whether language models share neural workings across sentence patterns, testing filler-gap dependencies and negative polarity licensing with causal methods. Using activation patching, they pin down the roles of attention heads and MLP blocks. They find a local, shared mechanism for filler-gap links in early to middle layers, while negative polarity handling shows no single mechanism. Patching-found mechanisms carry over out of distribution, while distributed alignment search fails on narrow cases. Finally, tweaking the found components improves acceptability judgment scores.
Catalog record
| CREATOR | Ryoma Kumon, Hitomi Yanaka |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45791 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.22166