Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models

Ryoma Kumon, Hitomi Yanaka · 2026

Read Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models by Ryoma Kumon, Hitomi Yanaka (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22166.txt.

Description

Kumon and Yanaka ask whether language models share neural workings across sentence patterns, testing filler-gap dependencies and negative polarity licensing with causal methods. Using activation patching, they pin down the roles of attention heads and MLP blocks. They find a local, shared mechanism for filler-gap links in early to middle layers, while negative polarity handling shows no single mechanism. Patching-found mechanisms carry over out of distribution, while distributed alignment search fails on narrow cases. Finally, tweaking the found components improves acceptability judgment scores.

Catalog record

CREATORRyoma Kumon, Hitomi Yanaka
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45791
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22166

Browse the full library · About the Tower of Babel