Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?
Atrey Desai, Sathvik Nair · 2026
Read Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints? by Atrey Desai, Sathvik Nair (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14459.txt.
Description
Desai and Nair ask whether language models trained on child-sized data build one shared representation for filler-gap dependencies across constructions. Using distributed alignment search, they probe models trained on varying BabyLM data amounts for transfer between wh-questions and topicalization, which differ greatly in input frequency. Shared yet item-sensitive mechanisms seem to develop with limited data. Still, models need far more data than children to reach similar generalizations, suggesting language-specific biases belong in models of language learning.
Catalog record
| CREATOR | Atrey Desai, Sathvik Nair |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38547 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14459