Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints?

Atrey Desai, Sathvik Nair · 2026

Read Filling in the Mechanisms: How do LMs Learn Filler-Gap Dependencies under Developmental Constraints? by Atrey Desai, Sathvik Nair (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14459.txt.

Description

Desai and Nair ask whether language models trained on child-sized data build one shared representation for filler-gap dependencies across constructions. Using distributed alignment search, they probe models trained on varying BabyLM data amounts for transfer between wh-questions and topicalization, which differ greatly in input frequency. Shared yet item-sensitive mechanisms seem to develop with limited data. Still, models need far more data than children to reach similar generalizations, suggesting language-specific biases belong in models of language learning.

Catalog record

CREATORAtrey Desai, Sathvik Nair
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38547
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.14459

Browse the full library · About the Tower of Babel