When the LM misunderstood the human chuckled : Analyzing garden path effects in humans and language models
Samuel Joseph Amouyal, Aya Meltzer-Asscher, Jonathan Berant λ \lambda Blavatnik School of Computer Science, Tel Aviv University, Israel † \dagger Department of Linguistics, Tel Aviv University, Israel ⋆ \star Sagol School of Neuroscience, Tel Aviv University, Israel {samuel.amouyal, joberant}.cs.tau · 2025
Read When the LM misunderstood the human chuckled : Analyzing garden path effects in humans and language models by Samuel Joseph Amouyal, Aya Meltzer-Asscher, Jonathan Berant λ \lambda Blavatnik School of Computer Science, Tel Aviv University, Israel † \dagger Department of Linguistics, Tel Aviv University, Israel ⋆ \star Sagol School of Neuroscience, Tel Aviv University, Israel {samuel.amouyal, joberant}.cs.tau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09307.txt.
Description
Samuel Amouyal, Aya Meltzer-Asscher, and Jonathan Berant compare readers in twenty twenty-five. Garden-path sentences mislead readers into a wrong parse, then force a re-read. Humans find them notoriously hard. The authors ask whether large language models stumble in the same places, and for the same reasons.
They turn psycholinguistic hypotheses into tests. Human participants and a large suite of models answered comprehension questions on garden-path items. Both groups struggled with specific syntactic complexities. Some models correlated strongly with human comprehension patterns. Paraphrase and image generation tasks gave matching results. Readers see where machine parsing mirrors human parsing, and where the resemblance breaks. The study treats models as subjects in a classic language experiment.
Catalog record
| CREATOR | Samuel Joseph Amouyal, Aya Meltzer-Asscher, Jonathan Berant λ \lambda Blavatnik School of Computer Science, Tel Aviv University, Israel † \dagger Department of Linguistics, Tel Aviv University, Israel ⋆ \star Sagol School of Neuroscience, Tel Aviv University, Israel {samuel.amouyal, joberant}.cs.tau |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-09307