Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs
William Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li · 2025
Read Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs by William Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04534.txt.
Description
William Sheffield and co-authors tested in 2025 whether large language models truly grasp the small word 'just' and its many jobs in English. The word can shut out options, mark a recent moment, or add force, among other senses. Expert linguists built and labeled the test data with care.
Models showed some skill at telling the broad sense families apart. They faltered, though, on the finer shades that separate one use from its near twin. The authors read this as a real gap in how models handle discourse particles, the small words that steer meaning. Such words are easy to overlook in training data. The study gives formal semantics a fresh test bed for probing machine understanding.
Catalog record
| CREATOR | William Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-04534