Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs

William Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li · 2025

Read Is It JUST Semantics? A Case Study of Discourse Particle Understanding in LLMs by William Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04534.txt.

Description

William Sheffield and co-authors tested in 2025 whether large language models truly grasp the small word 'just' and its many jobs in English. The word can shut out options, mark a recent moment, or add force, among other senses. Expert linguists built and labeled the test data with care.

Models showed some skill at telling the broad sense families apart. They faltered, though, on the finer shades that separate one use from its near twin. The authors read this as a real gap in how models handle discourse particles, the small words that steer meaning. Such words are easy to overlook in training data. The study gives formal semantics a fresh test bed for probing machine understanding.

Catalog record

CREATORWilliam Sheffield Kanishka Misra Valentina Pyatkin, Ashwini Deo Kyle Mahowald Junyi Jessy Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04534

Browse the full library · About the Tower of Babel