Estimating Commonsense Plausibility through Semantic Shifts
Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng · 2025
Read Estimating Commonsense Plausibility through Semantic Shifts by Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13464.txt.
Description
Wanqing Cui and co-authors present a new way to grade common sense in language models in a 2025 paper. Judging whether a sentence sounds plausible is a fine-grained skill, and models asked to score sentences outright, or by raw likelihood, often blur close calls.
Their method, ComPaSS, adds a commonsense detail to a sentence and measures how far the meaning shifts. A sensible addition barely moves the meaning, while a silly one knocks it far off course. Across tasks and model types, the shift test beat the standard scoring methods. Vision-language models did best on picture-grounded tasks, and a contrast-sharpened training style made every backbone keener.
Catalog record
| CREATOR | Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13464