Estimating Commonsense Plausibility through Semantic Shifts

Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng · 2025

Read Estimating Commonsense Plausibility through Semantic Shifts by Wanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13464.txt.

Description

Wanqing Cui and co-authors present a new way to grade common sense in language models in a 2025 paper. Judging whether a sentence sounds plausible is a fine-grained skill, and models asked to score sentences outright, or by raw likelihood, often blur close calls.

Their method, ComPaSS, adds a commonsense detail to a sentence and measures how far the meaning shifts. A sensible addition barely moves the meaning, while a silly one knocks it far off course. Across tasks and model types, the shift test beat the standard scoring methods. Vision-language models did best on picture-grounded tasks, and a contrast-sharpened training style made every backbone keener.

Catalog record

CREATORWanqing Cui, Keping Bi, Jiafeng Guo, Xueqi Cheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13464

Browse the full library · About the Tower of Babel