WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorization
Ine Gevers, Victor De Marez, Luna De Bruyne, Walter Daelemans · 2025
Read WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorization by Ine Gevers, Victor De Marez, Luna De Bruyne, Walter Daelemans (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-23779.txt.
Description
Ine Gevers and co-authors paraphrase a famous commonsense benchmark and watch language model scores fall sharply, in work from twenty twenty-five. WinoGrande sentences test pronoun sense through twin-sentence choices. Their WinoWhat corpus rewrites every validation item in new words. Five commonsense categories tag what kind of knowledge each item needs. Models of many sizes face both versions of the test.
Every model performs notably worse on the paraphrased corpus. That gap suggests benchmark scores overstate true reasoning skill. Training-data matches are checked to rule out simple memorisation. Memorisation, it turns out, explains little of the score gap. Surface wording, not knowledge, props up headline results. Paraphrase twins make a sterner test for machine commonsense.
Catalog record
| CREATOR | Ine Gevers, Victor De Marez, Luna De Bruyne, Walter Daelemans |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-23779