Lexical hints of accuracy in LLM reasoning chains
Arne Vanhoyweghen, Brecht Verbeken, Andres Algaba, Vincent Ginis · 2026
Read Lexical hints of accuracy in LLM reasoning chains by Arne Vanhoyweghen, Brecht Verbeken, Andres Algaba, Vincent Ginis (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13478320.txt.
Description
Arne Vanhoyweghen and three colleagues asked in 2026 whether wording inside a language model's reasoning chain betrays wrong answers. Models often sound confident while wrong, especially on hard tests, so the team tested chain length, sentiment swings, and hedging words as confidence signals.
Across three benchmarks and three models, uncertainty words like guess, stuck, and hard were the strongest clues, with sentiment shifts adding weaker support. Chain length helped only where accuracy was already high, not on the hardest test. Uncertainty cues stood out more than confidence cues, making errors easier to spot than correct answers. The findings support lightweight post-hoc calibration.
Catalog record
| CREATOR | Arne Vanhoyweghen, Brecht Verbeken, Andres Algaba, Vincent Ginis |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |