RiddleBench: A New Generative Reasoning Benchmark for LLMs
Deepon Halder, Alan Saji Thanmay Jayakumar, Ratish Puduppully, Anoop Kunchukuttan Raj Dabre · 2025
Read RiddleBench: A New Generative Reasoning Benchmark for LLMs by Deepon Halder, Alan Saji Thanmay Jayakumar, Ratish Puduppully, Anoop Kunchukuttan Raj Dabre (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24932.txt.
Description
Deepon Halder, Alan Saji, Thanmay Jayakumar, Ratish Puduppully, Anoop Kunchukuttan, and Raj Dabre built RiddleBench, a set of hard riddles to test chat models. Known test sets lean on math and set skills, yet miss the loose, multi side thought that riddles need. Riddle work joins logic, space sense, and rule checks in one ask.
The bench holds more than seventeen hundred riddles in English. Top paid models scored just above sixty in a hundred on it. Checks showed deep flaws: models took bad thought from peers as true, and could not fix their own slips due to self trust bias. Their thought was also frail to small shifts in an ask.
Catalog record
| CREATOR | Deepon Halder, Alan Saji Thanmay Jayakumar, Ratish Puduppully, Anoop Kunchukuttan Raj Dabre |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24932