Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information
Youcheng Huang, Bowen Qin, Chen Huang, Duanyu Feng, Xi Yang, Wenqiang Lei, Jiancheng Lv · 2025
Read Beyond Solving Math Quiz: Evaluating the Ability of Large Reasoning Models to Ask for Information by Youcheng Huang, Bowen Qin, Chen Huang, Duanyu Feng, Xi Yang, Wenqiang Lei, Jiancheng Lv (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11252.txt.
Description
Youcheng Huang and co-authors test reasoning models on problems that lack key facts in a 2025 paper. Large reasoning models score well on math tests, but those tests always give complete problems. A real helper should notice when facts are missing and ask for them, rather than guessing ahead. That skill is rarely measured.
The team built a data set of incomplete problems in varied settings. Across models, the systems rarely asked for the missing facts. Instead they often over-thought the problem or made up facts that were never given. Extra training taught the skill in part, but with clear limits. The study argues that asking good questions should count as a core part of machine intelligence.
Catalog record
| CREATOR | Youcheng Huang, Bowen Qin, Chen Huang, Duanyu Feng, Xi Yang, Wenqiang Lei, Jiancheng Lv |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-11252