KWBench: Measuring Unprompted Problem Recognition in Knowledge Work
Ankit Maloo · 2026
Read KWBench: Measuring Unprompted Problem Recognition in Knowledge Work by Ankit Maloo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15760.txt.
Description
Maloo offers KWBench, a test of whether LLMs spot the job itself, unprompted, before they try to solve it. Its 223 tasks, drawn from pros in deals, pacts, drug care, and fraud watch, each hide a game shape, with expert reads and flop paths on file. Models get raw data and a prompt, with no hint of job type, and face a three-step rule with a must-pass joint check. The best of 16 models clears 27.9 percent, and the top two agree on just 31.7 percent of passes.
Catalog record
| CREATOR | Ankit Maloo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 105167 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.15760