KWBench: Measuring Unprompted Problem Recognition in Knowledge Work

Ankit Maloo · 2026

Read KWBench: Measuring Unprompted Problem Recognition in Knowledge Work by Ankit Maloo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15760.txt.

Description

Maloo offers KWBench, a test of whether LLMs spot the job itself, unprompted, before they try to solve it. Its 223 tasks, drawn from pros in deals, pacts, drug care, and fraud watch, each hide a game shape, with expert reads and flop paths on file. Models get raw data and a prompt, with no hint of job type, and face a three-step rule with a must-pass joint check. The best of 16 models clears 27.9 percent, and the top two agree on just 31.7 percent of passes.

Catalog record

CREATORAnkit Maloo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105167
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15760

Browse the full library · About the Tower of Babel