HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models

Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng · 2026

Read HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models by Emmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19341.txt.

Description

Emmy Liu and colleagues define hallucination against an explicit reference world, then build HalluWorld to measure it cleanly. Existing benchmarks mix settings and labels, so a fix that helps one task may not transfer, and many rely on human labels or hard-to-reproduce observations. Their synthetic and semi-synthetic worlds fully specify the truth, control what the model sees, and generate labels automatically across gridworlds, chess, and terminal tasks. Frontier models nearly solve direct perceptual checks, yet multi-step tracking, causal simulation, and knowing when to abstain remain unsolved.

Catalog record

CREATOREmmy Liu, Varun Gangal, Michael Yu, Zhuofu Tao, Karan Singh, Sachin Kumar, Steven Y. Feng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE110077
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.19341

Browse the full library · About the Tower of Babel