When to Trust the Cheap Check: Weak and Strong Verification for Reasoning

Shayan Kiyani, Sima Noorani, George Pappas, Hamed Hassani · 2026

Read When to Trust the Cheap Check: Weak and Strong Verification for Reasoning by Shayan Kiyani, Sima Noorani, George Pappas, Hamed Hassani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17633.txt.

Description

Shayan Kiyani and team formalize a choice that shapes reasoning with language models: when to trust a cheap internal check, and when to ask a human to look. Cheap checks, such as self-consistency or proxy rewards, are fast and scalable but noisy. Human inspection is trustworthy but costly, so it cannot run on every output.

The authors frame weak-strong verification policies that decide when to accept, reject, or defer to strong checks. Over a population, optimal policies take a two-threshold form, and calibration and sharpness set the value of weak checkers. An online algorithm then provably controls acceptance and rejection errors, with no assumptions on the query stream, the model, or the weak checker. The result is a principled dial for trust versus cost.

Catalog record

CREATORShayan Kiyani, Sima Noorani, George Pappas, Hamed Hassani
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE69 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17633

Browse the full library · About the Tower of Babel