Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math

Guijin Son, Donghun Yang, Hitesh Laxmichand Patel, Hyunwoo Ko, Amit Agarwal, Sunghee Ahn, Kyong-Ha Lee, Youngjae Yu · 2026

Read Judging What We Cannot Solve: A Consequence-Based Approach for Oracle-Free Evaluation of Research-Level Math by Guijin Son, Donghun Yang, Hitesh Laxmichand Patel, Hyunwoo Ko, Amit Agarwal, Sunghee Ahn, Kyong-Ha Lee, Youngjae Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06291.txt.

Description

Grading research-level math solutions normally takes scarce expert time, since no answer key exists. Guijin Son and 7 co-authors propose judging a solution by its consequences. If a solution's method is sound, using it as an example should help solve nearby related problems that can be checked.

Their evaluator scores each candidate by that downstream usefulness, with no oracle needed. They tested it on fresh research-level problems, each with one expert solution and nine machine-made ones. The method ranked solutions better than reward models and language model judges. It also kept a wider gap between correct and wrong solutions, even on problems the underlying solver usually failed.

Catalog record

CREATORGuijin Son, Donghun Yang, Hitesh Laxmichand Patel, Hyunwoo Ko, Amit Agarwal, Sunghee Ahn, Kyong-Ha Lee, Youngjae Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06291

Browse the full library · About the Tower of Babel