MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs

Viresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh · 2026

Read MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs by Viresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08498.txt.

Description

Viresh Pati and six coauthors build MathConstraint, a test of step by step reasoning built from constraint problems with solver checked answers. Fixed tests grow stale as models improve, and judging by model opinion is weak. Their generator makes fresh problems at rising difficulty, each one verified by trusted solving software rather than by another model.

Two released sets show a steep drop from easy to hard problems for twelve leading models. Giving models solver tools roughly doubles hard set accuracy, though halving the tool budget erases much of that gain. Test makers gain a durable way to measure reasoning and tool use as models keep improving.

Catalog record

CREATORViresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE111 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08498

Browse the full library · About the Tower of Babel