MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs
Viresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh · 2026
Read MathConstraint: Automated Generation of Verified Combinatorial Reasoning Instances for LLMs by Viresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08498.txt.
Description
Viresh Pati and six coauthors build MathConstraint, a test of step by step reasoning built from constraint problems with solver checked answers. Fixed tests grow stale as models improve, and judging by model opinion is weak. Their generator makes fresh problems at rising difficulty, each one verified by trusted solving software rather than by another model.
Two released sets show a steep drop from easy to hard problems for twelve leading models. Giving models solver tools roughly doubles hard set accuracy, though halving the tool budget erases much of that gain. Test makers gain a durable way to measure reasoning and tool use as models keep improving.
Catalog record
| CREATOR | Viresh Pati, Zhengyu Li, Piyush Jha, Rahul Garg, Yatharth Sejpal, Vijay Ganesh |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 111 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08498