MathConstruct : Challenging LLM Reasoning with Constructive Proofs

Mislav Balunović*, Jasper Dekoninck*, Nikola Jovanović, Ivo Petrov, Martin Vechev · 2025

Read MathConstruct : Challenging LLM Reasoning with Constructive Proofs by Mislav Balunović*, Jasper Dekoninck*, Nikola Jovanović, Ivo Petrov, Martin Vechev (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10197.txt.

Description

Mislav Balunović and co-authors build a new test set for judging how well large language models do math. Their 2025 paper notes that many math tests use fixed answers, and models can pass them by guessing or by recall. Such tests also cover only a thin slice of real math work.

Their set, MathConstruct, holds 121 hard problems drawn from math contests. Each one asks the solver to build an object with stated traits, a form of proof used often in math. Right answers can be checked by machine, and the team can spin off changed forms of each problem to test how robust a model is. The best models solved only about six in ten problems.

Catalog record

CREATORMislav Balunović*, Jasper Dekoninck*, Nikola Jovanović, Ivo Petrov, Martin Vechev
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10197

Browse the full library · About the Tower of Babel