Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs
Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, Martin Vechev · 2026
Read Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs by Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, Martin Vechev (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-00674.txt.
Introduction
Jasper Dekoninck and 6 co-authors present MathArena as a living evaluation platform for mathematical reasoning in language models, arguing that static benchmarks have run out of road. Fixed test sets are narrow, saturate quickly, and date fast, which makes comparing models reliably and tracking progress genuinely hard. A platform, by contrast, is continuously maintained: it runs, aggregates, and analyzes evaluations across many benchmarks under one protocol.
MathArena has grown from final-answer olympiad problems to proof-based competitions, research-level problems drawn from arXiv, and formal proof generation in Lean. New benchmarks are designed as capabilities improve, keeping the platform challenging. The strongest current model reaches near-ceiling scores on the 2026 USA Math Olympiad and about three-quarters on research-level questions. The authors read those numbers as the point: frontier models now solve extremely hard mathematics, so only a maintained platform can keep score honestly.
Catalog record
| CREATOR | Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, Martin Vechev |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 156 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.00674