The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?
Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli · 2026
Read The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math? by Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03334.txt.
Description
Bianca Raimondi and three colleagues test whether language models are ready for university-level mathematics. Most existing tests stop at school problems, contests, or formal proof, leaving graduate computational work unexamined. Their 2026 dataset, CompMath-MCQ, fills the gap with fifteen hundred multiple-choice questions written fresh by professors of graduate courses, so no model can have memorized them from training data.
Questions span linear algebra, numerical optimization, vector calculus, probability, and scientific computing in Python, each with three options and one correct answer. Quality was checked through disagreement between models, followed by expert review. Scoring runs through a standard evaluation library, keeping results objective and repeatable. Baseline results with leading models show advanced computational reasoning is still a significant challenge.
Catalog record
| CREATOR | Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34279 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.03334