The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?

Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli · 2026

Read The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math? by Bianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03334.txt.

Description

Bianca Raimondi and three colleagues test whether language models are ready for university-level mathematics. Most existing tests stop at school problems, contests, or formal proof, leaving graduate computational work unexamined. Their 2026 dataset, CompMath-MCQ, fills the gap with fifteen hundred multiple-choice questions written fresh by professors of graduate courses, so no model can have memorized them from training data.

Questions span linear algebra, numerical optimization, vector calculus, probability, and scientific computing in Python, each with three options and one correct answer. Quality was checked through disagreement between models, followed by expert review. Scoring runs through a standard evaluation library, keeping results objective and repeatable. Baseline results with leading models show advanced computational reasoning is still a significant challenge.

Catalog record

CREATORBianca Raimondi, Francesco Pivi, Davide Evangelista, Maurizio Gabbrielli
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34279
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03334

Browse the full library · About the Tower of Babel