CombiGraph-Vis: A Curated Multimodal Olympiad Benchmark for Discrete Mathematical Reasoning
Hamed Mahdavi, Pouria Mahdavinia, Alireza Farhadi, Pegah Mohammadipour, Samira Malek, Majid Daliri, Pedram Mohammadipour, Alireza Hashemi, Amir Khasahmadi, Vasant Honavar, [4mm] Pennsylvania State University · 2025
Read CombiGraph-Vis: A Curated Multimodal Olympiad Benchmark for Discrete Mathematical Reasoning by Hamed Mahdavi, Pouria Mahdavinia, Alireza Farhadi, Pegah Mohammadipour, Samira Malek, Majid Daliri, Pedram Mohammadipour, Alireza Hashemi, Amir Khasahmadi, Vasant Honavar, [4mm] Pennsylvania State University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27094.txt.
Description
Hamed Mahdavi leads nine co-authors in studying whether top language models can grade olympiad proofs fairly. Leading systems now solve most recent olympiad problems, so attention shifts to judging: spotting errors, weighing their severity, and awarding partial credit beyond right or wrong. The team grades a corpus of generated solutions plus contest solution sets on detailed scales.
Models reliably flag incorrect solutions, including subtly flawed ones, but their partial credit calibration gaps show in scoring. To help, the authors build agent workflows that extract reference solutions and derive problem-specific rubrics for multi-step grading. Those workflows agree with human grades more closely and handle partial credit more consistently. All code, data, and logs are released for future grading research.
Catalog record
| CREATOR | Hamed Mahdavi, Pouria Mahdavinia, Alireza Farhadi, Pegah Mohammadipour, Samira Malek, Majid Daliri, Pedram Mohammadipour, Alireza Hashemi, Amir Khasahmadi, Vasant Honavar, [4mm] Pennsylvania State University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 131 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-27094