How Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling
Yuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao · 2026
Read How Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling by Yuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04791.txt.
Description
Liu and five colleagues test how far large language models lag human experts in mathematical modelling contests. They build a stage-wise scheme with expert criteria and check it against human scores on problems from a China postgraduate modelling contest. The scheme lines up with judges better than prior schemes. Results show a grasp-versus-execution gap: models frame problems well but falter at solving, coding, and analysis, and bigger scale does not close it. Faults trace to thin specs, missing checks, and weak validation, with errors spreading across stages.
Catalog record
| CREATOR | Yuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103950 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.04791