How Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling

Yuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao · 2026

Read How Far Are We? Systematic Evaluation of LLMs vs. Human Experts in Mathematical Contest in Modeling by Yuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04791.txt.

Description

Liu and five colleagues test how far large language models lag human experts in mathematical modelling contests. They build a stage-wise scheme with expert criteria and check it against human scores on problems from a China postgraduate modelling contest. The scheme lines up with judges better than prior schemes. Results show a grasp-versus-execution gap: models frame problems well but falter at solving, coding, and analysis, and bigger scale does not close it. Faults trace to thin specs, missing checks, and weak validation, with errors spreading across stages.

Catalog record

CREATORYuhang Liu, Heyan Huang, Yizhe Yang, Hongyan Zhao, Zhizhuo Zeng, Yang Gao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE103950
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04791

Browse the full library · About the Tower of Babel