Theoretical Physics Benchmark (TPBench) - a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics
Daniel J.H. Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, Sai Chaitanya Tadepalli · 2025
Read Theoretical Physics Benchmark (TPBench) - a Dataset and Study of AI Reasoning Capabilities in Theoretical Physics by Daniel J.H. Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, Sai Chaitanya Tadepalli (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15815.txt.
Description
Chung and co-authors introduce a benchmark, published in 2025, for testing how well AI systems solve theoretical physics problems. The set holds 57 fresh problems, from undergraduate level to live research level. They cover high-energy theory and cosmology. Because the problems were written for this test, they do not appear in public collections that AI systems may have trained on. That makes the scores a cleaner measure of reasoning.
The team ran the set on a range of leading language models, open and closed. The newest models scored impressively on the easier tiers. Research-level problems, though, stayed mostly unsolved. The authors also discuss what makes scoring such work hard, since physics answers come in many equivalent forms. The benchmark gives the field a yardstick for tracking real reasoning gains, separate from memorized problem banks.
Catalog record
| CREATOR | Daniel J.H. Chung, Zhiqi Gao, Yurii Kvasiuk, Tianyi Li, Moritz Münchmeyer, Maja Rudolph, Frederic Sala, Sai Chaitanya Tadepalli |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 142 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-15815