Evaluating Large Language Models on Quantum Mechanics: A Comparative Study Across Diverse Models and Tasks
S. K. Rithvik · 2026
Read Evaluating Large Language Models on Quantum Mechanics: A Comparative Study Across Diverse Models and Tasks by S. K. Rithvik (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19006.txt.
Description
How well do language models actually solve quantum mechanics problems? S. K. Rithvik, writing alone in 2026, presents a systematic evaluation of fifteen models from five providers, spanning three capability tiers, on twenty tasks covering derivations, creative problems, non-standard concepts, and numerical computation.
Results show clear tier stratification. Flagship models average about four-fifths correct, ahead of mid-tier and fast models. Derivations prove easiest, with flagship models reaching perfect average performance. Numerical computation proves hardest, at well under half correct on average. Tool augmentation on numerical tasks gives modest overall improvement at triple token cost, masking wide heterogeneity from large gains to notable degradation. Reproducibility analysis shows average variance across runs, with flagship models exceptionally stable. Tasks, verifiers, and results are publicly released.
Catalog record
| CREATOR | S. K. Rithvik |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-19006