FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning DOI: XXXXXXX.XXXXXXX Conference: KDD 2026 Datasets and Benchmarks Track ; Aug 06 2026; Jeju, Korea ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence Computer vision Computer vision tasks CCS: General and reference Cross-computing tools and techniques Evaluation CCS: Applied computing Physical sciences and engineering Physics
Zeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei · 2026
Read FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning DOI: XXXXXXX.XXXXXXX Conference: KDD 2026 Datasets and Benchmarks Track ; Aug 06 2026; Jeju, Korea ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence Computer vision Computer vision tasks CCS: General and reference Cross-computing tools and techniques Evaluation CCS: Applied computing Physical sciences and engineering Physics by Zeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03893.txt.
Description
Zeyu Wang and colleagues built FeynmanBench, a test set of over 2,000 tasks asking models to read Feynman diagrams in full. Each item pairs a diagram with short text and asks for vertex lists, propagator types, topology, momentum routing, and the full scattering amplitude. Nineteen leading multimodal models scored 70 to 95 percent on local spotting but fell to 13 to 17 percent on rebuilding topology, near zero on full derivation. The gap shows a deep weak spot in diagram-based science reasoning.
Catalog record
| CREATOR | Zeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37140 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.03893