FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning DOI: XXXXXXX.XXXXXXX Conference: KDD 2026 Datasets and Benchmarks Track ; Aug 06 2026; Jeju, Korea ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence Computer vision Computer vision tasks CCS: General and reference Cross-computing tools and techniques Evaluation CCS: Applied computing Physical sciences and engineering Physics

Zeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei · 2026

Read FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning DOI: XXXXXXX.XXXXXXX Conference: KDD 2026 Datasets and Benchmarks Track ; Aug 06 2026; Jeju, Korea ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence Computer vision Computer vision tasks CCS: General and reference Cross-computing tools and techniques Evaluation CCS: Applied computing Physical sciences and engineering Physics by Zeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-03893.txt.

Description

Zeyu Wang and colleagues built FeynmanBench, a test set of over 2,000 tasks asking models to read Feynman diagrams in full. Each item pairs a diagram with short text and asks for vertex lists, propagator types, topology, momentum routing, and the full scattering amplitude. Nineteen leading multimodal models scored 70 to 95 percent on local spotting but fell to 13 to 17 percent on rebuilding topology, near zero on full derivation. The gap shows a deep weak spot in diagram-based science reasoning.

Catalog record

CREATORZeyu Wang, Xiaogang Li, Peiyao Xiao, Qinhao Kong, Ben Wang, Chengliang Xu, Zichao Chen, Bing Zhao, Hu Wei
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37140
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.03893

Browse the full library · About the Tower of Babel