MMVU : Measuring Expert-Level Multi-Discipline Video Understanding

Yilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu · 2025

Read MMVU : Measuring Expert-Level Multi-Discipline Video Understanding by Yilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12380.txt.

Description

Yilun Zhao and co-authors benchmark video understanding across expert fields in a 2025 paper. Their test holds three thousand expert-written questions in total. Twenty-seven subjects span science, health, humanities, and engineering fields together.

Ordinary video tests mostly check basic visual perception skills of models. These questions demand real domain knowledge plus careful reasoning instead. Answers require watching specialized footage with an expert's eye for detail. Model scores reveal how far video AI still is from expertise. The benchmark was annotated and checked by subject specialists throughout. Readers see a harder yardstick for machines that claim to understand video.

Catalog record

CREATORYilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE60 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-12380

Browse the full library · About the Tower of Babel