MMVU : Measuring Expert-Level Multi-Discipline Video Understanding
Yilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu · 2025
Read MMVU : Measuring Expert-Level Multi-Discipline Video Understanding by Yilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12380.txt.
Description
Yilun Zhao and co-authors benchmark video understanding across expert fields in a 2025 paper. Their test holds three thousand expert-written questions in total. Twenty-seven subjects span science, health, humanities, and engineering fields together.
Ordinary video tests mostly check basic visual perception skills of models. These questions demand real domain knowledge plus careful reasoning instead. Answers require watching specialized footage with an expert's eye for detail. Model scores reveal how far video AI still is from expertise. The benchmark was annotated and checked by subject specialists throughout. Readers see a harder yardstick for machines that claim to understand video.
Catalog record
| CREATOR | Yilun Zhao Lujing Xie * Haowei Zhang * Guo Gan Yitao Long Zhiyuan Hu Tongyan Hu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-12380