Improve MLLM Benchmark Efficiency through Interview
Farong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai · 2025
Read Improve MLLM Benchmark Efficiency through Interview by Farong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00883.txt.
Description
Farong Wen and co-authors address a cost problem in testing multimodal language models in this 2025 paper. Full test sets with thousands of questions take much time and computing power to run. The team proposes an interview style of testing, where the model faces a small set of questions first and harder ones step by step.
To build the test, they took an existing data set and added difficulty labels based on how typical models scored. The interview then starts with a few items, gauges the model's level, and pushes toward its limits. Trials on standard benchmark sets show the method finds a model's score faster, using far fewer questions. That makes routine checks cheaper for labs that test many models.
Catalog record
| CREATOR | Farong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-00883