Improve MLLM Benchmark Efficiency through Interview

Farong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai · 2025

Read Improve MLLM Benchmark Efficiency through Interview by Farong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-00883.txt.

Description

Farong Wen and co-authors address a cost problem in testing multimodal language models in this 2025 paper. Full test sets with thousands of questions take much time and computing power to run. The team proposes an interview style of testing, where the model faces a small set of questions first and harder ones step by step.

To build the test, they took an existing data set and added difficulty labels based on how typical models scored. The interview then starts with a few items, gauges the model's level, and pushes toward its limits. Trials on standard benchmark sets show the method finds a model's score faster, using far fewer questions. That makes routine checks cheaper for labs that test many models.

Catalog record

CREATORFarong Wen, Yijin Guo, Junying Wang, Jiaohao Xiao, Yingjie Zhou, Chunyi Li, Zicheng Zhang, Guangtao Zhai
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-00883

Browse the full library · About the Tower of Babel