MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models
Anh Thai, Stefan Stojanov, Zixuan Huang, Bikram Boote, James M. Rehg · 2025
Read MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models by Anh Thai, Stefan Stojanov, Zixuan Huang, Bikram Boote, James M. Rehg (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20122.txt.
Description
Thai and co-authors present a 2025 benchmark on a word-learning trick in vision-language models. Young children use mutual exclusivity: a new word likely names the object they do not know. Models face the same choice in apps, yet the bias was untested. Their bench, MEBench, adds spatial reasoning to make scenes harder and more real.
A flexible pipeline builds varied annotated scenes for controlled tests. Fresh scoring rules capture how models use the bias. The tested models showed only weak bias, the team found. Extra spatial clues did help them settle some hard cases. The bench and pipeline are public for other labs to extend.
Catalog record
| CREATOR | Anh Thai, Stefan Stojanov, Zixuan Huang, Bikram Boote, James M. Rehg |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20122