OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models
Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen · 2026
Read OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models by Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18094.txt.
Description
Ling Lin and colleagues present OODBench, a benchmark for testing large vision-language models on out-of-distribution data. Existing models train on massive datasets assuming independent and identically distributed data, but real-world data often break that assumption. Mishandling out-of-distribution objects can create safety risks in driving and medical assistance, yet no valid benchmark has comprehensively assessed model performance on such data.
OODBench offers a mostly automated method with minimal human verification for building benchmarks and evaluating how models handle out-of-distribution data. It contains forty thousand out-of-distribution instance-category pairs, and current models still degrade notably, even when image categories are common. A reliable automated metric uses Basic-to-Advanced Progression questions to measure how out-of-distribution data affect questions of varying difficulty. The authors summarize findings to guide future research on acquiring and evaluating out-of-distribution data.
Catalog record
| CREATOR | Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 162 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-18094