OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models

Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen · 2026

Read OODBench: Out-of-Distribution Benchmark for Large Vision-Language Models by Ling Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-18094.txt.

Description

Ling Lin and colleagues present OODBench, a benchmark for testing large vision-language models on out-of-distribution data. Existing models train on massive datasets assuming independent and identically distributed data, but real-world data often break that assumption. Mishandling out-of-distribution objects can create safety risks in driving and medical assistance, yet no valid benchmark has comprehensively assessed model performance on such data.

OODBench offers a mostly automated method with minimal human verification for building benchmarks and evaluating how models handle out-of-distribution data. It contains forty thousand out-of-distribution instance-category pairs, and current models still degrade notably, even when image categories are common. A reliable automated metric uses Basic-to-Advanced Progression questions to measure how out-of-distribution data affect questions of varying difficulty. The authors summarize findings to guide future research on acquiring and evaluating out-of-distribution data.

Catalog record

CREATORLing Lin, Yang Bai, Heng Su, Congcong Zhu, Yaoxing Wang, Yang Zhou, Huazhu Fu, Jingrun Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE162 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-18094

Browse the full library · About the Tower of Babel