Vulca-Bench : A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding

Haorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi · 2026

Read Vulca-Bench : A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding by Haorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07986.txt.

Description

Haorui Yu and colleagues release Vulca-Bench, a benchmark for cultural understanding in vision-language models. Existing benchmarks mostly measure basic skills: object recognition, scene description, and factual questions. Higher-order cultural interpretation is under-evaluated. Vulca-Bench holds 7,410 matched image and critique pairs. They span eight cultural traditions, with Chinese and English bilingual coverage. Their 2026 preprint introduces the resource.

Matched critiques let evaluators check whether a model reads an artwork's cultural meaning, not just its content. Bilingual coverage tests whether understanding survives language change. Eight traditions guard against benchmarks that quietly center one culture. The resource gives model builders a fair test they can rerun. The paper serves the multimodal AI community. It extends a companion line of work on cross-cultural art critique into a full benchmark.

Catalog record

CREATORHaorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-07986

Browse the full library · About the Tower of Babel