Vulca-Bench : A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding
Haorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi · 2026
Read Vulca-Bench : A Multicultural Vision-Language Benchmark for Evaluating Cultural Understanding by Haorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-07986.txt.
Description
Haorui Yu and colleagues release Vulca-Bench, a benchmark for cultural understanding in vision-language models. Existing benchmarks mostly measure basic skills: object recognition, scene description, and factual questions. Higher-order cultural interpretation is under-evaluated. Vulca-Bench holds 7,410 matched image and critique pairs. They span eight cultural traditions, with Chinese and English bilingual coverage. Their 2026 preprint introduces the resource.
Matched critiques let evaluators check whether a model reads an artwork's cultural meaning, not just its content. Bilingual coverage tests whether understanding survives language change. Eight traditions guard against benchmarks that quietly center one culture. The resource gives model builders a fair test they can rerun. The paper serves the multimodal AI community. It extends a companion line of work on cross-cultural art critique into a full benchmark.
Catalog record
| CREATOR | Haorui Yu, Ramon Ruiz-Dolz, Diji Yang, Hang He, Fengrui Zhang, Qiufeng Yi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-07986