IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs
Ali Faraz, Akash, Shaharukh Khan, Raja Kolla, Akshat Patidar, [2pt] Suranjan Goswami, Abhinav Ravi, Chandra Khatri, Shubham Agarwal, [10pt] Krutrim AI, Bangalore, India, [2pt] OLA Electric, Bangalore, India · 2025
Read IndicVisionBench: Benchmarking Cultural and Multilingual Understanding in VLMs by Ali Faraz, Akash, Shaharukh Khan, Raja Kolla, Akshat Patidar, [2pt] Suranjan Goswami, Abhinav Ravi, Chandra Khatri, Shubham Agarwal, [10pt] Krutrim AI, Bangalore, India, [2pt] OLA Electric, Bangalore, India (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-04727.txt.
Description
Ali Faraz leads nine authors with IndicVisionBench, a benchmark rooted in the Indian subcontinent. Most vision-language tests center Western content and English text. That leaves open how models handle India's languages and cultural scenes.
The bench covers English plus ten Indian languages across three task families. Tasks include optical character recognition, multimodal translation, and visual question answering. About five thousand images carry over thirty-seven thousand question pairs. Thirteen culturally grounded topics shape the content throughout. A paired annotation corpus in ten Indic languages is released alongside, supporting fairer tests for a huge user base.
Catalog record
| CREATOR | Ali Faraz, Akash, Shaharukh Khan, Raja Kolla, Akshat Patidar, [2pt] Suranjan Goswami, Abhinav Ravi, Chandra Khatri, Shubham Agarwal, [10pt] Krutrim AI, Bangalore, India, [2pt] OLA Electric, Bangalore, India |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-04727