CURVE : A Benchmark for Cultural and Multilingual Long Video Reasoning
Darshan Singh, Arsha Nagrani, Kawshik Manikantan, Harman Singh, Dinesh Tewari, Tobias Weyand, Cordelia Schmid, Anelia Angelova, Shachi Dave · 2026
Read CURVE : A Benchmark for Cultural and Multilingual Long Video Reasoning by Darshan Singh, Arsha Nagrani, Kawshik Manikantan, Harman Singh, Dinesh Tewari, Tobias Weyand, Cordelia Schmid, Anelia Angelova, Shachi Dave (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10649.txt.
Introduction
Darshan Singh and co-authors present CURVE, a benchmark for video reasoning across cultures and languages. Video models have made fast progress on long clips. Yet most tests lean on western content and English text. That skew can hide weak spots when models face other cultures or tongues.
CURVE stands for Cultural Understanding and Reasoning in Video Evaluation. It tests models on long videos drawn from many cultures, with questions in many languages. Scores on it show whether a model truly grasps scenes beyond its home turf. The benchmark gives builders a fairer way to measure progress in global video understanding.
Catalog record
| CREATOR | Darshan Singh, Arsha Nagrani, Kawshik Manikantan, Harman Singh, Dinesh Tewari, Tobias Weyand, Cordelia Schmid, Anelia Angelova, Shachi Dave |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-10649
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CURVE : A Benchmark for Cultural and Multilingual Long Video Reasoning Darshan Singh, Arsha Nagrani, Kawshik Manikantan, Harman Singh, Dinesh Tewari, Tobias Weyand, Cordelia Schmid, Anelia Angelova, Shachi Dave Abstract Recent advancements in video models have shown tremendous progress, particularly in long video understanding. However, current benchmarks predominantly feature western-centric data and English as the dominant language, introducing significant biases in evaluation. To address this, we introduce CURVE ( C ultural U nderstanding and R easoning in V ideo E valuation), a challenging benchmark for multicultural and multilingual video reasoning. CURVE comprises high-quality, entirely human-generated annotations from diverse, region-specific cultural videos across 18 global locales. Unlike prior work that relies on automatic translations, CURVE provides complex questions, answers, and multi-step reasoning steps, all crafted in native languages. Making progress on CURVE requires a deeply situated understanding of visual cultural context. Furthermore, we leverage CURVE ’s reasoning traces to construct evidence-based graphs and propose a novel iterative strategy using these graphs to identify fine-grained errors in reasoning. Our evaluations reveal that SoTA Video-LLMs struggle significantly, performing substantially below human-level accuracy, with errors primarily stemming from the visual perception of cultural elements. CURVE will be publicly available here . 1 Introduction Figure 1 : CURVE for benchmarking Cultural and Multilingual Video Reasoning . We show examples from four distinct global locales: en-GB (English, UK), es-MX (Spanish, Mexico), ta-IN (Tamil, India), ar-EG (Arabic, Egypt) within the CURVE benchmark. Each example consists of a video , a complex native-language question , the ground-truth answer, and two representative and incorrect SOTA VLM responses . We also show the associated reasoning skills and annotated human reasoning steps provided with each question. Video understanding has made significant strides, yet current evaluation benchmarks exhibit a pervasive bias towards western and English-centric content ( 52 ; 46 ; 41 ) . This bias severely limits the ability of vision-language models (VLMs) to effectively operate in diverse global contexts, where cultural nuances, varied linguistic expressions, and unique visual characteristics are paramount ( 44 ) . While expanding language coverage, existing datasets often rely on direct translation of annotations ( 41 ; 31 ; 46 ) , but their visual content remains situated in Western concepts. To address this critical gap, we introduce CURVE , a challenging open-ended video question-answering benchmark. CURVE comprises a rich collection of videos depicting diverse cultural events (for example local sports, diverse festivals, local cuisine) from 18 regions and languages across the world (see Figure 1 for a snapshot). A crucial aspect of our dataset is that each video is uploaded with audio in a native language, and is paired with complex, multi-step reasoning questions crafted in the same native language. Each question is annotated by locally-situated experts with native cultural and language proficiency, and requires models to reason along different axes - (i) locale-specific cultural understanding, (ii) multimodal inference from audio-visual cues, and (iii) intricate temporal relationships (as videos vary in length from 1 minute to upto 1 hour). Benchmark Task Human Curated † Multi-cultural (# Regions) Multi-lingual (# Langs) Human Reasoning Traces Human Evals Video Durn. (avg) Percept.Test( 40 ) M ✓ ✗ ✗ ✗ ✓ 0.40 Cinepile( 42 ) M ✗ ✗ ✗ ✗ ✓ 2.66 MV-Bench( 28 ) M ✗ ✗ ✗ ✗ ✗ 16.00 TempCompass( 33 ) M ✗ ✗ ✗ ✗ ✓ 0.50 MMBench-Vid.( 15 ) O ✓ ✗ ✗ ✗ ✗ 2.75 Video-MME( 16 ) M ✓ ✗ ✗ ✗ ✗ 16.95 LongVideoB.( 56 ) M ✓ ✗ ✗ ✗ ✗ 7.91 MLVU( 66 ) M ✓ ✗ ✗ ✗ ✗ 15.5 EgoSchema( 34 ) M ✗ ✗ ✗ ✗ ✓ 3.00 Percept.Test’24( 18 ) M ✓ ✗ ✗ ✗ ✓ 60.00 VRBench( 60 ) B ✓ ✗ ✓ (8) ✓ ✗ 96.60 Neptune( 36 ) M ✗ ✗ ✗ ✗ ✓ 2.50 MINERVA( 35 M ✓ ✗ ✗ ✓ ✓ 12.00 M3-Med( 32 ) M ✓ ✗ ✓ (2) ✗ ✗ – ViMUL-Bench( 46 ) B ✗ ✓ (14) ✓ (14) ✗ ✗ 4.6 CURVE (Ours) O ✓ ✓ (18) ✓ (18) ✓ ✓ 12.62 Table 1 : Representative VideoQA benchmarks and key differences with CURVE . Task formats include Multiple Choice (M), Open-ended QA (O), or Both (B). CURVE introduces multilingual reasoning traces and human-curated multicultural videos spanning diverse regions. † Fully human curated (No synthetic data involved). Existing cultural benchmarks, such as ViMUL-Bench ( 46 ) , often focus solely on the final answer and overlook the underlying reasoning process. To allow for a more detailed analysis of model failures, each question in our dataset also comes with a detailed, multi-step reasoning trace. We use these reasoning traces to dig deeper into analyzing the failures made by models via a evidence-style Directed Acyclic Graph (DAG) analysis. This provides far more insight than a simple measure of final accuracy, allowing us to probe where and how models make mistakes. In our graph, nodes represent atomic evidences and edges signify prerequisites. A single error can cause cascading failures, creating a dilemma: penalizing all subsequent errors over-counts them, while stopping at the first limits diagnostic insight. Hence, we propose Iterative Error Isolation , a three-stage loop that traverses the evidence-style DAG, detects and tags errors, and iteratively re-evaluates with corrective hints until the problem is solved correctly. This process allows independent assessment of each intermediate step, revealing that the biggest mistakes are in the visual perception of cultural elements. Our contributions are: (i) We introduce CURVE , a multi-cultural, multi-lingual, long video reasoning benchmark for measuring and fostering the development of more equitable and globally competent multimodal AI systems. CURVE consists of 2400 questions from 540 videos from 18 locales all over the world. Questions in CURVE require various skills along with cultural understanding to answer. The benchmark will be publicly released to accelerate research towards truly multicultural video understanding. (ii) We benchmark SoTA frontier VLMs, and find a significant gap to human performance. We perform extensive analysis showing the effect of modalities, number of frames, and test-time computational scaling. (iii) We perform extensive error tagging using evidence graphs to analyze where the models tend to make mistakes, and show that 75% of all failures can be attributed to cultural visual perception. 2 Related Work Figure 2 : CURVE Benchmark Statistics. (a) CURVE contains a wide range of video durations, ranging from one minute to over an hour. (b) Human-authored reasoning traces are long and detailed, often spanning hundreds of words, while the corresponding answers are concise. (c) Questions are spread across six core cultural domains. Multicultural-Multimodal Evaluations. Equitable development and evaluation of foundation models requires cultural and linguistic inclusivity ( 41 ; 44 ; 37 ; 31 ; 65 ; 11 ; 52 ; 24 ) . Early efforts like xGQA ( 41 ) and MaRVL ( 31 ) expanded language coverage by translating English annotations, but retained Western-centric imagery and potential translation artifacts. Subsequent benchmarks such as CVQA ( 44 ) and CulturalVQA ( 37 ) incorporated culturally specific imagery, revealing significant performance gaps in modern VLMs while CUBE ( 23 ) showed a lack of diversity in generations from T2I models. Other works have targeted multilingual image captioning with diverse geography (Crossmodal-3600 ( 49 ) ),