VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models

Pritam Sarkar Queen’s University, Canada and Vector Institute [email protected] &Ali Etemad Queen’s University, Canada [email protected] Website Code Data · 2025

Read VCRBench: Exploring Long-form Causal Reasoning Capabilities of Large Video Language Models by Pritam Sarkar Queen’s University, Canada and Vector Institute [email protected] &Ali Etemad Queen’s University, Canada [email protected] Website Code Data (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08455.txt.

Description

Pritam Sarkar and Ali Etemad published this benchmark study in 2025. Video language models can describe clips, but can they reason about cause and effect across a long activity? The team built VCRBench to find out. It uses videos of simple everyday tasks whose steps have been deliberately shuffled. Each clip captures one key causal event. The model must work out the right order of steps to reach the goal.

The design blocks shortcut answers that multiple-choice formats allow. Leading models struggled badly on the benchmark. Their main weakness was tracking long-range cause and effect straight from the images. As a remedy, the authors split the job in two: first recognize what happens, then reason about the sequence separately. That decomposition raised accuracy by up to twenty-five points. Analysis also showed the models lean heavily on language knowledge rather than truly watching the causal chain.

Catalog record

CREATORPritam Sarkar Queen’s University, Canada and Vector Institute [email protected] &Ali Etemad Queen’s University, Canada [email protected] Website Code Data
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08455

Browse the full library · About the Tower of Babel