How does longer temporal context enhance multimodal narrative video processing in the brain?
Prachi Jindal, Anant Khandelwal, Manish Gupta, Bapi S. Raju, Subba Reddy Oota, Tanmoy Chakraborty · 2026
Read How does longer temporal context enhance multimodal narrative video processing in the brain? by Prachi Jindal, Anant Khandelwal, Manish Gupta, Bapi S. Raju, Subba Reddy Oota, Tanmoy Chakraborty (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-07570.txt.
Description
Watching a movie engages the brain over many timescales, but how does clip length change brain-model match? Prachi Jindal and five co-authors vary time context from 3 to 24 seconds and test narrative-task prompting while participants watch full-length films in an fMRI scanner.
Longer clips substantially improve match for multi-modal large language models, while unimodal video models gain little. Short windows align with perceptual and early language regions, while longer windows preferentially align higher-order integrative regions. Brain experts and AI teams get evidence that narrative understanding needs time: models, like brains, integrate story over seconds, not frames.
Catalog record
| CREATOR | Prachi Jindal, Anant Khandelwal, Manish Gupta, Bapi S. Raju, Subba Reddy Oota, Tanmoy Chakraborty |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-07570