SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding

Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu · 2025

Read SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding by Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10810.txt.

Description

Zhenyu Yang and co-authors built a new test set for models that watch video as it plays, in a paper from 2025. Most past tests ask one question about one clip. Real use, like a live stream, needs a model to follow events over time and answer a chain of linked questions.

Their test set, called SVBench, holds close to fifty thousand question and answer pairs drawn from more than a thousand videos. The questions come in chains, so later ones build on earlier ones across the stream. The team tested fourteen models in both chat and live modes. A closed model from OpenAI did best, while most open models struggled with long streams. The team also built a chat model that beat the other open models on the new test.

Catalog record

CREATORZhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10810

Browse the full library · About the Tower of Babel