SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding
Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu · 2025
Read SVBench: A Benchmark with Temporal Multi-Turn Dialogues for Streaming Video Understanding by Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10810.txt.
Description
Zhenyu Yang and co-authors built a new test set for models that watch video as it plays, in a paper from 2025. Most past tests ask one question about one clip. Real use, like a live stream, needs a model to follow events over time and answer a chain of linked questions.
Their test set, called SVBench, holds close to fifty thousand question and answer pairs drawn from more than a thousand videos. The questions come in chains, so later ones build on earlier ones across the stream. The team tested fourteen models in both chat and live modes. A closed model from OpenAI did best, while most open models struggled with long streams. The team also built a chat model that beat the other open models on the new test.
Catalog record
| CREATOR | Zhenyu Yang, Yuhang Hu, Zemin Du, Dizhan Xue, Shengsheng Qian, Jiahong Wu, Fan Yang, Weiming Dong, Changsheng Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-10810