Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge

Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu · 2025

Read Streaming Video Understanding and Multi-round Interaction with Memory-enhanced Knowledge by Haomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13468.txt.

Description

Haomiao Xiong and six colleagues present StreamChat, a training-free framework for understanding streaming video and holding multi-round conversations about it. Existing video language models struggle with long sequences, multi-turn dialogue, and changing real-world scenes. StreamChat uses a hierarchical memory system to compress video features efficiently over extended streams.

A parallel scheduling strategy cuts processing latency for real-time interaction. The team also introduces StreamBench, a benchmark spanning diverse media types and interactive scenarios, including complex reasoning tasks. On StreamBench and other public benchmarks, StreamChat beat state-of-the-art models in both accuracy and response time. The authors release the code for the framework.

Catalog record

CREATORHaomiao Xiong, Zongxin Yang, Jiazuo Yu, Yunzhi Zhuge, Lu Zhang, Jiawen Zhu, Huchuan Lu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13468

Browse the full library · About the Tower of Babel