Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction

Rui Qian Shuangrui Ding Xiaoyi Dong Pan Zhang, Yuhang Zang Yuhang Cao Dahua Lin Jiaqi Wang · 2025

Read Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction by Rui Qian Shuangrui Ding Xiaoyi Dong Pan Zhang, Yuhang Zang Yuhang Cao Dahua Lin Jiaqi Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-03218.txt.

Description

A video assistant that answers only after watching the whole clip is useless for live interaction. Rui Qian and seven co-authors present Dispider, a system that watches a stream, decides when to speak, and responds, all at once. The trick is splitting the three jobs apart, since they fight each other inside a single model. Deciding and replying want detail, while watching wants speed, and generating text normally freezes the watching.

A light module tracks the stream and spots good moments to interact. When one fires, a separate module composes a detailed reply while the watcher keeps watching. Tests show the system holds its own on standard video questions and clearly beats earlier online models on live streaming responses. Code and model are public, from a 2025 paper.

Catalog record

CREATORRui Qian Shuangrui Ding Xiaoyi Dong Pan Zhang, Yuhang Zang Yuhang Cao Dahua Lin Jiaqi Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-03218

Browse the full library · About the Tower of Babel