Proact-VL: A Proactive VideoLLM for Real-Time AI Companions

Weicai Yan, Yuhong Dai, Qi Ran, Haodong Li, Wang Lin, Hao Liao, Xing Xie, Tao Jin, Jianxun Lian · 2026

Read Proact-VL: A Proactive VideoLLM for Real-Time AI Companions by Weicai Yan, Yuhong Dai, Qi Ran, Haodong Li, Wang Lin, Hao Liao, Xing Xie, Tao Jin, Jianxun Lian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03447.txt.

Description

Weicai Yan and eight colleagues shape video language models into companions that speak up on their own. A real-time companion watching a live stream must react fast, decide for itself when to talk, and control how much it says. Their 2026 framework, Proact-VL, is tested in gaming, where companions act as commentator or guide, roles that allow automatic scoring.

The team also built the Live Gaming Benchmark, a large dataset covering solo commentary, co-commentary with a partner, and guidance of a user. Proact-VL reshapes a multimodal model into an agent that perceives the environment and interacts proactively in a human-like rhythm. Extensive experiments show better response latency and quality than rivals, while video understanding stays strong. The authors present it as practical for real-time interactive applications.

Catalog record

CREATORWeicai Yan, Yuhong Dai, Qi Ran, Haodong Li, Wang Lin, Hao Liao, Xing Xie, Tao Jin, Jianxun Lian
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE102935
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03447

Browse the full library · About the Tower of Babel