Online Reasoning Video Segmentation with Just-in-Time Digital Twins

Yiqing Shen, Bohan Liu, Chenjia Li, Lalithkumar Seenivasan, Mathias Unberath · 2025

Read Online Reasoning Video Segmentation with Just-in-Time Digital Twins by Yiqing Shen, Bohan Liu, Chenjia Li, Lalithkumar Seenivasan, Mathias Unberath (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21056.txt.

Description

Yiqing Shen, Bohan Liu, Chenjia Li, Lalithkumar Seenivasan, and Mathias Unberath present a 2025 paper on finding objects in video from vague text requests. A user might describe a target in indirect words, and the system must outline it frame by frame, live. Such skill would let embodied AI agents follow high-level commands.

Current methods lean on the raw perception of large vision-language models and need costly fine-tuning. The authors split the job: a language model plans, and specialist vision tools gather facts. Given a query, the planner builds a light digital twin of the scene, just in time, requesting only the details it needs. This avoids retraining the big model and works on streaming video.

Catalog record

CREATORYiqing Shen, Bohan Liu, Chenjia Li, Lalithkumar Seenivasan, Mathias Unberath
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-21056

Browse the full library · About the Tower of Babel