TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion

Weishi Mi, Yong Bao, Xiaowei Chi, Xiaozhu Ju, Zhiyuan Qin, Kuangzhi Ge, Kai Tang, Peidong Jia, Shanghang Zhang, Jian Tang · 2026

Read TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion by Weishi Mi, Yong Bao, Xiaowei Chi, Xiaozhu Ju, Zhiyuan Qin, Kuangzhi Ge, Kai Tang, Peidong Jia, Shanghang Zhang, Jian Tang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18323.txt.

Description

Weishi Mi and co-authors ground video generation so its plans can drive a real robot. Their 2026 preprint presents TC-IDM. The vision-language-action paradigm gives strong robot control, but it feeds on large sets of costly robot data. Generative world models promise an alternative: dream the scene, then act. The gap is that pixel-level plans are not yet physical actions a robot can run. TC-IDM aims to close that gap for zero-shot robot motion, with no task-specific training.

Executable plans would let one world model serve many robots. The grounding step is where many embodied AI ideas fail. Success here would loosen the field's dependence on hand-collected robot data.

Catalog record

CREATORWeishi Mi, Yong Bao, Xiaowei Chi, Xiaozhu Ju, Zhiyuan Qin, Kuangzhi Ge, Kai Tang, Peidong Jia, Shanghang Zhang, Jian Tang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18323

Browse the full library · About the Tower of Babel