DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos

Shenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg · 2026

Read DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos by Shenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06949.txt.

Description

Shenyuan Gao and colleagues present DreamDojo, a foundation world model for robots learned from 44,000 hours of egocentric human video. Simulating what actions do in varied environments could transform generalist robot development, but dexterous tasks suffer from limited data coverage and scarce action labels. Their data mixture is described as the largest video dataset yet assembled for world model pretraining, spanning everyday scenarios, objects, and skills.

To handle missing action labels, the team introduces continuous latent actions as unified proxy actions, improving transfer from unlabeled video. After post-training on small amounts of robot data, DreamDojo shows strong physics understanding and precise action control. A distillation pipeline speeds it to real-time operation at about 10.8 frames per second while improving context consistency, enabling live teleoperation, policy evaluation, and model-based planning, with strong results on out-of-distribution benchmarks.

Catalog record

CREATORShenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-06949

Browse the full library · About the Tower of Babel