DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos
Shenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg · 2026
Read DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos by Shenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-06949.txt.
Description
Shenyuan Gao and colleagues present DreamDojo, a foundation world model for robots learned from 44,000 hours of egocentric human video. Simulating what actions do in varied environments could transform generalist robot development, but dexterous tasks suffer from limited data coverage and scarce action labels. Their data mixture is described as the largest video dataset yet assembled for world model pretraining, spanning everyday scenarios, objects, and skills.
To handle missing action labels, the team introduces continuous latent actions as unified proxy actions, improving transfer from unlabeled video. After post-training on small amounts of robot data, DreamDojo shows strong physics understanding and precise action control. A distillation pipeline speeds it to real-time operation at about 10.8 frames per second while improving context consistency, enabling live teleoperation, policy evaluation, and model-based planning, with strong results on out-of-distribution benchmarks.
Catalog record
| CREATOR | Shenyuan Gao 1,2† William Liang 1,3† Kaiyuan Zheng 1,4∗ Ayaan Malik 1,5∗ Seonghyeon Ye 1,6 Sihyun Yu 6 Wei-Cheng Tseng 1,7 Yuzhu Dong 1 Kaichun Mo 1 Chen-Hsuan Lin 1 Qianli Ma 1 Seungjun Nah 1 Loic Magne 1 Jiannan Xiang 8 Yuqi Xie 1 Ruijie Zheng 1 Dantong Niu 1,3 You Liang Tan 1 K.R. Zentner 1 Georg |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-06949