DreamGen : Unlocking Generalization in Robot Learning through Video World Models
Joel Jang Seonghyeon Ye Zongyu Lin Jiannan Xiang, Johan Bjorck Yu Fang Fengyuan Hu Spencer Huang Kaushil Kundalia Lin Yen-Chen Loic Magne, Ajay Mandlekar Avnish Narayan You Liang Tan Guanzhi Wang Jing Wang Qi Wang Yinzhen Xu, Xiaohui Zeng Kaiyuan Zheng Ruijie Zheng, Ming-Yu Liu Luke Zettlemoyer Diet · 2025
Read DreamGen : Unlocking Generalization in Robot Learning through Video World Models by Joel Jang Seonghyeon Ye Zongyu Lin Jiannan Xiang, Johan Bjorck Yu Fang Fengyuan Hu Spencer Huang Kaushil Kundalia Lin Yen-Chen Loic Magne, Ajay Mandlekar Avnish Narayan You Liang Tan Guanzhi Wang Jing Wang Qi Wang Yinzhen Xu, Xiaohui Zeng Kaiyuan Zheng Ruijie Zheng, Ming-Yu Liu Luke Zettlemoyer Diet (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12705.txt.
Description
Joel Jang and a large team of co-authors let robots learn from videos that never happened in a 2025 paper. Teleoperating robots to gather training data is slow and narrow. Their pipeline, DreamGen, instead asks video generators to dream the robot doing tasks, then works backward from dream to action.
The dreamed clips are photorealistic and varied, spanning new tasks and rooms. A second model recovers the joint commands that would produce each dream. Trained on dreams seeded by a single real pick-and-place demonstration, a humanoid performed twenty-two new behaviors in familiar and novel rooms alike. A companion benchmark grades dream quality, and its scores track real robot success.
Catalog record
| CREATOR | Joel Jang Seonghyeon Ye Zongyu Lin Jiannan Xiang, Johan Bjorck Yu Fang Fengyuan Hu Spencer Huang Kaushil Kundalia Lin Yen-Chen Loic Magne, Ajay Mandlekar Avnish Narayan You Liang Tan Guanzhi Wang Jing Wang Qi Wang Yinzhen Xu, Xiaohui Zeng Kaiyuan Zheng Ruijie Zheng, Ming-Yu Liu Luke Zettlemoyer Diet |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-12705