Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization
Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London · 2026
Read Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization by Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26282.txt.
Description
Xiaoyuan Cheng and seven colleagues' 2026 paper names a basic block in scaling world-model reinforcement learning: search and value learning pull apart. They propose Model-Based Diffusion Policy Optimization, which joins search with policy optimization via diffusion policy forms. Rather than a planner, policy learning becomes a flow over searched paths in latent world models, held by a hidden energy function. Tests across offline pretraining, online learning, and fine-tuning from offline to online show steady, rising gains with model size.
Catalog record
| CREATOR | Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 114969 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.26282