Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization

Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London · 2026

Read Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimization by Xiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26282.txt.

Description

Xiaoyuan Cheng and seven colleagues' 2026 paper names a basic block in scaling world-model reinforcement learning: search and value learning pull apart. They propose Model-Based Diffusion Policy Optimization, which joins search with policy optimization via diffusion policy forms. Rather than a planner, policy learning becomes a flow over searched paths in latent world models, held by a hidden energy function. Tests across offline pretraining, online learning, and fine-tuning from offline to online show steady, rising gains with model size.

Catalog record

CREATORXiaoyuan Cheng, Wenxuan Yuan, Zhancun Mu, Yuanzhao Zhang, Yiming Yang, Hai Wang, Zhuo Sun, Che Liu, [0.3cm] Dynamic Systems Lab, University College London
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE114969
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.26282

Browse the full library · About the Tower of Babel