Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data
Yi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen · 2025
Read Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data by Yi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19544.txt.
Description
Yi Zhao leads a team squeezing value from messy, unlabeled robot data to train agents faster. Their paper appeared in two thousand twenty-five. Offline data can warm-start online reinforcement learning. The richest pools are non-curated: reward-free, mixed quality, from many robot bodies. World models seem ideal for digesting such data. Naive fine-tuning, though, fails to speed later learning.
The team traces the failure to a distribution shift. Offline and online data pull the model in different directions. Experience rehearsal keeps old lessons alive during fine-tuning. Execution guidance steers how the model is used in action. Together they turn junk data into real speedups. Sample efficiency rises across many tasks. Robot learning gets thriftier.
Catalog record
| CREATOR | Yi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-19544