Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data

Yi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen · 2025

Read Efficient Reinforcement Learning by Guiding World Models with Non-Curated Data by Yi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19544.txt.

Description

Yi Zhao leads a team squeezing value from messy, unlabeled robot data to train agents faster. Their paper appeared in two thousand twenty-five. Offline data can warm-start online reinforcement learning. The richest pools are non-curated: reward-free, mixed quality, from many robot bodies. World models seem ideal for digesting such data. Naive fine-tuning, though, fails to speed later learning.

The team traces the failure to a distribution shift. Offline and online data pull the model in different directions. Experience rehearsal keeps old lessons alive during fine-tuning. Execution guidance steers how the model is used in action. Together they turn junk data into real speedups. Sample efficiency rises across many tasks. Robot learning gets thriftier.

Catalog record

CREATORYi Zhao Aidan Scannell Wenshuai Zhao Yuxin Hou Tianyu Cui, Le Chen Dieter Büchler Arno Solin Juho Kannala Joni Pajarinen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19544

Browse the full library · About the Tower of Babel