Reinforcement Learning with Inverse Rewards for World Model Post-training

Yang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1 · 2025

Read Reinforcement Learning with Inverse Rewards for World Model Post-training by Yang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23958.txt.

Description

A video world model that ignores the player's commands is a movie, not a game. Yang Ye and 3 co-authors improve action-following in such models with Reinforcement Learning with Inverse Rewards. Human preference labels at scale are too costly, and rule-based video checkers are impractical. Their trick inverts the problem: an inverse dynamics model recovers the input actions from generated video, turning high-dimensional video into a low-dimensional, checkable action signal.

That recovered action supplies a verifiable reward for group-based policy optimization. Across autoregressive and diffusion world models, action-following rose by five to ten percent, with visual quality and human preference also improving.

Catalog record

CREATORYang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23958

Browse the full library · About the Tower of Babel