Reinforcement Learning with Inverse Rewards for World Model Post-training
Yang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1 · 2025
Read Reinforcement Learning with Inverse Rewards for World Model Post-training by Yang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23958.txt.
Description
A video world model that ignores the player's commands is a movie, not a game. Yang Ye and 3 co-authors improve action-following in such models with Reinforcement Learning with Inverse Rewards. Human preference labels at scale are too costly, and rule-based video checkers are impractical. Their trick inverts the problem: an inverse dynamics model recovers the input actions from generated video, turning high-dimensional video into a low-dimensional, checkable action signal.
That recovered action supplies a verifiable reward for group-based policy optimization. Across autoregressive and diffusion world models, action-following rose by five to ten percent, with visual quality and human preference also improving.
Catalog record
| CREATOR | Yang Ye 1 Tianyu He 1 Shuo Yang Jiang Bian 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23958