Policy Learning from Large Vision-Language Model Feedback Without Reward Modeling
Tung M. Luu, Donghoon Lee, Younghwan Lee, Chang D. Yoo · 2025
Read Policy Learning from Large Vision-Language Model Feedback Without Reward Modeling by Tung M. Luu, Donghoon Lee, Younghwan Lee, Chang D. Yoo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23391.txt.
Description
Luu, Donghoon Lee, Younghwan Lee, and Yoo train robots from the opinions of a vision-language model. Offline reinforcement learning learns from stored experience, but the stored data usually need reward labels, and writing reward functions is slow expert work. Their method, PLARE, skips rewards entirely: the model is shown pairs of video segments and asked which better matches the spoken task.
The robot's policy is then trained straight from those preference labels with a contrastive objective, never building a reward model. On MetaWorld manipulation tasks, the approach matches or beats methods that do construct rewards from such models. A real robot arm confirms the gains carry into the physical world, which the authors offer as proof of practical value.
Catalog record
| CREATOR | Tung M. Luu, Donghoon Lee, Younghwan Lee, Chang D. Yoo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-23391