Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning
Udita Ghosh, Dripta S. Raychaudhuri, Jiachen Li, Konstantinos Karydis, Amit Roy-Chowdhury · 2025
Read Preference VLM: Leveraging VLMs for Scalable Preference-Based Reinforcement Learning by Udita Ghosh, Dripta S. Raychaudhuri, Jiachen Li, Konstantinos Karydis, Amit Roy-Chowdhury (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-01616.txt.
Description
Udita Ghosh and co-authors use vision-language models to cut the human feedback needed to train robots by preference, in a paper from 2025. Preference learning steers a policy by asking which of two tries looks better. Human labels are dear, so the bill grows fast. Their framework, PrefVLM, lets a vision model draft the labels first.
Only the cases where the model feels unsure go to a human. A self-taught loss also tunes the vision model to match the shifting policy. On robot arm tasks, the method matched or beat top rivals. It used up to half the human labels of those rivals. Skills learned on one task carried to others through the tuned model. The authors say the blend makes preference training far more practical.
Catalog record
| CREATOR | Udita Ghosh, Dripta S. Raychaudhuri, Jiachen Li, Konstantinos Karydis, Amit Roy-Chowdhury |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-01616