Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Language
Minyoung Hwang, Alexandra Forsey-Smerek, Nathaniel Dennler, Andreea Bobu · 2025
Read Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Language by Minyoung Hwang, Alexandra Forsey-Smerek, Nathaniel Dennler, Andreea Bobu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14565.txt.
Description
Minyoung Hwang and three co-authors propose Masked IRL, a method for teaching robots user preferences from demos plus language. Demos alone show how to act, but not what matters, so reward models can latch onto spurious details. Language can state what matters, yet instructions are often vague on their own. The key idea is that the two signals complement each other.
Large language models help combine demos and language to resolve reward ambiguity. The framework masks state details that language marks as irrelevant. That focus should improve generalization from limited data. The method targets household and assistive settings where preferences differ per user. Better reward learning means robots adapt without extensive new demos.
Catalog record
| CREATOR | Minyoung Hwang, Alexandra Forsey-Smerek, Nathaniel Dennler, Andreea Bobu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-14565