Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations
Huy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma · 2025
Read Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations by Huy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21182.txt.
Description
Huy Hoang and co-authors teach a learning agent from both good and bad examples in a 2025 paper. Offline imitation learning trains an agent on recorded behavior without live trial and error. It usually leans on expert records and unlabeled ones, and throws away a rich signal: records that are explicitly marked as undesirable. The authors ask what those negative lessons can teach.
Their formulation scores a policy by the gap between how closely it matches the expert records and how closely it matches the bad ones. That objective is a difference of convex parts, a form that is often hard. They prove it turns convex when expert data outweighs bad data, which yields a stable training method with no adversarial game. Tests on standard imitation benchmarks judge the gain. Learning what not to do, it turns out, sharpens what to do.
Catalog record
| CREATOR | Huy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-21182