Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations

Huy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma · 2025

Read Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations by Huy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21182.txt.

Description

Huy Hoang and co-authors teach a learning agent from both good and bad examples in a 2025 paper. Offline imitation learning trains an agent on recorded behavior without live trial and error. It usually leans on expert records and unlabeled ones, and throws away a rich signal: records that are explicitly marked as undesirable. The authors ask what those negative lessons can teach.

Their formulation scores a policy by the gap between how closely it matches the expert records and how closely it matches the bad ones. That objective is a difference of convex parts, a form that is often hard. They prove it turns convex when expert data outweighs bad data, which yields a stable training method with no adversarial game. Tests on standard imitation benchmarks judge the gain. Learning what not to do, it turns out, sharpens what to do.

Catalog record

CREATORHuy Hoang, Tien Mai, Pradeep Varakantham, Tanvi Verma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21182

Browse the full library · About the Tower of Babel