Enabling Off-Policy Imitation Learning with Deep Actor-Critic Stabilization
Sayambhu Sen, Shalabh Bhatnagar · 2025
Read Enabling Off-Policy Imitation Learning with Deep Actor-Critic Stabilization by Sayambhu Sen, Shalabh Bhatnagar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07288.txt.
Description
Sayambhu Sen and Shalabh Bhatnagar make adversarial imitation learning sample-efficient by moving it off-policy. Imitation learning copies an expert from demonstrations and avoids hand-engineering rewards, but the leading method, GAIL, builds on on-policy algorithms like TRPO and so burns through setting interactions. Their algorithm keeps the adversarial system while learning from stored experience instead of only fresh trajectories.
Two auxiliary tools do the stabilizing: a double Q-network curbs overestimation in the value estimates, and value learning proceeds without inferring a reward function. The combination reduces the number of samples needed to robustly match expert behavior. The work targets a useful bottleneck, since expert demonstrations and setting time are usually the scarcest resources in imitation learning pipelines.
Catalog record
| CREATOR | Sayambhu Sen, Shalabh Bhatnagar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-07288