Enabling Off-Policy Imitation Learning with Deep Actor-Critic Stabilization

Sayambhu Sen, Shalabh Bhatnagar · 2025

Read Enabling Off-Policy Imitation Learning with Deep Actor-Critic Stabilization by Sayambhu Sen, Shalabh Bhatnagar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07288.txt.

Description

Sayambhu Sen and Shalabh Bhatnagar make adversarial imitation learning sample-efficient by moving it off-policy. Imitation learning copies an expert from demonstrations and avoids hand-engineering rewards, but the leading method, GAIL, builds on on-policy algorithms like TRPO and so burns through setting interactions. Their algorithm keeps the adversarial system while learning from stored experience instead of only fresh trajectories.

Two auxiliary tools do the stabilizing: a double Q-network curbs overestimation in the value estimates, and value learning proceeds without inferring a reward function. The combination reduces the number of samples needed to robustly match expert behavior. The work targets a useful bottleneck, since expert demonstrations and setting time are usually the scarcest resources in imitation learning pipelines.

Catalog record

CREATORSayambhu Sen, Shalabh Bhatnagar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-07288

Browse the full library · About the Tower of Babel