Latent Wasserstein Adversarial Imitation Learning

Siqi Yang, Kai Yan, Alexander G. Schwing, Yu-Xiong Wang · 2026

Read Latent Wasserstein Adversarial Imitation Learning by Siqi Yang, Kai Yan, Alexander G. Schwing, Yu-Xiong Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05440.txt.

Introduction

Siqi Yang and team offer Latent Wasserstein Adversarial Imitation Learning. Imitation learning mimics experts. Older methods require high-quality demonstrations and expert actions. To reduce this need, Wasserstein distance in a space is used. This latent space, obtained via pre-training, has the Aim Conditioned Value Function capturing state-space structure. This lifts policy grasp of state shifts, letting expert-level scores with few episodes. Through MuJoCo tests, their method outperforms prior Wasserstein-based and adversarial methods, achieving better results.

Catalog record

CREATORSiqi Yang, Kai Yan, Alexander G. Schwing, Yu-Xiong Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE84160
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.05440

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Latent Wasserstein Adversarial Imitation Learning Siqi Yang, Kai Yan, Alexander G. Schwing, Yu-Xiong Wang Abstract Imitation Learning (IL) enables agents to mimic expert behavior by learning from demonstrations. However, traditional IL methods require large amounts of medium-to-high-quality demonstrations as well as actions of expert demonstrations, both of which are often unavailable. To reduce this need, we propose Latent Wasserstein Adversarial Imitation Learning (LWAIL), a novel adversarial imitation learning framework that focuses on state-only distribution matching. It benefits from the Wasserstein distance computed in a dynamics-aware latent space. This dynamics-aware latent space differs from prior work and is obtained via a pre-training stage, where we train the Intention Conditioned Value Function (ICVF) to capture a dynamics-aware structure of the state space using a small set of randomly generated state-only data. We show that this enhances the policy’s understanding of state transitions, enabling the learning process to use only one or a few state-only expert episodes to achieve expert-level performance. Through experiments on multiple MuJoCo environments, we demonstrate that our method outperforms prior Wasserstein-based IL methods and prior adversarial IL methods, achieving better results across various tasks. 1 Introduction As a powerful tool for solving sequential decision-making problems, Reinforcement Learning (RL) has achieved remarkable success in recent years across various fields, such as gaming ( 67 ) and training large language models ( 61 ; 26 ) . However, RL relies heavily on well-defined reward signals ( 34 ; 51 ) , which can be difficult to obtain in real-world settings (e.g., robot control ( 33 ) with varied target tasks) or may require careful, environment-specific considerations ( 87 ) . Imitation Learning (IL) offers a compelling alternative by learning directly from expert demonstrations, thus bypassing the need for explicit reward design. Within IL, a major challenge are the frequently missing expert actions, encouraging adoption of Imitation Learning from Observations (LfO), which uses only sequences of expert states. However, even these state-only demonstrations are often expensive to acquire, making it crucial to develop methods that can learn from a minimal amount of expert data. Adversarial Imitation Learning (AIL) methods ( 73 ; 94 ) , which learn by matching the distribution of agent states to that of the expert, are a popular approach to LfO. Many such AIL methods measure the distribution difference by f -divergence, such as KL-, \chi^{2} - or JS-divergence. However, f -divergences require “distribution coverage”, i.e., the distributions in f -divergence must be on the same support set to avoid numerical error. This could lead to additional theoretical constraints, especially when an additional dataset of non-expert data is involved; for example, offline LfO methods such as SMODICE ( 56 ) require the non-expert state distribution to fully cover the expert state distribution, which does not necessarily hold in practice especially when the non-expert data quality is low (e.g., random). To address these limitations, the Wasserstein distance has gained popularity ( 5 ; 92 ) . However, many prior Wasserstein IL works that employ the Kantorovich-Rubinstein (KR) dual ( 92 ; 21 ; 69 ) overlook an important issue: the distance metric between individual states is rather simplistic. Indeed, to compute the Wasserstein distance between distributions of states, a “ground metric” or cost for measuring the distance between any two individual states is first defined. For this, the Euclidean distance is common ( 92 ; 21 ) . However, it fails to capture the environment’s dynamics (see Fig. 1 (a) for an illustration). This deficiency can severely mislead the learning process. Existing solutions bypass the issue by using the primal Wasserstein distance, which however introduces other complexities and requires surrogates ( 84 ; 14 ) . This raises a critical question: Can we learn a distance metric that encodes the environment’s dynamics from a minimum amount of state-only low-quality data , allowing an agent to learn efficiently from just a few, state-only expert trajectories? (a) Better distance metric (b) Pre-training stage (c) Imitation stage Figure 1: Illustrating our motivation for a better distance metric and an outline of the algorithm. Panel (a) illustrates a case where the Euclidean distance between states is not a good metric: state B is closer to expert state C, but it is apparently less desirable than more distant state A, as it cannot reach C. To fix, we use random data and ICVF to find a more meaningful embedding space, as shown in the lower half. Panel (b), together with the lower half of panel (a), shows our pre-training stage: we first train ICVF to obtain \phi(s) , which serves as a reward for our agent in the online stage shown in panel (c). Fire indicates trainable modules, and snowflakes indicate frozen modules. To answer this question, we introduce a novel Wasserstein AIL algorithm that directly addresses this foundational issue. We propose a two-stage process. First, in the pre-training stage, we leverage a small ( 1\% of online rollouts) number of unstructured, low-quality (e.g., random) state-only data to train an Intention-Conditioned Value Function (ICVF) ( 22 ) . The resulting embedding captures a rich, dynamics-aware notion of reachability between states. Second, during the imitation stage, we freeze this ICVF embedding and use the Euclidean distance in this new latent space as the cost function within a standard Wasserstein AIL framework. By modifying the agent and discriminator to operate in this learned space, we fix the core issue of prior methods. We find that this simple approach allows an agent to effectively match the expert’s state distribution, enabling highly efficient imitation from minimal expert data. We validate our approach on Umaze and challenging locomotion tasks in the MuJoCo environment from the D4RL benchmark ( 18 ) , and show that the latent space grasps the transition dynamics much better than the vanilla Euclidean distance. In terms of imitation learning, we achieve strong results using only a single trajectory of state-based expert data. We summarize our contributions as follows: 1) We show that the ICVF latent space captures a dynamics-aware ground metric even if only a small amount of (possibly low-quality) state-only data is available; this is a first direct remedy for the known geometric limitations of the Euclidean distance in prior Wasserstein occupancy matching methods. 2) We propose a simple but effective improvement to current Wasserstein AIL methods achieving expert-level performance with a single state-only expert trajectory; 3) We empirically show that our method outperforms various baselines on multiple testbeds, proving that a better distance metric can greatly benefit AIL. 2 Preliminaries Markov Decision Process (MDP). A Markov Decision Process (MDP) is a mathematical framework which describes the interactions of an agent with an environment at discrete time steps. It is defined by the tuple (\mathcal{S},\mathcal{A},P,R,\gamma) , where \mathcal{S} represents the state space, and \mathcal{A} denotes the action space. The state transition probability function P(s^{\prime}|s,a) defines the likelihood of transitioning to a new state s^{\prime}\in\mathcal{S} after taking an action a\in\mathcal{A} in the current state s\in\mathcal{S} . The reward function R(s,a)\in\mathbb{R} specifies the immediate reward received after taking action a in state s . \gamma\in[0,1{\color[rgb]{0,0,0})} is the discount factor, determining the importance of future rewards relative to immediate ones. At each time step t , the agent observes the current state s_{t}\in\mathcal{S} , selects an action

Read the complete text