Ergodic Imitation for Adaptive Exploration around Demonstrations

Ziyi Xu, Cem Bilaloglu, Yiming Li, Sylvain Calinon · 2026

Read Ergodic Imitation for Adaptive Exploration around Demonstrations by Ziyi Xu, Cem Bilaloglu, Yiming Li, Sylvain Calinon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13996.txt.

Introduction

Xu, Bilaloglu, Li and Calinon offer ergodic imitation, a robot drill that stays near taught demos while it probes on its own. When live scenes stray from training, a bot that just tracks a set path can stall and fail. Their fix builds a goal spread from demo shapes, then makes paths that blend tracking with probing. A fetch-based look-ahead frame runs the blend, pushing ergodic control past mere area cover and search.

Catalog record

CREATORZiyi Xu, Cem Bilaloglu, Yiming Li, Sylvain Calinon
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE14752
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13996

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Ergodic Imitation for Adaptive Exploration around Demonstrations Ziyi Xu, Cem Bilaloglu, Yiming Li, Sylvain Calinon Abstract In robotics, a common challenge in imitation learning is the mismatch between training and deployment conditions, caused, for example, by environmental changes or imperfect observation and control. When a robot follows a nominal trajectory under such mismatch, it may become stuck and fail to complete the task. This calls for adaptive online exploration strategies that remain grounded in demonstrations. To this end, we propose an adaptive ergodic imitation approach that constructs a target distribution from the geometry of the retrieved demonstrations and uses it to generate trajectories that adaptively interpolate between tracking and exploration. Our method extends ergodic control beyond its traditional role in area-coverage and search by incorporating demonstrations into a retrieval-based receding-horizon framework for adaptive imitation. I INTRODUCTION Fig. 1 : Overview of Adaptive Ergodic Imitation . A nominal trajectory induces tracking behavior when execution remains aligned with the demonstration. Under mismatch, the target particle distribution expands and the ergodic planner promotes exploration around the reference. Once the obstacle is bypassed, the score-based kernel contracts the distribution and pulls the agent back toward the demonstrated trajectory. Imitation learning (IL) is a practical paradigm for robot programming, where the intended behavior is learned by demonstrations without an explicit reward. In practice, however, the objective is rarely to replicate a demonstration exactly; rather, the agent must adapt the observed behaviors to novel environments that might diverge from the training distribution. Recent critiques of deep generative models in robotics suggest that these architectures often overfit the demonstration data, essentially memorizing specific action sequences rather than learning generalizable policies [ 3 , 8 ] . Consequently, current IL approaches remain notoriously brittle even under minimal distribution shifts. This necessitates a shift toward imitation paradigms that prioritize situational adaptation over pure trajectory replay. Although recent adaptive exploration methods allow systems to sample different modes from the dataset, they are largely restricted to discrete transitions [ 11 , 6 ] . Consequently, they lack the continuous exploration capabilities needed for the subtle, state-space adjustments typical of tasks like robotic assembly. We employ a different perspective that treats demonstrations as references to be tracked under nominal conditions, by using them as informed priors for exploration when environmental shifts render pure tracking insufficient. Instead of considering tracking and exploring as two different objectives, we use the ergodic control methodology to formulate tracking as a special case of exploration. This results in a continuous behavioral spectrum, from rigid imitation to adaptive exploration, governed by the same underlying controller. Ergodic control synthesizes trajectories whose time-averaged state visitation statistics converge to a target spatial distribution [ 9 ] . Traditionally, this framework is employed for exploration [ 10 ] , area coverage [ 1 ] , or search operations [ 5 ] , using mobile robots and UAVs. More recently, several works have extended ergodic control to the IL setting, applying it to tasks such as robotic insertion [ 12 , 13 ] , cart-pole inversion, and surface cleaning [ 7 ] . In this imitation context, demonstrations are leveraged to construct a static target distribution that the ergodic controller used for task reproduction. A primary advantage of ergodic imitation is its reliance on statistical state-visitation rather than strict temporal sequencing; this allows the robot to synthesize successful behaviors without being tethered to the demonstrator’s specific timeline [ 7 ] . While this flexibility is beneficial for tasks like cleaning or insertion, temporal dependency remains critical for most manipulation tasks. To address this, we propose an adaptive formulation that strikes a balance between these two paradigms. Our approach reproduces the temporal characteristics of the demonstration when the environment permits, yet autonomously transitions to ergodic exploration if the robot becomes "stuck" and can not track the reference motion. In this work, we introduce an adaptive ergodic imitation approach that brings the target distribution-driven exploration mechanism of ergodic control into imitation learning for runtime adaptation. Our work positions ergodic control as a promising direction for generalization in imitation learning, not only as a tool for search or coverage, but as a principled mechanism for adaptive execution around demonstrations. Our contributions are as follows: • a unified method for a continuous spectrum of tracking and exploration behaviors using adaptive ergodic imitation; • task progress estimation via demonstration to adaptively modify target distributions; • geometry-guided anisotropic diffusion for synthesizing target distributions inducing tracking and exploration; • using Maximum Mean Discrepancy (MMD) ergodic metric [ 4 ] in a retrieval-based imitation learning framework. II METHODOLOGY We consider a dataset \mathcal{D}=\left\{\Gamma^{(i)}\right\}_{i=1}^{N} of N expert demonstrations, where each demonstration is a state trajectory \Gamma^{(i)}=\left\{\bm{x}_{t}^{(i)}\right\}_{t=0}^{T_{i}} . Since the proposed framework utilizes an ergodic controller to derive control laws based on target distributions, the expert actions \bm{u}_{t}^{(i)} are omitted, and we focus exclusively on the state trajectories. We define the aggregate state set \mathcal{P}=\bigcup_{i,t}\left\{\bm{x}_{t}^{(i)}\right\} as the flattened collection of all expert states. II-A Background A dynamical system’s time-averaged state trajectory defines an empirical distribution, or coverage, as p(t,\bm{x})=\frac{1}{t}\int_{0}^{t}\delta(\bm{x}-\bm{x}(\tau))d\tau , where \delta(\cdot) is the Dirac delta function. The system is ergodic with respect to a target distribution q(\bm{x})\in\mathcal{S}(\mathcal{X}) if its coverage converges weakly to the target: \lim_{T\rightarrow\infty}\mathbb{E}_{\bm{x}\sim p_{T}}[\phi(\bm{x})]=\mathbb{E}_{\bm{x}\sim q}[\phi(\bm{x})],\quad\forall\phi\in\mathcal{C}(\mathcal{X}). (1) The objective of ergodic control is to synthesize control commands that ensure the system’s long-term coverage matches the desired distribution q(\bm{x}) , typically by minimizing a discrepancy measure between p and q . MMD provides a discrepancy measure for ergodicity [ 4 ] , particularly effective when the target distribution is known only through a set of discrete samples \left\{\bm{q}_{i}\right\}_{i=1}^{N}\subset\mathcal{X} . For a trajectory represented by discrete points \bm{x}_{t} , the squared MMD between the trajectory distribution p_{\bm{x}} and the target distribution q is approximated as: \displaystyle\overline{\mathrm{MMD}} {}_{k}^{2}(p,q)=\frac{1}{T^{2}}\sum_{t=0}^{T-1}\sum_{t^{\prime}=0}^{T-1}k(\bm{x}_{t},\bm{x}_{t^{\prime}}) \displaystyle-\frac{2}{TN}\sum_{t=0}^{T-1}\sum_{i=1}^{N}k(\bm{x}_{t},\bm{q}_{i})+z(q), (2) where z(q)=\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j=1}^{N}k(\bm{q}_{i},\bm{q}_{j}) is a constant term that depends solely on the target distribution. II-B Phase Retrieval At each re-planning interval, the system queries the dataset \mathcal{D} using the current robot state \bm{x}_{q}=\bm{x}(t) to identify the most relevant expert context. We define the projected state \bm{x}^{\prime} as the element in \mathcal{P} that minimizes the distance to the query state: \bm{x}^{\prime}=\underset{\bm{x}_{k}^{(i)}\in\mathcal{P}}{\arg\min}\|\bm{x}_{q}-\bm{x}_{k}^{(i)}\|_{2}. (3) Each \bm{x}^{\prime} is associated with a specific temporal index t^{\prime} from its source trajectory \Gamma^{(i)} , representing the "expert phase". To evaluate progress relative to the demonstration, we introduce a phase error e(t)=\tau(t)-t^{\prime} , where \tau(t) is a virtual reference clock. Unlike the continuous execution time t , the reference clock \tau(t) adapts to the robot’s performance. The update logic for the reference clock and a stagnation counter c is governed by a threshold \epsilon : • Progressing \rightarrow Tracking: If e(t)\leq\epsilon , the robot is successfully tracking the demonstration. We set c=0 and allow the reference clock to advance ( \dot{\tau}=1 ). • Stagnating \rightarrow Exploration: If e(t)>\epsilon , the robot is lagging behind the expert context. We fix the reference clock (\dot{\tau}=0) and increment the stagnation counter c to signal the exploration. This stagnation signal c provides a heuristic measure of task progress and determines whether to track or explore around demonstrations. II-C Distribution Generation We use anisotropic diffusion to generate a particle-based distribution \left\{\bm{q}_{j}\right\}_{j=1}^{N}\subset\mathcal{X}^{(i)} along the nominal trajectory \Gamma^{(i)} that unifies tracking and exploration within a single stochastic differential equation

Read the complete text