Theoretical Closed-loop Stability Bounds for Dynamical System Coupled with Diffusion Policies

Gabriel Lauzier, Alexandre Girard, François Ferland · 2025

Read Theoretical Closed-loop Stability Bounds for Dynamical System Coupled with Diffusion Policies by Gabriel Lauzier, Alexandre Girard, François Ferland (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15520.txt.

Introduction

Gabriel Lauzier, Alexandre Girard, and François Ferland study stability when a robot uses a diffusion policy, a controller learned from demonstrations that can capture several plausible actions. Such policies work well for manipulation under random disturbances. Their weakness is speed: producing an action needs a step-by-step denoising process, which is costly for real-time control.

The authors ask what happens if denoising runs only partly while the robot starts moving, so plant motion and computer denoising evolve together and affect each other. They derive bounds that say when the combined closed loop stays stable. The result also gives a measure, based on how varied the demonstrations are, for predicting whether a given controller will remain stable.

Catalog record

CREATORGabriel Lauzier, Alexandre Girard, François Ferland
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-15520

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Theoretical Closed-loop Stability Bounds for Dynamical System Coupled with Diffusion Policies Gabriel Lauzier, Alexandre Girard, François Ferland Abstract Diffusion policy has shown great performance in robotic manipulation tasks under stochastic perturbations, due to its ability to model multimodal action distributions. Nonetheless, its reliance on a computationally expensive reverse-time diffusion (denoising) process, for action inference, makes it challenging to use for real-time applications where quick decision-making is mandatory. This work studies the possibility of conducting the denoising process only partially before executing an action, allowing the plant to evolve according to its dynamics in parallel to the reverse-time diffusion dynamics ongoing on the computer. In a classical diffusion policy setting, the plant dynamics are usually slow and the two dynamical processes are uncoupled. Here, we investigate theoretical bounds on the stability of closed-loop systems using diffusion policies when the plant dynamics and the denoising dynamics are coupled. The contribution of this work gives a framework for faster imitation learning and a metric that yields if a controller will be stable based on the variance of the demonstrations. I INTRODUCTION Robotic control and decision making increasingly rely on generative models to capture complex and multimodal action distributions. Vision-Language-Action (VLA) models [ 1 , 2 ] have demonstrated great performance in manipulation tasks and the interest in this field is growing with the uprising of large language models. The benefits of such models are the usage of pretrained models on large, Internet-scale datasets that allow the use of generic architecture [ 1 ] . On the other hand, such datasets and training infrastructure are often hard to obtain, resulting in closed-source models hard to fine-tune for new platforms. Other works leveraging diffusion policies [ 3 , 4 ] have shown efficient learning that requires fewer expert demonstrations, allowing someone to easily build a dataset for a given task. Despite its efficient learning, the hyperparameters of the model are often convoluted [ 5 ] and given the iterative nature of diffusion models, it requires several steps of inference, making them challenging to use for real-time applications. In this work, we propose a partial diffusion policy that executes only one step of the denoising diffusion process before taking action in the environment. Rather than expressing the system made of the plant dynamics and the denoising diffusion process as two decoupled systems, we analyze its stability as a coupled dynamical system. This allows us to show that the full denoising diffusion process from Diffusion Policy and the proposed partial diffusion policy are equivalent in stability depending on the variance of the expert demonstrations, the diffusion coefficient and the time-scale ratio between the plant dynamics and the controller. The main contribution is a theoretical bound for asymptotic stability for a linear time-invariant plant under full and partial diffusion policies. This gives a metric for estimating the quality of a dataset given the variance of the expert demonstrations, the responses of the environment and the parametrization of the diffusion-based policy. It also shows a framework for trading off denoising depth against latency given proper variance in the expert demonstrations. This understanding of the diffusion-based policy brings such a framework closer to real-time robotics. II RELATED WORKS II-A Score-Based Generative Modeling Song et al. [ 6 ] introduced a mathematical framework for denoising diffusion models that leverage stochastic differential equations (SDEs). Unlike likelihood-based methods, the suggested approach doesn’t need to learn the partition function of the Boltzmann/Gibbs distribution by approximating the score function of the distribution with a neural network and a score matching objective [ 7 , 8 , 9 ] . The process of learning and inference is defined respectively by two SDEs: the forward equation and the reverse-time equation: d\mathbf{x}=\mathbf{f}(\mathbf{x},\tau)d\tau+\mathbf{g}(\mathbf{x},\tau)d\mathbf{w} (1) d\mathbf{x}=\left[\mathbf{f}(\mathbf{x},\tau)-\mathbf{g}^{2}(\tau)s_{\theta}(\mathbf{x},\tau)\right]d\tau+\mathbf{g}(\tau)d\hat{\mathbf{w}} (2) where \mathbf{w} is a standard Brownian motion, \mathbf{f} is the drift coefficient, \mathbf{g} is the diffusion coefficient and \mathbf{s_{\theta}} is the approximation of the score function, the gradient of the log-probability. The reverse-time SDE in equation 2 is then used to generate samples from the learned prior distribution. The discretization of variance-preserving SDE has been shown to yield Denoising Diffusion Probabilistic Models (DDPM) [ 10 , 11 , 6 ] . II-B Diffusion Policy Diffusion Policy [ 3 ] applies a conditional DDPM model to learn a policy observed from expert demonstrations. This is done by learning the score function of the conditional distribution over actions. Then, at inference time, it iteratively optimizes the score function through a series of stochastic Langevin dynamics steps to generate a sequence of actions within a given horizon. In this work, the model is conditioned on visual observations. By choosing to predict a sequence of actions instead of a single action, Diffusion Policy reduces jitter in the policy due to different valid modes between steps. It also allows handling idle actions where the agent shouldn’t move, which is often difficult for single-step policies. Another main advantage against other imitation learning methods is its capacity to model multimodal action distributions because of the stochastic nature of DDPM. This results directly in better performance with position control than with velocity control. For a simple task in a linear dynamic system in standard state-space form where the demonstrations are given by a linear feedback policy: \mathbf{u}_{t}=-\mathbf{K}\mathbf{x}_{t} , the DDPM sampling will converge to the expert policy given a sequence of actions of length one. With a bigger horizon, this policy will converge to \mathbf{u}_{t+t^{\prime}}=-\mathbf{K}(\mathbf{A}-\mathbf{B}\mathbf{K})^{t^{\prime}}\mathbf{x}_{t} , which means that the policy must implicitly learn the dynamics of the system \mathbf{A}-\mathbf{B}\mathbf{K} . III METHOD First, we define a linear time-invariant dynamic system in a standard state-space form: \mathbf{\dot{x}}=\mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u} (3) Like Diffusion Policy, we aim to obtain demonstrations from a linear feedback policy: \mathbf{u}_{t}=-\mathbf{K}\mathbf{x}_{t} . Then we want a surrogate policy that imitates the expert behavior. Instead of using DDPM like Diffusion Policy, we use the score-based generative modeling framework [ 6 ] to model the denoising process. Equation 2 is used for this purpose. Within this equation, the score function in the drift coefficient could be seen as a vector field that guides toward the learned prior probability density function. This last idea is what motivates this work. Instead of doing the full denoising process to sample an action, we directly take an action towards the direction of the most probable action. In a closed-loop system, this yields: \begin{bmatrix}d\mathbf{x}=\left[\mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u}\right]dt\\ d\mathbf{u}=\left[\mathbf{f}(\mathbf{u},\tau)+\mathbf{g}^{2}(\tau)\mathbf{s}_{\theta}(\mathbf{u},\tau)\right]d\tau+\mathbf{g}(\tau)d\mathbf{\hat{w}_{\tau}}\end{bmatrix} (4) Where \tau if the diffusion time and the score function \mathbf{s}_{\theta}(\mathbf{u},\tau) is defined as the gradient of the log-probability of the expert policy with respect to the action \mathbf{u}_{t} : \mathbf{s}(\mathbf{u})=\nabla_{\mathbf{u}}\log p(\mathbf{u}) (5) If the expert demonstration is deterministic, as we defined earlier, the score function will be ill-defined and could be interpreted as a Dirac delta function \delta(\mathbf{u}-\mathbf{u}_{0}\exp(\mathbf{A}-\mathbf{B}\mathbf{K})t) which is a distribution that is zero everywhere except at the point \mathbf{u}=\mathbf{u}_{0}\exp((\mathbf{A}-\mathbf{B}\mathbf{K})t) . This means that anywhere other than the expert policy trajectory, there is no direction to go towards. This is a problem if we want to use the score function to guide the system towards the most probable action. To this matter, we model the expert policy as a stochastic policy using a Gaussian distribution centered on the deterministic policy: p(u)=\frac{1}{\sqrt{2\pi\sigma^{2}}}\exp\left(-\frac{(u+Kx)^{2}}{2\sigma^{2}}\right) (6) With this probability density function, we can compute the score function as: \mathbf{s}(\mathbf{u})=-\frac{1}{\sigma^{2}}\left(\mathbf{u}+\mathbf{K}\mathbf{x}\right) (7) \bm{\pm} \dot{\mathbf{u}}=-\frac{1}{\sigma^{2}}\left(\mathbf{u}+\mathbf{K}\mathbf{x}\right) \dot{\mathbf{x}}=\mathbf{A}\mathbf{x}+\mathbf{B}\mathbf{u}

Read the complete text