FRMD: Fast Robot Motion Diffusion with Consistency-Distilled Movement Primitives for Smooth Action Generation

Xirui Shi, Jun Jin · 2025

Read FRMD: Fast Robot Motion Diffusion with Consistency-Distilled Movement Primitives for Smooth Action Generation by Xirui Shi, Jun Jin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02048.txt.

Introduction

Diffusion models make fine robot motions but work too slowly for live control, and their paths can jerk. Xirui Shi and Jun Jin speed them up by marrying them to movement primitives, compact codes for whole trajectories. A consistency training trick, borrowed from image models, lets the system produce a motion in one step. An encoder-decoder holds the smooth, time-spanning code. The slow denoising chain disappears.

Tests on two standard robot benchmarks, from easy to hard tasks, beat strong rivals. Motions came out smooth and steady in time, not just accurate at the end. One-step generation cuts the wait that blocks real-time use. The design shows robot learning can borrow speed tricks from image generation.

Catalog record

CREATORXirui Shi, Jun Jin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-02048

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FRMD: Fast Robot Motion Diffusion with Consistency-Distilled Movement Primitives for Smooth Action Generation Xirui Shi, Jun Jin Abstract We consider the problem of using diffusion models to generate fast, smooth, and temporally consistent robot motions. Although diffusion models have demonstrated superior performance in robot learning due to their task scalability and multi-modal flexibility, they suffer from two fundamental limitations: (1) they often produce non-smooth, jerky motions due to their inability to capture temporally consistent movement dynamics, and (2) their iterative sampling process incurs prohibitive latency for many robotic tasks. Inspired by classic robot motion generation methods such as DMPs and ProMPs, which capture temporally and spatially consistent dynamic of trajectories using low-dimensional vectors — and by recent advances in diffusion-based image generation that use consistency models with probability flow ODEs to accelerate the denoising process, we propose Fast Robot Motion Diffusion (FRMD) . FRMD uniquely integrates Movement Primitives (MPs) with Consistency Models to enable efficient, single-step trajectory generation. By leveraging probabilistic flow ODEs and consistency distillation, our method models trajectory distributions while learning a compact, time-continuous motion representation within an encoder-decoder architecture. This unified approach eliminates the slow, multi-step denoising process of conventional diffusion models, enabling efficient one-step inference and smooth robot motion generation. We extensively evaluated our FRMD on the well-recognized Meta-World and ManiSkills Benchmarks, ranging from simple to more complex manipulation tasks, comparing its performance against state-of-the-art baselines. Our results show that FRMD generates significantly faster, smoother trajectories while achieving higher success rates. I Introduction Recently, diffusion models [ 1 ] have gained increasing attention in robot learning due to their scalability [ 2 ] to complex tasks and their flexibility [ 3 ] in incorporating high-dimensional, multi-modal observations. Modern advancements [ 2 , 4 ] in embodied artificial intelligence have shown that an “action expert” [ 4 ] using diffusion models to generate robot actions can handle various manipulation tasks and that its multi-task capabilities scale up with dataset and model size. This highlights the promise of a unified model architecture that employs diffusion models as the action decoder (motion generation) for general-purpose robotic task solvers. However, existing diffusion-based robot motion generation methods still face two fundamental challenges. First, they often produce non-smooth, jerky motions because they fail to capture temporally consistent movement dynamics [ 5 ] . Specifically, this limitation arises from the fact that these methods commonly generate raw action sequences (i.e., waypoints) without accounting for the trajectory-level temporal consistency imposed by the robot’s structured dynamic constraints. Second, the iterative sampling process inherent to these diffusion models’ denoising sampling process (Denoising Diffusion Probabilistic Model (DDPM) [ 6 ] , Denoising Diffusion Implicit Models (DDIM) [ 7 ] ) introduces significant latency when generating robot actions. Inspired by classic dynamic-system–based robot motion generation methods, such as dynamic movement primitives (DMPs [ 8 ] ) and probabilistic movement primitives (ProMPs [ 9 ] ), which capture temporally and spatially consistent dynamic of trajectories using low-dimensional vectors, we propose rethinking diffusion-based robot motion generation by shifting from modeling the conditional action distribution at the raw action (waypoint) level to the trajectory level (movement primitives). This approach essentially learns the prior distribution in the trajectory parameter space. Moreover, instead of simply combing diffusion models with movement primitives (MPs) like previous methods [ 5 , 10 ] , to further accelerate the action denoising process, we incorporate recent advances in diffusion-based image generation [ 11 ] that employ consistency models with Probability Flow ODEs [ 11 ] and consistency distillation [ 12 ] for fast motion generation. These two effective ingredients enable our method to generate fast and smooth robot motions. Specifically, we introduce Fast Robot Motion Diffusion (FRMD), a novel consistency-distilled movement primitives framework that integrates Consistency Models [ 11 ] and Probabilistic Dynamic Movement Primitives (ProDMPs) [ 8 ] to achieve both high inference efficiency and smooth motion generation. Instead of directly generating raw actions via iterative denoising, we employ a movement primitives framework [ 13 , 14 ] to produce smooth motions with guaranteed initial conditions for planning consecutive action sequences. In this setup, the diffusion model predicts ProDMPs weight vectors to ensure structured and smooth trajectories. Moreover, unlike recent methods [ 5 , 10 ] that simply combine diffusion models with movement primitives—which incur inference latency from multi-step denoising and limit their applicability in many robotic tasks—we observe that shifting from predicting raw actions (waypoints) to predicting trajectory parameters (movement primitives) naturally enables the use of consistency models [ 11 ] for accelerated inference. We employ consistency distillation [ 11 , 12 ] to train a model that directly maps noisy actions to movement primitive parameters. This single-step inference approach eliminates multi-step denoising while preserving the expressive power of Movement Primitive Diffusion (MPD) [ 10 ] , our teacher model. As a result, FRMD produces structured motion faster and smoother than traditional diffusion policies, broadening its applicability in many robotic tasks. Our contributions can be summarized as follows: • We propose FRMD, a novel framework that combines Consistency Models with Movement Primitives, enabling fast inference while maintaining structured and smooth motion generation. • We introduce a consistency-distillation strategy to eliminate iterative denoising in diffusion-based motion generation, achieving one-step inference without sacrificing motion quality. We extensively evaluate FRMD across multiple robotic manipulation tasks, demonstrating significant improvements in both inference speed and task success rates over existing diffusion-based methods. • By bridging structured motion representation and fast generative modeling, FRMD paves the way for real-time, high-quality robotic action generation, unlocking new possibilities in robot learning. We evaluate FRMD on a diverse set of robotic manipulation tasks, ranging from simple to complex, using the MetaWorld [ 15 ] and ManiSkill [ 16 ] benchmarks following [ 17 ] . We compare FRMD with state-of-the-art diffusion policies, including the vanilla Diffusion Policy (DP) [ 1 ] and a recent method that combines diffusion models with movement primitives (MPD) [ 10 ] . The results show that FRMD achieves the highest success rate (64.8%) while operating 10× faster than MPD (the teacher model) and 7× faster than DP, enabling the generation of fast and smooth robot motions. Fig. 1: Overview of FRMD Training Framework. Given observations o_{i} , raw action sequence \tau^{0} and initial state (y_{0},\dot{y_{0}}) from the robot datasets, we first perform a forward diffusion to introduce noise over n+k steps. The resulting noisy sequence \tau^{n+k} is then fed into both the student model and the teacher model to predict the action sequence \tau^{0} and \tau^{n} . The target model uses the teacher network’s k -step estimation results to predict the action sequence. The student model, trained via consistency distillation and its weights are updated through an Exponential Moving Average (EMA). II Related Work Our work draws inspiration from a diverse range of topics, including recent advances in diffusion models for robot learning, consistency models, and classic dynamic-system–based methods for robot motion generation. Diffusion Models for Robot Learning: Recent research in robot learning has increasingly adopted diffusion models as the action decoder to generate smooth and diverse trajectories from noisy inputs. Diffusion models, which were first introduced in computer vision for denoising tasks [ 6 ] , have been adapted for robotics by leveraging U-Net based architectures to decode actions

Read the complete text