From Diffusion to Flow: Efficient Motion Generation in MotionGPT3

Jaymin Ban, JiHong Jeon, SangYeop Jeong · 2026

Read From Diffusion to Flow: Efficient Motion Generation in MotionGPT3 by Jaymin Ban, JiHong Jeon, SangYeop Jeong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-26747.txt.

Introduction

Bhan, Jeon, and Jeong compare diffusion and rectified flow objectives inside the MotionGPT3 motion generation system. The study holds the model, training plan, and test setup fixed, so only the generative objective changes. On HumanML3D data, rectified flow converges in fewer epochs and reaches strong quality sooner. It matches or beats diffusion quality under the same conditions. Flow priors also stay stable across many sampling step counts, giving good results with fewer steps at run time.

Catalog record

CREATORJaymin Ban, JiHong Jeon, SangYeop Jeong
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29763
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.26747

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Diffusion to Flow: Efficient Motion Generation in MotionGPT3 Jaymin Ban, JiHong Jeon, SangYeop Jeong Abstract Recent text-driven motion generation methods span both discrete token-based approaches and continuous-latent formulations. MotionGPT3 exemplifies the latter paradigm, combining a learned continuous motion latent space with a diffusion-based prior for text-conditioned synthesis. While rectified flow objectives have recently demonstrated favorable convergence and inference-time properties relative to diffusion in image and audio generation, it remains unclear whether these advantages transfer cleanly to the motion generation setting. In this work, we conduct a controlled empirical study comparing diffusion and rectified flow objectives within the MotionGPT3 framework. By holding the model architecture, training protocol, and evaluation setup fixed, we isolate the effect of the generative objective on training dynamics, final performance, and inference efficiency. Experiments on the HumanML3D dataset show that rectified flow converges in fewer training epochs, reaches strong test performance earlier, and matches or exceeds diffusion-based motion quality under identical conditions. Moreover, flow-based priors exhibit stable behavior across a wide range of inference step counts and achieve competitive quality with fewer sampling steps, yielding improved efficiency–quality trade-offs. Overall, our results suggest that several known benefits of rectified flow objectives do extend to continuous-latent text-to-motion generation, highlighting the importance of the training objective choice in motion priors. 1 Introduction Text-driven motion generation has seen significant progress through two parallel paradigms. One prominent approach relies on discrete latent representations, where motion sequences are quantized into symbolic tokens using Vector-Quantized Variational Autoencoders (VQ-VAEs) ( 8 ; 3 ; 15 ) . By treating motion as a foreign language, these methods have successfully leveraged the powerful autoregressive capabilities of large language models (LLMs). However, discretization introduces inherent limitations, including quantization artifacts and a reconstruction gap, which can suppress high-frequency details and fluid motion characteristics important for natural human movement, as noted in MotionGPT3 ( 18 ) . As an alternative to discrete tokenization, MotionGPT3 adopts a continuous-latent formulation by pairing a pretrained GPT-2–based text encoder with a continuous motion latent space and a text-conditioned diffusion prior for generation (see Figure 1 ). This design avoids explicit motion tokenization while retaining the semantic expressiveness of pretrained language representations, offering a compelling pathway toward continuous text-to-motion synthesis. While diffusion-based priors have demonstrated strong performance in this setting, recent work in image and audio generation has shown that rectified flow objectives can exhibit favorable optimization and inference-time properties relative to diffusion ( 2 ; 12 ) . These include earlier convergence during training and improved efficiency–quality trade-offs at inference. Whether such objective-level advantages transfer cleanly to continuous-latent motion generation, however, remains an open question. MotionGPT3 provides a natural testbed for investigating this question, as it isolates the motion prior within an otherwise fixed continuous-latent architecture. In this work, we conduct a controlled empirical study comparing diffusion and rectified flow objectives within the MotionGPT3 framework. We replace the diffusion objective with a rectified flow objective ( 10 ) while holding the model architecture, dataset, and evaluation protocol fixed, allowing us to isolate the effect of the generative objective on training dynamics, final performance, and inference-time behavior. Unlike diffusion, which learns to reverse a stochastic noising process, rectified flow models a direct ODE trajectory between noise and data, providing a conceptually simpler optimization path. Figure 1: Overview of the MotionGPT-3 architecture with alternative motion priors. The dual-stream GPT-2 backbone produces text-conditioned motion latents, which are fed into either a diffusion-based or flow-based motion prior. The and symbols indicate frozen and trainable modules, respectively. Our experiments on the HumanML3D dataset suggest that several properties commonly associated with rectified flow in other generative domains also emerge in the motion generation setting: • Earlier Convergence: Rectified flow reaches strong test performance in fewer training epochs under matched training conditions. • Comparable or Improved Final Performance: Flow-based priors match or exceed diffusion-based motion quality in terms of R-Precision and FID. • Inference-Time Robustness: Generation quality remains stable across a wide range of inference step counts, with reduced sensitivity to step selection. • Improved Efficiency–Quality Trade-offs: Competitive generation quality is achieved with fewer sampling steps, enabling lower inference latency. Taken together, these findings highlight the importance of the generative training objective in continuous-latent text-to-motion models. Within the MotionGPT3 framework, our results indicate that rectified flow constitutes a practical alternative to diffusion-based priors, particularly in settings where training efficiency and inference-time latency are key considerations. 2 Related Work Text-to-Motion Dataset and Evaluation Protocol HumanML3D ( 4 ) is the current standard dataset for text-driven motion generation, providing a large-scale collection of paired motion–language data. Along with the dataset, the authors introduced a standardized evaluation framework utilizing a learned joint text–motion embedding space. This framework enables the calculation of retrieval-based metrics—such as R-Precision (R@K) and Matching Score—and has become the primary evaluation for assessing the performance and cross-modal alignment of motion generation models. Motion Generation Paradigms. Discrete motion generation methods rely on vector-quantized representations combined with autoregressive sequence models ( 16 ; 3 ; 8 ) . These approaches benefit from language-model-style generation but may be constrained by quantization artifacts. In contrast, continuous-latent approaches model motion directly in a continuous space, avoiding explicit tokenization and enabling finer-grained temporal representations ( 14 ; 1 ; 17 ) . Diffusion and Flow-Based Objectives. Diffusion models have demonstrated strong performance for conditional generative modeling across multiple domains, including motion generation ( 6 ; 14 ; 17 ) . However, diffusion-based sampling relies on iterative stochastic denoising, which can incur significant inference-time cost. Rectified flow matching ( 10 ) has been proposed as an alternative objective that learns a deterministic vector field transporting noise to data, and has been shown in image generation to enable efficient sampling with fewer inference steps ( 2 ) . Although flow-based objectives have been explored in motion generation, prior work typically introduces simultaneous architectural or training changes, making it difficult to isolate the effect of the generative objective itself ( 7 ) . 3 Method To evaluate the effect of the generative training objective in isolation, we adopt the MotionGPT3 architecture ( 18 ) as our experimental baseline. All architectural components, training hyperparameters, and data processing steps are kept identical to the original MotionGPT3 setup. The only modification is the choice of motion prior objective: we replace the diffusion-based prior with a rectified flow formulation, both applied to the same text-conditioned motion latents. This design enables a controlled comparison between stochastic denoising and deterministic probability flow objectives with respect to training dynamics, motion quality, and inference efficiency. 3.1 Architectural Overview As illustrated in Figure 1 , MotionGPT3 treats human motion as a trajectory within a continuous latent space and consists of three primary components: 1. Motion Latent Space: A variational autoencoder (VAE) that compresses raw motion sequences into a continuous latent representation. 2. Text Conditioning: A dual-stream GPT-2 backbone that encodes natural language prompts into a rich conditioning embedding, c . 3. Motion Prior Network: The motion prior of the system, mapping Gaussian noise \epsilon\sim\mathcal{N}(0,\mathbf{I}) to motion latents z conditioned on the text embedding c . Generated latents are decoded by the VAE to produce motion feature representations, which are subsequently converted to 3D joint coordinates through a deterministic kinematic post-processing step. 3.2 Training Protocol All experiments are conducted

Read the complete text