ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment

Wanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, Hongsong Wang · 2025

Read ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment by Wanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, Hongsong Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19217.txt.

Introduction

Wanjiang Weng and five co-authors improve text-to-motion generation, which turns a written prompt into three-dimensional human movement for games, film, and robotics. Diffusion models make varied and realistic motion, but the result often drifts from the prompt's meaning. Their method, ReAlign, scores alignment during generation rather than only at the end.

A step-aware reward model judges each noisy intermediate motion for both semantic fit and realism, using one module for text alignment and one for motion quality. A reward-guided strategy then steers sampling toward better aligned results. On generation and retrieval tests, ReAlign beats leading methods in prompt match and motion quality.

Catalog record

CREATORWanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, Hongsong Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-19217

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment Wanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, Hongsong Wang Abstract Text-to-motion generation, which synthesizes 3D human motions from text inputs, holds immense potential for applications in gaming, film, and robotics. Recently, diffusion-based methods have been shown to generate more diversity and realistic motion. However, there exists a misalignment between text and motion distributions in diffusion models, which leads to semantically inconsistent or low-quality motions. To address this limitation, we propose Re ward-guided sampling Align ment ( ReAlign ), comprising a step-aware reward model to assess alignment quality during the denoising sampling and a reward-guided strategy that directs the diffusion process toward an optimally aligned distribution. This reward model integrates step-aware tokens and combines a text-aligned module for semantic consistency and a motion-aligned module for realism, refining noisy motions at each timestep to balance probability density and alignment. Extensive experiments of both motion generation and retrieval tasks demonstrate that our approach significantly improves text-motion alignment and motion quality compared to existing state-of-the-art methods. Code — https://wengwanjiang.github.io/ReAlign-page Introduction With the rising demand for realistic 3D human motions in gaming, filmmaking, virtual reality, and robotics, along with recent advances in motion modeling ( 14 ; 40 ) , there is an increasing need for intuitive and controllable motion generation techniques. Text-to-motion generation, which aims to synthesize human motion directly from natural language descriptions, has emerged as a key research topic ( 3 ; 4 ; 8 ; 14 ; 41 ; 42 ) . Diffusion has emerged as the mainstream approach for text-driven motion generation ( 36 ; 46 ) . However, diffusion-based models often struggle with text-motion alignment due to their reliance on text embeddings encoded by CLIP ( 30 ) , which is trained on text-image pairs rather than text-motion pairs. Consequently, these models often fail to capture the semantic alignment between text and motion. The motions synthesized by most existing diffusion-based methods lack coherence with the input descriptions (see Figure ). Prior works aiming to improve text-to-motion alignment, such as reinforcement learning with reward functions ( 10 ; 20 ; 35 ) , primarily focus on fine-tuning generative models to enhance motion quality. These approaches lack the capability to handle noisy motion inputs. Moreover, the misalignment issue should be addressed during the denoising process itself, rather than corrected retrospectively after the final motion is generated. Another limitation of existing diffusion-based methods is that, although they can generate high-quality motions, the generated motions may still lack smoothness and realism in some cases. During the reverse diffusion process, motion generation relies solely on the diffusion model without access to real motion references for guidance. To address this issue, we shift our focus to motion retrieval. In fact, motion generation and retrieval are closely related tasks; however, most existing works investigate them separately, with few efforts dedicated to exploring their interconnection or developing a unified model for both tasks. To address these problems, we propose a novel Re ward-guided sampling Align ment strategy ( ReAlign ) to enhance text-motion alignment quality with the guidance of a well-aligned reward distribution. We derive the reward distribution from a step-aware reward comprising two modules: a text-aligned module to ensure semantic consistency, and a motion-aligned module to assess realism. Together, these modules adapt to noisy motions and variations across timesteps, guiding diffusion model toward a distribution that not only maximizes probability density but also maintains strong text-motion alignment. By explicitly addressing both semantic misalignment and motion quality degradation, this approach improves the coherence and realism of the generated motion. The proposed reward model is plug-and-play and can be seamlessly integrated into any motion diffusion model without requiring additional fine-tuning. Our main contributions are as follows: \bullet Theoretical reward-guided denoising analysis: We theoretically demonstrate that the reward gradient, derived from both text-aligned and motion-aligned rewards, progressively influences the denoising process, guiding the sampling trajectory toward a distribution that better reflects the intended motion semantics. \bullet Versatile module for diffusion-based generation: We propose ReAlign, which comprises a step-aware reward model and a reward-guided sampling strategy to improve text-motion alignment. Extensive experiments demonstrate that our approach significantly enhances existing diffusion-based motion generation models. Related Works Alignment in Text-to-Motion Generation. Text-to-motion generation represents a critical task in computer vision, exhibiting rapid advancements in recent years ( 48 ; 49 ; 47 ; 43 ) . Diffusion models have been adopted for text-driven motion generation ( 36 ; 46 ) . MotionLCM ( 4 ) refines motion-latent diffusion to enable precise spatiotemporal control via few-step inference. Alignment represents a versatile technique widely employed across the domains of language modeling ( 31 ) , image generation ( 38 ) , and policy optimization ( 2 ) . Recently, human preference alignment is studied in text-to-motion generation. ReinDiffuse ( 10 ) refines the diffusion model through reinforcement learning to enhance the physical plausibility of generated motions. MotionRL ( 20 ) focus aligning human preferences using the proposed multi-reward reinforcement learning framework. SoPo ( 35 ) combines the strengths of online and offline direct preference optimization to overcome their individual shortcomings, delivering enhanced motion generation quality and preference alignment. However, these methods focus on fine-tuning generative models to align preferences or enhance motion quality without explicitly addressing text-motion misalignment. In contrast, we tackle this issue with a plug-and-play reward model in the inference process. Diffusion-Based Reward-Guided Generation. Guidance in diffusion models can be driven by either derivative-free or gradient-based reward models. Derivative-free methods include Sequential Monte Carlo (SMC)-based guidance ( 6 ; 1 ) and Value-Based Importance Sampling ( 16 ) , where the former samples across the entire batch, while the latter performs sampling independently for each sample without global interaction. The typical gradient-based reward is classifier guidance ( 5 ) . Inference-time guidance is specifically designed for discrete diffusion models ( 37 ) . Exact sampling from the optimal policy is feasible within discrete diffusion models under certain limited scenarios. A practical implementation of classifier guidance for discrete diffusion models proposed ( 25 ) . Search-based algorithms ( 39 ) are also proposed to enhance inference alignment. For image synthesis, Liu et al. ( 21 ) introduce a unified framework for multi-modal guidance with both language and image input. While reward-guided generation has received little attention in motion synthesis, this work addresses that gap. Methods Motivation and Overview Framework Figure 1: Illustration of the sampling process in diffusion-based motion generation frameworks. The blue region represents the sampling distribution p_{t}(\cdot) learned by the diffusion model, while the green region depicts the ideal sampling distribution p_{t}^{I}(\cdot) achieved by incorporating our proposed reward-guided sampling strategy with the sampling distribution p_{t}(\cdot) . Preliminaries. Existing diffusion-based motion generation methods ( 3 ; 36 ) operate via a forward process and a reverse process. The forward process gradually adds noise into the real motion distribution p_{\text{data}}(\cdot) over timestep, and can be modeled as a stochastic differential equation (SDE) ( 34 ) : \mathrm{d}\mathbf{x}=\mathbf{f}(\mathbf{x},t)\mathrm{d}t+g(t)\mathrm{d}\mathbf{w}, (1) where t is timestep, \mathbf{f}(\cdot,\cdot) and g(\cdot) are the drift and diffusion coefficients, and \mathbf{w} is the Wiener process. For reverse process, motions \mathbf{x} are generated via trajectory sampling ( 34 ) : \mathrm{d}\mathbf{x}=[\mathbf{f}(\mathbf{x},t)-g(t)^{2}\nabla\log p_{t}(\mathbf{x})]\mathrm{d}t+g(t)\mathrm{d}\mathbf{w}, (2) where \nabla\log p_{t}(\mathbf{x}) is the score function of p_{t}(\mathbf{x})

Read the complete text