MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation

Xiaofeng Tan*, Wanjiang Weng*, Hongsong Wang, Fang Zhao, Xin Geng, Senior Member, IEEE, Liang Wang, Fellow, IEEE · 2026

Read MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation by Xiaofeng Tan*, Wanjiang Weng*, Hongsong Wang, Fang Zhao, Xin Geng, Senior Member, IEEE, Liang Wang, Fellow, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27185.txt.

Introduction

Tan and co-authors present MotionRFT, a unified reinforcement fine-tuning framework for text-to-motion generation. Supervised training alone misses high-level goals like realism and human preference. Earlier post-training targeted one motion format or one quality aspect at a time. Their MotionReward model maps varied motion formats into one text-anchored semantic space. That enables rewards across several quality dimensions at once. EasyTune then updates step by step instead of over full denoising paths. The framework cut memory sharply while improving FID and text alignment scores.

Catalog record

CREATORXiaofeng Tan*, Wanjiang Weng*, Hongsong Wang, Fang Zhao, Xin Geng, Senior Member, IEEE, Liang Wang, Fellow, IEEE
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74527
LICENSEhttp://creativecommons.org/publicdomain/zero/1.0/
RIGHTSCC0
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27185

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MotionRFT: Unified Reinforcement Fine-Tuning for Text-to-Motion Generation Xiaofeng Tan*, Wanjiang Weng*, Hongsong Wang, Fang Zhao, Xin Geng, Senior Member, IEEE, Liang Wang, Fellow, IEEE Abstract Text-to-motion generation has advanced with diffusion- and flow-based generative models, yet supervised pre-training remains insufficient to align models with high-level objectives such as semantic consistency, realism, and human preference. Existing post-training methods have key limitations: they (1) target a specific motion representation, such as joints, (2) optimize a particular aspect, such as text-motion alignment, and may compromise other factors; and (3) incur substantial computational overhead, data dependence, and coarse-grained optimization. We present a reinforcement fine-tuning framework that comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward, and an efficient, fine-grained fine-tuning method, EasyTune. To obtain a unified semantics representation, MotionReward maps heterogeneous motions into a shared semantic space anchored by text, enabling multi-dimensional reward learning; Self-refinement Preference Learning further enhances semantics without additional annotations. For efficient and effective fine-tuning, we identify the recursive gradient dependence across denoising steps as the key bottleneck, and propose EasyTune, which optimizes step-wise rather than over the full trajectory, yielding dense, fine-grained, and memory-efficient updates. Extensive experiments validate the effectiveness of our framework, achieving FID 0.132 at 22.10 GB peak memory for MLD model and saving up to 15.22 GB over DRaFT. It reduces FID by 22.9% on joint-based ACMDM, and achieves a 12.6% R-Precision gain and 23.3% FID improvement on rotation-based HY Motion. Our project page with code is publicly available. I Introduction Text-to-motion generation aims to synthesize realistic and coherent human motions from natural language, enabling applications in animation [ 1 ] , robotics [ 28 ] , and virtual reality [ 42 ] . The rapid advancement of generative model [ 14 , 21 ] has propelled significant progress in text-to-motion generation. Mainstream generative paradigms, including diffusion models [ 24 , 44 , 4 , 37 ] and autoregressive models [ 17 , 57 , 10 ] , have demonstrated remarkable capabilities in synthesizing realistic human motions from natural language descriptions [ 65 ] . Despite significant advances in generation quality and efficiency, existing models [ 14 , 44 , 4 , 58 ] are still primarily trained under supervised learning, fitting annotated data distributions [ 11 ] . However, maximizing data likelihood does not necessarily optimize high-level objectives such as perceptual quality [ 46 ] or semantic consistency [ 40 , 38 ] , thereby limiting real-world performance. As a result, current text-to-motion models still face considerable challenges in semantic alignment [ 40 ] , human preferences [ 46 ] , and motion realism [ 13 ] . Reinforcement learning (RL) [ 36 ] has been explored to align diffusion-based generation with task-specific goals via reinforcement learning from human feedback (RLHF) [ 19 , 66 ] . Existing approaches include differentiable reward methods [ 5 ] , policy-gradient RL [ 3 , 55 , 22 ] , and direct preference optimization (DPO) [ 45 ] , which treat the denoising trajectory as a Markov Decision Process and optimize reward signals from motion discriminative models [ 32 , 49 , 48 , 47 , 50 ] to correct behaviors not addressed by supervised training [ 52 ] . In motion generation, recent works have explored use RL to enhance the motion quality. Specifically, SoPo [ 40 ] constructs semi-online preference pairs for semantic consistency, MotionCritic [ 46 ] employs PPO for human-preference alignment, and ReinDiffuse [ 13 ] adopts rule-based rewards for motion realism. However, these methods still face several limitations. First , they are typically developed for generative models with a specific motion representation, which makes it difficult to generalize to alternative representations. Beyond the widely used kinematic representation [ 11 ] , recent studies [ 26 , 61 , 8 , 43 , 2 ] have explored generating motions directly in rotation or joint-based representations [ 61 , 43 , 26 ] and have demonstrated improved scalability [ 43 ] ; however, existing methods do not readily transfer to these settings. Second , most prior approaches optimize for a specific aspect, such as text-motion alignment [ 40 ] or human preference [ 46 ] , which may bias the model toward that objective and potentially compromise other dimensions of motion quality. Finally , these methods often come with substantial computational overhead [ 52 ] , coarse-grained optimization, or heavy data dependence, which limits practical adoption in real-world scenarios. Contributions. We propose a reinforcement fine-tuning framework for text-to-motion generation. It comprises a heterogeneous-representation, multi-dimensional reward model, MotionReward , and an efficient and effective fine-tuning method, EasyTune . Our main contributions are summarized as follows. Our first contribution lies in a unified heterogeneous-representation reward model, MotionReward, for multi dimensional evaluation in text conditioned motion generation. MotionReward aims to overcome three challenges: (1) the discrepancy across heterogeneous motion representations that makes reward signals hard to compare and generalize, (2) the limitation of single objective rewards that fail to jointly capture motion realism, motion to text alignment, and human perceived preference, and (3) the scarcity of large scale, high quality preference annotations. To address the representation discrepancy, (1) we establish a unified semantic representation, where the heterogeneous motion representations are mapped into a shared embedding space via lightweight linear projections and aligned with the textual anchor. This unified semantic representation serves as the natural text-motion alignment reward and provides a unified representation for heterogeneous motions. (2) Building upon this unified semantic representation, we adopt a lightweight LoRA parameter to transfer the semantic representation into other reward domain, where the specific reward, such as human preference and motion authenticity, can be captured and learned in an efficient way. (3) To address the lack of large-scale datasets, we propose Self-refinement Preference Learning (SPL) to refine the pre-trained reward model without additional data. SPL dynamically constructs preference pairs by using pre-training samples as positives and failed retrieval results as negatives, and use them to fine-tune the pre-trained model. With this unified semantic modeling, MotionReward can be seamlessly applied to reinforcement fine-tuning of text-to-motion generators across kinematic-, rotation-, and joint-based representations in semantics, preference, and authenticity aspects. Second, we analyze the drawbacks of existing differentiable reward based methods and propose an efficient reinforcement fine tuning method, EasyTune. Specifically, we theoretically, in Corollary 1 , and empirically, in Fig. 6 , identify the key factor of computational and memory overhead: optimization is recursively coupled across the multi step denoising trajectory, meaning that the reward of the final motion \mathcal{R}_{\phi}(\mathbf{x}^{\theta}) depends on all intermediate states and the gradient \nabla_{\theta}\mathbf{x}_{t}^{\theta} must backpropagate through the entire reverse chain, which requires storing the full computational graph. We further show in Fig. 6 that such coarse-grained chain optimization causes vanishing gradients, making early denoising steps difficult to improve. Fortunately, intermediate motions are more perceivable than intermediate images or videos, which makes step aware optimization feasible, as illustrated in Fig. 6 in Sec. IV-A . Building upon these, we propose EasyTune , which performs reinforcement fine tuning at each denoising step to decouple gradients from the full trajectory, enabling dense and fine-grained optimization and expensive memory saving. To accurately estimate the reward of noisy intermediate motions, we design step specific reward perception mechanisms, including single step prediction rewards for ODE based models

Read the complete text