Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation
Deli Cai, Haoyang Ma, Changxing Ding, Senior Member, IEEE · 2026
Read Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation by Deli Cai, Haoyang Ma, Changxing Ding, Senior Member, IEEE (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13729.txt.
Introduction
Deli Cai, Haoyang Ma and Changxing Ding propose CMC, a split frame mixing text and trajectory rules for path-controlled human motion making. Old methods suffer when text and path rules conflict during cleaning, harming motion quality or trajectory tracking. CMC uses a divide-and-conquer design with two linked stages: Path Control and Motion Finish. Path Control uses a spread model to make a simple guided-joint sketch under path lead. Tests on HumanML3D and KIT show state-of-the-art control hits and motion quality.
Catalog record
| CREATOR | Deli Cai, Haoyang Ma, Changxing Ding, Senior Member, IEEE |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63580 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13729
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation Deli Cai, Haoyang Ma, Changxing Ding, Senior Member, IEEE Abstract Trajectory-controlled human motion generation aims to synthesize realistic human motions conditioned on both textual descriptions and spatial trajectories. However, existing methods suffer from two critical limitations: first, the conflict between text and trajectory conditions disrupts the denoising process, resulting in compromised motion quality or inaccurate trajectory following; second, the use of redundant motion representations introduces inconsistencies between motion components, leading to instability during trajectory control. To address these challenges, we propose CMC, a decoupled framework that effectively coordinates text and trajectory conditions through a divide-and-conquer strategy. CMC follows a divide-and-conquer paradigm, comprising two cascaded stages: Trajectory Control and Motion Completion. In the first stage, a diffusion model generates a simplified representation of the controlled joints under trajectory guidance, based on the given trajectories, ensuring accurate and stable trajectory following. In the second stage, a text-conditioned diffusion inpainting model generates full-body motions using the simplified representation from the first stage as partial observations. To mitigate overfitting caused by limited inpainting training data, we further introduce the Selective Inpainting Mechanism (SIM), which alternates between text-to-motion generation and motion inpainting tasks during training. Experiments on HumanML3D and KIT datasets demonstrate that CMC achieves state-of-the-art performance in control accuracy and motion quality, demonstrating its effectiveness in coordinating multimodal conditions and representations. Project page: https://cdlchoi.github.io/cmc_page I Introduction Trajectory-controlled human motion generation [ 44 , 59 , 57 , 2 ] aims to synthesize realistic human motions conditioned on both textual descriptions and spatial trajectories. This technology has significant potential applications in fields such as animation [ 18 ] , gaming [ 17 ] , and robotics [ 30 ] . However, it still presents a challenge due to the inherent disparity in information granularity between text and trajectory data. Specifically, language is typically abstract and conveys coarse-grained information, whereas trajectory data is significantly more detailed and fine-grained. To improve controllability in motion generation, recent diffusion-based methods [ 44 , 39 , 22 , 51 ] have proposed pre-assigning the spatial trajectories of specific human joints during motion generation, which can be divided into inpainting-based methods and guidance-based methods. Inpainting-based methods [ 44 , 39 ] use a diffusion model to complete a full-body motion representation using the local positions of controlled joints. However, the practicality is relatively poor because the local positions are typically unknown [ 51 ] . To address this issue, guidance-based methods [ 22 , 51 ] integrate the trajectory conditions into each denoising step in a classifier-guidance manner [ 7 ] , where trajectory guidance and text-based denoising are applied alternatively. Fig. 1: Comparison of frameworks between our approach and two mainstream frameworks. (a) GMD [ 22 ] applies both text and trajectory conditions at both stages, particularly during the motion generation stage. (b) Omnicontrol [ 51 ] generates motion in a single integrated stage, utilizing both text and trajectory conditions simultaneously. (c) In contrast, our CMC decouples the trajectory control and motion generation stages, thereby avoiding conflicts between the text and trajectory conditions. However, we identify several fundamental limitations in prior works [ 22 , 51 ] that follow the paradigm of simultaneous trajectory control and motion generation (Fig. 1 ). The primary issue lies in the conflict between text and trajectory conditions and the inconsistencies introduced by redundant motion representations. We argue that injecting the trajectory guidance into the diffusion process disrupts the original denoising process. This interference may hinder the network from following the learned text-to-motion mapping during training, as illustrated in Fig. 3 . As a result, the generated motion can become semantically inconsistent with the input text. Moreover, using the redundant representation (including local joint positions, rotations, and velocities) [ 22 , 51 ] for trajectory-controlled motion generation may undermine control accuracy and lead to instability during trajectory control, as illustrated in Fig. 4 . However, trajectory guidance is usually applied only to the local positions, which leads the inconsistency between the components in redundant representation [ 29 ] and leaves the diffusion network to implicitly reconcile inconsistencies among these components during denoising. Motivated by these observations, we explore coordinating the multiple conditions and representations during motion generation Fig. 2: We propose to Coordinate Multiple Conditions (CMC) for trajectory-controlled human motion generation. We visualize examples using different conditions: (a) Single-Joint Control: the specified trajectory for one joint covers partial or all frames. (b) Multi-Joint Control: the specified trajectories for multiple joints are available. (c) Keyframe control: joint locations are available only on several keyframes. We visualize the human body while keeping its joints visible to demonstrate the high controllability of our CMC. Fig. 3: (a) With conflict: insufficient understanding of the text condition leads to suboptimal motion quality. (b) Without conflict: the network fully maps the text to motion while achieves accurate trajectory following. Hence, we propose to coordinate multiple conditions (CMC) for trajectory-controlled human motion generation. CMC is a decoupled framework that leverages text and trajectory conditions separately. It follows a divide-and-conquer paradigm, comprising two cascaded stages: Trajectory Control and Motion Completion, based on diffusion models. In the Trajectory Control stage, a diffusion model first generates plausible local joint positions for the pelvis and controlled joints conditioned on the input text, which we denote as a simplified representation . Following the diffusion model, we apply trajectory guidance in a classifier-guidance manner to refine this simplified representation according to the given trajectories. This simplified representation makes the control process more stable and results in better control accuracy. Once optimized, the positions of the controlled joints are fixed and passed to the next stage. Fig. 4: Comparisons of the control error for the redundant and simplified representations across all denoising steps. The Motion Completion stage is also built on a diffusion model and leverages only the text condition to produce the full-body motions [ 13 , 2 , 57 ] . It utilizes the simplified representation from the first stage (i.e., the local positions of the pelvis and the controlled joints) as partial observations and generates full-body motion using a diffusion inpainting model [ 44 ] . Crucially, these trajectory-relative components remain frozen throughout this stage, thereby preserving the refined spatial path. Further, we observe that training the diffusion inpainting model exclusively on the inpainting task leads to overfitting, primarily due to the limited size of the motion inpainting data. To address this issue, we propose a Selective Inpainting Mechanism (SIM) for training, which dynamically alternates between two training modes: (1) standard text-to-motion generation, and (2) motion inpainting. The former task serves as a regularization approach, enhancing the generalization of the inpainting model while maintaining its inpainting capability. Our CMC can perform various tasks, such as single-joint control, multi-joint control, and keyframe control, as illustrated in Fig. 2 . We emphasize that our CMC addresses the conflict between text and trajectory conditions. In the first stage, the use of a simplified representation minimizes the influence of text on trajectory control, enabling stable and accurate alignment with the desired path. This improvement is clearly reflected in the reduced and stable error observed in Fig. 4 . In addition, under the decoupled paradigm, the second stage generates full-body motion solely from