PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generation
Hongsong Wang, Yin Zhu, Qiuxia Lai, Yang Zhang, Guo-Sen Xie, Xin Geng · 2025
Read PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generation by Hongsong Wang, Yin Zhu, Qiuxia Lai, Yang Zhang, Guo-Sen Xie, Xin Geng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20056.txt.
Introduction
Hongsong Wang and co-authors generate long, physically plausible dances from music with a diffusion model in a 2025 paper called PAMD. Dance synthesis serves art, virtual reality, and games, yet past diffusion methods often produced impossible poses and sliding feet.
A plausible motion constraint uses learned distance fields of real poses to steer generation toward the valid pose manifold. Prior motion guidance adds standing poses beside the music features as extra conditioning. A refinement module then fixes foot skating by bridging joint-position goals and rotation-based data. Tests show clear gains in both musical fit and physical realism over prior methods. Longer sequences stay coherent instead of drifting into broken poses.
Catalog record
| CREATOR | Hongsong Wang, Yin Zhu, Qiuxia Lai, Yang Zhang, Guo-Sen Xie, Xin Geng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20056
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PAMD: Plausibility-Aware Motion Diffusion Model for Long Dance Generation Hongsong Wang, Yin Zhu, Qiuxia Lai, Yang Zhang, Guo-Sen Xie, Xin Geng Abstract Computational dance generation is crucial in many areas, such as art, human-computer interaction, virtual reality, and digital entertainment, particularly for generating coherent and expressive long dance sequences. Diffusion-based music-to-dance generation has made significant progress, yet existing methods still struggle to produce physically plausible motions. To address this, we propose Plausibility-Aware Motion Diffusion (PAMD), a framework for generating dances that are both musically aligned and physically realistic. The core of PAMD lies in the Plausible Motion Constraint (PMC), which leverages Neural Distance Fields (NDFs) to model the actual pose manifold and guide generated motions toward a physically valid pose manifold. To provide more effective guidance during generation, we incorporate Prior Motion Guidance (PMG), which uses standing poses as auxiliary conditions alongside music features. To further enhance realism for complex movements, we introduce the Motion Refinement with Foot-ground Contact (MRFC) module, which addresses foot-skating artifacts by bridging the gap between the optimization objective in linear joint position space and the data representation in nonlinear rotation space. Extensive experiments show that PAMD significantly improves musical alignment and enhances the physical plausibility of generated motions. This project page is available at: https://mucunzhuzhu.github.io/PAMD-page/ . I Introduction Dance is an art form that harmonizes rhythmic body movements with musical accompaniment. It allows for the expression of emotions [ 1 ] , the preservation of cultural heritage [ 2 ] , and the promotion of social connections [ 3 ] . However, traditional choreography is complex and labor-intensive, often requiring extensive professional expertise. Recent advances in computational methods provide an innovative alternative, pushing beyond the limitations of manual approaches by automatically generating dance movements synchronized with music. These methods can not only inspire new forms of artistic expression, but also offer immersive interaction experiences when paired with VR or AR technologies [ 4 ] . Furthermore, computational music-to-dance generation fosters interdisciplinary collaboration among computer science, performing arts, and human-computer interaction [ 5 , 6 ] . A key challenge of computational music-to-dance generation lies in generating realistic and physically plausible dance sequences over long durations. Early autoregressive music-to-dance methods [ 7 , 8 , 9 ] generate dance movements iteratively by predicting each frame based on previously generated frames and corresponding music features. These methods, however, suffer from error accumulation over time, leading to issues like motion freezing [ 4 ] and an inability to generate plausible long sequences. In contrast, diffusion-based models [ 4 ] generate entire dance sequences at once, partially mitigating the problem of error accumulation. Despite this advantage, both autoregressive and diffusion-based methods share a significant limitation: they rely on a latent space to represent motion, yet this latent space is not specifically designed to guarantee physically plausible poses. As a result, both approaches struggle to generate realistic and physically plausible outputs. Fig. 1: Motivation of our approach for diffusion-based music-to-dance generation. To generate plausible and correct motion sequences for music-to-dance, we introduce prior motion and plausible motion constraints during the training of the generative diffusion model. Fig. 2: Our PAMD model generates long dances that are better synchronized and visually coherent. The black dots on the pink music waveform indicate music beats, while the grey and blue motions denote dance beats generated by PAMD (ours) and EDGE, respectively. The underlined dance beats indicate close alignment, which falls within five frames of the nearest music beat. PAMD produces eight closely aligned dance beats compared to only four from EDGE. Moreover, PAMD generates more natural and fluid movements. For example, EDGE lacks hand motion in frame 233, while PAMD maintains consistent and expressive hand movements in frame 205. In general, seed motion is commonly utilized in autoregressive models, where the model predicts subsequent motions based on observed preceding motions [ 7 ] . Although the seed motion renders the predicted motions physically plausible in short-term prediction, prediction errors of these models may accumulate for long-term dance prediction. Besides, many existing methods, such as [ 7 ] and [ 10 ] , rely on auto-regressive inference to produce long dance sequences. This paradigm of long-term generation is inefficient and also leads to error accumulation. To address the limitations outlined earlier, this paper proposes Plausibility-Aware Motion Diffusion (PAMD), a diffusion-based framework for generating dances that are both musically aligned and physically realistic. Figure 1 illustrates the outline of the proposed method. PAMD consists of three key components, namely, Plausible Motion Constraint (PMC), Prior Motion Guidance (PMG), and Motion Refinement with Foot-Ground Contact (MRFC). PMC , inspired by human pose prior modeling [ 11 ] , employs Neural Distance Fields (NDFs) to represent plausible human poses in a continuous, high-dimensional space. This constraint ensures the generation of physically valid motions by guiding outputs toward a realistic pose manifold. To the best of our knowledge, this is the first work to leverage NDFs for signal-conditioned human motion generation tasks, such as music-to-dance synthesis. PMG further enhances dance generation by providing a stable starting point for sequence generation, utilizing standing poses as auxiliary conditions. While conceptually similar to seed motions [ 12 , 13 ] for predicting future movements, PMG does not depend on real motion data. Instead, it seamlessly integrates standing poses commonly seen in transitions, beginnings, or endings of dance sequences with music features, a strategy that has proven highly effective in our experiments. MRFC is inspired by [ 14 ] and designed to improve the realism of complex movements by bridging the gap between the optimization objective in linear joint position space and the data representation in nonlinear rotation space. Unlike [ 14 ] , MRFC is more lightweight, avoiding multiple training stages and reducing the computational costs, while still achieving comparable refinement quality. As shown in Figure 2 , these components enable PAMD to generate high-quality, plausible dance sequences efficiently and effectively. Our contributions are summarized as follows: • We introduce Plausibility-Aware Motion Diffusion (PAMD), a diffusion-based framework for music-to-dance generation. PAMD generates dances that are both musically aligned and physically realistic. • To ensure realistic dance generation, we design Plausible Motion Constraint (PMC), which uses NDFs to model plausible human poses on a continuous manifold, the first application of NDFs in music-to-dance generation. • To provide a stable starting point for sequence generation, we introduce Prior Motion Guidance (PMG), which uses standing poses as auxiliary conditions. II Related Work II-A Music-Driven Dance Generation Music-driven dance generation has become increasingly popular recently. Early approaches [ 15 , 16 , 17 , 18 ] generate dance by viewing this task as a motion retrieval problem. However, these approaches are limited by the quality and diversity of motion databases, lack flexibility in adapting to new music, and often produce mechanically repetitive movements. Subsequently, deep learning approaches have gradually been applied to music-driven dance generation task, including autoregression-based methods [ 7 , 19 , 10 ] , GAN-based methods [ 8 , 20 ] , VAE-based methods [ 9 , 21 ] , and Diffusion-based methods [ 4 , 14 , 22 , 23 , 24 ] . Li et al. [ 7 ] learn the correspondence between music and motion with a cross-modal transformer block.