Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion
Haim Sawdayee, Chuan Guo, Guy Tevet, Bing Zhou, Jian Wang, Amit H. Bermano · 2025
Read Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion by Haim Sawdayee, Chuan Guo, Guy Tevet, Bing Zhou, Jian Wang, Amit H. Bermano (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19557.txt.
Introduction
Sawdayee and five co-authors introduced LoRA-MDM in twenty twenty-five, a light way to stylize computer-generated human motion. Text-to-motion models create three-dimensional actions from words but miss fine styles, such as moving like a chicken. Style examples are scarce, and pushing each output toward a reference often degrades quality.
Their key idea is to adapt the model's prior, its built-in sense of plausible motion, rather than each motion separately. A low-rank adaptation, trained on just a few style samples, shifts that prior in a meaningful direction. The styled prior then serves many text prompts, even for actions absent from the samples. Keeping the distribution's shape also allows style blending and motion editing. Tests showed a good balance of text fidelity and style consistency.
Catalog record
| CREATOR | Haim Sawdayee, Chuan Guo, Guy Tevet, Bing Zhou, Jian Wang, Amit H. Bermano |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19557
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion Haim Sawdayee, Chuan Guo, Guy Tevet, Bing Zhou, Jian Wang, Amit H. Bermano Abstract Text-to-motion generative models span a wide range of 3D human actions but struggle with nuanced stylistic attributes such as a ”Chicken” style. Due to the scarcity of style-specific data, existing approaches pull the generative prior towards a reference style, which often results in out-of-distribution low quality generations. In this work, we introduce LoRA-MDM , a lightweight framework for motion stylization that generalizes to complex actions while maintaining editability. Our key insight is that adapting the generative prior to include the style, while preserving its overall distribution, is more effective than modifying each individual motion during generation. Building on this idea, LoRA-MDM learns to adapt the prior to include the reference style using only a few samples. The style can then be used in the context of different textual prompts for generation. The low-rank adaptation shifts the motion manifold in a semantically meaningful way, enabling realistic style infusion even for actions not present in the reference samples. Moreover, preserving the distribution structure enables advanced operations such as style blending and motion editing. We compare LoRA-MDM to state-of-the-art stylized motion generation methods and demonstrate a favorable balance between text fidelity and style consistency. Project page at https://haimsaw.github.io/LoRA-MDM/ Figure 1 : LoRA-MDM generates stylized motions using Low-Rank Adaptation of the Motion Diffusion Model (MDM). Left: motions generated using the base model. Middle: stylized generation using LoRA-MDM . Left: Style mixing. 1 Introduction Recent advancements in human motion generation conditioned on text have demonstrated impressive results, producing diverse and high-fidelity motions closely aligned with textual descriptions [ 61 , 8 , 70 ] . However, these models struggle to convey nuanced attributes, including profiles such as that of an elderly person, and fail to interpret abstract descriptions, such as ”a Superman style”. This limitation stems from the nature of existing text-to-motion datasets [ 47 , 45 , 18 ] , which lack rich representations of styles and subtle nuances. In contrast, style-specific datasets [ 1 , 40 ] provide better coverage of these nuances, but they are limited to a small and simple set of motions per style, falling short of capturing the wide variety of human actions. Using the latter datasets, a handful of approaches tackle motion stylization, or infusing a given style into the generation process of another motion. Specifically the two leading approaches either inject attention features from a given style motion into the generation process [ 49 ] , or combine a ControlNet [ 71 ] with a classifier trained over the styles to perform guidance during the generation process [ 73 ] . In both cases, the generative process, based on a diffusion prior, is pushed out of its distribution to match the desired style, yielding lower overall quality, generalization, and editability. In this paper we introduce LoRA-MDM , a simple and efficient stylized motion generation method, yielding quality and editability for diverse stylized actions. Drawing inspiration from image editing and, specifically, personalization literature [ 13 , 50 ] , our main insight is that augmenting the generation process with a given style means venturing into regions unfamiliar to the generative prior. Instead, it is better to shift the generative prior itself to include the style, while preserving its overall distribution structure. Equipped with this insight, a simple solution, of employing Low-Rank Adaptation (LoRA) [ 26 ] , becomes clear; LoRA augments the weights of the prior in a low-ranked manner, which can be intuitively thought of as low frequency deformations to the target distribution manifold. Using LoRA, LoRA-MDM learns to embed the style within the output domain of the model, which can then be used to synthesize stylized motions in the context of different textual prompts. As we demonstrate, this offers two advantages: First, low-ranked augmentation diminishes forgetting, which in turn increases generation quality for stylized actions not seen during training. For example, directly fine-tuning a generated walking motion to walk like a chicken has little (or negative) influence on how the prior will generate kicks. Instead, our low-ranked tuning and structure preservation loss encourage the model to also kick like a chicken, since the changes affect the entire manifold. Second, because the generated stylized motions are in-distribution for the model, advanced operations such as stylized motion editing and style blending are supported, in sharp contrast to traditional motion style-transfer method. Using a pretrained state-of-the-art motion diffusion generative model and a LoRA trained on one or more styles, we demonstrate LoRA-MDM produces superior results compared to leading motion stylization approaches, qualitatively, quantitatively, and through a user study, both in terms of motion quality, and adherence to style or styles. Since the prior’s capabilities are not hampered, we also demonstrate that LoRA-MDM naturally combines with existing motion editing techniques. Specifically, we demonstrate trajectory control capabilities for generated stylized actions. In the future, we hope this personalization take on motion style can help set common practices for applying LoRA to other motion domains, and will inspire extending motion generation capabilities to other underrepresented axes. 2 Related Work 2.1 Human Motion Synthesis Multimodal generation. In recent year, human motion generation has experienced significant progress with the advent of Transformers [ 66 ] and deep generative models [ 24 , 15 , 32 ] . Motions can be now generated from a variety of input modalities, including action categories [ 42 , 17 , 68 , 38 ] , music signals [ 2 , 55 , 34 , 64 ] , and textual descriptions [ 61 , 18 , 59 , 72 , 20 , 19 , 53 , 6 ] . Early works achieved plausible 3D motion generation by leveraging variational autoencoders (VAEs) equipped with temporal models such as RNNs [ 18 , 17 , 34 ] and Transformers [ 42 , 43 ] . However, these models often struggled to learn complex actions and handle fine-grained conditionality. To address this, a subsequent line of research modeled human motion generation in a discrete space. These approaches utilized VQ-VAE [ 65 ] to map human motions into discrete motion tokens, which were then generated from various modalities using language models like GPTs [ 70 , 19 , 55 , 38 ] or generative masked Transformers [ 20 , 44 ] . Diffusion models have also emerged as a powerful tool for motion synthesis due to their exceptional expressiveness. MDM [ 61 ] and MotionDiffuse [ 72 ] applied diffusion models directly in the raw motion feature space, while techniques like latent diffusion [ 6 ] and latent consistency models [ 9 ] have been adopted to enhance the efficiency of diffusion-based human motion synthesis. Motion style transfer. Motion style transfer aims to reenact an existing motion while incorporating the style characteristics specified by a reference style motion or label. Traditional methods, such as motion matching [ 67 ] and gram matrix optimization [ 25 , 11 ] , are limited in scalability and efficiency. Recent advances in data-driven deep learning models have enabled more efficient and faithful motion style transfer. Abermann et al. [ 1 ] introduced the use of AdaIN to disentangle style and content features, which