Motion Diffusion Autoencoders: Enabling Attribute Manipulation in Human Motion Demonstrated on Karate Techniques
Anthony Richardson, Felix Putze · 2025
Read Motion Diffusion Autoencoders: Enabling Attribute Manipulation in Human Motion Demonstrated on Karate Techniques by Anthony Richardson, Felix Putze (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18729.txt.
Introduction
Anthony Richardson and Felix Putze showed in 2025 how to edit single traits of human motion data, using karate moves as their test case. Editing one trait, like speed or height, without touching the rest is a standing challenge for motion data. Their key step is a new way to encode poses as smooth rotations. The encoding splits the skeleton from its path through space while keeping the body shape faithful.
A transformer encoder learns the high-level meaning of each move, and a diffusion model handles the random detail left over. The meaning space turned out orderly: each trait runs along a straight direction in it. Sliding a code along that line changes just that trait. The authors release their code and data, calling it a first for motion editing.
Catalog record
| CREATOR | Anthony Richardson, Felix Putze |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18729
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Motion Diffusion Autoencoders: Enabling Attribute Manipulation in Human Motion Demonstrated on Karate Techniques Anthony Richardson, Felix Putze Abstract . Attribute manipulation deals with the problem of changing individual attributes of a data point or a time series, while leaving all other aspects unaffected. This work focuses on the domain of human motion, more precisely karate movement patterns. To the best of our knowledge, it presents the first success at manipulating attributes of human motion data. One of the key requirements for achieving attribute manipulation on human motion is a suitable pose representation. Therefore, we design a novel continuous, rotation-based pose representation that enables the disentanglement of the human skeleton and the motion trajectory, while still allowing an accurate reconstruction of the original anatomy. The core idea of the manipulation approach is to use a transformer encoder for discovering high-level semantics, and a diffusion probabilistic model for modeling the remaining stochastic variations. We show that the embedding space obtained from the transformer encoder is semantically meaningful and linear. This enables the manipulation of high-level attributes, by discovering their linear direction of change in the semantic embedding space and moving the embedding along said direction. All code and data is made publicly available. Conference: Proceedings of the 27th International Conference on Multimodal Interaction; October 13–17, 2025; Canberra, ACT, Australia Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), October 13–17, 2025, Canberra, ACT, Australia DOI: 10.1145/3716553.3750773 ISBN: 979-8-4007-1499-3/2025/10 CCS: Computing methodologies Learning latent representations CCS: Computing methodologies Unsupervised learning CCS: Computing methodologies Cluster analysis Figure 1 . Manipulating the technique and skill level of karate athletes with our proposed Motion Diffusion Autoencoder. T. 1. Introduction © Anthony Richardson 2025. This is the author’s version of the work. It is posted here for your personal use. Not for redistribution. Please cite the definitive version when referring to this work. The definitive version was published in Proceedings of the 27th International Conference on Multimodal Interaction (ICMI ’25), https://doi.org/10.1145/3716553.3750773 . Diffusion probabilistic models have driven significant progress in generative modeling ( Ho et al., 2020 ; Song et al., 2020 ; Dhariwal and Nichol, 2021 ) , inspiring a range of new applications. One such application is attribute manipulation ( Preechakul et al., 2022 ) , which aims to modify specific attributes of data while preserving others. Applied to human motion, this could open up new possibilities across disciplines: In kinesiology, it could help analyze how diseases or injuries affect movement, aiding in the design of more accessible environments. In sport science, it could serve as a training tool, showing athletes how to refine or adapt their skills. In computer animation, it could eliminate the need for repetitive manual work by generating diverse motions from a single example. While attribute manipulation has seen success in other domains, such as manipulating facial attributes in images ( Shen and Liu, 2017 ; Akhtar et al., 2020 ; Wang et al., 2018 ; Preechakul et al., 2022 ) , human motion manipulation is confronted with additional challenges. Firstly, human motion is sequential and its attributes can be time-variant . For example, the skill levels of two athletes might be highly distinguishable during the most challenging segments of the executed motion, but show little difference in other parts. Secondly, current state-of-the-art pose representations discard all anatomical details, which is suboptimal for tasks involving attribute manipulation. In sports science, for instance, this abstraction results in motion demonstrations based on generic, uniform skeletons that fail to capture the recorded individual’s true body proportions. The resulting mismatch can can degrade the user’s experience and limit the practical utility of these demonstrations, as viewers may struggle to relate the motions to their own bodies. Therefore, motion demonstrations intended for personalized training should preserve the individual’s anatomical structure. We address both challenges mentioned above and present the first success at manipulating attributes of human motion data, in the context of karate techniques. For such karate motions, we aim to manipulate either the technique or the skill level of the recorded athlete. When changing one of those attributes, the other one should be preserved. Both of these attributes can be regarded as high-level semantics. However, given the stochastic nature of human motion, not all technique executions by athletes, sharing the same semantics, will be the same ( Bartlett, 2008 ; Aliakbarian et al., 2020 ) . Those differences can be interpreted as stochastic variations, specific to the athlete or the recording. During manipulation, those stochastic variations should not be lost, so that generated samples are not reduced to few templates. Accordingly, when manipulating a particular attribute, both the remaining semantics and the stochastic variations should be preserved during the process. To achieve this preservation, we follow Preechakul et al. ( Preechakul et al., 2022 ) and explicitly model the high-level semantics and low-level stochastic variations in two separate embeddings. Manipulation is performed only on the semantic embedding. If the semantic embedding space is linearly separable with regards to the attributes, motion manipulation becomes possible by moving the semantic embedding in the linear direction of change for the specific attribute. Such a manipulation would preserve the remaining semantics of the embedding. As the stochastic embedding is not changed, the aspects captured by it are also maintained. Afterwards, the original stochastic and the new semantic embedding are used to reconstruct a corresponding motion. To support future research on attribute manipulation, all code and data is made publicly available 1 1 1 The repository can be found at https://github.com/anthony-richardson/MoDiffAE . 2. Related Work Human Motion Manipulation : To the best of our knowledge, there exists no system that can manipulate individual high-level attributes in human motion, while preserving the remaining attributes as well as low-level details, unique to the specific motion execution. However, attribute manipulation has seen various success in the image domain. Elarabawy et al. ( Elarabawy et al., 2022 ) introduce direct inversion , which models a trade-off between the preservation of the original sample and the realization of the desired manipulation. A diffusion model is expected to preserve the semantic aspects not targeted by the manipulation. However, according to Preechakul et al. ( Preechakul et al., 2022 ) , the embedding space of diffusion models lacks such abilities due to their stochastic nature. Instead, they propose to explicitly model the separation of a semantic and a stochastic embedding space. Manipulation is then achieved by moving the semantic image embedding in the linear directions of change for specific attributes. In a work parallel to this, Kim et al. ( Kim et al., 2023 ) transfer the idea from Preechakul et al. ( Preechakul et al., 2022 ) to the task of face video editing, where the main challenge is to achieve temporal consistency among edited frames. High-level semantics are assumed to be time-invariant and are computed frame wise. However, this approach is predetermined to fail on human motion and any other sequence domain, where the semantic aspects require temporal context to be observed and are time-variant , meaning their evidence may vary throughout time. Most recently, video diffusion models have demonstrated capabilities of manipulating videos through text guidance. However, they struggle in fields such as human motion, were intricate details and small variations can