SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation

Jiaming Zhang, Shengming Cao, Rui Li, Xiaotong Zhao, Yutao CuiXinglin Hou, Gangshan Wu, Haolan Chen, Limin Wang, Kai Ma · 2025

Read SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation by Jiaming Zhang, Shengming Cao, Rui Li, Xiaotong Zhao, Yutao CuiXinglin Hou, Gangshan Wu, Haolan Chen, Limin Wang, Kai Ma (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-19320.txt.

Introduction

Jiaming Zhang and ten co-authors present SteadyDancer, a system that animates a still photo of a person so they move to a desired pose sequence without losing their identity. The dominant paradigm binds image to motion in a way that overlooks mismatches between the reference frame and the driving poses, causing identity drift and visual artefacts in real use. SteadyDancer instead builds on an image-to-video design with a mechanism that reconciles the two conflicting conditions, the photo that must stay fixed and the motion that must be followed.

Pose modulation modules shape an adaptive pose representation that fits the reference image, and a staged training pipeline separates the objectives so each is learned in turn. The authors claim robust first-frame preservation, a first for this task, with precise control kept. The system targets animation tools where the opening frame is the product's promise.

Catalog record

CREATORJiaming Zhang, Shengming Cao, Rui Li, Xiaotong Zhao, Yutao CuiXinglin Hou, Gangshan Wu, Haolan Chen, Limin Wang, Kai Ma
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-19320

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

SteadyDancer: Harmonized and Coherent Human Image Animation with First-Frame Preservation Jiaming Zhang, Shengming Cao, Rui Li, Xiaotong Zhao, Yutao CuiXinglin Hou, Gangshan Wu, Haolan Chen, Limin Wang, Kai Ma Abstract Preserving first-frame identity while ensuring precise motion control is a fundamental challenge in human image animation. The Image-to-Motion Binding process of the dominant Reference-to-Video (R2V) paradigm overlooks critical spatio-temporal misalignments common in real-world applications, leading to failures such as identity drift and visual artifacts. We introduce SteadyDancer, an Image-to-Video (I2V) paradigm-based framework that achieves harmonized and coherent animation and is the first to ensure first-frame preservation robustly. Firstly, we propose a Condition-Reconciliation Mechanism to harmonize the two conflicting conditions, enabling precise control without sacrificing fidelity. Secondly, we design Synergistic Pose Modulation Modules to generate an adaptive and coherent pose representation that is highly compatible with the reference image. Finally, we employ a Staged Decoupled-Objective Training Pipeline that hierarchically optimizes the model for motion fidelity, visual quality, and temporal coherence. Experiments demonstrate that SteadyDancer achieves state-of-the-art performance in both appearance fidelity and motion control, while requiring significantly fewer training resources than comparable methods. Figure 1 : We introduce SteadyDancer, an Image-to-Video (I2V) paradigm-based human animation framework, with Motion-to-Image Alignment to achieve harmonized and coherent animation with first-frame preservation. † † * Work is done during internship at Tencent PCG. ‡ Equal contribution. † Corresponding author ([email protected]). 1 Introduction Human image animation, which aims to generate video from a single static image with controllable motion pose, has emerged as a prominent research frontier in video generation and holds immense potential for applications like film production, advertising, and video game development. While significant progress has been made by prior works, the breakthroughs of diffusion models 16 ; 28 have recently unlocked new capabilities in generating high-fidelity and temporally coherent customized videos. Most existing methods adhere to the Reference-to-Video (R2V) paradigm ( 14 ) , as illustrated in the left part of Fig. 2 (b). Specifically, the R2V paradigm defines the animation task as binding the reference image onto the driven pose, which inherently relaxes the alignment constraints between the inputs, thereby reducing the objective to only achieving a natural appearance transfer. However, inherent discrepancies exist between the image and pose inputs in practical scenarios as shown in Fig. 2 (a), manifesting as spatial misalignments limb structure and proportion) and temporal misalignments (movement type and amplitude). In such scenarios, the relaxation of alignment constraints within the R2V paradigm leads to unacceptable results, including severe artifacts, appearance distortions, and temporal incoherence. Moreover, in the common start-gap scenario due to the temporal misalignments, the R2V paradigm instantly binds the reference image to the first pose, completely omitting any smooth transition from the reference state. This abrupt jump, combined with the appearance distortion, makes it unsuitable for applications that require high visual and temporal fidelity, such as keyframe-based or static photo animation, and Post-Production & VFX. Figure 2 : (a) The spatio-temporal misalignment in practical scenarios. (b) The illustration of Reference-to-Video (R2V) and Image-to-Video (I2V) paradigms for human image animation. While R2V only cares about how to bind the reference image to driven motion, I2V additionally needs to carefully deal with the misalignment from the driven motion to the reference image. In contrast, the Image-to-Video (I2V) paradigm, which inherently guarantees first-frame preservation by generating consistent and coherent videos starting from the initial frame, maximizes the fidelity and emerges as a preferable solution. However, I2V-based animation research remains scarce and technically challenging, due to its stringent requirement for first-frame coherence, which demands that all input conditions and generated results strictly adhere to the initial frame. Specifically, it requires the pose to be modulated into a suitable control signal for the reference image, necessitating much tighter alignment than R2V. Failure to achieve such strict alignment will severely impact performance, especially when there is insufficient resources to train high-capacity models for learning motion control. In this paper, we propose SteadyDancer, an animation model built on the I2V paradigm with first-frame preservation. Firstly, to resolve the trade-off between appearance preservation and motion control within the I2V paradigm, we propose Condition-Reconciliation Mechanism that achieves precise motion-driven control without sacrificing first-frame fidelity by optimizing at three levels, including condition fusion, injection, and augmentation. Secondly, to address the spatio-temporal misalignment between the reference image and the driving pose, we introduce Synergistic Pose Modulation Modules to extract pose representations that are better adapted to the reference image, comprising the Spatial Structure Adaptive Refiner, the Temporal Motion Coherence Module, and the Frame-wise Attention Alignment Unit. Finally, to achieve efficient and stable model training, we propose a Staged Decoupled-Objective Training Pipeline , including an Action Supervision Stage to establish precise motion control, a Condition-Decoupled Distillation Stage to enhance the generated video quality, and a Motion Discontinuity Mitigation Stage that aims to generate smooth and continuous results.Ultimately, building on these strategies, SteadyDancer successfully activates the ability of first-frame preservation in the human animation task, enabling it to robustly handle misalignments and generate videos with high fidelity and accurate motion. Moreover, SteadyDancer achieves superior performance over existing methods while requiring substantially fewer training resources. In summary, the main contribution of this work lies in: (i) We propose SteadyDancer, a novel high-fidelity animation framework that achieves first-frame preservation firstly and significant training resource efficiency; (ii) To address the conflict and mismatch between the reference image and the driven pose, and to improve training efficiency, we propose a Condition-Reconciliation Mechanism, Synergistic Pose Modulation Modules, and a Staged Decoupled-Objective Training Pipeline; (iii) Extensive quantitative and qualitative results on multiple benchmarks validate the superiority and effectiveness of our proposed method. 2 Related Work Diffusion for video generation Diffusion-based models have become state-of-the-art for generative tasks, achieving remarkable success in both image 20 ; 37 and video generation 36 ; 4 . With the introduction of OpenAI’s Sora 2 , which leverages the Diffusion Transformer (DiT) architecture 19 , DiT-based approaches have since supplanted UNets 21 , as their pure Transformer structure facilitates massive parameter scaling and has become the mainstream technical route. Concurrently, to efficiently handle video data, many recent DiT models 28 ; 16 ; 35 adopt 3D VAEs over standard 2D VAEs 20 ; 15 to compress data across both spatial and temporal dimensions. The proliferation of powerful, open-source foundational models has further accelerated this field. Consequently, human image animation, as one of downstream tasks, directly benefits from the enhanced power and fidelity of these new models. Human Image Animation. Earlier works in image animation primarily relied on warping-based feature representations and GAN-based architectures 22 ; 23 ; 42 . Recently, this field has pivoted to diffusion models, yielding significant performance improvements. Early diffusion-based methods, such as DisCo 29 , leveraged ControlNet 37 for pose guidance and integrated motion modules to enhance cross-frame consistency. A key breakthrough came with Animate Anyone 10 and subsequent studies 34 ; 45 ; 26 , which utilize a UNet-based ReferenceNet to inject appearance features, achieving excellent identity preservation. To further enhance controllability, other works 45 ; 9 ; 43 ; 31 ; 41 incorporated camera parameters and rich 3D geometric guidance, such as depth, SMPL, hand. Mirroring the trend in general video generation, DiT-based architectures 19 have recently been adapted for human animation 40 ;

Read the complete text