DynaPose4D: High-Quality 4D Dynamic Content Generation via Pose Alignment Loss

Jing Yang, Yufeng Yang · 2025

Read DynaPose4D: High-Quality 4D Dynamic Content Generation via Pose Alignment Loss by Jing Yang, Yufeng Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22473.txt.

Introduction

Jing Yang and co-author Yufeng Yang propose DynaPose4D for turning one still image into moving four dimensional content. Making high quality dynamic scenes from a single picture is hard because motion over time and changing camera views must stay consistent. Their method builds a three dimensional model with Gaussian splatting and then predicts pose key points across views.

A pose alignment loss ties the predicted motion together, using guidance from one chosen view as support. Experiments show strong coherence, consistency, and smooth motion in the generated scenes. The authors point to uses in computer vision research and in animation production.

Catalog record

CREATORJing Yang, Yufeng Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22473

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DynaPose4D: High-Quality 4D Dynamic Content Generation via Pose Alignment Loss Jing Yang, Yufeng Yang Abstract † † address: Sun Yat-sen University Recent advancements in 2D and 3D generative models have expanded the capabilities of computer vision. However, generating high-quality 4D dynamic content from a single static image remains a significant challenge. Traditional methods have limitations in modeling temporal dependencies and accurately capturing dynamic geometry changes, especially when considering variations in camera perspective. To address this issue, we propose DynaPose4D, an innovative solution that integrates 4D Gaussian Splatting (4DGS) techniques with Category-Agnostic Pose Estimation (CAPE) technology. This framework uses 3D Gaussian Splatting to construct a 3D model from single images, then predicts multi-view pose keypoints based on one-shot support from a chosen view, leveraging supervisory signals to enhance motion consistency. Experimental results show that DynaPose4D achieves excellent coherence, consistency, and fluidity in dynamic motion generation. These findings not only validate the efficacy of the DynaPose4D framework but also indicate its potential applications in the domains of computer vision and animation production. 1 Introduction Implicit neural rendering techniques have been widely adopted for various tasks, such as pose and shape estimation, novel view synthesis (NVS), and static 3D or dynamic 4D generation . Among these, 4D Gaussian Splatting (4DGS) has pioneered the generation of dynamic scenes from single images or video sequences by enforcing strict temporal consistency across frames [ 1 , 2 ] [ 3 , 4 ] . Despite its progress, 4D Gaussian Splatting still faces limitations, particularly in handling temporal sequence dependencies and dynamic changes. Issues arise when dealing with changes in camera perspectives, which lead to insufficient visual consistency and hinder the generation of long-duration, complex 3D motions. Consequently, dynamic scenes generated using these methods may lack coherence, appearing unnatural or inconsistent, which impairs the transition from static to dynamic content. Figure 1: Overview of DynaPose4D. From a single image, 3D Gaussians are deformed into 4D content. Temporal consistency is enforced by \mathcal{L}_{\text{SCL}} , while \mathcal{L}_{\text{Ref}}+\mathcal{L}_{\text{KML}} supervise visual consistency and refine pose transitions. To address these challenges, we propose the DynaPose4D, which integrates 4D Gaussian Splatting (4DGS) techniques with Category-Agnostic Pose Estimation (CAPE) technology [ 5 ] . The framework, shown as Fig. 1 , begins by using 3D Gaussian Splatting to construct a 3D model from a single static view. CAPE is then employed to extract pose keypoints, utilizing CNNs for image feature extraction and GCNs for graph-structured data, thereby improving the accuracy of pose pattern detection in images. This allows precise pose keypoints to be extracted from dynamic objects in video sequences, which serve as supervisory signals to ensure smooth transitions between static and dynamic content. In this research, we make several key contributions: 1. We introduce DynaPose4D, a 4D content generation framework that supports multimodal inputs, including single image and video sequences. 2. Involving the use of advanced pose estimation techniques, DynaPose4D can accurately infer the position of keypoints and their temporal aspects for dynamic objects in any given image or video. 3. We propose a novel method that uses pose keypoints as conditions to guide 4D video generation, ensuring a high level of spatio-temporal consistency of the generated 4D content, while accurately preserving the keypoint trajectories. Experimental results demonstrate that DynaPose4D achieves significant improvements in dynamic objects generation, both for single images and video sequences. We observe that the performance of the model under dynamic objects is significantly improved by the accurate detection of CAPE [ b6 ] [ b12 ] . 2 Prior work Generating high-quality 4D dynamic content from a single static image integrates challenges from multiple fields in computer vision, including 3D reconstruction, dynamic scene modeling, and pose estimation. Traditional 3D reconstruction methods, such as Neural Radiance Fields (NeRF) [ 6 ] , have made significant strides in novel view synthesis by representing scenes as volumetric radiance fields. However, NeRF requires multiple views for training, limiting its effectiveness for single-image reconstruction. Zero-1-to-3 [ 7 ] addresses this issue by leveraging large-scale diffusion models to predict novel views from a single image but is restricted to static scenes, lacking the ability to model temporal dynamics. In dynamic scene generation and 4D modeling, methods like Dynamic NeRFs and 4D Gaussian Splatting (4DGS) extend NeRF to incorporate time as an additional input, allowing for the modeling of temporal changes. While these methods enable dynamic scene generation, they still rely on multi-view and multi-frame data for training. Even with advancements like DreamGaussian4D, which enforces temporal coherence, accurately capturing dynamic geometry and temporal dependencies remains challenging, particularly when dealing with varying camera perspectives, resulting in artifacts and inconsistencies in generated content. Pose estimation plays a critical role in understanding and generating motion within dynamic scenes. Techniques like OpenPose [ 8 ] and SMPL [ 9 ] have laid the groundwork for 2D and 3D pose estimation, while category-agnostic frameworks like PoseAnything have improved keypoint localization across diverse object categories. Generative models, such as Text2Video-Zero [ 10 ] , utilize pose information to guide video generation, yet they often depend on multi-view or pose data inputs during inference, leaving gaps in their ability to generate dynamic 4D content directly from single images. 3 Method Here, we introduce our proposed dynamic 4D Gaussian Splatting generation process and a pose-supervision loss designed to enforce visual consistency during training. First, we review the process of generating 3D models from a single image, followed by an analysis of the transition from static 3D models to dynamic 4D content. Finally, we present a pose-supervision loss to ensure high-quality and coherent motion generation. 3.1 Neural Rendering for 4D Content Static 3D generation from single images. For the image-to-3D task, we employ the Zero-1-to-3 model, which uses a single RGB image to generate novel 3D viewpoints by leveraging geometric priors learned from large-scale diffusion models. The model predicts different perspectives of the object by conditioning on camera transformations, enabling high-quality 3D reconstructions from a single view. Image-to-video diffusion for dynamic prior. We employed the Stable Video Diffusion (SVD) model [ 11 ] to generate the transformation process from a single input image to a driving video. SVD is a diffusion-based generative model initially used to create high-quality static images. To extend this model for video generation tasks, we leveraged its temporal dependency by inputting a single image and introducing random noise \epsilon , generating a time-sequence video. This process can be described by the following equation: \left\{I^{\text{Ref}}\right\}_{\tau=1}^{T}=f_{\psi}\left(\epsilon;I^{\text{Sup}}\right), (1) where I^{\text{Sup}} represents the input image, \left\{I^{\text{Ref}}\right\}_{\tau=1}^{T} is the generated video sequence, \epsilon is the random noise, f_{\psi} is the image-to-video diffusion model, and T is the number of time steps in the video. Through this model, we can generate a driving video containing dynamic motion information from a single input image. Figure 2: KML minimizes MSE between poses extracted from rendered and predicted frames under one-shot support. Dynamic 4DGS generation. Next, we use 4D Gaussian Splatting (4DGS) to extend the static 3D model into dynamic 4D content [ 12 ] [ 13 ] . 4DGS explicitly models dynamic changes in both spatial and temporal dimension, predicting spatial position, rotation, and scaling variations while ensuring temporal consistency of the generated content. The deformation process can be formulated as: S_{\tau}^{\prime}=\phi(S,\tau), (2) where \phi is the deformation network,

Read the complete text