Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors
Tian Guo 1 , Hui Yuan 1 , Philip Xu 1 , and David Elizondo 1 · 2025
Read Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors by Tian Guo 1 , Hui Yuan 1 , Philip Xu 1 , and David Elizondo 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-22295.txt.
Introduction
Tian Guo, Hui Yuan, Philip Xu, and David Elizondo propose SirenPose, a loss function for rebuilding dynamic 3D scenes from video. Fast-moving scenes with several targets strain current rebuild methods, which lose motion accuracy and consistency across space and time. SirenPose combines the wavy activation behavior of sine-based networks with geometric priors drawn from keypoint structure.
Its physics-inspired constraints push keypoint predictions to stay coherent across both spatial and temporal dimensions. The team also expanded the training data to 600,000 annotated instances for robust learning. Models trained with SirenPose improved greatly on consistency measures compared with prior methods. The authors report the strongest gains exactly where reconstruction usually fails: rapid motion and complex scene changes.
Catalog record
| CREATOR | Tian Guo 1 , Hui Yuan 1 , Philip Xu 1 , and David Elizondo 1 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-22295
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning Dynamic Scene Reconstruction with Sinusoidal Geometric Priors Corresponding author: [email protected] Tian Guo 1 , Hui Yuan 1 , Philip Xu 1 , and David Elizondo 1 Abstract This paper proposes a novel loss function "SirenPose" that innovatively combines the periodic activation characteristics of SIREN (Sinusoidal Representation Networks) with geometric prior information of keypoint structures to improve the accuracy of dynamic 3D scene reconstruction. Existing methods often struggle to maintain motion modeling accuracy and spatiotemporal consistency when handling fast-moving multi-target scenes. By introducing physics-based constraint mechanisms, SirenPose ensures coherence of keypoint predictions in both spatial and temporal dimensions. We expanded the training dataset to 600,000 annotated instances. Experimental results show that models based on SirenPose achieved significant improvements in spatiotemporal consistency metrics compared to existing methods, demonstrating superior performance in handling rapid motion and complex scene changes. I Introduction In computer vision, reconstructing dynamic 3D scenes from video sequences remains a fundamental and crucial task [ 12 , 52 , 49 , 1 , 13 , 2 ] . Traditional computer graphics approaches primarily rely on multi-view [ 23 , 41 , 40 , 19 , 5 ] reconstruction and template-based techniques, which require specialized hardware setups or complex multi-camera arrays, thus limiting their practical applications. With the advancement of deep learning technologies, learning-based dynamic scene reconstruction methods have demonstrated unique advantages in reconstructing 3D structures and capturing complex dynamic changes, such as human motion and object deformation, from monocular video sequences. These technologies have revolutionized creative industries, showing tremendous potential in film visual effects, game development, and virtual reality applications [ 6 , 50 , 51 ] . Recent breakthroughs in video-to-3D generation methods [ 22 ] [ 35 ] [ 19 ] based on predictive generative models have significantly advanced dynamic content creation. Current approaches predominantly rely on multi-view data for scene reconstruction, combining differentiable rendering techniques such as Neural Radiance Fields (NeRF) [ 22 ] or Gaussian Splatting with generative models based on Score Distillation Sampling (SDS) [ 33 ] [ 26 ] . While recent studies have begun exploring monocular video-based 3D dynamic [ 35 ] scene reconstruction, incorporating SE(3) [ 32 ] motion basis representations for explicit modeling of long-term 3D motion trajectories and employing "decomposition-reconstruction" [ 4 , 42 ] strategies for complex scenes with multiple dynamic objects, significant challenges remain in handling real-world dynamic scenes. Particularly when scenes contain rapidly moving multiple objects or frequent occlusions and interactions, existing methods show limitations in two critical aspects: motion modeling accuracy and spatiotemporal consistency. In terms of motion modeling precision, current approaches primarily rely on discrete pose keypoints or SE(3)-based motion basis functions, struggling to capture detailed features during rapid movements and non-rigid deformations, resulting in trajectory jitter and temporal inconsistencies. Additionally, the lack of effective geometric constraints makes it difficult to maintain 3D structural consistency during object occlusions and multi-object [ 53 ] interactions, often leading to penetration and incorrect occlusion issues [ 16 ] . These limitations significantly impact the practical effectiveness in high-precision applications such as virtual reality. latex Fig. 1: This figure illustrates the overall workflow of the SirenPose framework. At the input stage, the system accepts three different types of data: 3D frames, static images, and video frames. All input data are processed by the “Pose Estimation” module to extract the target’s keypoint information. Subsequently, the predicted keypoints are compared with the ground truth keypoints (GT Keypoint). Using the loss function L_{\text{SirenPose}} , the framework optimizes the keypoint predictions to ensure accuracy and consistency. This process adapts to various input formats while achieving efficient pose estimation. In this paper, we present "SirenPose," a novel loss function that leverages the periodic activation characteristics of SIREN [ 29 ] [ 3 ] (Sinusoidal Representation Networks) and the geometric priors of keypoint [ 36 , 48 ] structural correlations - inherent geometric constraints exist between object keypoints, and this structured information can be more accurately expressed through SIREN [ 30 ] networks’ high-frequency modeling capabilities. By enforcing this structural consistency, we introduce a powerful, physics-based constraint mechanism that ensures predicted keypoints maintain coherence in both spatial and temporal dimensions. This approach enables precise capture of detailed features during rapid motion while preserving geometric constraints between keypoints, effectively addressing complex scenarios involving occlusions and fast movements. This methodology has enabled stable training on large-scale [ 21 ] [ 24 ] dynamic scene datasets, including various motion scenarios, multi-view capture data, and in-the-wild videos with complex backgrounds. Our method demonstrates significant improvements in dynamic scene reconstruction accuracy, reducing motion trajectory jitter and discontinuities while maintaining natural scene transitions [ 34 ] . Our approach enables the integration of the UniKPT dataset from X-Pose [ 36 ] (338 keypoints, 1,237 categories, 400K instances) with additionally collected data, expanding the training set to 600K annotated instances. This enriched dataset provides more reliable supervision signals for dynamic scene reconstruction [ 28 ] . Experimental results demonstrate that SirenPose significantly enhances the model’s ability to capture high-frequency features and maintain spatiotemporal consistency in dynamic scenes. In standard video reconstruction benchmarks, SirenPose-based methods show approximately 6% improvement in spatiotemporal consistency metrics compared to existing approaches, demonstrating superior robustness in handling rapid motion and complex scene changes. This validates the innovation and effectiveness of SirenPose as a core loss design for dynamic scene reconstruction. Our key contributions are summarized as follows: 1. We propose the novel SirenPose loss function, combining SIREN networks’ periodic activation characteristics with geometric priors of keypoint structures to effectively supervise video reconstruction of dynamic scenes. 2. We establish a large-scale keypoint supervision training framework, integrating and expanding to 600K annotated instances, utilizing graph neural networks to model keypoint constraint relationships. 3. We introduce a high-frequency feature supervision strategy that significantly improves detail fidelity and spatiotemporal consistency [ 30 ] in dynamic scene reconstruction through the SirenPose loss function. II Related Work II-A Video-to-3D Reconstruction Dynamic scene reconstruction has evolved from multi-view to monocular video approaches [ 25 ] [ 23 ] [ 17 ] [ 19 ] [ 1 ] . Early methods primarily relied on multi-view videos and precise camera calibration, achieving reconstruction of visible areas through techniques like Dynamic NeRF [ 22 ] or Dynamic Gaussian Splatting [ 12 ] . These methods performed well in static or slow-moving scenes but required complex capture equipment and calibration processes.Recent research has shifted towards the more challenging task of monocular video reconstruction. The key breakthrough lies in decomposing complex scene dynamics into more manageable components: camera motion, object deformation, and rigid transformations. This decomposition strategy [ 11 ] not only simplifies the optimization process but also allows separate handling of different motion patterns [ 32 , 44 ] . By incorporating physical priors and data-driven supervision signals, new methods can better handle complex scenarios such as rapid motion, multi-object scenes, and occlusions, while achieving reasonable completion of unobserved regions. II-B Dynamic Keypoint Representation Keypoint detection and motion modeling are fundamental for understanding dynamic scenes [ 18 , 36 , 46 , 47 ] . Traditional methods are often limited to specific categories (such as humans or animals) and struggle to generalize to new categories. Recent work has introduced the category-agnostic keypoint detection (CAPE) [ 8 ] paradigm,