ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos

Shi Chen, Erik Sandström, Sandro Lombardi, Siyuan Li, Martin R. Oswald · 2025

Read ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos by Shi Chen, Erik Sandström, Sandro Lombardi, Siyuan Li, Martin R. Oswald (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-17864.txt.

Introduction

Shi Chen and 4 co-authors rebuild moving 3D scenes from a single camera video, live as it plays. Their 2025 method, ProDyG, uses Gaussian splatting, a way to paint a scene from soft 3D blobs. Past live tools tend to drop moving parts or need a depth camera.

Their system splits the scene into still and moving parts. A mask keeps pose tracking steady, while a motion graph guides how the moving parts are rebuilt bit by bit. New views from the rebuild rival off-line tools in tests. Tracking also matched top dynamic tools, and both colour and depth inputs work.

Catalog record

CREATORShi Chen, Erik Sandström, Sandro Lombardi, Siyuan Li, Martin R. Oswald
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-17864

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

ProDyG: Progressive Dynamic Scene Reconstruction via Gaussian Splatting from Monocular Videos Shi Chen, Erik Sandström, Sandro Lombardi, Siyuan Li, Martin R. Oswald Abstract Achieving truly practical dynamic 3D reconstruction requires online operation, global pose and map consistency, detailed appearance modeling, and the flexibility to handle both RGB and RGB-D inputs. However, existing SLAM methods typically merely remove the dynamic parts or require RGB-D input, while offline methods are not scalable to long video sequences, and current transformer-based feedforward methods lack global consistency and appearance details. To this end, we achieve online dynamic scene reconstruction by disentangling the static and dynamic parts within a SLAM system. The poses are tracked robustly with a novel motion masking strategy, and dynamic parts are reconstructed leveraging a progressive adaptation of a Motion Scaffolds graph. Our method yields novel view renderings competitive to offline methods and achieves on-par tracking with state-of-the-art dynamic SLAM methods. 1 Introduction Dynamic scene reconstruction is fundamental to problems like action recognition, scene understanding, autonomous driving, robotics and augmented reality, because it provides a temporally consistent spatial understanding of how objects and agents move and interact in their environment — an essential prerequisite for any system to perceive, predict, and act in the world. This problem has been tackled in various ways e.g. as an online SLAM task [ 4 , 89 , 34 ] , an offline reconstruction task [ 29 , 73 , 79 , 39 , 56 , 55 ] , and lately as a feedforward task [ 86 , 87 , 11 , 69 ] . However, most dynamic SLAM works ignore the dynamic parts [ 89 , 4 ] , only reconstructing the static world, only track rigid objects [ 85 , 43 ] or are restricted to object-centric reconstruction [ 45 , 58 ] . Offline methods typically separate the reconstruction task into pose estimation followed by reconstruction [ 55 , 56 ] , or are not scalable to long input videos due to their reliance on global optimization over all past frames [ 29 , 73 ] . Feed-forward methods train large transformers for online dynamic scene reconstruction, but are yet to achieve global pose consistency and only produce point clouds [ 69 , 11 , 86 ] . Despite recent progress, existing methods fall short of at least one of the requirements for practical dynamic scene reconstruction: (1) online operation , tightly coupling pose estimation and dense map reconstruction for scalability; (2) global pose and map consistency ; (3) expressive representations like 3DGS [ 23 ] for detailed appearance and geometry; and (4) flexibility to handle both RGB and RGB-D input. SLAM systems often ignore dynamics or lack detail; feedforward methods trade consistency and accuracy for speed; and offline methods are challenging to scale. We propose ProDyG 1 1 1 Pronounced “Prodigy”. , a method for Pro gressive, Dy namic scene reconstruction with G aussians from monocular input, that meets all four criteria. Our contributions are: • A motion mask prediction strategy, using dynamic flow and refined with semantic point prompting [ 50 ] . This is integrated into the SLAM backend for robust pose estimation. • An online dynamic reconstruction pipeline that uses Motion Scaffolds [ 29 ] to propagate 3DGS in space and time. • Support for both RGB and RGB-D, with ProDyG as the first online RGB-only method. • Competitive novel view synthesis results against state-of-the-art offline methods. Figure 1: ProDyG: Given an online RGB stream, ProDyG robustly tracks the camera and progressively reconstructs the static background and the dynamic foreground using 3D Gaussian Splatting [ 23 ] (3DGS). ProDyG reconstructs dynamic scenes with high-quality novel view synthesis, by fusing observations from different timestamps into a consistent dynamic 3DGS [ 23 ] representation with Motion Scaffolds [ 29 ] . On the right, we show a rendering from a novel view overlaid with the estimated camera trajectory and dynamic Gaussian trajectories. 2 Related Work Dynamic Dense SLAM. Dynamic dense SLAM methods mainly address two scenarios: filtering dynamic content to reconstruct only static environments [ 4 , 13 , 90 , 89 , 77 , 19 , 15 , 81 , 44 , 5 , 74 , 59 , 48 , 57 , 82 , 37 , 32 , 88 , 1 , 24 , 7 , 64 , 26 , 72 , 51 ] , or explicitly tracking and reconstructing dynamic objects alongside static structures [ 85 , 3 , 43 , 17 , 52 , 53 , 78 , 14 , 45 , 58 , 8 , 65 , 34 , 35 , 31 ] . Basic strategies for handling dynamic elements include outlier filtering, robust loss functions [ 90 , 44 , 5 ] , or covariance scaling [ 1 ] , but these fail with extensive dynamic content. Instead, a motion mask is estimated with optical flow [ 88 , 82 ] , semantic segmentation [ 26 , 14 , 35 , 57 , 4 , 33 , 77 , 51 , 81 , 74 , 85 , 59 , 72 , 53 ] or a combination of the two [ 34 , 19 , 15 , 52 ] , with uncertainty-guided loss functions [ 89 , 37 ] , with motion segmentation networks [ 57 ] , point tracking [ 65 ] , unsupervised clustering [ 17 ] or via conditional random fields [ 32 ] . In many real-world scenarios, reconstructing dynamic objects is critical. Some traditional works track rigid dynamic content during SLAM [ 85 , 3 , 43 , 17 , 52 , 53 , 78 , 14 ] or perform object-centric non-rigid reconstruction [ 45 , 58 , 8 ] . While the above-mentioned methods can reconstruct dynamic point clouds [ 4 , 85 , 74 , 59 , 88 , 17 ] , surfels [ 52 , 53 ] , oct-trees [ 81 ] or signed distance functions [ 48 , 45 , 58 , 8 ] , they struggle to model photometric details and lighting effects needed for scene understanding and photorealistic rendering. In response, neural implicit methods [ 33 , 77 , 51 , 19 ] have been proposed, but they are too slow for real-time SLAM. Dynamic 3DGS [ 23 ] SLAM has emerged as a solution [ 15 , 34 , 35 , 31 , 89 , 32 , 72 , 26 ] with concurrent works such as DynaGSLAM [ 34 ] using dynamic scene flow from optical flow to update the Gaussian means, [ 65 ] adapts [ 79 ] to the online setting and [ 35 ] uses an MLP to deform the dynamic parts. These works cannot handle pure RGB input. For a recent survey on dynamic SLAM, we refer to [ 71 ] . Dynamic 3D Reconstruction. Dynamic 3D reconstruction shares many similarities with dynamic SLAM, but videos are processed offline [ 29 , 73 , 79 , 9 , 21 , 80 , 27 , 28 , 36 , 40 , 41 , 49 , 60 , 84 , 2 , 16 , 39 , 25 , 91 , 18 , 38 , 68 ] . These works typically implement a strategy

Read the complete text