Fast and Robust Deformable 3D Gaussian Splatting
Han Jiao, Jiakai Sun, Lei Zhao*, Zhanjie Zhang, Wei Xing, Huaizhong Lin* · 2026
Read Fast and Robust Deformable 3D Gaussian Splatting by Han Jiao, Jiakai Sun, Lei Zhao*, Zhanjie Zhang, Wei Xing, Huaizhong Lin* (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20857.txt.
Introduction
Jiao and co-authors present FRoG, a fast and robust method for dynamic 3D scene reconstruction. It builds on 3D Gaussian Splatting, which renders new views of static scenes in real time. Dynamic methods often render slowly, need a good starting point cloud, or fail in dim scenes. FRoG embeds time data per Gaussian and fuses it early to speed rendering. A depth and error guided step adds Gaussians where detail is weak. Tests show faster rendering with top level visual quality.
Catalog record
| CREATOR | Han Jiao, Jiakai Sun, Lei Zhao*, Zhanjie Zhang, Wei Xing, Huaizhong Lin* |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50545 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20857
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fast and Robust Deformable 3D Gaussian Splatting Han Jiao, Jiakai Sun, Lei Zhao*, Zhanjie Zhang, Wei Xing, Huaizhong Lin* Abstract 3D Gaussian Splatting has demonstrated remarkable real-time rendering capabilities and superior visual quality in novel view synthesis for static scenes. Building upon these advantages, researchers have progressively extended 3D Gaussians to dynamic scene reconstruction. Deformation field-based methods have emerged as a promising approach among various techniques. These methods maintain 3D Gaussian attributes in a canonical field and employ the deformation field to transform this field across temporal sequences. Nevertheless, these approaches frequently encounter challenges such as suboptimal rendering speeds, significant dependence on initial point clouds, and vulnerability to local optima in dim scenes. To overcome these limitations, we present FRoG, an efficient and robust framework for high-quality dynamic scene reconstruction. FRoG integrates per-Gaussian embedding with a coarse-to-fine temporal embedding strategy, accelerating rendering through the early fusion of temporal embeddings. Moreover, to enhance robustness against sparse initializations, we introduce a novel depth- and error-guided sampling strategy. This strategy populates the canonical field with new 3D Gaussians at low-deviation initial positions, significantly reducing the optimization burden on the deformation field and improving detail reconstruction in both static and dynamic regions. Furthermore, by modulating opacity variations, we mitigate the local optima problem in dim scenes, improving color fidelity. Comprehensive experimental results validate that our method achieves accelerated rendering speeds while maintaining state-of-the-art visual quality. I Introduction The exponential growth of video content and pervasive adoption of VR/AR technologies have positioned dynamic scene reconstruction as a pivotal research domain in computer vision and graphics, garnering substantial academic interest. Early geometry-driven approaches [ 6 ; 7 ] employed dynamic primitive-based representations for scene modeling, yet were constrained by inadequate realism and insufficient detail preservation. Recently, Neural Radiance Fields (NeRF) [ 37 ] have set a new benchmark for high-fidelity static view synthesis, stimulating extensive research in diverse areas [ 54 ; 47 ; 38 ; 49 ; 3 ] . Notably, a significant line of these follow-up works focuses on adapting this powerful paradigm to dynamic scenes [ 42 ; 31 ; 57 ; 40 ; 39 ; 27 ; 28 ; 12 ; 32 ; 41 ; 14 ; 5 ; 63 ; 62 ; 13 ; 45 ] . While these extensions are capable of representing dynamic content, they also inherit NeRF’s major limitation: the rendering pipeline requires costly per-pixel ray sampling and MLP evaluations, resulting in prohibitive latency for real-time use. To break this real-time barrier, 3D Gaussian Splatting (3DG-S) [ 24 ] introduced an explicit 3D Gaussian (3DG) representation and a rasterization-based pipeline, achieving high-quality real-time rendering. This success has spurred numerous extensions to dynamic scenes, with deformation field-based methods being a prominent direction. Among these, E-D3DGS [ 2 ] introduces per-Gaussian and temporal embeddings. It employs a dual-deformation mechanism, where coarse and fine temporal embeddings are separately fed into the deformation network—a form of late-fusion architecture. However, this method faces several critical challenges. First, applying the deformation field twice per forward pass incurs significant rendering overhead. Second, and more fundamentally, it inherits a strong sensitivity to the initial point cloud. Most deformation-based methods rely on gradient-based densification, which performs poorly in sparsely initialized regions. This forces the deformation field to navigate a highly complex and ill-conditioned optimization landscape in order to learn long-range transformations and “pull” distant 3DGs into empty areas, a process that often leads to motion artifacts and detail loss. Third, these methods are often prone to local optima in dim scenes, causing color shifts. To overcome these limitations, we propose FRoG, a fast and robust framework for dynamic scene reconstruction. We utilize per-Gaussian embedding and temporal embedding as input to the deformation field. Unlike E-D3DGS, which employs coarse and fine deformations, we significantly improve the rendering speed through Hadamard product-based early temporal embedding fusion, while preserving high-quality rendering results. Moreover, we address the challenge of sparse initialization with a novel canonical field sampling strategy. While depth-guided initialization or error-based sampling has been explored in static reconstruction [ 11 ] and curve-based methods [ 30 ] , these approaches are often limited to static or explicitly structured scenes. In contrast, our work explores the far more challenging problem of sampling within an unstructured canonical field. By systematically observing the canonical field’s stability and experimentally investigating the trade-offs of inverse deformation mappings, we propose to solve this by injecting low-deviation anchors. This approach significantly simplifies the deformation field’s learning task without the instability of solving ill-posed inverse problems. Additionally, we implement an aggressive opacity reduction strategy to address the issue of local optima, which occurs in dim scenes when 3DGs near the camera introduce color compensation to training views. These improvements make our method more robust when handling both sparse initial point regions and dim scenarios. In conclusion, our main contributions are outlined as follows: • We propose FRoG, a fast and robust method for dynamic scene reconstruction using deformable 3D Gaussians. • We systematically explore temporal embedding fusion strategies and introduce an early-stage fusion based on the Hadamard product, which significantly improves rendering speed by avoiding the costly dual deformation of late-fusion schemes, while maintaining comparable rendering quality. • We design a canonical field sampling strategy that injects low-deviation anchors to ease the deformation field’s optimization challenge, enhancing reconstruction quality in regions with sparse point cloud initialization. • We propose an aggressive opacity reduction strategy that mitigates local optima in dim scenes with challenging shadows, improving color fidelity. II Related work II-A NeRF-Based Dynamic Scene Reconstruction Dynamic scene reconstruction is a challenging task in computer vision. Given the remarkable capability of NeRF to model static scenes, many researchers have extended its application to dynamic scenes. We categorize NeRF-based dynamic scene reconstruction methods as follows. Flow-based methods [ 31 ; 32 ] introduce optical flow priors to model and supervise scene flow, achieving notable results in dynamic scene reconstruction. However, these methods heavily depend on 2D priors. Spatiotemporal coupling methods [ 12 ; 53 ; 14 ; 41 ; 50 ; 5 ; 28 ] incorporate a time input t ; they directly query the radiance field using spatiotemporal coordinates, significantly reducing storage requirements and achieving high rendering quality. Per-frame training methods [ 52 ; 27 ] learn an initial neural radiance field from the first frame and update it in subsequent frames, enabling fast and online reconstruction of dynamic scenes. However, the constraints of online reconstruction prevent direct rendering of any frame and require more storage. Deformation field-based methods [ 42 ; 57 ; 48 ; 39 ; 40 ; 44 ; 19 ; 29 ; 13 ] assume that scene motion stems from the deformation of the static structure. They employ the deformation field to transform the canonical radiance field to any time step, yielding promising results. Despite their advancements, most of these methods struggle to achieve real-time rendering due to the high computational demands inherent to neural radiance fields. II-B 3DG-S-Based Dynamic Scene Reconstruction The field of 3DG-S-based dynamic reconstruction can also be categorized along similar lines to the NeRF-based methods. Flow-based methods [ 17 ; 67 ; 33 ] leverage optical flow priors to assist in dynamic scene reconstruction, achieving impressive reconstruction results. However, these