GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments

Lin Zeng, Boming Zhao, Jiarui Hu, Xujie Shen, Hujun Bao, Zhaopeng Cui · 2025

Read GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments by Lin Zeng, Boming Zhao, Jiarui Hu, Xujie Shen, Hujun Bao, Zhaopeng Cui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08867.txt.

Introduction

Lin Zeng, Boming Zhao, Jiarui Hu and colleagues present GaussianUpdate, a way to keep 3D scene models current as real places change in 2025. Scenes built with 3D Gaussian Splatting look sharp, but rebuilding them from scratch whenever furniture moves or objects vanish is wasteful.

Their method updates the Gaussian scene with new data while keeping what is still valid from before. A multi-stage strategy handles different kinds of change explicitly, and a visibility-aware learning scheme with generative replay refreshes the model without storing old images. Benchmarks showed high-quality, real-time rendering, plus the ability to visualise how a scene looked at different times.

Catalog record

CREATORLin Zeng, Boming Zhao, Jiarui Hu, Xujie Shen, Hujun Bao, Zhaopeng Cui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08867

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments Lin Zeng, Boming Zhao, Jiarui Hu, Xujie Shen, Hujun Bao, Zhaopeng Cui Abstract Novel view synthesis with neural models has advanced rapidly in recent years, yet adapting these models to scene changes remains an open problem. Existing methods are either labor-intensive, requiring extensive model retraining, or fail to capture detailed types of changes over time. In this paper, we present GaussianUpdate, a novel approach that combines 3D Gaussian representation with continual learning to address these challenges. Our method effectively updates the Gaussian radiance fields with current data while preserving information from past scenes. Unlike existing methods, GaussianUpdate explicitly models different types of changes through a novel multi-stage update strategy. Additionally, we introduce a visibility-aware continual learning approach with generative replay, enabling self-aware updating without the need to store images. The experiments on the benchmark dataset demonstrate our method achieves superior and real-time rendering with the capability of visualizing changes over different times. Please refer to our project webpage for more informations: https://zju3dv.github.io/GaussianUpdate . Figure 1 : GaussianUpdate . We present GaussianUpdate, a novel framework that can update the existing Gaussian fields by generatively replaying past data and currently captured data. The updated Gaussian fields can synthesize novel views across different times, capturing geometric layout changes between consecutive timestamps. 1 Introduction High-quality scene reconstruction and realistic image rendering are crucial for many applications, such as AR/VR, 3D gaming, and robotics. In recent years, neural scene representations, represented by Neural Radiance Fields (NeRF) [ 22 ] , have demonstrated outstanding performance for novel view synthesis. While these rendering results are impressive, the same scene often undergoes many changes over time. For example, the same room will have different objects, different furniture placements, and different lighting conditions at different times. Therefore, updating the original neural model is crucial to render images that align with the current appearance of the scene. Additionally, the knowledge from previous moments is valuable, and we hope to retain information from various times after updating the original 3D model. This would enable high-quality images of the same scene to be rendered at different moments when needed. A simple approach is to retrain a neural model with the images from the current moment. However, preserving neural models from each moment would increasingly consume storage space. Another strategy is to directly update the neural model with data from the current time, but this may lead to catastrophic forgetting [ 21 ] , causing the neural model to lose appearance and geometric information from previous times and be unable to accurately render scenes from earlier times. Existing methods [ 31 , 33 , 30 , 24 , 34 ] for 4D modeling can represent continuous dynamics through dynamic fields, but they are not directly applicable to scenarios with abrupt scene changes, such as the sudden appearance or disappearance of objects. Several attempts [ 2 , 32 ] have been made to integrate continual learning strategies [ 5 ] into the learning of NeRF models. For instance, CLNeRF [ 2 ] combines continual learning with NeRF, updating the NeRF model with new data while preserving the geometric and appearance information from previous moments. However, due to the inherent drawbacks, NeRF-based methods fail to locate the changes over time and suffer from low rendering efficiency. Recently, 3D Gaussian Splatting [ 12 ] introduced a tile-based splatting rendering framework that uses 3D Gaussians to represent the scene. This explicit representation simplifies modeling and detection of fine-grained geometric and photometric changes, while enabling real-time rendering, offering a promising solution to the challenges mentioned earlier. However, designing a system that combines 3D Gaussian representation with continual learning is far from straightforward and presents substantial challenges. Firstly, a temporal model is needed for the 3D Gaussian representation to address the differences in the appearance of the scene at different moments in time. Furthermore, when objects that originally existed in the scene are no longer present at the current moment, or new objects have appeared, we need a strategy to explicitly update the 3D Gaussian model. This process entails the precise removal of the 3D Gaussians associated with the objects that have disappeared and the addition of new 3D Gaussians at the necessary locations. The most relevant work is dynamic 3D Gaussian modeling [ 31 , 33 , 16 ] . These methods use an additional network to learn the deformation fields at different moments, warping the attributes of each 3D Gaussian from the canonical space to render temporally varying scene images. However, these methods assume that scene changes are caused by the continuous motion of objects and cannot model discrete changes in our tasks such as the appearance or disappearance of objects. In this paper, to tackle these challenges, we introduce GaussianUpdate, a novel method that combines 3D Gaussian representation with continual learning for the first time. Our method effectively updates the previous neural model with current data while preserving as much information as possible from all past scenes. Compared to past moments, the changes in the current 3D scene are explicitly modeled as three types: lighting changes, disappearance of existing objects, and the addition of new objects. To model these discrete changes, we first propose a global appearance updating method for 3D Gaussians that utilizes hash encoding to store temporal information, thereby fitting the lighting variations at different moments. Additionally, we developed a 3D change detection method to identify and remove the 3D Gaussians corresponding to objects that have disappeared at the current moment. We also designed a simple but efficient COLMAP-based point addition strategy to represent newly added objects. With these meticulous designs, the proposed GaussianUpdate can ensure real-time photorealistic rendering of the scene’s appearance at different times while also supporting the visualization of detailed changes over time. Our contributions can be summarized as follows: • We present GaussianUpdate, a novel framework that innovatively integrates 3D Gaussian representation with continual learning, leveraging images of the current scene to update the previous model and render novel view images at different moments. • We develop a novel multi-stage update strategy including hash encoding attribute updating, 3D change detection for object removal, and COLMAP-based point addition, which can adapt the previous 3D Gaussian model to the current changes in real-world scenes. • We present the visibility-aware continual learning with generative replay which enables self-aware updating to various types of changes without the need to store additional images. • Experimental results demonstrate our method achieves superior rendering quality compared to existing methods while ensuring real-time rendering and visualization of changes over different times. 2 Related Work NeRF and 3D Gaussian Splatting. Neural Radiance Field(NeRF) possesses robust multi-view synthesis capabilities. The vanilla NeRF [ 22 ] uses multi-layer perceptrons (MLPs) to map the input 3D coordinates and view direction into different colors and opacities and then employs volume rendering [ 11 ] to produce images from novel views of the scene. However, NeRF can only reconstruct static scenes [ 9 ] . To capture the dynamic changes in a scene, dynamic NeRF [ 24 , 8 , 3 , 7 ] trains a deformable field that deforms information from the canonical space to match different scene changes over time. NeRF-w [ 20

Read the complete text