1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering

Yuan Yuheng, Qiuhong Shen, Xingyi Yang, Xinchao Wang · 2025

Read 1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering by Yuan Yuheng, Qiuhong Shen, Xingyi Yang, Xinchao Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-16422.txt.

Introduction

Rendering a moving three-dimensional scene in real time takes huge storage and heavy computing. Yuheng Yuan and three colleagues tackle that problem in 2025 with 4DGS-1K, a faster version of a method called 4D Gaussian Splatting.

The authors trace the waste to two sources: many short-lived scene elements, and elements that get drawn even when they add nothing to a frame. A new score prunes the short-lived ones, and a stored mask skips the idle ones. The result runs above one thousand frames per second on modern graphics cards. Storage drops sharply and drawing runs about nine times faster, with picture quality close to the original method.

Catalog record

CREATORYuan Yuheng, Qiuhong Shen, Xingyi Yang, Xinchao Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-16422

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

1000+ FPS 4D Gaussian Splatting for Dynamic Scene Rendering Yuan Yuheng, Qiuhong Shen, Xingyi Yang, Xinchao Wang Abstract 4D Gaussian Splatting (4DGS) has recently gained considerable attention as a method for reconstructing dynamic scenes. Despite achieving superior quality, 4DGS typically requires substantial storage and suffers from slow rendering speed. In this work, we delve into these issues and identify two key sources of temporal redundancy. (Q1) Short-Lifespan Gaussians : 4DGS uses a large portion of Gaussians with short temporal span to represent scene dynamics, leading to an excessive number of Gaussians. (Q2) Inactive Gaussians : When rendering, only a small subset of Gaussians contributes to each frame. Despite this, all Gaussians are processed during rasterization, resulting in redundant computation overhead. To address these redundancies, we present 4DGS-1K , which runs at over 1000 FPS on modern GPUs. For Q1, we introduce the Spatial-Temporal Variation Score, a new pruning criterion that effectively removes short-lifespan Gaussians while encouraging 4DGS to capture scene dynamics using Gaussians with longer temporal spans. For Q2, we store a mask for active Gaussians across consecutive frames, significantly reducing redundant computations in rendering. Compared to vanilla 4DGS, our method achieves a 41 × 41\times reduction in storage and 9 × 9\times faster rasterization speed on complex dynamic scenes, while maintaining comparable visual quality. Please see our project page at 4DGS-1K . Figure 1 : Compressibility and Rendering Speed. We introduce 4DGS-1K , a novel compact representation with high rendering speed. In contrast to 4D Gaussian Splatting (4DGS) [ 40 ] , we can achieve rasterization at 1000+ FPS while maintaining comparable photorealistic quality with only 2\% of the original storage size. The right figure is the result tested on the N3V [ 18 ] datasets, where the radius of the dot corresponds to the storage size. 1 Introduction Novel view synthesis for dynamic scenes allows for the creation of realistic representations of 4D environments, which is essential in fields like computer vision, virtual reality, and augmented reality. Traditionally, this area has been led by neural radiance fields (NeRF) [ 25 , 21 , 18 , 2 , 12 ] , which model opacity and color over time to depict dynamic scenes. While effective, these NeRF-based methods come with high training and rendering costs, limiting their practicality, especially in real-time applications and on devices with limited resources. Recently, point-based representations like 4D Gaussian Splatting (4DGS) [ 40 ] have emerged as strong alternatives. 4DGS models a dynamic scene using a set of 4D Gaussian primitives, each with a 4-dimensional mean and a 4\times 4 covariance matrix. At any given timestamp, a 4D Gaussian is decomposed into a set of conditional 3D Gaussians and a marginal 1D Gaussian, the latter controlling the opacity at that moment. This mechanism allows 4DGS to effectively capture both static and dynamic features of a scene, enabling high-fidelity dynamic scene reconstruction. However, representing dynamic scenes with 4DGS is both storage-intensive and slow. Specifically, 4DGS often requires millions of Gaussians, leading to significant storage demands (averaging 2GB for each scene on the N3V [ 18 ] dataset) and suboptimal rendering speed. In comparison, mainstream deformation field methods [ 39 ] require only about 90MB for the same dataset. Therefore, reducing the storage size of 4DGS [ 40 ] and improving rendering speed are essential for efficiently representing complex dynamic scenes. We look into the cause of such an explosive number of Gaussian and place a specific emphasis on two key issues. (Q1) A large portion of Gaussians exhibit a short temporal span. In empirical experiments, 4DGS tends to favor “flicking” Gaussians to fit complex dynamic scenes, which just influence a short portion of the temporal domain. This necessitates that 4DGS relies on a large number of Gaussians to reconstruct a high-fidelity scene. As a result, substantial storage is needed to record the attributes of these Gaussians: (Q2) Inactive Gaussians lead to redundant computation. During rendering, 4DGS needs to process all Gaussians. However, only a very small portion of Gaussians are active at that moment. Therefore, most of the computation time is spent on inactive Gaussians. This phenomenon greatly hampers the rendering speed. In this paper, we introduce 4DGS-1K , a framework that significantly reduces the number of Gaussians to minimize storage requirements and speedup rendering while maintaining high-quality reconstruction. To address these issues, 4DGS-1K introduces a two-step pruning approach: • Pruning Short-Lifespan Gaussians. We propose a novel pruning criterion called the spatial-temporal variation score , which evaluates the temporal impact of each Gaussian. Gaussians with minimal influence are identified and pruned, resulting in a more compact scene representation with fewer Gaussians with short temporal span. • Filtering Inactive Gaussians. To further reduce redundant computations during rendering, we use a key-frame temporal filter that selects the Gaussians needed for each frame. On top of this, we share the masks for adjacent frames. This is based on our observation that Gaussians active in adjacent frames often overlap significantly. Besides, the pruning in step 1 enhances the masking process in step 2. By pruning Gaussians, we increase the temporal influence of each Gaussian, which allows us to select sparser key frames and further reduce storage requirements. We have extensively tested our proposed model on various dynamic scene datasets including real and synthetic scenes. As shown in Fig. 1 , 4DGS-1K reduces storage costs by 41 \times on the Neural 3D Video datasets [ 18 ] while maintaining equivalent scene representation quality. Crucially, it enables real-time rasterization speeds exceeding 1,000 FPS. These advancements collectively position 4DGS-1K as a practical solution for high-fidelity dynamic scene modeling without compromising efficiency. In summary, our contributions are three-fold: • We delve into the temporal redundancy of 4D Gaussian Splatting, and explain the main reason for the storage pressure and suboptimal rendering speed. • We introduce 4DGS-1K , a compact and memory-efficient framework to address these issues. It consists of two key components, a spatial-temporal variation score-based pruning strategy and a temporal filter. • Extensive experiments demonstrate that 4DGS-1K not only achieves a substantial storage reduction of approximately 41 \times but also accelerates rasterization to 1000+ FPS while maintaining high-quality reconstruction. 2 Related Work 2.1 Novel view synthesis for static scenes Recently, neural radiance fields(NeRF) [ 25 ] have achieved encouraging results in novel view synthesis. NeRF [ 25 ] represents the scene by mapping 3D coordinates and view dependency to color and opacity. Since NeRF [ 25 ] requires sampling each ray by querying the MLP for hundreds of points, this significantly limits the training and rendering speed. Subsequent studies [ 5 , 32 , 35 , 38 , 11 , 26 , 31 , 37 ] have attempted to speed up the rendering by introducing specialized designs. However, these designs also constrain the widespread application of these models. In contrast, 3D Gaussian Splatting(3DGS) [ 14 ] has gained significant attention, which utilizes anisotropic 3D Gaussians to represent scenes. It achieves high-quality results with intricate details, while maintaining real-time rendering performance. 2.2 Novel view synthesis for dynamic scenes Dynamic NVS poses new challenges due to the temporal variations in the input images. Previous NeRF-based dynamic scene representation methods [ 21 , 24 , 18 , 17 , 2 , 12 , 30 , 4

Read the complete text