Frequency-Aware Density Control via Reparameterization for High-Quality Rendering of 3D Gaussian Splatting

Zhaojie Zeng, Yuesong Wang, Lili Ju, Tao Guan · 2025

Read Frequency-Aware Density Control via Reparameterization for High-Quality Rendering of 3D Gaussian Splatting by Zhaojie Zeng, Yuesong Wang, Lili Ju, Tao Guan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-07000.txt.

Introduction

Zhaojie Zeng, Yuesong Wang, Lili Ju, and Tao Guan present a 2025 computer graphics paper on 3D Gaussian Splatting, a method that renders scenes from many soft blobs. Fine detail needs many small blobs in busy regions. The base method lacks a rule that ties blob density to blob scale, so wrong-sized blobs can blur detail.

Their fix rewrites the scale settings so density and scale stay linked by clear constraints. A control plan then adds blobs where detail is high and deletes blobs of the wrong size, using a threshold that shifts with the scene. On varied data sets, the method beat current rivals in both measured scores and how the images look. It did so with fewer blobs.

Catalog record

CREATORZhaojie Zeng, Yuesong Wang, Lili Ju, Tao Guan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-07000

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Frequency-Aware Density Control via Reparameterization for High-Quality Rendering of 3D Gaussian Splatting Zhaojie Zeng, Yuesong Wang, Lili Ju, Tao Guan Abstract By adaptively controlling the density and generating more Gaussians in regions with high-frequency information, 3D Gaussian Splatting (3DGS) can better represent scene details. From the signal processing perspective, representing details usually needs more Gaussians with relatively smaller scales. However, 3DGS currently lacks an explicit constraint linking the density and scale of 3D Gaussians across the domain, leading to 3DGS using improper-scale Gaussians to express frequency information, resulting in the loss of accuracy. In this paper, we propose to establish a direct relation between density and scale through the reparameterization of the scaling parameters and ensure the consistency between them via explicit constraints (i.e., density responds well to changes in frequency). Furthermore, we develop a frequency-aware density control strategy, consisting of densification and deletion, to improve representation quality with fewer Gaussians. A dynamic threshold encourages densification in high-frequency regions, while a scale-based filter deletes Gaussians with improper scale. Experimental results on various datasets demonstrate that our method outperforms existing state-of-the-art methods quantitatively and qualitatively. Code — https://github.com/whoiszzj/FDS-GS Figure 1: By linking the density and scale (the purple circle corresponds to the scale, constrained by the density), our FDS-GS can better reflect the frequency changes than 3DGS via density control (more similar to the GT’s spectrum), achieving better render quality with fewer Gaussians. Introduction Novel View Synthesis (NVS) is a significant task in computer vision, aiming to generate images from arbitrary viewpoints based on scene representations reconstructed from captured images. Neural Field Rendering (NeRF) ( Mildenhall et al. 2020 ) has emerged as a leading solution for NVS, renowned for its realistic rendering quality, and has inspired numerous subsequent works ( Barron et al. 2021 ; Barron et al. 2022 ; Barron et al. 2023 ; Zhang et al. 2020 ; Chen et al. 2023b ) . However, 3D Gaussian Splatting (3DGS) ( Kerbl et al. 2023 ) has recently gained attention for its combination of high-quality rendering and speed, offering a novel point-based real-time rendering solution for NVS applications in the VR and AR industries ( Xu et al. 2023a ; Deng et al. 2022b ; Li et al. 2018 ) . In addition to the flexibility offered by organizing a set of 3D Gaussian primitives (termed Gaussians in the following), each defined by position, opacity, spherical harmonic, and covariance matrix decomposed to scale and rotation, another pivotal innovation of 3DGS is its adaptive density control. As practical NVS demands accurate representation of scene details and sparse points are commonly insufficient for scene coverage and detail representation, using dense point cloud as an initial input is necessary for many point-based methods ( Kopanas et al. 2021 ; Zhang et al. 2022 ; Franke et al. 2024 ) . The adaptive density control liberates point-based methods from the necessity of relying on dense point clouds as input. Consequently, 3DGS can produce high-quality results by leveraging only sparse points generated for free as part of the Structure-from-Motion (SfM) process ( Schönberger and Frahm 2016 ) . The underlying principle of density control can be analyzed within the signal processing framework. According to the Fourier transform, complex signals can be represented as a superposition of sine and cosine waves at different frequencies. Sufficient high-frequency components are necessary to capture complex signals, especially abrupt changes, accurately. Similarly, 3DGS uses alpha blending to render scenes by blending Gaussians of different colors to produce the final output. It requires the superposition of many Gaussians to represent complex color variations and sufficient smaller-scale ones to capture high-frequency details. In practice, 3DGS typically generates more Gaussians through view space gradients and performs split operations on larger Gaussians. However, due to the extensive parameter space in 3DGS, this process does not always result in appropriate scale, sometimes producing overly large Gaussians in regions with details (i.e., local optimal solution), as illustrated in Fig. 1 . Moreover, the decision to split a Gaussian is often made based on its current scale but without understanding the region it represents. This inflexibility will prevent the generation of sufficiently small Gaussians needed to capture fine details as high-frequency components. From the above discussions and comparison results shown Fig. 1 , we can identify the following inconsistencies in 3DGS. When dealing with high-frequency regions that represent fine details, 3DGS increases the density of Gaussians through densification. Although there is a tendency for the scale to decrease, the lack of explicit and direct regularization constraints leads to that the Gaussians generated through densification do not fully contribute to representing high-frequency information. To ensure consistency in F requency, D ensity, and S cale in 3DGS, we propose to establish a direct relation between density and scale across the domain by reparameterization. Specifically, we first quantify the density information at each position as the number of Gaussians in a unit volume centered at the position. We then reparameterize the scale and decompose it into four variables, including an absolute scalar and a relative scale vector which respond to its anisotropy in three dimensions. Following the principle that higher-density areas will have smaller scales, making them act as high-frequency components, we establish an explicit constraint between the absolute scale scalar and the density. We further introduce a frequency-aware density control to enhance the density quality, i.e., using fewer Gaussians to get better rendering quality. Following the framework of 3DGS, we use both densification and deletion strategies for density control. First, we propose a dynamic threshold for densification: we sort the gradients of the Gaussians and adjust the threshold based on their distribution. This adaptive approach provides finer control over densification compared to the fixed threshold used in vanilla 3DGS, making the density more responsive to changes in frequency. Subsequently, we design a scale-based confidence filter to effectively remove overfitting Gaussians with improper scales, further ensuring consistency in frequency, density, and scale. We will call our proposed method “FDS-GS” for brevity. In summary, our main contributions are as follows: • We quantify the density and reparameterize the scale of the Gaussian to establish a direct relation between density and scale, based on which we use the density to constrain the covariance of the Gaussian to ensure that the density can better respond to changes in frequency. • We develop a frequency-aware density control strategy consisting of densification with a dynamic threshold and a scale confidence filter, optimizing the density to be more expressive of frequency variations to improve the accuracy of the rendering. • Our method, FDS-GS, outperforms existing state-of-the-art methods by achieving higher rendering quality with a reasonable number of Gaussians. Related Works NeRF-based Rendering. Different from traditional methods that synthesize novel views based on explicit geometry representations such as meshes or points, recently emerged neural rendering methods utilize a set of MLPs to represent the scenes implicitly. Neural Radiance Field (NeRF) ( Mildenhall et al. 2020 ) can be seen as a landmark work in this direction. By encoding a continuous volume with a view-dependent appearance using the parameters of a deep fully-connected neural network, NeRF can achieve both photorealistic rendering quality and low storage pressure. Many subsequent works have been proposed to further advance the progress.

Read the complete text