Gaussian Set Surface Reconstruction through Per-Gaussian Optimization

Zhentao Huang, Di Wu, Zhenbang He, Minglun Gong · 2025

Read Gaussian Set Surface Reconstruction through Per-Gaussian Optimization by Zhentao Huang, Di Wu, Zhenbang He, Minglun Gong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-18923.txt.

Introduction

Zhentao Huang and three co-authors rebuild accurate surfaces from 3D Gaussian splats. Their 2025 paper presents GSSR, a reconstruction method. Gaussian splatting renders new views well but places its blobs loosely. Uneven blobs blur the true surface and fight later editing.

The method tunes each Gaussian's place on its own. Normal agreement is enforced at pixel and blob levels, across one view and many. A fading penalty prunes needless blobs. Periodic resets, guided by depth and normals, tidy the spread. Rebuilt scenes showed far sharper placement precision. Clean surfaces also make scenes easier to edit and reuse.

Catalog record

CREATORZhentao Huang, Di Wu, Zhenbang He, Minglun Gong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-18923

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Gaussian Set Surface Reconstruction through Per-Gaussian Optimization Zhentao Huang, Di Wu, Zhenbang He, Minglun Gong Abstract 3D Gaussian Splatting (3DGS) effectively synthesizes novel views through its flexible representation, yet fails to accurately reconstruct scene geometry. While modern variants like PGSR introduce additional losses to ensure proper depth and normal maps through Gaussian fusion, they still neglect individual placement optimization. This results in unevenly distributed Gaussians that deviate from the latent surface, complicating both reconstruction refinement and scene editing. Motivated by pioneering work on Point Set Surfaces, we propose Gaussian Set Surface Reconstruction (GSSR), a method designed to distribute Gaussians evenly along the latent surface while aligning their dominant normals with the surface normal. GSSR enforces fine-grained geometric alignment through a combination of pixel-level and Gaussian-level single-view normal consistency and multi-view photometric consistency, optimizing both local and global perspectives. To further refine the representation, we introduce an opacity regularization loss to eliminate redundant Gaussians and apply periodic depth- and normal-guided Gaussian reinitialization for a cleaner, more uniform spatial distribution. Our reconstruction results demonstrate significantly improved geometric precision in Gaussian placement, enabling intuitive scene editing and efficient generation of novel Gaussian-based 3D environments. Extensive experiments validate GSSR’s effectiveness, showing enhanced geometric accuracy while preserving high-quality rendering performance. Our code will be released upon acceptance of the paper. Figure 1 : Comparison of 3D Gaussian distributions from 2DGS, PGSR, and our method across two scenes. Gaussian centers are visualized as colored points, with hue indicating surface distance error. Areas where the white background is visible through the Gaussians denote incomplete surface coverage. Red and green boxes highlights regions with improved accuracy and coverage, respectively. Quantitative and visual results confirm that our approach achieves both a lower reconstruction error and a more complete Gaussian representation. Figure 2 : Overview of the proposed GSSR pipeline. Given multi-view posed images and initialized 3D Gaussians, we render the depth map, depth-based normals and alpha-blended normals. The optimization stage includes four major components: (1) single-view normal consistency, (2) multi-view photometric consistency (3) RGB rendering loss, and (4) opacity regularization. Additionally, Gaussians are periodically resampled using our view-based opacity-guided strategy, resulting in a more uniform and accurate distribution. 1 Introduction Novel View Synthesis (NVS) remains a fundamental challenge in computer vision and graphics, enabling photorealistic scene rendering from sparse inputs for applications ranging from eXtended Reality (XR) to autonomous systems. Modern NVS approaches can be broadly categorized by their treatment of scene geometry: explicit modeling methods that reconstruct accurate 3D surfaces [ 27 , 28 ] and implicit scene representations that bypass explicit reconstruction [ 23 , 25 , 21 ] . With advances in deep neural networks, research on implicit scene representations has gained momentum. In particular, Neural Radiance Fields (NeRF) [ 25 ] leverages MLPs to model scenes volumetrically, achieving unprecedented rendering quality. However, these methods incur significant costs: slow training, high computational demands, and limited editability. 3D Gaussian Splatting [ 21 ] overcomes these limitations through its explicit, Gaussian-based representation. By employing differentiable rasterization of 3D Gaussians, it achieves real-time rendering while maintaining NeRF-comparable visual quality. The discrete nature of Gaussians offers advantages in scene manipulation and surface reconstruction compared to continuous implicit representations. However, standard 3DGS prioritizes rendering quality over geometric precision, often producing misaligned Gaussian distributions that poorly approximate physical surfaces. Recent extensions like 2DGS [ 15 ] , PGSR [ 3 ] , and GauSurf [ 30 ] address this through geometric regularization, optimizing Gaussians to produce accurate depth and normal maps. While these methods improve surface estimation, they focus on aggregate outputs rather than individual Gaussian placement. Consequently, the Gaussians remain unevenly distributed and misaligned with underlying surfaces, limiting their effectiveness for geometry-sensitive applications including scene editing and dynamic object deformation. Inspired by the foundational work on Point Set Surfaces [ 1 ] , we propose Gaussian Set Surface Reconstruction (GSSR), a method that optimizes Gaussian distributions by: (1) anchoring centers to latent scene surfaces, (2) enforcing spatial uniformity, and (3) aligning dominant normals with surface geometry. GSSR’s explicit geometric formulation enables direct compatibility with point-based manipulation techniques, opening new possibilities for precise scene editing, animation workflows, and dynamic scene processing – effectively bridging the gap between neural rendering quality and practical 3D content creation needs. GSSR optimize individual Gaussian location and orientation through two key mechanisms: (1) The application of explicit geometric constraints derived from estimated planar structures and surface smoothness, encouraging Gaussians to coalesce onto latent scene surfaces; and (2) A joint optimization of opacity and position coupled with adaptive pruning, which aggressively eliminates Gaussians with low contributions while strategically repositioning others at underrepresented areas. Our experiments across multiple datasets demonstrate that our approach yields a cleaner scene representation with more geometrically consistent ellipsoids while preserving high rendering quality. In summary, our key contributions include: • GSSR, a 3DGS framework optimizing Gaussians for uniform placement and precise surface alignment • A geometric regularization technique that advances beyond prior 3DGS methods in per-Gaussian depth/normal accuracy and visual coherence • An opacity-position optimization strategy that prunes redundancies while enforcing surface adherence • State-of-the-art Gaussian accuracy and completeness across three datasets, without rendering quality loss 2 Related Work 2.1 Gaussian Splatting for 3D Reconstruction Due to the explicit Gaussian ellipsoids representation of 3DGS, many effective works [ 13 , 15 , 29 , 33 , 3 , 8 , 5 , 30 ] that focus on applying geometric regularization to improve geometry accuracy are proposed. SuGaR [ 13 ] introduces a signed distance function and density supervision to guid Gaussians toward object surfaces. 2DGS [ 15 ] replaces volumetric ellipsoids with the elliptical disk-shaped Gaussians, which better align with local geometry and improve surface fidelity during rendering. GOF [ 34 ] integrates implicit neural representation within the Gaussian framework to enhance geometric detail and multi-view consistency. PGSR [ 3 ] introduces a depth estimation strategy and geometric constraints that yield smoother and more accurate surface reconstruction. GausSurf [ 30 ] incorporates geometric guidance by combining patch-match MVS for texture-rich regions and normal priors for texture-less areas. Additionally, methods like GSDF [ 33 ] and NeuSG [ 4 ] combine 3DGS with Signed Distance Field (SDF) networks to improve reconstruction quality. While these approaches effectively improve surface smoothness and depth accuracy, they often result in overly dense and irregular Gaussian distributions. In contrast, our method not only enhances surface reconstruction but also produces a cleaner and more spatially consistent Gaussian representation, while maintaining high-fidelity rendering quality. 2.2 Scene Editing 3D Gaussian Splatting offers photorealistic rendering and holds great promise for applications in XR, content creation, and digital twins. However, its ellipsoidal representation lacks geometric consistency with real-world surfaces, making intuitive and semantically meaningful scene editing difficult compared to mesh-based approaches. Recent methods such as GaussianEditor [ 6 ] , Gaussian Grouping [ 32 ] , Point’n Move [ 16 ] , and Feng et al. [ 9 ] explore 3DGS editing through semantic prompts, mask-based grouping, or improved Gaussian splitting, but are largely limited to basic object-level operations such as removal, rotation, and translation. In parallel, some efforts have explored scene editing in NeRF-based representations [ 31 , 17 , 35 , 14 ] , addressing similar challenges of geometry-awareness and user interaction. While

Read the complete text