Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision

Yu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, Qi Liu · 2025

Read Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision by Yu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, Qi Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-10316.txt.

Introduction

Yu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, and Qi Liu improve 3D Gaussian Splatting for scenes where near and far regions fight each other. Distant areas get inaccurate depth, while close-range structure degrades, because supervision signals conflict across the depth range. Their framework attacks both problems by unifying the physics of camera defocus with multi-view geometric constraints.

A monocular depth estimator supplies scale-recovered depth priors, and defocus convolution synthesizes physically accurate blurred images, tied together by a new depth-of-field loss. Separately, semi-dense feature matching across views feeds a least-squares step that enforces depth consistency on reliable matches. On the Waymo Open Dataset the method improved peak signal-to-noise ratio by just under one decibel over the previous best.

Catalog record

CREATORYu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, Qi Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-10316

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Depth-Consistent 3D Gaussian Splatting via Physical Defocus Modeling and Multi-View Geometric Supervision Yu Deng, Baozhu Zhao, Junyan Su, Xiaohan Zhang, Qi Liu Abstract Three-dimensional reconstruction in scenes with extreme depth variations remains challenging due to inconsistent supervisory signals between near-field and far-field regions. Existing methods fail to simultaneously address inaccurate depth estimation in distant areas and structural degradation in close-range regions. This paper proposes a novel computational framework that integrates depth-of-field supervision and multi-view consistency supervision to advance 3D Gaussian Splatting. Our approach comprises two core components: (1) Depth-of-field Supervision employs a scale-recovered monocular depth estimator (e.g., Metric3D) to generate depth priors, leverages defocus convolution to synthesize physically accurate defocused images, and enforces geometric consistency through a novel depth-of-field loss, thereby enhancing depth fidelity in both far-field and near-field regions; (2) Multi-View Consistency Supervision employing LoFTR-based semi-dense feature matching to minimize cross-view geometric errors and enforce depth consistency via least squares optimization of reliable matched points. By unifying defocus physics with multi-view geometric constraints, our method achieves superior depth fidelity, demonstrating a 0.8 dB PSNR improvement over the state-of-the-art method on the Waymo Open Dataset. This framework bridges physical imaging principles and learning-based depth regularization, offering a scalable solution for complex depth stratification in urban environments. [orcid=0000-0001-5378-6404] keywords 3D scene reconstruction ,Depth-of-field ,Differentiable rendering,Novel view synthesis, † † credit: Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft, Writing - Review & Editing, Visualization † † credit: Conceptualization, Methodology, Investigation, Resources, Writing - Original Draft, Writing - Review & Editing † † credit: Validation, Formal analysis, Writing - Review & Editing † † credit: Writing - Review & Editing † † credit: Supervision, Writing - Review & Editing, Funding acquisition † † corresponding: Corresponding author 1 Introduction Three-dimensional scene reconstruction from multi-view images remains a cornerstone capability for applications ranging from autonomous driving to immersive virtual reality. While Neural Radiance Fields (NeRF) ( 36 ) revolutionized photorealistic novel view synthesis, subsequent advances in 3D Gaussian Splatting (3DGS) ( 25 ) have achieved unprecedented real-time rendering speeds through differentiable Gaussian primitives. However, these methods face critical limitations when reconstructing scenes with substantial depth variations, as distant structures often exhibit positional inaccuracies due to insufficient supervision signals and overfitting to training viewpoints. Current approaches for large-scale scene reconstruction rely predominantly on multi-view stereo (MVS) techniques ( 14 ) or volumetric neural representations ( 2 ) . The former establishes geometric consistency through hand-crafted features ( 51 ) , while the latter optimizes implicit fields through photometric loss ( 45 ) . Recent extensions such as GaussianPro ( 7 ) attempt to mitigate extreme distant interference through sky segmentation masks, yet introduce new artifacts from imperfect matting and sparse depth supervision. This limitation stems from a fundamental challenge: when objects appear at varying distances across frames, conventional supervision struggles to resolve scale ambiguities, particularly for distant regions receiving insufficient pixel-level gradients. Our work addresses these limitations through two synergistic innovations. First, we leverage depth-of-field effects as implicit geometric supervision. By modeling the physical correlation between defocus blur and scene depth via adaptive kernel convolution, we derive gradient signals that guide Gaussians towards their geometrically consistent positions. Second, we introduce a hybrid depth estimation framework that integrates multi-view feature matching (LoFTR) ( 46 ) with monocular depth completion (Metric3D) ( 19 ) , resolving scale ambiguities while preserving structural details. This dual strategy effectively constrains Gaussian distributions across varying depth layers, particularly benefiting distant regions reconstruction where traditional methods fail. The technical contributions of this work are threefold: • A physics-aware defocus convolution model that translates optical principles into geometric constraints, using adaptive kernel designs (Gaussian, Polygonal, or SmoothStep) to adapt to different types of cameras and dynamic focus optimization to enhance depth consistency. • A multiscale depth alignment framework combining global monocular depth recovery with local grid-based correction, achieving view-consistent depth estimation without manual masking. • A gradient-aware density control mechanism that prioritizes structurally critical regions through depth-gradient statistics. Extensive validation on urban (Waymo) and unbounded (Mip-NeRF 360) scenes demonstrates the efficacy of our approach. Quantitative results show that our model achieves 35.17 PSNR on Waymo, outperforming SOTA methods. Qualitative analyzes reveal significant improvements in near region structure recovery, particularly for vehicles and buildings. These advancements establish new state-of-the-art performance for depth-aware scene reconstruction while preserving the computational efficiency central to 3DGS frameworks. 2 Related Work 2.1 Multi-View Stereo MVS represents a fundamental computer vision task that aims to reconstruct high-fidelity 3D models from a collection of calibrated images. The existing MVS methods can be broadly categorized into traditional geometry-based approaches and contemporary learning-based approaches. Traditional MVS approaches typically derive their camera parameters predominantly from Structure-from-Motion (SfM) methods ( 42 ; 40 ) or Simultaneous Localization and Mapping (SLAM) frameworks ( 38 ; 10 ) . Within this paradigm, seminal methods proposed by 5 , 14 , and 51 establish explicit pixel correspondences through hand-crafted features and rigorous geometric constraints. Learning-based approaches ( 24 ; 22 ; 55 ) have revolutionized MVS, initiated by the pioneering end-to-end architecture introduced by 52 . Contemporary methods ( 33 ; 12 ) leverage learned representations for robust depth regression, while advanced techniques such as cascade cost volumes ( 17 ) and feature matching networks ( 15 ) have substantially enhanced both performance and computational efficiency. 2.2 Neural Radiance Field A radiance field establishes a mapping from a 3D spatial coordinate (x,y,z) and viewing directions parameterized by polar angle \theta and azimuthal angle \phi to a nonnegative radiance value, characterizing the light-matter interaction within the environment ( 6 ) . NeRF ( 36 ) pioneered the paradigm of representing a scene as an emissive volumetric function, implemented via a position-encoded neural network that enables differentiable rendering through volumetric quadrature. Volumetric representations integrated with deep-learning techniques and volumetric ray-marching were initially proposed by 41 and 18 . Significant advancements in this domain include Instant-NGP ( 37 ) , which employs multi-resolution hash grids, and Plenoxels ( 13 ) , which utilizes sparse voxel grids for efficient optimization. Several approaches enhance rendering efficiency through scene reparameterization to generate more compact representations, notably Mip-NeRF 360 ( 2 ) , Zip-NeRF ( 3 ) , and F2-NeRF ( 45 ) . Recent research has increasingly focused on addressing defocus blur in neural rendering, with seminal works including RawNeRF ( 35 ) , AR-NeRF ( 23 ) , and NeRFocus ( 47 ) , all capable of synthesizing depth-of-field effects. Subsequently, Deblur-NeRF ( 32 ) mitigates image blurriness resulting from defocus by implementing a Deformable Sparse Kernel module, while DP-NeRF ( 27 ) specifically addresses geometric and appearance consistency challenges in defocused scenarios. Nevertheless, as noted by ( 48 ) , these approaches continue to encounter substantial limitations regarding computational efficiency and real-time rendering capabilities. 2.3 3D Gaussian Splatting Novel view synthesis has evolved significantly with NeRF ( 36 ) setting a milestone for photorealistic rendering. Building upon this foundation, 3DGS ( 25 ) introduces a paradigm shift that models scenes as a collection of 3D Gaussian primitives rendered via differentiable rasterization, simultaneously achieving high-quality reconstruction and real-time rendering capabilities. This approach extends traditional splatting-based rasterization ( 56 ) by optimizing Gaussian primitives with explicit geometry and appearance

Read the complete text