StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors

Wenhao Yuan, Yiyuan Ge, Deli Cai · 2026

Read StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors by Wenhao Yuan, Yiyuan Ge, Deli Cai (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30545.txt.

Introduction

Wenhao Yuan, Yiyuan Ge, and Deli Cai rebuild three-dimensional scenes from very few photos by borrowing depth from stereo vision. Gaussian splatting renders new views quickly and beautifully. With sparse input views it overfits and warps the geometry. Monocular depth hints help but carry no absolute scale. Different views also disagree with each other under monocular hints. Stereo priors promise consistent, scaled depth instead.

Virtual stereo pairs are constructed during optimization. A foundation stereo model supplies depth for those pairs. That depth enforces absolute scale and two-view consistency. A gradient-aware opacity decay prunes redundant Gaussians dynamically. It penalizes them by their relative opacity gradients. Dense initialization from zero-shot multi-view depth anchors the rest. Primitives start near accurate surfaces. Four standard datasets host the experiments. Performance reaches the state of the art in sparse-view settings. No extra cost appears at inference time.

Catalog record

CREATORWenhao Yuan, Yiyuan Ge, Deli Cai
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30545

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

StereoGS: Sparse-View 3D Gaussian Splatting via Stereo Priors Wenhao Yuan, Yiyuan Ge, Deli Cai Abstract 3D Gaussian Splatting (3DGS) has achieved remarkable success in real-time novel view synthesis, yet it suffers from severe overfitting under sparse-view settings due to insufficient geometric constraints. While recent methods introduce monocular depth priors to mitigate this, they inherently struggle with scale ambiguity and cross-view inconsistency, leading to defective geometry. In this paper, we propose StereoGS, a novel sparse-view 3DGS framework that integrates stereo priors to establish reliable binocular consistency. Unlike scale-agnostic monocular constraints, StereoGS introduces a Stereo Depth Regularization by constructing virtual stereo pairs during optimization and leveraging a foundation stereo model to enforce absolute scale and binocular-consistent structures. To further suppress overfitting and eliminate redundant primitives, we design a Gradient-Aware Opacity Decay strategy that dynamically penalizes Gaussians based on their relative opacity gradient magnitudes. Combined with a Consistency-Aware Dense Initialization using zero-shot multi-view depth estimation, StereoGS effectively anchors primitives to accurate scene surfaces. Extensive experiments on LLFF, DTU, Mip-NeRF360, and Blender datasets demonstrate that StereoGS achieves state-of-the-art performance in sparse-view settings without incurring any additional inference overhead. Project Page: https://stringerywh00.github.io/StereoGS_project_page/ 1 Introduction Novel View Synthesis (NVS) [ 23 , 35 , 1 ] aims to synthesize realistic images from unseen viewpoints. While Neural Radiance Field (NeRF) [ 19 ] and 3D Gaussian Splatting (3DGS) [ 13 ] have revolutionized high-fidelity NVS, 3DGS has gained particular attention for its real-time rendering capabilities. However, vanilla 3DGS relies heavily on dense input views. Under sparse-view settings [ 23 , 35 , 50 , 7 ] , limited view overlap and insufficient appearance correspondences restrict sufficient geometric constraints, inevitably leading to severe overfitting. Consequently, achieving robust sparse-view 3DGS remains a critical yet challenging task for real-world applications. To mitigate this, existing methods for sparse-view 3DGS [ 4 , 17 , 50 ] attempt to introduce geometric priors by using monocular depth estimation for geometry regularization. While effective, we argue that relying solely on monocular cues suffers from two fundamental limitations. First, monocular depth inherently suffers from scale ambiguity, producing relative rather than absolute depth estimates. Since these relative depth maps lack a unified metric scale, enforcing them as regularization often conflicts with photometric consistency derived from the input images, thereby misguiding the optimization and leading to defective geometry. Second, monocular depth maps lack cross-view consistency because they are inferred independently for each view. Consequently, the same spatial point is frequently assigned inconsistent depth values across different viewpoints. When back-projected into 3D space, these view-wise inconsistencies lead to spatial misalignments, destabilizing the Gaussian optimization and causing severe artifacts. Recent studies [ 7 , 39 , 48 ] attempt to integrate cross-view consistency for Gaussian optimization. For instance, Binocular3DGS [ 7 ] applies photometric loss on binocular images via the rendered depth map. However, this indirect pixel-level constraint lacks direct gradient guidance to 3D Gaussian primitives, failing to anchor them accurately to scene surfaces. NexusGS [ 48 ] leverages the traditional epipolar geometry for initialization, but relying solely on initialization without continuous binocular consistency constraints during optimization still results in geometric degradation, especially in textureless or occluded regions. In this paper, we propose StereoGS , a sparse-view Gaussian Splatting framework that integrates stereo priors to establish reliable binocular consistency during Gaussian optimization. StereoGS is optimized via stereo priors during both initialization and optimization, compelling the 3D Gaussians to learn scale-accurate geometry. Specifically, unlike previous methods that rely on scale-agnostic monocular depth constraints, StereoGS regularizes the geometry by imposing a novel stereo depth constraint during optimization. For each training view, we construct a virtual right camera to form a stereo pair and render the right-view image, and depth maps for training views. By feeding the RGB pair of the ground-truth left-view and the rendered right-view into a foundation stereo model [ 37 ] , we obtain a highly accurate reference depth map that serves as an external stereo depth prior. We then perform supervision between the rendered and reference depth maps. This process effectively integrates a scale-aware stereo depth prior into the optimization, compelling the 3D Gaussians to learn absolute scale and binocular-consistent structures. To suppress overfitting and eliminate redundant primitives, we introduce a general gradient-aware opacity decay strategy. Our core insight is that the opacity gradient of a Gaussian inherently reflects its necessity for scene reconstruction. Therefore, we dynamically penalize Gaussians based on their gradient magnitudes: primitives with smaller gradients are deemed less critical for rendering and receive a stronger decay penalty, while those with larger update demands are preserved. These mechanisms, alongside the stereo priors, serve as structural signals that backpropagate through the rendering pipeline to Gaussian primitives. As a result, StereoGS learns highly accurate, binocular-consistent geometric representations, directly mitigating the severe overfitting inherent in sparse-view settings. Additionally, for Gaussian initialization, we employ a pre-trained multi-view depth estimator [ 9 ] combined with strict consistency filtering to estimate the multi-view-consistent depth maps for each view. By further applying cross-view reprojection errors, we eliminate outliers in the depth maps and fuse them into a dense point cloud, which serves as a robust initialization. Notably, because the approaches in StereoGS are training-time strategies, employing StereoGS incurs no additional computational overhead during inference compared to the vanilla 3DGS. Extensive experiments on LLFF, DTU, Mip-NeRF360, and Blender datasets demonstrate that StereoGS achieves state-of-the-art performance in sparse-view settings, effectively balancing geometric accuracy with rendering quality. Our main contributions are summarized as follows: • We introduce a Stereo Depth Regularization that utilizes stereo priors to enforce absolute scale constraints during optimization, fundamentally resolving the scale ambiguity inherent in monocular depth priors. • We design Gradient-Aware Opacity Decay to dynamically adjust opacity during training based on the gradient, effectively eliminating redundant primitives and suppressing overfitting to ensure a compact and robust scene representation. • Extensive experiments on LLFF, DTU, Mip-NeRF360, and Blender datasets demonstrate that our method achieves state-of-the-art results compared to existing sparse-view methods. 2 Related Works 2.1 Radiance Fields Radiance Fields are employed for reconstructing 3D scenes and synthesizing novel views. Neural Radiance Fields (NeRFs) [ 19 ] have seen significant advances that learn neural volumetric representations of 3D scenes and render images via volume rendering. While NeRF enables high-quality reconstruction from images, it requires high computational costs and suffers from slow training and inference speeds. Subsequent research has focused on addressing these bottlenecks by improving the rendering quality [ 1 , 2 ] , computational efficiency [ 21 , 33 ] , frequency regularization [ 41 ] . A significant breakthrough in achieving real-time rendering is 3D Gaussian Splatting (3DGS) [ 13 ] , which represents scenes using a set of explicit 3D Gaussian primitives. 3DGS reconstructs high-quality scenes rapidly by utilizing differentiable splatting, excelling particularly in handling high-frequency details and offering intuitive interpretability. Building on its significant performance, many follow-up works focus on improving the rendering details [ 44 , 47 , 14 , 40 ] , compressing the Gaussians and accelerating rendering [ 16 , 22 ] . 2.2 Novel View Synthesis from Sparse View NeRF-based and 3DGS-based methods have demonstrated impressive rendering performance, whereas in real-world scenarios, only a small number of images are available (i.e., sparse input views). Directly applying the methods in a dense-view setting leads to

Read the complete text