VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM
Anh Thuan Tran, Jana Košecká · 2026
Read VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM by Anh Thuan Tran, Jana Košecká (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-09673.txt.
Introduction
Anh Thuan Tran and Jana Kosecka introduce VarSplat, an uncertainty-aware 3D Gaussian Splatting system for RGB-D SLAM. Existing splatting SLAM handles measurement reliability only implicitly, so pose estimates and global alignment drift in low-texture regions or on tricky surfaces. VarSplat learns per-splat appearance variance and renders a per-pixel uncertainty map in one pass using the law of total variance and alpha compositing together. That map steers tracking, submap registration, and loop detection toward reliable regions, improving robustness on standard datasets.
Catalog record
| CREATOR | Anh Thuan Tran, Jana Košecká |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48547 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.09673
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM Anh Thuan Tran, Jana Košecká Abstract Simultaneous Localization and Mapping (SLAM) with 3D Gaussian Splatting (3DGS) enables fast, differentiable rendering and high-fidelity reconstruction across diverse real-world scenes. However, existing 3DGS-SLAM approaches handle measurement reliability implicitly, making pose estimation and global alignment susceptible to drift in low-texture regions, transparent surfaces, or areas with complex reflectance properties. To this end, we introduce VarSplat, an uncertainty-aware 3DGS-SLAM system that explicitly learns per-splat appearance variance. By using the law of total variance with alpha compositing, we then render differentiable per-pixel uncertainty map via efficient, single-pass rasterization. This map guides tracking, submap registration, and loop detection toward focusing on reliable regions and contributes to more stable optimization. Experimental results on Replica (synthetic) and TUM-RGBD, ScanNet, and ScanNet++ (real-world) show that VarSplat improves robustness and achieves competitive or superior tracking, mapping, and novel view synthesis rendering compared to existing studies for dense RGB-D SLAM. 1 Introduction Simultaneous Localization and Mapping (SLAM) 18 i s significant for robotics and AR/VR applications by giving the autonomous systems ability to actively explore unknown environments while incrementally building a map. Classical methods 27 ; 26 ; 44 based on voxels, TSDFs, surfels, or meshes can achieve real-time tracking and mapping but struggle to capture photorealistic appearance. To improve visual fidelity, Neural radiance fields (NeRF) 25 have been integrated into SLAM systems 39 ; 55 , though implicit volumetric rendering remains costly for real-time performance. 3D Gaussian Splatting (3DGS) 17 further addresses this bottleneck by rasterizing anisotropic Gaussians with alpha compositing, enabling fast, differentiable rendering. Recent 3DGS-SLAM systems 16 ; 48 ; 51 ; 54 adopt this explicit representation and jointly optimize camera poses and Gaussian parameters. Despite these advances, a key limitation exists: measurement reliability is rarely modeled explicitly . Uniform photometric weighting leaves pose estimation vulnerable in low-texture regions, around depth discontinuities, and on reflective surfaces. For safety-critical systems, explicitly quantifying uncertainty is important, providing confidence alongside rendering quality. Prior efforts address uncertainty primarily on the geometric side (e.g., depth variance or probabilistic filters) 11 ; 31 ; 42 or rely on pretrained predictors 53 . The appearance uncertainty that directly reflects instability in 3DGS rendering has not been treated as a first-class quantity in online dense SLAM. To address these challenges, we introduce VarSplat , an uncertainty-aware RGB-D SLAM system leveraging 3D Gaussian Splatting. VarSplat integrates explicit uncertainty into the map representation by learning per-splat appearance variance \sigma^{2} that is trained along with standard Spherical Harmonics (SH) coeffcients. Using the law of total variance, we derive propagation through 3DGS rasterizer to obtain differentiable per-pixel uncertainty map V . The system is trained end-to-end online, jointly optimizing poses, Gaussian parameters, and \sigma^{2} as the map grows. For downstream tasks, uncertainty is applied across all three stages. In tracking , the per-pixel uncertainty map V provides short-horizon reliability measure within each submap, improving frame-to-frame pose updates. In registration , the same per-pixel reliability supports mid-range alignment between overlapping submaps. For loop detection , per-splat variances \sigma^{2} modulate submap similarity and correct long-range drift. To the best of our knowledge, VarSplat is the first 3DGS–SLAM system to learn per-splat appearance variance and render per-pixel uncertainty in online setting. In summary, our contributions can be shown as follows: • We introduce VarSplat , an RGB-D 3DGS–SLAM system that learns per-splat appearance variance \sigma^{2} to render differentiable per-pixel uncertainty map V while maintaining single-pass rasterization efficiency. • We integrate uncertainty at both representation and renderer levels: poses, Gaussian parameters, and variance \sigma^{2} are optimized together in a fully online, end-to-end submap pipeline. • Extensive experiments on Replica, TUM-RGBD, ScanNet, ScanNet++ show improved robustness and competitive or superior tracking, reconstruction, and novel view synthesis rendering against 3DGS-SLAM baselines. 2 Related Work In this section, we briefly go through various approaches to dense SLAM that leverage radiance fields for 3D representations and uncertainty quantification. Dense Visual SLAM. Traditional dense SLAM, with DTAM 27 and KinectFusion 26 , achieved real-time tracking and mapping from RGB-D images using explicit volumetric or surface representations such as TSDFs 6 ; 43 , voxels 49 ; 13 and surfels 33 ; 44 . While effective for real-time pose estimation, these approaches struggle to provide texture-rich, high-fidelity reconstructions and synthesize novel views. iMap 39 integrated Neural Radiance Fields (NeRF) 25 into tracking and mapping, and Nice-SLAM 55 improved scalability with multi-resolution feature grids. Additional efficiency representations and global consistency concerns were addressed in 15 ; 41 ; 52 ; 23 ; 30 ; 46 . Recently, SplaTAM 16 used 3D Gaussians 17 for dense RGB-D SLAM with per-pixel photometric losses, and submap-based systems 24 ; 48 ; 51 balanced accuracy and efficiency through incremental mapping. VarSplat follows this general 3DGS-SLAM architecture but treats measurement reliability explicitly. Uncertainty Quantification for Radiance Fields. In Neural Radiance Fields, Stochastic NeRF 35 and Conditional-Flow NeRF 34 attempted to quantify uncertainty by reparametrizing NeRF with Bayesian model. However, these approaches focused on predictive uncertainty and do not capture parameter uncertainty grounded in observed information. Bayes’ Rays 9 introduced spatially parameterized perturbation field and interpolates uncertainty at NeRF query points, estimating it with Laplace approximation. Information-driven studies used uncertainty for decision making. FisherRF 14 modeled uncertainty with Fisher information for active view selection. Bayesian NeRF 21 modeled explicitly quantifies uncertainty in the geometric volume structure, particularly volume density and occupancy. ActiveNeRF 7 rendered per-pixel variance map using the neural network to guide model acquisition. Recent methods 1 ; 22 integrated stochasticity into 3D Gaussian Splatting 17 pipeline through variational inference and Monte Carlo sampling to approximate pixel-level variance. A probabilistic view also treats 3DGS optimization as SGLD sampling 19 . Sensitivity-based 10 approach related uncertainty to model sensitivity for pruning. Semantic 3DGS 45 estimates per-pixel variance while rasterizing semantic maps. VarSplat introduces a conceptually different method that learns per-splat appearance variance and leverage Gaussian rasterizer to render per-pixel uncertainty map. This approach allows us to maintain single-pass rasterization efficiency in online setting. Uncertainty in SLAM. Incorporating uncertainty in SLAM is important for robust localization and consistent maps under real-time constraints. REMODE 29 models per-pixel depth distributions, then uses their variance into mapping, while SVO 8 provides depth filters with explicit uncertainty updates. DeepFactors 4 presents a real-time probabilistic framework for dense monocular SLAM. Within neural SLAM, UncLe-SLAM 31 learns per-pixel depth uncertainty and reweights tracking and mapping. Uni-SLAM 42 defines predictive uncertainty for neural implicit fields and uses it to reweight pixel losses. CG-SLAM 11 builds 3D Gaussian field and models depth uncertainty to guide optimization and select informative Gaussians. For dynamic scenes, WildGS-SLAM 53 predicts an uncertainty map from DINOv2 features to filter moving distractors, and GLC-SLAM 47 uses uncertainty for keyframe selection to stabilize submap optimization. Rather than modeling geometric depth variance 11 ; 31 or relying on pretrained predictors 53 , we learn per-splat appearance variance and propagate it through alpha compositing and the law of total variance to obtain a differentiable per-pixel uncertainty map. In contrast to variance from ray-termination probabilities 42 , our variance is learned from the rasterizer and updated frame by frame. Figure 2 : VarSplat architecture. During mapping, each 3D Gaussian jointly learns position, appearance, and variance \sigma^{2} . The