VCS-SLAM: Geometry-Validated Semantic Evidence Fusion for 3D Gaussian SLAM

Raman Jha, Shuaihang Yuan, Yi Fang · 2026

Read VCS-SLAM: Geometry-Validated Semantic Evidence Fusion for 3D Gaussian SLAM by Raman Jha, Shuaihang Yuan, Yi Fang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29494.txt.

Introduction

Jha, Yuan, and Fang improve 3D Gaussian SLAM, a system that tracks a camera while building a labelled three-dimensional map. SLAM, short for simultaneous localization and mapping, underpins robot navigation. Such systems borrow labels from two-dimensional models, yet those labels are not equally trustworthy. Occlusions, weak boundaries, and vague ray paths can bake lasting errors into the map.

Their system, VCS-SLAM, checks each label for geometric reliability before fusing it. Tests cover visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty. A reliability-aware objective then suppresses occluded updates and reduces label bleeding, where a label spills onto the wrong surface. Vague regions receive labels later, once evidence firms up. On the Replica dataset, consistency, boundary preservation, and reconstruction quality all improved. On ScanNet, with real RGB-D input, tracking stayed competitive.

Catalog record

CREATORRaman Jha, Shuaihang Yuan, Yi Fang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29494

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

VCS-SLAM: Geometry-Validated Semantic Evidence Fusion for 3D Gaussian SLAM Raman Jha, Shuaihang Yuan, Yi Fang Abstract Visual SLAM performance often deteriorates in complex real-world applications. Semantic 3D Gaussian SLAM commonly fuses 2D semantic priors into a persistent 3D map using uniform optimization weights. However, such priors are not equally reliable in online mapping: occlusions, unsupported semantic boundaries, and ambiguous ray geometry can introduce persistent semantic artifacts into the global Gaussian map. We propose VCS-SLAM, a geometry-validated semantic evidence fusion framework for RGB-D 3D Gaussian SLAM. Instead of treating all semantic observations as uniformly valid supervision, VCS-SLAM evaluates their geometric reliability through visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty. The resulting reliability-aware objective suppresses occluded semantic updates, reduces unsupported semantic bleeding, and delays premature label assignment in ambiguous regions. Experiments on Replica demonstrate improved semantic consistency, boundary preservation, and reconstruction quality. Results on ScanNet further show that VCS-SLAM maintains competitive tracking performance under real RGB-D inputs. I INTRODUCTION Visual SLAM fundamentally challenges robotics by requiring simultaneous camera pose estimation and incremental environment mapping [ 1 ] . While traditional frameworks perform robustly in static settings, their performance deteriorates during complex real-world applications. Although multi-modal sensor fusion addresses visual degradation in extreme conditions [ 2 ] , ensuring spatial and geometric consistency of semantic representations in 3D environments remains a critical bottleneck. Integrating visibility consistency and surface coupling improves structural mapping and object association, providing richer representations for advanced deployments. Fig. 1: Overview of VCS-SLAM. Top: High-fidelity global semantic reconstruction. Bottom: Effectiveness of our modules against 3DGS baselines. (1) Overall : Enhanced global consistency. (2) VCSU : Mitigates occlusion artifacts via a depth-gated mask. (3) SCEA : Reduces unsupported semantic bleeding (4) CAUW : Down-weights ambiguous observations via conflict-aware uncertainty mapping. Recent breakthroughs in Neural Radiance Fields (NeRF) [ 3 ] and 3D Gaussian Splatting (3DGS) [ 4 ] prompted their integration into visual SLAM, mitigating surface continuity limitations inherent in discrete mapping. Notably NeRF-based architectures (e.g. iMAP [ 5 ] NICE-SLAM [ 6 ] ) and 3DGS frameworks (e.g. MonoGS [ 7 ] SplaTAM [ 8 ] SGS-SLAM [ 9 ] ) achieve state-of-the-art performance producing consistent geometric and photorealistic maps. Semantic mapping increasingly relies on 2D data-driven models. Lacking inherent spatial and temporal continuity, these models introduce severe semantic inconsistencies during online mapping. Accumulating spatial discrepancies significantly degrades semantic scene reconstruction accuracy. While recent frameworks [ 10 , 11 , 9 ] successfully integrate semantic features, they fail to adequately mitigate the propagation of these mapping errors. By overlooking visibility consistency, principled uncertainty weighting, and explicit coupling of semantic boundaries to geometric surfaces, current methods exhibit limited robustness in real-world applications. Existing semantic 3D Gaussian SLAM methods treat per-frame labels as uniformly reliable supervision, so geometrically inconsistent priors from occluded boundaries, smooth-surface bleeding, or ambiguous geometry are repeatedly fused into the map and become persistent artifacts. VCS-SLAM instead validates each observation’s geometric reliability before fusion, preventing inconsistent updates from accumulating in the Gaussian map. To address these critical issues, this paper introduces a novel semantic SLAM framework featuring the following core contributions: • We formulate online semantic 3D Gaussian SLAM as a geometry-validated semantic evidence fusion problem, where 2D semantic priors are assimilated into the Gaussian map according to their geometric reliability rather than treated as uniformly valid supervision. • We instantiate this reliability model using three complementary geometric cues: visibility consistency, surface-supported boundary evidence, and ray-level conflict uncertainty. These cues jointly suppress occluded semantic updates, unsupported semantic bleeding, and premature label assignment in ambiguous regions. • We integrate the proposed reliability model into a differentiable semantic Gaussian optimization objective and validate its effect through targeted analyses on depth-inconsistent pixels, semantic boundary regions, and uncertainty-error calibration. II RELATED WORKS II-A Traditional Semantic SLAM-based methods Conventional semantic SLAM frameworks struggle to seamlessly integrate semantic information into mapping processes. Historically, these methods relied on discrete or implicit geometries like point clouds, voxels, and signed distance fields (SDFs) [ 12 , 13 , 14 ] . Point clouds and voxel grids are highly vulnerable to sparsity and degradation during rapid camera motion. Conversely, SDFs offer continuous surfaces but incur severe computational overhead, restricting real-time scalability. Consequently, a critical need remains for efficient continuous mapping methods providing robust semantic-geometric representations. II-B SLAM with Neural Radiance Fields The remarkable performance of Neural Radiance Fields (NeRF) in high-fidelity novel view synthesis [ 3 ] motivated their integration into SLAM frameworks [ 5 , 6 ] . For instance, SNI-SLAM [ 15 ] employs hierarchical semantic encoding and cross-attention mechanisms fusing appearance, geometric, and semantic features. However, the computational overhead inherent in NeRF volume rendering makes it challenging for dense SLAM methods to simultaneously model and optimize environmental semantic maps in real time [ 16 , 17 ] . Additionally, the implicit nature of these representations fundamentally limits overall NeRF-based system performance [ 18 ] . II-C SLAM with Gaussian Splatting 3D Gaussian Splatting (3DGS) has emerged as a promising representation utilizing 3D Gaussians parameterized by position anisotropic covariance opacity and color [ 4 ] . Existing 3DGS methods predominantly focus on RGB reconstruction, refining camera pose estimation via differentiable rendering with less attention devoted to semantic reconstruction [ 7 , 8 , 19 , 20 ] . Recent approaches typically integrate semantics by encoding ground-truth labels as an auxiliary color channel within Gaussian parameters [ 9 , 10 , 21 , 11 ] . Because these methods lack explicit modeling of semantic reliability, there remains substantial potential to improve semantic-geometric fusion during online mapping. While recent works such as NEDS-SLAM [ 22 ] investigate consistency, they overlook surface coupling and conflict-awareness weighting, prioritizing representation over observation reliability, the gap VCS-SLAM targets. III METHODOLOGY Fig. 2: System architecture of VCS-SLAM. Left (Input): Processes RGB images, depth maps, and Per-frame Semantic Label. The VCSU block evaluates spatial and depth features to generate a depth-gated visibility mask, filtering occluded or inconsistent observations. Middle (Optimization): 3D Semantic Gaussians and camera tracking are iteratively refined via joint multi-channel optimization. This integrates the SCEA constraint, aligning semantic boundaries with physical geometric edges, and the CAUW module, dynamically down-weighting ambiguous updates using intermediate depth variance. Right (Results): The optimized Gaussians enable high-fidelity novel view synthesis and globally consistent dense semantic mapping. III-A Semantic Scene Representation using Gaussian Splatting Our framework extends 3D Gaussian Splatting [ 4 ] by integrating semantic information. The environment is modeled as 3D Gaussians characterized by position \mu , opacity \sigma , and geometric scale. For dense semantic mapping, we augment each Gaussian with channels encoding visual appearance c_{i} and a C -dimensional semantic logit vector s_{i} where C is the number of semantic classes. Per-frame semantic labels are converted into one-hot class targets for cross-entropy supervision. During rendering, semantic logits are alpha-composited and supervised using pixel-wise cross-entropy. To optimize parameters against 2D observations, 3D Gaussians are differentiably projected onto the image plane. Overlapping Gaussians are aggregated via depth-sorted front-to-back volume rendering. Let \alpha_{i} denote the evaluated 2D influence of a Gaussian including its occlusion term. The semantic map S_{p} is rendered by blending semantic vectors of contributing Gaussians: S_{p}=\sum_{i=1}^{n}s_{i}\alpha_{i}\prod_{j=1}^{i-1}(1-\alpha_{j}) Similarly expected depth D_{p} is synthesized by accumulating depth d_{i} of each Gaussian center in camera coordinates:

Read the complete text