GS 3 LAM: Gaussian Semantic Splatting SLAM Conference: Proceedings of the 32nd ACM International Conference on Multimedia; October 28-November 1, 2024; Melbourne, VIC, Australia Proceedings of the 32nd ACM International Conference on Multimedia (MM ’24), October 28-November 1, 2024, Melbourne, VIC, Australia DOI: 10.1145/3664647.3680739 ISBN: 979-8-4007-0686-8/24/10 CCS: Computing methodologies Reconstruction

Linfei Li, Lin Zhang, Zhong Wang, Ying Shen · 2026

Read GS 3 LAM: Gaussian Semantic Splatting SLAM Conference: Proceedings of the 32nd ACM International Conference on Multimedia; October 28-November 1, 2024; Melbourne, VIC, Australia Proceedings of the 32nd ACM International Conference on Multimedia (MM ’24), October 28-November 1, 2024, Melbourne, VIC, Australia DOI: 10.1145/3664647.3680739 ISBN: 979-8-4007-0686-8/24/10 CCS: Computing methodologies Reconstruction by Linfei Li, Lin Zhang, Zhong Wang, Ying Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-27781.txt.

Introduction

Li and co-authors build GS3LAM for dense semantic mapping in real time. The system fuses color, depth, and semantic data streams. Scenes become a field of semantic Gaussians. Camera poses and the field train jointly under mixed error terms. A depth-adaptive scale rule fixes misalignments between cues. Past explicit maps were coarse and could not fill gaps. Implicit maps were too slow for live use. Gaussian splatting sidesteps both flaws.

Catalog record

CREATORLinfei Li, Lin Zhang, Zhong Wang, Ying Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64116
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.27781

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GS 3 LAM: Gaussian Semantic Splatting SLAM Conference: Proceedings of the 32nd ACM International Conference on Multimedia; October 28-November 1, 2024; Melbourne, VIC, Australia Proceedings of the 32nd ACM International Conference on Multimedia (MM ’24), October 28-November 1, 2024, Melbourne, VIC, Australia DOI: 10.1145/3664647.3680739 ISBN: 979-8-4007-0686-8/24/10 CCS: Computing methodologies Reconstruction Linfei Li, Lin Zhang, Zhong Wang, Ying Shen Abstract . Recently, the multi-modal fusion of RGB, depth, and semantics has shown great potential in the domain of dense Simultaneous Localization and Mapping (SLAM), as known as dense semantic SLAM. Yet a prerequisite for generating consistent and continuous semantic maps is the availability of dense, efficient, and scalable scene representations. To date, existing semantic SLAM systems based on explicit scene representations (points/meshes/surfels) are limited by their resolutions and inabilities to predict unknown areas, thus failing to generate dense maps. Contrarily, a few implicit scene representations (Neural Radiance Fields) to deal with these problems rely on time-consuming ray tracing-based volume rendering technique, which cannot meet the real-time rendering requirements of SLAM. Fortunately, the Gaussian Splatting scene representation has recently emerged, which inherits the efficiency and scalability of point/surfel representations while smoothly represents geometric structures in a continuous manner, showing promise in addressing the aforementioned challenges. To this end, we propose GS 3 LAM , a G aussian S emantic S platting SLAM framework, which takes multimodal data as input and can render consistent, continuous dense semantic maps in real-time. To fuse multimodal data, GS 3 LAM models the scene as a Semantic Gaussian Field (SG-Field), and jointly optimizes camera poses and the field by establishing error constraints between observed and predicted data. Furthermore, a Depth-adaptive Scale Regularization (DSR) scheme is proposed to tackle the problem of misalignment between scale-invariant Gaussians and geometric surfaces within the SG-Field. To mitigate the forgetting phenomenon, we propose an effective Random Sampling-based Keyframe Mapping (RSKM) strategy, which exhibits notable superiority over local covisibility optimization strategies commonly utilized in 3DGS-based SLAM systems. Extensive experiments conducted on the benchmark datasets reveal that compared with state-of-the-art competitors, GS 3 LAM demonstrates increased tracking robustness, superior real-time rendering quality, and enhanced semantic reconstruction precision. To make the results reproducible, the source code is available at https://github.com/lif314/GS3LAM . Figure 1 . Our proposed GS 3 LAM utilizes the 3D semantic Gaussian representation and the differentiable splatting rasterization pipeline, and jointly optimizes camera poses and field for appearance, geometry and semantics, achieving robust tracking, real-time high-quality rendering, and precise 3D semantic reconstruction. 1. Introduction By integrating semantic understanding into map, semantic Simultaneous Localization and Mapping (SLAM) achieves simultaneous estimation of camera poses while constructing maps that maintain consistency across geometry, appearance, and semantics. In comparison to conventional SLAM techniques, it excels in the identification, classification, and correlation of entities within scenes. Nowdays, semantic SLAM systems have been applied in various domains, such as robotics ( 18 ; 1 ) and autonomous driving ( 28 ; 15 ; 1 ) . To date, existing semantic SLAM systems based on explicit scene representations often resort to points/surfels ( 20 ; 32 ; 36 ; 38 ) , grids ( 21 ) , or voxels ( 8 ; 16 ; 22 ) to construct maps. Although these representations offer advantages in geometry, storage, computational efficiency, and scalability, they face challenges in predicting unknown regions and are constrained by limited resolutions, thus being unable to generate dense semantic maps. Contrarily, recent emerging neural rendering techniques based on implicit scene representations, such as Neural Radiance Fields (NeRF) ( 19 ) , have shown potentials to deal with these challenges. NeRF portrays scenes as continuous implicit volume functions, enabling realistic novel view synthesis with minimal storage requirements. Based on it, several studies ( 46 ; 4 ) incorporate additional MLP channels to encode and decode semantic labels, while jointly optimizing camera poses and semantic scenes. However, due to the computationally expensive ray tracing-based volume rendering technique of NeRF, these methods fail to meet the real-time demands of SLAM. (a) w/ LCKM (b) w/ Our RSKM Figure 2 . Illustration of optimization bias on Replica “Office 3”. Fortunately, we observe the emergence of 3D Gaussian Splatting (3DGS) ( 10 ) , which demonstrates exceptional capabilities in dense 3D reconstruction. This method represents the scene as dense Gaussian clouds and achieves efficient rendering through tile-based rasterization. We show that 3DGS has great potential in addressing the aforementioned challenges. As a semantic SLAM scene representation, it inherits the efficiency, locality, and modifiability of point/surfel representations while smoothly and differentiably representing the geometric structure in a continuous manner, enabling the reconstruction of rich and complex details in dense maps. To further improve the capabilities of semantic SLAM in tracking, rendering, and semantic reconstruction, it is a natural idea to extend 3DGS as a semantic scene representation, but surprisingly such a simple idea has seldom been explored in existing literature. In this work, based on the above-mentioned findings, we propose a dense semantic SLAM framework, GS 3 LAM ( G aussian S emantic S platting SLAM ), to fully leverage the advantages of 3DGS. However, the effective embedding and real-time optimization of high-dimensional semantic categories pose profound challenges for GS 3 LAM. To deal with these issues, GS 3 LAM models the scene as a Semantic Gaussian Field (SG-Field), wherein semantic categories are represented as low-dimensional implicit features. By means of a simple decoder, GS 3 LAM efficiently transforms these features into semantic categories, facilitating the conversion between 3D implicit features and 2D semantic labels. Furthermore, within the SG-Field, irregular Gaussian scales hinder the accurate representation of geometric surfaces, making it unacceptable for pixel-level semantic reconstruction. To address this issue, we propose a Depth-adaptive Scale Regularization (DSR) strategy. This strategy constrains scales within a depth-dependent range, indirectly aligning Gaussians with geometric surfaces, thus can effectively reduce blurring on object surfaces and enhance both tracking robustness and semantic reconstruction accuracy. Finally, to address the forgetting phenomenon in GS 3 LAM, we propose a Random Sampling-based Keyframe Mapping (RSKM) strategy, which proves to be more effective than the Local Covisibility Keyframe Mapping (LCKM) strategy commonly adopted in 3DGS-based SLAM systems. Our observation suggests that the latter method introduces a considerable bias during the optimization of the Gaussian field, thereby leading to poor global map consistency. In particular, as depicted in Fig. 2(a) , frames with dense co-observations (dense camera trajectories) and increased optimization iterations (large point radii) exhibit lower PSNR values (darker color), suggesting challenges in achieving convergence of the Gaussian field under the LCKM strategy. Conversely, as shown in Fig. 2(b) , our proposed RSKM strategy not only enhances the rendering quality of the global map (higher mean PSNR, \mu_{PSNR} ) but also ensures high consistency among all perspectives (smaller PSNR variance, \sigma_{PSNR} ), effectively reducing the optimization bias. Our contributions are summarized as follows: (1) As depicted in Fig. 1 , GS 3 LAM is a G aussian S platting S emantic SLAM framework, which models the scene as a Semantic Gaussian Field (SG-Field) to efficiently facilitate the conversion between 3D semantic features and 2D labels. By the joint optimization of camera poses and field for appearance, geometry, and semantics, it achieves robust tracking, real-time high-quality rendering, and precise semantic reconstruction. (2) A Depth-adaptive

Read the complete text