LiDAR-enhanced 3D Gaussian Splatting Mapping

Jian Shen, Huai YuJi WuWen Yang, Gui-Song Xia · 2025

Read LiDAR-enhanced 3D Gaussian Splatting Mapping by Jian Shen, Huai YuJi WuWen Yang, Gui-Song Xia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05425.txt.

Introduction

Jian Shen, Huai Yu, Ji Wu, Wen Yang, and Gui-Song Xia improve three-dimensional mapping by adding laser data in a 2025 study. A popular rendering method builds scenes from soft three-dimensional blobs fitted to photos alone. Photo-only starting points are sparse, and poses can drift, hurting both shape and appearance.

Their framework uses LiDAR point clouds to start the scene with denser, more reliable points. A joint loss over images and laser data refines poses and sensor alignment together. Depth maps drawn from laser scans add supervision during rendering. Experiments on public and self-collected data sets show gains over comparison methods.

Catalog record

CREATORJian Shen, Huai YuJi WuWen Yang, Gui-Song Xia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05425

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LiDAR-enhanced 3D Gaussian Splatting Mapping Jian Shen, Huai YuJi WuWen Yang, Gui-Song Xia Abstract This paper introduces LiGSM, a novel LiDAR-enhanced 3D Gaussian Splatting (3DGS) mapping framework that improves the accuracy and robustness of 3D scene mapping by integrating LiDAR data. LiGSM constructs joint loss from images and LiDAR point clouds to estimate the poses and optimize their extrinsic parameters, enabling dynamic adaptation to variations in sensor alignment. Furthermore, it leverages LiDAR point clouds to initialize 3DGS, providing a denser and more reliable starting points compared to sparse SfM points. In scene rendering, the framework augments standard image-based supervision with depth maps generated from LiDAR projections, ensuring an accurate scene representation in both geometry and photometry. Experiments on public and self-collected datasets demonstrate that LiGSM outperforms comparative methods in pose tracking and scene rendering. I Introduction 3D Gaussian Splatting (3DGS) is a technology used for 3D scene modeling and rendering, achieving an efficient rendering process by representing the scene as a set of learnable 3D Gaussian functions [ 1 ] . It has applications in various fields such as 3D mapping [ 2 ] and autonomous driving [ 3 ] . However, pure 3DGS relying solely on visual information struggles to accurately initialize the 3D space due to its dependence on sparse and potentially unreliable point clouds from Structure from Motion (SfM). LiDAR provides precise 3D information, unaffected by the lighting conditions and capable of capturing intricate environmental details over a much longer effective range compared to depth cameras, making it ideal for enhancing the geometric fidelity of reconstructed scenes. By incorporating LiDAR measurements, we can achieve more precise initialization of the 3DGS, thereby improving the overall quality and reliability of the reconstructed geometry. This fusion strategy also mitigates the dependency on sparse initialization typical in purely visual approaches, thus potentially increasing the robustness of the reconstruction process. Fig. 1 : LiGSM takes images and LiDAR point clouds as inputs, simultaneously optimizing the LiDAR-camera extrinsics and estimating the poses. The 3DGS reconstruction results is capable of rendering color images and depth maps. Integrating LiDAR into visual 3DGS brings advantages but also new challenges. The first challenge lies in how to effectively fuse LiDAR data within the 3DGS rendering pipeline. Traditional 3DGS methods rely heavily on image-based cues for both training and inference, but integrating LiDAR requires developing novel mechanisms to incorporate depth information seamlessly alongside color data. This necessitates modifications to the existing architecture and loss functions to ensure that the LiDAR data contributes meaningfully to the optimization process without overshadowing the visual cues that are essential for texturing and appearance synthesis. The second challenge involves ensuring and continuously maintaining accurate extrinsic calibration between the LiDAR and the camera. In real-world scenarios, even minor misalignments or variations in the relative positioning of sensors can lead to significant errors in the final mapping. Moreover, environmental changes or modifications in sensor mounting can cause alignment shifts over time, necessitating dynamic recalibration methods. Addressing this issue requires a robust system capable of detecting and correcting any drift or misalignment during operation. In this work, we propose a novel mapping framework that leverages LiDAR-enhanced 3DGS, designed to improve the accuracy and robustness of 3D scene mapping, as shown in Fig. 1 . We optimize the extrinsic parameters by constructing a joint observation of LiDAR point clouds and images, while simultaneously optimizing the pose. This joint optimization allows our system to adapt dynamically to variations in sensor alignment, ensuring that the reconstruction remains accurate even when the relative positions of the camera and LiDAR change. Furthermore, we leverage LiDAR point clouds for initializing the 3DGS, providing a denser and more reliable starting point than traditional sparse feature matches. For scene rendering, we augment the standard image-based supervision with depth maps generated from LiDAR projections. This dual-supervision strategy ensures that both the geometric and photometric aspects of the scene are accurately represented, leading to higher-quality reconstructions. The key contributions are summarized as: • We propose LiGSM, which leverages LiDAR to enhance the performance of visual 3D scene mapping. • We design a novel pose estimation framework fusing LiDAR and a monocular camera, which can optimize the camera-LiDAR extrinsic parameter without re-calibration when estimating poses. • We integrate the geometric information of LiDAR data into the initialization and training process of 3DGS, which reduces the dependence on the depth of visual reconstruction and improves the robustness of the reconstruction on image quality. • Experiments conducted on both public and self-collected datasets demonstrate the robustness and effectiveness of the proposed LiGSM on extrinsic-updating and dense mapping. II Related Work II-A Dense Scene Representation for SLAM Dense 3D scene representation is typically based on grids, points and neural networks. Grid-based methods [ 4 , 5 , 6 , 7 , 8 , 9 ] manage memory effectively by using dense grids, hierarchical octrees, and voxel hashing. Although they are simple and fast for finding neighborhoods and integrating context, they require predefined resolution, which can lead to inefficient memory usage in sparse areas and fail to capture details. Point-based methods [ 10 , 11 , 12 , 13 , 14 , 15 , 16 ] address the limitations of grids by varying density, focusing points on surfaces, avoiding modeling of empty space, but struggle to model continuous scenes. Neural methods [ 17 , 18 , 19 , 20 , 21 , 22 ] implicitly model continuous scenes through coordinate networks, capable of capturing high-quality maps and textures, but network training is time-consuming and not suitable for online reconstruction. Recently, new representation methods such as surfels [ 23 , 24 ] and neural planes [ 25 , 26 ] have been proposed, but they have limitations in modeling flexibility. NeRF [ 27 ] uses neural radiance fields to represent scenes, effectively modeling continuous scenes, and can render realistic images from neural radiance fields through volumetric rendering. However, training and rendering of neural radiance fields involve inevitable sampling of empty areas, which is computationally expensive, especially when dealing with large-scale scenes. Kerbl et al. proposed the use of 3DGS [ 1 ] , representing volumetric elements in the scene with three-dimensional Gaussian functions. Compared to NeRF, 3DGS has a significant advantage in rendering and training speed because it does not need to represent empty areas, and it allows for more direct scene editing and manipulation. However, pure visual 3DGS rely on the accuracy of sparse point clouds generated by SfM for initialization, which can affect the quality of the final 3D reconstruction. II-B Multimodal Dense 3D Reconstruction Recently, numerous SLAM studies have adopted neural radiance fields and 3D Gaussians as methods for 3D scene representation [ 19 , 28 , 20 , 29 , 30 , 31 , 32 , 33 , 2 ] . Initially, these works utilized images as inputs [ 30 ] , employing gradient descent algorithms to optimize both the scene and poses through differentiable rendering. However, supervision using images alone often yields subpar results. SLAM algorithms that use RGB-D inputs [ 20 , 29 ] , which incorporate depth supervision for rendering depth maps, can better leverage geometric information, leading to reconstructions that boast both superior visual appearance and robust structure. Yet, pose calculation based on gradient

Read the complete text