Stereo-LiDAR Fusion by Semi-Global Matching with Discrete Disparity-Matching Cost and Semidensification

Yasuhiro Yao, Ryoichi Ishikawa, and Takeshi Oishi · 2025

Read Stereo-LiDAR Fusion by Semi-Global Matching with Discrete Disparity-Matching Cost and Semidensification by Yasuhiro Yao, Ryoichi Ishikawa, and Takeshi Oishi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05148.txt.

Introduction

Yasuhiro Yao, Ryoichi Ishikawa, and Takeshi Oishi fuse laser ranging with stereo cameras for real-time depth sensing, in a 2025 robotics paper. Lasers give sparse but exact distances, while stereo images are dense but error-prone. Their method runs a classic stereo matcher with a new cost term seeded by laser points. The laser data are first spread into a semi-dense map, and a consistency check rejects clashes between the two sources.

On the KITTI driving benchmark, the method reached a 2.79 percent error rate, beating the prior real-time best of 3.05 percent. Tests varied laser density, weather, and indoor scenes to show flexibility. Every step runs in parallel on a graphics processor, and no learning is involved. The authors pitch it for robots that need depth now, not after training.

Catalog record

CREATORYasuhiro Yao, Ryoichi Ishikawa, and Takeshi Oishi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-05148

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Stereo-LiDAR Fusion by Semi-Global Matching with Discrete Disparity-Matching Cost and Semidensification Yasuhiro Yao, Ryoichi Ishikawa, and Takeshi Oishi Abstract We present a real-time, non-learning depth estimation method that fuses Light Detection and Ranging (LiDAR) data with stereo camera input. Our approach comprises three key techniques: Semi-Global Matching (SGM) stereo with Discrete Disparity-matching Cost (DDC), semidensification of LiDAR disparity, and a consistency check that combines stereo images and LiDAR data. Each of these components is designed for parallelization on a GPU to realize real-time performance. When it was evaluated on the KITTI dataset, the proposed method achieved an error rate of 2.79%, outperforming the previous state-of-the-art real-time stereo-LiDAR fusion method, which had an error rate of 3.05%. Furthermore, we tested the proposed method in various scenarios, including different LiDAR point densities, varying weather conditions, and indoor environments, to demonstrate its high adaptability. We believe that the real-time and non-learning nature of our method makes it highly practical for applications in robotics and automation. I Introduction Real-time depth estimation is crucial for a wide range of robotics and automation applications. Depth data are needed not only by autonomous vehicles but also by various mobile systems and robots to understand and navigate their environment. Standard methods for depth measurements include triangulation and time of flight (ToF). The widely used devices for these methods are stereo cameras, which rely on triangulation, and Light Detection And Ranging (LiDAR), which operates using ToF principles. Both stereo cameras and LiDAR systems have distinct advantages and limitations. Stereo cameras deliver depth information with a high resolution that is equivalent to the resolution of the input images. However, they perform poorly on untextured surfaces, repetitive patterns, and low-light environments due to challenges in finding correspondences. In contrast, LiDAR provides more precise depth measurements and is robust against variations in lighting and surface texture. However, LiDAR data are sparse because LiDAR captures depth information only at specific points where the laser beam intersects the target scene. Sensor fusion overcomes these drawbacks of sensor systems. In particular, a stereo-LiDAR fusion system can obtain a highly accurate, high-resolution depth map without being affected by the environment or by scenes [ 1 ] . As with other research topics, sensor fusion systems employ a learning-based [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 ] or non-learning-based [ 11 , 12 , 1 , 13 ] strategy. For stereo-LiDAR fusion, the performance of learning-based methods can be domain-dependent, as training is conducted on a specific dataset. Meanwhile, non-learning methods have the advantage of being less dependent on specific datasets and domains. However, the previous state-of-the-art real-time non-learning methods are not robust due to outlier-sensitive costs and direct use of LiDAR disparities, including misprojections [ 13 ] . Therefore, we propose a stereo-LiDAR fusion method, particularly a non-learning approach that operates in real time and achieves an accuracy comparable to that of learning-based methods. Fig. 1 is an overview of the proposed method. We employ Semi-Global Matching (SGM) [ 14 ] as the base stereo algorithm. The primary reason for the suboptimal accuracy of stereo-LiDAR fusion is that the integration between the stereo camera and LiDAR is insufficient. The proposed method addresses this issue by introducing the following three key approaches: • Discrete Disparity-matching Cost (DDC) discretely evaluates sparse disparities in the SGM framework. • Semidensification partially densifies sparse disparities to provide prior information to SGM using DDC. • Stereo-LiDAR consistency check ensures consistency in the disparity estimation by leveraging three views from the stereo cameras and LiDAR. In addition, we demonstrate that the proposed method surpasses previous state-of-the-art (SOTA) real-time stereo-LiDAR fusion techniques and exhibits strong adaptability across various domains. Sec. II reviews related works to position the proposed methodology within the existing literature. In Sec. III , we provide a brief overview of stereo SGM to facilitate a better understanding of the proposed approach and the variable notation. Secs. IV-A , IV-B , and IV-C describe DDC, semidensification, and the stereo-LiDAR consistency check, respectively. We evaluate the performance of the proposed method in Sec. V and present the conclusions in Sec. VI . Fig. 1 : Flow chart of the proposed method. The semidensification process takes stereo images and the sparse disparity map ( \bar{D} ) and outputs the semidense disparity map ( \hat{D} ). SGM with DDC takes stereo images with either a semidense disparity map ( \hat{D} ) or a sparse disparity map ( \bar{D} ) and outputs a dense disparity map. The stereo-LiDAR consistency check annotates invalid disparities based on the consistency of the three views to obtain a consistent dense disparity map ( D^{c*} ). II Related Work We consider the stereo system to be a parallel dense stereo setup, in which disparity maps are generated from a pair of images captured by two cameras aligned parallel to their image planes. Although modern deep learning methods estimate relative depth from a monocular image [ 15 ] , stereo images are still required to obtain depth at the real scale, which is our subject matter. In the following sections, we present a brief overview of related work on parallel dense stereo and stereo-LiDAR fusion methods. II-A Parallel dense stereo The stereo system estimates disparities by analyzing the local similarity between two images along their epipolar lines. A widely used method for finding the optimal solution is the energy minimization approach [ 16 , 17 , 14 ] , which includes pixel-matching cost and smoothness terms in 2D space. In most cases, this minimization problem is considered NP-hard [ 16 ] . Although several methods, such as graph cuts [ 16 ] and belief propagation [ 17 ] , have been proposed to solve this problem, these methods are computationally expensive. In contrast, SGM [ 14 ] reduces computational costs by approximating the 2D smoothness constraint using a combination of multiple one-dimensional (1D) constraints. Currently, SGM is one of the most widely used stereo-matching methods due to its high performance. Several variants of SGM have also been developed [ 18 , 19 , 20 ] . Learning-based dense stereo techniques have recently been introduced [ 21 , 22 ] ; however, training a model to handle all potential and unforeseen scenarios remains a challenge. For this reason, we consider SGM to be a suitable foundational algorithm for stereo matching. II-B Stereo-LiDAR fusion Non-learning stereo-LiDAR fusion has evolved and improved over the years. Badino et al. utilized LiDAR data to narrow the stereo matching search space and introduced predefined paths for dynamic programing [ 11 ] . Maddern et al. proposed a probabilistic model to fuse LiDAR and stereo disparities by combining the priors of individual sensors [ 12 ] . Yao et al. proposed a method for selecting, using belief propagation, appropriate depth values from LiDAR projections in the surrounding area [ 1 ] and then smoothing using total generalized variation [ 23 ] . Forkel et al. incorporated a LiDAR-based matching cost into SGM stereo to determine whether an estimated depth was similar to the LiDAR measurement of the depth [ 13 ] . Many recent studies have adopted learning-based approaches and have led to significant improvements in accuracy. Park

Read the complete text