Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving

Geonho Bang, Geunju Baek, Dongyoung Lee, Wonjun Jeong, Jun Won Choi · 2026

Read Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving by Geonho Bang, Geunju Baek, Dongyoung Lee, Wonjun Jeong, Jun Won Choi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-31096.txt.

Introduction

Bang and four colleagues present Horizon3D, a system for long-range 3D object detection in self-driving cars. At highway speeds, a car must spot distant objects early. Radar-camera fusion methods struggle at long range, though. Distant objects return only a few radar points and cover only a few image pixels. Bird's-eye-view methods keep scene context but lose fine detail. Query-based methods keep detail but lose context.

Horizon3D mixes both views. It places Gaussian primitives at object keypoints drawn from radar and camera features. A fusion step refines those primitives and splats them onto the bird's-eye-view plane. A dual-path temporal fusion then tracks scene context and per-object motion across frames. On the TruckScenes benchmark, the system beat the previous best method by 3.0 points of NDS and 1.6 points of mAP. It also kept a sparse representation and competitive speed.

Catalog record

CREATORGeonho Bang, Geunju Baek, Dongyoung Lee, Wonjun Jeong, Jun Won Choi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.31096

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Horizon3D: Sparse Radar-Camera Fusion for Long-Range 3D Perception in Autonomous Driving Geonho Bang, Geunju Baek, Dongyoung Lee, Wonjun Jeong, Jun Won Choi Abstract Long-range 3D object detection is critical for safe autonomous driving at highway speeds, yet existing radar-camera fusion methods face notable limitations at extended ranges. BEV-based approaches effectively encode scene-level context but incur rapidly growing computational cost and struggle to preserve fine-grained object-level detail, while query-based methods provide efficient object-centric encoding but lack sufficient scene-level context. Temporal fusion introduces additional challenges: distant objects produce only a few radar returns and occupy only a few image pixels, requiring scene-level accumulation, while high-speed motion causes large inter-frame displacements that require object-level motion modeling. BEV-based aggregation alleviates sparsity through multi-frame accumulation but is less suited to individual object motion, whereas query-based modeling captures object-level motion but provides limited scene-level temporal context. In this paper, we propose Horizon3D, a sparse radar-camera fusion framework for long-range 3D object detection that jointly captures object-level detail and scene-level context in both spatial and temporal dimensions through a hybrid representation that combines Gaussian primitives with sparse BEV features. Horizon3D first employs Keypoint-Guided Gaussian Initialization (KGGI) to initialize Gaussian primitives at object keypoints estimated from radar and camera features. Object-Centric Sparse Fusion (OCSF) aggregates cross-modal features around these primitives and splats the refined Gaussians onto the BEV plane, where they are fused with sparse radar BEV features to combine object-level detail with scene-level context. Finally, Dual-Path Temporal Fusion (DPTF) aggregates temporal cues through a BEV path for multi-frame feature accumulation and a Gaussian path for propagating primitives across frames to encode per-object motion. Extensive evaluations on TruckScenes demonstrate that Horizon3D achieves state-of-the-art performance for radar-camera 3D object detection. On the validation set, our approach outperforms the previous best method by + 3.0 \mathbf{+3.0} NDS and + 1.6 \mathbf{+1.6} mAP while maintaining a sparse representation with competitive inference speed. Figure 1 : Comparison of radar-camera fusion paradigms. (a) Dense BEV fusion encodes scene-level context but incurs growing cost with range, and its uniform temporal aggregation struggles to capture per-object motion. (b) Sparse query-based fusion achieves efficient object-centric encoding but lacks scene-level context in the spatiotemporal domain. (c) Our hybrid BEV fusion employs Gaussian primitives and sparse BEV features for object- and scene-level encoding, and uses dual-path temporal fusion. 1 Introduction Reliable long-range perception is essential for safe autonomous driving, as substantially longer braking distances at highway speeds require accurate detection of objects beyond 150\,\mathrm{m} [ 10 , 1 , 36 , 31 ] . Prior work on long-range 3D object detection has predominantly focused on LiDAR-based methods [ 22 , 28 , 40 ] . However, the high deployment and maintenance costs of LiDAR, along with its vulnerability to adverse weather, limit its practicality for large-scale deployment. As a cost-effective alternative, radar-camera fusion methods [ 21 , 17 , 16 , 15 , 19 , 4 ] have gained significant attention. Recent work has increasingly adopted 4D radar, which provides spatial measurements including elevation, remains robust to adverse weather, and offers velocity cues for dynamic objects [ 44 , 35 , 2 ] . Despite this progress, widely used benchmarks (e.g., nuScenes, VoD, and TJ4D) [ 5 , 26 , 43 ] primarily target short- to mid-range urban perception. Although TruckScenes [ 10 ] was introduced for long-range, high-speed scenarios, fully exploiting 4D radar-camera complementarity in this setting remains largely unexplored. Existing radar-camera 3D object detection frameworks can be broadly categorized into BEV-based and query-based methods. As illustrated in Fig. 1 , both paradigms exhibit clear limitations in long-range detection, especially in sensor fusion and temporal fusion. For sensor fusion, BEV-based methods [ 17 , 21 , 15 , 16 , 35 , 2 , 44 ] project multimodal features into a unified dense BEV representation, which effectively captures scene-level context but incurs rapidly increasing computational cost as the detection range grows. Moreover, the dense BEV grid encodes the scene uniformly at a fixed spatial resolution, limiting fine-grained object-level detail (Fig. 1 (a)). Query-based methods [ 7 , 19 , 32 ] instead perform object-centric encoding around learnable queries, enabling efficient long-range perception. However, the queries are optimized to represent foreground objects, and the resulting representation lacks sufficient scene-level context [ 20 , 29 ] (Fig. 1 (b)). In this paper, we tackle the challenges of modeling both scene-level and object-level information for temporal fusion in long-range 3D perception. First, long-range objects produce sparse radar returns and occupy only a few image pixels, making single-frame modeling insufficient for robust perception. Consequently, an effective temporal fusion strategy is required to aggregate complementary information across multiple frames. Second, dynamic objects exhibit substantial motion between consecutive frames, particularly at highway speeds. Therefore, temporal feature propagation should be performed at the object level to accurately capture the motion of individual objects over time. Several temporal fusion methods have been proposed for 3D perception. BEV-based methods [ 17 , 21 ] aggregate dense BEV features over the entire spatial grid, effectively mitigating the sparsity of single-frame observations through multi-frame fusion. Although methods such as CRT-Fusion [ 16 ] introduce local motion compensation, they still rely on dense BEV representations, resulting in high computational cost, particularly for long-range perception. In contrast, query-based methods [ 19 , 32 ] propagate compact object queries across frames, enabling each query to capture the temporal evolution of an individual object. While this design efficiently models object motion, it provides limited scene-level temporal context. Therefore, effective long-range 3D perception requires a temporal fusion method that jointly models scene-level context and object-level dynamics while maintaining high computational efficiency for extended-range perception. In this paper, we propose Horizon3D, a sparse radar-camera fusion framework for long-range 3D object detection that addresses these challenges through a hybrid representation combining Gaussian primitives and sparse BEV features (Fig. 1 (c)). Our approach comprises three key modules. The Keypoint-Guided Gaussian Initialization (KGGI) module estimates object keypoints from both radar and camera features and initializes a compact set of Gaussian primitives at these locations to guide object-centric encoding. Unlike existing Gaussian-based methods [ 13 , 12 ] that locate numerous primitives across the scene, KGGI places primitives only at estimated object locations, enabling efficient computation even at extended detection ranges. Given the initialized Gaussians, the Object-Centric Sparse Fusion (OCSF) module aggregates cross-modal features around these primitives and iteratively refines their parameters to align with object structures. The refined Gaussians are then splatted onto the BEV plane to produce a sparse object-centric BEV representation, which is fused with sparse radar BEV features to encode fine-grained object-level detail and scene-level context. The Dual-Path Temporal Fusion (DPTF) module integrates temporal information through two complementary paths. The BEV path accumulates foreground features over multiple frames to compensate for the per-frame sparsity of distant objects, while the Gaussian path propagates past Gaussians to the current frame and encodes per-object motion across frames. In both paths, predicted velocities are used to compensate for the motion of dynamic objects, mitigating temporal misalignment in high-speed scenarios. We evaluate Horizon3D on the TruckScenes benchmark [ 10 ] , which features high-speed and long-range driving scenarios. On the validation split, Horizon3D outperforms the previous best radar-camera fusion method by \mathbf{+3.0} NDS and \mathbf{+1.6} mAP, achieving state-of-the-art performance

Read the complete text