HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection
Esteban RiveraSurya PrabhakaranMarkus LienkampTechnical University of Munich and Munich Institute of Robotics and Machine IntelligenceMunich, Germany [email protected] · 2025
Read HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection by Esteban RiveraSurya PrabhakaranMarkus LienkampTechnical University of Munich and Munich Institute of Robotics and Machine IntelligenceMunich, Germany [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-00507.txt.
Introduction
Esteban Rivera, Surya Prabhakaran, and Markus Lienkamp present, in a 2025 paper, a smarter way to choose training examples for self-driving perception. Labeling 3D scenes is expensive, so active learning picks the scenes a model most needs. Earlier pickers judged scenes only by model uncertainty, ignoring practical knowledge about what makes a scene hard. The team's method, HeAL3D, adds common-sense cues such as object distance and point counts.
The picker blends those cues with the model's own doubts about where objects are and what they are. On the standard KITTI benchmark, the method matched rival state-of-the-art accuracy. More impressively, it matched the accuracy of training on everything while using only 24 percent of the samples. The authors argue that practical heuristics, long used by engineers, deserve a formal seat in the selection loop.
Catalog record
| CREATOR | Esteban RiveraSurya PrabhakaranMarkus LienkampTechnical University of Munich and Munich Institute of Robotics and Machine IntelligenceMunich, Germany [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-00507
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
HeAL3D: Heuristical-enhanced Active Learning for 3D Object Detection Esteban RiveraSurya PrabhakaranMarkus LienkampTechnical University of Munich and Munich Institute of Robotics and Machine IntelligenceMunich, Germany [email protected] Abstract Active Learning has proved to be a relevant approach to perform sample selection for training models for Autonomous Driving. Particularly, previous works on active learning for 3D object detection have shown that selection of samples in uncontrolled scenarios is challenging. Furthermore, current approaches focus exclusively on the theoretical aspects of the sample selection problem but neglect the practical insights that can be obtained from the extensive literature and application of 3D detection models. In this paper, we introduce HeAL (Heuristical-enhanced Active Learning for 3D Object Detection) which integrates those heuristical features together with Localization and Classification to deliver the most contributing samples to the model’s training. In contrast to previous works, our approach integrates heuristical features such as object distance and point-quantity to estimate the uncertainty, which enhance the usefulness of selected samples to train detection models. Our quantitative evaluation on KITTI shows that HeAL presents competitive mAP with respect to the State-of-the-Art, and achieves the same mAP as the full-supervised baseline with only 24% of the samples. † † footnotetext: * Corresponding author 1 Introduction Figure 1 : Typical driving scenario for an autonomous vehicle. Even for LiDAR detectors, which can measure depth directly, farther away objects are more difficult to localize, estimate and classify because of the distance and the lower number of points they reflect. Such information can be leveraged by Active Learning approaches to find the better samples to be labeled. Even human labelers require the support of camera image to label such examples, which reinforces the idea of difficulty and uncertainty. 3D object detection performs well in controlled scenarios and specific data distributions Shi et al. 2020 ; Roddick et al. 2019 . However, once the data distribution changes, it is necessary to record, collect and label new data, followed by retraining of the object detector. Additionally, issues like class novelty and class imbalance significantly restrict the overall performance of 3D object detection in real-world settings Geiger et al. 2012 ; Caesar et al. 2020 ; Fent et al. 2024 . A straightforward approach to solve those issues involves a supervised training procedure with extensive labelled data across all scenarios. However, labelling data for every possible distribution is impractical, if not impossible. Therefore, a scalable method is essential for robust generalization of 3d object detectors across diverse environments. Scalability can be pursued through several approaches. Among these, self-supervised and semi-supervised learning offer strategies that reduce the reliance on labelled data. However, initial human input remains necessary to provide the model with a foundational knowledge of objects, scenes, and defining characteristics. Despite this, minimizing human intervention is a key goal, making Active Learning (AL) an appealing strategy. AL focuses on identifying samples that are particularly challenging to classify or detect, enabling selective human analysis and labelling rather than annotating the entire dataset. In this way, human expertise is only required for specific, high-impact cases, delivering substantial improvements to the models in training while keeping the labelling effort manageable. These strategies are broadly applicable across various deep learning domains; however, in the specific context of autonomous driving, they play a crucial role in 3D object detection. This application poses unique challenges, as it demands reliable detection in outdoor environments with constantly changing weather, lighting conditions, and a multitude of diverse scenarios. Additionally, while cameras are often the standard for perception tasks, LiDAR has become the preferred sensor for autonomous driving due to its advantageous 3D information capture capabilities. Although algorithms originally developed for camera-based detection, including active learning, have been adapted to LiDAR data, new challenges arise with the distinct data structure associated with this sensor. These challenges necessitate specialized approaches tailored to the unique characteristics of LiDAR-based detection Luo et al. 2023b . Furthermore, uncertainty estimation is a widely utilized approach by AL to identify the most optimal samples for labeling, as indicated by previous studies Hekimoglu et al. 2022 ; Elezi et al. 2022 ; Du et al. 2021 ; Gal and Ghahramani 2016 . It is therefore evident that objects for which the model is less confident should provide the most value to improve its generalization capability and consequently its performance. Moreover, the results of several 3D detection architectures Sun et al. 2021 ; Fent et al. 2024 suggests that objects situated at a distance from the sensor or enclosing a smaller number of points are consistently more challenging to detect. Such practical knowledge is given in the literature already and can be heuristically implemented into the uncertainty estimation to enhance and complement it. Therefore, in this paper, we present a new active learning strategy for 3D LiDAR object detection based on both localization and classification uncertainty of augmented samples. We introduce a Gaussian mixture model (GMM) approach to calculate an inconsistency score while incorporating detection distance and class features to improve the information quality of the selected samples. Our contributions are summarized as below: • We propose HeAL, an explainable uncertainty estimation method derived from the real-world application of 3D detection. It uses heuristic information like object distance to enhance an augmentation inconsistency AL approach. • Our sample selection strategy represents the output of the 3D detection as a Gaussian Mixture Model and therefore is agnostic to arbitratry model architectures, facilitating its implementation. • We conducted several experiments with the KITTI dataset which show an improvement of 3% mAP with respect to the current AL state-of-the-art methods (To the best of our knowledge) and the same accuracy as a fully supervised model training with 24% of the data. • We show that previous state-of-the-art approaches perform satisfactorily in a restricted data-quantity domain and propose a piece-wise approach for the AL problem, where different methods are used for different data-domains to keep a constant improvement above random sampling 2 Related Work 2.1 Active Learning Active learning is a strategy to minimize labeling costs by selecting the most informative samples for annotation, particularly useful in data-intensive tasks like object detection Seung et al. 1992 . In recent years, several AL approaches have been developed, targeting the optimization of acquisition functions, uncertainty measures, and efficiency improvements in the labeling process. Classical AL methods rely on uncertainty-based sampling Du et al. 2021 ; Caramalau et al. 2021 ; Gudovskiy et al. 2020 ; Ash et al. 2020 ; Gal and Ghahramani 2016 , where samples with the highest uncertainty scores are prioritized for labeling. Techniques such as BALD (Bayesian Active Learning by Disagreement) Cao and Tsang 2021 and entropy-based sampling Wu et al. 2022 have been used to capture the most ambiguous samples for model improvement. Ensemble models, like those proposed by Beluch et al. Beluch et al. 2018 , increase robustness by training multiple networks and aggregating predictions. Bayesian approaches utilize Monte Carlo dropout Houlsby et al. 2011 ; Gal et al. 2017 or variational autoencoders (VAEs) Tran et al. 2019 to estimate uncertainty, which is then used as a acquisition function to select the most valuable samples. For low-dimensional problems, the uncertainty calculation is tractable, but for high-dimensional domains like image classification the uncertainty can only estimated, giving as