Learnability-Driven Submodular Optimization for Active Roadside 3D Detection

Ruiyu Mao, Baoming Zhang, Nicholas Ruozzi, Yunhui Guo · 2026

Read Learnability-Driven Submodular Optimization for Active Roadside 3D Detection by Ruiyu Mao, Baoming Zhang, Nicholas Ruozzi, Yunhui Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-01695.txt.

Introduction

Roadside cameras can watch traffic for self-driving systems, but labeling their images is costly, and distant or blurred objects may be impossible to label reliably from the roadside view alone. Ruiyu Mao, Baoming Zhang, Nicholas Ruozzi, and Yunhui Guo argue that annotation budgets should skip such inherently ambiguous scenes rather than waste effort on them.

Their active learning framework, LH3D, selects scenes that are both informative and dependably labelable, while keeping coverage of the road scene broad. Using only a quarter of the annotation budget on a standard vehicle-infrastructure dataset, it reached most of the full-data performance for vehicles, pedestrians, and cyclists. The results beat uncertainty-based selection and make a case that labelability matters more than uncertainty.

Catalog record

CREATORRuiyu Mao, Baoming Zhang, Nicholas Ruozzi, Yunhui Guo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-01695

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learnability-Driven Submodular Optimization for Active Roadside 3D Detection Ruiyu Mao, Baoming Zhang, Nicholas Ruozzi, Yunhui Guo Abstract Roadside perception datasets are typically constructed via cooperative labeling between synchronized vehicle and roadside frame pairs, but real deployment is often limited roadside-only data due to hardware and privacy constraints. The observation that even human experts struggle to produce accurate labels without vehicle-side data reveals a fundamental learnability problem: many roadside-only scenes contain distant, blurred, or occluded objects whose 3D properties are ambiguous from a single view and can only be reliably annotated by cross-checking paired vehicle–roadside frames. We refer to such cases as inherently ambiguous samples. In this work, we develop an active learning framework for roadside monocular 3D object detection and propose a learnability-driven framework that selects scenes which are both informative and reliably labelable, suppressing inherently ambiguous samples while ensuring coverage. Experiments demonstrate that our method significantly outperforms uncertainty-based baselines, which suggests that learnability, not uncertainty, matters for roadside 3D perception. 1 Introduction Modern autonomous driving systems primarily rely on ego-vehicle sensors (cameras, LiDAR, radar) to perceive their surroundings 7 ; 4 ; 26 . Yet an ego-only viewpoint suffers from occlusions, intersection blind spots, and limited long-range visibility in dense traffic, motivating vehicle–infrastructure cooperation, where roadside sensors extend the field of view and are widely regarded as a key enabler for Level 5 autonomy 39 ; 37 ; 14 . Among infrastructure options, roadside cameras can be deployed densely at relatively low cost and are naturally suited to bird’s-eye-view (BEV) perception: recent vision-centric BEV frameworks lift monocular or multi-camera images into a top-down representation on the ground plane and then perform 3D object detection and mapping in the BEV space based on monocular depth estimation, where object locations, lanes, and trajectories are explicitly defined 10 ; 15 ; 16 ; 35 ; 31 . Roadside cameras are typically installed at intersections, highway ramps, and other traffic hubs, where traffic flows from multiple directions and vehicles, pedestrians, and cyclists share the same space in complex ways, leading to dense scenes with long-range targets and frequent occlusions 37 ; 42 ; 43 ; 33 . To train reliable roadside BEV detectors in such environments, large-scale datasets must capture the countless variety of objects, layouts, and conditions 39 ; 37 ; 35 ; 31 . To label each frame, annotators must reason about 3D bounding boxes under heavy occlusion and severe depth ambiguity, cross-checking multiple visual cues and contextual references to obtain consistent labels. At city scale, the time required to annotate every frame becomes unsustainable, turning 3D labeling into the dominant cost of deploying roadside perception systems. To reduce the cost of annotating such large data sets, a natural direction is active learning (AL) 27 , which assumes a small labeled subset and a large unlabeled pool of roadside scenes, and iteratively selects the most valuable images for human annotation. However, most existing AL frameworks for this task implicitly assume that monocular images can be labeled reliably in isolation . For roadside BEV 3D detection, accurate labels in current benchmarks are typically obtained with cross-modality or cross-view verification, such as synchronized vehicle–roadside frame pairs or auxiliary LiDAR scans 39 ; 37 ; 35 ; 31 . In real deployment, these cooperative signals are often unavailable, and annotators must infer the class and 3D location of distant or heavily occluded objects from a single view. These inherently ambiguous samples can be problematic in an AL setting: Traditional uncertainty-based AL and diversity-based AL prioritize high-uncertainty or distributionally “novel” scenes, which in roadside BEV often coincide with inherently ambiguous samples. As a result, much of the annotation budget is wasted on intrinsically unlearnable labels instead of filtering them out; Fig. 1 shows that detectors trained on these ambiguous scenes perform worse than those trained on learnable ones under the same budget and class distribution, indicating weaker supervision. We therefore rephrase this challenge of AL for 3D roadside BEV perception through the lens of learnability . Since inherently ambiguous samples are difficult to annotate and impossible to verify without additional views or sensors, directly detecting them in the unlabeled pool is infeasible. Instead, we characterize learnability by how well a scene supports unambiguous depth estimation, balanced semantics, and informative yet resolvable geometry. Reliable depth estimation is critical for monocular BEV as it yields stable BEV geometry and accurate 3D bounding boxes, whereas ambiguous depth cascades into large localization errors and inconsistent labels. At the same time, roadside traffic is dominated by vehicles, but a safe detector cannot be biased toward the majority class; it must still learn pedestrians and cyclists robustly from limited annotations. Finally, roadside environments exhibit a wide range of layouts, e.g., different intersection shapes, lane configurations, and traffic patterns, and an effective model should be exposed to diverse but still learnable geometric configurations rather than repeatedly oversampling a few common patterns. We decompose learnability into three complementary aspects: (i) depth confidence , (ii) semantic balance , and (iii) geometric variation . Depth confidence reflects how stable and predictable the image-to-depth projection is in monocular BEV lifting; samples with confident depth estimates form the foundation for learnable geometry. Semantic balance captures whether an image provides a balanced exposure across rare and common classes, preventing the model from overfitting to dominant vehicle categories and neglecting vulnerable road users. Geometric variation quantifies how the spatial configuration of objects differs across scenes in the roadside BEV scene space, encouraging exploration of novel yet still learnable layouts. Together, these criteria form a unified measure of learnability, allowing our framework to select samples that are not only informative and diverse, but also reliably learnable and less likely to be inherently ambiguous. To jointly optimize these three learnability factors in a principled way, we formulate active selection as a concave-over-modular submodular maximization problem, which provides a natural structure for modeling balanced coverage and enables efficient greedy optimization with theoretical guarantees 20 ; 32 ; 3 ; 19 ; 12 . To the best of our knowledge, this is the first work to explicitly identify inherently ambiguous samples in roadside BEV perception and to design a systematic learnability-driven active learning framework that directly addresses them. Our contributions can be summarized as follows. • We identify and formalize inherently ambiguous samples in roadside BEV perception, showing how single-view, single-modality constraints systematically lead to samples that are fundamentally unlearnable. • We introduce a learnability-driven formulation for active learning, grounded in three complementary factors—depth confidence, semantic balance, and geometric variation—that together characterize when a monocular roadside scene can be reliably learned. • We propose LH3D ( L earnable H ierarchical 3D ), a three-stage submodular active learning framework that operationalizes these learnability factors through concave-over-modular objectives, enabling efficient greedy selection that suppresses inherently ambiguous samples while maintaining semantic and geometric coverage. Figure 1 : Human study: learnable vs. ambiguous samples. Images are categorized as learnable or ambiguous based on how difficult they are to interpret from a single monocular view. Using this partition (while training only with the dataset’s original ground-truth labels), detectors trained on the ambiguous split achieve lower AP on cars and pedestrians under the same annotation budget and class balance, with cyclists remaining similar. This indicates that ambiguous samples provide

Read the complete text