Segmenting Objectiveness and Task-awareness Unknown Region for Autonomous Driving
Mi Zheng, Guanglei Yang, Zitong Huang, Zhenhua Guo, Kevin Han, Wangmeng Zuo · 2025
Read Segmenting Objectiveness and Task-awareness Unknown Region for Autonomous Driving by Mi Zheng, Guanglei Yang, Zitong Huang, Zhenhua Guo, Kevin Han, Wangmeng Zuo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19183.txt.
Introduction
Mi Zheng and co-authors present a method for self-driving cars to spot unknown hazards on the road, in a 2025 paper. Road-scene systems learn from fixed training sets, so objects outside those sets can be missed. Past hazard tools often flag odd regions poorly or mark items that pose no threat to driving.
The new framework sharpens the outline of unknown objects by fusing scene features at two scales. A second part reads the wider scene and filters out odd items that do not affect navigation. Tests on three public benchmark sets show steady gains across several base detectors. The authors say the two checks together give fuller, cleaner hazard maps for driving scenes.
Catalog record
| CREATOR | Mi Zheng, Guanglei Yang, Zitong Huang, Zhenhua Guo, Kevin Han, Wangmeng Zuo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-19183
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Segmenting Objectiveness and Task-awareness Unknown Region for Autonomous Driving Mi Zheng, Guanglei Yang, Zitong Huang, Zhenhua Guo, Kevin Han, Wangmeng Zuo Abstract . With the emergence of transformer-based architectures and large language models (LLMs), the accuracy of road scene perception has substantially advanced. Nonetheless, current road scene segmentation approaches are predominantly trained on closed-set data, resulting in insufficient detection capabilities for out-of-distribution (OOD) objects. To overcome this limitation, road anomaly detection methods have been proposed. However, existing methods primarily depend on image inpainting and OOD distribution detection techniques, facing two critical issues: (1) inadequate consideration of the objectiveness attributes of anomalous regions, causing incomplete segmentation when anomalous objects share similarities with known classes, and (2) insufficient attention to environmental constraints, leading to the detection of anomalies irrelevant to autonomous driving tasks. In this paper, we propose a novel framework termed Segmenting Objectiveness and Task-Awareness (SOTA) for autonomous driving scenes. Specifically, SOTA enhances the segmentation of objectiveness through a Semantic Fusion Block (SFB) and filters anomalies irrelevant to road navigation tasks using a Scene-understanding Guided Prompt-Context Adaptor (SG-PCA). Extensive empirical evaluations on multiple benchmark datasets, including Fishyscapes Lost and Found, Segment-Me-If-You-Can, and RoadAnomaly, demonstrate that the proposed SOTA consistently improves OOD detection performance across diverse detectors, achieving robust and accurate segmentation outcomes. Conference: Proceedings of the 33rd ACM International Conference on Multimedia; October 27–31, 2025; Dublin, Ireland. Proceedings of the 33rd ACM International Conference on Multimedia (MM ’25), October 27–31, 2025, Dublin, Ireland ISBN: 979-8-4007-2035-2/2025/10 DOI: XXXXXXX.XXXXXXX CCS: Computing methodologies Scene anomaly detection 1. Introduction The integration of transformer-based architectures ( Zheng et al., 2021 ; Cheng et al., 2021 ; Cheng et al., 2022 ; Ranftl et al., 2021 ; Strudel et al., 2021 ) and large-scale pretrained models ( Liu et al., 2021 ; Kirillov et al., 2023 ; Dosovitskiy et al., 2020 ) has significantly advanced semantic segmentation in road scene perception. These methods have shown outstanding performance in closed-set environments, characterized by predefined object categories like vehicles and pedestrians ( Cordts et al., 2016 ) . However, real-world environments are inherently open, often containing unexpected out-of-distribution (OOD) objects—from accident debris to rare natural phenomena—that significantly challenge traditional segmentation models. Misclassifying these anomalies as background or assigning them to known categories introduces substantial safety hazards, particularly in autonomous driving. For instance, failing to detect a fallen tree or a misplaced construction sign can result in severe accidents. Consequently, reliably identifying and segmenting OOD objects without compromising in-distribution (ID) segmentation accuracy has emerged as a critical challenge, termed road anomaly detection, for robust and safe scene understanding ( Yang et al., 2021 ; Bogdoll et al., 2022 ) . Current road anomaly detection methods follow two main technical routes. The first paradigm relies primarily on reconstruction-based approaches ( Lis et al., 2023 ; Zavrtanik et al., 2020 ; Xia et al., 2020 ; Lis et al., 2019 ; Wang et al., 2018 ) , which use generative models to learn normal scene distributions and detect anomalies via high reconstruction errors. These methods proved effective in controlled environments but faced inherent challenges in complex real-world scenarios. Recently, the field has shifted towards score-based methods ( Tian et al., 2022 ; Di Biase et al., 2021 ; Liu et al., 2023 ; Nayal et al., 2023 ; Rai et al., 2023 ; Delić et al., 2024 ) , framing anomaly detection as a confidence estimation problem. These approaches typically leverage prediction uncertainty or feature-space distances to compute pixel-level anomaly scores. Representative works include RPL ( Liu et al., 2023 ) and RbA ( Rai et al., 2023 ) . RPL ( Liu et al., 2023 ) uses residual pattern learning to identify deviations from expected feature distributions, while RbA ( Nayal et al., 2023 ) employs a multi-head rejection mechanism, treating mask classification results as multiple one-vs-all classifiers for robust anomaly identification. Despite significant advancements, prior approaches often treat road anomaly detection as a general anomaly detection problem, where all outliers are deemed anomalous. This leads to two fundamental limitations hindering their practical deployment in autonomous driving systems, as shown in Fig. . First, they struggle with incomplete segmentation of partial anomalies. Current methods fail to capture all OOD-specific pixels, especially when anomalies share partial visual characteristics with in-distribution objects. For example, state-of-the-art RbA ( Nayal et al., 2023 ) (Fig. , column 1–2) detects anomalies like damaged tires but fails to segment them entirely, leaving residual regions misclassified as background. This issue arises because its multi-head rejection mechanism suppresses uncertain pixels without explicitly modeling OOD shape continuity, leading to fragmented masks with undetected anomaly parts. Second, they suffer from task-agnostic overdetection. Existing methods overlook road scene constraints, leading to false alarms in irrelevant regions. As shown in Fig. (column 3–5), non-critical objects outside drivable areas, such as roadside vegetation and distant buildings, are erroneously flagged as OOD. This happens because RbA’s ( Nayal et al., 2023 ) threshold-based scoring system prioritizes anomaly likelihood without considering autonomous driving task requirements, resulting in overdetection in areas unrelated to navigation safety. These two issues—partial under-segmentation and task-irrelevant overdetection—highlight the need for anomaly detection frameworks that explicitly model OOD objective attributes while incorporating scene constraints to enhance autonomous driving reliability. In this paper, we propose Segmenting Objectiveness and Task-awareness (SOTA), a unified framework integrating semantic feature fusion with scene-understanding guided prompt learning. It comprises two modules: the Semantic Fusion Block (SFB) and the Scene-understanding Guided Prompt-Context Adapter (SG-PCA). For SFB, the pixel-wise segmentor’s vanilla OOD prediction is aligned to the latent vision space via projection and alignment blocks. Fusing the OOD map with vision features improves objectiveness segmentation precision by refining OOD-specific attribute detection and reducing partial anomaly errors. SG-PCA, conversely, extracts road scene priors (e.g., lane topology, drivable areas) through task-aware aggregation, adapting Erosion and Dilation operations to resolve partial occlusion issues. Subsequently, scene priors and the vanilla OOD prediction combine to generate task-aware prompts via multi-aware cross-attention. By suppressing navigation-irrelevant anomalies through prompt learning, SG-PCA ensures focus on safety-critical regions, overcoming environmental constraints inherent in distribution-based methods. Moreover, we introduce parameter-efficient adaptation of SAM’s ( Kirillov et al., 2023 ) mask decoder using Low-Rank Adaptation (LoRA) ( Hu et al., 2022 ) , enabling seamless integration of enriched OOD embeddings without additional manual threshold tuning or postprocessing. Our contributions are summarized as follows: • We are the first to incorporate both objectiveness and task-awareness into road anomaly detection, significantly improving its practical utility for real-world autonomous driving applications. • We propose Segmenting Objectiveness and Task-awareness (SOTA), comprising two key modules—Semantic Fusion Block (SFB) and Scene-understanding Guided Prompt-Context Adapter (SG-PCA)—that together achieve spatial and semantic completeness in anomaly detection. • Extensive experiments on benchmark datasets demonstrate that our approach significantly outperforms the baseline and state-of-the-art method in both pixel-level and component-level anomaly segmentation results, achieving marked improvements across key evaluation metrics. 2. Related Work Semantic Segmentation. Semantic segmentation, the task of assigning a class label to each pixel in an image, is critical in scene understanding and computer vision ( Cordts et al., 2016 ; Thisanke et al., 2023 ; Minaee et al., 2020 ) . Early models ( Long et al., 2015 ; Ronneberger et al., 2015 ; Badrinarayanan et al., 2017