Out-of-Distribution Object Detection in Street-Scenes via Synthetic Outlier Exposure and Transfer Learning

Sadia Ilyas 1,3 , Annika Mütze 2 , Klaus Friedrichs 3 , Thomas Kurbiel 3 , Matthias Rottmann 2 · 2026

Read Out-of-Distribution Object Detection in Street-Scenes via Synthetic Outlier Exposure and Transfer Learning by Sadia Ilyas 1,3 , Annika Mütze 2 , Klaus Friedrichs 3 , Thomas Kurbiel 3 , Matthias Rottmann 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16122.txt.

Introduction

Ilyas and co-authors teach street detectors to flag objects they were never trained on. Missed odd objects leave driving stacks blind and silent. Their SynOE-OD method paints fake outliers into street scenes offline. Text-guided inpainting builds the fakes, and a checker verifies them. Fine-tuning then adds one OOD class to a stock detector. No extra branch runs at test time in the design. OoDIS scores beat the best open-vocabulary baseline clearly. In-distribution skill stays competitive in street scenes.

Catalog record

CREATORSadia Ilyas 1,3 , Annika Mütze 2 , Klaus Friedrichs 3 , Thomas Kurbiel 3 , Matthias Rottmann 2
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54255
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.16122

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Out-of-Distribution Object Detection in Street-Scenes via Synthetic Outlier Exposure and Transfer Learning Sadia Ilyas 1,3 , Annika Mütze 2 , Klaus Friedrichs 3 , Thomas Kurbiel 3 , Matthias Rottmann 2 Abstract Out-of-distribution (OOD) object detection is an important yet underexplored task. A reliable object detector should be able to handle OOD objects by localizing and correctly classifying them as OOD. However, a critical issue arises when such atypical objects are completely missed by the object detector and incorrectly treated as background. Existing OOD detection approaches in object detection often rely on complex architectures or auxiliary branches and typically do not provide a framework that treats in-distribution (ID) and OOD in a unified way. In this work, we address these limitations by enabling a single detector to detect OOD objects, that are otherwise silently overlooked, alongside ID objects. We present SynOE-OD , a Syn thetic O utlier- E xposure-based O bject D etection framework, that leverages strong generative models, like Stable Diffusion, and Open-Vocabulary Object Detectors (OVODs) to generate semantically meaningful, object-level data that serve as outliers during training. The generated data is used for transfer-learning to establish strong ID task performance and supplement detection models with OOD object detection robustness. Our approach achieves state-of-the-art average precision on an established OOD object detection benchmark, where OVODs, such as GroundingDINO, show limited zero-shot performance in detecting OOD objects in street-scenes. 1. Introduction In real-world applications, the semantic distribution of encountered data often deviates from that provided during training [ 28 , 31 ] . Under such conditions, object detectors may overlook objects that fall outside the semantic distribution of typical scenes, implicitly treating them as background. These unknown objects, that extend beyond the categories observed during training, are referred to as out-of-distribution (OOD) objects [ 10 ] . In safety-critical applications, e.g., autonomous driving [ 23 ] or medicine [ 19 ] , it becomes crucial to detect OOD objects [ 4 , 58 ] as accurately as in-distribution (ID) objects in a perceived scene. A significant amount of work [ 21 , 33 , 29 , 59 , 56 , 38 , 49 ] has been carried out that addresses OOD image classification and OOD semantic segmentation [ 6 , 70 , 55 , 3 ] , while OOD object detection remains comparatively understudied [ 15 , 25 ] . The lack of unified benchmarks for ID/OOD object detection and the scarcity of realistic OOD data has hindered the progress on this topic. Moreover, recent surveys [ 1 , 64 ] emphasize these gaps and highlight inconsistent problem formulation, evaluation protocols, and trade-offs in OOD object detection [ 43 ] . With the emergence of Open-Vocabulary Object Detectors (OVODs), it is worthwhile to consider whether the traditional concerns about OOD detection have been explained away [ 44 , 45 , 46 ] . Recent study [ 25 ] shows that OVODs, despite their strong zero-shot generalization, exhibit limited performance in certain OOD detection scenarios. Furthermore, it is unclear how to best utilize them in real-world scenarios, given that they are fundamentally prompt-dependent. Since unknown objects cannot be anticipated beforehand, prompt-based methods may overlook them. Moreover, the effect of fine-tuning OVODs for joint ID and OOD object detection is not well established, nor is it clear whether such an adaptation preserves or degrades their capability to detect OOD objects. (a) Synthetic outliers and real ID objects (b) Real OOD and ID objects Fig. 1 : Qualitative results for OOD and ID object detection. The model is trained using synthetic outlier data and evaluated on real-world street-scene OOD data. Bounding boxes illustrate accurate localization and classification of ID and OOD objects. One influential idea in OOD image classification is Outlier Exposure (OE) [ 22 ] , a strategy that exposes a model to auxiliary outlier data to encourage high uncertainty for OOD samples at test time. Virtual Outlier Synthesis (VOS) [ 15 ] adapts this principle to object detection by synthesizing virtual outliers in feature space and operates on Pascal-VOC/COCO setting, where one dataset is treated as ID and the other is used to construct OOD samples. However, this setup is less challenging than real-world street-scenes, where OOD objects appear in complex environments. Recent advances in image generation and inpainting via Stable Diffusion [ 57 ] make OE more scalable beyond Pascal-VOC/COCO [ 17 , 35 ] style setting. Inspired by the recent line of work [ 15 , 57 ] , we propose SynOE-OD, a framework that incorporates synthetic outliers directly into the image space of street-scene datasets, illustrated in Fig. 1 (left), and explicitly fine-tunes object detectors to classify unusual objects as OOD during inference, cf. Fig. 1 (right). More specifically, we combine two complementary paradigms, i.e., OE and transfer-learning [ 24 ] , into a unified framework by utilizing a strong generative model, Stable Diffusion and an OVOD, e.g. GroundingDINO (GDINO) [ 37 ] , to enable object-level OE that supplements object detectors with OOD object detection capability, shown in Fig. 2 . This formulation bridges the critical gap between the underexplored task of OOD object detection and prior OOD research, which has largely focused on image classification and segmentation. In addition, we consider application scenarios such as autonomous driving, which require coupled reasoning over localization and classification for both known and unknown objects, a setting that remains challenging for existing approaches. Our key contributions are as follows: • We propose SynOE-OD, an object detection framework that integrates seamlessly with standard object detectors, adapting them to jointly detect OOD and ID objects. • We introduce a data generation pipeline that enables OOD object detection by systematically incorporating synthetic object-level outliers in a given dataset. The curated datasets are utilized in fine-tuning both closed-vocabulary detectors as well as OVODs, resulting in state-of-the-art OOD object detection performance while retaining competitive ID object detection. • We provide a comprehensive evaluation across diverse training configurations and dataset compositions, demonstrating the robustness and generality of the proposed framework. Fig. 2 : Overview of SynOE-OD, illustrating an augmented NuImages training set and evaluation on an unseen test set, i.e., RoadAnomaly, at inference time. During transfer-learning synthetic outliers are incorporated into the training data, and object detectors are fine-tuned to achieve joint ID and OOD object detection capability. 2. Related Work In this section, we organize prior contributions on OOD detection into three core tasks: (i) image classification, that operates on image-level and lacks spatial reasoning about OOD objects, whereas (ii) semantic segmentation exposes per-pixel class probability, and (iii) object detection couples localization and classification, addressing salient object-level OOD detection failures. OOD detection in image classification . Conventional OOD detection relied on softmax confidence-scores [ 21 , 33 ] , and distance-based techniques, i.e., Mahalanobis [ 29 ] and nearest neighbors [ 59 ] . Later work introduced likelihood ratios [ 56 ] and energy-based models [ 38 ] . A major advancement led to the concept of OE [ 22 ] , which incorporates auxiliary outliers during training and has inspired numerous extensions [ 60 , 2 , 32 , 69 ] . With the emergence of VLMs, e.g., CLIP [ 54 ] , OOD detection shifted toward two conceptual categories, first, prompt-based methods [

Read the complete text