AD4AD: Benchmarking Visual Anomaly Detection Models for Safer Autonomous Driving

Fabrizio Genilotti, Arianna Stropeni, Gionata Grotto, Francesco Borsatti, Manuel Barusco, Davide Dalle Pezze, Gian Antonio Susto · 2026

Read AD4AD: Benchmarking Visual Anomaly Detection Models for Safer Autonomous Driving by Fabrizio Genilotti, Arianna Stropeni, Gionata Grotto, Francesco Borsatti, Manuel Barusco, Davide Dalle Pezze, Gian Antonio Susto (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-15291.txt.

Introduction

Genilotti and six colleagues benchmark eight visual anomaly detection methods for self-driving cars, which must flag road hazards never seen in training. Using AnoVox, the largest synthetic set for this task, they test four network backbones from heavy to light. The methods transfer well to road scenes, and Tiny-Dinomaly gives the best accuracy-for-efficiency balance for on-board use at a fraction of the memory cost.

Catalog record

CREATORFabrizio Genilotti, Arianna Stropeni, Gionata Grotto, Francesco Borsatti, Manuel Barusco, Davide Dalle Pezze, Gian Antonio Susto
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29611
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.15291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AD4AD: Benchmarking Visual Anomaly Detection Models for Safer Autonomous Driving Fabrizio Genilotti, Arianna Stropeni, Gionata Grotto, Francesco Borsatti, Manuel Barusco, Davide Dalle Pezze, Gian Antonio Susto Abstract The reliability of a machine vision system for autonomous driving depends heavily on its training data distribution. When a vehicle encounters significantly different conditions, such as atypical obstacles, its perceptual capabilities can degrade substantially. Unlike many domains where errors carry limited consequences, failures in autonomous driving translate directly into physical risk for passengers, pedestrians, and other road users. To address this challenge, we explore Visual Anomaly Detection (VAD) as a solution. VAD enables the identification of anomalous objects not present during training, allowing the system to alert the driver when an unfamiliar situation is detected. Crucially, VAD models produce pixel-level anomaly maps that can guide driver attention to specific regions of concern without requiring any prior assumptions about the nature or form of the hazard. We benchmark eight state-of-the-art VAD methods on AnoVox, the largest synthetic dataset for anomaly detection in autonomous driving. In particular, we evaluate performance across four backbone architectures spanning from large networks to lightweight ones such as MobileNet and DeiT-Tiny. Our results demonstrate that VAD transfers effectively to road scenes. Notably, Tiny-Dinomaly achieves the best accuracy–efficiency trade-off for edge deployment, matching full-scale localization performance at a fraction of the memory cost. This study represents a concrete step toward safer, more responsible deployment of autonomous vehicles, ultimately improving protection for passengers, pedestrians, and all road users. I Introduction Autonomous driving systems rely on machine learning models trained on large datasets to perceive and interpret the surrounding environment. However, the reliability of such systems is fundamentally bounded by their training data distribution. When a vehicle encounters conditions that deviate significantly from what was seen during training, e.g. atypical obstacles, unusual road configurations, or rare events, its perceptual capabilities can degrade substantially. Unlike many domains where prediction errors carry limited consequences, failures in autonomous driving translate directly into physical risk for passengers, pedestrians, and other road users. A system must therefore not only perform well under expected conditions, but also handle edge cases correctly, or recognize its own limitations when encountering them. A natural response to this challenge is to equip the system with the ability to detect when something unknown is present in the scene. Visual Anomaly Detection (VAD) can offer precisely this capability: VAD models are trained exclusively on normal, expected data and, at inference time, identify anything that deviates from the learned distribution without requiring labeled anomalous examples and without presupposing what form the anomaly may take. This open-world property is critical in the autonomous driving context, where the space of possible road hazards is virtually unbounded and cannot be exhaustively anticipated at training time. In addition, VAD models produce pixel-level anomaly maps that spatially localize the detected deviation within the scene. This is particularly valuable for driver-assistance applications: rather than issuing a generic alert, the system can direct the driver’s attention to the specific region of the scene where the anomaly is located, effectively reducing reaction time and supporting faster, more informed decision-making. Despite the growing maturity of VAD in industrial inspection and medical imaging, its application to autonomous driving remains unexplored. Existing benchmarks and evaluation protocols are not designed for road scenarios, and it remains unclear whether state-of-the-art VAD methods generalize effectively to the visual characteristics of on-road environments. Our contributions can be summarized as follows: • We present the first systematic evaluation of Visual Anomaly Detection in the autonomous driving domain , benchmarking eight state-of-the-art VAD methods on AnoVox, a dataset specifically designed for anomaly detection on the road. • We perform an edge-oriented analysis , by comparing the use of feature extractors of varying size and nature, including CNN-based backbones and lightweight Vision Transformers, providing concrete guidance on the performance–efficiency trade-off for onboard deployment in resource-constrained automotive hardware. • The results obtained on AnoVox demonstrate that VAD models are effective at this task, and that the anomaly maps provide meaningful spatial guidance for driver alerting, providing strong baselines for future work in the field. The remainder of this paper is organized as follows: section II reviews related work on VAD in general and its applications to autonomous driving; section III describes the dataset, the evaluated models and the chosen feature extractors; section IV presents and discusses the results; finally, Section V draws conclusions and outlines future directions. Fig. 1 : Anomaly maps produced by Dinomaly using a DeiT-Small backbone. We report two cases where the model performs well and two failure cases. II Related Work II-A Visual Anomaly Detection Visual Anomaly Detection (VAD) plays a crucial role in many computer vision applications, including manufacturing and healthcare. VAD models offer two main advantages: first, they are typically trained in an unsupervised manner using only anomaly-free samples, bypassing the costly process of collecting and labeling large numbers of anomalous samples; second, they produce pixel-level anomaly maps that enhance interpretability, operator decision-making, and end-user trust. State-of-the-art VAD models generally fall into two main categories: reconstruction-based methods and feature embedding–based methods. Reconstruction-based methods employ generative models to learn the distribution of normal data, identifying anomalies at inference time through high reconstruction errors. Common approaches include Autoencoders, GANs, and Diffusion Models [ 1 , 2 ] . However, these methods are often computationally expensive, which can limit their applicability in real-time or resource-constrained scenarios. Feature embedding–based methods , in contrast, exploit representations extracted from pretrained neural networks, avoiding explicit image reconstruction and achieving higher computational efficiency. This family is further organized into three subcategories: Teacher–Student methods , which detect anomalies through discrepancies between teacher and student feature maps (e.g. STFPM [ 3 ] ); Memory Bank methods , which store normal feature representations for comparison at inference time (e.g., PaDiM [ 4 ] , PatchCore [ 5 ] , CFA [ 6 ] ); and Normalizing Flow methods , which map data distributions to a normal distribution for likelihood-based anomaly detection (e.g. FastFlow [ 7 ] ). II-B Anomaly Detection for Autonomous Driving As anticipated, the detection of hazards that may compromise the safety of road users is crucial. To tackle this problem, numerous datasets and benchmarks have been presented in literature as discussed in [ 8 ] , each characterized by different properties, such as their synthetic or real-world nature and their underlying definition of normality. [ 9 ] introduces two real-world datasets: RoadAnomaly21, containing internet-collected road images with anomalous objects, and RoadObstacle21, recorded in a limited set of street environments. Authors of [ 10 ] propose StreetHazards, a synthetic CARLA-based dataset featuring 250 anomalies across multiple town scenes and weather conditions, and BDD-Anomaly, derived from BDD100K by treating motorcycles, trains, and bicycles as anomalous classes at test time. Lost and Found [ 11 ] targets small hazard detection across diverse real road scenarios, covering 42 object types. Fishyscapes [ 12 ] is an evaluation benchmark providing two validation sets: FS Lost and Found and FS Static, the latter obtained by overlaying anomalous objects onto Cityscapes images. However, these datasets present limitations in the context of Anomaly Detection for Autonomous Driving (AD4AD): Lost and Found lacks road diversity; Fishyscapes relies on artificial anomalies; StreetHazards includes unrealistic anomalies in irrelevant

Read the complete text