Snowball Adversarial Attack on Traffic Sign Classification

Anthony Etim, Jakub Szefer · 2025

Read Snowball Adversarial Attack on Traffic Sign Classification by Anthony Etim, Jakub Szefer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19757.txt.

Introduction

Anthony Etim and Jakub Szefer fool traffic sign readers with snow in this 2025 paper. Most attacks on image classifiers hide in tiny invisible tweaks. People, though, read signs through heavy occlusion with ease, while networks fall apart; the authors turn that gap into a weapon.

Their Snowball attack covers signs with visible white blotches that mimic snow. A human driver shrugs and reads on. State-of-the-art sign classifiers, tested across many images, stumbled badly under the fake weather. The trick needs little effort, no access to the model's insides, and works in the open rather than in pixel noise. The authors argue that defenses trained on invisible perturbations miss the visible kind, and that road-facing systems need testing against the weather-like clutter of the real world.

Catalog record

CREATORAnthony Etim, Jakub Szefer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19757

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Snowball Adversarial Attack on Traffic Sign Classification Anthony Etim, Jakub Szefer Abstract . Adversarial attacks on machine learning models often rely on small, imperceptible perturbations to mislead classifiers. Such strategy focuses on minimizing the visual perturbation for humans so they are not confused, and also maximizing the misclassification for machine learning algorithms. An orthogonal strategy for adversarial attacks is to create perturbations that are clearly visible but do not confuse humans, yet still maximize misclassification for machine learning algorithms. This work follows the later strategy, and demonstrates instance of it through the Snowball Adversarial Attack in the context of traffic sign recognition. The attack leverages the human brain’s superior ability to recognize objects despite various occlusions, while machine learning algorithms are easily confused. The evaluation shows that the Snowball Adversarial Attack is robust across various images and is able to confuse state-of-the-art traffic sign recognition algorithm. The findings reveal that Snowball Adversarial Attack can significantly degrade model performance with minimal effort, raising important concerns about the vulnerabilities of deep neural networks and highlighting the necessity for improved defenses for image recognition machine learning models. 1. Introduction Deep neural networks (DNNs) have become integral to modern computer vision applications, particularly in autonomous driving, where traffic sign recognition plays a crucial role in decision-making. However, these models remain highly susceptible to adversarial attacks, where subtle, carefully placed perturbations can mislead classification systems ( Szegedy, 2013 ; Goodfellow et al., 2014 ) . Such attack strategies focus on minimizing the visual perturbation for humans so they are not confused while maximizing the misclassification for machine learning algorithms. In an orthogonal attack strategy, growing number of adversarial image attacks focuses on exploiting image perturbations that are visible, but not confusing, to humans, while they make machine learning algorithms to fail to correctly recognize images. In particular, researchers have focused on naturally occurring artifacts such as leaves, shadows, and lighting variations that can be exploited to create adversarial perturbations to images, subtly altering a model’s perception without drawing human suspicion ( Zhong et al., 2022 ; Hsiao et al., 2024 ; Etim and Szefer, 2024 ) . These physical attacks are particularly concerning because they can also be executed in natural environments by physically modifying the objects being captured with digital cameras, without requiring direct access to a machine learning model or digital input data. These attacks highlight the fragility of deep learning models in uncontrolled environments, where even minor environmental changes can cause misclassification. Building upon this idea, we introduce the Snowball Adversarial Attack, a novel physical attack that simulates snow accumulation on traffic signs to induce misclassification. This attack leverages the idea that naturally occurring snowfall or snowballs stuck to a traffic sign are naturally ignored by drivers, yet they can correctly recognize the street signs – but machine learning models are confused. Unlike traditional adversarial attacks that require small, precisely crafted perturbations, the Snowball Adversarial Attack utilizes a progressive occlusion strategy, where obvious patches of snow are placed at strategic locations over street signs. A search algorithm is utilized to find the best location and is shown to be effective for many street sign types and various snowballs or snow patches. This attack is particularly dangerous in real-world winter conditions, where snowfall already introduces visual obstructions and reduces the contrast of traffic signs against their backgrounds. Because autonomous vehicles and traffic monitoring systems rely heavily on visual perception, an attacker can exploit this vulnerability by carefully various amounts of snow in key regions of a sign, disrupting classification while maintaining a seemingly natural appearance. This makes the Snowball Adversarial Attack both stealthy and highly effective, as it blends seamlessly into common winter weather conditions, making it difficult to detect even through human oversight. In addition to presenting the new attack, a secondary contribution of this work is a digital framework used to evaluate the attacks. It makes the evaluation more scalable, but also removes need to physically alter the street signs – which could be impractical, dangerous or even illegal. The digital framework is used in our evaluation and analysis of the attack’s effectiveness. The Snowball Adversarial Attack poses a significant threat to autonomous vehicles and intelligent transportation systems. Since self-driving cars rely heavily on computer vision-based classifiers for real-time decision-making, an attack that misclassifies a stop sign as a speed limit sign could have severe consequences. Given the stealthy and naturally occurring nature of snow this attack may be hard to detect and there is need for creating more robust traffic sign recognition models. 1.1. Contributions The contributions of this work are as follows: (1) We propose the Snowball Adversarial Attack, a novel adversarial attack that leverages snow-like obstructions to manipulate traffic sign classification. (2) We demonstrate that strategic snow placement on specific regions of traffic signs maximizes misclassification rates across different sign types, showing a universal effect across various traffic sign classes. (3) We evaluate the attack’s effectiveness using our digital framework, leveraging real street signs from Street View and various realistic snowballs or patches created using generative machine learning models. (4) We demonstrate the ability of the attack to cause misclassification of all the street signs we have tested. 2. Background In this section, we provide an overview of adversarial attacks in machine learning, focusing on their implications for traffic sign classification systems. We also introduce the LISA dataset, a widely used benchmark for U.S. traffic sign recognition, and discuss LISA-CNN, the convolutional neural network model used in this work as a victim classifier. 2.1. Adversarial Attacks Adversarial attacks exploit the vulnerabilities of deep neural networks (DNNs) by introducing small, carefully crafted perturbations to input data, causing the model to misclassify objects while remaining undetectable to human observers ( Szegedy, 2013 ; Goodfellow et al., 2014 ) . Other attacks have explored adding larger perturbations, which are ignored by humans, but confuse machine learning models ( Eykholt et al., 2018 ; Zhong et al., 2022 ) . Adversarial attacks can be broadly categorized into digital attacks, where perturbations are applied directly to input images, and physical attacks, where adversarial modifications are introduced into real-world environments. Among digital attacks, early attack methods, such as the Fast Gradient Sign Method (FGSM) ( Goodfellow et al., 2014 ) and Projected Gradient Descent (PGD) ( Madry, 2017 ) , rely on perturbing individual images to fool a model. More advanced approaches, like universal adversarial perturbations (UAPs), generate a single perturbation that generalizes across multiple inputs, making attacks more practical for real-world scenarios ( Moosavi-Dezfooli et al., 2017 ) . Among physical attacks, recent studies have shown that physical adversarial attacks pose a significant challenge to vision-based models. Attacks using stickers, shadows, and light-based manipulations have successfully deceived traffic sign classifiers by subtly altering key regions of an image without requiring direct access to the model ( Eykholt et al., 2018 ; Zhong et al., 2022 ; Hsiao et al., 2024 ) . Attacks using leaves have also been used to mislead traffic sign classifiers as well ( Etim and Szefer, 2024 ) . These attacks highlight the importance of understanding adversarial vulnerabilities beyond digital spaces and accounting for real-world environmental factors that may compromise model robustness. (a) Stop Test Image (b) Yield Test

Read the complete text