Texture- and Shape-based Adversarial Attacks for Vehicle Detection in Synthetic Overhead Imagery
Mikael Yeghiazaryan Sai Abhishek Siddhartha Namburu Emily Kim Stanislav Panev, [0.5em] Celso de Melo Brent Lance Fernando De la Torre Jessica K. Hodgins · 2024
Read Texture- and Shape-based Adversarial Attacks for Vehicle Detection in Synthetic Overhead Imagery by Mikael Yeghiazaryan Sai Abhishek Siddhartha Namburu Emily Kim Stanislav Panev, [0.5em] Celso de Melo Brent Lance Fernando De la Torre Jessica K. Hodgins (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16358.txt.
Introduction
Mikael Yeghiazaryan and co-authors craft in 2024 texture and shape attacks that fool vehicle detectors in fake overhead images. Air view detection is already hard from clutter, small size, shade, and blocks. Attack tests show how far that trust can bend under set real-world limits.
Texture attacks cap resolution, area, and color range to stay practical. Shape tweaks are graded beside them for attack power. Three detector designs face the attacks in wide tests, which expose a clear trade: practical tweaks bite less, and strong ones look less real. Code and data ship with the study.
Catalog record
| CREATOR | Mikael Yeghiazaryan Sai Abhishek Siddhartha Namburu Emily Kim Stanislav Panev, [0.5em] Celso de Melo Brent Lance Fernando De la Torre Jessica K. Hodgins |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-16358
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Texture- and Shape-based Adversarial Attacks for Vehicle Detection in Synthetic Overhead Imagery Mikael Yeghiazaryan Sai Abhishek Siddhartha Namburu Emily Kim Stanislav Panev, [0.5em] Celso de Melo Brent Lance Fernando De la Torre Jessica K. Hodgins Abstract Detecting vehicles in aerial images can be very challenging due to complex backgrounds, small resolution, shadows, and occlusions. Despite the effectiveness of SOTA detectors such as YOLO, they remain vulnerable to adversarial attacks (AAs), compromising their reliability. Traditional AA strategies often overlook the practical constraints of physical implementation, focusing solely on attack performance. Our work addresses this issue by proposing practical implementation constraints for AA in texture and/or shape. These constraints include pixelation, masking, limiting the color palette of the textures, and constraining the shape modifications. We evaluated the proposed constraints through extensive experiments using three widely used object detector architectures, and compared them to previous works. The results demonstrate the effectiveness of our solutions and reveal a trade-off between practicality and performance. Additionally, we introduce a labeled dataset of overhead images featuring vehicles of various categories. We will make the code/dataset public upon paper acceptance. Figure 1 : We enforced various constraints on common adversarial attacks to improve their implementation in the real world. Our pipeline perturbs objects’ texture, shape, or both. The latter demonstrates superior efficacy in deceiving object detectors. 1 Introduction Robust object detection in aerial and satellite images is vital for automating critical tasks like traffic management, urban planning, and disaster response. State-of-the-art (SOTA) detectors such as YOLO and RetinaNet, which rely on deep neural networks (DNNs), have become foundational in this domain. However, recent studies such as Szegedy et al . [ 74 ] have revealed that DNNs can be susceptible to adversarial examples. Given the importance of these applications, understanding this vulnerability is crucial, especially in object detection in Remote Sensing Imagery (RSI). Furthermore, there are scenarios where utilizing AAs to impede vehicle detection by computer vision systems in overhead images could offer strategic advantages, such as military camouflage. Our primary objective is to investigate the resilience of object detectors against adversarial vehicles in RSI scenarios under realistic constraints. Traditional AA strategies often neglect the constraints of physical implementation, focusing solely on task performance. For example, adversarial texture patterns typically resemble those depicted in Figure 1 (unconstrained texture-only attack). However, it is essential to consider how these patterns could be realistically applied, such as attaching a printed vinyl wrap to a vehicle. In this study, we propose a set of constraints within AAs to ensure the attack is feasible for implementation on vehicles, effectively camouflaging them. We define a practical adversarial mesh as a mesh that is perturbed from its original state (by modifying texture and/or shape) such that replicating the same set of modifications in real life would not require specialized equipment or significant resources, or at least would facilitate the process of implementing these modifications. We outline the following aspects that define practicality: installation cost, installation difficulty, and difficulty of operation. Our results indicate that while constrained attacks are less successful than traditional AAs [ 75 , 73 , 72 , 38 ] , they provide much better practicality of implementation. Shape-only attacks exhibit lower effectiveness than unconstrained texture attacks. Nevertheless, combining constrained texture with practical shape modifications enhances performance, reaching levels comparable to unconstrained texture attacks, see Figure 1 . Our work contributes to the field in several ways. (1) We introduce constrained AAs for shape and/or texture, designed to create practical camouflages capable of deceiving object detectors in RSI. These constraints facilitate more straightforward implementation compared to unconstrained camouflages. (2) We thoroughly examine how practicality and adversarial performance relate, finding that they have an inverse relationship. (3) We provide a new object detection dataset comprising labeled real overhead images featuring eight distinct vehicle classes covering a broad spectrum of commonly encountered categories. (4) We developed a tool for generating synthetic overhead images, contributing to creating synthetic datasets. Figure 2 : Illustration of the three types of adversarial attacks. 2 Related Work Over the past ten years, adversarial attacks (AAs) have become an increasingly important research topic in computer vision. Szegedy et al . [ 74 ] addressed the problem of neuron explainability but sparked a new branch of research that targeted the stability and robustness of a range of computer vision models [ 41 , 67 , 36 , 20 , 30 , 52 , 19 , 43 , 65 , 32 , 31 , 62 , 61 , 48 , 46 ] . Recent research has proposed various methods for generating adversarial examples [ 33 , 56 , 55 , 49 , 13 , 16 ] . This direction of research evolved into the class of digital AAs [ 78 , 23 , 50 , 53 , 45 , 64 , 58 , 76 ] , which according to [ 3 ] are defined as attacks where the attacker is capable of directly modifying image pixels to fool the victim model (see Figure 2 ). These attacks are usually imperceptible. A very different type of AAs is the class of physical AAs [ 15 , 27 , 29 , 68 , 69 , 9 , 24 , 28 , 11 ] , which according to [ 3 ] are defined as attacks where the attacker has no direct access to the images supplied to the model, but threatens the model performance by physically manipulating the objects expected to appear in the target images. Some works attempt to perform imperceptible physical attacks, but the majority of such attacks are perceptible. We also highlight that there are works that lie at the intersection of digital and physical attacks, which we call simulated AAs [ 42 , 35 , 8 , 51 , 25 , 75 , 7 , 66 ] , because the majority of these works perform perceptible attacks but test them in simulated realistic environments, avoiding any real tests. Because we utilize synthetically generated data for testing, our work belongs to this class of AAs. Additionally, we utilize a realistic physics-based rendering to test our results, which also separates us from prior works. Moreover, considerable effort has been invested in crafting adversarial 3D geometry, as demonstrated in [ 83 , 44 , 79 , 82 ] . Most of the works that generate adversarial shapes concentrate on point clouds. This focus is a direct consequence of extensive research in autonomous driving, where LIDARs play a crucial role in localization and mapping. Some works also conduct AAs in a coupled LIDAR+RGB setting [ 12 ] . However, it differs from our task, as we assume only RGB data in RSI. Unlike our work, most of the studies highlighted above completely avoid constraining the adversarial search space or use ”light” constraints expressed as regularization terms in the loss functions. Recent years have seen increased attention by the research community to remote sensing imagery [ 89 , 87 , 40 , 70 , 81 , 26 , 2 , 6 ] . These images represent a top-down view and typically have resolutions ranging from several centimeters to several meters or even tens of meters per pixel. The exponential growth of this type of data necessitates robust