Impact Analysis of Inference Time Attack of Perception Sensors on Autonomous Vehicles
Hanlin Chen*, Corresponding Author, ORCID: 0000-0001-6508-7715 Buildings and Transportation Science Division Oak Ridge National Laboratory, Oak Ridge, USA, 37932 Email: [email protected] Simin Chen Department of Computer Science, The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 7508 · 2025
Read Impact Analysis of Inference Time Attack of Perception Sensors on Autonomous Vehicles by Hanlin Chen*, Corresponding Author, ORCID: 0000-0001-6508-7715 Buildings and Transportation Science Division Oak Ridge National Laboratory, Oak Ridge, USA, 37932 Email: [email protected] Simin Chen Department of Computer Science, The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 7508 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-03850.txt.
Introduction
Hanlin Chen and co-authors study a quiet kind of attack on self-driving cars: slowing down the car's own seeing. Their 2025 paper looks at perception sensors, the cameras and other gear a car uses to read the road. An attacker who makes the car's software take longer to judge each scene may leave it too late to brake or steer.
The team built a test system that runs the attack in simulation. They show that the added delay can put the target car at risk and can also endanger other road users nearby. Most past work on sensor attacks asked whether the car sees the wrong thing. This study asks a plainer question: what happens when the car sees the right thing, but too slowly? The answer points to speed of thought as a safety need in its own right.
Catalog record
| CREATOR | Hanlin Chen*, Corresponding Author, ORCID: 0000-0001-6508-7715 Buildings and Transportation Science Division Oak Ridge National Laboratory, Oak Ridge, USA, 37932 Email: [email protected] Simin Chen Department of Computer Science, The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 7508 |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-03850
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Impact Analysis of Inference Time Attack of Perception Sensors on Autonomous Vehicles Hanlin Chen*, Corresponding Author, ORCID: 0000-0001-6508-7715 Buildings and Transportation Science Division Oak Ridge National Laboratory, Oak Ridge, USA, 37932 Email: [email protected] Simin Chen Department of Computer Science, The University of Texas at Dallas 800 W Campbell Rd, Richardson, TX 75080 Email: [email protected] Wenyu Li Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology 2006 Xiyuan Ave, HongKong, China 611731 Email: [email protected] Wei Yang, Ph.D. Department of Computer Science, The University of Texas at Dallas ECSS 4.225, 800 W. Campbell Rd, Richardson, TX 75080 Email: [email protected] Yiheng Feng, ORCID: 0000-0001-5656-3222 Lyles School of Civil Engineering, Purdue University 550 Stadium Mall Drive, West Lafayette, IN 47907 Email: [email protected] Abstract \AuthorHeaders Chen, Chen, Li, Yang and Feng 1 Abstract As a safety-critical cyber-physical system, cybersecurity and related safety issues for Autonomous Vehicles (AVs) have been important research topics for a while. Among all the modules on AVs, perception is one of the most accessible attack surfaces, as drivers and AVs have no control over the outside environment. Most current work targeting perception security for AVs focuses on perception correctness. In this work, we propose an impact analysis based on inference time attacks for autonomous vehicles. We demonstrate in a simulation system that such inference time attacks can also threaten the safety of both the ego vehicle and other traffic participants. Keywords : Autonomous Vehicle, Cybersecurity, Perception security, Cyber-physical sytstem 2 Introduction Autonomous vehicles shed light on the development of smart transportation system as they can improve the safety, mobility and energy efficiency, as shown in the report issued by USDOT [ 1 ] . One critical feature of autonomous vehicles, comparing to their human counterpart, is that they have a very short response time between an event and the vehicle reaction. The processing time for the perception-decision-response process for the human-driven vehicle is determined by human response time, which can be affected by driver’s distraction [ 2 ] , age [ 3 ] and the possibility of DUI [ 4 ] . For autonomous vehicles, the computer-vision-based system has a shorter response time [ 5 ] , and the reaction time is mainly determined by the mechatronic properties of the vehicle itself. Given the fact that autonomous vehicles are safety-critical cyber-physical systems, there has been a lot of research work targeting the vulnerabilities within the system. Lot of research on vulnerabilities in computer vision [ 6 , 7 , 8 ] is also applicable for launching an attack on perception module of autonomous vehicles [ 9 , 10 , 11 ] . Generative adversarial examples can also enable generating perturbations that slowdown the inference speed of deep-learning modules, as shown in [ 12 , 13 , 14 , 15 ] . For example, if a neural network normally inference around 0.1 second(which is a common standard for autonomous driving perception system), when given some attack data, some neural networks’ inference speed can be greatly reduced. As our work shows in section 7.1 , the vitim model will have an average inferece time around 0.1 seconds. But, when under attack its inference time goes up to 3 seconds. This means when the inference process finished, the vehicle though it is responding to the most current environment state, which was actually happened 3 seconds ago. This is way too slow for safety-critical cyber-physical system like autonomous vehicles. In this work, we perform an initial impact analysis of inference time attack on autonomous vehicle’s perception system. As the first target, we apply such an attack towards an autonomous vehicle at a signalized intersection, since in such scenario, the perception system of the autonomous vehicle not only need to detect other road users, but also the traffic signals. With such a threat model, the attack goal in this paper is to maximize the degradation of the performance of an autonomous vehicle in the intersection navigation scenario. The attacker’s primary target is to downgrade the autonomous vehicle’s safety performance while also seeking to compromise mobility and comfort. To achieve the attack goal, smart selection of attack policy is essential, which is the main contribution of this paper. The contribution and novelty of our work are shown below: • We are the first to study the effect of inference time attacks on the perception module of autonomous vehicles. Specifically, we find that the optimal attack policy requires a careful selection of both attack intensity and the launch time. This attack is different from the DoS attack occurring in V2V communication shown in previous work [ 16 ] , as the information retrieval from the environment is not completely blocked out and we also involve the prediction module in state-of-the-art middleware system, which is designed to mitigate the information retrieval delay from perception. • We performed end-to-end evaluation on both inference time only and performance from vehicle side and transportation engieering side. We showed that such attack can threaten both the ego vehicle and other traffic participants as well. The structure of this paper is as follows: Section 3 shows current work regarding autonomous vehicle perception time and inference time attack. Section 4 shows the threat model in this work, mainly about attacker assumption and attacker’s goal. Section 5 shows how the attack is formulated and how adversarial example is obtained. Section 6 shows the impact analysis with end-to-end simulation system, on inference only, vehicle and transportation side. Section 8 shows potential future work with regard to defense, and section 9 concludes this work. 3 Literature Review Figure 1: Problem definition of Object detection and the working mechanism of Pixel2Seq 3.1 The perception and control of autonomous vehicles Autonomous vehicles use the information obtained from sensors to make control decisions. In this paper, we focus on attacking the object detection perception module for the following reasons: (1) the object detection module plays a vital role in autonomous vehicles; (2) the object detection module is the most widely deployed in autonomous vehicles [ 17 ] . As shown in Figure 1 (a), given one input image, the object detection module seeks to output the bounding box and the class labels of each object in the image. Here, we briefly introduce several working mechanisms of the neural networks-based object detection module. The first type is R-CNN, the masterpiece of two-stage object detection neural networks. In the first stage, R-CNN applies its region extraction neural networks to generate object region proposals. In the second stage, R-CNN uses its feature extraction neural networks to extract features for each extracted object region and uses an SVM to classify the category of each region. The second type is YOLO, which combines the stage of object region proposal and region classification into one neural network. YOLO applies only one network to divide the input image into regions and predicts bounding boxes and probabilities for each region. Recently, state-of-the-art, Pixel2Seq, applies a language model (LM) to cast the object detection task. Object descriptions ( e.g., coordinate of the bounding boxes and class labels) are expressed as sequences of discrete tokens, and the neural networks are trained to generate the desired sequence. Figure 1 (b) shows the working mechanism of Pixel2Seq. The