Physics-Based Adversarial Attack on Near-Infrared Human Detector for Nighttime Surveillance Camera Systems Conference: ACM MM’23: ACM International Conference on Multimedia; Oct.29–Nov.3, 2023; Ottawa, Canada
Muyao Niu, Zhuoxiao Li, Yifan Zhan, Huy H. Nguyen, Isao Echizen, Yinqiang Zheng · 2024
Read Physics-Based Adversarial Attack on Near-Infrared Human Detector for Nighttime Surveillance Camera Systems Conference: ACM MM’23: ACM International Conference on Multimedia; Oct.29–Nov.3, 2023; Ottawa, Canada by Muyao Niu, Zhuoxiao Li, Yifan Zhan, Huy H. Nguyen, Isao Echizen, Yinqiang Zheng (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-13709.txt.
Introduction
Muyao Niu and co-authors show how night cameras can be fooled in a 2024 security paper. Many watch cameras switch to near-infrared light at night. Images in that light lose color and texture, which leaves AI detectors with weak clues.
The team found that simple tapes stuck on clothes can reshape how bright body parts look to such a camera. Reflective tape and insulation tape bend the light pattern in planned ways. They designed patterns by querying a common human detector, then built them with tape. The fake patterns hid people from the detector in real tests. The work warns that night systems meant to guard sites have real flaws.
Catalog record
| CREATOR | Muyao Niu, Zhuoxiao Li, Yifan Zhan, Huy H. Nguyen, Isao Echizen, Yinqiang Zheng |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-13709
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Physics-Based Adversarial Attack on Near-Infrared Human Detector for Nighttime Surveillance Camera Systems Conference: ACM MM’23: ACM International Conference on Multimedia; Oct.29–Nov.3, 2023; Ottawa, Canada Muyao Niu, Zhuoxiao Li, Yifan Zhan, Huy H. Nguyen, Isao Echizen, Yinqiang Zheng Abstract . Many surveillance cameras switch between daytime and nighttime modes based on illuminance levels. During the day, the camera records ordinary RGB images through an enabled IR-cut filter. At night, the filter is disabled to capture near-infrared (NIR) light emitted from NIR LEDs typically mounted around the lens. While RGB-based AI algorithm vulnerabilities have been widely reported, the vulnerabilities of NIR-based AI have rarely been investigated. In this paper, we identify fundamental vulnerabilities in NIR-based image understanding caused by color and texture loss due to the intrinsic characteristics of clothes’ reflectance and cameras’ spectral sensitivity in the NIR range. We further show that the nearly co-located configuration of illuminants and cameras in existing surveillance systems facilitates concealing and fully passive attacks in the physical world. Specifically, we demonstrate how retro-reflective and insulation plastic tapes can manipulate the intensity distribution of NIR images. We showcase an attack on the YOLO-based human detector using binary patterns designed in the digital space (via black-box query and searching) and then physically realized using tapes pasted onto clothes. Our attack highlights significant reliability concerns for nighttime surveillance systems, which are intended to enhance security. Codes Available: https://github.com/MyNiuuu/AdvNIR . 1. Introduction Surveillance camera systems are widely deployed for security purposes, capturing ordinary RGB images during the daytime while filtering out the infrared component with an IR-cut filter. However, in extremely low-light conditions such as midnight, clear visible images are difficult to obtain due to limited camera sensitivity. To address this, NIR LEDs are often utilized for auxiliary illumination. The central wavelength of these NIR LEDs is typically around 850nm, falling within the spectral response range of silicon-based image sensors. This NIR light is invisible to human eyes and helps to reduce light pollution while concealing surveillance coverage. With the rapid development of deep learning-based visual AI, surveillance cameras are becoming more intelligent, with advanced functionalities of human detection ( Redmon et al. 2016 ; Carion et al. 2020 ) , face recognition ( Wen et al. 2016 ; Schroff et al. 2015 ) , low-light enhancement ( Niu et al. 2023a ; Niu et al. 2023b ) , etc. . The vulnerability of RGB-based vision has also been extensively studied, and countermeasures to protect them from attack have been proposed ( Hu and Shi 2022 ; Duan et al. 2021a ; Hu et al. 2022 ; Suryanto et al. 2022 ; Wang et al. 2022 ; Tan et al. 2021 ) . However, the potential vulnerabilities of NIR-based vision have rarely been explored. Although there have been attempts to use electronic devices worn on the face that emits near-infrared lights to hide human faces from cameras ( Yamada et al. 2013 ; Yamada et al. 2012 ) , the potential vulnerabilities of NIR-based human body detectors have never been reported in existing studies. Given its wide application scenarios in security fields ( e.g. , surveillance cameras), these vulnerabilities may leave a window for attackers to exploit, causing great security issues. In this paper, we bring this critical concern to the community and warn of the potential risks of using AI algorithms in the NIR domain ( Redmon et al. 2016 ; Wu et al. 2017 ; Zhang et al. 2022 ) for existing surveillance systems. By considering the imaging principle of NIR nighttime surveillance camera systems and the installation method of auxiliary lighting, several inherent vulnerabilities make NIR-based AI more brittle and less robust compared to RGB-based AI. Specifically, compared to visible illuminations, the NIR LED illumination around 850 nm has the disadvantage that the color and texture of scene objects captured by the camera tend to disappear (Fig. 1 ), and this phenomenon particularly happens on textures of dyed fabrics such as human clothing ( Wu et al. 2020b ; Liu et al. 2022 ) . The reason lies in the fundamental imaging principle in the NIR range, which can be divided into two aspects. First, the spectral sensitivities of the three color filters of the silicon camera tend to overlap with each other in the NIR region, so monochromatic images can be acquired (Fig. 1 ), though the sensor itself is trichromatic and the scene has brilliant colors. Second, the reflectance spectra of different materials also tend to coincide, resulting in severe texture loss. As a result, the intensity variation of NIR images is very limited, and the robustness of AI models trained on such data tends to be unstable. This implies the immediate vulnerability of AI models when the intensity distribution of the input image is somehow altered. Regarding how to easily manipulate the intensity distribution in the physical world, we further recognize an intrinsic limitation of the geometric location of the auxiliary illuminant and the camera. To save installation space and reduce occlusion, LEDs are usually mounted around the lens (Fig. 2 ), which leads to a nearly co-located geometry setup. Given such a setting, the scene surface perpendicular to the viewing/lighting direction will lead to saturation when encountering certain materials. In reality, we propose to use retro-reflective tape, which by design has a micro surface perpendicular to the viewing/lighting direction, irrespective of the geometric shape of the tape. This can be used to brighten an image area when needed. On the other hand, black plastic tapes (e.g., electrical insulating tape) have low NIR reflectance, allowing to reduce the intensity of an image area. Since retro-reflective tape’s visible color (to human eyes) is independent of its reflectivity in the NIR range, we can use different color combinations to make the physical attacker appear natural to human eyes, further improving its stealthiness. Based on the aforementioned vulnerabilities of NIR and the intensity manipulation via tapes with different materials, we showcase an attack instance on the YOLO-based human detector by first designing binary patterns in the digital space via black-box query and searching, neither requiring knowing the model parameters nor using any gradient-based optimization techniques. We leverage the 3D human models and render-er during the design process to consider real-world physical attributes. As a result, the designed pattern can be physically realized with appropriate tapes pasted onto clothes whose attack ability has been verified. Since retro-reflective tapes and insulation tapes are easily available at very low prices and the cost to make a physical attacker is extremely low, the proposed method will pose tremendous reliability concerns on NIR-based AI, widely introduced in nighttime surveillance systems for security and convenience. The contribution of this work can be summarized as follows: • We disclose the shortage of intensity variation in the NIR domain and analyze its physical mechanism from the perspective of camera spectral response and reflectance of dyed cloth, which finally leads to fundamental vulnerabilities of AI models on such images. • We propose a fully passive yet effective method to manipulate the intensity distribution of NIR images captured by a co-located camera/illuminant system, mainly grounded on the well-known principle of retro-reflective materials. • We show an instance of how to attack the deep-learning based human detector