IAP: Invisible Adversarial Patch Attack through Perceptibility-Aware Localization and Perturbation Optimization
Subrat Kishore Dutta CISPA Helmholtz Center for Information Security [email protected] Xiao Zhang CISPA Helmholtz Center for Information Security [email protected] · 2025
Read IAP: Invisible Adversarial Patch Attack through Perceptibility-Aware Localization and Perturbation Optimization by Subrat Kishore Dutta CISPA Helmholtz Center for Information Security [email protected] Xiao Zhang CISPA Helmholtz Center for Information Security [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06856.txt.
Introduction
A small sticker can flip what a vision model sees, yet loud stickers get spotted and stripped. Subrat Dutta and Xiao Zhang craft stickers that hide in plain sight, in work from twenty twenty-five. Their method first picks a spot the model cares about but the eye skims past.
A tuned loss then shapes the sticker's colors to stay true to the scene. Tests across image sets and model types hit strong success in aimed attacks. Human notice dropped sharply against older stickers. Several top patch defenses also failed to catch them. The work is a warning: stealth, not size, is the patch threat now.
Catalog record
| CREATOR | Subrat Kishore Dutta CISPA Helmholtz Center for Information Security [email protected] Xiao Zhang CISPA Helmholtz Center for Information Security [email protected] |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06856
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
IAP: Invisible Adversarial Patch Attack through Perceptibility-Aware Localization and Perturbation Optimization Subrat Kishore Dutta CISPA Helmholtz Center for Information Security [email protected] Xiao Zhang CISPA Helmholtz Center for Information Security [email protected] Abstract Despite modifying only a small localized input region, adversarial patches can drastically change the prediction of computer vision models. However, prior methods either cannot perform satisfactorily under targeted attack scenarios or fail to produce contextually coherent adversarial patches, causing them to be easily noticeable by human examiners and insufficiently stealthy against automatic patch defenses. In this paper, we introduce IAP, a novel attack framework that generates highly invisible adversarial patches based on perceptibility-aware localization and perturbation optimization schemes. Specifically, IAP first searches for a proper location to place the patch by leveraging classwise localization and sensitivity maps, balancing the susceptibility of patch location to both victim model prediction and human visual system, then employs a perceptibility-regularized adversarial loss and a gradient update rule that prioritizes color constancy for optimizing invisible perturbations. Comprehensive experiments across various image benchmarks and model architectures demonstrate that IAP consistently achieves competitive attack success rates in targeted settings with significantly improved patch invisibility compared to existing baselines. In addition to being highly imperceptible to humans, IAP is shown to be stealthy enough to render several state-of-the-art patch defenses ineffective. 1 Introduction Deep neural networks (DNNs) are highly susceptible to inputs, often known as adversarial examples [ 42 , 12 ] , crafted with small perturbations designed to fool the model. Most existing attacks constrain perturbations using \ell_{p} -norm bounds to ensure imperceptibility [ 12 , 31 , 30 , 4 ] , where any pixel of the entire input image is allowed to be modified to achieve the attack goal. Imposing an \ell_{p} -norm constraint limits perturbation size, ensuring generated perturbations remain visually imperceptible to humans. In contrast, a different but related line of work investigated adversarial patch attacks [ 2 , 21 ] , which modify localized regions without magnitude restrictions. The general framework of this approach facilitates adaptability to physical-world scenarios, increasing threats to security-critical machine learning systems. Nevertheless, adversarial patches, though applicable in physical space, often attract suspicion due to their highly visible, salient nature and inability to maintain homogeneity with the host image. To overcome this, several techniques [ 9 , 40 , 26 , 45 ] have been proposed to generate adversarial patches that mimic realistic images, ensuring their placement does not raise suspicion. These approaches focus on maintaining context homogeneity and high realism, which are crucial for achieving imperceptibility to human perception. Despite these advancements, the perturbations required to sustain high attack efficacy, particularly in targeted scenarios, often result in large patches that are still visually noticeable. Defense mechanisms [ 27 , 5 , 51 , 43 , 10 , 20 ] have also been developed that leverage the contiguous high saliency of the patch region, proving successful in detecting adversarial patches. The strict constraints on patch size and the need to avoid saliency make balancing stealth and targeted attack success particularly difficult. Therefore, a natural question arises: Are targeted attack goals at all achievable with visually imperceptible adversarial patches? In this paper, we answer the above question affirmatively by developing a general perceptibility-aware framework that can generate invisible patches with strong attack capabilities. Below, we summarize the key contributions of our work. Figure 1 : The overall pipeline of IAP for conducting targeted attacks with imperceptible adversarial patches, consisting of both patch localization and iterative patch update blocks. Contributions. We propose IAP 1 1 1 The code is available at https://github.com/subratkishoredutta/IAP . , a novel patch attack framework based on a series of perceptibility-aware optimization schemes, which can realize targeted attack goals with I nvisible A dversarial P atches (see Figure 1 for the overall pipeline). Most prior work overlooked human visual sensitivity in patch placement and perturbation, a key factor in our design’s success. Specifically, IAP first locates a patch region in the host image that optimally balances the class localization and sensitivity scores, enabling a strategic advantage in both attack capabilities and imperceptibility (Section 4.1 ). After the localization step, it iteratively updates the patch by restricting the changes in base color and regularizing the adversarial loss using a human perception-based distance, which reduces the saliency of the resulting patch and further promotes imperceptibility (Section 4.2 ). Through extensive experiments on benchmark datasets for image classification and recognition, we show that IAP matches or surpasses state-of-the-art patch attacks in success rate while significantly enhancing invisibility across various targeted scenarios (Section 5.2 ). Moreover, we demonstrate that adversarial patches generated by IAP can successfully bypass various patch defenses, confirming the stealthiness of our attack (Section 5.4 ). The high stealthiness and strong capabilities of the adversarial patches generated by our method call for an urgent need to develop better defense strategies to detect and mitigate such stealthy attacks. 2 Related Work Adversarial Patches. Brown et al. 2017 and Karmon et al. 2018 showed that small yet prominent adversarial patches can mislead classifiers, with the former introducing universal patches and the latter extending them to image-specific attacks. Later works focused on blending patches into their environment for stealth. Eykholt et al. 2018 concealed perturbations in traffic signs, while Sharif et al. 2016 , Liu et al. 2019 , and Li and Ji 2021 leveraged generative models for adversarial eyeglasses, high-fidelity stickers, and GDPA-based soft-masked blending. Zolfi et al. 2021 used blending to generate translucent patches for detection tasks, and VRAP [ 45 ] refined perturbations using \ell_{p} -norm constraints [ 30 ] . Some aimed for complete invisibility, with Bai et al. 2021 using cascaded GANs at high cost and Qian et al. 2020 leveraging perceptual sensitivity but lacking spatial constraints. However, balancing stealth and attack efficacy remains a challenge, limiting most to untargeted attacks. Imperceptibility in Adversarial Examples. Imperceptibility in adversarial attacks ensures that adversarial examples closely resemble the original image, making them undetectable to humans. The human visual system is less sensitive to variations in high-textured regions than to perturbations on edges [ 25 ] . Studies have shown that perturbations in high-variance areas are less noticeable [ 29 ] , while restricting modifications to horizontal and vertical edges enhances imperceptibility [ 6 ] . Additionally, altering only saturation and brightness to mimic different quantized levels of the same base color can effectively camouflage perturbations in textured regions [ 6 , 25 , 8 ] . Common noise constraints, such as \epsilon -budgets and \ell_{p} -norm regularization, help limit perturbation magnitudes [ 42 , 12 , 31 , 30 , 9 ] , but they do not account for human perception. To address this, Luo et al. 2018 introduced a perception-aware distance metric that considers human sensitivity when curating perturbations. Given the space constraints of adversarial patch attacks, achieving targeted goals with limited perturbation is non-trivial. With our proposed method, IAP, we argue that human perception-oriented strategies, primarily studied in perturbation-restricted global attacks, are essential to store strategic large perturbations, which we hypothesize are crucial for stronger and stealthier targeted adversarial patch attacks. Adversarial Examples in Face Recognition. Adversarial attacks on facial recognition systems have shown that even subtle perturbations can undermine system reliability. Goswami et