From Pixels to Reality: Physical-Digital Patch Attacks on Real-World Camera
Victoria Leonenkova, Ekaterina Shumitskaya, Dmitriy Vatolin, Anastasia Antsiferova · 2026
Read From Pixels to Reality: Physical-Digital Patch Attacks on Real-World Camera by Victoria Leonenkova, Ekaterina Shumitskaya, Dmitriy Vatolin, Anastasia Antsiferova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28425.txt.
Introduction
Leonenkova and three co-authors present DiPA, an attack that fools face recognition cameras with a patch shown on a phone screen. Printed patches are the usual route; a screen patch deploys fast and needs no print-friendly smoothing. An ensemble of face models makes the patch transfer to unseen commercial systems. In a live demo, the attack hid authorized users from a working camera in real time. It beat older physical attacks on success rate and confidence drop.
Catalog record
| CREATOR | Victoria Leonenkova, Ekaterina Shumitskaya, Dmitriy Vatolin, Anastasia Antsiferova |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 12287 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.28425
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Pixels to Reality: Physical-Digital Patch Attacks on Real-World Camera Thanks: This work was supported by the The Ministry of Economic Development of the Russian Federation in accordance with the subsidy agreement (agreement identifier 000000C313925P4H0002; grant No 139-15-2025-012). Victoria Leonenkova, Ekaterina Shumitskaya, Dmitriy Vatolin, Anastasia Antsiferova Abstract This demonstration presents Digital-Physical Adversarial Attacks (DiPA) , a new class of practical adversarial attacks against pervasive camera-based authentication systems, where an attacker displays an adversarial patch directly on a smartphone screen instead of relying on printed artifacts. This digital-only physical presentation enables rapid deployment, removes the need for total-variation regularization, and improves patch transferability in black-box conditions. DiPA leverages an ensemble of state-of-the-art face-recognition models (ArcFace, MagFace, CosFace) to enhance transfer across unseen commercial systems. Our interactive demo shows a real-time dodging attack against a deployed face-recognition camera, preventing authorized users from being recognized while participants dynamically adjust patch patterns and observe immediate effects on the sensing pipeline. We further demonstrate DiPA’s superiority over existing physical attacks in terms of success rate, feature-space distortion, and reductions in detection confidence, highlighting critical vulnerabilities at the intersection of mobile devices, pervasive vision, and sensor-driven authentication infrastructures. Fig. 1: The overview of the proposed demonstration. The user uploads a photo to the server and receives a set of digital adversarial patches designed to cause dodging physical attack for a real camera in a completely black-box setting, along with images for white-box attacks on online face recognition models. I Introduction and Related Work Pervasive computing environments increasingly rely on vision-based authentication, particularly face recognition (FR), to enable frictionless access control for smart buildings, workplaces, homes, and public infrastructures. As FR modules become embedded in IoT devices, smart cameras, edge processors, and mobile systems, their security becomes a critical requirement for trustworthy pervasive computing ecosystems. Unfortunately, modern FR models—whether deployed on-device or in the cloud—remain vulnerable to adversarial attacks. Such attacks can be launched either digitally (e.g., manipulating input images before they reach the model) or physically (e.g., using adversarial objects or printed patterns visible to a camera). Physical adversarial attacks are especially concerning in pervasive systems because they bypass digital safeguards and exploit the uncontrolled sensing pipeline, which includes illumination variations, sensor noise, optics, autofocus, and proprietary model preprocessing. Defense mechanisms for these attacks remain limited, and testing is often performed in purely digital setups that fail to represent complex real-world camera behavior. Several digital [ 6 ] and physical [ 11 , 5 ] attacks on FR models have been proposed. However, their evaluations rarely include commercial, closed‑source camera systems—a setting highly relevant to deployed intelligent environments. Some early works incorporate cameras [ 4 , 9 ] , but real-world transferability is not their main target, and the adversarial patterns are typically printed, requiring careful color calibration and offering limited flexibility. In pervasive scenarios, users constantly carry a smartphone , enabling both routine and potentially harmful interactions with camera systems. Surprisingly, no prior physical attack leverages the smartphone display itself as an adversarial surface, despite its ubiquity, mobility, brightness control, and dynamic rendering capabilities. This demonstration presents DiPA , the first adversarial attack designed for digital-physical deployment on smartphone screens, enabling practical dodging attacks against real-world FR cameras. Our contributions include: • Novel attack modality for pervasive environments : We introduce the first adversarial patch that is displayed directly on a smartphone screen and is immediately usable in real-world physical spaces, aligning with pervasive contexts where mobile devices interact continuously with sensing infrastructures. • A method for generating high-resolution adversarial patches that successfully transfer from surrogate FR models to commercial cameras, even in a strict black-box setting. • A working end-to-end system , accessible through a web interface, that allows participants to upload a picture and receive personalized adversarial patches for online and physical FR systems. • Comprehensive evaluation across commercial APIs and a real camera, demonstrating that DiPA and DiPA TV outperform prior physical and digital attacks in dodging performance. II Approach Our work focuses on a dodging attack designed to prevent the successful verification of an authorized user already enrolled in a face recognition (FR) system. Such attacks are highly relevant to pervasive environments where FR-enabled devices (e.g., smart doors, kiosks, embedded access-control terminals) operate autonomously at the edge and must handle uncontrolled user behavior and mobile-device interactions. Let \mathcal{F}:\mathbb{R}^{H\times W\times 3}\rightarrow\mathbb{R}^{d} denote an FR model that maps an input face image x to an embedding vector. Given an enrolled identity with reference embedding s=\mathcal{F}(x_{\text{ref}}) , the goal of the dodging attack is to generate an adversarial patch p\in\mathbb{R}^{D\times D\times 3} that minimizes the cosine similarity between \mathcal{F}(A(x,p)) —where A(\cdot) is a patch-application operator—and the reference embedding s : p^{*}=\arg\min_{\mathbf{p}}\sum_{i=1}^{N}\cos\!\left(\mathcal{F}_{i}\!\left(A\left(\mathbf{x},\text{MedianPool}(\mathbf{p})\right)\right),\mathcal{F}_{i}(\mathbf{x})\right), (1) where \{\mathcal{F}_{i}\}_{i=1}^{N} is the ensemble of surrogate FR models (ArcFace [ 2 ] , CosFace [ 10 ] , MagFace [ 8 ] ) and \mathbf{x} is the attacker’s face image. Direct optimization against a real-world camera is infeasible due to the lack of gradients and strict limitations on the number of queries. Therefore, we rely on transferability from surrogate models. Since these models accept low-resolution inputs ( 3\times 112\times 112 ), generating high-resolution physically deployable adversarial patches is challenging. To address this, we introduce a median-pooling mechanism that allows the patch to be trained at high resolution while still matching model constraints. II-A Regularization Variants We study two versions of the proposed method: • DiPA+TV : Uses Total Variation (TV) regularization [ 7 ] to enforce spatial smoothness: TV(p)=\sum_{i,j}\sqrt{(p_{i,j}-p_{i+1,j})^{2}+(p_{i,j}-p_{i,j+1})^{2}}. (2) • DiPA : Removes TV regularization entirely. Because our patches are displayed digitally on smartphone screens (rather than printed), enforcing smoothness is less critical, and removing it increases the expressive power of the patch. II-B Evaluation Across Digital–Physical and Real Pervasive Settings We compare DiPA with existing attacks on two online FR systems (Face++ [ 3 ] , MobileFaceNet [ 1 ] ) and a commercial off-the-shelf FR camera operating as a strict black box. For online models, we compute cosine similarity between embeddings of clean and patched images; lower values indicate stronger dodging performance. For the real-world camera, we use two metrics: 1. Attack Success Rate (ASR) : ASR=\frac{1}{N}\sum_{i=1}^{N}\mathbb{I}\!\left(\mathcal{F}(A(x_{i},p))\neq\mathcal{F}(x_{i})\right), (3) where N is the number of trials. The camera system reports the predicted identity \mathcal{F}(\cdot) directly, so this metric measures how often the presence of the smartphone screen with the adversarial patch causes the system to output an identity different from that of the authorized user. 2. Mean confidence of face presence , which ensures the attack does not disrupt the face-detection module—an important property for pervasive deployments, where detection and recognition are typically decoupled embedded subsystems. II-C Experimental Protocol Five volunteers (three female, two male) participated in our evaluation. For each participant and for each method under comparison, we generated five personalized adversarial patches. Every patch was tested in five independent trials, with the smartphone positioned near the mouth following the setup used in our demonstration. Table I summarizes all results. TABLE I: Comparison with existing adversarial patch approaches in a digital–physical setup. Patches are displayed on a smartphone screen and tested against the Face++ [ 3 ] system, MobileFaceNet (MF-Net) [ 1 ] , and a real-world FR camera. Face++ MF-Net Real Camera Method Sim. \downarrow Sim. \downarrow