Optimization-Free Patch Attack on Stereo Depth Estimation
Hangcheng Liu, Xu Kuang, Xingshuo Han, Xingwan Wu, Haoran Ou, Shangwei Guo, Xingyi Huang, Tao Xiang, Tianwei Zhang · 2025
Read Optimization-Free Patch Attack on Stereo Depth Estimation by Hangcheng Liu, Xu Kuang, Xingshuo Han, Xingwan Wu, Haoran Ou, Shangwei Guo, Xingyi Huang, Tao Xiang, Tianwei Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17632.txt.
Introduction
Hangcheng Liu and co-authors attack the stereo depth systems that self-driving cars use to judge distance. Earlier attacks tweak pixels one by one, which ties them to a single camera view and a lab setting. The team first built a test frame that stretches such attacks across the four stages of stereo matching, from feature extraction to depth output. Across nine depth models, the old attacks transferred poorly between systems.
Their new attack, PatchHunter, skips pixel tuning entirely. It searches a structured space of printed patterns that break the assumptions stereo matching rests on, guided by a reinforcement learning policy. On the KITTI driving dataset it beat pixel attacks in strength and in black-box transfer. In a simulator and on real cars it held up in poor light, where pixel attacks faded to nothing.
Catalog record
| CREATOR | Hangcheng Liu, Xu Kuang, Xingshuo Han, Xingwan Wu, Haoran Ou, Shangwei Guo, Xingyi Huang, Tao Xiang, Tianwei Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 75 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-17632
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Optimization-Free Patch Attack on Stereo Depth Estimation Hangcheng Liu, Xu Kuang, Xingshuo Han, Xingwan Wu, Haoran Ou, Shangwei Guo, Xingyi Huang, Tao Xiang, Tianwei Zhang Abstract Stereo Depth Estimation (SDE) is a core technique for scene perception in vision-based systems, particularly in autonomous driving. Recent studies have revealed that SDE models are vulnerable to optimization-based adversarial attacks. However, these attacks are often confined to impractical settings, such as separately perturbing stereo views, restricted to the digital level, and static scenes, rendering them ineffective in real-world scenarios. A fundamental question remains: how can we design physically deployable, scene-adaptive, and transferable attacks against SDE models under realistic constraints? We make two key contributions to address the above question. First, we construct a unified attack framework that extends optimization-based attack techniques to four key stages of stereo matching, i.e., feature extraction, cost-volume construction, cost aggregation, and disparity regression. Leveraging this framework, we perform a comprehensive stage-wise evaluation of adversarial patches across 9 mainstream SDE models under realistic constraints (e.g., photometric consistency). Our evaluations show that optimization-based patches consistently suffer from poor transferability. Yet, a subset of patches exhibiting partial transferability leads to a valuable insight: patterns may contribute more to transferable adversarial attacks. Inspired by this, we introduce PatchHunter , the first optimization-free adversarial patch attack against SDE. PatchHunter formulates patch generation as a search task over a structured space of visual patterns designed to disrupt key SDE assumptions. Guided by reinforcement learning policy, PatchHunter efficiently navigates this large space to discover effective and transferable adversarial patterns. We evaluate PatchHunter across three levels, i.e., autonomous driving dataset, high-fidelity simulator, and real-world deployment, to comprehensively assess its effectiveness, transferability, and practicality. On the KITTI dataset, PatchHunter achieves superior attack effectiveness over optimization-based methods, while significantly improving transferability under black-box scenarios. Further evaluations in the CARLA simulator and on a vehicle equipped with industrial-grade stereo cameras highlight that PatchHunter is resilient to physical-world variations. Even under challenging conditions, such as low lighting, PatchHunter can achieve a D1-all error (ranging from 0 to 1) exceeding 0.4, whereas optimization-based attacks yield results close to 0. I Introduction Vision-based depth estimation infers scene depth information using RGB cameras, offering a low-cost and lightweight method in automated platforms such as autonomous driving systems [ 1 , 2 , 3 , 4 , 5 ] . Existing vision-based methods can be categorized as Monocular depth estimation (MDE) [ 6 ] and Stereo depth estimation (SDE) [ 7 ] . MDE estimates depth from a single camera image by learning semantic and geometric priors from data. While attractive for its minimal hardware requirements, MDE suffers from fundamental limitations, including scale ambiguity and poor generalization to unseen environments [ 8 , 9 , 10 ] . In contrast, SDE emulates human binocular vision by analyzing disparities between two images captured from horizontally displaced viewpoints. By leveraging explicit geometric constraints, SDE produces more accurate depth estimates across diverse scenes [ 11 , 12 , 13 ] . Recent studies [ 14 , 15 , 16 , 17 ] have shown that SDE is susceptible to adversarial attacks, posing a significant threat to its reliability in safety-critical applications. These works typically employ optimization-based strategies to craft perturbations that manipulate the predicted disparities, potentially causing critical perception failures. However, despite promising results in controlled settings, these attacks remain limited by impractical assumptions, restricting their applicability in real-world scenarios. Specifically, most works [ 14 , 15 , 16 ] focus on digital-level attacks by applying different digital perturbations to the left and right views separately. This design violates the photometric consistency constraint inherent to stereo vision in the real world. Moreover, they typically require perturbations to be injected into the image streams in real time, making the physical-world deployment infeasible. Liu et al. [ 17 ] considered the photometric consistency and deployed their patch in the real world. However, it is only evaluated on static images, while its effectiveness, transferability, and robustness in dynamic, real traffic scenarios remain unverified. These limitations raise some critical questions: is the optimization-based strategy still promising to generate adversarial patches that are deployable, adaptive to various scenes, and transferable across diverse SDE models? If not, what other strategies can we use to effectively satisfy these attack requirements? Motivated by these questions, our work delivers the following novel contributions: (1) We construct the first optimization-based adversarial attack framework that systematically generates adversarial patches by targeting the four key stages of stereo matching. This stage-wise attack design is motivated by the insight that the stereo matching module in SDE involves four interdependent stages: feature extraction, cost volume construction, cost aggregation, and disparity generation, that collectively determine the final depth output (Figure 1 ). Disrupting any of these stages may lead to the failure of depth estimation. Our framework not only covers existing attacks, such as those targeting disparity generation for effectiveness [ 18 , 17 ] and feature extraction for transferability [ 16 ] , but also identifies two previously underexplored stages, cost volume construction and cost aggregation, as new attack surfaces. Perturbing these intermediate stages can achieve comparable adversarial effects to those on the previously explored stages. We comprehensively evaluate different attacks originating from this framework on the KITTI dataset [ 1 ] using nine representative SDE models. This leads to two important findings. 1) Optimization-based attacks struggle to achieve high transferability under practical constraints such as photometric consistency [ 18 ] and scene adaptability [ 15 ] . Among the 72 source-target model pairs formed from 9 SDE models, adversarial patches generated at the feature extraction, cost volume construction, cost aggregation, and disparity regression stages show poor transferability in 85% (61 pairs), 92% (66 pairs), 100% (72 pairs) and 100% (72 pairs) of the cases, respectively (Table III ), indicating severe cross-model transferability limitations. Although attacks targeting feature extraction show relatively better transferability, the advantage is marginal, challenging the optimistic conclusions drawn in [ 16 ] . 2) Repetitive textures offer a promising path to improving transferability. We find that transferable patches share a common trait: visually repetitive textures across different regions. This pattern exploits a fundamental architecture-independent weakness in SDE: the reliance on local matching consistency. Repetitive textures naturally introduce multiple similar candidate matches along the epipolar line, which increases correspondence ambiguity and leads to a noisier disparity distribution, ultimately making such a pattern more transferable than model-specific adversarial textures. Guided by this insight, we revise our framework: instead of optimizing a full patch, we optimize a small texture block and tile it to create explicit self-similar textures. Surprisingly, this simple change significantly improves transferability across all four stages (Figure 5 ). Although 19%–29% source-target model pairs remain non-transferable, the results underscore a promising direction: enhancing attack transferability can be achieved by leveraging structural vulnerabilities in SDE. (2) We propose the first optimization-free adversarial attack framework, named PatchHunter , which efficiently searches for adversarial patterns within a predefined space to attack SDE. These patterns directly determine the visual appearance of the adversarial patch, without requiring any additional texture optimization or gradient-based fine-tuning. To achieve this, we first construct a Pattern Primitive Dictionary (PPD) that encodes 9 basic visual attributes, including patch shape, hollowness, block repetition, brightness, color, texture type, frequency,