AdvIRL: Reinforcement Learning-Based Adversarial Attacks on 3D NeRF Models

Tommy Nguyen, Mehmet Ergezer, Christian Green · 2024

Read AdvIRL: Reinforcement Learning-Based Adversarial Attacks on 3D NeRF Models by Tommy Nguyen, Mehmet Ergezer, Christian Green (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-16213.txt.

Introduction

Tommy Nguyen, Mehmet Ergezer, and Christian Green introduced this 2024 method for attacking 3D scene models. Neural radiance fields build lifelike 3D views from photos and now serve in key applications. Their safety holes are far less studied than those of flat image models. The authors use reinforcement learning to craft poisoned versions of such models.

The crafted noise survives spinning and resizing of the scene, which makes black-box attacks realistic. Tests ran from small objects like bananas to whole lighthouse scenes. Targeted attacks made classifiers call a banana a slug and a truck a cannon with high confidence. The poisoned models can also serve as training fodder to toughen vision systems. Code is public for checking and follow-up work.

Catalog record

CREATORTommy Nguyen, Mehmet Ergezer, Christian Green
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-16213

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AdvIRL: Reinforcement Learning-Based Adversarial Attacks on 3D NeRF Models Tommy Nguyen, Mehmet Ergezer, Christian Green Abstract The increasing deployment of AI models in critical applications has exposed them to significant risks from adversarial attacks. While adversarial vulnerabilities in 2D vision models have been extensively studied, the threat landscape for 3D generative models, such as Neural Radiance Fields (NeRF), remains underexplored. This work introduces AdvIRL , a novel framework for crafting adversarial NeRF models using Instant Neural Graphics Primitives (Instant-NGP) and Reinforcement Learning. Unlike prior methods, AdvIRL generates adversarial noise that remains robust under diverse 3D transformations, including rotations and scaling, enabling effective black-box attacks in real-world scenarios. Our approach is validated across a wide range of scenes, from small objects (e.g., bananas) to large environments (e.g., lighthouses). Notably, targeted attacks achieved high-confidence misclassifications, such as labeling a banana as a slug and a truck as a cannon, demonstrating the practical risks posed by adversarial NeRFs. Beyond attacking, AdvIRL -generated adversarial models can serve as adversarial training data to enhance the robustness of vision systems. The implementation of AdvIRL is publicly available at https://github.com/Tommy-Nguyen-cpu/AdvIRL/tree/MultiView-Clean , ensuring reproducibility and facilitating future research. Introduction The widespread integration of artificial intelligence (AI) across fields such as autonomous driving and robotics ( Li, Chen, and Shen 2019 ; Ravindran, Santora, and Jamali 2020 ; Zhang et al. 2020 ; Hu et al. 2019 ; Hossain, Capi, and Jindai 2016 ; Finn et al. 2015 ) has led to an increase in incidents where AI models misclassify objects ( Kurakin, Goodfellow, and Bengio 2017 ) . These errors can have severe consequences, including the loss of life, particularly in safety-critical systems like AI-driven vehicles. To mitigate such risks, research on adversarial AI has expanded, aiming to expose vulnerabilities in AI algorithms and strengthen their robustness against adversarial attacks ( Xu et al. 2020 ; Behzadan and Munir 2017 ; Goodfellow, Shlens, and Szegedy 2015 ) . While adversarial attacks on 2D vision models have been widely studied ( Li, Lian, and Chen 2023 ; Huang et al. 2023 ; Madry et al. 2017 ) , the transition to the physical world introduces new challenges. Many real-world attacks rely on 3D transformations—such as changes in angle, scale, and lighting—which 2D approaches cannot adequately capture. Further, many existing approaches assume white-box access to the model’s architecture and gradients ( Sarkar et al. 2023 ; Lin et al. 2020 ; Andriushchenko and Flammarion 2020 ; Dong et al. 2018 ; Hull, Wang, and Chau 2024 ) . In contrast, real-world adversarial attacks are often black-box in nature, where attackers do not have direct access to the model but instead exploit vulnerabilities through inputs captured by external devices like cameras ( Athalye et al. 2018 ; Sharif et al. 2019 ; Ergezer et al. 2024 ) . Recent efforts have begun exploring adversarial attacks on 3D objects ( Li, Lian, and Chen 2023 ) , but this domain remains underdeveloped. Applying adversarial noise to 3D objects is more resource-intensive, often requiring high-quality 3D models and complex manipulation of textures. Moreover, existing datasets for 3D adversarial research are sparse compared to their 2D counterparts, creating a bottleneck for advancements. Although recent studies have explored the 3D adversarial landscape using rendering techniques such as Gaussian Splatting ( Zeybey, Ergezer, and Nguyen 2024 ) , these investigations remain largely limited to white-box settings. AdvIRL innovatively overcomes these limitations, relying solely on the input and output of the target model to generate effective adversarial attacks. This gradient-free approach simplifies the process and sidesteps challenges associated with adversarial transferability in existing methods. In the context of this research, we focus on digital adversarial attacks targeting machine learning vision systems. These attacks simulate how carefully crafted perturbations can manipulate neural rendering models, potentially compromising perception in critical applications like autonomous vehicles, surveillance systems, and robotic navigation. Practical Scenario Example: Consider a virtual reality (VR) training application for first responders, where environments like urban landscapes or rescue scenes are reconstructed using Neural Radiance Fields (NeRFs). The system must render objects accurately from various angles and distances to ensure realism and usability. An adversarial attack on such a system could introduce imperceptible yet impactful distortions, disrupting the visual consistency and undermining the training’s effectiveness. Our AdvIRL framework addresses this challenge by generating robust adversarial noise that maintains its efficacy across diverse transformations, allowing for more consistent 3D renderings that minimize distortions and visual inconsistencies when using the adversarial noise generated by AdvIRL during training. AdvIRL can also be used to enhance the security of 3D object detection algorithms used in various applications, such as security cameras and advanced tolling systems. By training these algorithms on adversarial noise, they become more resilient to both intentional (perturbations) and unintentional attacks (adversarial rotation). This can help protect sensitive information and improve the accuracy of these systems. In this work, we propose AdvIRL , a novel black-box reinforcement learning framework that generates adversarial 3D models using a state-of-the-art NeRF architecture ( Mildenhall et al. 2020 ) . Unlike previous approaches that assume model access, AdvIRL operates in a black-box setting, leveraging Instant Neural Graphics Primitives (Instant-NGP) ( Müller et al. 2022 ) to introduce adversarial noise resilient to diverse transformations. Furthermore, by incorporating image segmentation ( Wu et al. 2019 ) , our method can target specific objects within a scene, allowing for more precise and controlled adversarial attacks. Our approach can be utilized to significantly improve the robustness of vision models when adversarial NeRF models are used for adversarial training. This work introduces an innovative approach to black-box adversarial robustness for 3D vision models, with the following key contributions: 1. We propose AdvIRL , the first black-box adversarial framework for Neural Radiance Fields (NeRF). Our approach leverages reinforcement learning and Instant Neural Graphics Primitives (Instant-NGP) to efficiently generate robust adversarial noise under diverse 3D transformations, such as rotations, scaling, and viewpoint changes in AI vision models. To our knowledge, AdvIRL is the first black-box adversarial framework for targeting 3D vision models. 2. We demonstrate the efficacy of AdvIRL across a wide range of scenes, from small-scale objects (e.g., transforming a banana’s classification) to large environmental scenes (e.g., manipulating lighthouse representations). Our targeted attacks achieved high-confidence misclassifications (e.g., labeling a banana as a slug and a truck as a cannon), highlighting the practical risks posed by adversarial NeRFs. Method In this section, we introduce our proposed framework, AdvIRL , designed to generate adversarial noise for 3D models. Unlike traditional adversarial methods, our framework operates on a higher dimension leading to more robust results. We describe the components apart of our AdvIRL pipeline in the following subsections. The adversarial noise generation process comprises four primary steps. First, input images are processed through an image segmentation algorithm, generating segmented images X . Secondly, segmented images X are classified using the Contrastive Language-Image Pretraining (CLIP) model to ensure only correctly classified images are used ( Radford et al. 2021 ) . Next, Instant-NGP is employed to render the 3D object from various viewpoints, creating snapshots of the NeRF model. Finally, AdvIRL is configured with appropriate parameters to generate adversarial examples from different perspectives. Figure 1 illustrates how AdvIRL adjusts the parameters of Instant-NGP to generate

Read the complete text