RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses

1 st Mohamed Djilani, 2 nd Salah Ghamizi †, 3 rd Maxime Cordy · 2024

Read RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses by 1 st Mohamed Djilani, 2 nd Salah Ghamizi †, 3 rd Maxime Cordy (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-20987.txt.

Description

Abstract Although adversarial robustness has been extensively studied in white-box settings, recent advances in black-box attacks (including transfer- and query-based approaches) are primarily benchmarked against weak defenses, leaving a significant gap in the evaluation of their effectiveness against more recent and moderate robust models (e.g., those featured in the Robustbench leaderboard).

Catalog record

CREATOR1 st Mohamed Djilani, 2 nd Salah Ghamizi †, 3 rd Maxime Cordy
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE106 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-20987

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defenses 1 st Mohamed Djilani, 2 nd Salah Ghamizi †, 3 rd Maxime Cordy Abstract Although adversarial robustness has been extensively studied in white-box settings, recent advances in black-box attacks (including transfer- and query-based approaches) are primarily benchmarked against weak defenses, leaving a significant gap in the evaluation of their effectiveness against more recent and moderate robust models (e.g., those featured in the Robustbench leaderboard). In this work, we argue that this gap is problematic and the previous benchmarks conclusions do not hold under more robust evaluation frameworks, leading to contradicting conclusions on the transferability of adversarial attacks. We extensively evaluate the effectiveness of 13 popular black-box attacks, representative of the top ten popular transferability theories. We benchmark these attacks against eight top-performing and standard defense mechanisms on the ImageNet dataset. Our empirical evaluation reveals the following key findings: (1) the most advanced black-box attacks struggle to succeed even against simple adversarially trained models; (2) robust models that are optimized to withstand strong white-box attacks, such as AutoAttack, also exhibit enhanced resilience against black-box attacks; and (3) robustness alignment between the surrogate models and the target model can significantly impact the success rate of transfer-based attacks. $\dagger$ $\dagger$ footnotetext: This work was conducted while at Luxembourg Institute of Health. I Introduction TABLE I: Theories about the transferability of adversarial examples. In bold when the evaluation protocol of the black-box is sufficiently robust . We report the largest image dataset, the smallest perturbation distance, and the deep learning model families used in the evaluation. FNN for feed-forward neural network, CNN for convolutional neural network Theory Dataset Representative attacks Attack budgets Target architectures Target defenses Linearity [ 1 ] ImageNet FGSM L_{2}=0.25 Convolution FGSM AT Decision Boundary Similarity [ 2 ] ImageNet FGSM L_{0}=23 Convolution \varnothing Gradient Similarity [ 3 ] MNIST FGSM L_{2}=1 FNN \varnothing Adversarial Subspace [ 4 ] MNIST FGSM L_{2}=4 FNN, CNN FGSM AT Non-Robust Features [ 5 ] ImageNet PGD L_{2}=3 CNN PGD AT Model complexity [ 6 ] ImageNet IFGSM L_{2}=15 CNN \varnothing Perturbation interactions [ 7 ] ImageNet PGD L_{\infty}=16/255 CNN FGSM AT Spectral properties [ 8 ] CIFAR-10 PGD \mathbf{L_{\infty}=2/255} FNN, CNN PGD AT Knowledge transferability [ 9 ] STL-10 PGD \mathbf{L_{2}=2/255} CNN \varnothing Generalization gap [ 10 ] CIFAR-100 PGD L_{\infty}=8/255 CNN \varnothing Proposed protocol ImageNet AutoAttack, BASES \mathbf{L_{\infty}=4/255} CNN, Transformers Robustbench Since the discovery that deep learning models are susceptible to minor input disturbances, resulting in adversarial examples (AE) [ 1 ] , the development of robust models has become one of the most active topics in the machine learning community. This topic has been thoroughly explored for white-box settings, settings where attackers have full knowledge of the target, leading to powerful standardized attacks [ 11 ] and benchmarks [ 12 ] . In parallel, more ”realistic” studies of adversarial vulnerability involved restricting the knowledge or the capabilities of the attacker. These scenarios are referred to as black-box scenarios (or gray-box when partial knowledge of the target is available [ 13 ] ). State-of-the-art (SoTA) black-box attacks leverage one or both of the following mechanisms: (1) adversarial examples transferability and (2) iterative queries with meta-heuristics. Transferability methods craft adversarial examples on a specifically built surrogate model (with white-box access) such that this example should transfer to (i.e. also fools) a target model (to which they have black-box access). Iterative query methods use query feedback from the target model to guide or refine the optimization process to uncover its vulnerabilities. Black-box attacks are powerful tools to study real-world attack scenarios and to better understand the mechanisms behind adversarial vulnerabilities of deep learning models, in particular through the study of attacks’ transferability. However, they still rely on outdated and unstandardized evaluation practices (cf. Table I ) . In contrast to white-box research, recent black-box attacks are still designed and evaluated using large epsilon noise budgets, small architectures, and non-robust target models. Traditional theories such as the decision boundary similarity [ 2 ] , the adversarial subspace theory [ 4 ] , and the non-robust features [ 5 ] are still well recognized, but have been demonstrated using simple transfer attacks, with very large budgets and non-robust models. In contrast, Robustbench [ 12 ] recommended much stronger models and enforced a budget of \epsilon=4/255 as a reasonable evaluation budget. At least ten of the main theories have been empirically demonstrated using incomplete black-box attacks protocols (Table I ): linearity [ 1 ] , gradient similarity [ 3 ] , adversarial subspace [ 4 ] , non-robust features [ 5 ] , model complexity [ 6 ] , adversarial perturbation interactions [ 7 ] , spectral properties [ 8 ] , knowledge transferability [ 9 ] , generalization gap [ 10 ] , and decision boundary similarity [ 2 , 14 ] . Our work does not intend to challenge all previous work and theories about the vulnerability of deep learning models to adversarial attacks; indeed, many theories were evaluated with the best protocols at that time. We would rather like to demonstrate that adhering to more stringent evaluation protocols leads to different insights and conclusions on the vulnerability of the models and effectiveness of the black-box attacks. In addition, given the importance of black-box attacks in assessing real-world robustness of ML systems put in production, we also hypothesize that the previous evaluation results of black-box attacks can be misleading. Even old robustification mechanisms enable us to successfully evade supposedly SoTA black-box attacks. LGV [ 15 ] , one of the strongest transfer attacks, sees its success rate drop from 87.39\% to 2.17\% under a 2020 defense by Salman et al. [ 16 ] . Our first insight suggests to reconsider previous claims of effectiveness of black-box attacks and demonstrates that simple adversarial training is sufficient against the majority of SoTA black-box attacks. While our first insight challenges the effectiveness of SoTA attacks, we tackle the benefits of SoTA defenses in our second study. We evaluated 9 defenses from Robustbench white-box ImageNet benchmark [ 17 ] and categorized these defenses into four families. Our analysis demonstrates that within the same family of defenses, architectures and model size have limited impact on the effectiveness of defenses. Our main insight from this study is that SoTA defenses are starting to plateau and show diminishing returns against SoTA black-box attacks. Finally, our work argues that an incomplete evaluation of the interactions between new adversarial defenses and black-box attacks can actually lead new defenses to strengthen already existing black-box attacks, opening new surface attacks that were not considered before. For example, BASES [ 18 ] can leverage recent robust models from Liu et al. [ 19 ] to increase its effectiveness against the most robust models from 1.26\% to 12.55\% . In fact, we analyzed the top 20 defenses in the Robustbench ImageNet leaderboard ( L_{\infty} , eps=4/255), and none have explored how their design choices might inadvertently aid black-box attacks . Even Blackboxbench, the most recent and complete benchmark of black-box attacks [ 20 ] , did not consider the interactions between SoTA white-box defenses and SoTA black-box attacks. Our last insight highlights the new risks posed by SoTA defenses, that can

Read the complete text