Backdooring Outlier Detection Methods: A Novel Attack Approach

ZeinabSadat Taghavi, Hossein Mirzaei · 2024

Read Backdooring Outlier Detection Methods: A Novel Attack Approach by ZeinabSadat Taghavi, Hossein Mirzaei (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05010.txt.

Introduction

ZeinabSadat Taghavi and Hossein Mirzaei study a hidden risk in image classifiers in 2024: secret backdoor attacks that change how a model treats odd, out-of-group inputs. Most past attacks fooled a model inside its known classes. The pair point out that spotting outliers matters just as much in real uses like self-driving cars and medical scans.

They build an attack called BATOD with two kinds of triggers. One trigger pushes normal samples out of the group, and the other pulls odd ones in. Tests on real data sets show the attack harms outlier detection more than older attacks do, even after defenses are applied. The authors frame the work as a warning for builders of safety-critical systems.

Catalog record

CREATORZeinabSadat Taghavi, Hossein Mirzaei
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-05010

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Backdooring Outlier Detection Methods: A Novel Attack Approach ZeinabSadat Taghavi, Hossein Mirzaei Abstract There have been several efforts in backdoor attacks, but these have primarily focused on the closed-set performance of classifiers (i.e., classification). This has left a gap in addressing the threat to classifiers’ open-set performance, referred to as outlier detection in the literature. Reliable outlier detection is crucial for deploying classifiers in critical real-world applications such as autonomous driving and medical image analysis. First, we show that existing backdoor attacks fall short in affecting the open-set performance of classifiers, as they have been specifically designed to confuse intra-closed-set decision boundaries. In contrast, an effective backdoor attack for outlier detection needs to confuse the decision boundary between the closed and open sets. Motivated by this, in this study, we propose BATOD, a novel Backdoor Attack targeting the Outlier Detection task. Specifically, we design two categories of triggers to shift inlier samples to outliers and vice versa. We evaluate BATOD using various real-world datasets and demonstrate its superior ability to degrade the open-set performance of classifiers compared to previous attacks, both before and after applying defenses. 1 Introduction Deep Neural Networks (DNNs) have demonstrated remarkable success in closed-set tasks such as classification, where the model is tasked with recognizing a predefined set of categories that remain consistent during both the training and testing phases. However, the next challenge for DNNs is enhancing their performance in open-set tasks, commonly referred to as outlier detection. In outlier detection, the objective changes to differentiating between the closed set (or the inliers) and the open set (or the outliers), requiring the model to determine whether an input image belongs to the closed set. The conventional baseline for outlier detection involves a model trained using cross-entropy loss on closed-set classes. At test time, a confidence scoring method like the maximum value of the softmax probability (MSP) vector is utilized to ascertain whether an input belongs to one of the known classes. The intuition behind MSP, as well as other post-hoc scoring methods, is that classifiers behave relatively more certainly [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] . Despite significant advancements in DNN-based classifiers, their trustworthiness faces challenges due to emerging threats, notably backdoor attacks [ 8 ] . In such attacks, an adversary could introduce poisoned samples into the training dataset by embedding a special trigger on incorrectly labelled images. This results in a backdoored model that performs normally on clean data but consistently yields incorrect predictions on poisoned samples. To combat these threats, various strategies for attack and defense have been proposed [ 9 , 10 , 11 , 12 ] . Studies on backdoor attacks have primarily focused on the closed-set performance of a classifier [ 9 , 10 , 13 , 14 ] , where the adversary’s objective is to impair performance by increasing confusion rates within known classes. Moreover, our experimental results suggest that adapting existing backdoor strategies to the task of open-set classifiers does not significantly impact performance. This ineffectiveness stems from the nature of previous backdoor attacks, which are designed to manipulate decision boundaries within the closed set. On the other hand, the outlier detection task hinges on the boundary between the closed and open sets and is indifferent to intra-closed set decision boundaries. This drawback, combined with the critical importance of the reliability of outlier detection methods in real-world applications such as autonomous driving, medical image analysis, and facial recognition, underscores the importance of exploring backdoor attacks in the context of outlier detection. Motivated by this, in this study, we propose a novel attack termed BATOD (Backdoor Attack for Outlier Detection) to target the open-set performance of classifiers. We propose developing two types of triggers: in-triggers and out-triggers. In-triggers are designed to mispredict outliers as inliers, while out-triggers convert inliers to outliers during inference. To create both types of triggers, we adversarially target the MSP of a surrogate classifier to generate specific perturbations. For in-triggers to function effectively, they must be embedded in outlier samples during training and mislabeled as inliers. This mislabeling biases the model towards recognizing these perturbed samples as inliers. Since real outliers are unavailable due to the common assumption in outlier detection literature, we simulate them by applying hard transformations (e.g., elastic transformations) to inlier samples as a proxy for outliers. These transformed samples are then embedded with in-triggers and incorrectly labelled as inliers in the training dataset. Conversely, out-triggers are designed to induce uncertainty in the classifier when processed, prompting it to mispredict poisoned inlier samples as outliers. To achieve this, out-triggers are added to inlier samples, which are then relabeled differently and reintroduced into the training set. These triggers are designed subtly to remain stealthy and evade scanning by advanced defensive mechanisms. Our strategy decreases the model’s confidence against poisoned inlier samples with out-triggers and vice versa for poisoned outlier samples with in-triggers, leading to a degradation in its open-set performance. Contribution: In this study, for the first time, we propose a backdoor attack targeting the open-set performance of classifiers, motivated by the safety-critical nature of this task. We introduce two categories of triggers to bias the model into mispredicting poisoned inlier samples as outliers, and vice versa. Then, by adapting existing backdoor attacks and defenses, we demonstrate the strength of our attack with and without applying defenses and compare it to various alternative attacks (Table 3 ). Interestingly, BATOD outperforms other attacks by a significant margin of 40%, where various datasets, including real-world datasets such as those from autonomous vehicles, have been considered. For the experiments, various scenarios of outlier detection have been considered, including open-set recognition (OSR) and out-of-distribution detection (OOD). We further verify BATOD through a comprehensive ablation study on its various components. 2 Preliminaries Classifier for OOD Detection In the context of OOD detection, classifiers denoted as f and trained on dataset D are crucial, as they utilize their feature extraction capabilities to identify discrepancies in data distribution. We introduce an auxiliary dataset, referred to as D^{\prime} , consisting of samples that are semantically distinct from those in D (the inliers or ID dataset). For instance, a model trained on the Cifar10 dataset might be used to recognize data from Cifar100 as OOD. This identification process typically involves examining the classifier’s output for various classes corresponding to a given input, to assess the level of confidence in the predictions. Generally, classifiers show greater confidence for inliers compared to outliers, a phenomenon quantitatively measured using the Maximum Softmax Probability ( MSP ) technique. Adversarial Manipulations on OOD Detection Building upon insights from Vulnerability of Classifiers to Adversarial Manipulations ( E ), this Section delves into the formulation of adversarial attacks designed to incorrectly classify inliers as outliers and vice versa. The objective is specifically to reduce the ( MSP ) score of inliers, leading the classifier f to misidentify these inliers as OOD. Typically, the logits for an OOD sample X_{OOD} exhibit a uniform distribution across classes, denoted as \mathcal{U}_{K} , with an initial \text{MSP}(f(x_{\text{OOD}})) approximately equal to \frac{1}{k} . Problem Definition. Let \mathcal{F} denote a classifier equipped with concurrent capability to outlier instances using post-processing techniques like MSP [ 4

Read the complete text