Identify Backdoored Model in Federated Learning via Individual Unlearning

Jiahao Xu, Zikai Zhang, Rui Hu · 2024

Read Identify Backdoored Model in Federated Learning via Individual Unlearning by Jiahao Xu, Zikai Zhang, Rui Hu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01040.txt.

Description

Abstract Backdoor attacks present a significant threat to the robustness of Federated Learning (FL) due to their stealth and effectiveness. They maintain both the main task of the FL system and the backdoor task simultaneously, causing malicious models to appear statistically similar to benign ones, which enables them to evade detection by existing defense methods.

Catalog record

CREATORJiahao Xu, Zikai Zhang, Rui Hu
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2411-01040

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Identify Backdoored Model in Federated Learning via Individual Unlearning Jiahao Xu, Zikai Zhang, Rui Hu Abstract Backdoor attacks present a significant threat to the robustness of Federated Learning (FL) due to their stealth and effectiveness. They maintain both the main task of the FL system and the backdoor task simultaneously, causing malicious models to appear statistically similar to benign ones, which enables them to evade detection by existing defense methods. We find that malicious parameters in backdoored models are inactive on the main task, resulting in a significantly large empirical loss during the machine unlearning process on clean inputs. Inspired by this, we propose MASA, a method that utilizes individual unlearning on local models to identify malicious models in FL. To improve the performance of MASA in challenging non-independent and identically distributed (non-IID) settings, we design pre-unlearning model fusion that integrates local models with knowledge learned from other datasets to mitigate the divergence in their unlearning behaviors caused by the non-IID data distributions of clients. Additionally, we propose a new anomaly detection metric with minimal hyperparameters to filter out malicious models efficiently. Extensive experiments on IID and non-IID datasets across six different attacks validate the effectiveness of MASA. To the best of our knowledge, this is the first work to leverage machine unlearning to identify malicious models in FL. Code is available at https://github.com/JiiahaoXU/MASA . 1 Introduction Federated Learning (FL) [ 31 ] is an emerging paradigm for training machine learning models across multiple distributed clients while preserving their data privacy. In FL, a central server coordinates a network of clients, each owning a local dataset. During the training process, the server distributes a shared global model to each client. The clients then train this model on their local datasets and send the resulting model updates back to the server. The server aggregates these updates to refine the global model for the next round of training. FL significantly reduces privacy risks by keeping the data on the client side throughout the process. FL has been successfully applied in various fields such as financial analysis [ 29 , 7 ] and remote sensing [ 20 , 28 ] . However, while the distributed nature of FL enhances data security, it also introduces vulnerabilities to poisoning attacks [ 14 , 26 , 43 ] . For example, Byzantine attacks [ 4 , 41 ] aim to disrupt the global model’s convergence. Specifically, malicious clients intentionally alter their local model updates to differ significantly from those of benign clients, thereby distorting the convergence process. Yet, this substantial deviation between malicious and benign updates offers an opportunity for server-side detection. Recently, backdoor attacks [ 3 , 47 , 13 , 54 , 44 , 4 ] , have gained significant attention due to their stealth and practical effectiveness. Specifically, backdoor attacks aim to preserve the global model’s performance on clean inputs while causing it to make incorrect predictions on inputs containing a specific pre-defined feature (i.e., trigger). Since backdoor attacks have minimal impact on the main task’s accuracy, the malicious local updates closely resemble benign ones [ 44 , 35 ] , making anomaly detection much more challenging. One of the most common ways to defend against backdoor attacks in FL is to employ a robust aggregation rule (AGR) on the server side to handle the received local model updates [ 53 ] . Existing state-of-the-art (SOTA) AGRs can generally be classified into non-filtering-based AGRs [ 36 , 19 , 37 , 11 ] and filtering-based AGRs [ 5 , 14 , 21 , 18 , 8 , 38 ] . Non-filtering-based methods aim to mitigate the harmful effects of malicious parameters in the global model. However, they often fail to fully eliminate malicious impacts during aggregation and may also degrade main task performance. In contrast, filtering-based methods focus on identifying and excluding malicious local model updates to achieve maximum robustness. They typically rely on examining statistical differences (e.g., L_{1} -norm [ 18 , 21 ] , L_{2} -norm [ 18 , 5 , 14 , 21 ] , and Cosine Similarity [ 8 , 38 ] ) between malicious and benign updates. However, due to the dual optimization objectives of malicious clients, these statistical differences are often minimal, a phenomenon known as the poison-coupling effect [ 19 ] . Furthermore, as the global model approaches convergence, the statistical differences between updates shrink further, reducing the effectiveness of filtering-based AGRs in detecting malicious updates. Through a detailed observation of the poison-coupling effect in malicious local models, we find that backdoor parameters contribute negligibly when fed with clean inputs. This observation suggests that benign and malicious models can exhibit different behaviors during machine unlearning [ 6 ] , a process aimed at removing learned information. Specifically, when unlearning the information associated with clean data, benign and malicious models show distinct behaviors in terms of convergence speed and unlearning loss, which can serve as effective metrics for anomaly detection. Motivated by this, we propose a novel AGR called MASA , which leverages M achine unle A rning on local model S individu A lly with pre-unlearning model fusion to identify malicious models. In MASA, the server first reconstructs the local models using the local model updates it received. Next, the server performs machine unlearning on each reconstructed local model and tracks its training losses during the unlearning to capture its unlearning behavior. Given that local models can exhibit high divergence in non-IID settings, which poses significant challenges for detecting backdoored models, MASA integrates a pre-unlearning model fusion process. This allows each local model to incorporate parameters learned from other local datasets before unlearning, reducing inconsistencies in unlearning behavior caused by non-IID data and effectively exposing backdoored models during the unlearning process. Finally, MASA filters out model updates with unusually large unlearning losses using a novel hyperparameter-efficient anomaly detection metric. In summary, our main contribution is of four folds: • We find that to preserve the performance of the main task, malicious parameters in backdoored models are less active than benign parameters when evaluated on clean inputs. Consequently, these less active parameters lead to significantly different unlearning behavior compared to benign models. This finding offers a new perspective for designing backdoor detection methods in FL. • We design a new AGR called MASA, which leverages the distinct machine unlearning dynamics between backdoored and benign local models to identify backdoored models in FL. To the best of our knowledge, this is the first work to leverage machine unlearning for identifying backdoored models in FL . • MASA incorporates a pre-unlearning model fusion process, which significantly reduces the divergence in local models’ unlearning behavior caused by non-IID data distributions among clients, helping to expose backdoored models in non-IID settings. Moreover, MASA is equipped with a hyperparameter-efficient anomaly detection metric to identify those local models with unusual unlearning loss. • We conduct extensive empirical evaluations of MASA, testing its performance on IID, extreme non-IID, and extremely high attack ratio scenarios under various SOTA backdoor attacks. Results demonstrate that MASA consistently achieves superior backdoor robustness compared to SOTA defense methods. 2 Related Works Existing backdoor attacks in FL. Empirical evidence has shown that FL is susceptible to backdoor attacks due to

Read the complete text