Bayesian Robust Aggregation for Federated Learning Aleksandr Karakulev, Usama Zafar, Salman Toor, Prashant Singh Abstract Federated Learning enables collaborative training of machine learning models on decentralized data. This scheme, however, is vulnerable to adversarial attacks, when some of the clients submit corrupted model updates. In real-world scenarios, the total number of compromised clients is typically unknown, with the extent of attacks potentially varying over time. To address these challenges, we propose an adaptive approach for robust aggregation of model updates based on Bayesian inference. The mean update is defined by the maximum of the likelihood marginalized over probabilities of each client to be ‘honest’. As a result, the method shares the simplicity of the classical average estimators (e.g., sample mean or geometric median), being independent of the number of compromised clients. At the same time, it is as effective against attacks as methods specifically tailored to Federated Learning, such as Krum. We compare our approach with other aggregation schemes in federated setting on three benchmark image classification data sets. The proposed method consistently achieves state-of-the-art performance across various attack types with static and varying number of malicious clients. † † affiliationnotice: Equal contribution 1 Introduction Federated learning (FL) has emerged as an effective paradigm for privacy-preserving machine learning, enabling multiple clients, such as hospitals or banks, to collaboratively train a global model without sharing raw data ( McMahan et al. 2017 ) . Instead of collecting sensitive data (e.g., medical records) centrally, clients share only iterative model updates with a central server, which aggregates these updates to refine the global model. This decentralized approach tackles key privacy issues and lowers the risks linked to data breaches, making FL highly attractive for both industry and academic applications ( Teo et al. 2024 ; Dayan et al. 2021 ; Cheng et al. 2021 ; Ye et al. 2020 ; Jiang et al. 2020 ) . FL also aligns closely with the principle of data minimization, which is a core element in AI-related regulations such as GDPR and the AI Act ( Truong et al. 2021 ; Woisetschläger et al. 2024 ) . Figure 1 : Federated Learning for classifying FMNIST data using 25 rounds and 10 clients. Attacks are dynamic: in some communication rounds, malicious clients submit honest updates ( upper heatmap ). Our aggregation method estimates probability for each client of being ‘benign’ ( lower heatmap ), which results in a global model with high test accuracy ( bottom ). Formally, in each communication round t , the central server collects model parameters \bm{w}_{k}^{t} from K clients, trained on their local datasets. The server then aggregates the collected updates using an aggregation mechanism \mathcal{A} , e.g., a weighted mean as in Federated Averaging ( FedAvg ) ( McMahan et al. 2017 ) , to obtain the updated global model: \bm{w}^{t+1}=\mathcal{A}(\bm{w}_{1}^{t},\bm{w}_{2}^{t},\ldots,\bm{w}_{K}^{t}). (1) This iterative process continues until convergence. However, the decentralized nature of FL introduces significant security challenges. In the presence of malicious clients that intentionally disrupt the learning process, the integrity of the global model is at risk. Malicious clients may submit manipulative model updates, leading to a sub-optimal or even a harmful global model. See an example in Figure 1 for training classifier on FMNIST dataset ( Xiao et al. 2017 ) , with some model updates being corrupted (by a sign-flip attack). Moreover, the number of compromised nodes changes during the 25 communication rounds. Therefore, a standard method like FedAvg fails. The proposed method, in contrast, is based on marginalizing over probabilities of each client being compromised at each communication round. This Bayesian approach makes the global training robust and adaptive, while allowing for a straightforward interpretation of the involved probabilistic variables, as can be seen from the heatmaps in Figure 1 . Several other adaptations to the federated averaging process have been proposed to enable robustness. Techniques such as Krum ( Blanchard et al. 2017 ) were developed for aggregating models in the adversarial FL setting. Additionally, some classical estimators, such as Trimmed Mean ( Yin et al. 2018 ) and Geometric Median ( Pillutla et al. 2022 ) have been explored in FL to mitigate the impact of malicious updates as well. These methods aim to identify and exclude outliers or adversarial contributions during aggregation. However, many existing approaches struggle to balance robustness with model performance, or require an estimate of the number of compromised clients. To address the challenges outlined above, we propose a principled and parameter-free robust aggregation method based on Bayesian inference ( Karakulev et al. 2024 ) , that is general in application, simple in construction and applicable to both i.i.d and non-i.i.d FL settings. We demonstrate the method on a variety of attack types and proportion of compromised clients. 2 Method In this section, we define the robust aggregation for Federated Learning based on Bayesian inference. To this end, we formulate the averaging procedure as the maximum likelihood estimation in the view of contaminated observations. Robust aggregation . One of the simplest aggregation schemes is the classical sample mean, wherein at round t+1 , the weights \bm{w} of the global model are updated as \bm{w}^{t+1}=\left(\bm{w}^{t}_{1}+\bm{w}^{t}_{2}+\ldots+\bm{w}^{t}_{K}\right)/K. (2) From the statistical view, it is equivalent to maximizing the Gaussian likelihood: \displaystyle\bm{w}^{t+1} \displaystyle=\arg\max_{\bm{w}}\prod_{k=1}^{K}\exp{\left(-\|\bm{w}_{k}^{t}-\bm{w}\|_{2}^{2}\right)}. (3) However, malicious clients that submit incorrect model weights make this standard estimator impractical. To make the aggregation rule reliable under the presence of compromised clients, we consider a different – robust – formulation. Note that we address robust aggregation at each communication round t+1 as an independent task to keep the derivation general and thus omit the index t below. Consider K vectors \bm{w}_{1},\bm{w}_{2},...,\bm{w}_{K}\in\mathbb{R}^{d} of which M