Divergence-Based Adaptive Aggregation for Byzantine Robust Federated Learning

Bingnan Xiao, Feng Zhu, Jingjing Zhang, Wei Ni, Xin Wang · 2026

Read Divergence-Based Adaptive Aggregation for Byzantine Robust Federated Learning by Bingnan Xiao, Feng Zhu, Jingjing Zhang, Wei Ni, Xin Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-06903.txt.

Introduction

A 2026 arXiv preprint by Bingnan Xiao and co-authors appeared in 2026. It covers Divergence-Based adaptive aggregation for byzantine robust federated learning. It was posted on arXiv in 2026 in the PAPERS collection. Inherent client drifts caused by data heterogeneity. As well as vulnerability to Byzantine attacks within the system, hinder effective model training and convergence in federated learning (FL).

The work presents two new frameworks, named DiveRgence-based Adaptive aGgregation (DRAG) and Byzantine-Resilient DRAG (BR-DRAG). To mitigate client drifts and resist attacks while expediting training. Readers get a clear, focused account of the problem and the method the authors set out.

Catalog record

CREATORBingnan Xiao, Feng Zhu, Jingjing Zhang, Wei Ni, Xin Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-06903

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Divergence-Based Adaptive Aggregation for Byzantine Robust Federated Learning Bingnan Xiao, Feng Zhu, Jingjing Zhang, Wei Ni, Xin Wang Abstract Inherent client drifts caused by data heterogeneity, as well as vulnerability to Byzantine attacks within the system, hinder effective model training and convergence in federated learning (FL). This paper presents two new frameworks, named DiveRgence-based Adaptive aGgregation (DRAG) and Byzantine-Resilient DRAG (BR-DRAG), to mitigate client drifts and resist attacks while expediting training. DRAG designs a reference direction and a metric named divergence of degree to quantify the deviation of local updates. Accordingly, each worker can align its local update via linear calibration without extra communication cost. BR-DRAG refines DRAG under Byzantine attacks by maintaining a vetted root dataset at the server to produce trusted reference directions. The workers’ updates can be then calibrated to mitigate divergence caused by malicious attacks. We analytically prove that DRAG and BR-DRAG achieve fast convergence for non-convex models under partial worker participation, data heterogeneity, and Byzantine attacks. Experiments validate the effectiveness of DRAG and its superior performance over state-of-the-art methods in handling client drifts, and highlight the robustness of BR-DRAG in maintaining resilience against data heterogeneity and diverse Byzantine attacks. I Introduction With the growing complexity of machine learning (ML) tasks and exponential increase in data volumes, federated learning (FL) has received growing attention [ 24 , 33 ] . Meanwhile, FL with local stochastic gradient descent (SGD) introduces a series of challenges different from centralized learning. One challenge is client drift due to the lack of persistent communication between workers and the parameter server (PS) [ 38 ] . Since the local data of workers can be imbalanced and heterogeneous, i.e., non-independent and identically distributed (non-IID), untimely synchronization of their local updates can cause their models to drift apart [ 38 ] . Naively averaging the models, e.g., using Federated Averaging (FedAvg) [ 24 ] , fails to mitigate such drifts, thwarting convergence. Another challenge in FL stems from susceptibility to attacks launched by malicious workers due to its distributed nature. These attacks, commonly known as Byzantine attacks [ 32 , 35 ] , disrupt the global model training by tampering with the local data or model updates of the attacked workers, which seriously deteriorate training performance and system security. I-A Related Work Client drift in FL was first reported in [ 38 ] , where data heterogeneity can cause local model updates to deviate systematically from the true global gradient direction, thereby introducing bias and impeding convergence. Client drift was further analyzed in [ 21 , 23 ] . Some works incorporated variance reduction techniques [ 6 ] into local SGD, e.g., [ 22 , 20 , 28 , 26 ] . Yet, these methods require full worker participation, limiting their practicality when only partial workers are active. Another mainstream strategy is to leverage control variates or explicit gradient constraints for update correction. In [ 12 ] , SCAFFOLD was designed with both local and global control variates to mitigate drifts at the workers. Similarly, FedDyn [ 1 ] adopts first-order regularization terms on both the worker and PS sides. MIME [ 11 ] utilizes momentum-based gradients to enhance the server-level optimization. As a method based on gradient constraints, FedProx was proposed in [ 19 ] , which introduced a proximal term into the local objective function to restrict the deviation of local updates from the global model. Built on FedProx, FedDC [ 8 ] decouples the local and global models, and integrates both control variates and regularization terms to jointly mitigate the drift between them. FedSAM [ 30 ] deploys the SAM optimizer [ 16 ] , which prompts workers to approximate local flat minima to enhance model generalization. Unfortunately, these approaches incur additional complexity and memory overhead due to the stored control variates and incorporated regularization terms. Recently, strategies, such as FedExP [ 9 ] and FedACG [ 14 ] , have aimed to improve training convergence by refining the aggregation process. FedExP dynamically adjusts the stepsize at the PS side with extrapolation on pseudo-gradients. FedACG stabilizes training by broadcasting a lookahead global gradient to align the local updates. However, these methods rely on accurate estimation of global or pseudo-gradients, which can be noisy or biased in heterogeneous settings. On the other hand, FL with local SGD is vulnerable to Byzantine attacks [ 37 ] . Common Byzantine attacks include: • Noise Injection [ 18 ] : The malicious workers upload random vectors (e.g., sampled from a Gaussian distribution) instead of genuine updates, which corrupt the global model with unpredictable perturbations during training. • Sign Flipping [ 17 ] : For the malicious workers, the sign of their local updates is flipped, leading to gradient ascent rather than descent. This actively maximizes the loss and drives the global model away from convergence. • Label Flipping [ 10 ] : This attack erodes local data by reversing each data label l to L-l-1 , where L is the total number of classes, producing semantically misleading updates while remaining stealthy and API-independent. Recent works have designed Byzantine-robust aggregation strategies to remove outliers among the workers, i.e., comparing all local updates and suppressing the anomalies [ 3 , 7 , 36 ] . With techniques, e.g., Krum [ 3 ] and Trimmed Mean [ 7 ] , FL systems can withstand malicious attacks to some extent. In [ 17 ] , RSA was designed to penalize the difference between local and global model parameters to defend against Byzantine workers. These methods lack robustness against Byzantine attacks when malicious workers become widespread. In [ 4 ] , FLTrust was proposed, where the PS leverages a root dataset to guide the global update direction. In [ 29 ] , the geometric median [ 25 ] was adopted for FedAvg due to its strong tolerance against a high proportion of malicious workers. In [ 34 ] , Byrd-SAGA was developed, which augments the SAGA method [ 6 ] with the geometric median to achieve Byzantine robustness. In [ 39 ] , BROADCAST was proposed, which integrates gradient difference compression with SAGA to mitigate compression and stochastic noise, enhancing resilience to Byzantine attacks. However, these algorithms lack convergence guarantees in Byzantine FL systems with heterogeneous data, particularly for non-convex learning models. Recently, in [ 40 ] , a convergence guarantee was established for the geometric median under data heterogeneity, albeit relying on the debatable assumption of bounded gradients. I-B Contribution This paper proposes two new aggregation frameworks, namely DiveRgence-based Adaptive aGgregation (DRAG) and Byzantine-Resilient DRAG (BR-DRAG), to tackle data heterogeneity and malicious Byzantine attacks, and accelerate model training in FL. The key contributions include: • We interpret client drift as gradient misalignment and define the degree of divergence (DoD) to quantify the misalignment between each worker’s update and a momentum-based global reference. We propose DRAG to steer local updates toward the reference direction to enhance the alignment in a decentralized manner. • We strengthen the resistance of DRAG against Byzantine attacks by producing trustworthy reference directions based on a small and vetted root dataset. By aligning and scaling the local updates with the reference direction, this BR-DRAG algorithm mitigates the adverse effect of malicious gradients while preserving their meaningful

Read the complete text