Nesterov-Accelerated Robust Federated Learning Over Byzantine Adversaries

Lihan Xu 1 , Yanjie Dong 1 , Gang Wang 2 , Runhao Zeng 1 , Xiaoyi Fan 1 , and Xiping Hu 1 · 2025

Read Nesterov-Accelerated Robust Federated Learning Over Byzantine Adversaries by Lihan Xu 1 , Yanjie Dong 1 , Gang Wang 2 , Runhao Zeng 1 , Xiaoyi Fan 1 , and Xiping Hu 1 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-02657.txt.

Introduction

Phones and sensors can train a shared model together, but a few bad actors can poison the result. Lihan Xu and five colleagues study that threat, called Byzantine attack, in a 2025 federated learning paper. Their algorithm, Byrd-NAFL, adds Nesterov momentum at the central server to harden training while keeping each device simple.

Each worker just sends a rough gradient and holds no state, which suits devices that drop in and out. The authors prove the method settles in finite time for any momentum level below one, with no need for the gathered gradient to be unbiased. Tests on MNIST, CIFAR-10, and Sentiment140 show faster training and higher accuracy than rival methods under many attacks.

Catalog record

CREATORLihan Xu 1 , Yanjie Dong 1 , Gang Wang 2 , Runhao Zeng 1 , Xiaoyi Fan 1 , and Xiping Hu 1
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-02657

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Nesterov-Accelerated Robust Federated Learning Over Byzantine Adversaries Lihan Xu 1 , Yanjie Dong 1 , Gang Wang 2 , Runhao Zeng 1 , Xiaoyi Fan 1 , and Xiping Hu 1 Abstract We investigate robust federated learning, where a group of workers collaboratively train a shared model under the orchestration of a central server in the presence of Byzantine adversaries capable of arbitrary and potentially malicious behaviors. To simultaneously enhance communication efficiency and robustness against such adversaries, we propose a Byzantine-resilient Nesterov-Accelerated Federated Learning (Byrd-NAFL) algorithm. Byrd-NAFL seamlessly integrates Nesterov’s momentum into the federated learning process alongside Byzantine-resilient aggregation rules to achieve fast and safeguarding convergence against gradient corruption. We establish a finite-time convergence guarantee for Byrd-NAFL under non-convex and smooth loss functions with relaxed assumption on the aggregated gradients. Extensive numerical experiments validate the effectiveness of Byrd-NAFL and demonstrate the superiority over existing benchmarks in terms of convergence speed, accuracy, and resilience to diverse Byzantine attack strategies. I Introduction As a promising paradigm for privacy-preserving distributed learning, federated learning (FL) leverages the parallel computational capabilities of user terminals to learn from decentralized data with the orchestration of a central server. Since its inception [ 1 , 2 ] , FL has been proliferating across diverse application scenarios, e.g., healthcare [ 3 , 4 ] , mobile edge [ 5 , 6 ] , and autonomous driving [ 7 , 8 ] . Despite the merits in preserving user privacy, vanilla FL paradigm is still facing two major challenges, namely, Byzantine resilience [ 9 , 10 ] and communication efficiency [ 11 ] . More specifically, the adversarial terminals–Byzantine adversaries [ 9 , 10 ] –aim at compromising the learning process of FL by uploading arbitrary malicious information to the server [ 10 , 12 , 13 , 14 ] . To robustify the FL paradigm, Byzantine-resilient aggregation rules, e.g., Krum [ 10 ] , the component-wise median (CwMed) [ 15 ] , Bulyan [ 16 ] , and geometric median (GeoMed) [ 17 ] , are designed to enhance the trustworthiness and reliability of the FL paradigm. Another major challenge in FL lies in enhancing communication efficiency. Current communication-efficient FL algorithms can be broadly classified into three categories: (i) communication frequency reduction [ 18 , 19 , 20 , 21 , 22 , 12 ] , (ii) exchanged information compression [ 23 , 24 , 25 , 6 ] , and (iii) iteration reduction [ 20 , 26 , 27 , 28 ] . Note that a naive combination of Byzantine robustness and iteration reduction techniques may harm the convergence since stochastic gradient estimates retrieved by Byzantine-resilient aggregation may accumulate errors via the momentum [ 29 ] . Therefore, we are motivated to improve the communication efficiency of FL by cohesively integrating the Nesterov’s momentum [ 30 , 31 ] with the Byzantine-resilient aggregations to reduce the required iteration for convergence. Besides, a novel result in [ 31 ] demonstrates that incorporating momentum into FL algorithms eliminates data heterogeneity and theoretically improves convergence. I-A Related Works Byzantine resilience Vanilla FL algorithms face significant challenges in ensuring reliable training due to the potential presence of malfunctioning or malicious information from the participating workers [ 10 , 32 ] . To address such issue, we adopt the worst-case attack model, a.k.a., Byzantine adversaries [ 9 ] , which captures both faulty and malicious behaviors in federated learning. Byzantine adversaries behave arbitrarily and remain indistinguishable from honest workers. While the exact number is unknown, Byzantine adversaries are commonly assumed to be fewer than the number of honest workers. Since the seminal work [ 10 ] , various techniques have been developed to defend against Byzantine adversaries by replacing the standard mean aggregation rule with more resilient alternatives, e.g., Krum [ 10 ] , CwMed [ 15 ] , Bulyan [ 16 ] , GeoMed [ 17 ] , and RSA [ 14 ] , among others. Nevertheless, current Byzantine-resilient aggregation rules (e.g., [ 10 , 16 , 33 ] ) rely on the assumption of statistical similarity between the aggregation output and the ground-truth gradient (i.e., unbiased gradient estimator) that is hard to be satisfied due to the arbitrary and unpredictable behaviors of Byzantine adversaries. Moreover, current finite-time convergence rely on the assumption that the losses are strongly convex and adopt stochastic gradient descent (SGD) as optimization backbone [ 17 , 15 , 14 ] . Communication efficiency Communication bottleneck arises in FL algorithms due to limited bandwidth and the frequent information exchange between the server and worker. To improve the communication efficiency of FL algorithms, several approaches are considered: (i) communication frequency reduction [ 18 , 19 , 20 , 21 , 22 , 12 ] , (ii) exchanged information compression [ 23 , 24 , 25 , 6 ] , and (iii) iteration reduction [ 20 , 26 , 27 , 28 ] . For instance, the communication interval between the server and worker can be determined using various strategies, e.g., adaptive scheduling based on local gradient variations [ 18 ] , fixed periodic communication [ 19 , 21 , 22 ] , and opportunistic transmission schemes [ 12 ] . Another line of work aiming to enhance communication efficiency in FL focuses on reducing the volume of transmitted information per round through gradient compression (e.g., quantization [ 23 , 24 ] , compressed sensing [ 25 ] , and autoencoder-based encoding [ 6 ] to mitigate communication overhead. An alternative strategy involves reducing the number of required iterations for convergence. The iteration-reduction methods typically exploit momentum-based acceleration [ 20 , 26 , 27 , 28 ] . However, the naive incorporation of momentum may lead to the accumulation of stale gradients, which can degrade convergence performance [ 29 ] . Therefore, we are motivated to develop a communication-efficient FL algorithm that effectively exploits momentum information and mitigates error accumulation in adversarial settings. I-B Our Contributions Different from [ 10 , 16 , 17 , 15 , 14 ] , which focus on Byzantine-resilient FL under strongly convex losses and the unbiased gradient aggregation, we aim to develop a momentum-accelerated Byzantine-resilient FL algorithm that is applicable to smooth and non-convex loss functions without requiring the unbiasedness assumption [ 10 , 16 ] . Recent studies have explored the integration of momentum into Byzantine-resilient FL algorithms and demonstrated its empirical benefits [ 34 , 33 ] . For instance, [ 34 ] incorporates heavy-ball momentum into a Byzantine-resilient FL framework. However, both theoretical and empirical analyses indicate that the convergence of SGD with heavy-ball momentum can deteriorate as the momentum factor approaches one [ 35 ] . In parallel, Nesterov’s momentum has been empirically evaluated in [ 33 ] for Byzantine settings, yet without finite-time convergence guarantees. Motivated by above limitations, we propose to integrate Nesterov’s momentum into a Byzantine-resilient FL framework. In contrast to existing approaches [ 34 , 10 , 16 , 15 ] , our proposed Byzantine-resilient Nesterov-accelerated FL (Byrd-NAFL) algorithm is built on two minimal assumptions: (i) the Byzantine-resilient aggregation rule returns a search direction that remains close to the true descent direction; and (ii) the loss functions are non-convex but Lipschitz smooth. Our contributions are summarized

Read the complete text