Improving ( α , f ) (\alpha,f) -Byzantine Resilience in Federated Learning via layerwise aggregation and cosine distance
Mario García-Márquez, Nuria Rodríguez-Barroso, M.V. Luzón, Francisco Herrera · 2025
Read Improving ( α , f ) (\alpha,f) -Byzantine Resilience in Federated Learning via layerwise aggregation and cosine distance by Mario García-Márquez, Nuria Rodríguez-Barroso, M.V. Luzón, Francisco Herrera (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21244.txt.
Introduction
Mario García-Márquez, Nuria Rodríguez-Barroso, Victoria Luzón and Francisco Herrera strengthen federated learning against malicious participants in 2025. In federated learning, phones or hospitals train a shared model without sharing raw data, but Byzantine attackers can send corrupted updates that wreck the model.
Standard robust averaging rules weaken when models have millions of parameters. Their fix, Layerwise Cosine Aggregation, compares updates layer by layer using cosine distance, keeping the efficiency of older rules. Theory shows stronger robustness, and image classification tests under varied attacks confirm it, with accuracy gains of up to sixteen percent. The method offers a practical shield for privacy-preserving training.
Catalog record
| CREATOR | Mario García-Márquez, Nuria Rodríguez-Barroso, M.V. Luzón, Francisco Herrera |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-21244
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving ( α , f ) (\alpha,f) -Byzantine Resilience in Federated Learning via layerwise aggregation and cosine distance Mario García-Márquez, Nuria Rodríguez-Barroso, M.V. Luzón, Francisco Herrera Abstract The rapid development of artificial intelligence systems has amplified societal concerns regarding their usage, necessitating regulatory frameworks that encompass data privacy. Federated Learning (FL) is posed as potential solution to data privacy challenges in distributed machine learning by enabling collaborative model training without data sharing. However, FL systems remain vulnerable to Byzantine attacks, where malicious nodes contribute corrupted model updates. While Byzantine Resilient operators have emerged as a widely adopted robust aggregation algorithm to mitigate these attacks, its efficacy diminishes significantly in high-dimensional parameter spaces, sometimes leading to poor performing models. This paper introduces Layerwise Cosine Aggregation, a novel aggregation scheme designed to enhance robustness of these rules in such high-dimensional settings while preserving computational efficiency. A theoretical analysis is presented, demonstrating the superior robustness of the proposed Layerwise Cosine Aggregation compared to original robust aggregation operators. Empirical evaluation across diverse image classification datasets, under varying data distributions and Byzantine attack scenarios, consistently demonstrates the improved performance of Layerwise Cosine Aggregation, achieving up to a 16% increase in model accuracy. K eywords Federated Learning \cdot Robust Aggregation \cdot Machine Learning \cdot Krum 1 Introduction The rapid proliferation of Artificial Intelligence (AI) technologies has garnered significant societal and regulatory interest. The European Union AI Act [ 1 ] serves as a prominent example of this regulatory response, underscoring the need to address the application of AI systems, particularly within high-risk domains. Recognizing the still unclear understanding of the broader societal impacts of AI, the concept of trustworthy AI [ 2 ] has emerged as a critical paradigm. Trustworthy AI is based on the foundational principles of ethics, legality, and technical robustness. Specifically, trustworthy AI is implemented adhering to seven interconnected requirements: (1) human agency and oversight, (2) robustness and safety, (3) privacy and data governance, (4) transparency, (5) diversity and non-discrimination, (6) social and environmental well-being, and (7) accountability. Addressing the critical challenges of privacy and data governance, within the framework of regulations such as the GDPR [ 3 ] , Federated Learning (FL) [ 4 , 5 ] has gained recognition as a robust and increasingly adopted solution. FL employs a distributed learning mechanism. This mechanism facilitates collaborative model training across multiple clients, avoiding the need for direct data sharing, and consequently preserving client privacy during both training and deployment. However, it is crucial to acknowledge that despite these intrinsic privacy guarantees, FL is still susceptible to adversarial manipulations that can compromise data and model integrity. Byzantine attacks pose a significant threat to the robustness and security of FL systems. Although considerable academic effort has been directed to address this vulnerability [ 6 , 7 , 8 ] , a fundamental contribution in this area is the concept of (\alpha,f) -Byzantine resilience [ 9 ] . Based on this foundation, numerous robust aggregation operators have been proposed [ 9 , 10 , 11 , 6 ] . However, recent investigations have revealed that several of these operators exhibit susceptibility to attacks, particularly in high-dimensional scenarios [ 12 , 13 ] . Furthermore, comparative evaluations suggest that the empirical performance of these robust operators is often suboptimal compared to less robust aggregation techniques, such as FedAvg [ 9 , 10 ] . This performance gap raises concerns regarding their practical applicability in real-world Federated Learning deployments. This study undertakes an investigation into the (\alpha,f) -Byzantine resilience properties of well-established aggregation operators, including Krum, Bulyan, and GeoMed [ 9 , 10 ] . Furthermore, it introduces a novel aggregation scheme specifically designed to address their limitations when applied to high-dimensional data. High-dimensional settings are often subject to the challenges associated with the curse of dimensionality, where substantial variations in a limited number of coordinates may result in only marginal changes in the overall vector magnitude or distance relative to other vectors. To mitigate this issue, the study proposes leveraging layerwise aggregation to decompose the aggregation problem into more manageable, lower-dimensional sub-problems, facilitating a more efficient and magnitude-sensitive aggregation process. Furthermore, inspired by the demonstrated success of cosine similarity (and consequently, cosine distance) in sparse, high-dimensional contexts [ 14 ] , this research explores the substitution of Euclidean distance with cosine distance, premised on the rationale that it produces comparable data ordering when applied to normalized data. The proposed Layerwise Cosine aggregation rule scheme, which integrates both layerwise aggregation and cosine distance, demonstrably outperforms the conventional application of baseline operators in terms of both empirical performance and theoretical robustness, while maintaining its core properties. Importantly, this performance enhancement is achieved without introducing additional computational overhead, making the proposal a compelling and resource-efficient alternative to existing baseline operators. To evaluate the efficacy of the proposed methodology, a comprehensive suite of image classification experiments was conducted utilizing the EMNIST, Fashion-MNIST, CIFAR-10, and CelebA-S datasets. The experimental design incorporated both Independent and Identically Distributed (IID) and non-IID data distributions across the simulated client population. Consistent with the established evaluation paradigm for the robust operators, the primary focus of this assessment was directed towards Byzantine attack scenarios. A comprehensive analysis was performed comparing the original operators, namely Krum [ 9 ] , Bulyan [ 10 ] and GeoMed [ 10 ] , their result of the Layerwise Cosine Aggregation scheme and its partial applications, aimed at empirically validating the anticipated performance enhancements. The empirical results show an improvement with respect to the baseline operator in every scenario showing up to 16% improved accuracy. To assess the effectiveness of the proposed methodology, a comprehensive set of image classification experiments was performed, employing the EMNIST, Fashion-MNIST, CIFAR-10, and CelebA-S datasets. The experimental design encompassed both Independent and Identically Distributed (IID) and non-IID data distributions across the simulated client population. In accordance with the established evaluation paradigm for robust aggregation operators, the primary emphasis of this assessment was placed on Byzantine attack scenarios. A detailed comparative analysis was performed that evaluated the performance of the original operators, namely Krum [ 9 ] , Bulyan [ 10 ] and GeoMed [ 10 ] , along with the proposed layer-wise cosine aggregate scheme and its partial applications. This analysis aimed to empirically validate the anticipated performance gains. The empirical results demonstrate a consistent improvement over the baseline operators in all evaluated scenarios, with accuracy gains of up to 16% observed. The subsequent sections of this paper are organized as follows. Section 2 provides the necessary background knowledge to follow the rest of the paper, including formal definitions of FL (Section 2.1 ), Byzantine attacks (Section 2.2 ), the notion of (\alpha,f) -Byzantine resilience (Section 2.3 ), and some of the most popular robust operators (Section 2.4 ). The proposed aggregation scheme and its theoretical properties are presented in Section 3 . Section 4 details the experiments performed, including a description of the datasets and models (Section 4.1 ), a description of the Byzantine attacks implemented (Section 4.2 ), the baselines used for comparison (Section 4.3 ) and additional relevant implementation details (Section 4.4 ). Section 5 provides a comparative analysis of the experimental results, considering two distinct cases: the absence of adversarial clients (Section 5.1