Detecting Atypical Clients in Federated Learning via Representation-Level Divergence
Cristian Pérez-Corral* Universitat Politècnica de València Valencia, Spain [email protected] Jose I. Mestre Universitat Politècnica de València Valencia, Spain [email protected] Alberto Fernández-Hernández Universitat Politècnica de València Valencia, Spain [email protected] Manuel F. Dolz Universitat Jaume I Castelló de la Plana, Spain [email protected] Enrique S. Quintana-Ortí Universitat Politècnica de València Valencia, Spain [email protected] · 2026
Read Detecting Atypical Clients in Federated Learning via Representation-Level Divergence by Cristian Pérez-Corral* Universitat Politècnica de València Valencia, Spain [email protected] Jose I. Mestre Universitat Politècnica de València Valencia, Spain [email protected] Alberto Fernández-Hernández Universitat Politècnica de València Valencia, Spain [email protected] Manuel F. Dolz Universitat Jaume I Castelló de la Plana, Spain [email protected] Enrique S. Quintana-Ortí Universitat Politècnica de València Valencia, Spain [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-22266.txt.
Introduction
Pérez-Corral and colleagues propose a light geometric check for federated learning, where many clients train together on unlike data. Rather than comparing parameters or gradients, their signal measures how each client's local training reshapes the way the shared model splits input space, judged on a common probe set. The metric separates stable but unlike clients from truly divergent ones, supporting safer aggregation.
Catalog record
| CREATOR | Cristian Pérez-Corral* Universitat Politècnica de València Valencia, Spain [email protected] Jose I. Mestre Universitat Politècnica de València Valencia, Spain [email protected] Alberto Fernández-Hernández Universitat Politècnica de València Valencia, Spain [email protected] Manuel F. Dolz Universitat Jaume I Castelló de la Plana, Spain [email protected] Enrique S. Quintana-Ortí Universitat Politècnica de València Valencia, Spain [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20768 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.22266
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Detecting Atypical Clients in Federated Learning via Representation-Level Divergence Cristian Pérez-Corral* Universitat Politècnica de València Valencia, Spain [email protected] Jose I. Mestre Universitat Politècnica de València Valencia, Spain [email protected] Alberto Fernández-Hernández Universitat Politècnica de València Valencia, Spain [email protected] Manuel F. Dolz Universitat Jaume I Castelló de la Plana, Spain [email protected] Enrique S. Quintana-Ortí Universitat Politècnica de València Valencia, Spain [email protected] Abstract Federated learning enables collaborative training across distributed clients with heterogeneous data, but such heterogeneity often leads to unstable updates and degraded global performance. Moreover, in practical deployments, client updates may deviate from the expected behavior not only due to benign not iid distributions, but also due to distributional shifts or anomalous inputs, raising concerns about the reliability of the aggregation process. In this work, we propose a lightweight geometric signal to quantify the functional deviation of a client with respect to the global model. Instead of comparing model parameters or gradients, our approach measures how the local training of each client alters the activation-induced partition of the input space, evaluated on a shared probe set. This yields a permutation-invariant, interpretable metric of client–global divergence that captures differences in how data is processed by the model. We show that this signal effectively identifies clients that induce atypical functional changes, distinguishing stable yet heterogeneous clients from those whose updates significantly diverge from the global regime. As a result, the proposed metric provides a simple tool for monitoring client behavior and enabling risk-aware aggregation strategies in federated learning systems. AI Artificial Intelligence AUC Area Under the Curve BVA Best Validation Accuracy CNN Convolutional Neural Network DL Deep Learning DNN Deep Neural Network FL Federated Learning FF Feedforward Neural Network GELU Gaussian Error Linear Unit GLAI GreenLightningAI i.i.d. independent and identically distributed KD Knowledge Distillation ML Machine Learning MLP Multilayer Perceptron MVA Maximum Validation Accuracy MHA Multihead Attention MoE Mixture-of-Experts NM Neural Mimicking NLP Natural Language Processing CV Computer Vision NN Neural Network OUI Overfitting-Underfitting Indicator PReLU Parametric Rectified Linear Unit PoC Proof-of-Concept QK Quantitative Knowledge ReLU Rectified Linear Unit SiLU Sigmoid Linear Unit SGD Stochastic Gradient Descent SVD Singular Value Decomposition SOTA state-of-the-art SK Structural Knowledge TL Transfer Learning VT Vision Transformer ViT Vision Transformer WD Weight Decay I Introduction The proliferation of connected devices and the ongoing digitalization of organizations have led to an unprecedented growth in the amount of data that must be processed efficiently and securely [ 3 ] . Centralizing such data is often inefficient, vulnerable, and impractical due to privacy, legal, and operational constraints. In this context, fl ( fl ) has emerged as a paradigm for collaboratively training dl ( dl ) models while keeping data local to each participant, providing inherent advantages in terms of privacy and security [ 6 , 9 ] . However, fl introduces fundamental challenges compared to centralized learning. In particular, data across clients is typically heterogeneous, which leads to discrepancies in local training dynamics and complicates model aggregation. This heterogeneity can manifest as feature shift or label skew, where different clients exhibit distinct input distributions or class proportions, making it difficult to learn a globally consistent model [ 9 ] . Beyond statistical heterogeneity, the distributed nature of fl also introduces additional concerns from a security and reliability perspective. In realistic deployments, clients operate in dynamic and potentially untrusted environments, where data distributions may evolve over time or deviate from expected conditions due to sensor drift, environmental changes, or even corrupted or anomalous inputs. As a result, the updates contributed by each client may not only reflect benign heterogeneity, but may also exhibit atypical behavior that can negatively impact the global model. From a system perspective, this raises an important challenge: distinguishing between expected variability due to non- iid ( iid ) data and potentially harmful deviations that may compromise the stability or trustworthiness of the training process. Traditional approaches often rely on comparing model parameters or gradients, which may fail to capture how such deviations affect the functional behavior of the model. In particular, they provide limited insight into how different clients induce changes in the internal representation and decision boundaries learned by the nn ( nn ). Motivated by these limitations, we adopt a complementary perspective centered on the geometric structure induced by nn . In particular, we focus on how different clients organize the input space through their activation patterns, and how these differences can be leveraged to better understand heterogeneity and detect atypical behaviors in federated settings. From this standpoint, the main contributions of this work can be summarized as follows: • We introduce a geometric perspective to analyze client behavior in fl , based on the partition of the input space induced by activation patterns in nn . • We propose a lightweight metric to quantify the functional divergence between client and global models, capturing differences in how inputs are organized and processed beyond parameter-level comparisons. • We show that this metric provides an interpretable signal to monitor data heterogeneity and to identify atypical client contributions in federated learning systems. The remainder of the paper is organized as follows. Section II reviews related work on nn structure and fl security and privacy. Section III introduces the necessary definitions and the proposed metric. Section IV evaluates the metric on two different models and demonstrates its usefulness. Section V concludes the paper and discusses future research directions. II Related Work fl enables collaborative learning across distributed data holders without transferring raw data, typically through iterative rounds of local optimization and server-side aggregation [ 9 ] . In practice, a major obstacle is the statistical heterogeneity of client data, which often leads to unstable convergence and degraded global performance. Classical approaches such as FedProx mitigate excessive local deviation by introducing a proximal term [ 7 ] , while adaptive server-side optimization methods such as FedOpt improve robustness under heterogeneous data distributions [ 14 ] . Other works address specific forms of non- iid ness, such as feature shift affecting batch-normalization statistics ( FedBN ) [ 8 ] , or personalization-oriented methods that decompose the model into shared and client-specific components, such as FedRep and FedBABU [ 4 , 11 ] . In contrast to these approaches, our goal is not to modify the optimization procedure itself, but to provide a lightweight signal that characterizes how much a client functionally deviates from the global model. A complementary line of work studies nn through the geometry induced by their activation patterns. In relu ( relu )-based nn , sign patterns define a partition of the input space into linear or piecewise-affine regions, whose number and geometry are closely related to expressivity and representation structure [ 10 , 13 ] . Recent works have begun to analyze the dynamics of such patterns during training. Xu et al. [ 15 ] study convergence properties once activations are fixed, whereas Hartmann et al. [ 5 ] analyze activation statistics such as entropy and similarity. From a security and privacy viewpoint, fl is often motivated by keeping raw data local, but this alone does not guarantee either privacy or integrity. On the privacy side, secure aggregation protocols aim to prevent the server from inspecting individual client updates by only revealing their aggregate sum [ 2