FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning

Fu Peng, Ming Tang · 2025

Read FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning by Fu Peng, Ming Tang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21054.txt.

Introduction

Peng and Tang design a federated learning system that adapts when each client's data drifts in this 2025 paper. Federated learning trains a shared model across many devices without pooling data. Drift comes in flavors: the input mix shifts, the label mix shifts, or the true input-output link changes.

Most defenses handle only the last kind, and they forget useful history. FedDAA first decides how many client clusters exist. A detector then separates true link drift from mere mix drift. The adaptation module discards stale data only for true drift and keeps history otherwise. Grouping similar clients adds stability. The design targets the catastrophic forgetting that plagues drifting federations.

Catalog record

CREATORFu Peng, Ming Tang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-21054

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning Fu Peng, Ming Tang Abstract In federated learning (FL), the data distribution of each client may change over time, introducing both temporal and spatial data heterogeneity, known as concept drift. Data heterogeneity arises from three drift sources: real drift (shift in P ⁡ ( y | 𝐱 ) P(y|\mathbf{x}) ), virtual drift (shift in P ⁡ ( 𝐱 ) P(\mathbf{x}) ), and label drift (shift in P ⁡ ( y ) P(y) ). However, most existing FL methods addressing concept drift primarily focus on real drift. When clients experience virtual or label drift, these methods often fail to selectively retain useful historical knowledge, leading to catastrophic forgetting. A key challenge lies in distinguishing different sources of drift, as they require distinct adaptation strategies: real drift calls for discarding outdated data, while virtual or label drift benefits from retaining historical data. Without explicitly identifying the drift sources, a general adaptation strategy is suboptimal and may harm generalization. To address this challenge, we propose FedDAA, a dynamic clustered FL framework designed to adapt to multi-source concept drift while preserving valuable historical knowledge. Specifically, FedDAA integrates three modules: a cluster number determination module to determine the optimal number of clusters; a real drift detection module to distinguish real drift and virtual/label drift; and a concept drift adaptation module to adapt to new data while retaining useful historical data. We provide theoretical convergence guarantees, and experiments show that FedDAA achieves 7.84%–8.52% accuracy improvements over state-of-the-art methods on Fashion-MNIST, CIFAR-10, and CIFAR-100. 1 Introduction Federated learning (FL) is a decentralized machine learning approach that enables multiple clients to collaboratively train a shared model while keeping their data localized. Existing FL frameworks assume that client data distributions remain stable over time. However, this assumption often fails to hold in real-world scenarios. For example, user preferences for movie genres may evolve temporally; respiratory disease case counts commonly exhibit seasonal variations. As the data distributions of different clients could change at different time steps and even evolve in different directions (e.g., towards distinct distributions), this leads to data heterogeneity in both time and space in FL. This phenomenon is referred to as concept drift in FL, shown in Fig. 1 . Data heterogeneity in either time or space can be triggered by three sources [ 1 , 2 , 3 ] . Specifically, suppose there are two datasets that follow joint probability distributions P_{1}(\mathbf{x},y) and P_{2}(\mathbf{x},y) respectively, where \mathbf{x} is the feature vector and y is the label. The two datasets may be collected from the same client at different time steps (temporal heterogeneity), or from different clients at the same time step (spatial heterogeneity). Concept drift occurs if P_{1}(\mathbf{x},y)\neq P_{2}(\mathbf{x},y) . The joint probability P(\mathbf{x},y) can be decomposed into two components, i.e., P(\mathbf{x},y)=P(\mathbf{x})P(y|\mathbf{x}) or P(y)P(\mathbf{x}|y) . Three sources 1 1 1 Readers may raise concerns about the absence of a fourth source: P_{1}(\mathbf{x}|y)\neq P_{2}(\mathbf{x}|y) while P_{1}(y)=P_{2}(y) . The reasons are twofold: (i) existing works are primarily interested in predicting labels given features (i.e., P(y|\mathbf{x}) ) [ 1 , 2 , 4 ] ; (ii) for the fourth scenario, identical labels y may correspond to different features \mathbf{x} (e.g. due to seasonal variations or cultural heterogeneity) [ 2 ] , which can be viewed as a form of virtual drift. of data heterogeneity can be defined as: (i) Real drift arises when P_{1}(y|\mathbf{x})\neq P_{2}(y|\mathbf{x}) while P_{1}(\mathbf{x})=P_{2}(\mathbf{x}) ; (ii) Virtual drift occurs when P_{1}(\mathbf{x})\neq P_{2}(\mathbf{x}) while P_{1}(y|\mathbf{x})=P_{2}(y|\mathbf{x}) ; (iii) Label drift occurs when P_{1}(y)\neq P_{2}(y) while P_{1}(\mathbf{x}|y)=P_{2}(\mathbf{x}|y) . Among these three sources, real drift refers to changes in the conditional distribution P(y\mid\mathbf{x}) , leading to a change in the decision boundary, 2 2 2 A decision boundary is the surface that separates different classes for a classification task in machine learning. whereas virtual drift and label drift involve changes in the marginal distribution P(\mathbf{x}) or the label distribution P(y) , without affecting the decision boundary. The change in the decision boundary significantly degrades model performance. Thus, we classify real drift as a separate category, while grouping virtual drift and label drift together. Figure 1 : FL under concept drift. There are two dimensional data heterogeneity: time and space. “A”, “B” and “C” represent different data distributions. To deal with data heterogeneity in space (i.e., across clients), existing works [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 ] proposed clustered FL methods. Clustered FL is a framework that groups clients into clusters based on their local data distributions to address the data heterogeneity across clients. The idea of clustered FL stems from the fact that no single global model can perform well for all clients, especially when clients’ data follow different conditional distributions P(y|\mathbf{x}) (i.e., decision boundaries) [ 8 , 14 ] . Recently, [ 4 ] proposes a principle of clustering: separating clients with different data conditional distributions into different clusters. However, existing clustered FL methods [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 ] fail to consider temporal data heterogeneity and consequently do not adjust the number of clusters over time. When considering temporal heterogeneity, most existing FL concept drift methods [ 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ] primarily focus on real drift and often overlook the presence of other drift types such as virtual and label drift. As a result, these methods often mistakenly treat virtual and label drift as real drift, and discard historical data that could still be beneficial when virtual or label drift occurs. However, in such cases, the decision boundaries between historical and current data often remain similar, meaning that historical data can still provide useful knowledge and enhance model generalization. Consequently, these methods fail to simultaneously adapt to new data distributions and retain valuable historical knowledge, leading to catastrophic forgetting and making them suboptimal for handling complex and dynamic environments. To address the above challenges, we aim to: (i) adaptively adjust the number of clusters, and (ii) distinguish real drift from virtual or label drift, enabling adaptation to new data distributions while retaining useful historical knowledge. Specifically, we formulate three research questions (RQs) as follows. RQ 1 (Dynamic Number of Clusters) . The number of clusters (the number of data conditional distributions P(y|\mathbf{x}) ) dynamically evolves over time in clustered FL. How to determine the optimal number of clusters at each time step? RQ 2 (FL Concept Drift Detection and Adaptation) . How can we detect real drift to distinguish it from virtual and label drift? Based on the distinction between real drift and virtual/label drift, how can we design two adaptation strategies: one for real drift and another for virtual/label drift? RQ 3 (Convergence Analysis) . What are the theoretical convergence guarantees in clustered FL under concept drift? To resolve the aforementioned research questions, we propose FedDAA, a novel clustered FL approach that dynamically adapts to multi-source concept drift while preserving useful historical knowledge. Our approach consists of three main modules. First, since the number of clusters evolves over time ( RQ

Read the complete text