Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset
Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham · 2025
Read Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset by Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12031.txt.
Introduction
Nguyen, Nguyen, and Pham tackle federated continual learning when each site's data arrive as a stream and sites may hold unrelated tasks. Hospitals and phones, for instance, learn in sequence without sharing raw data, but clashing data and noise breed false patterns and forgetting. Their 2025 method, STAMP, replaces generated replay data with compact prototypes.
A model free rule picks sample subsets that act as prototypes of past tasks. Gradient matching then runs across time at each client and across space at the server, aligning updates so old skills survive new data. Prototypes stand in for task gradients on the client side. Wide experiments show the method beating prior baselines, the authors report, in this harder, mixed task setting.
Catalog record
| CREATOR | Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-12031
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving Generalization in Heterogeneous Federated Continual Learning via Spatio-Temporal Gradient Matching with Prototypical Coreset Minh-Duong Nguyen, Le-Tuan Nguyen, Quoc-Viet Pham Abstract Federated Continual Learning (FCL) has recently emerged as a crucial research area, as data from distributed clients typically arrives as a stream, requiring sequential learning. This paper explores a more practical and challenging FCL setting, where clients may have unrelated or even conflicting data and tasks. In this scenario, statistical heterogeneity and data noise can create spurious correlations, leading to biased feature learning and catastrophic forgetting. Existing FCL approaches often use generative replay to create pseudo-datasets of previous tasks. However, generative replay itself suffers from catastrophic forgetting and task divergence among clients, leading to overfitting in FCL. To address these challenges, we propose a novel approach called S patio- T emporal gr A dient M atching with network-free P rototype (STAMP). Our contributions are threefold: 1) We develop a model-agnostic method to determine subset of samples that effectively form prototypes when using a prototypical network, making it resilient to continual learning challenges; 2) We introduce a spatio-temporal gradient matching approach, applied at both the client-side (temporal) and server-side (spatial), to mitigate catastrophic forgetting and data heterogeneity; 3) We leverage prototypes to approximate task-wise gradients, improving gradient matching on the client-side. Extensive experiments demonstrate our method’s superiority over existing baselines. 1 Introduction Federated Learning (FL) is a distributed and privacy-preserving approach that facilitates collaboration among various entities, such as organizations or devices ( Lim et al. 2020 ) . In FL, multiple clients train a shared model in coordination with a central server without exchanging personal data. Recently, FL has gained significant attention across multiple domains, including healthcare ( Nguyen et al. 2023a ) , Internet-of-Things (IoT) ( Nguyen et al. 2023b ) , and autonomous driving ( Fantauzzo et al. 2022 ) . While conventional FL studies often assume static data classes and domains, real-world scenarios involve the continuous emergence of new classes and evolving data distributions ( Elsayed & Mahmood 2024 ) . Training entirely new models to accommodate these changes is impractical due to the substantial computational resources required. An alternative approach is transfer learning from pre-trained models; however, this method is prone to catastrophic forgetting ( Dong et al. 2024 ) , which degrades performance on previously learned classes. To address catastrophic forgetting in FL, recent research ( Luo et al. 2023 ) has introduced the concept of Federated Continual Learning (FCL), which integrates the principles of FL and Continual Learning (CL) ( Yang et al. 2023 ) . (a) Generalization gap on CIFAR100 (b) Generalization gap on ImageNet1K Figure 1: The illustration highlights the challenges encountered by current FCL methods (e.g., AF-FCL ( Wuerkaixi et al. 2024 ) , FedSSI ( Li et al. 2025a ) ) when applied in heterogeneous settings. A notable gap between local and global test accuracy arises due to client-specific data heterogeneity within each task at every time step. STAMP demonstrates superior robustness over current baselines by mitigating inter-client divergence throughout the learning process, leading to a reduced local-global generalization gap. In FCL, clients collaboratively learn models for their private, sequential tasks while preserving data privacy. However, due to the sequential nature of these tasks, each client only has access to a limited amount of data from the current task. This constraint often leads to the loss of previously acquired knowledge, resulting in catastrophic forgetting. Unlike existing studies, our focus is on a more challenging setting of FCL, coined heterogeneous FCL. This setting introduces two major challenges beyond those encountered in conventional FCL. First, in heterogeneous FCL, clients are engaged in non-identical tasks at any given time, resulting in a non-uniform learning environment (see Figure 1b ). As a consequence, the aggregated global model is subject to domain shifts at each communication round, impeding efficient convergence. Second, existing Federated Class-Incremental Learning (FCIL) methods perform poorly under heterogeneous FCL conditions. This is primarily because these methods typically assume a homogeneous class distribution across clients, enabling a straightforward model design where new task-specific heads are incrementally added. In contrast, heterogeneous FCL involves clients encountering different sets of classes at each task, leading to discrepancies in the architecture of local model heads. This variability substantially reduces the effectiveness of existing FCIL approaches in heterogeneous settings. To address the aforementioned challenges, we propose a novel method, dubbed Federated Continual Learning via S patio- T emporal gr A dient M atching with network-free P rototype (STAMP). Our approach offers three key contributions: First, we apply unified gradient matching across both spatial and temporal dimensions of the FCL system. Specifically, the spatial dimension refers to task differences across clients at a given time, while the temporal dimension refers to sequential tasks within a single client. By aligning gradients across both dimensions, our method enables the identification of aggregated gradients that minimize negative transfer both across sequential tasks and between clients, thereby improving knowledge retention and overall model performance. Second, we utilize a prototypical network to generate prototypes that serve as stable gradient approximators, even in the presence of data perturbations. This design ensures more consistent gradient estimates of previous tasks when viewed through the lens of the current model—a property often overlooked in existing gradient-based approaches ( Luo et al. 2023 ; Lopez-Paz & Ranzato 2017 ; Saha et al. 2021 ; Deng et al. 2021 ) . Third, we introduce a prototypical coreset selection strategy for efficient and robust prototype approximation. This approach offers two key advantages. (1) By carefully selecting a compact set of representative samples (coresets), our method maintains prototype quality and diversity over time with significantly reduced dependence on the prototypical networks or generative replay mechanisms used in prior work ( Wei et al. 2023 ; Li et al. 2024a ; Chen et al. 2023 ; Goswami et al. 2023 ; Qi et al. 2023 ; Zhen et al. 2020 ) , both of which are prone to catastrophic forgetting in FCL settings. (2) The coreset-based strategy substantially reduces memory usage, as only a small number of representative samples per class per task are required to approximate the prototypes effectively. 2 Backgrounds & Preliminaries 2.1 Federated Continual Learning FCL refers to a practical learning scenario that melds the principles of FL and CL. Suppose that there are U clients. On each client u , the model is trained on a sequence of T tasks. At a given step t\times R+r , where R represents the number of communication rounds per task and r is the current round of task t , client u holds model parameters \theta^{t,r}_{u} and only has access to the data from task t . On client u , data {\mathcal{D}}^{t}_{u} of task t consists of N^{t}_{u} pairs of samples and their labels, i.e., {\mathcal{D}}^{t}_{u}=\{(x^{t}_{i},y^{t}_{i})^{N^{t}_{u}}_{i=1}\} . In existing literature, the primary focus is on a specific task reshuffling setting, wherein the task set is identical for all clients, yet the arrival sequence of tasks differs ( Yoon et al. 2021 ) . In practical scenarios, it may be observed that the task set of clients is not necessarily correlated. There is no guaranteed relation among the tasks \{{\mathcal{D}}^{1}_{u},{\mathcal{D}}^{2}_{u},\ldots,{\mathcal{D}}^{T}_{u}\} of client u at different steps. Similarly, there is no guaranteed