Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning
Shunxin Guo, Jiaqi Lv, and Xin Geng · 2025
Read Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning by Shunxin Guo, Jiaqi Lv, and Xin Geng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-19103.txt.
Introduction
Federated learning trains one model across many devices without gathering their data, but dropping the central server makes sharing harder. Shunxin Guo, Jiaqi Lv, and Xin Geng tackle that trade-off in 2025 for devices linked in a ring, where each talks only to its neighbors.
When each device's data differs, purely personal models drift apart and lose the gains of teamwork. The proposed framework splits the job in two: one module learns features tuned to local data, and a second module pulls out knowledge that stays the same everywhere. Tests against leading methods showed stronger results across varied data splits. The design aims to keep personal fit and shared strength at the same time.
Catalog record
| CREATOR | Shunxin Guo, Jiaqi Lv, and Xin Geng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-19103
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning Shunxin Guo, Jiaqi Lv, and Xin Geng Abstract We introduce Ring-topology Decentralized Federated Learning (RDFL) for distributed model training, aiming to avoid the inherent risks of centralized failure in server-based FL. However, RDFL faces the challenge of low information-sharing efficiency due to the point-to-point communication manner when handling inherent data heterogeneity. Existing studies to mitigate data heterogeneity focus on personalized optimization of models, ignoring that the lack of shared information constraints can lead to large differences among models, weakening the benefits of collaborative learning. To tackle these challenges, we propose a Divide-and-conquer RDFL framework (DRDFL) that uses a feature generation model to extract personalized information and invariant shared knowledge from the underlying data distribution, ensuring both effective personalization and strong generalization. Specifically, we design a PersonaNet module that encourages class-specific feature representations to follow a Gaussian mixture distribution, facilitating the learning of discriminative latent representations tailored to local data distributions. Meanwhile, the Learngene module is introduced to encapsulate shared knowledge through an adversarial classifier to align latent representations and extract globally invariant information. Extensive experiments demonstrate that DRDFL outperforms state-of-the-art methods in various data heterogeneity settings. I introduction Federated Learning (FL) is an emerging machine learning paradigm that enables multiple clients to collaboratively train a global model while keeping each client’s private raw data stored locally [ 1 , 2 , 3 ] . One of the main challenges in FL is data heterogeneity, which arises from significant distributional differences among clients [ 4 , 5 , 6 , 7 ] . Generally, effective solutions to mitigate this issue tend to focus on improving model generalization to better accommodate a wider range of clients, or enhancing model personalization to better adapt to local data distributions. Several classic Centralized Federated Learning (CFL) methods have been proposed, which regularize the model using a global model copy from the server [ 8 , 9 , 10 ] or decouple local models into shared parameters for server-side aggregation and partial personalization parameters for client-side customization [ 11 , 12 , 13 ] . In addition, some approaches use both global and personalized classifiers downloaded from the server for prediction [ 11 , 14 , 15 ] , aiming to mitigate inherent data shift issues due to data heterogeneity, thereby improving model performance. However, these methods generally rely on the presence of a central server, and this dependence on a centralized architecture can introduce problems such as single points of failure and server-client communication bottlenecks. Fig. 1 : Illustration of the optimized learning mechanism in RDFL and comparison of the personalization and generalization performance of various advanced methods. (a) shows the schematic of the ring optimization approach, where client 1 receives the model from client 4, trains and updates it using its private dataset, and then transmits the locally updated model to client 2. The model is iteratively optimized over the ring topology. (b) presents the performance comparison of personalization (Local-T) and generalization (Global-T) on different methods across four clients with different data distributions. The results show that our proposed method effectively achieves both personalization and generalization goals in federated learning scenarios with data heterogeneity. To avoid the limitations of server-based architecture, we focus on the Ring-topology Decentralized Federated Learning (RDFL) architecture that performs distributed model training [ 16 , 17 , 18 ] using a client-directed communication manner, as shown in Figure 1 (a). However, its inherent architecture is challenging in addressing data heterogeneity, mainly due to the drift of data distribution across clients and the weakening of information sharing efficiency by point-to-point communication. The lack of effective global shared information for guidance leads to instability in the local model optimization process, exacerbating inconsistencies between client models. Existing DFL methods mainly focus on modeling individual user preferences through partial personalization or sparse masking of the model [ 19 , 20 , 16 ] , but these often overlook the information necessary for global generalization. The decoupled partial personalization models capture individual knowledge but fail to effectively leverage the remaining shared parameters to encode more generalizable knowledge [ 21 , 22 ] . This is also confirmed by Figure 1 (b) with FL methods (local, FedRep [ 12 ] , and DFedPGP [ 18 ] ) focusing on personalized optimization. Inspired by the recently proposed Learngene paradigm 1 1 1 ”Learngene” refers to a learning paradigm, while ” Learngene ” denotes a model component encapsulating shared knowledge. [ 23 , 24 , 25 ] , which encapsulates shared knowledge within lightweight models to facilitate efficient task model initialization and adaptation to diverse data distributions, we propose a D ivide-and-conquer R ing-topology D ecentralized F ederated L earning (DRDFL) method to enhance communication efficiency and personalized learning capabilities. To extract globally generalized knowledge that is independent of specific clients, we design a transferable Learngene component that adversarial training using an adversarial classifier under a uniform class distribution constraint, allowing it to learn unbiased shared knowledge. Meanwhile, each client model adapts to its specific class distribution to address inconsistencies in data distributions across clients. To achieve this, we design a unique PersonaNet component for each client, which undergoes personalized training based on the Gaussian distribution modeled by the mean and covariance of each class, and use the global Gaussian distribution of the corresponding class for auxiliary learning. The resulting personalized representation not only incorporates global information relevant to the target class but also enhances adaptation to heterogeneous data distributions. During inference, the invariant representation from Learngene is combined with the personalized representation from PersonaNet and fed into a decoder to reconstruct augmented data and generate the final prediction. Within the ring-topology framework, the Learngene component and global implicit class representations undergo iterative optimization and sharing across clients, improving both generalization and personalization capabilities. Comprehensive experimental results demonstrate that the proposed DRDFL method effectively addresses data heterogeneity while improving communication efficiency, achieving a dual optimization of model generalization and personalization. To sum up, our contributions are as follows: • We analyze the challenges of data heterogeneity and inefficient information sharing faced by RDFL and argue that the essence is to harmonize the high generalization and personalized effectiveness of the model. • We propose a novel method DRDFL that uses a feature generation process to extract both underlying shared knowledge across clients and personalized knowledge specific to each client. • We empirically show the effectiveness of DRDFL, which outperforms eight SOTA methods by up to 3.28% in local test accuracy with only 0.58 MB of shared information among clients in each iteration. II Related Work II-A Decentralized Federated Learning Federated learning (FL) has emerged as a promising research direction in the field of data privacy preservation [ 26 , 27 , 28 , 29 ] . Due to the heterogeneity of computational and communication resources among clients, decentralized federated learning (DFL) has gained increasing attention in recent years. In this paradigm, clients connect solely with their neighbors through point-to-point communication to collaboratively train a consensus model. We investigate various approaches to personalization in DFL under multi-round local optimization. The classic DisPFL [ 16 ] designs personalized models and pruning masks for each client, thereby accelerating personalized convergence. KD-PDFL [ 30