FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification
Ziqiao Weng, Weidong Cai, and Bo Zhou · 2025
Read FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification by Ziqiao Weng, Weidong Cai, and Bo Zhou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18981.txt.
Introduction
Ziqiao Weng, Weidong Cai, and Bo Zhou present a new way for hospitals to train AI together in a 2025 paper. Federated learning lets sites train models without sharing raw patient data. Sites often need distinct model shapes to fit their own machines. Past methods lean on a central server, which slows scale.
Their system, FedSKD, passes models from site to site in a ring. No central gathering step is needed at all. Knowledge moves through a teaching method that works at three levels of detail. Tests cover brain scans for autism checks and skin lesion sorting. FedSKD beat prior methods on both personal fit and cross-site skill. The authors see it as a robust path for medical AI.
Catalog record
| CREATOR | Ziqiao Weng, Weidong Cai, and Bo Zhou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18981
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FedSKD: Aggregation-free Model-heterogeneous Federated Learning via Multi-dimensional Similarity Knowledge Distillation for Medical Image Classification Ziqiao Weng, Weidong Cai, and Bo Zhou Abstract Federated learning (FL) enables privacy-preserving collaborative model training without direct data sharing. Model-heterogeneous FL (MHFL) enables clients to train personalized models with heterogeneous architectures, but existing methods mainly rely on centralized aggregation or require partially identical architectures, limiting scalability and efficiency. Current peer-to-peer (P2P) FL frameworks, though removing server dependence, have not been adapted to heterogeneous models and suffer from model drift and knowledge dilution. To address these challenges, we propose FedSKD, a novel P2P MHFL framework for medical image classification that facilitates direct knowledge exchange through round-robin model circulation, eliminating the need for centralized aggregation while allowing fully heterogeneous model architectures across clients. FedSKD’s key innovation lies in multi-dimensional similarity knowledge distillation, which enables bidirectional cross-client knowledge transfer at batch, pixel/voxel, and region levels for heterogeneous models in FL. This approach mitigates catastrophic forgetting and model drift through progressive reinforcement and distribution alignment while preserving model heterogeneity. Extensive evaluations on fMRI-based autism spectrum disorder diagnosis and skin lesion classification demonstrate that FedSKD outperforms state-of-the-art heterogeneous and homogeneous FL baselines, achieving superior personalization and cross-institutional generalization. These findings underscore FedSKD’s potential as a scalable and robust solution for real-world medical federated learning. I Introduction Artificial intelligence (AI)-based medical image analysis is fundamentally constrained by limited annotated data and strict privacy regulations. Medical imaging data are costly to acquire and annotate, while regulations such as HIPAA [ 1 ] , GDPR [ 2 ] , and the Cyber Security Law [ 3 ] severely restrict centralized data sharing across institutions. These challenges motivate learning paradigms that can leverage multi-institutional data without direct data aggregation. Fig. 1: Comparison of traditional MHFL schemes (a), traditional sequential P2P FL schemes (b), and our proposed MHFL framework (c). Traditional MHFL methods rely on a central server for knowledge fusion and require partially identical model structures across clients. Traditional P2P FL trains a homogeneous model sequentially across clients. In contrast, our FedSKD framework operates without a central server and allows fully heterogeneous model architectures while enabling direct knowledge transfer among clients. Federated learning (FL) enables collaborative model training across multiple institutions without direct data sharing [ 4 , 5 , 6 ] . Existing FL paradigms can be broadly categorized into central server-based and peer-to-peer approaches. In server-based FL, data remain decentralized at local institutions, while a centralized server coordinates the iterative optimization of a shared global model. The widely adopted FedAvg algorithm [ 7 ] updates the global model by averaging client model weights; however, its naive aggregation strategy often performs poorly under non-IID data distributions, which are prevalent in medical imaging due to heterogeneous acquisition protocols, scanner types, and patient demographics [ 8 , 9 ] . Such distribution mismatches can impair convergence and generalization [ 10 , 11 ] . To mitigate these issues, several extensions have been proposed, including FedProx [ 12 ] , FedBN [ 13 ] , as well as approaches based on domain adaptation [ 4 , 14 , 15 ] and knowledge distillation [ 16 , 11 , 17 , 18 ] . Peer-to-peer federated learning (P2P FL) [ 19 , 20 , 21 ] removes the centralized server and enables direct model exchange among clients. As in server-based FL, non-IID data remains a fundamental challenge. However, classic P2P FL typically adopts a sequential learning protocol, where a single model is iteratively fine-tuned across clients rather than synchronously aggregated. This sequential exposure to heterogeneous data distributions causes the model to repeatedly adapt to shifting optimization objectives, leading to pronounced parameter oscillations and unstable convergence [ 22 ] , a phenomenon commonly referred to as model drift . Moreover, as the model progressively adapts to each new client, knowledge acquired from earlier clients is susceptible to interference or overwriting, resulting in a gradual erosion of previously learned representations, which we denote as knowledge dilution . While other challenges in decentralized FL, such as communication overhead, computational redundancy, and privacy risks, have been extensively studied and partially mitigated [ 23 , 24 , 25 , 26 , 27 , 28 ] , the absence of effective mechanisms to address sequential model drift and knowledge dilution remains a key obstacle to robust convergence and sustained performance in P2P FL. While data heterogeneity has been widely studied in FL, model heterogeneity remains a critical yet underexplored challenge. Variations in clients’ computational resources and application-specific requirements often necessitate the use of heterogeneous model architectures [ 29 , 30 , 31 ] . Existing model-heterogeneous FL (MHFL) methods rely heavily on centralized aggregation (Figure 1 (a)) and generally follow two strategies, representation-level knowledge transfer [ 29 , 32 , 33 ] or partial weight aggregation [ 34 , 35 , 36 ] , but both face limitations such as reliance on public data, restricted architectural flexibility, vulnerability to gradient conflicts under non-IID settings, and scalability constraints imposed by the central server. In contrast, P2P FL enables collaboration among fully heterogeneous client models without server-side aggregation, offering a more flexible and scalable alternative. However, existing P2P FL frameworks still suffer from model drift and knowledge dilution. Furthermore, whether based on parameter aggregation or alternative synchronization strategies, they generally assume a single homogeneous model that is sequentially trained across clients, as illustrated in Figure 1 (b). This assumption limits the flexibility of P2P frameworks in model-heterogeneous scenarios, where clients often use customized architectures tailored to their specific needs. To address these issues, we propose FedSKD, a novel model-heterogeneous P2P FL framework based on multi-dimensional similarity knowledge distillation (SKD). FedSKD introduces two key innovations. First, it enables decentralized knowledge transfer by circulating locally trained heterogeneous models among clients in a round-robin manner, eliminating the gradient conflicts and server overhead inherent in traditional MHFL methods, as shown in Figure 1 (c). In each training round, each client receives a model from the previous client, performs bidirectional knowledge distillation, and passes its enhanced model to the next client. This closed-loop knowledge reinforcement chain ensures that critical features are progressively amplified, rather than diluted, counteracting the problem of knowledge dilution. Second, FedSKD incorporates multi-dimensional similarity knowledge distillation to facilitate efficient cross-client knowledge reinforcement among heterogeneous models. Specifically, we introduce three novel SKD mechanisms that operate purely at the feature representation level: Batch-wise SKD (B-SKD) to align activation patterns across batches, Pixel/Voxel-wise SKD (P-SKD) to align spatial feature distributions, and Region-wise SKD (R-SKD) to align representations within anatomical regions. During training, clients simultaneously distill domain-specific knowledge into received models while absorbing cross-institutional insights from these models, creating a bidirectional knowledge exchange that facilitates knowledge reinforcement and prevents model drift through continuous distribution alignment. We evaluate FedSKD on two clinically significant medical imaging tasks: fMRI-based autism spectrum disorder (ASD) diagnosis and skin lesion classification. Experimental results demonstrate that FedSKD outperforms existing methods in both personalization (client-specific performance) and generalization (cross-institutional adaptability), while addressing the challenges outlined above. These findings highlight FedSKD’s potential to significantly advance federated learning in resource-constrained, need-variant, privacy-sensitive medical applications. Our key contributions can be summarized as follows: • We propose the first peer-to-peer model-heterogeneous federated learning (P2P-MHFL) framework tailored for medical image classification, enabling clients with