DFPL: Decentralized Federated Prototype Learning Across Heterogeneous Data Distributions

Hongliang Zhang, Fenghua Xu, Zhongyuan Yu, Shanchen Pang, Chunqiang Hu, Jiguo Yu · 2025

Read DFPL: Decentralized Federated Prototype Learning Across Heterogeneous Data Distributions by Hongliang Zhang, Fenghua Xu, Zhongyuan Yu, Shanchen Pang, Chunqiang Hu, Jiguo Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04947.txt.

Introduction

Hongliang Zhang and five co-authors propose DFPL, a way to run federated learning without a central server. In ordinary federated learning, many clients train a shared model and a server combines their updates, which leaves the whole system exposed if that server fails. DFPL removes the server and uses a blockchain so that training and mining run locally on each client. It adds prototype learning to cope with the fact that different clients hold very different data.

The design also cuts the number of parameters clients must exchange. The team analyzed how the method converges, modeling the computing resources that training and mining require. On four benchmark datasets with uneven data splits, DFPL beat the decentralized methods it was compared with on both model performance and communication efficiency.

Catalog record

CREATORHongliang Zhang, Fenghua Xu, Zhongyuan Yu, Shanchen Pang, Chunqiang Hu, Jiguo Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-04947

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DFPL: Decentralized Federated Prototype Learning Across Heterogeneous Data Distributions Hongliang Zhang, Fenghua Xu, Zhongyuan Yu, Shanchen Pang, Chunqiang Hu, Jiguo Yu Abstract Federated learning is a distributed machine learning paradigm through centralized model aggregation. However, standard federated learning relies on a centralized server, making it vulnerable to server failures. While existing solutions utilize blockchain technology to implement Decentralized Federated Learning (DFL), the statistical heterogeneity of data distributions among clients severely degrades the performance of DFL. Driven by this issue, this paper proposes a decentralized federated prototype learning framework, named DFPL, which significantly improves the performance of DFL under heterogeneous data distributions. Specifically, DFPL introduces prototype learning into DFL to mitigate the impact of statistical heterogeneity and reduces the amount of parameters exchanged between clients. Additionally, blockchain is embedded into our framework, enabling the training and mining processes to be executed locally on each client. From a theoretical perspective, we analyze the convergence of DFPL by modeling the required computational resources during both training and mining. The experiment results highlight the superiority of DFPL in both model performance and communication efficiency across four benchmark datasets with heterogeneous data distributions. I Introduction Federated Learning (FL) [ 1 ] , a distributed machine learning paradigm, allows multiple clients to collaboratively train a global model by centralized aggregation without sharing raw data. However, the centralized aggregation suffers from serious security threats. ( i ) The global model is inaccurate if the server is attacked (e.g., data tampering attack [ 2 ] ). ( ii ) FL is interrupted if the server fails due to physical damage [ 3 ] [ 4 ] . To solve these threats, Blockchain-assisted FL (BFL) frameworks [ 5 , 6 , 7 , 8 , 9 ] are proposed, replacing the central server with a third-party blockchain network. Nevertheless, the aggregation stage in BFL frameworks is susceptible to pooling and collusion attacks from third-party blockchain networks [ 10 , 11 , 12 ] . The risk of pooling or collusion attacks undermines the decentralization of blockchain, compromising its security and degrading FL performance. To mitigate these threats, Decentralized Federated Learning (DFL) solutions [ 13 , 14 , 2 , 15 , 16 , 17 ] have emerged. The DFL framework integrates training and mining into the clients, allowing them to communicate directly with neighboring nodes without relying on any third-party. ((a)) MNIST ((b)) CIFAR10 Fig. 1: The performance of DFL using the FedAvg [ 1 ] aggregation strategy in IID and Non-IID settings, where the Non-IID data is simulated using a Dirichlet distribution [ 18 ] with a concentration parameter of 0.1. However, in practical scenarios, the performance of DFL rapidly degrades when the data distribution is statistically heterogeneous [ 19 ] [ 20 ] . Specifically, the local data stored on clients is usually non-independent and identically distributed (Non-IID), which leads to data heterogeneity. The Non-IID data causes divergent local optimal solutions among clients, resulting in performance degradation of DFL, as shown in Fig. 1(b) . This is because each client’s model tends to adapt to its own data distribution during local training, thereby converging to a local optimum suitable for the local distribution rather than the global data distribution [ 21 ] . When these local optima are aggregated, the resulting global model deviates from the true global optimum, which leads to suboptimal FL performance. Researchers have been devoted to addressing the performance degradation of FL caused by Non-IID data [ 22 , 23 , 24 , 25 ] . They focus on aligning local models with global models to improve the consistency of local models. However, these works [ 22 , 23 , 24 , 25 ] rely on a fixed central server to perform model aggregation and global optimization. This design contradicts the decentralized nature of DFL, where no individual client can consistently act as a server throughout federation training process. Therefore, how to improve the performance of DFL under heterogeneous data distributions is a current challenge. Fortunately, prototype learning has gradually attracted attention in FL [ 26 ] [ 27 ] . These works leverage prototype learning to build global representations by aggregating prototypes submitted by clients. The so-called “prototype” can be viewed as an abstract representation of a class [ 28 ] . For instance, when recognizing a “cat”, each client has its unique “imaginary picture” or “prototype” to capture the feature of “cat”. By exchanging these prototypes, clients can share the abstract representations of classes like “cat”. This prototype-based knowledge integration is independent of local class distributions, mitigating the negative impacts caused by heterogeneous data distributions. Consequently, a critical question is: can we leverage prototype learning to address the statistical heterogeneity in decentralized federated learning and achieve high-performance federated training? To answer this question, we propose a decentralized federated prototype learning framework, named DFPL, which enhances the performance of distributed learning under heterogeneous data distributions while reducing the amount of parameters transmitted among clients. The key novelty of our work lies in: ( i ) DFPL integrates blockchain into federated prototype learning, enabling each client to perform both training and mining locally, which facilitates direct communication with neighboring nodes without relying on any third-party. ( ii ) A novel local optimization function is designed leveraging prototype learning, where clients exchange only prototypes instead of model parameters or gradients, which mitigates the impact of statistical heterogeneity in decentralized federated learning. ( iii ) We model the computational resources required by DFPL for both training and mining, and further derive the relationship between its convergence and computational resource requirements. Our main contributions can be highlighted as follows: • We introduce prototype learning into DFL and propose a decentralized federated prototype learning framework (DFPL), in which each client aligns its prototypes with those of the other clients while minimizing its own classification loss, which effectively address the challenges caused by Non-IID data in DFL. • We provide theoretical convergence analyses of DFPL by incorporating the computational resource allocation between training and mining processes, providing theoretical guidance for deploying DFPL in resource-constrained environments. • We experimentally examine the impact of the trade-off between training and mining on model performance under limited computational resource settings. In addition, the experiments show that the DFPL achieves communication efficiency and enhanced performance across four benchmark datasets with different Non-IID settings. The rest of the paper is organized as follows. Section II discusses recent works related to blockchain in FL and prototype learning. A detailed presentation of DFPL is provided in Section III . Section IV provides the theoretical analysis about DFPL. Section V reviews the experimental results. Finally, Section VI concludes this paper. II Related Work This section reviews research on federated learning and prototype learning schemes. II-A FL with Blockchain Due to privacy and security concerns, accessing data from distributed clients is challenging for the server. To address this concerns, Google proposed FL, where each client submits its local model update to a central server. The server then aggregates these local updates to obtain a global model update, thereby effectively preserving data privacy. However, centralized aggregation is vulnerable to central server attacks and server failures. To mitigate the potential threats posed by centralized aggregation, existing works [ 5 ,

Read the complete text