DySTop: Dynamic Staleness Control and Topology Construction for Asynchronous Decentralized Federated Learning

Yizhou Shi, Qianpiao Ma, Yan Xu, Junlong Zhou, Ming Hu · 2025

Read DySTop: Dynamic Staleness Control and Topology Construction for Asynchronous Decentralized Federated Learning by Yizhou Shi, Qianpiao Ma, Yan Xu, Junlong Zhou, Ming Hu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-01996.txt.

Introduction

Yizhou Shi and co-authors published this distributed computing paper in 2025. Federated learning trains models across many edge devices without pooling their data, but the usual design leans on a central server. Removing the server means devices trade models peer to peer. If they trade whenever they like, updates arrive stale and traffic balloons. The team's mechanism, DySTop, manages both problems at once.

DySTop controls staleness dynamically and rebuilds the trading network phase by phase to cut overhead and cope with uneven data across devices. A rigorous analysis ties the method's convergence to staleness limits, activation rates, and data spread. Two algorithms put the theory to work, one waking workers at the right times and one shaping the network. In simulations and a real testbed, DySTop cut completion time by about half and communication use by 57 percent, at equal model accuracy.

Catalog record

CREATORYizhou Shi, Qianpiao Ma, Yan Xu, Junlong Zhou, Ming Hu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-01996

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DySTop: Dynamic Staleness Control and Topology Construction for Asynchronous Decentralized Federated Learning Thanks: Corresponding authors: Qianpiao Ma, Junlong Zhou. Thanks: Yizhou Shi, Qianpiao Ma, Yan Xu and Junlong Zhou are with the School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, Jiangsu 210094, China (e-mail: [email protected], [email protected], [email protected], [email protected]). Thanks: Ming Hu is with the School of Computing and Information Systems, Singapore Management University, Singapore (e-mail: [email protected]). Thanks: Yunming Liao and Hongli Xu are with the School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui 230027, China (e-mail: [email protected], [email protected]). Yizhou Shi, Qianpiao Ma, Yan Xu, Junlong Zhou, Ming Hu Abstract Federated Learning (FL) has emerged as a potential distributed learning paradigm that enables model training on edge devices ( i.e. , workers) while preserving data privacy. However, its reliance on a centralized server leads to limited scalability. Decentralized federated learning (DFL) eliminates the dependency on a centralized server by enabling peer-to-peer model exchange. Existing DFL mechanisms mainly employ synchronous communication, which may result in training inefficiencies under heterogeneous and dynamic edge environments. Although a few recent asynchronous DFL (ADFL) mechanisms have been proposed to address these issues, they typically yield stale model aggregation and frequent model transmission, leading to degraded training performance on non-IID data and high communication overhead. To overcome these issues, we present DySTop, an innovative mechanism that jointly optimizes dynamic staleness control and topology construction in ADFL. In each round, multiple workers are activated, and a subset of their neighbors is selected to transmit models for aggregation, followed by local training. We provide a rigorous convergence analysis for DySTop, theoretically revealing the quantitative relationships between the convergence bound and key factors such as maximum staleness, activating frequency, and data distribution among workers. From the insights of the analysis, we propose a worker activation algorithm (WAA) for staleness control and a phase-aware topology construction algorithm (PTCA) to reduce communication overhead and handle data non-IID. Extensive evaluations through both large-scale simulations and real-world testbed experiments demonstrate that our DySTop reduces completion time by 51.8% and the communication resource consumption by 57.1% compared to state-of-the-art solutions, while maintaining the same model accuracy. I Introduction Recent advances in the Internet of Things (IoT) promote continuous generation of vast quantities of data generated from physical environments [ 1 , 2 ] . Generally, this data is sent to a remote cloud for processing, raising concerns about potential privacy leakage and considerable latency. In response, edge computing has evolved as a viable solution by distributing computation capacity to edge devices ( i.e. , workers) for accelerated local data processing. Furthermore, it promotes the adoption of federated learning (FL), which coordinates workers to perform distributed machine learning [ 3 ] while preserving data privacy. Traditional FL, also known as Centralized FL (CFL), comprises a set of workers and a centralized parameter server (PS) [ 3 , 4 , 5 ] . These workers collaboratively train a global model by performing local updates on their own datasets and subsequently transmitting their model parameters to the PS. The PS next aggregates these uploaded models and distributes the aggregated model back to the workers for the subsequent training round. This procedure usually maintains multiple training rounds until model convergence. Though CFL is a mature and widely used scheme, it comes with notable limitations. First, its reliance on a star topology inherently restricts system scalability. Second, frequent communication between the PS and workers generates a substantial traffic workload and creates a communication bottleneck at the PS, posing a significant risk of a single point failure. TABLE I: Comparison of different DFL mechanisms. DFL Mechanisms Handing Edge Heterogeneity Handing Edge Dynamic Communication Consumption Handling Non-IID Handling Staleness Synchronous BrainTorrent [ 6 ] Poor Poor High Poor - GossipFL [ 7 ] Poor Poor High Poor - D-Cliques [ 8 ] Poor Poor Medium Good - MATCHA [ 9 ] Poor Medium Low Poor - L2PL [ 10 ] Poor Medium Low Poor - FedHP [ 11 ] Poor Medium Low Good - Asynchronous HADFL [ 12 ] Good Poor High Poor Poor AsyNG [ 13 ] Good Good Medium Good Poor AsyDFL [ 14 ] Good Good Medium Good Poor SA-ADFL [ 15 ] Good Good High Poor Medium DySTop (Ours) Good Good Low Good Good Decentralized federated learning (DFL) has emerged as a solution to overcome the limitations of CFL [ 6 , 7 , 9 , 10 , 11 , 8 ] . In DFL, each worker performs local model updating and transmits the updated models to its neighbors through peer-to-peer (P2P) communications. Since DFL does not rely on a centralized PS, it can achieve high scalability and construct network topology more flexibly. When a worker in the system fails, it is only necessary for the affected workers to re-establish P2P connections with other workers. Moreover, each worker communicates with its neighbors, effectively distributing the communication overhead that was originally borne solely by the PS in CFL. However, achieving high efficiency in DFL presents unique challenges that need to be addressed. • Edge Heterogeneity: Each worker interacts with heterogeneous neighbors with varying computational and communication capabilities, data sizes, and physical locations [ 16 ] . Therefore, the coordination required for training in DFL is more complex compared with CFL. • Edge Dynamic: The network topologies may change over time due to intermittent connections among workers caused by their mobility [ 17 ] . Furthermore, a worker with poor communication conditions may lead to disconnections for its neighbors. • Communication Resource Constraint: On one hand, since the absence of the PS, DFL typically requires more training rounds to converge than CFL. On the other hand, in DFL, each worker sends its model to multiple workers rather than a single PS, resulting in more communication resource consumption compared to CFL [ 7 ] . • Data Non-IID: The data collected on workers is often not a uniform sample of the overall distribution, leading to data non-independent-and-identically-distributed (non-IID) [ 3 ] , which degrades FL training. Even more critically, unlike the PS in CFL, which trains the global model over all data, each worker in DFL may obtain models trained over more biased data [ 13 ] . There are two communication schemes for DFL: synchronous and asynchronous. Most of the existing DFL researches adopt synchronous communication [ 6 , 7 , 8 , 9 , 10 , 11 ] , where each worker waits for all its neighbors to finish local model updating and aggregates them with its own local model. However, due to edge heterogeneity, workers are compelled to wait for the slowest worker to complete its local updating and model transferring, known as the straggler problem . Moreover, edge dynamics may render synchronous DFL impractical, as dynamic network conditions and potential disconnections can lead to significant idle time while workers wait for model transmissions from all neighbors in each round. To handle heterogeneity and dynamics in edge networks, the asynchronous DFL (ADFL) scheme [ 12 , 13 , 14 , 15 ] further eliminates the synchronization barrier in synchronous DFL. In ADFL, each worker is allowed to exchange models with its neighbors at any time and

Read the complete text