Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning
Xi Chen, Qin Li, Haibin Cai, Ting Wang · 2025
Read Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning by Xi Chen, Qin Li, Haibin Cai, Ting Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16966.txt.
Introduction
Xi Chen and co-authors personalize federated learning for mixed fleets of devices using two reinforcement learning agents, in a paper from two thousand twenty-five. Federated learning trains shared models without pooling private data. But devices differ in shape and power, which drags accuracy and leaves slow stragglers.
One agent hands each client a right-sized model, and another tunes how hard each trains. Clients also run a small shared twin model, and the pair teach each other by distillation. That twin carries global knowledge back into local training. Benchmarks cut total training time by up to about forty percent and straggler delay by nearly half.
Catalog record
| CREATOR | Xi Chen, Qin Li, Haibin Cai, Ting Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-16966
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Heterogeneity-aware Personalized Federated Learning via Adaptive Dual-Agent Reinforcement Learning Xi Chen, Qin Li, Haibin Cai, Ting Wang Abstract Federated Learning (FL) empowers multiple clients to collaboratively train machine learning models without sharing local data, making it highly applicable in heterogeneous Internet of Things (IoT) environments. However, intrinsic heterogeneity in clients’ model architectures and computing capabilities often results in model accuracy loss and the intractable straggler problem, which significantly impairs training effectiveness. To tackle these challenges, this paper proposes a novel Heterogeneity-aware Personalized Federated Learning method, named HAPFL, via multi-level Reinforcement Learning (RL) mechanisms. HAPFL optimizes the training process by incorporating three strategic components: 1) An RL-based heterogeneous model allocation mechanism. The parameter server employs a Proximal Policy Optimization (PPO)-based RL agent to adaptively allocate appropriately sized, differentiated models to clients based on their performance, effectively mitigating performance disparities. 2) An RL-based training intensity adjustment scheme. The parameter server leverages another PPO-based RL agent to dynamically fine-tune the training intensity for each client to further enhance training efficiency and reduce straggling latency. 3) A knowledge distillation-based mutual learning mechanism. Each client deploys both a heterogeneous local model and a homogeneous lightweight model named LiteModel, where these models undergo mutual learning through knowledge distillation. This uniform LiteModel plays a pivotal role in aggregating and sharing global knowledge, significantly enhancing the effectiveness of personalized local training. Experimental results across multiple benchmark datasets demonstrate that HAPFL not only achieves high accuracy but also substantially reduces the overall training time by 20.9%-40.4% and decreases straggling latency by 19.0%-48.0% compared to existing solutions. I Introduction With the continuous advancement of Internet of Things (IoT) technology [ 1 ] , the deployment of IoT devices is experiencing unprecedented growth. As of 2021, there are already over 10 billion active IoT devices worldwide, and this number is expected to exceed 25.4 billion by 2030. By 2025, an estimated 152,200 IoT devices will be connecting to the Internet every minute, contributing to a projected increase in data generation to 73.1 Zettabytes by that year—a dramatic 422% rise from the 17.3 Zettabytes recorded in 2019. Traditionally, such vast amounts of data have been managed through centralized learning (CL) methods [ 2 ] [ 3 ] , which aggregate data to train or fine-tune models. Although CL is effective for developing high-precision models and straightforward to implement, it relies heavily on centralized data collection. This dependency raises substantial privacy and security concerns, particularly as IoT devices often hold sensitive information, such as medical records containing patients’ private health data or personal sensitive banking details. Data owners’ reluctance to share such data compromises the feasibility of CL for practical applications. Moreover, the CL model, necessitating extensive data collection, incurs significant communication costs, which make it impractical for resource-constrained devices operating in bandwidth-limited edge wireless networks, thereby restricting the scalability and applicability of the CL approach in the rapidly evolving IoT environment. Federated Learning (FL) embodies a collaborative learning paradigm, specifically designed to address privacy concerns. This approach utilizes a distributed learning algorithm that allows clients to independently process local datasets without the necessity to exchange raw data [ 4 ] . In FL system, the server distributes a global model to each participating client, who then utilizes their unique local dataset to train a local model. Upon completion of training, clients transmit their model parameters to the server, which aggregates these contributions to refine the global model. This strategy ensures that the global model can effectively leverage the collective data from all participating clients without compromising individual data privacy, thus making FL highly practical and broadly applicable across various data-sensitive sectors. Despite its vast potential, FL encounters significant implementation challenges that hinder its widespread adoption. In addition to managing data privacy, one primary issue in FL arises from the computational and communication efficiency constraints that occur within networks of heterogeneous devices [ 5 ] [ 6 ] [ 7 ] . In synchronous FL systems, performance disparity—defined here as any factors that impact client training in a given round and contribute to training latency—can include specific elements such as computational resources, dataset size, and model size. This disparity often leads to “straggler problems,” where high-performance devices must wait for slower devices to complete their tasks before proceeding to aggregation. This delay can severely impact resource utilization and model efficiency, underscoring the need for methods that can dynamically adapt to the varied capacities of different clients. In synchronous FL, the disparity in performance among heterogeneous devices often causes “straggler problems”, where high-performance devices are delayed as they must wait for lower-performance devices to complete training and data upload before aggregation can proceed [ 8 ] [ 9 ] . One common strategy to mitigate this issue and reduce straggling latency involves allocating different training intensities based on client performance, where “training intensity” in this context refers to varying the number of training epochs assigned to each client during federated training rounds. However, while this approach can enhance the efficiency of high-performance devices, it also risks exacerbating disparities. Specifically, allocating too many epochs to high-performance devices and too few to lower-performance ones can lead to an imbalance, which might not only decrease the accuracy of the aggregated model due to under-representation of data from less capable devices but also result in suboptimal utilization of all client data across the network. While Semi-FL [ 10 ] and Asyn-FL [ 11 ] approaches have been proposed to address training latency in federated learning, each comes with notable limitations. Asyn-FL improves scalability and efficiency by allowing clients to update models asynchronously, yet it often struggles with stability in model convergence, especially in environments with numerous nodes or significant computational delays. Similarly, Semi-FL seeks a balance between global consistency and performance efficiency by introducing a delay window for client updates. However, its effectiveness depends heavily on carefully tuning this window, which complicates deployment in large-scale systems. To address the challenges outlined above, this paper introduces a novel adaptive Heterogeneity-aware Personalized Federated Learning method, named HAPFL. HAPFL leverages two deep reinforcement learning (DRL) agents to dynamically allocate appropriately sized heterogeneous models and tailored training intensities to clients based on their performance capabilities, respectively. Furthermore, to tackle the obstacles posed by model heterogeneity during global model aggregation, each client is additionally equipped with a homogeneous lightweight model, named LiteModel. This setup allows each client to train its personalized local model under intensities specified by the reinforcement learning (RL) agent, while the LiteModel engages in mutual learning with the local model through knowledge distillation [ 12 ] . These well-learned homogeneous LiteModels are then used to perform global model aggregation. This dual-model approach not only facilitates efficient and effective global model aggregation but also enhances the local model with insights gained from the LiteModel, which accumulates and transfers knowledge across all clients. Moreover, to mitigate the adverse impacts of low-contributing clients on the accuracy and stability of the global model, this paper proposes a novel model aggregation method using information entropy and accuracy weighting. In summary, this paper makes the following three key contributions: 1. We propose a novel heterogeneity-aware personalized federated learning framework, named HAPFL, that leverages two functional RL agents. These two agents are respectively designed to adaptively determine