FedEL: Federated Elastic Learning for Heterogeneous Devices
Letian Zhang, Bo Chen, Jieming Bian, Lei Wang, Jie Xu · 2025
Read FedEL: Federated Elastic Learning for Heterogeneous Devices by Letian Zhang, Bo Chen, Jieming Bian, Lei Wang, Jie Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-16902.txt.
Introduction
Letian Zhang and co-authors speed federated learning on mixed devices in a 2025 paper. Phones and sensors train a shared model without sending raw data away. Devices vary widely in power, though. Slow stragglers hold up each round for everyone. Their elastic method, FedEL, lets weak devices do less while still contributing.
A tensor module balances local and global importance to fight data bias. Tests show time to a set accuracy fell by up to several fold. Final accuracy held or rose versus baselines. Elastic training could make federated systems practical on the messy mix of devices people actually own.
Catalog record
| CREATOR | Letian Zhang, Bo Chen, Jieming Bian, Lei Wang, Jie Xu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-16902
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FedEL: Federated Elastic Learning for Heterogeneous Devices Letian Zhang, Bo Chen, Jieming Bian, Lei Wang, Jie Xu Abstract Federated learning (FL) enables distributed devices to collaboratively train machine learning models while maintaining data privacy. However, the heterogeneous hardware capabilities of devices often result in significant training delays, as straggler clients with limited resources prolong the aggregation process. Existing solutions such as client selection, asynchronous FL, and partial training partially address these challenges but encounter issues such as reduced accuracy, stale updates, and compromised model performance due to inconsistent training contributions. To overcome these limitations, we propose FedEL, a federated elastic learning framework that enhances training efficiency while maintaining model accuracy. FedEL introduces a novel window-based training process, sliding the window to locate the training part of the model and dynamically selecting important tensors for training within a coordinated runtime budget. This approach ensures progressive and balanced training across all clients, including stragglers. Additionally, FedEL employs a tensor importance adjustment module, harmonizing local and global tensor importance to mitigate biases caused by data heterogeneity. The experiment results show that FedEL achieves up to 3.87× improvement in time-to-accuracy compared to baselines while maintaining or exceeding final test accuracy. 1 Introduction Federated learning (FL) [ 30 , 29 , 5 , 39 , 51 , 16 , 25 , 38 ] is a privacy-preserving machine learning paradigm where distributed clients, such as mobile devices and IoT systems, collaboratively train a global model while keeping their data local. Typically, FL involves devices performing local model training and sharing parameters with a central server for global model updates. However, heterogeneous hardware capabilities among devices lead to “straggler”, or slower clients, causing significant training delays as the server must wait for their updates. This challenge hinders the scalability of FL, particularly in large-scale cross-device scenarios. Status Quo and Their Limitations. To address computational constraints, existing solutions fall into three main categories: client selection, asynchronous FL, and partial training. Client Selection (Figure 1 , top-left). Selecting a subset of devices for training based on specific criteria can mitigate delays. However, significant differences in clients’ data distributions often leave stragglers underrepresented, reducing the global model’s accuracy. Asynchronous FL (Figure 1 , top-right). This approach decouples local training from global aggregation, allowing stragglers to train independently. While this reduces delays, the global model often relies on faster clients, leaving stragglers’ contributions infrequent and potentially outdated, which may harm convergence [ 45 ] . Partial Training (Figure 1 bottom left). Techniques like width and depth scaling adjust the model architecture to accommodate varying resources. Width scaling resizes convolutional layers, risking channel mismatches during aggregation [ 17 ] , while depth scaling limits training to early layers, leading to suboptimal task-specific features and reduced performance [ 43 ] . These limitations highlight the need for a novel training paradigm to overcome resource heterogeneity and enable high-performance FL in real-world deployments. Figure 1 : Existing works vs. FedEL. ElasticTrainer [ 14 ] introduces a method for selecting important deep neural network (DNN) tensors to meet runtime training requirements on a single device . By focusing on these key tensors, ElasticTrainer accelerates training. When applied to FL, it offers a potential solution for addressing stragglers by allowing each client to select important tensors based on its hardware capabilities under a unified runtime constraint. This ensures that all clients complete local training within a similar timeframe, enabling synchronized global model aggregation. However, directly deploying ElasticTrainer in FL scenarios has two limitations: Limitation \# 1 Limited Training Scope on Slower Clients: Due to the chained rule of DNN backward propagation, unselected tensors still compute gradients to propagate updates to the selected tensors. This constrains the selected tensors on slower devices to the back-end of the DNN, reducing training on the front-end feature extraction layers and degrading FL accuracy, especially with heterogeneous data distributions. Limitation \# 2 Exacerbated Local Model Drift: Variations in data distribution cause significant differences in tensor importance across clients. Training only the important tensors amplifies local model drift, where client models diverge from the global model, further reducing accuracy. Overview of the Proposed Approach. Motivated by the above limitations, we propose FedEL, a federated elastic learning framework that enhances federated training efficiency. To address the first limitation , we propose a window-based training approach that divides the DNN model into multiple blocks, ensuring that each part of the model is trained during FL rounds. Before training, we use a tensor timing profiler to measure the training time for each tensor, which is then aggregated into block-level training times. In each FL round, the window slides to include a set of blocks based on the runtime budget and current training status. The sliding window process involves moving the front edge to include deeper blocks and shrinking the end edge to exclude blocks that no longer require training. ElasticTrainer is then modified to select important training tensors within the selected window, allowing straggler clients to progressively train the crucial tensors of the entire DNN model. To address the second limitation , we design a tensor importance adjustment module. At the start of each FL round, the client estimates the global model’s tensor importance using the global models from the current and previous rounds, along with the learning rate. This global tensor importance is used to adjust the local tensor importance computed by ElasticTrainer, ensuring tensor selection considers both local and global data distributions. Evaluation. We implement FedEL on both a hardware testbed and software simulations. The hardware testbed consists of ten NVIDIA Jetson devices connected wirelessly to a server. To simulate large-scale scenarios, we extend the setup with a diverse client simulation. We evaluate FedEL using various DNN models and four real-world FL datasets across three key tasks: image classification, voice command recognition, and next-word prediction. Our results show that: (1) FedEL outperforms baselines on the time-to-accuracy. Specifically, FedEL outperforms FedAvg by 3.87\times in time-to-accuracy while final test accuracy is on par with or even higher than FedAvg. (2) FedEL reduces memory overhead and energy consumption during training compared to existing methods. (3) Ablation studies confirm the necessity and importance of each key component in FedEL’s design. 2 Related Work On-single-device training. Leveraging mobile and embedded computing for DNN model training has gained attention [ 53 ] . Some studies reduce computation by quantizing or pruning gradient propagation for certain neurons [ 4 , 11 , 35 , 15 ] . Others use a two-stage paradigm, where the system prepares the computing graph and generates a training plan before model training [ 32 , 46 , 40 , 27 , 10 , 14 ] . Our work builds on ElasticTrainer [ 14 ] , which dynamically selects important tensors during training. However, applying single-device methods directly to FL scenarios can be challenging due to heterogeneous systems and data. Heterogeneous federated learning. To address the challenges posed by low-end devices in FL, three main training methodologies have been proposed: (1) client selection, (2) asynchronous FL, and (3) partial training. Client selection methods [ 20 , 23 , 7 , 43 ] evaluate the utility of each client and select a subset to participate in FL rounds. For example, PyramidFL [ 23 ] ranks clients based