FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices

Dezhong Yao, Yuexin Shi, Tongtong Liu, Zhiqiang Xu · 2025

Read FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices by Dezhong Yao, Yuexin Shi, Tongtong Liu, Zhiqiang Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-08518.txt.

Introduction

Dezhong Yao and co-authors design federated learning for weak edge devices in a 2025 paper. Federated learning trains one model across many devices without pooling their raw data. Standard rounds of back-and-forth training strain small devices, so one-shot versions send updates just once. Trouble is, devices differ wildly in power, and mixed fleets break most one-shot schemes.

Their framework, FedMHO, splits the work by strength. Strong clients train deep classifiers, while weak ones run light generative models. The server then creates synthetic data and fuses everyone's knowledge in two stages, with variants that fight knowledge loss during fusion and clean up the synthetic samples. Across test setups, the approach beat leading rivals, suggesting mixed fleets can still train one shared model well.

Catalog record

CREATORDezhong Yao, Yuexin Shi, Tongtong Liu, Zhiqiang Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-08518

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices Dezhong Yao, Yuexin Shi, Tongtong Liu, Zhiqiang Xu Abstract Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional FL introduces significant computation and communication overhead, which is unfriendly for resource-constrained edge devices. One-shot FL has emerged as a promising approach to mitigate communication overhead, and model-heterogeneous FL solves the problem of diverse computing resources across clients. However, existing methods face challenges in effectively managing model-heterogeneous one-shot FL, often leading to unsatisfactory global model performance or reliance on auxiliary datasets. To address these challenges, we propose a novel FL framework named FedMHO, which leverages deep classification models on resource-sufficient clients and lightweight generative models on resource-constrained devices. On the server side, FedMHO involves a two-stage process that includes data generation and knowledge fusion. Furthermore, we introduce FedMHO-MD and FedMHO-SD to mitigate the knowledge-forgetting problem during the knowledge fusion stage, and an unsupervised data optimization solution to improve the quality of synthetic samples. Comprehensive experiments demonstrate the effectiveness of our methods, as they outperform state-of-the-art baselines in various experimental setups. Our code is available at https://github.com/YXShi2000/FedMHO . I Introduction With the advancement and adoption of Edge Computing (EC) [ 1 ] , edge devices continuously generate vast amounts of data [ 2 ] . This data is crucial for the development of Artificial Intelligence (AI). However, the traditional paradigm of centralized AI model training, which aggregates all data on a central server, has become increasingly difficult to achieve due to growing concerns over data privacy and security. Federated Learning (FL) has emerged as a promising paradigm for training machine learning models across distributed devices without sharing raw data [ 3 ] . Despite impressive theoretical and experimental advancements, FL still faces notable challenges [ 4 , 5 ] in EC scenarios, such as healthcare [ 6 ] , recommendation systems [ 7 ] , and financial services [ 8 ] . A considerable challenge is the need for multiple communication rounds between several clients and a central server, which can be costly and intolerable due to the associated time and energy constraints [ 9 , 10 ] . Moreover, frequent communication poses a high risk of privacy attacks [ 11 , 12 ] , such as a man-in-the-middle attack [ 13 ] or the potential for reconstructing training data from gradients [ 14 ] . To address the communication and security challenges in conventional FL, the concept of one-shot FL has been introduced [ 15 ] , which aims to obtain an acceptable global model within a single communication round. Fig. 1: The heterogeneous one-shot Federated Learning (FL) framework. The clients with varying computing capabilities deploy heterogeneous local models. Each client communicates with the central server only once. After fully training their local models on private data, clients send these models to the server, where they are aggregated into a global model. Another challenge in real-world FL scenarios for EC is the heterogeneity of client computing resources [ 16 ] . For example, a naive solution for developing a health monitoring model via FL involves training local models on each user’s smartphone, smartwatch, or smart wristband, followed by the weighted averaging of their parameters. Smartphones typically have 4GB-16GB of RAM, while the Xiaomi Smart Wristband 8 offers 1.4MB of RAM, and the Amazfit Band 7 provides 8MB of RAM. This heterogeneity of client computing resources is common in EC applications of FL [ 17 ] . Existing FL frameworks typically require clients to deploy homogeneous local models. However, deploying uniformly small models prevents clients with abundant computing resources from fully utilizing their computational potential. Conversely, deploying uniformly large models excludes clients with limited computing resources from participating in the FL process, thus preventing the global model from acquiring knowledge from these resource-constrained clients. As a result, the deployment of homogeneous local models can degrade the performance of the final global model. A prevalent method for addressing computing heterogeneity is to deploy heterogeneous models on different clients, according to their respective computing capabilities [ 18 ] . Therefore, in FL scenarios with a mix of resource-sufficient clients and resource-constrained clients, as illustrated in Figure 1 , studying model-heterogeneous one-shot FL becomes critical. The existing one-shot FL methods [ 19 , 20 ] still face various challenges in model heterogeneity scenarios. DOSFL [ 21 ] adopts dataset distillation [ 22 ] on each client. However, sending distilled data to the central server introduces additional communication costs, which goes against the original intention of addressing the communication bottleneck in one-shot FL. Some other methods [ 15 , 23 ] leverage knowledge distillation [ 24 ] to aggregate local models with auxiliary public data. However, the performance of these methods depends both on the size of the auxiliary data and on its domain similarity to the local data [ 25 ] . Several data-free methods [ 19 , 20 , 26 ] have been proposed to address these issues. DENSE [ 19 ] and Co-Boosting [ 26 ] train an additional generator on the server side to generate auxiliary data that resembles the local data distribution. However, the efficacy of this generator heavily depends on the local model used for its training. If clients with limited computing resources deploy lightweight local models, whose performance is restricted, the generator’s performance will suffer, thereby hindering the final global model from achieving high performance. FEDCVAE [ 20 ] deploys generative models on all participating clients, with the global model on the server being trained from scratch using the synthetic samples generated by these generators. Nevertheless, using synthetic samples to convey local data information is less effective than directly using local model parameters. A more detailed explanation can be found in Section III . In this paper, we propose a novel method, FedMHO, to address the challenge of M odel- H eterogeneous O ne-shot Fed erated Learning. Our method involves deploying deep classification models on resource-sufficient clients while utilizing lightweight generative models on resource-constrained clients. In this paper, we adopt Conditional Variational Autoencoders (CVAE) [ 27 ] as the generative models to validate the effectiveness of our methods. More complex generators can be employed for more complex tasks. During global model training, FedMHO introduces a two-stage process encompassing data generation and knowledge fusion. In the data generation stage, the decoders received from clients generate synthetic samples based on local label distribution. To improve the fidelity of the synthetic samples, we employ an unsupervised data optimization solution. In the subsequent knowledge fusion stage, the global model is initialized by the average parameters of the local classification models and then updated based on the synthetic samples generated in the data generation stage. Furthermore, during the global model training, we propose two solutions named FedMHO-MD and FedMHO-SD to prevent the forgetting of knowledge from classification models. In FedMHO-MD, the local classification models act as multiple teacher models to distill the global model. FedMHO-SD involves self-distillation, where the initialized global model acts as a teacher to guide the training of the global model. Our main contributions are summarized as follows. • We propose FedMHO, a

Read the complete text