GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning

Shunxin Guo, Jiaqi Lv, Qiufeng Wang, and Xin Geng · 2025

Read GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning by Shunxin Guo, Jiaqi Lv, Qiufeng Wang, and Xin Geng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14628.txt.

Introduction

Shunxin Guo and colleagues proposed a leaner way in 2025 to train AI models across many separate devices. In federated learning, phones or hospitals train locally and share updates, but real clients join and leave, and their data differ wildly. Sending full models back and forth wastes bandwidth and slows the start for each newcomer.

Their framework borrows a gene metaphor: small knowledge fragments, called learngenes, carry the core skill of a big model. Local training protects the most informative weights, squeezes the fragment down by sensitivity checks, and the server merges fragments into a stronger one. New clients can then start fast from the shared fragment instead of from scratch.

Catalog record

CREATORShunxin Guo, Jiaqi Lv, Qiufeng Wang, and Xin Geng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14628

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning Shunxin Guo, Jiaqi Lv, Qiufeng Wang, and Xin Geng Abstract Real-world F ederated L earning systems often encounter D ynamic clients with A gnostic and highly heterogeneous data distributions (DAFL), which pose challenges for efficient communication and model initialization. To address these challenges, we draw inspiration from the recently proposed Learngene paradigm, which compresses the large-scale model into lightweight, cross-task meta-information fragments. Learngene effectively encapsulates and communicates core knowledge, making it particularly well-suited for DAFL, where dynamic client participation requires communication efficiency and rapid adaptation to new data distributions. Based on this insight, we propose a Gene-driven parameter-efficient dynamic Federated Learning (GENE-FL) framework. First, local models perform quadratic constraints based on parameters with high Fisher values in the global model, as these parameters are considered to encapsulate generalizable knowledge. Second, we apply the strategy of parameter sensitivity analysis in local model parameters to condense the learnGene for interaction. Finally, the server aggregates these small-scale trained learnGene s into a robust learnGene with cross-task generalization capability, facilitating the rapid initialization of dynamic agnostic client models. Extensive experimental results demonstrate that GENE-FL reduces 4 × \times communication costs compared to FEDAVG and effectively initializes agnostic client models with only about 9.04 MB. I Introduction Federated Learning (FL) [ 1 ] has shown great promise in the field of distributed learning across devices, allowing multiple clients to collaboratively train a shared global model without exposing private data [ 2 , 3 ] . The integration of FL enhances the safety and efficiency of real-world AI applications in various domains, including medical impact analysis [ 4 , 5 ] , personalized recommendation systems [ 6 , 7 ] , and intelligent transportation systems [ 8 , 9 ] . In real-world applications, FL is increasingly being deployed in dynamic and diverse environments, where participating clients often exhibit agnostic classes and heterogeneous data distributions, forming a Dynamic Agnostic Federated Learning (DAFL) scenario. This introduces two critical challenges that must be addressed simultaneously: ❶Efficient Communication : The limited bandwidth and high communication costs in FL systems can significantly impact overall performance and scalability. ❷Rapid Adaptation to Agnostic Client Models : Dynamically joining clients may have agnostic data distributions, requiring effective model initialization to avoid expensive retraining and slow convergence. Common approaches to improve communication efficiency include model pruning and compression [ 10 , 11 ] , one-shot FL [ 12 , 13 , 14 , 15 ] , reducing local updates to control communication costs [ 10 ] , and using class prototypes to minimize communication overhead [ 16 , 17 ] . While these approaches are well-suited for FL scenarios with a stable number of clients and class distributions, they exhibit limitations in the DAFL setting. Specifically, pruned models or trained class prototypes are based only on information from fixed training data, making it difficult to generalize to agnostic clients with new data distributions or classes that were not included during the pruning process. This highlights the core goal of DAFL: How can we design a framework that enables efficient communication between server and clients while ensuring effective initialization of agnostic client models? Fig. 1: Illustration of Dynamic Agnostic Federated Learning and Learngene. In the accumulating, condensing, and inheriting processes of the Learngene, dynamic agnostic federated learning can achieve a corresponding organic integration. To achieve this goal, we are inspired by a novel Learngene paradigm [ 18 , 19 ] that involves learning from open-world data to accumulate knowledge for training a large-scale ancestry model. This model is then condensed into a lightweight learnGene 1 1 1 “Learngene” refers to the learning framework, while “ learnGene ” denotes the condensed information piece of the model. , which encapsulates shared meta-knowledge and is inheritable. Subsequently, models for different tasks inherit learnGene for rapid and effective initialization, as shown in Figure 1 (a). The ability of Learngene to efficiently encapsulate and transfer core knowledge makes it particularly well-suited for DAFL, where dynamic client participation demands both communication efficiency and rapid adaptation to new data distributions. Building on this insight, we propose a Gene-driven parameter-efficient dynamic Federated Learning framework (GENE-FL) tailored for DAFL scenarios, with the summarized process illustrated in Figure 1 (b): (i) Condensing : the known client models in the FL system are condensed into small-scale learnGene s through smooth updates. (ii) Collaborating : clients and the server engage in efficient communication based on learnGene s, where the server collaborates to aggregate the learnGene s uploaded by participated clients. (iii) Initializing : the agnostic clients dynamically join the FL system and utilize the learnGene s downloaded from the server to quickly initialize their models. The three processes in the framework complement each other and can be executed in parallel, enabling efficient communication and scalability. Specifically, to address the communication latency issue commonly encountered in FL systems (where the slowest participating client affects communication time when using a single global model) [ 20 ] , we introduce a one-shot clustering method to obtain multiple cluster models. Initially, each local model is smoothly updated based on its corresponding cluster model, absorbing the knowledge from other participants. To ensure the generalization of the condensed learnGene across data distributions, we impose quadratic constraints on the local model using the elastic learnGene that is obtained by partitioning the cluster model based on Fisher information. Second, each client performs sensitivity analysis on parameter updates to extract learnGene , facilitating ❶ efficient communication with the server. Finally, the server dynamically updates the cluster learnGene based on the uploaded learnGene s, enabling ❷ rapid adaptation to agnostic client models. In summary, we analyze the adaptability of the Learngene framework to the goal of DAFL, and based on this, propose the first method that leverages Learngene technology to simultaneously address the dual challenges. II Related Work Federated Learning. Federated Learning (FL) is a distributed learning framework designed to safeguard user data privacy [ 1 , 21 , 22 ] . Compared to centralized learning, FL faces several unique challenges, including non-independently identically distributed and imbalanced data, as well as limited communication bandwidth [ 23 , 24 , 25 , 26 ] . An intuitive approach to reduce communication costs is to quantify weights directly and upload them sparsely [ 27 , 28 , 29 , 30 ] . The FedDrop [ 31 ] reduces the computational burden of local training and the corresponding communication costs in FL. The transferable model design in FEDLPS [ 32 ] uses an adaptive channel model pruning algorithm. Efforts have also been directed towards one-shot FL [ 12 , 13 ] , which aims to achieve satisfactory models with only one round of communication, but requires the shared dataset. Additionally, using class prototypes for low-cost communication has gained attention [ 16 ] . Further, FedTGP [ 17 ] proposes using adaptive margin-enhanced contrastive learning to train global prototypes on the server. However, most existing studies focus on FL scenarios where the number of participating clients and the number of classes remain stable, with limited attention given to the critical issue of model initialization and optimization when new classes or clients are dynamically introduced. In contrast, we propose a Gene-driven dynamic agnostic FL framework, which

Read the complete text