Few-for-Many Personalized Federated Learning
Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li, Zhi-Ri Tang, Qingfu Zhang · 2026
Read Few-for-Many Personalized Federated Learning by Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li, Zhi-Ri Tang, Qingfu Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-11992.txt.
Introduction
Guo and co-authors reshape personalized federated learning as a few-for-many problem. Instead of one model per client, a few shared server models serve all clients at once. The team proves this setup can stay near optimal for each client. Error shrinks as the shared model count grows and as client data grows. Their FedFew method, with just three shared models, beat leading rivals on vision, language, and medical imaging tests.
Catalog record
| CREATOR | Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li, Zhi-Ri Tang, Qingfu Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67363 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.11992
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Few-for-Many Personalized Federated Learning Ping Guo, Tiantian Zhang, Xi Lin, Xiang Li, Zhi-Ri Tang, Qingfu Zhang Abstract Personalized Federated Learning (PFL) aims to train customized models for clients with highly heterogeneous data distributions while preserving data privacy. Existing approaches often rely on heuristics like clustering or model interpolation, which lack principled mechanisms for balancing heterogeneous client objectives. Serving M M clients with distinct data distributions is inherently a multi-objective optimization problem, where achieving optimal personalization ideally requires M M distinct models on the Pareto front. However, maintaining M M separate models poses significant scalability challenges in federated settings with hundreds or thousands of clients. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only K K shared server models ( K ≪ M K\ll M ) to collectively serve all M M clients. We prove that this framework achieves near-optimal personalization: the approximation error diminishes as K K increases and each client’s model converges to each client’s optimum as data grows. Building on this reformulation, we propose FedFew, a practical algorithm that jointly optimizes the K K server models through efficient gradient-based updates. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew automatically discovers the optimal model diversity through its optimization process. Experiments across vision, NLP, and real-world medical imaging datasets demonstrate that FedFew, with just 3 models, consistently outperforms other state-of-the-art approaches. Code is available at https://github.com/pgg3/FedFew . 1 Introduction Personalized Federated Learning (PFL) 37 aims to train client-specific models that best fit each client’s local data distribution by leveraging aggregated knowledge from collaborative learning across the federation. This paradigm overcomes a fundamental limitation of traditional Federated Learning (FL) 29 , where a single global model struggles to effectively serve all clients when their data are drawn from vastly different distributions P_{i} under the non-IID setting. The benefit of personalized collaborative learning has made PFL particularly valuable in domains such as healthcare 28 and finance 5 , where heterogeneous data distributions necessitate client-specific models while privacy constraints require decentralized training. The heterogeneity of client data changes the nature of the optimization landscape in PFL 22 ; 39 . When clients have distinct data distributions P_{i}\neq P_{j} , a model update that benefits one client may harm another due to their inherent conflicting objectives 45 . For instance, in a healthcare federation where hospitals in urban and rural areas have vastly different patient demographics, optimizing model accuracy for urban hospitals might learn feature representations that poorly capture rural patient characteristics 26 ; 13 . This inherent conflict naturally leads to a multi-objective optimization perspective 37 ; 16 . Rather than seeking a single consensus model, PFL must navigate trade-offs among M distinct client loss functions \{L_{1},L_{2},\ldots,L_{M}\} , where each client i requires a model tailored to its local data distribution P_{i} . While achieving optimal personalization would ideally require learning M distinct models, maintaining and training such a large number of separate models introduces prohibitive scalability challenges in federated settings involving hundreds or thousands of clients. Figure 1 : Paradigms of Personalized Federated Learning. Left: Centralized methods maintain a single global model for all M clients, failing to capture client heterogeneity. Center: Per-client methods train M independent models, sacrificing collaborative learning benefits and suffering from data scarcity. Right: Our proposed few-for-many approach maintains K server models ( K\ll M ) that collectively serve all clients. Each client selects the best-fitting model, achieving strong personalization while preserving collaboration. Given this scalability challenge, existing PFL methods struggle to effectively balance personalization and computational efficiency. Methods that explicitly adopt the multi-objective perspective, such as FedMGDA 16 and FedMTL 37 , only obtain a single model on the Pareto front 30 , which is the set of all optimal trade-off solutions. Consequently, they cannot provide individual optima for each client. Meanwhile, most PFL methods resort to heuristics without theoretical Pareto optimality guarantees: clustering-based methods like IFCA 12 and CFL 35 train one model per group; interpolation-based methods like APFL 8 and Ditto 21 mix global and local models using ad-hoc weights; and per-client methods like FedRep 6 train M independent models, thereby sacrificing collaboration benefits. In summary, existing approaches face a significant limitation: multi-objective methods produce only a single Pareto-optimal model without personalization, while heuristic methods generate M personalized models without Pareto optimality guarantee. To address this challenge, we reformulate PFL as a few-for-many optimization problem that maintains only K server models that collectively serve all M clients (where K\ll M ) as illustrated in Figure 1 , where each client selects the model that best fits its local data distribution. We rigorously prove that the K -for- M framework achieves near-optimal personalization through a precise error decomposition. Our analysis establishes two vanishing error components: the Pareto coverage gap from using K<M models diminishes as K increases, and the statistical error between empirical and population losses vanishes as client dataset sizes grow. These dual convergence guarantees distinguish our approach from existing heuristic PFL methods. Building upon this theoretical foundation, we propose FedFew (Federated Learning with Few Models), a novel algorithm that enables efficient gradient-based optimization in federated settings. The core challenge lies in jointly optimizing the K server models alongside discrete client-model assignments, which is incompatible with standard gradient-based methods. We address this through a two-level smoothing technique that transforms the discrete selection problem into a fully differentiable objective. This formulation enables clients to perform soft model selection via gradient descent, while the server jointly updates all K models to collaboratively cover all client needs. Unlike clustering-based approaches that require manual client partitioning or interpolation-based methods that demand careful hyperparameter tuning, FedFew can automatically discover the optimal model diversity through its optimization process. The algorithm seamlessly integrates with standard federated learning protocols, incurring only minimal communication overhead compared to single-model training. Our contributions are three-fold: • We introduce the few-for-many optimization framework that reformulates PFL as maintaining K shared models ( K\ll M ) to serve M clients, addressing the scalability challenge with rigorous convergence guarantees through Pareto coverage gap and statistical error decomposition. • We develop FedFew, a practical federated algorithm that solves the few-for-many problem via two-level smoothing, enabling automatic model diversity discovery through gradient-based optimization without manual client clustering or delicate hyperparameter tuning. • We demonstrate through extensive experiments on seven datasets, including real-world medical imaging application, that FedFew consistently outperforms existing methods while utilizing only 3 models. 2 Related Work 2.1 Standard and Personalized Federated Learning Standard Federated Learning. Traditional approaches in federated learning aim to learn a single global model by aggregating local updates from distributed clients. Starting with FedAvg 29 , which introduced weighted averaging, subsequent methods like FedProx 22 , SCAFFOLD 18 , and FedDyn 1 have brought improvements in training convergence and stability. While FL methods perform well under the IID data assumption, real-world FL problems often exhibit significant data heterogeneity across clients. This mismatch can lead to degraded performance and slow convergence 45 ; 20 . Therefore, personalized federated learning approaches have been proposed to address this challenge by tailoring models to individual client distributions while still leveraging collaborative learning benefits. Personalized Federated Learning. Existing PFL methods can be roughly grouped into