FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data

Eman M. AbouNassar, Amr Elshall, Sameh Abdulah · 2026

Read FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data by Eman M. AbouNassar, Amr Elshall, Sameh Abdulah (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13909.txt.

Introduction

AbouNassar, Elshall, and Abdulah propose FedPBS, a federated learning method built for uneven, non-identical client data. It adapts batch sizes to each client's resources, so participation stays balanced. Small-batch clients also get a proximal correction that keeps local updates near the global model. The method blends ideas from FedBS and FedProx. On CIFAR-10 and UCI-HAR under strong data skew, FedPBS beats FedBS, FedGA, MOON, and FedProx, with smooth, stable loss curves.

Catalog record

CREATOREman M. AbouNassar, Amr Elshall, Sameh Abdulah
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55550
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13909

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FedPBS: Proximal-Balanced Scaling Federated Learning Model for Robust Personalized Training for Non-IID Data Eman M. AbouNassar, Amr Elshall, Sameh Abdulah Abstract Federated learning (FL) enables a set of distributed clients to jointly train machine learning models while preserving their local data privacy, making it attractive for applications in healthcare, finance, mobility, and smart-city systems. However, FL faces several challenges, including statistical heterogeneity and uneven client participation, which can degrade convergence and model quality. In this work, we propose FedPBS, an FL algorithm that couples complementary ideas from FedBS and FedProx to address these challenges. FedPBS dynamically adapts batch sizes to client resources to support balanced and scalable participation, and selectively applies a proximal correction to small-batch clients to stabilize local updates and reduce divergence from the global model. Experiments on benchmarking datasets such as CIFAR-10 and UCI-HAR under highly non-IID settings demonstrate that FedPBS consistently outperforms state-of-the-art methods, including FedBS, FedGA, MOON, and FedProx. The results demonstrate robust performance gains under extreme data heterogeneity, with smooth loss curves indicating stable convergence across diverse federated environments. FedPBS consistently outperforms state-of-the-art federated learning baselines on UCI-HAR and CIFAR-10 under severe non-IID conditions while maintaining stable and reliable convergence. 1 Introduction In recent years, machine learning (ML) has made significant progress in domains such as autonomous driving [ 34 ] , computer vision [ 15 ] , image recognition [ 30 ] , and natural language processing [ 29 ] , thereby enhancing professional productivity and daily convenience. This rapid development has been fuelled by the availability of huge datasets and powerful computational resources [ 1 , 13 ] . However, the traditional method of accumulating extensive user data in centralized repositories raises significant privacy concerns, as aggregating sensitive information in a single location for training increases the risk of leakage and may reveal personal behavioural patterns and family-related information. These leaks can significantly impact privacy and raise concerns while AI technologies continue to advance across various fields. To address these issues, distributed computing has arisen as an alternative in which each client processes local data. It subsequently integrates with other local components to enhance the global system’s efficiency, reliability, and scalability. This approach has become increasingly prevalent, as growing awareness of data protection and tightening legal restrictions have severely constrained traditional data-sharing methods. Federated learning (FL) provides an effective framework for distributed machine learning by enabling clients to collaborate while preserving the privacy of their local data [ 3 , 19 , 22 , 24 ] . In FL, multiple clients train one model without exchanging raw data to reduce privacy risks and support secure collaboration across organizations [ 17 , 14 , 6 ] . Local updates are computed on each client and aggregated centrally, improving scalability, energy efficiency, and overall system performance. FL also enables compliant model development on sensitive datasets [ 42 , 39 , 33 , 8 ] , while enhancing robustness by leveraging diverse data sources. However, in practice, federated learning often faces significant heterogeneity challenges, including statistical and system heterogeneity [ 38 ] . Statistical heterogeneity arises from non-identically distributed data (non-iid) across clients, and system heterogeneity arises from differences in computational resources, hardware capabilities, and network stability. These variations complicate model aggregation and reduce the effectiveness of traditional algorithms designed for homogeneous environments. To address these issues, new optimization and coordination strategies are needed to balance contributions, handle unreliable clients, and maintain stable and fair global model convergence [ 17 , 16 , 23 ] . Building on the need to coordinate learning across distributed clients, Federated Averaging (FedAvg) is the earliest and perhaps the most widely used algorithm in FL [ 26 ] . It performs multiple local stochastic gradient descent (SGD) steps in parallel across a sampled subset of devices, and periodically averages the resulting model updates at a central server. Compared with traditional centralized SGD and its variants, FedAvg emphasizes local computation and significantly less communication, thereby improving efficiency in distributed environments. However, despite these advantages, FedAvg remains sensitive to non-IID data distributions and to fluctuations in client participation. When data heterogeneity and device imbalances are not properly managed, the global model may suffer from unstable convergence and biased outcomes [ 11 , 31 ] . To mitigate these limitations, several algorithms have been proposed in the literature to enhance the capabilities of federated optimization under heterogeneous conditions. For instance, FedProx introduces a proximal term into the local objective functions [ 41 ] , which serves as a regularizer, keeping local updates closer to the global model. This adjustment provides stronger theoretical convergence guarantees under statistical and system heterogeneity. It allows devices to perform varying amounts of local computation without destabilizing training. Previous studies show that FedProx consistently outperforms FedAvg in highly heterogeneous federated networks by improving robustness, fairness, and overall model stability. However, the FedProx approach applies the proximal correction to all clients, even when only a few are updated. This can negatively affect optimization speed and introduce additional regularization. It lacks a mechanism for determining whether a correction is necessary. This affects adaptability in non-IID scenarios that change over time [ 21 ] . Another example is the FedBS algorithm [ 12 ] , which extends prior methods by addressing data heterogeneity at a finer granularity. FedBS employs bias-mitigation techniques that reduce disparities in client distributions, leading to more balanced learning outcomes. It refines client selection and aggregation mechanisms to enhance performance and fairness in scenarios with strongly non-IID data. FedBS has demonstrated improvements over FedAvg in convergence stability, accuracy, and fairness. Its adaptive framework highlights an important trend in FL research, shifting from simple global aggregation to more context-aware, client-sensitive optimization strategies. However, FedBS can flag unreliable clients as unstable using batch and variance tests, but it does not provide a means to compensate for the inaccurate updates it identifies. Consequently, unreliable clients continue to degrade the globally shared model, particularly under severe label imbalance, where the Dirichlet concentration parameter \alpha , commonly used to model data heterogeneity across clients in federated learning, is small. In such highly heterogeneous settings, FedBS alone is insufficient to guarantee robust convergence, particularly when model diversity is pronounced. This work presents a hybrid FL model that improves personalization and addresses client and data heterogeneity. The proposed approach, Proximal-Balanced Scaling Federated Learning (FedPBS), combines key ideas from FedProx and FedBS to enhance robustness in heterogeneous environments. FedPBS relies on the FedProx model proximal regularization parameter to stabilize local updates and facilitate smoother convergence across different client capabilities. Moreover, balanced scaling reduces the impact of skewed data distributions in non-IID settings. The method identifies vulnerable clients via the FedBS mechanism and updates their models using a proximal method, preventing harmful contributions while enabling effective training. By integrating the complementary strengths of FedProx and FedBS, FedPBS overcomes their individual limitations and achieves stronger robustness under severe heterogeneity. Experimental results using two datasets, i.e., UCI-HAR and CIFAR-10, demonstrate that the proposed FedPBS algorithm consistently outperforms existing federated learning baselines, particularly under highly non-IID data distributions, where client-level heterogeneity is modeled using a Dirichlet distribution in which a smaller concentration parameter \alpha corresponds to more skewed and heterogeneous class allocations across clients. On UCI-HAR, FedPBS achieves up to

Read the complete text