Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization
Ratun Rahman, Dinh C. Nguyen · 2026
Read Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization by Ratun Rahman, Dinh C. Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-18083.txt.
Introduction
Rahman and Nguyen present Meta-BayFL, a personalized probabilistic federated learning method for messy client data. Bayesian neural nets model uncertainty but add runtime, latency, and bandwidth costs. Meta-BayFL joins meta-learning with Bayesian client models and adaptive learning rates. Personal updates help local training on non-IID data, and aggregation stays more robust. On CIFAR-10, CIFAR-100, and Tiny-ImageNet, it beat rival methods by up to 7.42% in test accuracy.
Catalog record
| CREATOR | Ratun Rahman, Dinh C. Nguyen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78788 |
| LICENSE | http://creativecommons.org/publicdomain/zero/1.0/ |
| RIGHTS | CC0 |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.18083
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization Ratun Rahman, Dinh C. Nguyen Abstract Conventional federated learning (FL) frameworks often suffer from training degradation due to data uncertainty and heterogeneity across local clients. Probabilistic approaches such as Bayesian neural networks (BNNs) can mitigate this issue by explicitly modeling uncertainty, but they introduce additional runtime, latency, and bandwidth overhead that has rarely been studied in federated settings. To address these challenges, we propose Meta-BayFL , a personalized probabilistic FL method that combines meta-learning with BNNs to improve training under uncertain and heterogeneous data. The framework is characterized by three main features: (1) BNN-based client models incorporate uncertainty across hidden layers to stabilize training on small and noisy datasets, (2) meta-learning with adaptive learning rates enables personalized updates that enhance local training under non-IID conditions, and (3) a unified probabilistic and personalized design improves the robustness of global model aggregation. We provide a theoretical convergence analysis and characterize the upper bound of the global model over communication rounds. In addition, we evaluate computational costs (runtime, latency, and communication) and discuss the feasibility of deployment on resource-constrained devices such as edge nodes and IoT systems. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that Meta-BayFL consistently outperforms state-of-the-art methods, including both standard and personalized FL approaches (e.g., pFedMe, Ditto, FedFomo), with up to 7.42% higher test accuracy. I Introduction Federated learning (FL) has gained considerable momentum due to its ability to facilitate data training at the network edge while ensuring the preservation of privacy [ 17 , 20 ] . FL enables local clients to independently train machine learning (ML) models on their local data, subsequently sharing only the model parameters with a centralized server for aggregation without exchanging raw data. This approach significantly improves data privacy and security by keeping local data private within local devices [ 18 ] . Furthermore, FL approaches mitigate communication latency by introducing a decentralized approach and not transmitting only the parameter of local models [ 15 ] . Despite such promising research efforts, traditional FL frameworks struggle with two practical challenges: uncertain and heterogeneous data , limiting training performance in federated settings [ 5 ] . 1. Uncertain data. Data in a collaborative environment can be uncertain and the quality of the dataset may vary for each global round and for every client. For example, some client information may be out of date, noisy, or very small in number, which could have a negative impact on the training process when local updates are combined with the global model. This discrepancy increases difficulties in ensuring that the global model remains stable and improves consistently over each training round. Furthermore, uncertainty in data quality can hamper the model’s convergence, since the learning algorithm may struggle to establish a generalizable pattern that adequately fits all data quality variations [ 28 ] . This leads to a critical question: a) How can inherent uncertain data be controlled to stabilize the training of the global model in FL? 2. Heterogeneous data. Another significant problem is data heterogeneity where data distributions across clients are naturally non-independent and identically distributed (non-IID). For instance, different clients may have various feature spaces that allow them to be biased towards certain features. These differences create challenges for clients to construct a global model that represents all clients equally, and it can significantly affect the overall effectiveness of FL [ 24 , 19 ] . This raises a critical question, b) How can FL effectively handle heterogeneous data distribution in non-IID settings? To address data uncertainty and heterogeneity challenges in FL, several works have been proposed that use advanced learning techniques. Personalized Federated Learning (PFL) has been developed to precisely address heterogeneous data settings. PFL methods ensure that the global model works well across various datasets by tailoring the training process to the features of individual or grouped clients. On the other hand, to address data uncertainty, probabilistic model training methods have been proposed, where probabilistic neural networks use a kernel function to approximate the probability density function of the training data and estimate the probability distribution of the data [ 23 ] . This allows the model to account for uncertainty in parameters, which is essentially important in training on noisy or unpredictable data. I-A Related Works We summarize the related works in BNN, probabilistic FL, and personalized FL. 1) BNN: One of the most common probabilistic methods is the bayesian neural network (BNN) which uses the Bayes theorem and observed data to update the model parameters given a prior distribution to estimate the posterior distribution [ 25 ] . BNN utilizes possibilities for all neural network layers, making the prediction more accurate for uncertain data than the basic FL method [ 4 , 7 , 1 ] . However, in the BNN, every client is handled equally and as a result, underperforming clients can significantly impact the global model, degrading the training performance. This also leads to poor generalization for individual clients, causing the international model to be biased towards some and irrelevant to others. As a result, the model becomes overly confident and performs poorly, especially on client datasets that differ significantly from the aggregated training data. Fig. 1 : A simple probabilistic model structure in which probabilistic properties link every layer to every layer after it. The likelihood graph shows the likelihood of moving from the current layer to the next. It shows that probabilistic (distributional) parameters spread uncertainty through depth. 2) Personalized FL: Federated distillation and contrastive learning have also been investigated in recent works to address heterogeneity and communication efficiency. Examples of these include efficient federated distillation for multitask time-series classification [ 27 ] and federated contrastive learning with feature-based distillation for human activity recognition [ 26 ] . [ 13 ] proposed FedProx that utilizes a proximal term to improve the stability of the FL process. Despite their effectiveness in representation sharing, they do not explicitly describe uncertainty or offer probabilistic personalization under noisy and small client datasets. Also methods like pFedMe [ 24 ] and Ditto [ 12 ] learned additional personalized models. However, only the desired information that improves the quality of the local model is beneficial for this method, and undesired information leads to poor generalization. Models like Meta-FL [ 19 ] , has continuously been used in various fields such as air quality [ 21 ] and load forecasting [ 16 ] . It uses local personalized aggregation to learn the local model, making the local model more accurate for the global aggregation. However, FedAMP and FedPHP performed their pFL without considering any local objectives. FedMFomo downloaded other client models for aggregation, creating a high communication overhead. 3) Personalized Probabilistic FL: [ 11 ] proposed Fedmask that learns a sparse binary mask for personalized. However, it is mainly used to reduce computational costs and does not enhance the overall performance. pFedBayes [ 31 ] and pFedBL [ 29 ] also used the pFL method on the Bayesian layer that introduces weight uncertainty for both the clients and server. The computational becomes very expensive in this method for large-scale data. However, the problem of uncertain and heterogeneous data training in FL settings has been