Federated Learning with Differential Privacy: An Utility-Enhanced Approach
Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne · 2025
Read Federated Learning with Differential Privacy: An Utility-Enhanced Approach by Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-21154.txt.
Introduction
Seven authors led by Kanishka Ranaweera improve a private way to train AI models across many phones or sites. In this method, called federated learning, raw data never leaves each device. Still, the updates sent to a central server can leak facts about the data. Adding noise, a step known as differential privacy, plugs the leak but often dulls the model a great deal.
Their change adds a wavelet step, which splits each update into broad and fine parts, plus a fresh noise scheme. The pair cuts the noise size needed for the same privacy pledge. Proofs show faster settling toward a good model than the plain private methods. Tests on real data sets beat past methods on score while the privacy pledge held. The gain matters most where data are both spread out and sensitive.
Catalog record
| CREATOR | Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-21154
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Federated Learning with Differential Privacy: An Utility-Enhanced Approach Kanishka Ranaweera, Dinh C. Nguyen, Pubudu N. Pathirana, David Smith, Ming Ding, Thierry Rakotoarivelo, Aruna Seneviratne Abstract Federated learning has emerged as an attractive approach to protect data privacy by eliminating the need for sharing clients’ data while reducing communication costs compared with centralized machine learning algorithms. However, recent studies have shown that federated learning alone does not guarantee privacy, as private data may still be inferred from the uploaded parameters to the central server. In order to successfully avoid data leakage, adopting differential privacy (DP) in the local optimization process or in the local update aggregation process has emerged as two feasible ways for achieving sample-level or user-level privacy guarantees respectively, in federated learning models. However, compared to their non-private equivalents, these approaches suffer from a poor utility. To improve the privacy-utility trade-off, we present a modification to these vanilla differentially private algorithms based on a Haar wavelet transformation step and a novel noise injection scheme that significantly lowers the asymptotic bound of the noise variance. We also present a holistic convergence analysis of our proposed algorithm, showing that our method yields better convergence performance than the vanilla DP algorithms. Numerical experiments on real-world datasets demonstrate that our method outperforms existing approaches in model utility while maintaining the same privacy guarantees. I Introduction The proliferation of Internet of Things (IoT) devices has led to exponential growth in data creation. Machine learning (ML) has become an essential tool to analyze this data and extract valuable insights for various applications, including facial recognition, data analytics, weather prediction, and speech recognition, among others [ 1 , 2 , 3 , 4 , 5 ] . However, in real-world settings, data — particularly personal data — is often created and stored on end-user devices. The majority of traditional ML algorithms require the centralization of these training data, which involves collecting and processing data at a potent cloud-based server [ 6 , 7 ] . This process carries significant risks to data integrity and privacy, particularly when it comes to personal data. This is because personal data may contain sensitive or private information, and the centralization of data creates a single point of failure that might jeopardize the integrity of the data. Exposing such data can lead to financial loss, lawsuits, reputation damage, and even physical violence. In addition, centralized data processing and administration impose restricted transparency and provenance, which could result in end-users losing trust in the system [ 8 ] . Federated learning (FL) [ 9 , 10 ] is a cutting-edge development in distributed ML, designed to enable collaborative learning in decentralized environments. In this context, an ML model is implemented by applying an algorithm to a variety of local datasets maintained by client devices. The updated model parameters are then sent to a central server for aggregation, eliminating the need for gathering and processing training data at a centralized data server. In a traditional machine learning system, a large dataset is partitioned across cloud servers and optimized using techniques such as stochastic gradient descent (SGD) [ 11 ] , adaptive moment estimation (ADAM) [ 12 ] , or adaptive gradient algorithm (Adagrad) [ 13 ] . However, in an FL environment, the data is dispersed unevenly across millions of devices with varying degrees of availability and connectivity. These bandwidth and latency restrictions lead to the development of Google’s Federated Averaging (FedAvg) algorithm [10], which uses far fewer communication rounds to train deep networks than a naive version of a federated machine learning optimizer. The key concept behind FL is to compute local updates using the powerful CPUs available on client devices, allowing for training to consume substantially less communication overhead. This is achieved by generating high-quality updates in fewer cycles than traditional gradient steps, resulting in a better model. With the increasing need for data security and privacy in big data applications and distributed learning systems, the preservation of privacy has become a major concern. FL provides a significant advantage by performing local training without transferring personal data between the server and clients, thus safeguarding client data from covert attackers. However, recent research has shown that variations in parameters trained and uploaded by clients can still reveal private information to some extent [ 14 , 15 ] . Differential privacy (DP) [ 16 , 17 ] has shown to be a promising solution for preventing adversaries from inferring information from FL applications. This DP characteristic confers strong privacy on a data-processing mechanism rather than the data itself. It is widely used for query release, synthetic data synthesis, and machine learning training, among other things [ 16 , 18 , 19 ] . The work in [ 20 ] introduced a novel optimization technique based on the concept of DP [ 17 ] applicable to deep learning techniques, which can also be adapted for use at the sample level in FL algorithms. More specifically, they presented a modification to the classic SGD technique, which is a cost function-based optimization algorithm commonly used in DL applications to fit linear classifiers and regressors. This algorithm is called differentially private stochastic gradient descent (DP-SGD) and is considered highly effective in achieving sample-level differential privacy in FL applications. However, users may occasionally provide several examples to the training dataset, in which case sample-level DP may not be strong enough to prevent user data from being memorized. To address this issue, [ 21 ] created the Differentially Private Federated Averaging (DP-FedAvg) algorithm, which provides user-level DP. This means that the output distribution of models remains constant even if we add or delete all of the training samples from any one client. However, these DP algorithms [ 20 , 21 ] exhibit a considerable loss in model utility when compared to non-private versions. The insertion of Gaussian noise into the gradient steps utilized by DP-SGD and the insertion of Gaussian noise into the global aggregation in DP-FedAvg cause this deterioration. Increasing the amount of noise increases privacy assurances, but it also reduces utility. Motivated by these limitations, our paper presents a novel modification to the vanilla DP-SGD and DP-FedAvg algorithms that delivers significantly enhanced utility while maintaining the same privacy guarantees, resulting in a superior utility-privacy tradeoff. This approach is inspired by the Privelet framework introduced by [ 22 ] for differentially private data publishing of range count queries. However, while Privelet is designed for static, non-learning scenarios, our work extends this technique to dynamic and iterative machine learning and FL environments. In these settings, we address unique challenges such as managing the sensitivity of gradient updates and ensuring robust differential privacy guarantees across multiple rounds of communication. Our method integrates wavelet transforms into the differentially private federated averaging process, providing theoretical convergence guarantees and practical performance improvements demonstrated through extensive empirical validation on real-world FL datasets.The strategy is based on a Haar wavelet transformation (HWT), a linear transformation approach commonly used in data compression, image, and signal processing [ 23 , 24 , 25 ] . [ 22 ] demonstrated that by performing Laplace noise injection in the resultant coefficients of HWT, the noise variance can be greatly reduced, hence enhancing the accuracy of information obtained from DP