Post-Processing in Local Differential Privacy: An Extensive Evaluation and Benchmark Platform
Alireza Khodaie, Berkay Kemal Balioglu, M. Emre Gursoy · 2025
Read Post-Processing in Local Differential Privacy: An Extensive Evaluation and Benchmark Platform by Alireza Khodaie, Berkay Kemal Balioglu, M. Emre Gursoy (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05875.txt.
Introduction
Alireza Khodaie and co-authors benchmark ways to clean up privacy-noised data, in a 2025 paper. Local differential privacy lets phones share statistics without exposing raw personal data. The noise it adds, though, dulls the data's usefulness. Post-processing methods try to restore that usefulness without weakening privacy.
Their benchmark spans many privacy protocols, cleanup methods, quality measures, and data sets. Cleanup helped a lot when privacy settings were strict and noise was heavy. Gains shrank as the privacy budget grew and noise eased. No single method won everywhere: the best choice hinged on the protocol, the data shape, and the quality measure. The team released an open platform so others can pick the right method for their case.
Catalog record
| CREATOR | Alireza Khodaie, Berkay Kemal Balioglu, M. Emre Gursoy |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05875
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Post-Processing in Local Differential Privacy: An Extensive Evaluation and Benchmark Platform Alireza Khodaie, Berkay Kemal Balioglu, M. Emre Gursoy Abstract Local differential privacy (LDP) has recently gained prominence as a powerful paradigm for collecting and analyzing sensitive data from users’ devices. However, the inherent perturbation added by LDP protocols reduces the utility of the collected data. To mitigate this issue, several post-processing (PP) methods have been developed. Yet, the comparative performance of PP methods under diverse settings remains underexplored. In this paper, we present an extensive benchmark comprising 6 popular LDP protocols, 7 PP methods, 4 utility metrics, and 6 datasets to evaluate the behaviors and optimality of PP methods under diverse conditions. Through extensive experiments, we show that while PP can substantially improve utility when the privacy budget is small (i.e., strict privacy), its benefit diminishes as the privacy budget grows. Moreover, our findings reveal that the optimal PP method depends on multiple factors, including the choice of LDP protocol, privacy budget, data characteristics (such as distribution and domain size), and the specific utility metric. To advance research in this area and assist practitioners in identifying the most suitable PP method for their setting, we introduce LDP 3 , an open-source benchmark platform. LDP 3 contains all methods used in our experimental analysis, and it is designed in a modular, extensible, and multi-threaded way for future use and development. 1 Introduction Over the last few years, local differential privacy (LDP) has become a popular notion for privacy-preserving data collection and analysis. In LDP, users perturb their sensitive data locally and then send it to a data collector. The added perturbation disables the data collector from making inferences regarding any particular user, but the data collector can still estimate population-level statistics from aggregates of perturbed data. This feature of LDP makes it well-suited for data collection in decentralized settings. Consequently, LDP has seen widespread adoption in various research fields [ 4 , 7 , 10 , 12 , 16 ] as well as industry products, e.g., Google Chrome, Apple iOS, and Microsoft Windows [ 6 , 8 , 13 ] . With the emerging popularity of LDP, numerous LDP protocols have been developed in the literature [ 5 , 9 , 14 ] . However, since these protocols inherently introduce perturbation to ensure privacy, the population-level statistics estimated by the data collector become noisy, leading to utility loss. To reduce utility loss, several post-processing (PP) methods were proposed [ 11 , 15 ] , e.g., Base-Pos, Norm, Norm-Cut, Norm-Sub, Norm-Mul, Power, and Power-NS. In essence, PP methods take as input the estimation results and process them in different ways to improve utility. On the other hand, although PP methods have been utilized in specific contexts or tested with specific protocols, to the best of our knowledge, there is no benchmark or detailed analysis of how different PP methods behave comparatively under diverse settings, e.g., under different combinations of LDP protocols, data characteristics, and utility metrics. In this paper, we create an extensive setup containing 6 LDP protocols, 7 PP methods, 4 utility metrics, and 6 datasets to benchmark the behaviors and optimality of PP methods under diverse settings. We perform an experimental analysis using our benchmark setup and report (a subset) of our results in the paper. Our results and analyses yield several interesting findings. First, we find that PP methods indeed yield substantial utility benefits when the privacy budget \varepsilon is small (strict privacy), but their utility results converge as \varepsilon becomes larger, and their benefits decrease. Second, we find that the best-performing PP method (in terms of utility improvement) is not static, but rather, it depends on a variety of factors including the LDP protocol, \varepsilon budget, data characteristics (e.g., statistical distribution, domain size), and utility metric. Third, we find that the best-performing PP method is more consistent when \varepsilon is small, but it becomes more inconsistent as \varepsilon increases. Next, considering there are multiple factors that influence the performance of PP methods and the choice of optimal PP method, we ask the following question: Can we design a practical system that helps a practitioner choose the best PP method for their setting, i.e., data, protocol, budget, and utility metric? To address this question, we design and develop LDP 3 (pronounced LDP-Cube): an open-source benchmark platform containing the implementations of all protocols, PP methods, utility metrics, and datasets used in our benchmark. LDP 3 is designed in a modular and extensible way so that new protocols, PP methods, utility metrics, etc. can be added in the future. Furthermore, it supports multi-threaded execution to reduce execution times so that researchers and practitioners can perform fast benchmarking of PP methods. Contributions. In short, in this paper we make three main contributions: • We create a benchmark setup with 6 LDP protocols, 7 PP methods, 4 utility metrics, and 6 datasets to analyze the behaviors and optimality of PP methods under diverse settings. • We perform an experimental analysis and show that the best-performing PP method depends on a variety of factors, including the LDP protocol, \varepsilon budget, data characteristics (e.g., statistical distribution, domain size), and utility metric. We show that when these factors are different, the best-performing PP method can also be different. • To advance research in this area and to assist researchers and practitioners in selecting the best PP method for their setting, we develop an open-source benchmark platform called LDP 3 which contains all protocols, PP methods, metrics, and datasets used in our analysis. Related work and main differences. While there exist a few past works in benchmarking LDP protocols or developing open-source LDP platforms, our work has some key differences. Earlier works such as [ 14 ] and [ 15 ] focus solely on protocols without post-processing or evaluate post-processing only on a single protocol. Pure-LDP proposed in [ 5 ] contains a subset of the LDP protocols and PP methods that exist in LDP 3 . The Multi-Freq-LDP package was proposed in [ 3 ] ; however, its focus is on multi-dimensional and longitudinal frequency estimation [ 1 , 2 ] rather than post-processing. LDPLens was proposed in [ 9 ] ; however, it is focused on protocols’ adversarial analysis and does not contain post-processing. Overall, our work differs in terms of its emphasis on post-processing as well as the novel experimental analysis and take-away messages concerning post-processing methods. Paper organization. The rest of the paper is organized as follows. In Section 2 , we provide background information related to LDP, LDP protocols, and PP methods. In Section 3 , we provide information regarding our experiment setup. In Section 4 , we present and discuss our experiment results; additionally, we summarize the main take-away messages from our experimental analysis. In Section 5 , we describe the design and implementation of LDP 3 . Finally, Section 6 concludes the paper. 2 LDP and Post-Processing Methods 2.1 Local Differential Privacy (LDP) Local differential privacy (LDP) is a popular notion for collecting data while protecting individuals’ privacy. In a typical LDP setup, there are multiple users (clients) and a server (data collector). We denote the user population by \mathcal{P} , a user by u\in\mathcal{P} , and the user’s