MCMC for Bayesian estimation of Differential Privacy from Membership Inference Attacks
Ceren Yıldırım, Kamer Kaya, Sinan Yıldırım, Erkay Savaş · 2025
Read MCMC for Bayesian estimation of Differential Privacy from Membership Inference Attacks by Ceren Yıldırım, Kamer Kaya, Sinan Yıldırım, Erkay Savaş (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-16683.txt.
Introduction
Ceren Yıldırım and co-authors estimate a system's privacy level from the results of membership inference attacks, using Bayesian sampling rather than a single worst-case guess. Their 2025 paper works in differential privacy auditing. The usual audit assumes the strongest possible attack on the worst possible data point, a pairing rarely seen in practice.
Their method runs a Markov chain Monte Carlo sampler that returns the whole posterior spread of the privacy parameter, not just an interval. It jointly infers how strong each attack was and how private the training algorithm is, which yields a more cautious verdict. They also give a cheap way to gather the attack measurements the sampler eats. Tests ran on synthetic and real data.
Catalog record
| CREATOR | Ceren Yıldırım, Kamer Kaya, Sinan Yıldırım, Erkay Savaş |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-16683
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MCMC for Bayesian estimation of Differential Privacy from Membership Inference Attacks Ceren Yıldırım, Kamer Kaya, Sinan Yıldırım, Erkay Savaş Abstract We propose a new framework for Bayesian estimation of differential privacy, incorporating evidence from multiple membership inference attacks (MIA). Bayesian estimation is carried out via a Markov chain Monte Carlo (MCMC) algorithm, named MCMC-DP-Est , which provides an estimate of the full posterior distribution of the privacy parameter (e.g., instead of just credible intervals). Critically, the proposed method does not assume that privacy auditing is performed with the most powerful attack on the worst-case (dataset, challenge point) pair, which is typically unrealistic. Instead, MCMC-DP-Est jointly estimates the strengths of MIAs used and the privacy of the training algorithm, yielding a more cautious privacy analysis. We also present an economical way to generate measurements for the performance of an MIA that is to be used by the MCMC method to estimate privacy. We present the use of the methods with numerical examples with both artificial and real data. 1 Introduction Differential privacy (DP) has emerged as a gold standard for quantifying and guaranteeing privacy in data analysis and machine learning ( Dwork,, 2006 ; Dwork and Roth,, 2014 ) . DP provides a mathematical framework to limit the impact of an individual’s data on the output of a random algorithm, enabling robust privacy guarantees regardless of the adversary’s knowledge other than that individual. DP is defined below. Definition 1 (DP) . An algorithm \mathcal{A} with input space \bigcup_{n=1}^{\infty}\mathcal{Z}^{n} and output space \Omega is (\epsilon,\delta) -DP if for every n\in\mathbb{N} , D\in\mathcal{Z}^{n} , z\in\mathcal{Z} , and E\subseteq\Omega , we have \displaystyle P(\mathcal{A}(D)\in E)\leq e^{\epsilon}P(\mathcal{A}(D\cup\{z\})\in E)+\delta, \displaystyle P(\mathcal{A}(D\cup\{z\})\in E)\leq e^{\epsilon}P(\mathcal{A}(D)\in E)+\delta. Quantifying the privacy of practical implementations remains a challenging task. Theoretical lower bounds for (\epsilon,\delta) have been extensively analyzed for a variety of mechanisms, such as noise-adding mechanisms (Laplace, Gaussian, etc.) ( Dwork et al.,, 2006 ; Dwork and Roth,, 2014 ) , subsampling ( Balle et al.,, 2018 ) , and their composition ( Kairouz et al.,, 2015 ) . Other theoretical definitions of DP have also been used to derive lower bounds of DP ( Bun and Steinke,, 2016 ; Dong et al.,, 2022 ; Mironov,, 2017 ) . However, theoretical lower bounds are not tight for many practical algorithms whose revealed outputs result from a series of calculations involving randomness. An example is when a training algorithm outputs only its final model, possibly following randomizing steps like random initialization, random updates (e.g., due to subsampling and/or noisy gradients), or output perturbation. In such a case, a large gap between the theoretical bounds and the actual privacy is shown to exist ( Nasr et al.,, 2025 ) . The privacy-auditing, or privacy estimation, of such complex but practical algorithms, through numerical estimation of (\epsilon,\delta) , has become an emerging research line ( Andrew et al.,, 2024 ; Hyland and Tople,, 2022 ; Jagielski et al.,, 2020 ; Lu et al.,, 2022 ; Maddock et al.,, 2023 ; Nasr et al.,, 2023 ; Nasr et al.,, 2021 ; Leemann et al.,, 2023 ; Nasr et al.,, 2025 ; Pillutla et al.,, 2023 ; Steinke et al.,, 2023 ; Zanella-Beguelin et al.,, 2023 ) . This study follows this line by proposing a new framework for Bayesian privacy estimation. Privacy auditing methods leverage the relation between DP and Membership Inference Attacks (MIA) ( Shokri et al.,, 2017 ; Yeom et al.,, 2018 ; Carlini et al.,, 2022 ; Ye et al.,, 2022 ) to estimate \epsilon and \delta . This is because DP particularly guarantees the protection of sensitive input data against MIAs. While there are different types of MIAs, by Def. 1 , the leave-one-out attack (L-attack in Ye et al., (2022) ) is the one directly relevant to the DP of a private algorithm. In an L-attack, a data set D , a point z\in\mathcal{Z} , and a random output of \mathcal{A} are given to the attacker who uses the given information to infer whether D or D\cup\{z\} was used by \mathcal{A} . For any D,z and any test, the type I and type II error probabilities are lower-bounded by a curve determined by the (\epsilon,\delta) of \mathcal{A} , see Theorem 1 . Two main issues deserve caution in empirical privacy analysis based on MIAs. 1. The attack strategy to audit privacy is typically not the strongest : For a given pair (D,z) , the strongest attack that decides based on \mathcal{A} ’s output is known to be the likelihood ratio test (LRT). However, practical MIAs to audit privacy merely approximate the LRT ( Sablayrolles et al.,, 2019 ) with a limited computational budget, e.g. using metrics that are loss-based ( Yeom et al.,, 2018 ; Ye et al.,, 2022 ; Carlini et al.,, 2022 ) , gradient-based ( Nasr et al.,, 2023 ) , etc. On the other hand, an adversary can design more powerful attacks than the one used for auditing, provided computational budget. It is also difficult to analytically characterize the gap between the performance of a given MIA and the LRT. Therefore, treating a given MIA as the strongest attack may lead to overconfident estimates about the privacy of an algorithm (‘overconfident’, because weaker attacks imply stronger privacy; see Remark 2 ). This is particularly dangerous since, based on overconfident estimations, private data may be leaked to a greater extent than it is permitted. 2. The challenge base (D,z) may not be the absolute “worst-case” challenge base . Moreover, existing efforts such as Nasr et al., (2021) ; Lu et al., (2022) ; Zanella-Beguelin et al., (2023) ; Pillutla et al., (2023) do not theoretically guarantee to find such a point. Therefore, even if the strongest attack is applied on the challenge base (D,z) , its observed false positive and false negative counts should not directly be used to upper bound \epsilon . This is because tests on another (D,z) challenge base could result in a larger upper bound on \epsilon . Considering the above issues, we propose a new Bayesian methodology for empirical privacy estimation. Our contributions are as follows. 1. A new posterior sampling method for DP estimation: We develop a joint probability distribution for the privacy parameter, attack strengths, and false negatives/positives involved in block-box auditing a private algorithm, where one has access only to the output of \mathcal{A} and not its intermediate results. Suited to this model, we propose a Markov Chain Monte Carlo (MCMC) method named MCMC-DP-Est (Alg. 1 ), adopted from Andrieu et al., (2020) , for Bayesian estimation of privacy. The advantages of MCMC-DP-Est are as follows: • Full posterior distribution: Beyond credible intervals, it returns the full posterior distribution of the privacy parameters as in Zanella-Beguelin et al., (2023) . • Combining multiple results: With the fully Bayesian treatment, the algorithm can combine the false negative and false positive counts from multiple (D,z) points (and possibly from multiple attack strategies). In particular, no tried attacks need to be thrown away . This also enables leveraging new (D,z) points or attack strategies to refine the privacy estimates coherently. • Cautious treatment of attack strengths: Related to the above discussion, the probabilistic model based on which MCMC is used in