Privacy Auditing with Zero (0) Training Run
Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet · 2026
Read Privacy Auditing with Zero (0) Training Run by Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-14591.txt.
Introduction
Cebere, Even, Bleistein and Bellet introduce Zero-Run privacy auditing, which audits a trained model after the fact with no retraining and no control over data inclusion. Two fixed sets are used: known training members and known non-members. Since membership is no longer randomised, score gaps may reflect distribution shift rather than leakage. Drawing on causal inference, the authors formalise this confounding and propose two corrections, one global and conservative, one sharper and instance-dependent.
Catalog record
| CREATOR | Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 141438 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.14591
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Privacy Auditing with Zero (0) Training Run Tudor Cebere, Mathieu Even, Linus Bleistein, Aurélien Bellet Abstract Privacy auditing provides empirical lower bounds on the differential privacy parameters of learning algorithms. Existing methods, however, require interventional access to the training pipeline, either to retrain multiple times or to randomize data inclusion. This is often infeasible for large deployed systems such as foundation models. We introduce Zero-Run privacy auditing, a post-hoc framework for auditing models using two fixed datasets: examples known to be training-set members and examples known to be non-members. In this observational regime, membership is no longer randomized; instead, member and non-member data often differ in distribution, so membership inference scores may reflect a distribution shift rather than algorithmic leakage. Drawing on ideas from causal inference, we formalize this confounding effect and propose two complementary corrections that yield valid privacy audits. Our first approach models the combined effect of distribution shift and algorithmic leakage as an adaptive composition, producing conservative global corrections. Our second approach conditions on observed data and adjusts pointwise membership guesses, yielding sharper instance-dependent bounds. Experiments on synthetic data and large-scale models show that Zero-Run auditing enables practical privacy evaluation when retraining or controlled data insertion is infeasible. 1 Introduction As AI systems increasingly process sensitive data, mitigating privacy leakage (i.e., the inadvertent disclosure of training examples) has become a critical priority. This risk is exacerbated by high-capacity foundation models trained across various modalities [ 39 , 21 , 32 , 7 , 18 , 8 ] . Trained on massive, web-crawled corpora, they are inherently susceptible to memorizing personal or proprietary information [ 4 , 3 , 24 ] , and 2 show that this memorization scales log-linearly with model size. Moreover, these vulnerabilities introduce challenges in domain-specific contexts; for instance, clinical foundation models memorize rare patient profiles despite rigorous data de-identification [ 38 ] . A countermeasure to these risks is differential privacy (DP) [ 12 ] , the de facto standard for privacy-preserving algorithms. By bounding the impact of any individual’s data on the result, DP protects against privacy attacks that infer sensitive information from released outputs. However, despite offering worst-case guarantees over the privacy leakage, its adoption in large-scale training pipelines is limited due to steep utility penalties [ 20 ] and the prohibitive cost of overhauling existing data-processing infrastructure [ 35 ] . Motivated by these practical barriers, privacy auditing offers an empirical complement to DP by lower-bounding an algorithm’s privacy parameters [ 9 , 19 , 29 , 5 ] . By probing a model’s behavior on its training data, audits can expose implementation flaws [ 40 ] , identify misconfigured privacy mechanisms [ 6 ] , or demonstrate that theoretical guarantees are tight (i.e., the auditing lower bound closely matches the DP upper bound) [ 30 , 29 ] . In contrast to worst-case theoretical upper bounds, auditing quantifies empirical leakage for a specific model architecture, dataset, and threat model, yielding deployment-relevant evidence for privacy claims, compliance, and risk assessment [ 15 ] . While traditional privacy auditing requires repeatedly retraining models on randomized datasets [ 19 , 29 , 5 ] , recent advances condense this requirement to a single randomized training run [ 37 , 25 ] . Yet, even this approach fails to scale to modern AI models. Their training consumes thousands of GPU days and relies on rigid, distributed data pipelines. Large language models perfectly capture this dilemma. Despite being trained on vast web corpora loaded with sensitive data, their immense scale, opaque data mixtures, and blackbox deployments render privacy auditing nearly impossible. Consequently, researchers and regulators lack tools to verify privacy guarantees, relying on trust and the voluntary cooperation of model providers. This creates an accountability gap: the systems posing the greatest privacy risks are the most insulated from independent privacy evaluation. Contributions. To address this gap, we introduce a Zero-Run auditing framework that enables auditors to assess the privacy leakage of an algorithm post-hoc from its release , without modifying the algorithm or data pipeline. We ground our approach in a fundamental analogy to causal inference. Existing privacy audits based on randomized data insertion are inherently interventional—analogous to randomized controlled trials—which avoid confounding by design. In contrast, our Zero-Run framework is the observational counterpart, estimating privacy leakage from static member and non-member datasets. Since these datasets often follow different underlying distributions, this distribution shift acts as a confounding factor [ 33 , 42 ] . Guided by this analogy, our main contributions are as follows: 1. Formalizing Zero-Run Auditing ( Section 3 ). We formalize the post-hoc auditing regime where the auditor relies on disjoint datasets of known members and non-members. We demonstrate that naive audits on such data can substantially overestimate privacy leakage: membership inference attacks can succeed merely by exploiting distributional shift rather than algorithmic leakage, causing standard procedures to wrongly attribute this spurious signal to the model. We explicitly connect this setting to observational causal inference, where valid correction requires overlap (positivity) between the member and non-member distributions. 2. Privacy Leakage Correction ( Sections 4 and 5 ). We extend One-Run analyses [ 37 , 25 ] to the post-hoc regime, correcting distribution shift effects on empirical privacy curves via: • The Composition Viewpoint ( Section 4 ) models the observed leakage as an adaptive composition of a distribution-shift mechanism and the audited algorithm, yielding a deconvolution-based estimate of privacy leakage, albeit a conservative one under severe distribution shift. • The Conditional Viewpoint ( Section 5 ) characterizes pointwise privacy leakage conditional on the example. We tamper an adversary’s membership guesses to remove overconfidence induced by confounding, yielding a robust, shift-dependent correction. 3. Propensity Scores and Uncertainty Quantification ( Section 6 ). Our composition and conditional correction principles rely on propensity scores (the probability that an example is a member, based solely on its features), which must be estimated. We explicitly quantify uncertainty in propensity score estimation in our privacy lower bounds via bootstrapping [ 14 ] . 4. Practical Scalability ( Section 7 ). We demonstrate that Zero-Run auditing enables scalable privacy audits of large, deployed models where retraining is computationally or logistically infeasible. 2 Background on Differential Privacy and Privacy Auditing Differential Privacy. Informally, Differential Privacy (DP) [ 12 , 13 ] ensures that a randomized algorithm’s output remains largely unchanged whether a single individual’s data is included or not in the input. DP is widely considered the standard for privacy-preserving data analysis because it provides a mathematically rigorous guarantee against adversaries with arbitrary side information. DP is immune to post-processing (protections cannot be degraded by further computation) and composes gracefully, enabling accurate tracking of the cumulative privacy loss across multiple analyses. Definition 1 ( (\varepsilon,\delta) -DP) . Let \varepsilon,\delta>0 . A randomized algorithm \mathcal{A} that takes as input a dataset \mathcal{D}\subset\mathcal{X} and outputs \mathcal{A}(\mathcal{D})\in\Theta is said to satisfy (\varepsilon,\delta) -differential privacy if, for any pair of adjacent datasets \mathcal{D} and \mathcal{D}^{\prime} (i.e., which are the same except one data sample) and any measurable set \mathcal{E}\subset\Theta : \mathbb{P}\left(\mathcal{A}(\mathcal{D})\in\mathcal{E}\right)\leq e^{\varepsilon}\mathbb{P}\left(\mathcal{A}(\mathcal{D}^{\prime})\in\mathcal{E}\right)+\delta\,. Pure \varepsilon -DP corresponds to the special case where \delta=0 . In this work, we will also rely on the more expressive notion of f -DP [ 10 ] ,