Observational Auditing of Label Privacy

Iden Kalemaj, Luca Melis, Maxime Boucher, Ilya Mironov, Saeed Mahloujifar · 2025

Read Observational Auditing of Label Privacy by Iden Kalemaj, Luca Melis, Maxime Boucher, Ilya Mironov, Saeed Mahloujifar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14084.txt.

Introduction

Iden Kalemaj, Luca Melis, Maxime Boucher, and two co-authors audit the privacy of training labels without touching the training data. Checking differential privacy guarantees usually requires altering the data set, planting canary examples or removing samples, which is costly at production scale. Their observational framework instead exploits randomness already present in the data distribution.

The method extends auditing beyond membership questions to protected attributes, with labels as one case. The authors supply theoretical foundations and test the approach on the Criteo and CIFAR-10 data sets, where it successfully evaluated label privacy guarantees. They present it as a practical route to privacy auditing inside large live systems.

Catalog record

CREATORIden Kalemaj, Luca Melis, Maxime Boucher, Ilya Mironov, Saeed Mahloujifar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14084

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Observational Auditing of Label Privacy Iden Kalemaj, Luca Melis, Maxime Boucher, Ilya Mironov, Saeed Mahloujifar Abstract Differential privacy (DP) auditing is essential for evaluating privacy guarantees in machine learning systems. Existing auditing methods, however, pose a significant challenge for large-scale systems since they require modifying the training dataset—for instance, by injecting out-of-distribution canaries or removing samples from training. Such interventions on the training data pipeline are resource-intensive and involve considerable engineering overhead. We introduce a novel observational auditing framework that leverages the inherent randomness of data distributions, enabling privacy evaluation without altering the original dataset. Our approach extends privacy auditing beyond traditional membership inference to protected attributes, with labels as a special case, addressing a key gap in existing techniques. We provide theoretical foundations for our method and perform experiments on Criteo and CIFAR-10 datasets that demonstrate its effectiveness in auditing label privacy guarantees. This work opens new avenues for practical privacy auditing in large-scale production environments. † † affiliation: Meta † † affiliation: FAIR at Meta † † contribution: Equal contribution † † date: August 10, 2026 † † correspondence: Iden Kalemaj [email protected] 1 Introduction Differential privacy (DP) auditing has become an important tool for evaluating privacy guarantees in machine learning systems. Recent advances in auditing methods that require only a single run have made it feasible to evaluate privacy for large-scale models without prohibitive computational costs ( Steinke et al. 2024 ; Mahloujifar et al. 2025b ) . However, most existing auditing approaches still require modifying the training dataset by injecting known entropy or canary data, which limits their applicability in industry-scale environments, where modifications to the training data pipeline require significant engineering overhead. In this work, we propose a novel auditing methodology that eliminates the need for dataset modification. Our approach enables privacy evaluation using the natural nondeterminism present in the data distribution itself. We formalize and empirically validate this methodology in the setting of auditing Label DP, generalizable to privacy guarantees for any protected attribute. This capability addresses a significant gap in current auditing techniques, which have primarily focused on membership inference attacks ( Shokri et al. 2017 ; Carlini et al. 2022 ) rather than attribute inference. In particular, existing methods for auditing Label DP either require adding out-of-distribution canaries to the training set ( Malek et al. 2021 ) or are applicable only to a limited set of mechanisms ( Busa-Fekete et al. 2024 ) . Observational privacy auditing cannot be done unconditionally, without making certain assumptions about the underlying distribution ( Hernán and Robins 2020 , Chapter 3) . Unlike anecdotal instances of privacy violations ( Barbaro and Zeller Jr 2006 ; Narayanan and Shmatikov 2008 ; Carlini et al. 2021 ) , auditing seeks to provide statistically valid measurements of memorization. In other words, the objective of privacy auditing is to establish causality —demonstrating that a model behaves in a certain way because it was trained on specific data. Traditionally, most rigorous membership inference attacks establish and measure causal effects through randomized control trials (RCTs), which require interference with the training data. By reframing auditing as a security game between two adversarial parties, our approach eliminates the need for training-time intervention. Our key assumption is the availability of a distribution that approximates the ground truth. Concretely, the Label DP auditing mechanism relies on access to a proxy label-generating distribution. The proxy does not need to match the ground truth distribution, provided the adversary cannot distinguish between them (with reasonable computational resources). Under this assumption, the counterfactual examples generated by the challenger can be used to evaluate the attack on the model’s claimed Label DP guarantees without training data intervention . The attack can be made practical by using a model other than the target model as the proxy distribution. Further, in an incremental learning setting, earlier model checkpoints can be used as the proxy distribution, thus requiring no additional model training and minimal engineering overhead. The We demonstrate that our observational auditing framework provides accurate privacy bounds that match those obtained by interventional methods and provable privacy guarantees. Through a series of cryptography-inspired games we establish the theoretical foundations for auditing privacy without training data manipulation. By lowering the complexity of privacy auditing, our approach enables its application in a wider variety of contexts. 2 Background and prior work Differential Privacy (DP) introduced by Dwork et al. 2006 is a leading framework for providing rigorous privacy guarantees in statistical data analysis and machine learning. In its standard formulation, DP bounds the impact that any single individual’s record has on the outcome of a computation by constraining how much the distribution of outputs may differ between neighboring datasets. The definition’s strong theoretical guarantees, resilience to arbitrary auxiliary data, and compositional properties have driven its adoption in academic research and industry deployments ( Fioretto and Hentenryck 2025 ) . The general DP framework can be adapted to settings where only certain parts of a dataset are considered private. This paper focuses on Label DP, which has emerged as an important objective for PPML, particularly in the domain of recommendation systems ( Chaudhuri and Hsu 2011 ; Ghazi et al. 2021 ; Malek et al. 2021 ; Wu et al. 2023 ) . The following factors motivate Label DP as a uniquely valuable privacy concept: • The label—representing the user’s choice, expressed preference, or the outcome of an action—may be the only sensitive part of the record, with the rest being publicly available, static, non-sensitive data. • In machine learning, labels are particularly vulnerable to memorization compared to other attributes since they most directly influence the loss function. • In settings with mixed public/private features, instead of applying privacy-preserving techniques to sensitive features, one may exclude them from the model, potentially sacrificing some accuracy. In supervised learning, however, labels are indispensable—there is no analogous alternative to omitting sensitive features. In further separation, once training completes, labels can be safely discarded, whereas features must be available for inference. Complementing the strong worst-case guarantees of DP that bound the privacy loss from above on all inputs, privacy auditing empirically measures the privacy loss on concrete instances, providing a lower bound on DP’s numerical parameters. Privacy auditing can be used for finding bugs in claimed implementations of DP algorithms ( Ding et al. 2018 ) , advancing understanding of complex DP mechanisms ( Malek et al. 2021 ; Nasr et al. 2023 ; Nasr et al. 2025 ) , or guardrailing models in a production environment ( Agrawal and Book 2025 ) . Privacy auditing consists of two components: a privacy game between the challenger and the attacker, and an auditing analysis that translates the attacker’s success into lower bounds on the ({\epsilon},\delta) -DP guarantee (or other forms of DP). The privacy game is characterized by the capabilities and resources of the parties, and the attacker’s goals, such as reconstruction, membership or attribute inference. Auditing analyses, typically used for membership inference, can be applied to any stochastic privacy game ( Swanberg et al. 2025 ) . We show theoretical results for our label inference attack building on Steinke et al. 2024 and Mahloujifar et al. 2025b . Membership inference attacks (MIAs)—where an adversary uses model access and knowledge

Read the complete text