Dynamic Probabilistic Noise Injection for Membership Inference Defense
Javad Forough, Hamed Haddadi · 2025
Read Dynamic Probabilistic Noise Injection for Membership Inference Defense by Javad Forough, Hamed Haddadi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13362.txt.
Introduction
Javad Forough and Hamed Haddadi present a privacy defense for machine learning in a 2025 paper. Attackers can probe a model to guess if a given record trained it. That risk is grave in fields like health care and finance. The usual fix adds a fixed dose of noise, which can fail or harm accuracy.
Their defense, DynaNoise, adjusts the noise for each query at answer time. It gauges how risky a query looks and scales the noise to match. A smoothing step then keeps the answers useful for honest users. The authors also coin a score that weighs privacy gain against lost accuracy. Tests on standard data sets cut attack success a great deal. Accuracy stayed close to that of rival defenses in the same tests.
Catalog record
| CREATOR | Javad Forough, Hamed Haddadi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13362
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Dynamic Probabilistic Noise Injection for Membership Inference Defense Javad Forough, Hamed Haddadi Abstract . Membership Inference Attacks (MIAs) expose privacy risks by determining whether a specific sample was part of a model’s training set. These threats are especially serious in sensitive domains such as healthcare and finance. Traditional mitigation techniques, such as static differential privacy, rely on injecting a fixed amount of noise during training or inference. However, this often leads to a detrimental trade-off: the noise may be insufficient to counter sophisticated attacks or, when increased, can substantially degrade model accuracy. To address this limitation, we propose DynaNoise, an adaptive inference-time defense that modulates injected noise based on per-query sensitivity. DynaNoise estimates risk using measures such as Shannon entropy and scales the noise variance accordingly, followed by a smoothing step that re-normalizes the perturbed outputs to preserve predictive utility. We further introduce MIDPUT (Membership Inference Defense Privacy-Utility Trade-off), a scalar metric that captures both privacy gains and accuracy retention. Our evaluation on several benchmark datasets demonstrates that DynaNoise substantially lowers attack success rates while maintaining competitive accuracy, achieving strong overall MIDPUT scores compared to state-of-the-art defenses. Volume: YYYY X DOI: XXXXXXX.XXXXXXX Conference: Proceedings on Privacy Enhancing Technologies; printacmref=false,printccs=false,printfolios=true 1. Introduction Machine learning has revolutionized many domains by leveraging vast amounts of data to achieve impressive performance ( Islam et al., 2024 ; Forough et al., 2023 ; Rayed et al., 2024 ; Zhang et al., 2019 ; Forough et al., 2022 ; Sahu et al., 2023 ) . However, this success comes at a cost, where sensitive information from training datasets may be inadvertently memorized, posing serious privacy risks ( Schwarzschild et al., 2024 ; El Mestari et al., 2024 ; Yang et al., 2024 ; Ye, 2024 ; Li et al., 2024 ) . For example, in a scenario where a hospital deploys a predictive model to diagnose diseases; if an attacker can determine whether a patient’s record was part of the training set, it may reveal that the patient has visited the hospital, thereby compromising their privacy. Similarly, in finance, membership leakage could expose clients’ investment histories. Such risks underscore the need for robust defenses against privacy attacks. Membership Inference Attacks (MIAs), as shown in Figure 1 , exploit subtle differences in a model’s output behavior to determine whether a specific data record was used during training, thereby threatening the confidentiality of individual data points ( Shokri et al., 2017 ) . Conventional defenses, such as differential privacy, introduce a fixed level of noise during training or inference to obscure membership information ( Abadi et al., 2016 ) . While these approaches offer formal privacy guarantees, they force an inherent trade-off between privacy and utility. In many practical settings, uniformly adding noise can significantly degrade model performance, yet reducing the noise level may leave the model susceptible to advanced membership inference attacks. Figure 1 . Illustration of the membership inference attack (MIA) process. To address these challenges, we introduce DynaNoise, an adaptive noise injection approach that modulates the privacy noise dynamically based on query sensitivity. Our method first assesses the risk of each query through sensitivity analysis, by utilizing metrics such as Shannon entropy ( Sepúlveda-Fontaine and Amigó, 2024 ) , and then adjusts the noise variance accordingly. A subsequent probabilistic smoothing step is applied to re-normalize the perturbed outputs, ensuring that the model retains high predictive accuracy while effectively obfuscating membership signals. Another key novelty of our approach is the introduction of a new empirical metric, the Membership Inference Defense Privacy-Utility Trade-off (MIDPUT) , which quantitatively captures the balance between the reduction in attack success rates and the preservation of model accuracy. Our experimental results on CIFAR-10, ImageNet-10, and SST-2 datasets demonstrate that DynaNoise not only significantly reduces success rates of membership inference attacks but also achieves notably higher improvement in the MIDPUT metric compared to existing defenses. The main contributions of this paper are threefold: (1) Proposal of a novel post-hoc defense mechanism against membership inference attacks called DynaNoise that dynamically adjusts the noise level based on query sensitivity, thereby providing stronger privacy protection with minimal impact on target model accuracy. (2) Proposal of a new empirical metric called MIDPUT, that quantifies the trade-off between the reduction in attack success rates and the loss in model performance. This metric enables a more detailed and precise evaluation of privacy defenses. (3) Comprehensive experimental study on multiple benchmarks (CIFAR-10, ImageNet-10, and SST-2), systematically varying DynaNoise parameters and benchmarking against state-of-the-art defenses (AdvReg, MemGuard, RelaxLoss, SELENA, and HAMP). We also evaluate robustness under strong membership inference attacks, including LiRA, ensuring a fair and rigorous assessment. The remainder of this paper is organized as follows. In Section 2 , we review the state-of-the-art in membership inference attacks and defenses. Section 3 and 4 explain our problem statement and threat model, respectively. Section 5 reviews the preliminary concepts related to this work. Section 6 details the design and implementation of DynaNoise, including our adaptive noise injection and the MIDPUT metric. Section 7 presents our experimental setup, results, and a discussion of the advantages and limitations of the proposed approach. Finally, Section 8 concludes the paper and discusses future research directions. 2. Related Work 2.1. Membership Inference Attacks (MIAs) Membership inference attacks can be broadly categorized into two types: shadow training-based attacks ( Shokri et al., 2017 ; Salem et al., 2018 ; Long et al., 2020 ) and metric-based attacks ( Yeom et al., 2018 ; Song and Mittal, 2021 ) . Shadow training-based attacks. Shokri et al. ( Shokri et al., 2017 ) introduced shadow-model MIAs, where an adversary trains auxiliary models on data from a similar distribution and uses their outputs to train an attack classifier distinguishing members from non-members. Salem et al. ( Salem et al., 2018 ) showed that this approach can be effective even with a single shadow model, substantially reducing the attacker’s cost. Long et al. ( Long et al., 2020 ) further improved practicality by refining shadow-model construction and the attack classifier to reduce query requirements. Overall, shadow-model attacks remain strong but typically rely on access to representative auxiliary data from the target distribution. Metric-based attacks. Metric-based attacks infer membership directly from the target model’s outputs by evaluating simple statistics, without training auxiliary models. Yeom et al. ( Yeom et al., 2018 ) propose a loss-based attack that exploits the tendency of models to incur lower loss on training samples, inferring membership via thresholding. Song et al. ( Song and Mittal, 2021 ) similarly introduce confidence-based attacks that predict membership based on whether the maximum confidence exceeds a predefined threshold. While lightweight and effective, these methods are sensitive to threshold selection and model calibration. Song and Mittal ( Song and Mittal, 2021 ) further extend this line of work by proposing entropy-based and modified entropy (M-Entropy) attacks, which exploit the observation that training samples typically produce lower-entropy and more structured output distributions. The entropy attack infers membership by thresholding Shannon entropy, while M-Entropy incorporates the probability assigned to the true label to improve robustness across confidence profiles. These attacks operate solely on output probabilities, making them well suited for black-box settings. Likelihood Ratio Attack (LiRA). Carlini et al. ( Carlini et al., 2022