Evaluating the Dynamics of Membership Privacy in Deep Learning

Yuetian Chen, Zhiqi Wang, Nathalie Baracaldo, Swanand Ravindra Kadhe, Lei Yu · 2025

Read Evaluating the Dynamics of Membership Privacy in Deep Learning by Yuetian Chen, Zhiqi Wang, Nathalie Baracaldo, Swanand Ravindra Kadhe, Lei Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-23291.txt.

Introduction

Yuetian Chen, Zhiqi Wang, Nathalie Baracaldo, Swanand Ravindra Kadhe, and Lei Yu ask when a deep learning model starts to leak who was in its training data. Their 2025 paper studies membership inference attacks, which probe whether a record was part of that data.

To follow the leak as training runs, the team tracks the weakness of each sample on an FPR-TPR plane. That view lets them gauge how dataset complexity, model architecture, and optimizer choice affect how fast samples grow exposed, and how badly. Two points emerge. A sample's own difficulty in being learned ties closely to its privacy risk. And for samples left most at risk in the final model, that risk was largely set early in training. The authors say the work points toward training methods that watch for privacy risk while training is under way.

Catalog record

CREATORYuetian Chen, Zhiqi Wang, Nathalie Baracaldo, Swanand Ravindra Kadhe, Lei Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-23291

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating the Dynamics of Membership Privacy in Deep Learning Yuetian Chen, Zhiqi Wang, Nathalie Baracaldo, Swanand Ravindra Kadhe, Lei Yu Abstract Membership inference attacks (MIAs) pose a critical threat to the privacy of training data in deep learning. Despite significant progress in attack methodologies, our understanding of when and how models encode membership information during training remains limited. This paper presents a dynamic analytical framework for dissecting and quantifying privacy leakage dynamics at the individual sample level. By tracking per-sample vulnerabilities on an FPR-TPR plane throughout training, our framework systematically measures how factors such as dataset complexity, model architecture, and optimizer choice influence the rate and severity at which samples become vulnerable. Crucially, we discover a robust correlation between a sample’s intrinsic learning difficulty, and find that the privacy risk of samples highly vulnerable in the final trained model is largely determined early during training. Our results thus provide a deeper understanding of how privacy risks dynamically emerge during training, laying the groundwork for proactive, privacy-aware model training strategies. Introduction Membership Inference Attacks (MIAs) ( Shokri et al. 2017 ) represent a significant threat to privacy in deep learning, enabling adversaries to infer whether a specific data point was used in model training. Beyond their immediate implications for individual privacy ( Long et al. 2018 ; Yeom et al. 2018 ; Carlini et al. 2022 ; Choquette-Choo et al. 2021a ; Watson et al. 2022 ) , MIAs also play a critical role as tools for auditing differentially private systems ( Jayaraman and Evans 2019a ; Nasr et al. 2021 ) , verifying machine unlearning ( Bourtoule et al. 2020 ; Chen et al. 2021 ) , and quantifying the privacy risks of model memorization ( Mireshghallah et al. 2022 ) . Despite considerable advances in developing increasingly sophisticated attacks, our fundamental understanding of how, when, and why membership information becomes encoded in models during the training process remains surprisingly limited. Figure 1: An illustration of sample-level vulnerability trajectories. Each dot on a path represents a sample’s vulnerability state, characterized by its membership attack metric (FPR, TPR), estimated from N model variations at a given epoch. The distinct paths illustrate how membership encoding evolves uniquely for each sample—nuances obscured by traditional post-hoc analyses that only assess the final state ( ). Current research primarily relies on post-hoc assessments of trained models ( Ye et al. 2022 ; Shokri 2022 ; Chen et al. 2024 ) , evaluating privacy vulnerabilities only after training completion. Although insightful, this static approach provides merely a snapshot of vulnerability at a single point in time (i.e, the final training epoch), ignoring the underlying dynamic processes through which privacy risks emerge and evolve. Consequently, it provides little insight into the dynamics of sample-level privacy leakage throughout training and offers limited guidance for proactively preventing privacy risks during the model training phase. To bridge this critical gap, we present a novel dynamic analytical framework designed to systematically dissect and quantify membership privacy leakage at the granularity of individual training samples. Central to our approach is the concept of tracking per-sample vulnerabilities over time on an FPR-TPR vulnerability plane, as illustrated in Figure 1 , allowing us to visualize and measure the evolving privacy risks throughout the entire training process. This dynamic, sample-centric perspective provides critical insights into how various training-time factors, including dataset complexity, model architecture, and optimizer choice, influence the rate and extent to which membership information is encoded into models. While previous works ( Yeom et al. 2018 ; Nasr, Shokri, and Houmansadr 2019 ; Long et al. 2018 ; Carlini et al. 2022 ) have explored the relationship between membership inference vulnerabilities and these factors, their analyses have predominantly been static, post-training evaluations focusing on qualitative assessments of final vulnerabilities. In contrast, our approach quantitatively measures the dynamic impact of these factors on membership privacy leakage throughout training, significantly advancing beyond existing static methodologies. Specifically, our contributions are threefold: 1. We propose a new framework and a suite of metrics, including membership encoding speed, center of mass displacement, and transition probabilities, to characterize the dynamic nature of privacy loss for individual samples and entire populations. The source code of our framework will be made publicly available. 2. Through extensive experiments, we present—for the first time—a systematic and quantitative characterization of the dynamic impact of dataset complexity, model architecture, and optimizer choice, based on per-sample membership encoding trajectories. Our approach reveals how and when privacy vulnerabilities emerge and evolve during training—a significant advancement beyond existing static, qualitative analyses. 3. We reveal a strong correlation between intrinsic sample hardness—quantified through metrics such as influence functions, learning iterations, and epistemic uncertainty—and membership vulnerability. Crucially, our analysis demonstrates that vulnerability of these “hard-to-learn” samples is predominantly determined early during training, indicating a critical window for proactive intervention. Moreover, we show that optimization methods explicitly promoting generalization, notably Sharpness-Aware Minimization (SAM) ( Foret et al. 2021 ) , substantially mitigate the encoding of membership information throughout training. Ultimately, our work shifts the perspective from reactive, post-hoc privacy auditing toward proactively characterizing how and when privacy risks emerge during training. By providing analytical tools and empirical insights into the dynamics of privacy leakage, our framework lays critical groundwork for developing targeted interventions and more effective privacy-aware training strategies. Related Work Membership Inference Attacks. Membership Inference Attacks aim to determine if a specific data record was used to train a model, a vulnerability extensively documented across various architectures ( Shokri et al. 2017 ; Truex et al. 2021 ; Hu et al. 2022 ; Rigaki and Garcia 2023 ; Leino and Fredrikson 2020 ) . Foundational approaches demonstrated that models behave differently for members versus non-members, often using shadow models to learn these distinguishing patterns ( Salem et al. 2018 ; Hayes et al. 2017 ; Choquette-Choo et al. 2021b ) . The state-of-the-art has since evolved to include more robust, reference-based methods that calibrate a target’s outputs against a baseline ( Jayaraman and Evans 2019b ; Mireshghallah et al. 2022 ; Song and Mittal 2021 ; Sablayrolles et al. 2019 ) . Among these, the Likelihood Ratio Attack (LiRA) is particularly effective at providing reliable, instance-specific scores ( Carlini et al. 2022 ) . Learning Dynamics as a Privacy Lens. Existing research has demonstrated that the learning process itself is a primary source of information leakage, primarily driven by model overfitting ( Yeom et al. 2018 ; Long et al. 2018 ; Jayaraman et al. 2020 ; Zanella-Béguelin et al. 2020 ) . A rich body of work connects a model’s vulnerability to its learning dynamics—the temporal evolution of its state and parameters. Specific indicators of memorization and heightened risk have been identified through signals like loss and prediction confidence trajectories ( Li and Zhang 2021 ; Song, Shokri, and Mittal 2019 ) , patterns of example “forgetting” ( Toneva et al. 2018 ; Feldman 2020 ) , and the information content revealed by gradient evolution ( Carlini et al. 2021 ; Nasr, Shokri, and Houmansadr 2019 ; Hitaj, Ateniese, and Perez-Cruz 2017 ) . Gradient information is a particularly potent leakage channel, directly exploited by severe attacks that can reconstruct training data ( Zhu, Liu, and Han 2019 ;

Read the complete text