Imitative Membership Inference Attack
Yuntao Du, Yuetian Chen, Hanshen Xiao, Bruno Ribeiro, Ninghui Li · 2025
Read Imitative Membership Inference Attack by Yuntao Du, Yuetian Chen, Hanshen Xiao, Bruno Ribeiro, Ninghui Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-06796.txt.
Introduction
Testing whether a specific person's data trained a model, without training hundreds of stand-in models, is the advance claimed by Yuntao Du and four co-authors in a 2025 machine learning security paper. Membership inference attacks measure how much a model leaks about its training set. Leading attacks rely on many shadow models unrelated to the target, which is expensive.
Their imitative attack instead builds a small number of models designed to copy the target model's behavior closely, informed by the target itself. Those imitators then support the membership judgment. Extensive experiments show the method substantially outperforming existing attacks across settings while using less than five percent of the computing cost of leading approaches. Cheaper, stronger membership tests matter both for auditing privacy honestly and for understanding how much information trained models really retain.
Catalog record
| CREATOR | Yuntao Du, Yuetian Chen, Hanshen Xiao, Bruno Ribeiro, Ninghui Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 94 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-06796
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Imitative Membership Inference Attack Yuntao Du, Yuetian Chen, Hanshen Xiao, Bruno Ribeiro, Ninghui Li Abstract A Membership Inference Attack (MIA) assesses how much a target machine learning model reveals about its training data by determining whether specific query instances were part of the training set. State-of-the-art MIAs rely on training hundreds of shadow models that are independent of the target model, leading to significant computational overhead. In this paper, we introduce Imitative Membership Inference Attack ( \mathsf{IMIA} ), which employs a novel imitative training technique to strategically construct a small number of target-informed imitative models that closely replicate the target model’s behavior for inference. Extensive experimental results demonstrate that \mathsf{IMIA} substantially outperforms existing MIAs in various attack settings while only requiring less than 5% of the computational cost of state-of-the-art approaches 1 1 1 Our code is available at https://github.com/zealscott/IMIA . . 1 Introduction Over the past decade, machine learning (ML) has seen remarkable advances, with models such as neural networks increasingly being trained on sensitive datasets. This trend has raised growing concerns about the privacy risks associated with these models. Membership inference attacks (MIAs) [ 48 ] have been proposed to measure the degree to which a model leaks information by determining whether some instances were part of its training set. Closely related to Differential Privacy (DP) [ 13 , 32 ] , MIAs have become a widely adopted technique for empirical auditing of various trustworthy risks in ML models [ 39 , 51 , 56 , 57 , 42 ] , and serve as key components for more sophisticated attacks [ 4 , 5 , 52 ] . MIAs in the black-box setting [ 59 , 50 , 58 , 3 ] typically exploit the model’s behavioral discrepancy between its training instances ( i.e., members) and non-training instances ( i.e., non-members) for inference. A widely-used strategy to explore the discrepancy is shadow training [ 48 ] , which involves training multiple shadow models on datasets drawn from the same distribution as the target model’s training set. For each query instance, the membership scores generated by the shadow models trained without the instance (shadow out models) represent the out distribution for the instance. In some settings, one also has shadow models that are trained with the instance as a member, and can use the scores generated by these shadow in models to compute an in distribution. Given a target model, one tries to tell which distribution the score from the target model fits better. MIAs [ 54 , 19 , 9 , 60 , 3 ] utilizing shadow training have shown strong attack performance. Figure 1: Distributions of membership scores ( i.e., scaled confidence scores, defined in Section 3.1 ) for six CIFAR-100 instances with varying attack difficulty (easy, medium, hard). A larger overlap between in (trained with the instance) and out (trained without the instance) score distributions indicates greater difficulty in determining membership. The dashed vertical line represents each instance’s score on the target model. Top row : target-agnostic shadow models show high predictive variance (with long tails and wide distributions) for both members and non-members, resulting in significant overlap that hampers reliable inference, especially for hard-to-attack instances. Bottom row : target-informed imitative models exhibit more stable and well-separated distributions, enabling effective inference across all levels of difficulty. More examples are in Figure 7 . Best viewed in color. A critical limitation of existing shadow-based MIAs lies in their substantial computational overhead . Training each shadow model incurs non-trivial overhead, and state-of-the-art attacks like LiRA [ 3 ] and PMIA [ 9 ] require training hundreds of shadow models to estimate the likelihood ratio for inference. This requirement imposes a substantial computational burden, which reduces the feasibility of using these state-of-the-art MIAs for practical privacy auditing [ 60 ] . In addition, it also impedes research reproducibility within the privacy community (see Section 2 for further discussion). We observe that this inefficiency stems from the target-agnostic design of shadow training. That is, the current shadow training process fails to take advantage of knowing the target model under attack. As a result, the shadow models learn only the general patterns of members and non-members, rather than the specific behavioral discrepancies of the target model. As shown in the top row of Figure 1 , this target-agnostic approach causes shadow models to exhibit high predictive variance across instances with varying levels of attack vulnerability. Consequently, existing MIAs need to train a multitude of shadow models to capture this variability, resulting in both substantial computational overhead and suboptimal performance. To address this challenge, we introduce Imitative Membership Inference Attack ( \mathsf{IMIA} ), a novel approach that improves both attack efficiency and effectiveness. At the core of \mathsf{IMIA} is imitative training , a new shadow training technique that trains target-informed imitative models to mimic the target model’s behavior. Specifically, we first train a set of imitative out models by applying weighted logits (log of output confidence) matching to the target model’s outputs, capturing the behavior of the target on non-member instances. We then leverage these models to continue training on specially selected “pivot” instances from the adversary’s dataset using standard cross-entropy loss, yielding a set of imitative in models that capture the target model’s predictions on member instances. This targeted approach enables us to pair each query instance with a set of in and out predictions estimated from the target model. As illustrated in the bottom row of Figure 1 , imitative models yield significantly more separable and stable estimations, enabling the adversary to better exploit behavioral discrepancies for more powerful inference. We note that some attacks [ 58 , 15 , 30 ] employ model distillation to construct shadow models from the target model. However, these methods directly apply the standard distillation technique [ 20 ] without tailoring it to capture the membership-related behaviors of the target model. Furthermore, the resulting distillation models can only reflect behaviors on non-member instances, making it infeasible to exploit behavioral discrepancies for effective membership inference. A more detailed discussion of the differences between our approach and distillation-based methods is provided in Section 3.2 . We compare the proposed attack with a broad range of MIAs on six benchmark datasets (4 image and 2 non-image datasets), under both non-adaptive and adaptive settings [ 9 ] (also referred to as “offline” and “online” [ 3 ] ; see Section 2 for details). Experimental results show that \mathsf{IMIA} consistently outperforms all evaluated attacks (including distillation-based attacks) while using significantly fewer shadow models, with particularly strong gains in the increasingly advocated non-adaptive setting. For example, \mathsf{IMIA} surpasses the state-of-the-art non-adaptive attack ( i.e., PMIA [ 9 ] ) by over \mathbf{9\times} in true positive rate at a 0.001% false positive rate on Fashion-MNIST, while using only 5% of its computational resources. Similarly, \mathsf{IMIA} outperforms all adaptive attacks with far less computation. We also conduct comprehensive analyses to show the effectiveness of imitative training over shadow training, evaluate the impact of components in \mathsf{IMIA} , and assess the effectiveness of existing defenses against the proposed attack. In summary, we make the following contributions: • We propose \mathsf{IMIA} , a new MIA that employs a novel target-informed imitative training technique to