A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks

Haoyang Li, Li Bai, Qingqing Ye, Haibo Hu, Yaxin Xiao, Huadi Zheng, Jianliang Xu · 2025

Read A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks by Haoyang Li, Li Bai, Qingqing Ye, Haibo Hu, Yaxin Xiao, Huadi Zheng, Jianliang Xu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-19070.txt.

Introduction

Haoyang Li and co-authors sharpen how privacy attacks on neural networks are judged in a 2025 paper. Model inversion attacks rebuild training data from a trained model, a real threat when the data are faces or medical records. Past work rebuilt class-level averages, like a generic face for one person, while the privacy of a single training photo got little study.

The team coins a score that grades each rebuilt sample on diversity and distance at once. Through that lens, many training samples shrug off even top attacks. They then boost the attacker with transfer learning, an entropy loss, and a natural gradient method, and the boosted attacks score better across the board. Flipping the lens, the same score can flag which samples in a dataset sit most exposed, aiding defense before any attack runs.

Catalog record

CREATORHaoyang Li, Li Bai, Qingqing Ye, Haibo Hu, Yaxin Xiao, Huadi Zheng, Jianliang Xu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-19070

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks Haoyang Li, Li Bai, Qingqing Ye, Haibo Hu, Yaxin Xiao, Huadi Zheng, Jianliang Xu Abstract Model Inversion (MI) attacks, which reconstruct the training dataset of neural networks, pose significant privacy concerns in machine learning. Recent MI attacks have managed to reconstruct realistic label-level private data, such as the general appearance of a target person from all training images labeled on him. Beyond label-level privacy, in this paper we show sample-level privacy, the private information of a single target sample, is also important but under-explored in the MI literature due to the limitations of existing evaluation metrics. To address this gap, this study introduces a novel metric tailored for training-sample analysis, namely, the Diversity and Distance Composite Score (DDCS), which evaluates the reconstruction fidelity of each training sample by encompassing various MI attack attributes. This, in turn, enhances the precision of sample-level privacy assessments. Leveraging DDCS as a new evaluative lens, we observe that many training samples remain resilient against even the most advanced MI attack. As such, we further propose a transfer learning framework that augments the generative capabilities of MI attackers through the integration of entropy loss and natural gradient descent. Extensive experiments verify the effectiveness of our framework on improving state-of-the-art MI attacks over various metrics including DDCS, coverage and FID. Finally, we demonstrate that DDCS can also be useful for MI defense, by identifying samples susceptible to MI attacks in an unsupervised manner. Introduction With today’s explosive growth in Machine Learning (ML), all businesses are collecting personal data more frequently than ever ( Li et al. 2023 ) . As such, there is an increasing demand of privacy preservation during model training and deployment ( Yan et al. 2022 ; Bai et al. 2024 ; Hu et al. 2024 ; Zhao et al. 2016 ) . However, it is well known that by exploiting the confidence score of neural networks, Model Inversion (MI) attacks can partially reconstruct the training dataset from an ML model ( Fredrikson, Jha, and Ristenpart 2015 ) . Compared with membership or property inference attacks that only infer certain membership or property information as a binary classification task ( Carlini et al. 2022 ) , an MI attack targets at the whole private information in the training dataset and reconstructs multiple private features simultaneously ( Song and Namiot 2022 ) , which poses even more severe threats than its inference counterpart. In line with previous studies ( Zhang et al. 2020 ; Chen et al. 2021 ; Struppek et al. 2022 ) , we consider white-box MI attacks, where the attacker has access to the specific parameters and architecture of the victim model. Given the inherent difficulty for MI attacks to recover every sample within a large-scale dataset, recent works ( Struppek et al. 2022 ; An et al. 2022 ) have lowered down their goal to recover one or few representatives in each class label. This trend was first motivated in the field of face recognition ( Wang et al. 2021b ) , where each label corresponds to the identity of a person. Since these label representatives can reveal the general appearance of a person, they arise as a significant concern in recent studies ( Wang et al. 2021a ; Nguyen et al. 2023 ) . However, the potential for privacy compromise varies across applications. In some domains beyond facial recognition, label representatives may convey little or even no private information, which in turn weakens such threats. For instance, in disease diagnosis, each label corresponds to a well-known disease, so label representatives merely encapsulate public knowledge about the conditions of this disease. However, MI attacks may still threaten sample-level privacy by reconstructing individual samples. To address the imperative to unearth and safeguard sample-level privacy, this study critiques existing evaluation metrics of MI by first demonstrating their inadequacy in fully capturing the multifaceted characteristics of MI attacks. Furthermore, metrics based on distributional similarity are sensitive to the distribution of reconstructed samples, so that their evaluation results can be easily fooled by controlling the sample distribution ( e.g. , producing redundant samples or approximating the distribution with fewer samples). To overcome these limitations, we introduce a novel metric, namely Diversity and Distance Compostie Score (DDCS), that assesses the integrity of sample-level privacy in the context of MI attacks. In essence, DDCS uniquely evaluates the extent of reconstruction by the attack for each sample in the training dataset, so that it can comprehensively incorporate diverse attributes ( e.g. , distance, coverage and distributional similarity) to indicate a good MI attack and is robust against sample distribution manipulations. From the defense’s perspective, DDCS quantifies the susceptibility of individual samples to MI attacks by assigning each with a reconstruction distance, enabling the identification and protection of vulnerable samples in an unsupervised manner. Leveraging Diversity and Distance Composite Score (DDCS) to evaluate the success of reconstruction, we observe that many samples in the training dataset remain un-reconstructable by even the most advanced MI attacks ( Struppek et al. 2022 ) . This observation coincides with the limited diversity and coverage of current MI methodologies, mainly because prior metrics focus primarily on label-level privacy. In response, we introduce a Generative Adversarial Network (GAN) augmentation framework designed to enhance both coverage and diversity of MI attacks, through mitigating the produced artifacts while reserving the advantages from the inversion-specific GAN ( Karras et al. 2020b ; Chen et al. 2021 ) . To achieve this, our framework transfers a pre-trained GAN with entropy loss regularized with natural gradient descent ( Martens 2020 ) , so as to expand the attacker’s generative ability and mitigate artifacts typically associated with deep GAN structures, such as those found in StyleGAN ( Karras et al. 2020b ) . To validate our methodologies, we conduct comprehensive experiments on face recognition and dog breed classification datasets under various model architectures and MI algorithms. The results affirm the comprehensiveness of DDCS, underscoring its utility in evaluating sample-level privacy and robustness against manipulations of sample distributions. Further, the experiments substantiate the efficacy of our GAN augmentation framework in enhancing the performance of leading MI attacks, as evidenced by improvements in Fréchet Inception Distance, coverage, and DDCS metrics. Additionally, we also conduct a series of ablation studies to scrutinize the influence of different experimental parameters, ensuring a thorough understanding of our framework’s dynamics and its implications for privacy-preserving ML. To summarize, our contributions in this paper are as follows: • We introduce DDCS, a comprehensive evaluation metric for MI attacks with great potential for sample-level privacy analysis. • We propose a GAN augmentation framework to improve MI attacks on DDCS while retaining the image quality. • We conduct extensive experiments on computer vision tasks to demonstrate DDCS and verify the effectiveness our framework. The rest of the paper is organized as follows. We first review recent work on MI attacks. Later, we point out the limitations of evaluation metrics and introduce DDCS. We then propose our GAN augmentation framework and conduct extensive experiments before the conclusion. DDCS: A New Evaluation Metric for MI Prior studies on MI attacks have employed a range of evaluation metrics, each with its own limitations.

Read the complete text