Winning the MIDST Challenge: New Membership Inference Attacks on Diffusion Models for Tabular Data Synthesis
Xiaoyu Wu, Yifei Pang, Terrance Liu, Steven Wu · 2025
Read Winning the MIDST Challenge: New Membership Inference Attacks on Diffusion Models for Tabular Data Synthesis by Xiaoyu Wu, Yifei Pang, Terrance Liu, Steven Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12008.txt.
Introduction
Xiaoyu Wu and co-authors attack fake data privacy in a 2025 paper that won a challenge. Diffusion models can make synthetic table data that mimic real records, promising a balance of use and privacy. Privacy checks, though, often rest on weak guesses or feeble attacks, so real risk stays hidden.
The team asked whether an attacker can tell if a given record was in the training set. Attacks built for image models failed on tables. Noise settings at the start of generation proved to shape attack success. Their learning based attack reads loss signals across noise levels and time steps with a small network, and picks up membership far better.
Catalog record
| CREATOR | Xiaoyu Wu, Yifei Pang, Terrance Liu, Steven Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-12008
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Winning the MIDST Challenge: New Membership Inference Attacks on Diffusion Models for Tabular Data Synthesis Xiaoyu Wu, Yifei Pang, Terrance Liu, Steven Wu Abstract Tabular data synthesis using diffusion models has gained significant attention for its potential to balance data utility and privacy. However, existing privacy evaluations often rely on heuristic metrics or weak membership inference attacks (MIA), leaving privacy risks inadequately assessed. In this work, we conduct a rigorous MIA study on diffusion-based tabular synthesis, revealing that state-of-the-art attacks designed for image models fail in this setting. We identify noise initialization as a key factor influencing attack efficacy and propose a machine-learning-driven approach that leverages loss features across different noises and time steps. Our method, implemented with a lightweight MLP, effectively learns membership signals, eliminating the need for manual optimization. Experimental results from the MIDST Challenge @ SaTML 2025 demonstrate the effectiveness of our approach, securing first place across all tracks. Code is available at https://github.com/Nicholas0228/Tartan_Federer_MIDST . 1 Introduction Tabular data synthesis has garnered significant attention for its potential to safeguard data privacy [ 11 , 1 , 4 , 10 , 6 ] , providing a means to facilitate downstream tasks while possibly mitigating the risk of user information leakage. Recent studies have explored the adaptation of advanced diffusion models for tabular data synthesis, claiming that these models achieve significantly higher generation quality while maintaining privacy [ 4 , 10 , 6 ] . Although the assertion that synthetic data enhances privacy is widely accepted in the literature, it has not been rigorously verified in many scenarios. For instance, prior works on diffusion-based tabular data synthesis have either primarily relied on basic privacy evaluation methods, such as Distance to Closest Records (DCR) [ 10 , 6 ] , or measure under membership inference attack (MIA) but using some basic attacking method [ 4 ] . These approaches serve as intuitive measurements but may underestimate the extent of potential privacy leakage. In this paper, we employ MIA to assess the privacy risks of diffusion models for tabular data, ensuring a broad coverage of attack methods to provide a comprehensive evaluation. MIA [ 7 ] offers a direct and informative measure of privacy leakage by explicitly quantifying an attacker’s ability to distinguish training data from non-training data. However, we show that existing MIA techniques designed for diffusion models are not directly applicable to tabular data synthesis. This necessitates a more rigorous evaluation, and we demonstrate that significantly stronger attacks can be developed for this setting. Specifically, we first apply SecMI [ 2 ] , one of the strongest established attacks for image-based diffusion models. However, we find that its performance on tabular data diffusion models is nearly indistinguishable from random guessing. This highlights the fundamental differences in data distribution and structure, suggesting that tabular data requires specialized attack methods tailored to its unique characteristics. Further investigation reveals that the choice of initialized noise significantly impacts attack performance, leading to high variance across different noise selections. A natural approach to addressing this issue is to identify an optimal noise configuration. However, this is non-trivial, as prior studies on diffusion models suggest that effective noise selection often requires complex optimization [ 5 ] or additional training data [ 12 ] . To circumvent this challenge, we propose a more direct, machine-learning-driven approach that mitigates the influence of noise selection and the time variable t . Specifically, we treat loss values as features, computing multiple losses for each data point using different noises and time variables. These loss features are then processed by a lightweight three-layer MLP to predict membership status. Intuitively, this approach enables the model to automatically learn the relationship between noise, time, and membership information, eliminating the need for manual optimization or heuristic selection. Experimental results from the MIDST Challenge @ SaTML 2025 [ 9 ] demonstrate the effectiveness of our method, securing first place across all tracks. 2 Related works 2.1 Diffusion Models for Tabular Data Generation Diffusion models have been increasingly adopted for tabular data generation, with various approaches designed to handle its unique structure and constraints. TabDDPM [ 4 ] employs denoising diffusion models with separate diffusion processes for numerical and categorical data. TabSyn [ 10 ] integrates a transformer-based variational autoencoder with latent diffusion modeling. ClavaDDPM [ 6 ] leverages clustering labels as intermediaries to capture relationships between tables, particularly enforcing foreign key constraints, thereby enabling multi-table tabular data generation. 2.2 Membership Inference Attack on Diffusion Models Diffusion models (DMs) [ 3 , 8 ] generate data by iteratively denoising a Gaussian-sampled variable. They consist of a forward process, where noise \varepsilon\sim\mathcal{N}(0,1) is progressively added to data x_{0} , forming noisy samples: x_{t}=\sqrt{\alpha_{t}}x_{0}+\sqrt{1-\alpha_{t}}\varepsilon. In the backward process, a model \epsilon_{\theta} predicts and removes noise from x_{t} . The training objective, is known as the diffusion loss: \mathcal{L}_{DM}=\mathbb{E}_{\varepsilon\sim\mathcal{N}(0,1),t}\left[\|\epsilon_{\theta}(x_{t},t)-\varepsilon\|_{2}^{2}\right]. This loss guides the model to reconstruct clean data from noisy inputs, enabling effective data generation. MIA on diffusion models varies in design, primarily focusing on deriving strong loss functions to better assess memorization. Among them, SecMI [ 2 ] is one of the most influential approaches. It leverages the DDIM-forwarding process to actively add noise to a given sample, represented as x_{t+1}=\phi_{\theta}(x_{t},t) , and the general backward process to perform one-step denoising, denoted as \psi_{\theta}(x_{t},t) . Membership is then determined using a t -error metric: \tilde{l}_{t,x_{0}}=\|\psi(\phi(\tilde{x}_{t},t),t)-\tilde{x_{t}}\|_{2}, (1) where \tilde{x}_{t}=\Phi_{\theta}(x_{0},t)=\phi_{\theta}(\dots\phi_{\theta}(\phi_{\theta}(x_{0},0),1),t-1). 3 Threat Model and Evaluation We consider a MIA against a diffusion model trained for tabular data synthesis. Let the target diffusion model be parameterized by \theta and denoted as \epsilon_{\theta} . Given a dataset \mathcal{D}=\{x_{i}\}_{i=1}^{N} , the model is trained to approximate the underlying data distribution and generate synthetic samples. Adversary’s Goal. The adversary aims to determine whether a given data point x was part of the training dataset \mathcal{D} . Specifically, given a query sample x , the adversary must infer the binary membership status m_{x}\in\{0,1\} , where m_{x}=1 if x\in\mathcal{D} and m_{x}=0 otherwise. Adversary’s Knowledge. We consider two attack settings: black-box and white-box. The setup consists of: 30 train-phase models \{\epsilon_{\theta^{(i)}}\}_{i=1}^{30} , where the adversary has access to their training samples. 20 dev-phase models \{\epsilon_{\theta^{(i)}}\}_{i=31}^{50} and 20 final-phase models \{\epsilon_{\theta^{(i)}}\}_{i=51}^{70} , for which training samples are not accessible. In the white-box setting, the adversary has access to: • The full model weights of the 30 train-phase models \{\epsilon_{\theta^{(i)}}\}_{i=1}^{30} , along with their corresponding training samples. • The model weights of the 20 dev-phase and 20 final-phase models \{\epsilon_{\theta^{(i)}}\}_{i=31}^{70} , but without access to their training samples. With this access, the adversary attempts to infer the membership status of a given sample x using an adversarial function A(\cdot) , which directly takes the model as input to compute a membership prediction: A(\epsilon_{\theta^{(i)}},x)\rightarrow\hat{m}_{x}. (2) In the black-box setting, the adversary has access to: • The training samples of the 30 train-phase models, but not the models themselves. • A synthetic dataset \{X_{g}^{(i)}\}_{i=1}^{70} , containing an equal number of samples generated by the given models across all phases, without access to their weights. In this case, the adversary relies on model-generated samples to make a membership prediction: A(X_{g}^{(j)},x,\{X_{g^{(i)}}\}^{30}_{i=1})\rightarrow\hat{m}_{x}. (3) For each model \epsilon_{\theta^{(j)}} , the adversary is provided with a balanced dataset consisting of an equal number of member