Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators

Rajdeep Pathak, Sayantee Jana · 2026

Read Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators by Rajdeep Pathak, Sayantee Jana (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10937.txt.

Introduction

Rajdeep Pathak, Amit Basak, and Sayantee Jana propose a practical way to measure how much membership information tabular synthetic data leaks about the real records behind it. Their method uses kernel density estimators on nearest-neighbor distances between synthetic and training records. It turns membership inference into a probabilistic score that ROC curves can judge. Tests across four datasets and six generators beat a prior baseline. A realistic attack variant needs only auxiliary data, sparing custodians expensive shadow models.

Catalog record

CREATORRajdeep Pathak, Sayantee Jana
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29661
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10937

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators Rajdeep Pathak, Sayantee Jana Abstract The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive domains such as healthcare, finance, and demography. However, the privacy assurances of synthetic data are not absolute, and remain susceptible to membership inference attacks (MIAs), where adversaries aim to determine whether a specific individual was present in the dataset used to train the generator. In this work, we propose a practical and effective method to quantify membership disclosure risk in tabular synthetic datasets using kernel density estimators (KDEs). Our KDE-based approach models the distribution of nearest-neighbour distances between synthetic data and the training records, allowing probabilistic inference of membership and enabling robust evaluation via ROC curves. We propose two attack models: a ‘True Distribution Attack’, which assumes privileged access to training data, and a more realistic, implementable ‘Realistic Attack’ that uses auxiliary data without true membership labels. Empirical evaluations across four real-world datasets and six synthetic data generators demonstrate that our method consistently achieves higher F1 scores and sharper risk characterization than a prior baseline approach, without requiring computationally expensive shadow models. The proposed method provides a practical framework and metric for quantifying membership disclosure risk in synthetic data, which enables data custodians to conduct a post-generation risk assessment prior to releasing their synthetic datasets for downstream use. The datasets and codes for this study are available at https://github.com/PyCoder913/MIA-KDE. I Introduction Imagine sharing rich and sensitive healthcare, financial, or demographic data without risking individual privacy - synthetic data generation makes this possible. By training statistical or Machine Learning models on real data to simulate datasets, synthetic data aims to retain analytical utility while protecting individual identities. However, despite growing adoption, synthetic datasets remain susceptible to membership inference attacks (MIAs) [ 2 ] , where adversaries infer whether specific individuals were present in the original training data. This is particularly concerning when the inclusion of an individual in the dataset reveals sensitive information (e.g., HIV status, rare diseases, financial default). State-of-the-art MIAs typically employ computationally expensive shadow modelling strategies [ 16 , 17 , 6 ] , training multiple shadow generators to simulate attack scenarios and meta-classifiers for membership prediction. A more practical alternative uses distance-based methods that calculate F1 scores via data partitioning [ 4 , 11 , 3 ] , but these produce hard membership labels rather than probabilistic predictions. In this work, we introduce a non-parametric, distance-based framework using Kernel Density Estimators (KDEs) to model empirical distance distributions between synthetic and training data. Extending the methodology used in [ 4 , 11 , 3 ] , our approach generates probabilistic membership predictions, enabling improved analysis through ROC curves at low false positive rates while remaining computationally efficient. Our contributions are threefold: (1) a KDE-based privacy scoring framework for tabular synthetic data, (2) comprehensive validation across diverse datasets and generation models, and (3) demonstration of superior F1 scores compared to the existing data-partitioning method without requiring expensive shadow model training. II Background and Related Work II-A Synthetic Data Generation Synthetic data is generated by first training a statistical or machine learning model on the original dataset, and then producing artificial (synthetic) records by sampling new values from the learned model. Contemporary literature highlights diverse methodologies for synthetic data generation. Bayesian networks (e.g., BayNet, PrivBayes [ 26 ] ) model attribute dependencies via directed acyclic graphs and conditionals. Generative Adversarial Networks (GAN)-based approaches like CTGAN (Conditional Tabular GAN) [ 22 ] use adversarial training to capture complex data distributions, though they face challenges in tabular data synthesis. Generative models such as Variational Autoencoders (VAEs) [ 20 , 10 ] and diffusion models [ 9 ] have gained significant attention in synthetic data generation. Recently, Large Language Models (LLMs) have demonstrated promising results in synthesizing tabular data, capturing patterns and distributions inherent in real-world datasets. II-B Membership Inference Attacks Membership inference attacks (MIAs) have emerged as a standard approach for assessing the privacy risks associated with synthetic data and Machine Learning models. For synthetic tabular data, one class of MIAs involves direct comparisons between synthetic and original records to identify exact or near matches [ 23 ] . However, these similarity-based approaches have been criticized for underestimating privacy risks [ 17 ] . The state-of-the-art alternatives are shadow modelling approaches [ 16 , 17 , 6 ] , which are practically infeasible for large datasets due to their heavy computational overhead. For instance, consider a data custodian who routinely generates and sells synthetic datasets derived from proprietary, high-dimensional, and dynamic data containing billions of records, refreshed weekly. To assess privacy risk before each release, employing shadow modelling would require training numerous additional generative models - an approach that is both time and resource-intensive. Given the dynamic nature of the data, any delay in release diminishes its utility and commercial value to potential users. Breugel et al. present DOMIAS [ 19 ] , a membership inference attack against generative models, which estimates synthetic data and reference data distributions using methods like BNAF and KDE. It can be used as a loss during the training of the generative model, as well as a post-training metric to evaluate the synthetic data MI risk. A more popular approach implemented in practice to evaluate membership disclosure risks of synthetic data is a distance-based approach [ 11 , 4 , 3 , 2 ] , which we elaborate in Section III-A . Our proposed method is grounded on the distance-based method, where we model the distance distributions using KDEs. II-C Kernel Density Estimation Kernel Density Estimation (KDE), also known as the Parzen’s window [ 14 ] , is a fundamental nonparametric technique used to estimate the probability density function (pdf) of a random variable based on a finite data sample. Unlike parametric models, KDE does not assume a specific functional form for the underlying distribution, making it especially useful for exploring complex, multimodal, or unknown distributions. Given a sample X_{1},\dots,X_{n}\in\mathbb{R}^{d} drawn independently from an unknown distribution with density p(x) , the kernel density estimator at a point x\in\mathbb{R}^{d} is defined as, \hat{p}_{n}(x)=\frac{1}{nh^{d}}\sum_{i=1}^{n}K\left(\frac{x-X_{i}}{h}\right), where K:\mathbb{R}^{d}\to\mathbb{R} is a smooth, symmetric kernel function (e.g., Gaussian or Epanechnikov), and h>0 is the bandwidth (smoothing parameter) controlling the scale of smoothing. III Methodology III-A Motivation This section discusses the practical approach for quantifying membership disclosure risk from El Emam et al. [ 4 ] and establishes our proposed framework. Membership inference attacks simulate a classification task where adversaries classify records as training set members or non-members. Given real dataset R (training set) generating synthetic data S , and unseen data U (not used in training), we form an attack dataset D_{\text{attack}} comprising records from R and U . The standard approach in [ 11 , 4 , 3 , 2 ] (which we hereafter refer to as Method 1 ) sets a distance threshold \tau , computes nearest-neighbour distances between D_{\text{attack}} records and synthetic data S , and classifies records below the threshold as members, and others as non-members. Finally, the F1 score for this classification quantifies the membership disclosure risk. However, Carlini et al. [ 1 ] argue that average-case metrics like F1 scores

Read the complete text