Disjoint Generative Models

Anton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp · 2025

Read Disjoint Generative Models by Anton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19700.txt.

Introduction

Fake data sets that stand in for private records are usually made by one model trained on the whole table. Anton Danholt Lautrup and four co-authors propose a split approach in 2025. They cut the table into separate pieces, let a different model learn each piece, and join the fake pieces afterward without shared keys. They call the frame disjoint generative models.

Case studies on table data show three gains. Privacy checks improved, some costly model types became workable, and mixing model kinds closed much of the gap between privacy and usefulness. Scores on later tasks stayed near the best while the risk of naming a real person fell sharply. The authors present the split as a flexible design choice for data sharing.

Catalog record

CREATORAnton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-19700

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Disjoint Generative Models Anton Danholt Lautrup, Muhammad Rajabinasab, Tobias Hyrup, Arthur Zimek, Peter Schneider-Kamp Abstract We propose a new framework for generating cross-sectional synthetic datasets via disjoint generative models. In this paradigm, a dataset is partitioned into disjoint subsets that are supplied to separate instances of generative models. The results are then combined post hoc by a joining operation that works in the absence of common variables/identifiers. The success of the framework is demonstrated through several case studies and examples on tabular data that helps illuminate some of the design choices that one may make. The principal benefit of disjoint generative models is significantly increased privacy at only a low utility cost. Additional findings include increased effectiveness and feasibility for certain model types and the possibility for mixed-model synthesis. 1 Introduction A common strategy for solving difficult tasks is to “divide and conquer”, i.e., to break the task into smaller, manageable sub-problems which are solved individually and then combined post hoc into a solution for the original problem. This paradigm can be applied effectively (and often recursively) to many branches of computer science (e.g., [ 1 , 2 , 3 ] ), but remains to be explored for generative modelling. Motivated by this gap, we propose a new generative model procedure for synthetic data generation in the tabular regime using partitioning. In the proposed Disjoint Generative Models (DGMs) framework, training data are partitioned column-wise into disjoint subsets of variables, these are then used to train separate generative models before a joining operation combines the independent outputs. Disjoint generation challenges the notion that using all available data together leads to a superior outcome. While learning the overall distribution is certainly useful for the utility of synthetic data, some models struggle to reliably capture high-dimensional spaces or overfit to the detriment of privacy. DGMs enable specifying partitions based on the strengths and weaknesses of the included models, mixed model generation, multi-modal data, and using the joining operation and its control parameters to balance utility and privacy effectively. This framework can be used with many types of models, datasets, and joining algorithms. In this work, we focus primarily on the special case where the joining operation is done by repeatedly querying a validation model with joined candidates. Additionally, we limit the experimental treatment to tabular data for ease of evaluation. We establish the framework through various case studies with promising results and propose future directions to be explored. We make an easy-to-use and extendable implementation of disjoint synthetic data generation available for the community to use and iterate upon. 2 Background Synthetic data are products of a generative model that mimic authentic samples (e.g., patient records, consumer profiles, etc.), following the same statistical distributions and multivariate relationships without copying. This makes synthetic data extremely attractive for purposes of privacy, amplification, and fairness augmentation [ 4 , 5 , 6 , 7 ] . Tabular data are one of the most promising applications, with many opportunities to be explored for national and private data banks and commercial cooperation scenarios between companies. Typically, the process of creating synthetic data in an application setting involves getting all the relevant data onto the same server, linking on primary and foreign keys, followed by various preprocessing steps such as imputation. The next steps are running a big generative model and, finally, conducting an evaluation to ensure that the generated data meet the required specifications [ 8 , 9 ] . One exception to this workflow is in the federated/distributed setting, where all records and/or features cannot (for practical or legal reasons) be joined on a single server. Under these conditions, it is a major challenge to create even half-decent generative models, and despite best efforts, state-of-the-art generative models in a horizontally or vertically federated learning setting (see e.g., [ 10 , 11 , 12 ] ) perform at best similarly compared to the non-distributed case. In our scenario, instead of working within the confines of necessity, we purposely partition the data, hoping to have some benefit towards utility, privacy, and efficiency. Conditional generation is perhaps the closest related concept, and conditioning based on chunks of already generated variables has been attempted previously in studies with Bayesian Network models and Auto-Regressive Generative Models [ 13 , 14 ] . Both examples show a limited but optimistic palette of results with good efficiency and performance compared to their baselines. Splitting data column-wise (or having data supplied as such) may give rise to a host of benefits, including computational speedup, access to mixed-model synthesis, and increased privacy, but also raises certain challenges. Namely, while there remains a relationship in the training data between rows of disjoint tables, unless conditioning is enforced in the model (as in [ 13 ] ), synthetic samples will generally not belong together with those modelled based on another slice of the dataset. This raises the problem of column-wise joining of datasets with no shared variables or keys and represents a challenge that few authors have attempted to solve, although some examples exist in record linkage [ 15 ] or federated learning [ 16 ] . In both cases, some information is shared between the datasets being joined, enabling rule-based, probabilistic, and machine learning methods of reassembly. Machine learning techniques and similarity learning generally outperform probabilistic methods [ 17 , 18 ] . Figure 1 : Disjoint Generation Conceptual Overview. The figure shows how our approach splits training data into a number of subsets with similar or different sizes and with no shared variables. Generative models can then be applied in parallel, and the resulting synthetic datasets are then joined using some joining operation into a final output. 3 Disjoint Generative Models The proposed disjoint generative models framework offers an alternative route to generating tabular synthetic datasets. Instead of using one generative model for the synthesis, DGMs distribute the labour by handling disjoint partitions of variables using different generative model instances and combining the results post hoc (see Figure 1 ). First, let the initial dataset \bf X be an (n\times k) matrix consisting of n independent observations in k variables, i.e., \bf X=\{\bf x_{1},\bf x_{2},\dots,\bf x_{k}\} . Allocate \bf X into the required number of column-wise partitions n_{p} , using some assignment function \mathtt{r}(\cdot) which distributes attributes according to some specifications or by random assignment. Next, each dataset partition d_{p} is handed to a unique generative model instance \mathtt{G}_{p}(\cdot) , which creates synthetic dataset partition s_{p} . Note that in practice, it may be helpful to oversample s_{p} such that n<|s_{p}| , to ensure enough valid matches are possible in the joining operation. As mentioned previously, this paper treats the special case when the joining operation is done using a model for validating joins. We define \mathtt{V}(\cdot) , the validator model, as an object that assigns a score z , of a query point \bf q to be a valid observation. In the case where it is a binary classifier, we train \mathtt{V} on \bf X as well as \bf X^{\prime}=[d^{\prime}_{1}|d^{\prime}_{2}|\dots|d^{\prime}_{n_{p}}] where the prime indicates that the partition has been randomly shuffled row-wise independently of the other partitions. Artificial labels are assigned accordingly (i.e., 0 for the random joins \bf X^{\prime} and 1 for authentic joins

Read the complete text