A Comparative Study of Open-Source Libraries for Synthetic Tabular Data Generation: SDV vs. SynthCity

Cristian Del Gobbo · 2025

Read A Comparative Study of Open-Source Libraries for Synthetic Tabular Data Generation: SDV vs. SynthCity by Cristian Del Gobbo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-17847.txt.

Introduction

Cristian Del Gobbo wrote a solo study in two thousand twenty-five comparing two open-source toolkits that make fake data tables. Fake, or synthetic, data copy the patterns of real data without exposing it. Good training data are hard to get, above all for small firms, so such tools have clear appeal.

The study tests six generators, three from each toolkit, on a real data set about energy use and weather in Belgium. Each tool was trained on just one thousand rows, a low-data test. It then had to make sets of the same size and ten times larger. Scores covered how close the fake data stayed to the real patterns.

Catalog record

CREATORCristian Del Gobbo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-17847

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Comparative Study of Open-Source Libraries for Synthetic Tabular Data Generation: SDV vs. SynthCity Cristian Del Gobbo Abstract High-quality training data is critical to the performance of machine learning models, particularly Large Language Models (LLMs). However, obtaining real, high-quality data can be challenging, especially for smaller organizations and early-stage startups. Synthetic data generators provide a promising solution by replicating the statistical and structural properties of real data while preserving privacy and scalability. This study evaluates the performance of six tabular synthetic data generators from two widely used open-source libraries: SDV (Gaussian Copula, CTGAN, TVAE) and Synthicity (Bayesian Network, CTGAN, TVAE). Using a real-world dataset from the UCI Machine Learning Repository, comprising energy consumption and environmental variables from Belgium, we simulate a low-data regime by training models on only 1,000 rows. Each generator is then tasked with producing synthetic datasets under two conditions: a 1:1 (1,000 rows) and a 1:10 (10,000 rows) input-output ratio. Evaluation is conducted using two criteria: statistical similarity, measured via classical statistics and distributional metrics; and predictive utility, assessed using a "Train on Synthetic, Test on Real" approach with four regression models. While statistical similarity remained consistent across models in both scenarios, predictive utility declined notably in the 1:10 case. The Bayesian Network from Synthicity achieved the highest fidelity in both scenarios, while TVAE from SDV performed best in predictive tasks under the 1:10 setting. Although no significant performance gap was found between the two libraries, SDV stands out for its superior documentation and ease of use, making it more accessible for practitioners. Keywords: Synthetic Data Generation, Tabular Data, Predictive Utility, Statistical Similarity, SDV, Synthicity, Machine Learning Benchmarking, Data-Centric AI Introduction The performance of machine learning models, particularly in supervised learning tasks, is heavily dependent on the availability of large volumes of high-quality data [ 1 ] . However, real-world datasets are often limited due to privacy concerns, regulatory constraints, high labeling costs, or data sparsity, especially in domains such as healthcare, energy systems, and finance [ 2 ] . As a result, synthetic data generation has emerged as a promising alternative that can supplement or even replace real data in certain use cases [ 3 ] . While synthetic data has been widely explored in computer vision and natural language processing [ 4 , 5 ] , its application to structured, tabular data, common in many real-world business and scientific settings, has gained traction more recently. To support this growing need, several open-source tools and libraries have been developed. Among the most prominent are the Synthetic Data Vault (SDV) and Synthicity, both of which provide user-friendly APIs for generating synthetic tabular data using a variety of models, ranging from classical probabilistic approaches to modern deep generative networks. Despite the growing popularity of these tools, few studies have systematically compared their performance under controlled, real-world conditions. This study aims to fill that gap by evaluating and comparing SDV and Synthicity in terms of both statistical fidelity and downstream predictive utility. Using a real-world energy consumption dataset, we generate synthetic data under two experimental conditions: one where the output data matches the input size (1:1), and another where it scales tenfold (1:10). We assess performance using both statistical similarity metrics and a Train-on-Synthetic, Test-on-Real (TSTR) predictive evaluation pipeline. Our findings offer practical insights into the strengths and limitations of each library and provide guidance for researchers and practitioners selecting synthetic data generation tools for tabular data tasks. Literature Review In the past few years, Large Language Models (LLMs) have become prominent, and with the release of commercial models like ChatGPT by OpenAI in November 2022 [ 6 ] , their power became available to anyone with internet access, greatly impacting many aspects of daily life [ 7 ] . A common belief behind the success of LLMs is the scaling law of computing, model size, and, perhaps most importantly, the high quality of pre-training data [ 8 ] . The biggest LLMs today are often pre-trained on trillions of tokens. For example, GPT - 3 was famously trained on nearly 500 billion tokens [ 9 ] , from a mixture of web text, books, and other sources. GPT - 4 is rumored to have used well over a trillion tokens [ 10 ] , and Anthropic’s Claude reportedly relies on a similarly large-scale corpus, likely in the hundreds of billions to trillions of tokens [ 11 ] . The exact numbers for GPT - 4 and Claude have not been officially disclosed by OpenAI and Anthropic, but external analyses report similar figures [ 12 ] . However, acquiring such a massive quantity of high-quality data has become more challenging [ 13 ] . Many sources are now gated behind paywalls, restricted by copyright, or filtered due to data quality concerns [ 14 ] . As the demand for high-quality training data grows, finding scalable solutions for future LLM development remains an open question. As a remedy, synthetic data has been widely adopted in training LLMs, offering a more accessible and controllable alternative to real-world data [ 15 , 16 ] . Chen et al. [ 8 ] conducted a study on the measurement of diversity in synthetic data and its impact on LLM performance. They examined how synthetic data diversity influences both pre-training and fine-tuning stages, introducing a new diversity metric called LLM Cluster-Agent, designed specifically to evaluate the diversity of synthetic datasets. They define LLM Cluster-Agent as “a diversity measure pipeline that leverages LLM’s ability to interpret semantic meanings and understand rich contexts of text samples for clustering”. This metric is particularly suited for text-based synthetic data, which is commonly used in the pre-training process of large LLMs, rather than for tabular data. Through a series of controlled experiments with 350M and 1.4B parameter models, Chen et al. demonstrated that higher diversity in synthetic data correlates positively with both pre-training and fine-tuning performance. Interestingly, their findings suggest that synthetic data diversity in pre-training has an even stronger effect on fine-tuning than on pre-training itself. Although this study differs from our goal of comparing tabular data generators rather than synthetic text data, it is still relevant because it highlights how synthetic data can be effectively leveraged in the pre-training of LLMs. The use of synthetic data generators for training LLMs, however, is not their only application. In fact, synthetic data generation is now widely used across multiple domains. Lu et al. [ 17 ] presented a comprehensive review of existing studies on employing machine learning for synthetic data generation, highlighting applications spanning computer vision, speech, natural language processing, healthcare, and business domains. Their review categorizes existing approaches based on machine learning techniques, with a particular emphasis on deep generative models, including GANs, VAEs, and reinforcement learning-based methods. One of the key findings of their study is that the effectiveness of synthetic data depends on the application domain. In computer vision, synthetic datasets are frequently used to train models for object detection, facial recognition, and domain adaptation when real-world labeled data is scarce. In speech processing, synthetic data has proven valuable in speech synthesis and voice cloning, reducing the need for extensive manually labeled datasets. In natural language processing, it is used to augment training datasets for tasks such

Read the complete text