TabGen-ICL : Residual-Aware In-Context Example Selection for Tabular Data Generation

Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu · 2025

Read TabGen-ICL : Residual-Aware In-Context Example Selection for Tabular Data Generation by Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-16414.txt.

Introduction

Liancheng Fang and co-authors improve how language models invent realistic table data. Their 2025 paper skips the usual fine-tuning, which is pricey, and instead prompts a fixed model with a few real examples. Random example choices, they show, drag quality down, so the choice of examples is where their method, TabGen-ICL, does its work.

The method runs in rounds. Each round finds real rows that cover the gap between what the model has made so far and the true data spread, then feeds those rows back as fresh examples. Locally this teaches better patterns; globally the invented tables creep closer to the real thing. Wide tests back the claim.

Catalog record

CREATORLiancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-16414

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TabGen-ICL : Residual-Aware In-Context Example Selection for Tabular Data Generation Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu Abstract Large Language models (LLMs) have achieved encouraging results in tabular data generation. However, existing approaches require fine-tuning, which is computationally expensive. This paper explores an alternative: prompting a fixed LLM with in-context examples. We observe that using randomly selected in-context examples hampers the LLM’s performance, resulting in sub-optimal generation quality. To address this, we propose a novel in-context learning framework: TabGen-ICL , to enhance the in-context learning ability of LLMs for tabular data generation. TabGen-ICL operates iteratively, retrieving a subset of real samples that represent the residual between currently generated samples and true data distributions. This approach serves two purposes: locally, it provides more effective in-context learning examples for the LLM in each iteration; globally, it progressively narrows the gap between generated and real data. Extensive experiments on five real-world tabular datasets demonstrate that TabGen-ICL significantly outperforms the random selection strategy. Specifically, it reduces the error rate by a margin of 3.5 % − 42.2 % 3.5\%-42.2\% on fidelity metrics. We demonstrate for the first time that prompting a fixed LLM can yield high-quality synthetic tabular data. The code is provided in the link . 1 Introduction Tabular data, despite being one of the most prevalent data modalities in real-world applications ( Benjelloun et al., 2020 ) , often encounters several issues in practical use. These include imbalanced data categories ( Cao et al., 2019 ) , privacy concerns ( Gascón et al., 2016 ) (as many tabular datasets contain sensitive personal information that cannot be directly shared), insufficient data quality Lin and Tsai (2020) , and high data collection costs ( Even et al., 2007 ) . Tabular generation is an important means to address these problems. Classic tabular generation methods such as GANs ( Xu et al., 2019 ) , VAEs ( Liu et al., 2023 ) , and diffusion models ( Kim et al., 2023 ; Lee et al., 2023 ; Kotelnikov et al., 2023 ; Zhang et al., 2024 ) have two main limitations. First, they require large amounts of tabular data for training, which leads to a noticeable decline in performance in low-resource scenarios. This is particularly problematic considering that most real-world situations requiring tabular generation lack abundant data. Second, they need special preprocessing to handle heterogeneous data types, making them less flexible. The rapid development of large language models (LLMs) brings new possibilities for solving table data generation problems with their powerful semantic understanding, reasoning, and generation capabilities. LLMs can understand and process various data types and structures without complicated data preprocessing, offering more flexible and principled solutions. Moreover, LLMs’ few-shot learning ability may alleviate data scarcity issues, enabling excellent performance in low-resource scenarios. Previous works ( Borisov et al., 2023 ; Solatorio and Dupriez, 2023 ; Zhang et al., 2023 ; Zhao et al., 2023 ; Gulati and Roysdon, 2023 ; Xu et al., 2024a ; Wang et al., 2024 ) resort to fine-tuning general-purpose LLMs on target tables. While effective, fine-tuning requires substantial computational resources, making it inapplicable in resource-scarce scenarios. (a) No in-context Ex. (b) Sampled in-context Ex. (c) Fixed in-context Ex. (d) Ground Truth Figure 1: Comparison of samples generated with different in-context learning examples. Plots show the latitude and longitude coordinates of California housing, with the solid line representing the state boundary. (a) 2000 samples generated by LLM with only the table header as input, without any in-context examples. (b) 2000 samples generated by LLM, giving in-context examples sampled from the real dataset. (c) 2000 synthetic samples generated by LLM, giving in-context examples with latitude and longitude in a fixed range. (d) 2000 samples from the ground truth training table. In-context learning effectively solves such problems. By adding examples to the context, distribution characteristics can be provided to LLMs, guiding them to generate data that conforms to the target distribution without specific fine-tuning ( Gao et al., 2023 ) . However, simple in-context learning strategies still face challenges. Figure 1 (a) shows that even without in-context examples (see the full prompt at Appendix A.2 ), LLMs can generate reasonable distributions, reflecting the influence of the LLM’s pre-training distribution. Figure 1 (b) demonstrates the strategy proposed by ( Seedat et al., 2024 ) , which involves random sampling from Ground Truth as in-context examples. Although the generated results are closer to the Ground Truth shown in Figure 1 (d) compared to Figure 1 (a), they are still mainly influenced by the LLM’s original distribution and struggle to fit the Ground Truth. This phenomenon reveals the importance of choosing in-context examples. In this work, we propose TabGen-ICL , a dynamic in-context example selection method. Inspired by the observation in Figure 1 (c), we found that using fixed range in-context examples leads to generated distributions closely mimicking those examples, significantly differing from the LLM’s original distribution. This indicates in-context learning’s ability to simulate distributions. By carefully selecting in-context examples, we can more effectively guide LLMs to generate distributions closer to the ground truth. Central to our framework is the design of an automated strategy for selecting effective in-context examples while ensuring global consistency with the real data distribution. Our key idea is to utilize simple, discernible patterns in subsets of real samples, which can effectively guide LLMs in generating realistic tabular data. Specifically, TabGen-ICL identifies subsets of real samples that exhibit simple patterns and closely match the residual between the current generated data distribution and the real data distribution. This idea can be categorized as a novel residual-aware RAG technique, where we retrieve in-context examples based on the residual between the generated and real data distributions. The residual-aware sampling measures the discrepancy between the generated and real data distributions, focusing on areas where the model needs improvement. This approach enables TabGen-ICL to progressively narrow the distribution gap while maintaining the use of easily learnable patterns in the in-context examples. Our sampling technique offers two key advantages: flexibility in selecting simple patterns for effective learning, and consistent generation through progressive distribution alignment. The contributions of this paper are as follows: 1. We propose TabGen-ICL , an in-context learning selection method that retrieves in-context examples by leveraging residual between currently generated samples and true data distributions. 2. We conduct extensive experiments on five datasets, evaluated under three distinct groups of synthetic data evaluation metrics. Experiment results show that TabGen-ICL outperforms the previous in-context learning method by a margin of 3.5\%-42.2\% across multiple fidelity metrics. Notably, TabGen-ICL surpasses state-of-the-art deep generative models under the data-scarce scenarios. 2 Related works Deep generative models for synthetic tabular data generation Generative models for tabular data have become increasingly important and have widespread applications Assefa et al. (2021) ; Zheng and Charoenphakdee (2022) ; Hernandez et al. (2022) . For example, CTGAN and TAVE ( Xu et al., 2019 ) deal with mixed-type tabular data generation using the basic GAN ( Goodfellow et al., 2014 ) and VAE ( Kingma and Welling, 2013 ) framework. GOGGLE ( Liu et al., 2023 ) incorporates Graph Attention Networks in a VAE framework such that the correlation between different data columns can be explicitly learned. Recently, inspired

Read the complete text