AIGT : AI Generative Table Based on Prompt
Mingming Zhang, Zhiqing Xiao, Guoshan Lu, Sai Wu, Weiqiang Wang, Xing Fu, Can Yi, Junbo Zhao · 2024
Read AIGT : AI Generative Table Based on Prompt by Mingming Zhang, Zhiqing Xiao, Guoshan Lu, Sai Wu, Weiqiang Wang, Xing Fu, Can Yi, Junbo Zhao (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18111.txt.
Introduction
Mingming Zhang and co-authors introduce AIGT, a system that fabricates realistic spreadsheet-style data on request. Tables hold over 80 percent of company data assets, but privacy rules often block sharing the real thing. Language models can fake table rows well because they grasp what the columns mean, yet older methods ignore much of what a table carries. AIGT feeds the model the table's own description and schema as part of the prompt, which sharpens the output.
A second trick splits long tables into parts so the model can handle tables of any scale within its token limit. The system reached the best scores on 14 of 20 public datasets. It also led on two real datasets inside the Alipay risk control system. The authors pitch AIGT for safe data sharing and model testing.
Catalog record
| CREATOR | Mingming Zhang, Zhiqing Xiao, Guoshan Lu, Sai Wu, Weiqiang Wang, Xing Fu, Can Yi, Junbo Zhao |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-18111
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
AIGT : AI Generative Table Based on Prompt Mingming Zhang, Zhiqing Xiao, Guoshan Lu, Sai Wu, Weiqiang Wang, Xing Fu, Can Yi, Junbo Zhao Abstract Tabular data, which accounts for over 80% of enterprise data assets, is vital in various fields. With growing concerns about privacy protection and data-sharing restrictions, generating high-quality synthetic tabular data has become essential. Recent advancements show that large language models (LLMs) can effectively generate realistic tabular data by leveraging semantic information and overcoming the challenges of high-dimensional data that arise from one-hot encoding. However, current methods do not fully utilize the rich information available in tables. To address this, we introduce AI G enerative T able ( AIGT ) based on prompt enhancement, a novel approach that utilizes metadata information, such as table descriptions and schemas, as prompts to generate ultra-high-quality synthetic data. To overcome the token limit constraints of LLMs, we propose long-token partitioning algorithms that enable AIGT to model tables of any scale. AIGT achieves state-of-the-art performance on 14 out of 20 public datasets and two real industry datasets within the Alipay risk control system. 1 1 footnotetext: Indicates equal contribution. 1 Introduction Given the prevalence of tabular data in practical applications, synthesizing high-quality tabular data is an essential task. It ensures privacy protection, enhances the generalization capabilities of machine learning models, and boosts performance in scenarios with limited samples. However, this task is fraught with unique challenges, including specificity, impurities, class imbalances, and privacy concerns. Traditional approaches to tabular data synthesis, such as generative models and statistical methods, often lose textual information and struggle with capturing complex feature relationships Choi et al. 2017 ; Park et al. 2018 ; Xu et al. 2019a ; Borisov et al. 2022 . Recent efforts have explored the use of language models to incorporate feature names for better contextual learning. For instance, GReaT Borisov et al. 2023 made the pioneering attempt to generate tabular data using large language models (LLMs), achieving satisfactory synthetic data. TapTap Zhang et al. 2023 further enhanced the quality of synthetic data through table pre-training and applied it to data augmentation in tabular data for improved performance in prediction tasks, achieving state-of-the-art (SOTA) results. However, these methods are constrained by token limits, limiting their ability to handle arbitrarily wide tables. Additionally, they fail to fully utilize crucial tabular elements such as headers and column names, resulting in incomplete information integration. Recently, Artificial Intelligence Generation and Creation (AIGC) technologies, including large-scale language models and automatic image generation techniques, are driving progress in AI’s capabilities for content creation and data generation. Prompt Learning, as an innovative approach, aligns the pre-training of language models with specific downstream tasks. By using well-designed prompts, which include task descriptions, input data, background information and output indicators, create a template that directs the model’s attention and thus improves the accuracy and relevance of the generated context. AIGC is more broadly applied to the generation of multimedia content, such as images, videos, and audio. Combined with Prompt Learning, AIGC systems can generate content more accurately according to user instructions or needs. However, The application of AIGC technology in the generation of tabular data is currently relatively limited. Although work on table generation based on large-scale language models has begun to explore, these efforts have mostly utilized only the cell values in the tables, without fully leveraging the more comprehensive information contained in the tabular data. We recognize that table metadata, such as headers and column names, provides valuable context often overlooked in analysis. Inspired by AIGC and Prompt learning, we propose a prompt-enhanced model that leverages this metadata to improve performance. Therefore, we propose AI G enerative T able ( AIGT ) technique based on prompt enhancement, which leverages metadata to enhance the quality and relevance of table generation. Methods based on language models inherently face limitations in generating long sequences of tokens, which is why previous approaches like GReaT and TapTap struggled to model tables with a large number of columns—something very common in real-world industry data. To address this, we developed a long-token partitioning algorithm specifically adapted for AIGT. This approach allows AIGT to effectively handle data synthesis tasks for tables of any size, overcoming the token length constraints. We further demonstrate AIGT’s effectiveness in real-world scenarios such as Alipay’s risk control system for commercial credit and merchant fraud detection. Our key contributions are as follows: (1) A Prompt-Enhanced LM for Tabular Data Synthesis: We utilize the metadata of tables to construct prompt-enhanced language model and design a series of training techniques to enhance the capability of table generation. (2) Scalability: Our proposed partitioning algorithm enables LM-based generation methods to be scaled to tables of any size, overcoming the token limit constraints of LM methods. (3) Superior Performance: We achieve state-of-the-art results on 14 out of 20 academic datasets and two real-world industry datasets from Alipay. 2 Related Works In this section, we provide a brief background on prompt engineering and tabular data synthesis approaches. Figure 1: The architecture of the proposed AIGT . Firstly, AIGT utilizes the collected pre-trained corpus for pre-training; Then perform fine-tuning training on each downstream table to learn the complex relationships between features; Finally, based on the trained language model model, sample rows can be composed. 2.1 Prompt Engineering Prompt engineering has recently garnered significant attention as a technique for enhancing the performance and usability of AI models, especially in natural language processing (NLP). As described by Lester et al. 2021 , a “prompt” involves providing natural language instructions or commands to guide an AI model in task completion. This approach offers several benefits, including increased model effectiveness, reduced training time and costs, and improved interpretability and controllability. Notable works in this area include those by Jiang et al. 2020 ; Liu et al. 2023 ; Brown et al. 2020 . 2.2 Tabular Data Synthesis Existing approaches for tabular data synthesis can be categorized into four main groups: Probabilistic Models. These models leverage probabilistic techniques to synthesize data. For instance, Gaussian copula models Patki et al. 2016 are effective for continuous variables but not for categorical ones. Conversely, Bayesian networks Zhang et al. 2017 ; Aviñó et al. 2018 are adept at handling categorical data but struggle with continuous variables. Generative Adversarial Networks. Generative Adversarial Networks (GANs) have been widely used to generate tabular data. MedGAN Armanious et al. 2020 and RGAN Esteban et al. 2017 produce healthcare records but face challenges with mixed data types. TableGAN Park et al. 2018 employs Convolutional Neural Networks (CNNs), demonstrating that synthetic data can perform comparably to real data. CTGAN and TVAE Xu et al. 2019a address multimodality with column-specific preprocessing and Variational Gaussian Mixture (VGM) models. Other notable works include Xu et al. 2019b ; Marti 2020 ; Jordon et al. 2018 ; Che et al. 2017 . Diffusion Models. These approaches utilize diffusion models for data synthesis. TabDDPM Kotelnikov et al. 2023 models both categorical and continuous values but encounters difficulties with correlations. SOS Kim et al. 2022 uses Score-based Generative Models (SGMs) to handle imbalanced data, though it lacks the ability to condition on both data types. Language Models. Self-attention models, which have revolutionized NLP Vaswani et al. 2017 , have also been