Generating Synthetic Invoices via Layout-Preserving Content Replacement
Bevin V, Ananthakrishnan P V, Ragesh KR, Sanjay M, Vineeth S, Bibin Wilson BEO AI · 2025
Read Generating Synthetic Invoices via Layout-Preserving Content Replacement by Bevin V, Ananthakrishnan P V, Ragesh KR, Sanjay M, Vineeth S, Bibin Wilson BEO AI (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-03754.txt.
Introduction
Fake invoices that look real can train bill-reading software without leaking private bills. Bevin V and co-authors built a pipeline for making them in work from twenty twenty-five. Real bills are scarce for training, since privacy rules and hand-label costs block sharing.
The pipeline reads a source bill's text and layout, then swaps set fields for fresh, lifelike values from a large language model. A paint-over step wipes the old text and sets the new text in the same place and style. Each fake bill comes with a matched data file that lists its fields. The team pitches the flow as a way to grow small, private bill sets into large training pools.
Catalog record
| CREATOR | Bevin V, Ananthakrishnan P V, Ragesh KR, Sanjay M, Vineeth S, Bibin Wilson BEO AI |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 18 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-03754
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generating Synthetic Invoices via Layout-Preserving Content Replacement Thanks: The code for this project is available at: https://github.com/BevinV/Synthetic_Invoice_Generation Bevin V, Ananthakrishnan P V, Ragesh KR, Sanjay M, Vineeth S, Bibin Wilson BEO AI Abstract . The performance of machine learning models for automated invoice processing is critically dependent on large-scale, diverse datasets. However, the acquisition of such datasets is often constrained by privacy regulations and the high cost of manual annotation. To address this, we present a novel pipeline for generating high-fidelity, synthetic invoice documents and their corresponding structured data. Our method first utilizes Optical Character Recognition (OCR) to extract the text content and precise spatial layout from a source invoice. Select data fields are then replaced with contextually realistic, synthetic content generated by a large language model (LLM). Finally, we employ an inpainting technique to erase the original text from the image and render the new, synthetic text in its place, preserving the exact layout and font characteristics. This process yields a pair of outputs: a visually realistic new invoice image and a perfectly aligned structured data file (JSON) reflecting the synthetic content. Our approach provides a scalable and automated solution to amplify small, private datasets, enabling the creation of large, varied corpora for training more robust and accurate document intelligence models. 1. Introduction The digital transformation of financial workflows has made the automated processing of invoices a cornerstone of modern business operations. Machine learning (ML), particularly deep learning models, has shown significant promise in accurately extracting structured data from these documents, thereby reducing manual labor, minimizing errors, and accelerating payment cycles. However, the performance of these sophisticated models is fundamentally bottlenecked by the availability of large-scale, diverse, and meticulously labeled training data. The acquisition of such datasets is fraught with challenges. Real-world invoices contain sensitive financial and personal information, making them subject to strict privacy regulations like GDPR. Furthermore, the manual process of collecting, scanning, and annotating thousands of invoices is prohibitively expensive and time-consuming. To overcome this data scarcity bottleneck, we propose a novel pipeline for generating high-fidelity, synthetic invoice documents paired with their corresponding structured data. Our method leverages a hybrid approach, combining the strengths of Optical Character Recognition (OCR) for layout preservation, Large Language Models (LLMs) for contextual data generation, and computer vision techniques for seamless image composition. The system first deconstructs a source invoice into its textual and spatial components, then intelligently replaces key information with realistic, synthetic content, and finally re-renders a new, visually coherent invoice. The key contributions of this work are: (1) A complete, end-to-end pipeline for generating paired image-JSON invoice data. (2) A method for leveraging LLMs to produce contextually aware, anonymized data that maintains document plausibility. (3) A robust text-rendering technique that preserves the original document’s layout and font scale. This paper will first review related work in synthetic data generation, then detail our methodology, present the results of our generation process, and finally discuss the implications and future directions of this research. 2. Related Work The creation of training data for document intelligence models has evolved significantly. We categorize prior work into three main areas: classical augmentation, generative modeling, and layout-aware language models. 2.1. Classical and Content-Agnostic Augmentation The foundational approach to expanding datasets is data augmentation. For image-based tasks, this involves applying transformations that preserve class labels. A comprehensive survey by Shorten and Khoshgoftaar [ 1 ] details common techniques like rotation, scaling, noise injection, and elastic distortions. While effective for making models robust to visual noise and scanning artifacts, these methods are content-agnostic; they do not alter the semantic information within the document. More advanced techniques like ”Cutout” [ 2 ] , which randomly erases rectangular regions of an image, force models to learn from less complete data but still do not modify the underlying text. These methods fail to enrich the variety of textual entities (e.g., company names, line items) that a document extraction model must learn. 2.2. Generative Models for Document Synthesis To generate novel content, researchers have turned to generative models, most notably Generative Adversarial Networks (GANs). Early applications focused on generating realistic images, with seminal work by Goodfellow et al. [ 3 ] laying the groundwork. Subsequent work has attempted to adapt GANs for document synthesis. For example, LayoutGAN [ 4 ] generates layouts by modeling the spatial relationships between document elements, but it does not generate the textual content itself. Conversely, models like MC-GAN [ 5 ] can generate images conditioned on text, but often struggle with the legibility and precise placement required for complex documents. The primary drawback of most GAN-based approaches for document AI is the difficulty in producing coherent, readable text and ensuring that the visual output perfectly matches any corresponding structured data label. Another line of work involves programmatic generation. The work by C. L. Tan et al. [ 6 ] shows an early approach using templates and rendering engines to create synthetic documents for OCR training. While this guarantees perfect data labels, such template-based systems often produce datasets with limited layout diversity, which may not generalize well to the variance of real-world documents. 2.3. Layout-Aware Language Models The most recent and successful approaches combine the power of Large Language Models (LLMs) with an understanding of document layout. The LayoutLM series from Microsoft represents a major breakthrough in this area. LayoutLM [ 7 ] was the first model to jointly pre-train text and layout information by incorporating 2D position embeddings. LayoutLMv2 [ 8 ] improved upon this by adding visual feature embeddings directly from the image, creating a truly multimodal model. The culmination, LayoutLMv3 [ 9 ] , simplified the architecture and used a unified text-image masking objective, setting a new state-of-the-art on many document AI benchmarks. Following this trend, other models like ”Donut” (Document Understanding Transformer) [ 10 ] proposed an end-to-end approach that bypasses traditional OCR, using a Transformer encoder-decoder to directly generate a structured data output from a document image. While incredibly powerful for information extraction, these models are primarily discriminative; they are designed to *read* documents, not *generate* new ones. The gap in the literature, therefore, lies in the generation of new, paired data. While generative models struggle with data-label consistency and layout-aware models are built for extraction, a practical pipeline for data *augmentation* through synthesis is still needed. Our work addresses this specific gap. By anchoring our generation process on the layout of a real document, we inherit its visual realism. By leveraging an LLM for its core competency—contextual text generation—and then re-rendering, we create a new, visually plausible document with a perfectly corresponding structured data label, a critical requirement for training the next generation of supervised document intelligence models. 3. Methodology To address the need for paired image and structured data, we developed a multi-stage pipeline that transforms a single seed invoice into a new, synthetic, and visually plausible counterpart. Our approach ensures that the layout and style of the original document are preserved while the textual content is replaced. The entire process, illustrated in Figure 1 , is composed of three core stages: OCR-based Content and Layout Extraction, LLM-Powered Content Generation, and finally, Image Inpainting and Text Rendering. Figure 1 . The end-to-end pipeline of our synthetic invoice generation system. The