Large-Scale Universal Defect Generation: Foundation Models and Datasets
Yuanting Fan, Jun Liu, Bin-Bin Gao, Xiaochen Chen, Yuhuan Lin, Zhewei Dai, Jiawei Zhan, Chengjie Wang · 2026
Read Large-Scale Universal Defect Generation: Foundation Models and Datasets by Yuanting Fan, Jun Liu, Bin-Bin Gao, Xiaochen Chen, Yuhuan Lin, Zhewei Dai, Jiawei Zhan, Chengjie Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08915.txt.
Introduction
Fan and seven colleagues build a large foundation model for creating defect images, the flawed examples used to train inspection systems. Earlier few-shot methods overfit narrow defect types because paired examples of normal and damaged parts were scarce, and varied defect sizes and shapes made the problem worse. Their UDG collection holds 300,000 normal, damaged, mask, and caption quadruples across many domains. Their UniDG model trains in two stages, first for variety and then for consistency, and it beats prior methods on standard benchmarks and on downstream detection tasks.
Catalog record
| CREATOR | Yuanting Fan, Jun Liu, Bin-Bin Gao, Xiaochen Chen, Yuhuan Lin, Zhewei Dai, Jiawei Zhan, Chengjie Wang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 89626 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.08915
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large-Scale Universal Defect Generation: Foundation Models and Datasets Yuanting Fan, Jun Liu, Bin-Bin Gao, Xiaochen Chen, Yuhuan Lin, Zhewei Dai, Jiawei Zhan, Chengjie Wang Abstract Existing defect/anomaly generation methods often rely on few-shot learning, which overfits to specific defect categories due to the lack of large-scale paired defect editing data. This issue is aggravated by substantial variations in defect scale and morphology, resulting in limited generalization, degraded realism, and category consistency. We address these challenges by introducing UDG , a large-scale dataset of 300K normal-abnormal-mask-caption quadruplets spanning diverse domains, and by presenting UniDG , a universal defect generation foundation model that supports both reference-based defect generation and text instruction-based defect editing without per-category fine-tuning. UniDG performs Defect-Context Editing via adaptive defect cropping and structured diptych input format, and fuses reference and target conditions through MM-DiT multimodal attention. A two-stage training strategy, Diversity-SFT followed by Consistency-RFT , further improves diversity while enhancing realism and reference consistency. Extensive experiments on MVTec-AD and VisA show that UniDG outperforms prior few-shot anomaly generation and image insertion/editing baselines in synthesis quality and downstream single- and multi-class anomaly detection/localization. Code will be available at https://github.com/RetoFan233/UniDG . † † affiliationnotice: Equal contribution 1 Introduction Anomaly detection is crucial in industrial inspection, medical diagnosis, and many safety-critical applications. However, abnormal samples are inherently scarce in real-world deployments, making it difficult to train reliable supervised detectors and localizers. Consequently, anomaly/defect generation has emerged as a practical way to synthesize abnormal samples and alleviate the data bottleneck for downstream detection and localization. Existing anomaly generation methods can be broadly grouped into two paradigms: (1) zero-shot approaches that edit normal images using textual descriptions with pre-trained generative models, and (2) few-shot approaches that condition on a small set of real abnormal samples to synthesize additional anomalies with similar appearance statistics. Figure 1: The framework of existing anomaly generation methods. The proposed UniDG maintains straightforward inference and fully open-sourced framework. Despite encouraging progress, both paradigms still struggle to produce defects that are realistic and visually consistent with the target scene. As illustrated in Fig. 1 , prior methods may generate highly anomalous regions, yet often fail to yield sufficiently realistic and category consistent defects . We attribute these limitations to two key factors. First, the field lacks large-scale paired defect editing data, so many methods resort to few-shot learning that overfits to a specific defect category and generalizes poorly across scenarios or objects. Second, defect appearance is inherently ambiguous, with large variations in scale and morphology; learning from only few-shot samples often fails to capture the underlying distribution, leading to degraded realism and reference consistency. In addition, pre-trained universal text-to-image or image editing models provide limited defect-specific prior knowledge, and thus struggle to precisely synthesize rare defects even with detailed text prompts (see Fig. 1 , even the most advanced Gemini-3-Pro-Image ( 6 ) remains limited performance in rare defects). These limitations also restrict cross-category defect synthesis ( e.g. , transferring similar defects across different objects). Recent work such as Anomagic ( 18 ) partially expands category coverage by retrieving relevant abnormal exemplars with an external MLLM, but still depends on additional models and a large offline repository, which makes the pipeline costly and hard to scale. To overcome the above issues, we propose UniDG , a universal defect generation foundation model that transfers defects from a reference image to a specified target region in a flexible and interactive manner. Crucially, UniDG is trained once on our newly constructed large-scale UDG dataset (300K normal-abnormal-mask-caption quadruplets spanning diverse domains), enabling strong generalization without per-category fine-tuning. Technically, UniDG adopts a Defect-Context Editing strategy that combines adaptive defect cropping with a structured diptych input, and fuses reference and target conditions via MM-DiT multimodal attention. Furthermore, we design a two-stage training recipe, Diversity-SFT and Consistency-RFT , to improve both synthesis diversity and defect realism/consistency, leading to consistent gains in generation quality as well as downstream single- and multi-class anomaly detection and localization. The key contributions are as follows: • We introduce UDG , a large-scale dataset of 300K normal-abnormal-mask-caption quadruplets spanning diverse domains, which supports research on multimodal anomaly detection and defect generation. • We present UniDG , a universal defect generation foundation model based on MM-DiT and Defect-Context Editing, enabling realistic and consistent defect synthesis that generalizes to unseen scenarios. • We propose a two-stage training strategy ( Diversity-SFT and Consistency-RFT ) together with diverse reward models to further improve defect realism and reference consistency. • Extensive experiments on MVTec-AD and VisA outperform prior few-shot anomaly generation and image insertion/editing baselines in synthesis quality and downstream anomaly detection/localization performance without per-category fine-tuning. 2 Related Work Few-shot Anomaly Generation Few-shot anomaly generation methods leverage a small number of abnormal samples during training to synthesize additional anomalies with similar visual characteristics, thereby alleviating data scarcity for downstream anomaly detection. Early works such as DFMGAN ( 8 ) employ GANs to produce high-fidelity defects, while more recent approaches predominantly adopt diffusion-based frameworks. AnomalyDiffusion ( 16 ) uses text inversion strategy to model anomaly semantics and mask distributions; AnoGen ( 14 ) and DefectFill ( 29 ) formulate anomaly synthesis as image inpainting to improve mask accuracy; and DualAnoDiff ( 19 ) improves synthesis quality by decoupling defect foreground from background modeling. SeaS ( 7 ) further binds anomaly attributes to learnable tokens, enabling a single model to cover multiple anomaly types. However, many of these methods still require training separate models for individual defect categories, which limits generalization to novel categories and increases training and storage costs. Anomagic ( 18 ) constructs multi-category anomaly datasets and retrieves relevant abnormal exemplars via an advanced MLLM during inference, partially expanding category coverage, but it relies on an additional online MLLM and a large anomaly repository, making the pipeline difficult to scale. Reference-Based Image Generation Reference-based image generation, also referred to as subject-driven generation, takes a reference image containing the subject as input and aims to generate a target image that preserves subject-specific details such as identity and appearance. These methods can be broadly categorized into two paradigms: test-time fine-tuning approaches ( e.g. , DreamBooth ( 28 ) and Textual Inversion ( 12 ) , which are commonly used in FSAG), and training-based approaches ( e.g. , ControlNet ( 35 ) and IP-Adapter ( 34 ) ). The former are less data-hungry and can adapt with only a few images, but their cross-category generalization is limited, making them less suitable for building universal defect generation foundation models. The latter typically require large-scale pre-training datasets to achieve high-fidelity synthesis in general domains; for example, AnyDoor ( 5 ) , FreeEdit ( 15 ) , and InsertAnything ( 30 ) construct paired data for subject-driven generation and improve both synthesis quality and semantic consistency. However, such models and datasets are primarily developed for natural images and can suffer from domain gaps when transferred to defect generation, due to distributional differences in defect scale, identity, and other factors. Following prior successes in natural-image generation, we construct the first large-scale defect-scenario dataset and build the first foundation model for universal defect generation, enabling robust cross-category synthesis and promoting high-quality defect generation in downstream applications. 3 UDG Dataset 3.1 Construction Pipeline We construct a novel multi-agent