Analyzing and Mitigating Model Collapse in Rectified Flow Models
Huminhao Zhu, Fangyikang Wang, Tianyu Ding, Qing Qu, Zhihui Zhu · 2024
Read Analyzing and Mitigating Model Collapse in Rectified Flow Models by Huminhao Zhu, Fangyikang Wang, Tianyu Ding, Qing Qu, Zhihui Zhu (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08175.txt.
Introduction
Zhu and four co-authors worked in twenty twenty-four on rot that hits flow models when they train on their own made data. Flow tools straighten their path by rounds of self-made samples, which can lose quality in time. Past work on that rot was mostly by eye or on plain tasks.
They prove loss for a class of clean-up tools trained on their own out-put in rounds. Flow tools show the same rot in their tests and in the proof sketch. Mixing real data in stops the rot in that proof frame. Their new reflow adds real data by a back flow trick and holds quality with few steps.
Catalog record
| CREATOR | Huminhao Zhu, Fangyikang Wang, Tianyu Ding, Qing Qu, Zhihui Zhu |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08175
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Analyzing and Mitigating Model Collapse in Rectified Flow Models Huminhao Zhu, Fangyikang Wang, Tianyu Ding, Qing Qu, Zhihui Zhu Abstract Training with synthetic data is becoming increasingly inevitable as synthetic content proliferates across the web, driven by the remarkable performance of recent deep generative models. This reliance on synthetic data can also be intentional, as seen in Rectified Flow models, whose Reflow method iteratively uses self-generated data to straighten the flow and improve sampling efficiency. However, recent studies have shown that repeatedly training on self-generated samples can lead to model collapse (MC) , where performance degrades over time. Despite this, most recent work on MC either focuses on empirical observations or analyzes regression problems and maximum likelihood objectives, leaving a rigorous theoretical analysis of reflow methods unexplored. In this paper, we aim to fill this gap by providing both theoretical analysis and practical solutions for addressing MC in diffusion/flow models. We begin by studying Denoising Autoencoders (DAEs) and prove performance degradation when DAEs are iteratively trained on their own outputs. To the best of our knowledge, we are the first to rigorously analyze model collapse in DAEs and, by extension, in diffusion models and Rectified Flow. Our analysis and experiments demonstrate that rectified flow also suffers from MC, leading to potential performance degradation in each reflow step. Additionally, we prove that incorporating real data can prevent MC during recursive DAE training, supporting the recent trend of using real data as an effective approach for mitigating MC. Building on these insights, we propose a novel Real-data Augmented Reflow (RA Reflow) and a series of improved variants, which seamlessly integrate real data into Reflow training by leveraging reverse flow. Empirical evaluations on standard image benchmarks confirm that RA Reflow effectively mitigates model collapse, preserving high-quality sample generation even with fewer sampling steps. Figure 1 : Two Scenarios for Studying Model Collapse. Top : Rectified Flow uses the Reflow method, iteratively relying on self-generated data to straighten the flow and improve sampling efficiency. Left : Model collapse occurs when models are repeatedly trained on their own outputs, progressively degrading performance. Right : Adding real data at each iteration mitigates collapse by preserving sample quality. Bottom : Visualization of correction streams after 10 iterations. The baseline lacks color and produces blurry, mixed outputs, whereas our approach maintains clarity and fidelity. 1 Introduction Generative modeling aims to produce synthetic data that is indistinguishable from genuine data distributions. While deep generative models have achieved remarkable success across images, audio, and text ( Rombach et al. 2022 ; Ramesh et al. 2022 ; Chen et al. 2020 ; Achiam et al. 2023 ; Touvron et al. 2023 ) , the increasing volume of synthetic data introduces significant challenges. As shown in Figure 1 , a critical issue is model collapse (MC) , where generative models trained iteratively on their own outputs progressively degrade in performance ( Shumailov et al. 2023 ) . This degradation not only affects the quality of generated data but also poses risks when synthetic data is inadvertently included in training datasets, leading to self-consuming training loops ( Alemohammad et al. 2023 ; Briesch et al. 2023 ) . Most recent work on model collapse centers on empirical observations ( Shumailov et al. 2023 ; Alemohammad et al. 2023 ) , and most of the theoretical study addresses regression problems ( Fu et al. 2024 ; Dohmatob et al. 2024a ; Gerstgrasser et al. 2024 ) or maximum likelihood objectives ( Bertrand et al. 2023 ) , leaving a rigorous theoretical analysis of diffusion/flow models unexplored. Simulation-free models and their variants—such as diffusion models ( Song & Ermon 2019 ; Song et al. 2020b ; Ho et al. 2020 ) , flow matching ( Lipman et al. 2022 ; Pooladian et al. 2023 ; Tong et al. 2023 ) , and rectified flow ( Liu et al. 2022 ) —have drawn increasing attention. Among these models, rectified flow stands out due to its rapid development and extensive foundational and large-scale work ( Esser et al. 2024 ) . Unlike conventional diffusion models, Rectified Flow employs a reflow procedure that iteratively uses self-generated data as training data to straighten the flow and improve sampling efficiency. While Rectified Flow can reduce the number of sampling steps, the quality of generated data tends to decline, particularly with a higher number of reflow steps. However, there is a lack of thorough analysis regarding this performance drop; prior studies on Rectified Flow have primarily focused on scaling up the model or applying distillation techniques ( Lee et al. 2024a ; Liu et al. 2023 ; Esser et al. 2024 ) . As a result, the observed decline in reflow’s generation quality is often attributed to error accumulation. Rectified flow and model collapse share a similar setting—both involve retraining models with self-generated data—but deliver conflicting messages, as rectified flow produces more efficient models while model collapse highlights model failure. In this paper, we aim to resolve this conflict by studying the following questions: Does rectified flow suffer from model collapse? If so, how can model collapse be prevented while leveraging the benefits of reflow? Contribution This paper investigates model collapse in diffusion models by (1) demonstrating that rectified flow also suffers from model collapse, (2) providing a theoretical analysis using a denoising autoencoder, and (3) proposing a novel method to prevent model collapse. Our contributions can be summarized as follows. • Empirical verification of model collapse. Complementing existing works that empirically demonstrate diffusion/flow models suffer from model collapse in self-consuming training loops ( Alemohammad et al. 2023 ) , we experimentally validate that the reflow training method also leads to a decline in model performance on both synthetic and real tasks. See Section 2 for the detailed description of related work. • Theoretical analysis of model collapse. We then delve into a theoretical analysis of MC. To facilitate the analysis, we study Denoising Autoencoders (DAEs) and uncover the underlying mechanisms that lead to performance degradation when DAEs are trained iteratively on their own outputs. To the best of our knowledge, we are the first to rigorously analyze model collapse in DAEs and, by extension, in diffusion models and Rectified Flow. Our analysis validates the causes of performance degradation arising from iterative training on self-generated data in Rectified Flow. In addition, we prove that incorporating real data can prevent model collapse during the recursive training of DAEs, supporting the recent development of using real data as an effective approach for mitigating model collapse ( Bertrand et al. 2023 ) . • RA Reflow: Real-data Augmented Rectified flow Following this analysis, a natural approach for preventing model collapse is by incorporating real data. However, the absence of direct noise-image pairs poses a significant challenge in directly using real data with rectified flow. Our proposed RA Reflow addresses this issue with a novel approach of incorporating real data by leveraging reverse processes. With balanced synthetic and real data, RA Reflow can straighten the flow trajectories effectively while maintaining training stability. We also propose an online RA Reflow that can greatly reduce the storage budget of naive reflow while progressively straightening the flow We validate our methods through extensive experiments on standard image datasets.