What happens when generative AI models train recursively on each others’ outputs?

Hung Anh Vu, Galen Reeves, & Emily Wenger · 2025

Read What happens when generative AI models train recursively on each others’ outputs? by Hung Anh Vu, Galen Reeves, & Emily Wenger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21677.txt.

Introduction

Hung Anh Vu, Galen Reeves, and Emily Wenger study AI models that train on each other's outputs in 2025. The web now mixes human and machine text, so cross feeding is likely. Past work tracked models eating their own output. This team tests the cross model case. They pair live experiments with a theory of the loop.

Tests confirm the theory's forecasts. Cross feeding can help by surfacing ideas a model missed in its first training. It can also flatten how alike models perform on shared tasks. The authors weigh both paths for a world of heavy AI use. Knowing which path rules matters for data policy.

Catalog record

CREATORHung Anh Vu, Galen Reeves, & Emily Wenger
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21677

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What happens when generative AI models train recursively on each others’ outputs? Hung Anh Vu, Galen Reeves, & Emily Wenger Abstract The internet serves as a common source of training data for generative AI (genAI) models but is increasingly populated with AI-generated content. This duality raises the possibility that future genAI models may be trained on other models’ generated outputs. Prior work has studied consequences of models training on their own generated outputs, but limited work has considered what happens if models ingest content produced by other models. Given society’s increasing dependence on genAI tools, understanding such data-mediated model interactions is critical. This work provides empirical evidence for how data-mediated interactions might unfold in practice, develops a theoretical model for this interactive training process, and experimentally validates the theory. We find that data-mediated interactions can benefit models by exposing them to novel concepts perhaps missed in original training data, but also can homogenize their performance on shared tasks. 1 Introduction Since the release of ChatGPT in 2022, generative AI (genAI) models have exploded in popularity. Now capable of generating highly realistic text, images, and videos, these models have been widely adopted for various use cases, from creative idea generation ( Notion, 2024 ; Ellis, 2024 ) to healthcare support ( Reddy, 2024 ) to national security settings ( Harding, 2024 ; GAO, 2024 ) . Given the significant uptick in genAI use across numerous industries, this technology is clearly here to stay. Consequently, interrogating potential ways genAI models could evolve—in positive or harmful ways—is critical. With few exceptions, today’s large-scale genAI models are trained on massive datasets sourced from the internet. Widely-accepted scaling laws for model performance say that training on more data aids learning ( Kaplan et al., 2020 ) , and the internet provides a rich, cheap, and ever-evolving source of training data. Although whitepapers for more recent genAI models withhold details about training set composition—potentially due to ongoing litigation about copyright concerns—evidence from earlier whitepapers indicates that scraped data was used to train models like Llama, Gemini, Phi, the GPT series, Claude, and others ( Dubey et al., 2024 ; Achiam et al., 2023 ; Team et al., 2024a ; Jiang et al., 2023 ; coh, 2024 ; Anthropic, 2023 ; Abdin et al., 2024 ) . Beyond privacy and copyright concerns, training on scraped data could have other downsides. Prior work has noted that genAI models trained recursively on their own generated outputs “collapse,” becoming unable to generate meaningful content ( Shumailov et al., 2024 ; Hataya et al., 2023 ; Martínez et al., 2023 ; Alemohammad et al., 2024 ) . This scenario is feasible, since AI-generated content abounds online ( Sun et al., 2025 ) and could be part of future training datasets. However, subsequent work has proposed ways to mitigate collapse via reuse of non-AI-generated data in subsequent training iterations ( Dey & Donoho, 2024 ; Kazdan et al., 2025 ; Dohmatob et al., 2025 ; Feng et al., 2024 ) . Model collapse remains an activate research area ( Schaeffer et al., 2025 ) . Yet, prior work studying the dynamics of model collapse has overlooked another reality: the internet teems with content from many genAI models. Today’s most popular models have millions of users ( Reuters, 2024 ; Handa et al., 2025 ; AI, 2024a ) , who leverage generative AI tools to create online content like web pages and social media posts ( gen, 2022 ) . Recent work showed that up to 40% of content on popular sites like Quora is now AI-generated ( Sun et al., 2025 ) . Given the increasing availability of these models for a variety of public-facing uses ( app, 2024 ; Notion, 2024 ; Ellis, 2024 ) , AI-generated content from many different models will continue to proliferate. The standard practice of training on scraped internet data and the increasing prevalence of AI-generated content online suggest the strong possibility that future generative AI models will be trained on other models’ outputs. Yet, this aspect of model training has received relatively little attention. Given the widespread adoption of generative AI models in critical settings like healthcare and national security, this phenomenon ought to be investigated to ensure models remain helpful and trustworthy. Contributions. To address this need, this work theoretically derives and experimentally evaluates the long-term evolutionary behavior of generative models trained on each other’s data. Specifically, we • Develop a framework, grounded in real-world evidence, describing data-mediated interactions between genAI models. • Derive concise formulas describing the dynamics of interactive training under varied regimes. • Run experiments on large language models to understand how data-mediated interactions affect model performance in practice. Both our theoretical analysis and experiments show that when training with a mixture of real and synthetic data, the implicit interaction between heterogeneous models and datasets can have both positive and negative impacts. At a high level, well known concepts in statistical learning theory anticipate this: recursive training on the same data is bad (e.g., overfitting and model “collapse”) but training on novel data, even if synthetic, can boost performance (e.g., transfer learning). Our experimental results provide concrete evidence that these phenomena can occur simultaneously. 2 Related Work Model collapse is a recently observed phenomenon in large-scale generative text and image models. It referred—in it’s earliest form—to the phenomenon of models performing much worse after generations of training on their own generated outputs ( Shumailov et al., 2024 ; Peterson, 2025 ; Wang et al., 2024 ; Alemohammad et al., 2024 ; Feng et al., 2024 ; Hataya et al., 2023 ; Dohmatob et al., 2025 ; Martínez et al., 2023 ) . Theoretical and empirical results from these works show that models, if trained on generated outputs from their prior versions, slowly degrade in performance as generations progress. One way this manifests is in models forgetting the tails of their original (real) training data, since generated content tends not to contain rare content from the original training data. Training repeatedly on truncated, synthetic data leads the model to forget the richness of its original distribution, resulting in degraded behavior (at best) and total failure (at worst). Mitigating model collapse. Despite the dire predictions of these papers, subsequent work has proposed a simple mitigation strategy: instead of discarding all prior (human-generated) training data, retain some fraction of this while augmenting it with generated data. Numerous works have observed that this choice to augment instead of discard the original training dataset results in a bounded error in future models, avoiding collapse ( Kazdan et al., 2025 ; Gerstgrasser et al., 2024 ; Marchi et al., 2024 ) . Although most of these results were discovered on small models, recent work claims that the observed bound in error \pi^{2}/6 exists for all models ( Dey & Donoho, 2024 ) . Further work ( Schaeffer et al., 2025 ) summarizes current research on collapse. Transfer learning and other model interactions. Significant prior work has studied transfer learning, in which information learned by one model is passed to another, often by reusing the trained weights of a “teacher” model to initialize a “student” model ( Zhuang et al., 2020 ) . Some prior work has further

Read the complete text