Machine-generated text detection prevents language model collapse

George Drayson, Emine Yilmaz and Vasileios Lampos · 2025

Read Machine-generated text detection prevents language model collapse by George Drayson, Emine Yilmaz and Vasileios Lampos (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15654.txt.

Introduction

Drayson, Yilmaz, and Lampos showed in twenty twenty-five that spotting machine text can stop a rot that hits chat models trained on web text. As machine text floods the web, new models may train on it and lose range and truth. They call that loss model collapse.

They first test how decode picks drive that rot across a few turns of new models. Then they train a spot tool and weigh more on text that looks human in the train set. On four models from two lines the fix stopped the rot. It even beat training on pure human text in their tests.

Catalog record

CREATORGeorge Drayson, Emine Yilmaz and Vasileios Lampos
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-15654

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Machine-generated text detection prevents language model collapse George Drayson, Emine Yilmaz and Vasileios Lampos Abstract As Large Language Models (LLMs) become increasingly prevalent, their generated outputs are proliferating across the web, risking a future where machine-generated content dilutes human-authored text. Since online data is the primary resource for LLM pre-training, subsequent models could be trained on an unknown portion of synthetic samples. This could lead to model collapse, a degenerative process whereby LLMs reinforce their own errors, reduce output diversity, and ultimately yield declining performance. In this study, we investigate the impact of decoding strategy on model collapse, analysing the text characteristics at each model generation, the similarity to human references, and the resulting model performance. Using the decoding strategies that lead to the most significant degradation, we evaluate model collapse in a more realistic scenario where the origin of the data (human or synthetic) is unknown. We train a machine-generated text detector and propose an importance resampling approach to prevent model collapse by up-sampling likely human content in the training data. Our method is validated on four LLMs from two model families ( GPT-2 and SmolLM2 ), across a range of model sizes ( 124 124 M to 1.7 1.7 B). We demonstrate that it not only prevents model collapse but also improves performance compared to training on purely human data, underscoring the benefit of synthetic samples and the importance of data curation. 1 1 1 Accepted by EMNLP 2025 (main). Source code: github.com/GeorgeDrayson/model_collapse 1 Introduction Large Language Models (LLMs) can generate high-quality, fluent language across a wide range of applications. A key factor that drives their capabilities is the vast amount of data used to train them, which is predominantly based on text published on the web Wenzek et al. (2020) . The extensive adoption of LLMs will inevitably result in an ever-increasing amount of synthetic data that will co-exist alongside or even dominate human-generated text Dohmatob et al. (2024) , especially within online ecosystems such as social media, news websites, and digital encyclopedias. Hence, there are legitimate concerns as to the effect this might have on future generations of language models trained on a mixed set of human and synthetic corpora. While synthetic data has proven beneficial in controlled scenarios, such as instruction tuning Wang et al. (2023) , distillation Hsieh et al. (2023) , self-improvement Wu et al. (2024) , and reinforcement learning Zhao et al. (2025) , these settings typically involve careful curation and limited reuse. In contrast, our focus is on the long-term effects of uncontrolled accumulation of synthetic content. Several works have attempted to simulate this scenario by recursively training language models on their own generated outputs Shumailov et al. (2023) ; Briesch et al. (2023) ; Guo et al. (2024) . The outcome of this recursive training is referred to as “ model collapse ” Shumailov et al. (2023) , a degenerative process caused by training on synthetic data from previous generations, leading to compounded errors and the convergence to a low variance output distribution. This has been shown to cause performance degradation Alemohammad et al. (2024a) and a drastic loss in diversity Briesch et al. (2023) ; Guo et al. (2024) ; Alemohammad et al. (2024a) . However, an unexplored factor in this recursive training process is the decoding strategy used to generate the synthetic data. Decoding strategies alter the distribution of model outputs, which could, in turn, affect how errors accumulate during recursive training. This work investigates the impact of decoding strategies on model collapse and the characteristics of the data that could be causing this behaviour. Subsequently, we explore the scenario where the training data is mixed (human and synthetic) in an unknown proportion, akin to training on web-crawled data. We propose a method for preventing model collapse by a guided resampling of the training data using the confidence estimates from a machine-generated text detector. Our method is motivated by prior work Bertrand et al. (2024) ; Alemohammad et al. (2024a) , which highlighted that when the proportion of human data in the training set is sufficient, model collapse can be prevented. Our contributions can be summarised as follows: (a) we evaluate model collapse from three perspectives: task performance, model generation quality, and semantic similarity to human text, (b) we show that model collapse is significantly affected by the choice of decoding strategy, demonstrating large discrepancies in performance and data quality, (c) we train a machine-generated text detector to provide calibrated confidence estimates for the origin of the training samples, (d) we propose a method that uses the detector’s outputs to prevent model collapse through a guided resampling of the training data, and (e) we substantiate our hypotheses by conducting experiments on four LLMs from two model families across a range of decoding strategies. 2 Prior work on model collapse Model collapse is a degenerative process in which models recursively trained on generational data exhibit a drop in performance compared to a model trained on the original human distribution Shumailov et al. (2023) . In the early stages of model collapse, information is lost at the tails of the distribution. Eventually the output distribution converges to a point estimate with very little variance, resulting in a model that cannot be restored back to the original generation trained on human data. This effect can also be viewed as a change to neural scaling laws, in which there reaches a point where training on additional synthetic samples does not improve model performance, and thus learning plateaus Dohmatob et al. (2024) . It has been argued that the two causes for this behaviour are finite sampling error leading to information being lost at the tails of the distribution, and functional approximation error introducing non-zero likelihoods outside of the support of the original distribution Shumailov et al. (2023) . Additionally, Dohmatob et al. (2024) theorised that the choice of generation algorithm is another contributing factor to model collapse. However, this has not been empirically evaluated in the case of LLMs, where decoding strategies that modify and sample from the model’s output distribution could also have a significant impact on how errors are propagated across generations. Currently, model collapse in LLMs has been studied with a fixed decoding strategy, and model degradation has been mostly assessed using task performance metrics such as perplexity Shumailov et al. (2024) and test loss Gerstgrasser et al. (2024) . Interestingly, Guo et al. (2024) also evaluate the diversity of the generated text. In our study, we have chosen to study model collapse across three perspectives: the quality of the generated text (including diversity and readability), its similarity to human text, and the downstream task performance. Recent studies have explored methods for mitigating model collapse, such as using synthetic samples as negative guidance in the image domain Alemohammad et al. (2024b) , pruning samples based on high perplexity Feng et al. (2024) , token-level editing Zhu et al. (2025) or filtering low-quality samples Zhang et al. (2024) . Bertrand et al. (2024) and Alemohammad et al. (2024a) show that when a high enough proportion of human data is added at each iteration, model collapse in diffusion models can be avoided. In the computational linguistics domain, Gerstgrasser et al. (2024)

Read the complete text