LLM-Generated Fake News Induces Truth Decay in News Ecosystem: A Case Study on Neural News Recommendation
Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, Danding Wang · 2025
Read LLM-Generated Fake News Induces Truth Decay in News Ecosystem: A Case Study on Neural News Recommendation by Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, Danding Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20013.txt.
Introduction
Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, and Danding Wang study what happens when machine-written fake news floods a news recommendation system, in a research paper from 2025. Detectors struggle with this fake news one item at a time. Its effect on the whole news ecosystem was less explored.
The team built a simulation with about fifty-six thousand generated news items of varied types. As fake items entered the rankings, real news slowly lost its edge, a pattern they call truth decay. They trace it to familiarity: text that reads smoothly tends to rank higher, whatever its truth. The authors discuss countermeasures and urge platforms to act early.
Catalog record
| CREATOR | Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, Danding Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-20013
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLM-Generated Fake News Induces Truth Decay in News Ecosystem: A Case Study on Neural News Recommendation Beizhe Hu, Qiang Sheng, Juan Cao, Yang Li, Danding Wang Abstract Online fake news moderation now faces a new challenge brought by the malicious use of large language models (LLMs) in fake news production. Though existing works have shown LLM-generated fake news is hard to detect from an individual aspect, it remains underexplored how its large-scale release will impact the news ecosystem. In this study, we develop a simulation pipeline and a dataset with ~56k generated news of diverse types to investigate the effects of LLM-generated fake news within neural news recommendation systems. Our findings expose a truth decay phenomenon, where real news is gradually losing its advantageous position in news ranking against fake news as LLM-generated news is involved in news recommendation. We further provide an explanation about why truth decay occurs from a familiarity perspective and show the positive correlation between perplexity and news ranking. Finally, we discuss the threats of LLM-generated fake news and provide possible countermeasures. We urge stakeholders to address this emerging challenge to preserve the integrity of news ecosystems. = Date : April 28, 2025 = Project : https://beanandrew.github.io/projects/TruthDecay = Venue : ACM SIGIR 2025 ( https://doi.org/10.1145/3726302.3730027 ) \emails { hubeizhe21s , shengqiang18z , caojuan , liyang23s , wangdanding }@ict.ac.cn Figure 1: Illustration of Truth Decay phenomenon, where real news gradually loses its advantageous position in rankings against fake news as LLM-generated news penetrates. 1 Introduction The proliferation of online fake news has been a long-standing problem across the world, posing significant threats across various critical domains such as politics Grinberg et al. 2019 and public health Caceres et al. 2022 . Fake news moderation has been a necessity for online information systems like social media Giansiracusa 2021 , search engines BBC News 2016 , and recommendation systems Fernandez and Bellogín 2020 . To combat the fake news spread, stakeholders have made much effort in developing automatic detection techniques Hu et al. 2022 , Bu et al. 2023 , Zhu et al. 2022 , monitoring and mitigating fake news spread Saxena et al. 2022 , providing reporting and debunking systems Li et al. 2021 , which effectively reduce fake news threats to the real world. Unfortunately, the breakthrough in text generation with large language models (LLMs) has largely complicated the situation Zhao et al. 2024 , Chen and Shu 2024b . Because of their excellent text generation capability, malicious actors have equipped themselves with LLMs to cheaply and efficiently produce fake news Jiang et al. 2024 . Recent coverage discloses that over 1k unreliable AI-generated news websites were identified Sadeghi et al. 2024 globally, and a news faking gang using LLMs has produced 28k fake news items, surprisingly resulting in 2.7B views Zhang 2024 . As LLM-generated news rushes into online news channels, the news ecosystem, like recommendation systems, faces an almost unavoidable news data distribution shift and credibility crisis. A crucial but still underexplored question arises: How does LLM-generated fake news impact the neural news recommendation system? To investigate the impact on news recommendation systems, we first tackle the unavailability of existing data and perform Data & Environment Preparation . By repurposing a fake news detection dataset GossipCop, we construct a dataset including news items, veracity labels, and user-news interaction records. To systematically analyze LLM-generated news, we first propose a news taxonomy for levels of LLMs’ news generation modes, given the degree of LLM involvement in generation. Then we prompt LLMs to generate news of different types with the assigned level as guidance and human news as reference. With this prepared data, we explore How Generated News Affects the Neural News Recommendation System . We initially test the system recommendation preference between human-written real and fake news, finding that overall, real news ranks higher than fake news. Subsequently, we examine the effects of LLM-generated news on the news recommendation system by simulating its intrusion into different components, including the news candidate list, user-news interaction history, and training set of news recommendation models. As illustrated in Figure 1 , we find that LLM-generated news’s involvement leads to the “Truth Decay” phenomenon, where real news is gradually losing its advantageous position against fake news and fake news (especially the LLM-generated) could be even more preferable and earns higher ranks than real news. Finally, we provide an analysis about Why Truth Decay Occurs from a familiarity perspective and reveal the perplexity’s high correlation with the changes of ranking advantages. We further discuss the potential implications of the truth decay phenomenon and possible countermeasures. Our main contributions include: • New Perspective: To the best of our knowledge, we are the first that investigate the impact of LLM-generated fake news on neural news recommendation systems, which is important for credibility analysis of information systems in the LLM era. • New Findings: We observe a Truth Decay phenomenon in neural news recommendation systems, which reflects that the involvement of LLM-generated news leads to relative advantages of real news over fake news being diminished and the overall deterioration, sounding a new alarm about such potential harms. • New Taxonomy & Data: We propose a new taxonomy for levels of LLM-based news generation modes given the degree of LLM involvement in generation, which facilitates systematic description and analysis of possible scenarios of LLM-based news generation. A human-LLM-mixed news recommendation dataset containing ~75k news items that follow this taxonomy is constructed. 2 Preliminaries 2.1 Personalized News Recommendation Personalized news recommendation aims at actively displaying news items that might be of a target user’s interest according to its news interaction history, mostly applied in news distribution channels like Google News 1 1 1 https://news.google.com/foryou , MSN 2 2 2 https://microsoftnews.msn.com/ , Toutiao 3 3 3 https://en.wikipedia.org/wiki/Toutiao , etc. Given a user’s news interaction history ( e.g., viewing or reposting records), the news recommendation model predicts the next news item the user will interact with. Like many other recommendation systems, news recommendation could work with collaborative filtering ( Konstan et al. 1997 , Wang and Blei 2011 ) , which predicts the next item according to other similar users’ news interaction history. However, it faces a serious challenge of cold-start situations for news recommendation because news pools are intrinsically updated fast to ensure timeliness, causing interaction records on outdated news items to be hardly useful. Therefore, content-based methods, most of which use neural networks for content representation ( Wu et al. 2019 , An et al. 2019 , Wu et al. 2021 , Li et al. 2022 ) , are often considered effective for addressing the unique challenges of news recommendation systems. Figure 2 illustrates how a typical content-based news recommendation model is trained and then serves for ranking candidate news items. During the training phase, the model learns to predict whether an item in the candidate news will be interacted with based on the recently interacted news list. For inference, the news recommendation model is frozen, and candidate news items will be scored and ranked by referring to a user’s interaction history. The ranked list will finally determine the display order of candidate news items to the target user. Figure 2: Illustration of a typical news recommendation