Wikipedia Contributions in the Wake of ChatGPT Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3701716.37

Liang Lyu, James Siderius, Hannah Li, Daron Acemoglu, Daniel Huttenlocher, Asuman Ozdaglar · 2025

Read Wikipedia Contributions in the Wake of ChatGPT Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3701716.37 by Liang Lyu, James Siderius, Hannah Li, Daron Acemoglu, Daniel Huttenlocher, Asuman Ozdaglar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00757.txt.

Introduction

Liang Lyu and five co-authors measure how ChatGPT changed Wikipedia activity in a 2025 paper. When a chatbot can answer and draft similar content, volunteers may edit and read the encyclopedia less. The effect should hit hardest where article content overlaps the chatbot's strengths.

A difference-in-differences design compares similar and dissimilar articles around the chatbot's late 2022 launch. New, popular articles with high overlap saw steeper drops in editing and views. The baseline articles faded less over the same span. The authors read this as uneven substitution rather than a blanket decline. Human knowledge projects may lose effort exactly where machines seem good enough, a pattern with long-term costs.

Catalog record

CREATORLiang Lyu, James Siderius, Hannah Li, Daron Acemoglu, Daniel Huttenlocher, Asuman Ozdaglar
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00757

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Wikipedia Contributions in the Wake of ChatGPT Conference: Companion Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Companion Proceedings of the ACM Web Conference 2025 (WWW Companion ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3701716.3715543 ISBN: 979-8-4007-1331-6/2025/04 CCS: Information systems Collaborative and social computing systems and tools; Information retrieval; Web mining CCS: Human-centered computing Human-computer interaction (HCI); User studies CCS: Computing methodologies Natural language processing; Language models Liang Lyu, James Siderius, Hannah Li, Daron Acemoglu, Daniel Huttenlocher, Asuman Ozdaglar Abstract . How has Wikipedia activity changed for articles with content similar to ChatGPT following its introduction? We estimate the impact using differences-in-differences models, with dissimilar Wikipedia articles as a baseline for comparison, to examine how changes in voluntary knowledge contributions and information-seeking behavior differ by article content. Our analysis reveals that newly created, popular articles whose content overlaps with ChatGPT 3.5 saw a greater decline in editing and viewership after the November 2022 launch of ChatGPT than dissimilar articles did. These findings indicate heterogeneous substitution effects, where users selectively engage less with existing platforms when AI provides comparable content. This points to potential uneven impacts on the future of human-driven online knowledge contributions. Figure 1 . Our Diff-in-Diff Methodology. Measuring Wikipedia activity before and after the introduction of ChatGPT for Wikipedia articles that rank similarly and dissimilarly to the same information generated by ChatGPT. † † cc-license: by 1. Introduction Generative AI tools like ChatGPT have fundamentally reshaped how information is disseminated and consumed online. Increasingly, users are turning to large language model (LLM) chatbots and AI-generated search summaries for information ( ( Dam et al. 2024 ) ; ( Glickman and Zhang 2024 ) ). This shift in behavior can create spillover effects on other platforms, as users substitute away from traditional crowdsourced knowledge sources such as Reddit ( ( Burtch et al. 2024 ) ) and Stack Overflow ( ( del Rio-Chanona et al. 2023 ) ; ( Shan and Qiu 2023 ) ). Importantly, these changes in user behavior can have downstream impacts not only on these traditional platforms, but also on future generative AI models. One concern is that if users substitute away from these platforms and viewership drops, then contributors may also be less inclined to create content and knowledge, which AI models are trained on. A decline in their quantity and quality can result in “model collapse” for future AI models ( Shumailov et al. 2024 ) . In this study, we focus on Wikipedia, a knowledge sharing platform with unique concerns and learning opportunities regarding the potential negative effects of LLMs. Wikipedia’s scope extends far beyond technical, directed questions, instead offering an encyclopedic repository of historical, cultural and scientific topics upon which many other sources draw. The platform also hosts two distinct groups of users – viewers and editors – who could differ in their responses to LLMs and their downstream implications. Several factors about Wikipedia necessitate our differences-in-differences (DiD) strategy, in contrast to the interrupted time series analysis that is often used in similar work on Reddit ( ( Burtch et al. 2024 ) ), Stack Overflow ( ( del Rio-Chanona et al. 2023 ) ; ( Shan and Qiu 2023 ) ), and Wikipedia itself ( ( Reeves et al. 2024 ) ). In addition to having a broader scope of topics, Wikipedia allows for more diverse user incentives than analogous platforms: viewers exhibit both shallow and deep information needs ( Singer et al. 2017 ) , while contributors are driven by both intrinsic ( Yang and Lai 2010 ) and extrinsic motivations ( Xu and Li 2015 ) . These factors may dampen the effects of ChatGPT on some users and articles. In fact, ( Reeves et al. 2024 ) analyze Wikipedia engagement in the aggregate, and do not identify significant drops in activity following the launch of ChatGPT. We hypothesize that their analysis do not fully capture the heterogeneity of Wikipedia, compared to similar platforms with more homogeneous contents and users. The heart of our empirical strategy is that the impact of ChatGPT on Wikipedia should vary between different types of articles. We classify a subset of articles as “similar” to ChatGPT’s output, capturing the capability of the chatbot to learn and summarize relevant information in these articles. A non-overlapping subset of articles are classified as “dissimilar”, meaning that there is less possibility that ChatGPT will replace Wikipedia as the main provider of information for these articles. We then estimate DiD models with dissimilar articles as our “control group”, in order to estimate the causal effect of ChatGPT on viewership and edits. More specifically, we define our two-year event study period centered at the launch of ChatGPT in November 2022, consider a pool of articles that had been popular before the end of this period and were created relatively recently, and gather their views and edits during the period. We pass their titles to GPT 3.5 and ask it to generate an encyclopedic article, and compute similarity scores between the ChatGPT outputs and the Wikipedia articles. This empirical strategy relies on two assumptions. The first is the “stable unit treatment value assumption”, which states that our control group, dissimilar articles, is not impacted by ChatGPT. This assumption is motivated by the fact that ChatGPT is less capable of replicating the content of these articles. If the assumption is violated due to indirect effects (e.g., ChatGPT makes Wikipedia contributors more productive), our strategy will still reveal the differential additional effect on similar articles relative to dissimilar ones (but not the full impact on dissimilar articles). Our second assumption is the standard “parallel trends” assumption of DiD: absent ChatGPT, similar and dissimilar articles would have evolved in similar fashion. Figure 2 below, which shows similar pre-ChatGPT evolution of these two groups, bolsters our confidence in this assumption. Our analysis reveals that, following the launch of ChatGPT, articles with similar content experienced a significant decline in views relative to dissimilar articles. For edits, we find suggestive but not statistically significant evidence of fewer edits on similar articles compared to dissimilar ones. These findings suggest that substitution effects of ChatGPT may vary depending on its capabilities relative to the content of interest. It highlights the need for further research into the potential effects of LLMs on voluntary knowledge contributions across crowdsourced platforms, and mechanisms to explain such behavior for both viewers and content contributors. 2. Data and Methods We perform a difference-in-differences (DiD) analysis of the growth in activities – views and edits in each month – of a pool of English Wikipedia pages from the 12-month period preceding ChatGPT’s launch to the 12-month period that follows it. Pages are classified as “similar” or “dissimilar” based on how much ChatGPT’s responses on the topic align with the actual Wikipedia pages. We analyze how the growths in activities differ for similar vs. dissimilar articles given fixed recency of articles, controlling for article length and the trend in overall growth of all articles. 2.1. Wikipedia Data Time frame. We define our event study period E , the period during which activities are measured, as the 24 months from December 2021 to November 2023 (inclusive). It consists of

Read the complete text