Machines in the Margins: A Systematic Review of Automated Content Generation for Wikipedia Journal: PACMHCI Volume: 9 7 CSCW445 11 DOI: 10.1145/3757626 CCS: Computing methodologies Natural language generation CCS: Human-centered computing Computer supported cooperative work CCS: Human-centered compu
Neal Reeves, Elena Simperl · 2025
Read Machines in the Margins: A Systematic Review of Automated Content Generation for Wikipedia Journal: PACMHCI Volume: 9 7 CSCW445 11 DOI: 10.1145/3757626 CCS: Computing methodologies Natural language generation CCS: Human-centered computing Computer supported cooperative work CCS: Human-centered compu by Neal Reeves, Elena Simperl (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-22443.txt.
Introduction
Neal Reeves and Elena Simperl reviewed research on machines that write Wikipedia content. Wikipedia holds tens of millions of articles, and bots already help with upkeep and admin tasks. Yet bots that create content are the smallest group, and most studies only cover bots already in use. The review instead scans the wider literature for proposed generation methods.
The pair mapped what these methods make, which models they use, and what source text they draw on. They also charted how the methods were tested and judged. The findings point to gaps for the Wikipedia community and for research on user-made content. The review is among the first on this understudied form of AI help.
Catalog record
| CREATOR | Neal Reeves, Elena Simperl |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 88 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-22443
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Machines in the Margins: A Systematic Review of Automated Content Generation for Wikipedia Journal: PACMHCI Volume: 9 7 CSCW445 11 DOI: 10.1145/3757626 CCS: Computing methodologies Natural language generation CCS: Human-centered computing Computer supported cooperative work CCS: Human-centered computing Wikis CCS: Human-centered computing Collaborative content creation Neal Reeves, Elena Simperl Abstract . Wikipedia is among the largest examples of collective intelligence on the Web with over 61 million articles covering over 320 languages. Although edited and maintained by an active workforce of human volunteers, Wikipedia is highly reliant on automated bots to fill gaps in its human workforce. As well as administrative and governance tasks, these bots also play a role in generating content, although to date such agents represent the smallest proportion of bots. While there has been considerable analysis of bots and their activity in Wikipedia, such work captures only automated agents that have been actively deployed to Wikipedia and fails to capture the methods that have been proposed to generate Wikipedia content in the wider literature. In this paper, we conduct a systematic literature review to explore how researchers have operationalised and evaluated automated content-generation agents for Wikipedia. We identify the scope of these generation methods, the techniques and models used, the source content used for generation and the evaluation methodologies which support generation processes. We also explore implications of our findings to CSCW, User Generated Content and Wikipedia, as well as research directions for future development. To the best of our knowledge, we are among the first to review the potential contributions of this understudied form of AI support for the Wikipedia community beyond the implementation of bots. † † cc-license: by 1. Introduction Wikipedia is one of the largest online examples of collective intelligence with over 61 million articles ( Lewoniewski 2023 ) from 321 active languages ( Moás and Lopes 2023 ) as of 2023 making it the “largest encyclopaedic work" ever created through human effort ( Arroyo-Machado et al. 2020 ) . Founded on the principle that anyone can create and edit content regardless of their expertise ( Yarovoy et al. 2020 ) , Wikipedia articles are developed through a highly collaborative process where editors collectively create, edit, refine and update articles either through modifying the articles themselves or through coordinated discussions on article ‘talk’ pages ( Crowston and Rezgui 2020 ) . It is perhaps unsurprising, then, that Wikipedia has long relied on bots and other forms of automation to manage the large quantity of content being created, edited and deleted each day ( Halfaker and Geiger 2020 ) . As many as 16.5% of edits in the English edition of Wikipedia are performed by bots and these bots make up a significant proportion of the most active Wikipedia editors ( Träris and Balsacq 2021 ) . Bots perform a wide variety of roles within Wikipedia, but most commonly they are employed to fix pages by correcting mistakes, linking pages or performing administrative tasks that would otherwise be time-consuming for the community ( Zheng et al. 2019 ) . This use of bots for automating tedious or labour-intensive tasks within Wikipedia has become widely accepted by the user-base ( Clément and Guitton 2015 ) . Nevertheless, even with the assistance of bots, editing Wikipedia remains a labour-intensive process. Experienced Wikipedia editors are often expected to spend a significant proportion of their time on administrative and quality assurance tasks, leaving them little time to devote to the creation of new content, while newer editors may not post new articles without approval from more experienced editors ( Ardati et al. 2024 ) . This reliance on human effort naturally leads to oversight in key editing tasks with over 50% of Wikipedia articles lacking an improvement in quality over their life-cycle ( Das et al. 2022 ) . The disparity between the availability of volunteer effort and the time required to curate and manage contributions is so extreme that the community has responded by increasing barriers to contribution and limiting the creation of new articles and content ( Halfaker and Geiger 2020 ) . To help fill this gap, Wikipedia has turned to content generation bots which take responsibility for creating articles and content ( Zheng et al. 2019 ) . Bot-generated content has a long history with the very first Wikipedia bot being a content generation bot intended to develop novel articles from US census results ( Livingstone 2016 ) . Even today, some language versions of Wikipedia remain highly reliant on automatically generated content. Over 50% of articles in the Serbo-Croatian and Vietnamese editions of Wikipedia are automatically generated, while a single user has automatically generated over a million articles (of disputed quality) in Egyptian Arabic Wikipedia ( Alshahrani et al. 2023 ) . Yet in spite of this, these so-called “generator” bots remain relatively rare compared to other types of bot and are comparatively responsible for significantly fewer edits ( Zheng et al. 2019 ) . Even under ORES, Wikipedia’s initiative to develop key machine learning models, most pipelines focused on quality assurance with a single pipeline focused on drafting new articles typically through predicting article topics ( Halfaker and Geiger 2020 ) 1 1 1 See also the pipeline repository on GitHub: https://github.com/wikimedia/drafttopic . As Large Language Models and other AI systems improve, it is possible that in future, editors may be more readily able to make use of such systems to generate Wikipedia content. Nevertheless, it is not yet possible to do so effectively and responsibly -- on the contrary, Wikipedia’s current policy on LLM usage strongly discourages writing article content 2 2 2 https://en.wikipedia.org/wiki/Wikipedia:Large_language_models . In this paper, we explore the state-of-the-art in research implementing automated generation models for Wikipedia. Through a structured review of research literature, we explore how researchers have operationalised AI models to support content creation tasks. We explore the scope of content generated through such models, the models used, input data for content generation and the methods used to evaluate such content. We further explore implications of this research for CSCW and User Generated Content (UGC) more broadly, as well as for Wikipedia specifically. We conclude with research directions for future work. By analysing how researchers design and implement content automation specifically within the context of Wikipedia, we offer insight into how AI might reshape notions of authorship in UGC communities, as well as the role of human users in such communities. 2. Background and Related Work 2.1. Automation in User-Generated Content Communities Previous CSCW work has explored and characterised the nature and role that bots play in peer-production, knowledge sharing and social online communities. Wessel et al. 2018 explored the role that bots play in Open Source Software projects, identifying 12 tasks performed by bots, largely with the role of supporting and notifying users rather than generating content. In a follow-up study, Wessel et al. 2021 interviewed users about their interactions with bots and found that users often characterised bots as distracting rather than the supportive force they were intended to be. Berkel and Pohl 2024 analysed the use of bots and automation in the online crowdsourced mapping platform OpenStreetMap which they contrast with Wikipedia. Their findings suggested that automation is less disputed in Wikipedia