LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities DOI: XXXXXXX.XXXXXXX Conference: PACM on Human-Computer Interaction; 2025; TBD CCS: Human-centered computing Empirical studies in collaborative and social computing

Moyan Zhou, Soobin Cho, Loren Terveen · 2025

Read LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities DOI: XXXXXXX.XXXXXXX Conference: PACM on Human-Computer Interaction; 2025; TBD CCS: Human-centered computing Empirical studies in collaborative and social computing by Moyan Zhou, Soobin Cho, Loren Terveen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-07819.txt.

Introduction

How are large language models changing who writes Wikipedia, and how? Moyan Zhou and 2 co-authors investigate that question in a 2025 study of online knowledge communities. Editors increasingly use these models in their writing workflow, yet the encyclopedia has no settled rules for the practice, leaving individuals to experiment alone.

Participation in such communities is social, not just a matter of producing text. The authors examine how model use affects contribution and community life, a gap that matters for writing future norms. Their findings can help communities decide where automated help supports volunteers and where it may weaken the human ties that keep a shared knowledge project alive and trustworthy.

Catalog record

CREATORMoyan Zhou, Soobin Cho, Loren Terveen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-07819

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

LLMs in Wikipedia: Investigating How LLMs Impact Participation in Knowledge Communities DOI: XXXXXXX.XXXXXXX Conference: PACM on Human-Computer Interaction; 2025; TBD CCS: Human-centered computing Empirical studies in collaborative and social computing Moyan Zhou, Soobin Cho, Loren Terveen Abstract . Large language models (LLMs) are reshaping knowledge production as community members increasingly incorporate them into their contribution workflows. However, participating in knowledge communities involves more than just contributing content - it is also a deeply social process. While communities must carefully consider appropriate and responsible LLM integration, the absence of concrete norms has left individual editors to experiment and navigate LLM use on their own. Understanding how LLMs influence community participation is therefore critical in shaping future norms and supporting effective adoption. To address this gap, we investigated Wikipedia, one of the largest knowledge production communities, to understand 1) how LLMs influence the ways editors contribute content, 2) what strategies editors leverage to align LLM outputs with community norms, and 3) how other editors in the community respond to LLM-assisted contributions. Through interviews with 16 Wikipedia editors who had used LLMs for their edits, we found that 1) LLMs affected the content contributions for experienced and new editors differently; 2) aligning LLM outputs with community norms required tacit knowledge that often challenged newcomers; and 3) as a result, other editors responded to LLM-assisted edits differently depending on the editors’ expertise level. Based on these findings, we challenge existing models of newcomer involvement and propose design implications for LLMs that support community engagement through scaffolding, teaching, and context awareness. 1. Introduction Knowledge production refers to the process of creating, maintaining, and propagating knowledge ( Alavi and Leidner 2001 ) . The laborious process has traditionally required contributors to carefully gather, verify, and synthesize information. But the advent of large language models (LLMs) has transformed this reality, as LLMs enable users to generate fluent and well-structured content through natural language prompts ( Brachman et al. 2024 ; Shao et al. 2024 ) . Prior work has indicated the usefulness of LLMs as writing assistants and imagined the potential of LLMs in knowledge production ( Wasi et al. 2024 ; Lee et al. 2024 ; Aubin Le Quéré et al. 2024 ; Shao et al. 2024 ) . However, participation in knowledge communities involves more than writing fluent or well-structured paragraphs. It is also highly epistemic (e.g., factuality, balance) and social ( Bryant et al. 2005 ; Ren et al. 2023 ) (e.g., aligning with norms ( Butler et al. 2008 ; Beschastnikh et al. 2008 ) , engaging with other community members ( Maddock et al. 2017 ; Ren et al. 2023 ) ). There is a mismatch between surface-level fluent content produced by LLMs and the unique epistemic and social needs of knowledge communities, such as knowledge integrity ( Heather Ford and Rizoiu 2023 ) . Furthermore, Wikipedia, one of the largest knowledge production platforms ( Ayers et al. 2008 ; Ren et al. 2023 ) has not yet reached a consensus or developed norms around LLMs ( Ayers et al. 2008 ; Ren et al. 2023 ) , which could exacerbate these challenges. As a result, individual editors must interpret, navigate, and experiment with appropriate and responsible usage of LLMs within the community. The tension between surface-level content generation and unique epistemic and social needs raises important questions about how LLMs are shaping community participation in knowledge communities, critical to support the credibility and sustainability of these knowledge communities. This is especially true given the rise of LLM-generated content in new articles observed in Wikipedia ( Brooks et al. 2024 ) , indicating that editors have already incorporated LLMs during their contributions. Our study aims to understand the use of LLMs by knowledge community members. Specifically we ask: • RQ1 : How does using LLMs influence the ways editors contribute content? • RQ2 : What strategies do editors leverage to conform to community norms for LLM-assisted contributions? • RQ3 : How do other editors in the community respond and engage with LLM-assisted contributions? Through semi-structured interviews with 16 participants who have used LLMs to edit Wikipedia, we unveil an expertise-based participation divide between editors. Experienced editors enhance their participation through LLMs, as they expand the range of their contributions in Wikipedia, leverage strategies to align LLM-generated content with community norms, and thus receive positive responses from other members. In contrast, LLMs raise the demands for new editors to participate in the community. LLMs lower the barriers for entry, and new editors tend to rely on LLMs to fill in their knowledge gaps. However, LLMs compel them to make judgments about AI-generated content, which requires skills they have not yet developed. This challenges them to produce high quality content, leading to rejections from others. Based on our findings, we discuss how the participation paradox for newcomers, namely lower barriers to entry yet higher demands to participate, arises from LLM usage, breaking the trajectory of Legitimate Peripheral Participation (LPP). LLMs interrupt learning pathways for newcomers, thus new editors miss opportunities to gain wiki skills gradually. At the same time, LLMs blur the boundaries between peripheral and central tasks and push newcomers from peripheral participants to editorial judges, which requires more central wiki skills. Building on our findings, we then propose design implications to mitigate the participation gap LLMs can lead to by fostering newcomers’ learning process and supporting editors with varying expertise in knowledge production platforms. We situate our work within the broader empirical HCI/CSCW research on human AI collaboration and communities of practices ( MacKenzie 2024 ; Oulasvirta and Hornbæk 2016 ; Erickson and Kellogg 2000 ) , particularly work around how tools reshape work ( Norman 1991 ; Dillahunt and Malone 2015 ) , how individuals align to norms and boundaries ( Hsieh et al. 2023 ; Halfaker et al. 2013a ; Kiene et al. 2016 ) , and how community respond and adapt to technology changes ( Smith et al. 2020 ; Kuo et al. 2024 ) . By examining how editors integrate LLMs into their practices, we made the following contributions: • We surfaced the invisible dynamics of how editors with different levels of experience adopt, adapt, and respond to LLMs in knowledge communities in practice; • We extend Legitimate Peripheral Participation by demonstrating that LLM usage interrupts newcomers’ traditional learning pathway; • We informed the design of future LLMs for knowledge communities, highlighting the need for AI tools to scaffold, teach, and be context aware to support community participation. 2. Related Work 2.1. Human AI collaboration in writing Knowledge production involves writing, summarizing, and drafting information in textual format. To contextualize this process, we surveyed the literature in the domain of AI-assisted writing and identified patterns of human-AI collaboration. Generative AI supports writing across a wide range of domains, from personal writing such as diary entries ( Kim et al. 2024 ) , to creative writing including stories ( Biermann et al. 2022 ; Yuan et al. 2022 ; Lee et al. 2022 ; Stefnisson and Thue 2018 ; Chung et al. 2022 ) , fictions ( Yang et al. 2022 ) , metaphors ( Gero and Chilton 2019 ) , poetry ( Chakrabarty et al. 2022 ) ,

Read the complete text