Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06
Travis Lloyd, Tung Nguyen, Karen Levy, Mor Naaman · 2025
Read Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 by Travis Lloyd, Tung Nguyen, Karen Levy, Mor Naaman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-15434.txt.
Introduction
Travis Lloyd leads a team that studies crowd written notes on social media posts. The best known case is Community Notes on X, where users add context to posts. Other platforms are adding like systems as a swap for paid fact checks. The team call these crowdsourced context systems and ask how they are built and what they change.
They review past studies of such systems and look at real cases in use. From that base they build a two part frame. One part defines what these systems are, and the other maps six design choices, from who takes part to how open the process is. They also flag the values at stake in each choice. The work sets terms for future study.
Catalog record
| CREATOR | Travis Lloyd, Tung Nguyen, Karen Levy, Mor Naaman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-15434
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Community Notes: A Framework for Understanding and Building Crowdsourced Context Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 Travis Lloyd, Tung Nguyen, Karen Levy, Mor Naaman Abstract . Social media platforms are increasingly developing features that display crowdsourced context alongside posts, modeled after X’s Community Notes. These systems, which we term Crowdsourced Context Systems (CCS), have the potential to reshape our information ecosystem as major platforms embrace them as alternatives to top-down fact-checking. To deeply understand the features and implications of such systems, we perform a systematic literature review of existing CCS research and analyze several real-world CSS implementations. Based on our analysis, we develop a framework with three distinct components. First, we present a theoretical model to help conceptualize and define CCS. Second, we identify a design space encompassing six key aspects of CCS: participation, inputs, curation, presentation, platform treatment, and transparency. Third, we identify key normative implications of different CCS design and implementation choices. Our framework integrates these theoretical, design, and ethical perspectives to establish a foundation for future human-centered research on Crowdsourced Context Systems. Two side-by-side screenshots showing Community Notes displayed on different platforms. The left image shows Instagram with a post containing a photo of two people walking, with a Community Note appearing below providing additional context. The right image shows X (formerly Twitter) with a post about Mars and a Community Note displayed underneath with contextual information and engagement metrics. Figure 1. Screenshots of a Community Note displayed on Instagram (left) and X (right). 1. Introduction Social media platforms play an important role as sites of entertainment ( Herder and Staring 2024 ) , news consumption ( Chouaki et al. 2024 ) , and interpersonal connection ( Preece 1999 ) . At the same time, these platforms must contend with problems stemming from harmful user-uploaded content, which can take the form of abuse, spam, or misinformation ( Scheuerman et al. 2021 ) . Specifically, platforms have tried various forms of content moderation ( Grimmelmann 2015 ; Lampe and Resnick 2004 ; Jhaver et al. 2019 ) to deal with misinformation, including centralized fact-checking efforts to label or remove it ( Gillespie 2018 ) , as well as distributed approaches where users are empowered to tackle these challenges themselves ( Seering 2020 ) . While these approaches have had some success, they have also evoked backlash and accusations of bias from both users and lawmakers, particularly on the political right ( Jiang et al. 2020 ; Johnson 2023 ) . Recently, a new approach has emerged: beginning with X/Twitter, major social media platforms are increasingly introducing systems that crowdsource “helpful context” to display alongside potentially problematic posts ( Coleman 2021 ) (e.g. Figure 1 ). These systems, which we term Crowdsourced Context Systems (CCS), have seen growing adoption and impact, with some of the world’s largest platforms, including Meta (Facebook, Instagram, and Threads), TikTok, and YouTube all recently launching their own implementations. Meta has gone one step further, announcing that their CCS will replace their fact-checking program in the U.S. ( Kaplan 2025 ) . As CCS adoption grows, these systems have the potential to reshape our information ecosystem. Early studies into such systems have produced promising results, finding that CCS can increase user informedness and decrease the spread of misinformation ( Wojcik et al. 2022 ; Chuai et al. 2024b ; Drolsbach and Pröllochs 2023 ) , although it is still unclear whether CCS can replace centralized fact-checking, as some platforms have done or may seek to do ( Kaplan 2025 ) . As CCS develop, novel implementation questions also come to the fore, including how to design these systems most effectively, for example, to provide reliable context quickly enough to prevent the spread of the most viral content ( Chuai et al. 2024b ; Chuai et al. 2024a ; Renault et al. 2024 ) . At this critical moment of change in the information ecosystem, it is important to understand how CCS function, how we should evaluate them, and how these systems may be better designed. This paper makes two contributions towards the goal of better understanding CCS. Our first contribution is a systematic literature review of existing research studying implementations of CCS. Our review finds significant research into CCS contributor behavior and the impact of displaying notes. However, the review highlights the fact that certain aspects of these systems are underexplored by existing research. We thus set out to identify additional facets of these systems that may affect their impact. Our second contribution is an analysis and comparison of real-world implementations of CCS by several major social media platforms, in order to create a framework to guide future HCI research on these novel and important systems. Our framework is composed of three parts. First, we offer a theoretical model of a CCS that identifies these systems’ core characteristics and what sets them apart from prior crowdsourced systems. Our model posits that while CCS use familiar moderation tools like crowdsourced information and context labels, they are unique in how they are natively integrated into platforms with the explicit goal of fostering user informedness. Second, we outline a design space ( MacLean et al. 1996 ) for CCS in order to taxonomize the important design decisions made in their implementations. Design spaces are a conceptual tool used in HCI research that can guide work on complex socioetechnical systems by structuring the space of design possibilities ( Zhang et al. 2024 ; Lee et al. 2024 ; Card et al. 1990 ) . We group these design decisions into six high-level aspects of CCS that combine and interact to shape their impact: participation, inputs, curation, platform treatment, and transparency. Third, we identify key normative implications of these systems in order to make sense of the ethical stakes involved in their design. As systems with the power to shape the information ecosystem, we emphasize three interconnected concerns: user informedness , distribution of power , and fairness . We discuss the role of CCS within the broader conversation about content moderation and establish a foundation for future human-centered research on Crowdsourced Context Systems. 2. Background In January 2021, Twitter (now X) launched a U.S. pilot for the first CCS, which they called Birdwatch , presenting it as, “a community-based approach to misinformation” ( Coleman 2021 ) . This system allowed a subset of users to suggest notes to, “provide informative context” about Tweets (posts) on the platform. The same subset of users then rated these suggested notes and these ratings were used as the inputs to a note curation algorithm that aimed to select notes that would be helpful to a broad audience. In this pilot, these notes were only visible on a separate site, but Twitter announced their plan, “to make notes visible directly on Tweets for the global Twitter audience, when there is consensus from a broad and diverse set of contributors.” In March 2022, Twitter expanded the pilot, announcing that now, “a small (and randomized) group of people on Twitter in the US will see Birdwatch notes directly on some Tweets” ( Coleman 2022 ) . Twitter was