What People See (and Miss) About Generative AI Risks: Perceptions of Failures, Risks, and Who Should Address Them DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing User studies CCS: Human-centered computing Empirical studies in HCI

Megan Li, Wendy Bickersteth, Ningjing Tang, Parv Kapoor, Khinezin Win, Peter Zhong, Jason I. Hong, Lorrie Faith Cranor, Hoda Heidari, Hong Shen · 2026

Read What People See (and Miss) About Generative AI Risks: Perceptions of Failures, Risks, and Who Should Address Them DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing User studies CCS: Human-centered computing Empirical studies in HCI by Megan Li, Wendy Bickersteth, Ningjing Tang, Parv Kapoor, Khinezin Win, Peter Zhong, Jason I. Hong, Lorrie Faith Cranor, Hoda Heidari, Hong Shen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22654.txt.

Introduction

Li and nine colleagues present a survey tool gauging views of GenAI failure modes, risks, and who must fix them. Despite growing concern about GenAI risks, public grasp of these risks and failure modes, recurring breakdown patterns across the AI lifecycle, remains limited. The tool, checked with eight experts and run on 960 United States participants, uses scenarios from public incidents and a taxonomy of failure modes. Findings suggest the tool gauges risk views in current use contexts, yet extensible to new contexts that will arise.

Catalog record

CREATORMegan Li, Wendy Bickersteth, Ningjing Tang, Parv Kapoor, Khinezin Win, Peter Zhong, Jason I. Hong, Lorrie Faith Cranor, Hoda Heidari, Hong Shen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23347
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.22654

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What People See (and Miss) About Generative AI Risks: Perceptions of Failures, Risks, and Who Should Address Them DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Human-centered computing User studies CCS: Human-centered computing Empirical studies in HCI Megan Li, Wendy Bickersteth, Ningjing Tang, Parv Kapoor, Khinezin Win, Peter Zhong, Jason I. Hong, Lorrie Faith Cranor, Hoda Heidari, Hong Shen Abstract . Despite growing concerns about the risks of Generative AI (GenAI), there is limited understanding of public perceptions of these risks and their associated failure modes —defined as recurring patterns of sociotechnical breakdown across the GenAI lifecycle that contribute to risks of real-world harm. To address this gap, we present a survey instrument, validated with eight subject matter experts and deployed on a sample of 960 U.S.-based participants, to assess awareness and perceptions of GenAI’s failure modes, their associated risks, and stakeholder responsibilities to address them. To support realism and content validity, our instrument is structured around scenarios grounded in publicly reported incidents and a taxonomy of GenAI’s failure modes. Findings suggest that our instrument is (1) effective for assessing risk awareness and perceptions in a way that is grounded in people’s current contexts of use, yet is extensible to new contexts that will inevitably arise; and (2) potentially useful for informing the design of AI literacy tools and interventions. We argue for AI literacy and governance approaches that align with how people encounter and reason about GenAI in everyday life. 1. Introduction Generative AI (GenAI) systems are increasingly embedded in everyday life, prompting growing public concern about the risks they pose ( thorbecke2023ai ; eliot2024generative ) . To define risk, we adapt the definition provided in the AI Risk Management Framework proposed by the U.S. National Institute of Standards and Technology: “the composite measure of an event’s probability of occurring and the magnitude or degree of the consequences of the corresponding event” ( nist_rmf ) and use risk to refer specifically to risk of harm . These risks—and their underlying sociotechnical failure modes, defined as “recurring patterns of sociotechnical breakdown across the GenAI lifecycle that contribute to real-world harms or risks of harm” ( mapping_2025 ) —have been documented and taxonomized in a growing body of work across Human-Computer Interaction (HCI), Fairness, Accountability, and Transparency (FAccT), and Science and Technology Studies (STS) communities, ( mapping_2025 ; weidinger_sociotechnical_2023 ; hutiri_not_2024 ; dominguez_foundation_risks ) . Yet, there is still little understanding about how members of the general public perceive these risks and their associated failure modes. As GenAI is deployed across highly variable domains, public perceptions of its limitations and risks are increasingly consequential. Having a calibrated understanding of GenAI’s failure modes and risks is critical for responsible use and governance of the technology ( ai_literacy ) . While prior work has investigated public perceptions of GenAI, it tends to focus narrowly on specific risks or applications such as privacy, job displacement, or loss of human connection ( ai_companions_perceptions ; ortega_data_flows_genai ; privacy_custom_GPT ) . In doing so, it often overlooks other types of risks, including those posed by basic failures of functionality ( raji_fallacy ) . To address this gap, we structure our work around a comprehensive taxonomy of GenAI failure modes ( mapping_2025 ) . In particular, we investigate perceptions of GenAI’s failure modes and the risks they pose, both broadly and in specific contexts of use. Because GenAI is a general-purpose technology that is constantly applied in new and emerging domains, thereby creating new risks, we argue that using failure modes rather than specific risks as a framing device can offer more generalizable insights into awareness and perceptions of GenAI’s limitations and risks. Our work is framed by three research questions. • RQ1: What GenAI failure modes are people aware of and how likely do they think they are? • RQ2: What risks are people aware of and do they associate them with particular failure modes? • RQ3: Who do people think should address the risks of GenAI? To answer these questions, we developed a scenario-based survey instrument grounded in real-world GenAI incidents and a taxonomy of GenAI failure modes developed by Li et al. ( mapping_2025 ) . Our survey elicits general attitudes toward GenAI, awareness of how GenAI’s failure modes can manifest and cause risks in specific contexts of use, and perceptions of which stakeholders should address the risks associated with GenAI. It is structured around 24 hypothetical scenarios, each of which we developed to illustrate how failure modes can manifest in a particular context. To establish expert consensus on the failure modes illustrated in each scenario, we recruited eight responsible AI experts to evaluate our scenarios. We administered our survey to 960 U.S.-based participants via Prolific. Our findings suggest that our survey instrument is an effective tool for mapping perceptions and understandings of GenAI’s risks across contexts of use. For instance, while prior work on AI risk perceptions has suggested that many people are aware of only a narrow set of societal risks ( Oppenlaender_2023 ) , our participants surfaced a rich array of perceived risks across different contexts of use. Additionally, our empirical results indicate that participants are widely aware of failure modes that are observable in generated outputs, such as hallucinations, but are often unaware of those reflecting upstream design and release choices, such as developers’ problematic data collection and use. This has implications for AI literacy: the gaps revealed by our survey suggest that people can be competent at using GenAI as a tool while being unaware of the potentially harmful ways in which GenAI systems are developed. This is particularly relevant because many participants called on not only government and industry actors but also users and the public to call attention to, mitigate, and address the risks of GenAI. In this work, we introduce and validate a scenario-based survey instrument to assess awareness of GenAI failure modes and their associated risks. We provide one of the first empirical studies mapping people’s understanding of GenAI failures and risks in diverse, real-world contexts of use. We offer our instrument as a tool to probe where misunderstandings of GenAI’s limitations lie and how they can materialize into risky patterns of misuse, ultimately informing literacy interventions and governance frameworks that are grounded in the needs of real people. 2. Related Work The following provides an overview of prior work mapping the risks and failure modes of AI, public perceptions of these risks, and the relevance of both these topics to perspectives on GenAI literacy. Throughout, we highlight work that focuses specifically on GenAI and its unique risk landscape. 2.1. Risks, Harms, and Failure Modes of GenAI Shelby et al. define harms from algorithmic systems as “adverse lived experiences resulting from a system’s deployment and operation in the world” ( shelby ) . Drawing on literature from HCI, FAccT, and STS communities, their work is grounded in the idea that algorithmic harms occur not in a vacuum, but are experienced by people and arise from interaction of technical system components with societal power dynamics ( shelby ) . A growing body of literature identifies and structures these into taxonomies of

Read the complete text