Effects of Generative AI Errors on User Reliance Across Task Difficulty Conference: Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772363.3798463 ISBN: 979-8-4007-2281-3/2026/04 CCS: Human-centered computing HCI theory, concepts and models CCS: Human-centered computing Empirical studies in HCI
Jacy Reese Anthis, Hannah Cha, Solon Barocas, Alexandra Chouldechova, Jake Hofman · 2026
Read Effects of Generative AI Errors on User Reliance Across Task Difficulty Conference: Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772363.3798463 ISBN: 979-8-4007-2281-3/2026/04 CCS: Human-centered computing HCI theory, concepts and models CCS: Human-centered computing Empirical studies in HCI by Jacy Reese Anthis, Hannah Cha, Solon Barocas, Alexandra Chouldechova, Jake Hofman (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04319.txt.
Introduction
Anthis and colleagues study how AI mistakes affect trust across easy and hard tasks. AI skill follows a jagged frontier, failing easy tasks while acing hard ones. The team built an incentive-compatible test around diagram generation, inducing errors at 10, 30 or 50 percent on easier or harder tasks. The 3-by-2 experiment ran with 577 participants. More errors cut use, as expected. Yet easy-task errors did not cut use significantly more than hard-task errors, suggesting no aversion to jaggedness here. The authors urge future work varying difficulty alongside other error features.
Catalog record
| CREATOR | Jacy Reese Anthis, Hannah Cha, Solon Barocas, Alexandra Chouldechova, Jake Hofman |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31815 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.04319
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Effects of Generative AI Errors on User Reliance Across Task Difficulty Conference: Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems (CHI EA ’26), April 13–17, 2026, Barcelona, Spain DOI: 10.1145/3772363.3798463 ISBN: 979-8-4007-2281-3/2026/04 CCS: Human-centered computing HCI theory, concepts and models CCS: Human-centered computing Empirical studies in HCI Jacy Reese Anthis, Hannah Cha, Solon Barocas, Alexandra Chouldechova, Jake Hofman Abstract . The capabilities of artificial intelligence (AI) lie along a jagged frontier, where AI systems surprisingly fail on tasks that humans find easy and succeed on tasks that humans find hard. To investigate user reactions to this phenomenon, we developed an incentive-compatible experimental methodology based on diagram generation tasks, in which we induce errors in generative AI output and test effects on user reliance. We demonstrate the interface in a preregistered 3x2 experiment ( N N = 577) with error rates of 10%, 30%, or 50% on easier or harder diagram generation tasks. We confirmed that observing more errors reduces use, but we unexpectedly found that easy-task errors did not significantly reduce use more than hard-task errors, suggesting that people are not averse to jaggedness in this experimental setting. We encourage future work that varies task difficulty at the same time as other features of AI errors, such as whether the jagged error patterns are easily learned. † † cc-license: by 1. Introduction The general-purpose capabilities of artificial intelligence (AI), particularly generative AI tools such as large language models (LLMs), create significant cognitive challenges for users ( Sturgeon et al. 2025 ; Subramonyam et al. 2024 ; Tankelevitch et al. 2024 ; Zamfirescu-Pereira et al. 2023 ) . Effective use requires forming and updating accurate expectations of the tool’s effectiveness across diverse tasks. Accurate calibration is made challenging by the complexity and opacity of the tool ( Zhao et al. 2024a ) , diversity of use cases ( Raiaan et al. 2024 ) , and highly personalized experiences that emerge over long periods of use ( Manoli et al. 2025 ) . This is challenging for users, developers who aim to build useful tools, and researchers aiming to understand tool development and use. A crucial challenge humans face is accounting for the “jagged frontier” ( Dell’Acqua et al. 2023 ; Gans 2026 ; Morris et al. 2026 ; Zhou et al. 2024 ) , the fact that modern AI systems are difficult to predict because they can fail on tasks that humans find easy despite superhuman performance on tasks that humans find hard. For example, AI systems can fail to count letters in common words or answer riddles correctly when rephrased ( McCoy et al. 2024 ; West et al. 2023 ; Zhou et al. 2024 ) , but they can also summarize vast amounts of text ( Skarlinski et al. 2024 ) and speak dozens of languages ( Zhao et al. 2024b ) . Even if a human observes an example of AI performance, they are often not able to effectively update their predictions of AI performance on a different task ( Vafa et al. 2024 ) . To understand human-AI interaction in light of such challenges, we developed an incentive-compatible experimental methodology to study human reactions to generative AI errors that balances realism, experimental control, and the ability to quantify user reliance. Our approach involves generating diagrams of the type used in project planning ( Workspace [n. d.] ) and slideshow presentations ( Lucidchart [n. d.] ) . In Phase 1, participants learn about the AI tool’s performance by observing a series of tasks in which they are shown a prompt given to the tool and the resulting diagram, with the pattern of successes and errors determined by random assignment to treatment groups. In Phase 2, participants apply what they have learned about the AI tool in a series of willingness - to - pay (WTP) tasks ( Becker et al. 1964 ) as a measure of reliance: in each task, they can pay to use the AI tool to attempt the task, and they receive a monetary reward if the AI tool successfully generates the intended diagram. This way, we can observe effects of error patterns in Phase 1 on user reliance in Phase 2. Using this methodology, we ran a preregistered 3x2 experiment ( N = 577) in which participants were randomly assigned to one of six Phase 1 treatments: observing an error rate of 10%, 30%, or 50%, with those errors being on either the hard tasks or the easy tasks. In this context, “easy” and “hard” refer to the complexity of the diagram that must be generated, where a human would find it easier to quickly create a simpler diagram. As expected, a larger number of errors led to lower WTP. However, contrary to expectations, we did not find a consistent effect of task difficulty. These findings raise important questions about how humans will under- or over-rely on AI tools and how human-AI collaboration will evolve as AI tools become more powerful. We propose follow-up research that further investigates human reactions by varying not just task difficulty but also how easy the error pattern is for humans to learn. People may be comfortable with non-humanlike AI systems as long as the systems are predictable, or they may prefer humanlike systems even if the systems are not more predictable. 2. Background People stop using relatively simple algorithmic tools after seeing even minor mistakes, penalizing the tool more harshly than they would a human advisor, a pattern termed algorithm aversion ( Dietvorst et al. 2015 ) , yet others have identified cases of automation bias ( Goddard et al. 2012 ) and algorithm appreciation ( Castelo et al. 2019 ; You et al. 2022 ) in which people favor algorithms over human judgment. These patterns can lead to underreliance and overreliance, where humans fail to correctly identify errors ( Bansal et al. 2019a ) . Human failures to respond to errors appropriately are a contributing factor in the failure of human–AI teams to attain “strong synergy,” an ideal interaction in which teams outperform humans and AI systems that complete tasks individually ( Vaccaro et al. 2024 ) . AI reliance depends on various factors, such as whether tasks are perceived as objective or subjective ( Castelo et al. 2019 ) , whether humans merely provide input to the algorithm versus being able to override the algorithmic outcome ( Cheng and Chouldechova 2023 ) , and whether the algorithm is described as a novice or an expert ( Khadpe et al. 2020 ) . Appropriate reliance can be facilitated through system design, including explanations of the algorithmic decision-making process ( Schoeffer et al. 2024 ; Vasconcelos et al. 2023 ) and encouragement for users to reflect thoughtfully on whether to rely on the algorithm ( Buçinca et al. 2021 ) . However, much of this evidence is based on interaction with conventional tools. For example, Bansal et al. 2019b tested advice that consisted of a binary choice (e.g., whether a product was defective), and Dhuliawala et al. 2023 mapped trust recovery following errors in which