Augmenting Human Cognition With Generative AI: Lessons From AI-Assisted Decision-Making Conference: CHI 2025 Workshop on Tools for Thought: Research and Design for Understanding, Protecting, and Augmenting Human Cognition with Generative AI; April 26, 2025; Yokohama, Japan CHI ’25 T4T: Workshop on T
Zelun Tony Zhang, Leon Reicherts · 2025
Read Augmenting Human Cognition With Generative AI: Lessons From AI-Assisted Decision-Making Conference: CHI 2025 Workshop on Tools for Thought: Research and Design for Understanding, Protecting, and Augmenting Human Cognition with Generative AI; April 26, 2025; Yokohama, Japan CHI ’25 T4T: Workshop on T by Zelun Tony Zhang, Leon Reicherts (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03207.txt.
Introduction
Generative tools usually hand users a finished answer to accept or reject, and Tony Zhang and Leon Reicherts argue that habit undersells both the tool and the user. In a 2025 position paper they draw lessons from their own studies of AI-assisted decision-making. The alternative they favor is process-oriented support, help that arrives in steps while the user still does the solving.
End-to-end suggestions, their work shows, carry real challenges, from overreliance to shallow checking. Incremental support can counter those failure modes by keeping the human in the reasoning loop. A recent comparison of both styles on a complex decision task with language models tests the transfer. The authors frame the findings as design guidance for tools meant to augment human cognition rather than replace it.
Catalog record
| CREATOR | Zelun Tony Zhang, Leon Reicherts |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03207
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Augmenting Human Cognition With Generative AI: Lessons From AI-Assisted Decision-Making Conference: CHI 2025 Workshop on Tools for Thought: Research and Design for Understanding, Protecting, and Augmenting Human Cognition with Generative AI; April 26, 2025; Yokohama, Japan CHI ’25 T4T: Workshop on Tools for Thought—Research and Design for Understanding, Protecting, and Augmenting Human Cognition with Generative AI, April 26, 2025, Yokohama, Japan CCS: Human-centered computing Interaction paradigms CCS: Human-centered computing HCI theory, concepts and models CCS: Information systems Decision support systems Zelun Tony Zhang, Leon Reicherts Abstract . How can we use generative AI to design tools that augment rather than replace human cognition? In this position paper, we review our own research on AI-assisted decision-making for lessons to learn. We observe that in both AI-assisted decision-making and generative AI, a popular approach is to suggest AI-generated end-to-end solutions to users, which users can then accept, reject, or edit. Alternatively, AI tools could offer more incremental support to help users solve tasks themselves, which we call process-oriented support. We describe findings on the challenges of end-to-end solutions, and how process-oriented support can address them. We also discuss the applicability of these findings to generative AI based on a recent study in which we compared both approaches to assist users in a complex decision-making task with LLMs. 1. Introduction In recent years, the perspective that AI should not replace, but augment and enhance human cognition has become increasingly popular ( Carter and Nielsen 2017 ; Akata et al. 2020 ; Shneiderman 2020 ) . Now, with the rapid progress in generative AI (GenAI), this perspective becomes even more relevant, as AI can generate satisfactory, or sometimes even impressive solutions to more and more tasks with little human effort. But how can we leverage these capabilities without sidelining human reasoning? Or more ambitiously, how can we combine artificial with human intelligence to produce better results than either could on its own? These questions have been extensively researched in the field of AI-assisted decision-making as one of the primary applications of human-AI collaboration prior to the popularization of GenAI. Therefore, to understand how to design GenAI systems to protect and augment human thinking, it seems natural to turn to AI-assisted decision-making for lessons to learn. In this position paper, we review our own work in AI-assisted decision-making for how it can inform the design of GenAI tools for thought. We start by outlining several challenges of using AI to propose end-to-end solutions ( Section 2 ), continue by presenting alternative approaches and how they can overcome these challenges ( Section 3 ), and end by discussing the applicability of these findings to GenAI, based on our recent study on supporting complex decisions with LLMs ( Section 4 ). 2. Challenges of End-to-End Solutions There is a tendency in AI-assisted decision-making to design AI in a recommendation-centric way: The AI provides an end-to-end recommendation, i.e., a recommendation for the final decision straight from the input data, without involving the user. The user can then decide whether to accept or overrule this recommendation. A similar tendency can be observed in GenAI as well, where the AI often assists by generating an end-to-end solution to the problem, and the user is only involved afterwards as an editor. In our research, we identified several challenges that come with this approach. Increasing overreliance over time. In ( Zhang et al. 2023b ) , we measured how participants’ overreliance developed over repeated interactions with a recommendation-centric AI, and found that overreliance increased over time. This was to be expected for long, tiring study sessions that are routinely used in AI-assisted decision-making research. However, overreliance even increased when participants only had short interactions with the AI, which were spread over a longer period. The results indicate that providing end-to-end recommendations makes it difficult for users to remain meaningfully engaged with the decision-making task, similar to the difficulty of supervisory control in automation ( Bainbridge 1983 ) . Overreliance in difficult tasks. We conducted two studies using different methods and decision tasks to study how users’ reliance on recommendation-centric AI depends on decision difficulty ( Zhang et al. 2024a ; Zhang et al. 2024c ) , as the vision is often for AI to augment human reasoning in difficult tasks. The results of both studies were consistent in that participants could often rely appropriately on AI recommendations in easy decisions, but became much more over-reliant in difficult decisions. The problem is that for decisions where users are highly uncertain about their answer, recommendation-centric support does little to improve their understanding of the decision. Consequently, users cannot meaningfully contribute and have little choice but to rely on the AI recommendation. Helpful when correct, but a burden when wrong. A common assertion in the discourse about using AI in high-stakes domains is that AI models are black boxes that need to be opened. However, when we interviewed professional pilots about their thoughts on deploying AI into cockpits ( Zhang et al. 2021 ) , transparency was not their main concern. Instead, they were much more concerned that AI might not be intelligent enough to respond appropriately to complex situations and become a burden when workload is already high—a phenomenon known as clumsy automation in automation research ( Cook et al. 1991 ) . This finding points to a seemingly trivial, but often neglected requirement for the design of AI systems: that imperfect AI outputs should have minimal negative consequences for users, which Gu et al. 2024 call AI-resilient . In a follow-up study ( Zhang et al. 2023a ) , we found why recommendation-centric support tends to violate this requirement. Pilots stated that they would not blindly trust AI, but they also rejected the idea that they would have to review every AI recommendation for its correctness. This seeming contradiction highlights that instead of augmenting users’ reasoning, end-to-end recommendations burden users with an effortful review task that is not integrated into their existing reasoning processes—and one that becomes more effortful the more complex the decision is. 3. Benefits of Process-Oriented Support Figure 1. Recommendation-centric vs. process-oriented support. Adapted from ( Zhang et al. 2024b ) . Providing end-to-end solutions is often the most obvious approach, but it is not the only one. Rather than solving tasks for users, we can also aim to help users solve the task themselves ( Dix 2007 ) by identifying and focusing on their primary challenges when solving a task. Rather than end-to-end solutions, the AI would provide more incremental support targeted at these primary challenges. The goal would be to help users reason forward through their reasoning processes, rather than pushing them to reason backward from an end-to-end solution. We call this alternative paradigm process-oriented support ( Figure 1 ). We empirically compared recommendation-centric to process-oriented support in the use case of diversions in commercial aviation ( Zhang et al. 2024b ) . When making diversion decisions, pilots’ primary challenge is to gather and integrate the required information from multiple sources. Our process-oriented support concept addressed this challenge by pulling relevant information together in a continuously updated table view. AI continuously evaluated the information and highlighted possible limitations at the surrounding airports within the table. We found that