The Impact of AI Explanations on Clinicians’ Trust and Diagnostic Accuracy in Breast Cancer
Olya Rezaeian, Onur Asan · 2024
Read The Impact of AI Explanations on Clinicians’ Trust and Diagnostic Accuracy in Breast Cancer by Olya Rezaeian, Onur Asan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-11298.txt.
Introduction
Olya Rezaeian and co-authors test how AI explanations affect doctors reading breast cancer cases in a paper from 2024. AI tools can support diagnosis, but their black-box nature can hurt trust. Explanations are often offered as the fix, yet more detail may not always help.
Twenty-eight clinicians with roles in breast cancer care took part in the experiment. They worked cases with varied levels of AI explanation. Trust and accuracy were tracked for each level. Richer explanations did not always raise trust or accuracy. Self-rated AI familiarity also varied with age and gender. The study warns clinics to test explanation styles rather than assume that more is better.
Catalog record
| CREATOR | Olya Rezaeian, Onur Asan |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-11298
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Impact of AI Explanations on Clinicians’ Trust and Diagnostic Accuracy in Breast Cancer Olya Rezaeian, Onur Asan Abstract Advances in machine learning have created new opportunities to develop artificial intelligence (AI)-based clinical decision support systems using past clinical data and improve diagnosis decisions in life-threatening illnesses such breast cancer. Providing explanations for AI recommendations is a possible way to address trust and usability issues in black-box AI systems. This paper presents the results of an experiment to assess the impact of varying levels of AI explanations on clinicians’ trust and diagnosis accuracy in a breast cancer application and the impact of demographics on the findings. The study includes 28 clinicians with varying medical roles related to breast cancer diagnosis. The results show that increasing levels of explanations do not always improve trust or diagnosis performance. The results also show that while some of the self-reported measures such as AI familiarity depend on gender, age and experience, the behavioral assessments of trust and performance are independent of those variables. 1 Introduction Given the data-intensive nature of healthcare, AI-based Clinical Decision Support Systems (CDSS) have become increasingly prevalent across various domains, including diagnosis [ 1 , 2 , 3 , 4 ] , treatment planning [ 5 , 6 ] , and symptom checking [ 7 ] . CDSS systems offer numerous advantages, such as reducing diagnostic errors, improving efficiency, and lowering costs if designed and utilized efficiently [ 8 , 9 , 10 ] . Many AI systems have been developed to enhance the prediction of clinical outcomes, especially for breast cancer detection [ 11 ] . However, despite their potential, one of the most significant challenges remains the establishment of factors that influence the interaction between clinicians and these AI-driven tools, such as trust, perceived usefulness, and ease of use. No matter how advanced or effective a CDSS may be, its success attaches to clinicians’ willingness to use and integrate its recommendations into their practice [ 12 ] . Thus, trust is a fundamental factor in successfully adopting and utilizing AI in healthcare settings [ 6 ] . Trust in AI systems is influenced by numerous factors, as highlighted by a substantial body of research. Among these, the explainability of AI systems has emerged as one of the most critical elements [ 13 ] , particularly due to the ”black-box” nature of many machine learning models, which can make it difficult for clinicians to understand how decisions are reached. It is known in the literature that the type and medium of information presented to the clinicians impact their tendency to agree with the system [ 14 ] . However, the exact role that explainability plays in shaping clinician behavior is not yet fully understood. In this experimental study, we explore the interaction between clinicians and our AI-based CDSS for breast cancer diagnosis, focusing on the impact of varying levels of explainability. Our primary aim is to evaluate how different degrees of AI explainability influence clinicians’ diagnostic accuracy, as well as their self-reported and behavioral trust in the system. To achieve this, we present the results of a human-subject experiment using a custom-built AI web application that provides diagnostic recommendations generated through machine learning techniques with varying levels of explainability in multiple interventions. The experimental data we present includes the clinicians’ interactions with system and their diagnostic decisions as well as pre- and post-experiment surveys assessing demographics and participants’ perceptions and trust levels. Data from these experiments enable us to compare outcomes across different levels of explainability in multiple interventions to determine the optimal balance that fosters trust, adoption, and effective decision-making. The findings provide valuable insights into best practices for integrating explainability into AI systems in healthcare, ultimately contributing to more informed, data-driven clinical decisions and improved patient outcomes. Aligned with the objectives of this study, we seek to address the following research questions: • RQ1: Does providing explanations improve both human decision-making performance and trust in AI-based clinical decision support systems for breast cancer detection? • RQ2: How do clinicians’ demographic characteristics, such as age, gender, and familiarity with AI, influence their trust in and performance with AI-based clinical decision support systems (CDSS) in breast cancer detection? 2 Related Work 2.1 Trust in Human-Technology Interaction Trust plays a key role in how much users rely on a system, influencing their willingness to use it [ 15 , 16 , 6 , 17 ] . There are many studies that extensively explore how establishing trust encourages users to depend on a system, ultimately shaping their decisions to engage with it. Muir’s work [ 18 ] serves as a valuable foundation for understanding human-machine interactions, proposing a trust model that explores various aspects of this relationship and provides a basis for testing future hypotheses. One of the earliest and most influential models of trust was introduced by Mayer et al., who proposed ability, benevolence, and integrity as factors affecting trustworthiness [ 19 ] . However, Mayer’s model primarily focused on interpersonal trust. In the context of human-technology interaction, Parasuraman et al. [ 12 ] expanded this understanding by examining how users interact with automation technology. Their study explored aspects such as usage patterns, reliance, and potential for misuse or overreliance on automation. They identified factors like trust, workload, and perceived risk as significant in determining whether individuals choose to engage with automated systems. Parasuraman et al. also introduced the concept of automation abuse, where system designs overly prioritize automation, potentially leading to diminished human oversight and performance. Such an approach can lead to both overreliance and neglect of automation by human operators. This finding is supported by other research, such as [ 20 ] , which emphasizes that overreliance on automation may result in errors and negative consequences, including insufficient system monitoring or biased decision-making. Thus, the studies on literature highlights the need for designing the technology for appropriate reliance or ”calibrated trust” [ 20 , 15 ] . Chiou [ 21 ] studies trust in workplace automation by exploring the interplay between humans and automation beyond traditional trust calibration. The study emphasizes responsive automation to enhance collaboration, aiming not just for trust calibration but for cultivating trust through the adaptability of automation systems. To build upon these insights, it is essential to recognize that measuring trust in technology is a complex task. Trust is often conceptualized as a latent variable, meaning it cannot be directly observed but must instead be inferred through indirect measures [ 22 ] . To address this, Kohn et.al [ 22 ] proposed three main categories of measures that can be used to assess trust: self-reported measures, behavioral measures, and physiological measures. In this study, we will focus on utilizing the first two — self-reported and behavioral measures — to assess clinicians’ trust in AI-based systems. 2.2 Factors Affecting Trust Formation in Technology Many researchers have explored factors influencing users’ trust, aiming to enhance trust in AI systems for effective human-AI collaboration. Hoff and Bashir [ 23 ] propose that the trust formation process comprises three layers: dispositional trust, situational trust, and learned trust. Building on those layers, researchers have identified user-related, AI-related, and wider contextual factors that influence trust formation between users and technological