Uncertainty Awareness and Trust in Explainable AI - On Trust Calibration using Local and Global Explanations
Carina Newen, Daniel Bodemer, Sonja Glantz, Emmanuel Müller, Magdalena Wischnewski, Lenka Schnaubert · 2025
Read Uncertainty Awareness and Trust in Explainable AI - On Trust Calibration using Local and Global Explanations by Carina Newen, Daniel Bodemer, Sonja Glantz, Emmanuel Müller, Magdalena Wischnewski, Lenka Schnaubert (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-08989.txt.
Introduction
An AI that explains itself can still mislead users about how much to trust it. Carina Newen and 5 co-authors study that trust calibration problem in a 2025 research paper. Their focus is uncertainty explanations, which say how unsure a model is, and global explanations, which describe the whole model rather than one decision. Both are understudied compared with local explanations.
The team picked an algorithm combining uncertainty, robustness, and global explanation in one scheme. They tested whether it calibrates user trust appropriately. They also asked whether a visually intuitive but complicated display could beat simpler ones on satisfaction and interpretability. Guidelines for explanation design are derived from the results. The stakes are practical: overtrust invites accidents, undertrust wastes good tools, and explanations are the dial meant to set trust correctly.
Catalog record
| CREATOR | Carina Newen, Daniel Bodemer, Sonja Glantz, Emmanuel Müller, Magdalena Wischnewski, Lenka Schnaubert |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-08989
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncertainty Awareness and Trust in Explainable AI - On Trust Calibration using Local and Global Explanations Thanks: Research Center Trustworthy Data Science and Security, TU Dortmund Carina Newen, Daniel Bodemer, Sonja Glantz, Emmanuel Müller, Magdalena Wischnewski, Lenka Schnaubert Abstract Explainable AI has become a common term in the literature, scrutinized by computer scientists and statisticians and highlighted by psychological or philosophical researchers. One major effort many researchers tackle is constructing general guidelines for XAI schemes, which we derived from our study. While some areas of XAI are well studied, we focus on uncertainty explanations and consider global explanations, which are often left out. We chose an algorithm that covers various concepts simultaneously, such as uncertainty, robustness, and global XAI, and tested its ability to calibrate trust. We then checked whether an algorithm that aims to provide more of an intuitive visual understanding, despite being complicated to understand, can provide higher user satisfaction and human interpretability. I Introduction In recent years, the efforts regarding machine-learning-assisted research have doubled and tripled, which has led to artificial intelligence applications being introduced into high-risk domains such as credit scoring [ 6 ] , medical diagnosis [ 17 ] or threat detection [ 25 ] . It has become a primary directive to ensure the trustworthiness of current and future technologies (e.g., European Commission 2019, IBM). A standard tool for establishing trust has been explainable artificial intelligence (XAI). While we see efforts from the psychological side to provide general guidelines for XAI research [ 8 ] or attempts to measure the usability of an algorithm on more minor scales for some proposed algorithms [ 32 ] , it has yet to become accepted that newly established explainability methods are evaluated and improved regarding the effectiveness of their trust calibration ability. It is common that from the computer science community, human interpretability is simply claimed or evaluated at really small scales with very few test subjects [ 32 ] . While this is often the only thing that can be done without proposing a large-scale human study every time for each algorithm, we recognize that detailed studies are necessary, at least for representatives. As we show here later, with our study, we were able to identify the essentials of a global XAI algorithm in the uncertainty domain without producing cognitive overload for the users. The concept we found can be extended to other algorithms potentially making the same mistakes, but we would not have been able to identify it without this in-depth study with open questions. While analyzing every single algorithm on its own also comes with enormous effort and perhaps little insight, we show that there is a knowledge gain given if we do not just analyze the most common algorithms but other representatives that are state-of-the-art in their own niche. For specific purposes, we still need new algorithms. To give an example: While LIME [ 32 ] , Integrated gradients [ 44 ] and its countless extensions cover locally faithful explanations, they have informatory value on how well an algorithm performs overall on an entire dataset. However, it might be helpful to evaluate a machine learning model on a global scale rather than only on local examples. The same can be said about other commonly used explanation algorithms: Shapley values [ 23 ] are often not considered human interpretable (at least regarding non-experts in the field) [ 18 ] . Analyzing only the most commonly used XAI methods, we might not learn anything new regarding methods with different goals, inputs, and details compared to the most popular methods, which means we will not spot important usability concerns. Other attempts to include more reliability/ robustness notions are, for example, a focus on the stability of the explanation, however again, without focusing on the robustness notion [ 43 ] . In Table I , we give an overview of some of the algorithms that tackle similar goals as our chosen algorithm and why we chose this particular one. TABLE I: Overview of algorithm choices and their related concepts global model-agnostic Uncertainties Robustness usable for non-experts LIME [ 32 ] ✗ ✓ ✗ ✗ ✓ SHAP [ 23 ] ✓ ✓ ✗ ✗ ✗ Integrated gradients [ 44 ] ✗ ✗ ✗ ✗ ✓ Uncertainty DeepSHAP [ 24 ] ✗ ✓ ✓ ✗ ✓ CLUE [ 2 ] ✗ ✗ ✓ ✗ ✓ Bayesian Local Explanations [ 43 ] ✗ ✓ ✓ ✗ ✗ Unsupervised DeepView [ 30 ] ✓ ✓ ✓ ✓ ✓ While this might only be feasible for some upcoming approaches, we chose a less studied area of XAI regarding the uncertainty explanation domain. Rather than going by popularity, we chose a representative from that area that covers a variety of properties present in the XAI taxonomy. We focused on uncertainties because it has been shown that explanations that are not uncertainty-aware can often be inconsistent [ 24 ] . The common approach tackled by most XAI researchers includes works on how to extend state-of-the-art methods to incorporate explanations [ 43 ] or why XAI and Uncertainties combined help to calibrate trust [ 39 ] . We consider global explanations to be explanations that do not just cover single instances and a model’s performance on those, but overall estimate the explainability on an entire dataset [ 38 ] . It is important to get more than just local snapshots of a model’s trustworthiness. Model-agnostic states that the explanation method does not require prior knowledge of the models’ internals [ 38 ] , which was already recognized as a desirable metric in the very first works on explainable AI [ 32 ] , and has since been established [ 18 ] . The robustness that we estimate with this explanation method denotes the likelihood of adversarial attacks [ 30 ] . Adversarial attacks describe instances that are altered in an inconspicuous manner but then misclassfied [ 13 ] . We define trust calibration as the balance between human trust and the actual trustworthiness of the application [ 27 ] . We focus on Unsupervised DeepView [ 30 ] , the first unsupervised explainable AI algorithm for high dimensional data that portrays uncertainty not just by providing conventional uncertainty estimation but also by including a robustness notion. The appeal of this method is that it should help laypersons and experts directly compare the quality of one model with the quality of another model for a set task by giving a visual comparison of certainty. To improve the general understanding of XAI method improvements, we propose that it is not enough to evaluate a very general setup of all common XAI methods. Suppose that we want to evaluate new methods that fulfill niche purposes that are not covered by the most popular methods. In that case, we have to evaluate those methods in detail and then derive general recommendations from researching the usability and trust calibration abilities of specific newly proposed methods. Hence, in this study, we evaluate the explainable AI algorithm Unsupervised DeepView [ 30 ] regarding trust calibration and explanation satisfaction using the Explanation Satisfaction Scale [ 14 ] as an additional evaluation scheme for the quality of the explainability method. We check our trust