Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations
Tabea E. Röber, Paul Festor, Rob Goedhart, Ş. İlker Birbil, Aldo Faisal · 2026
Read Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations by Tabea E. Röber, Paul Festor, Rob Goedhart, Ş. İlker Birbil, Aldo Faisal (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-09573.txt.
Introduction
Röber and four colleagues test which kinds of post-hoc explanations actually help users grasp and trust black-box models. In an online study, people faced four arms: no explanation, feature importance scores, point counterfactuals, and interval counterfactuals. Interval explanations clearly beat the rest at raising both model understanding and shown trust. Point counterfactuals, by contrast, showed no supported gain over the control group. Traits such as cognitive style shaped how well each explanation worked.
Catalog record
| CREATOR | Tabea E. Röber, Paul Festor, Rob Goedhart, Ş. İlker Birbil, Aldo Faisal |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45653 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.09573
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations Tabea E. Röber, Paul Festor, Rob Goedhart, Ş. İlker Birbil, Aldo Faisal Abstract Experimental user studies evaluating the effectiveness of different subtypes of post-hoc explanations for black-box models are largely nonexistent. Therefore, the aim of this study was to investigate and evaluate how different types of counterfactual explanations, namely single point explanations and interval-based explanations, affect both model understanding and (demonstrated) trust. We conducted an online user study using a within-subjects experimental design, where the experimental arms were (i) no explanation (control), (ii) feature importance scores, (iii) point counterfactual explanations, and (iv) interval counterfactual explanations. Our results clearly show the superiority of interval explanations over other tested explanation types in increasing both model understanding and demonstrated trust in the AI. We could not support findings of some previous studies showing an effect of point counterfactual explanations compared to the control group. Our results further highlight the role individual differences in, for example, cognitive style or personality, in explanation effectiveness. K eywords Counterfactual explanations, User study, Interpretable AI, Explainable AI INTRODUCTION Even though artificial intelligence (AI) models have enjoyed growing attention over the last decades, their implementation in high-stakes practice is still slow. A common critique centers around the difficulty to rely on such models especially in critical environments due to the black-box character of most models, which ultimately hinders user acceptance. On top of that, legal guidelines like the GDPR ( European Parliament and Council of the European Union 2016 ) and the European AI Act ( European Parliament and Council of the European Union 2024 ) , were put in place to ensure that AI-based decision processes are sufficiently transparent and intelligible, or in other words, explainable (XAI). A prominent type of explanations for predictions made by black-box models is counterfactual explanations , which show how changes in the feature space would change the model prediction. This way, users may be able to form a better understanding of the model’s decision boundaries. Specifically, algorithms finding counterfactual explanations search for a data point that is close to the original sample in terms of feature space, but which lies on the other side of the decision boundary. In a classification setting, the counterfactual would receive a different predicted class, whereas in a regression setting we would set a threshold that the counterfactual’s prediction should exceed ( Molnar 2022 ; Guidotti 2024 ) . It has been argued that counterfactual explanations are intuitive for humans because their contrastive nature ( ‘Why did the model predict class A and not class B?’ ) resembles human reasoning patterns in daily life ( Byrne 2019 ; Miller 2019 ) . To date, almost all counterfactual explanation methods generate a single counterfactual data point as explanation ( Russell 2019 ; Ustun et al. 2019 ; Kanamori et al. 2020 ; Kanamori et al. 2021 ; Karimi et al. 2020 ; Karimi et al. 2021 ; Mothilal et al. 2020 ; Mahajan et al. 2019 ; Maragno et al. 2022 , e.g. ,) . The feature values of this counterfactual can then be used to formulate an explanation in an if-then manner, e.g. ‘If x_{1} was smaller by x amount, then the model would have predicted class B’ . Instead of providing only a single counterfactual, Maragno et al. 2024 propose to generate a region of possible counterfactuals. This way, an explanation can be formulated that refers to ranges in feature values instead of single data points, potentially providing more insightful information about the model’s decisions. Methods generating these counterfactual intervals, also known as robust recourse, return a lower and upper bound for each feature. It is believed that this can be effectively translated into more user-friendly explanations that may give more agency to the user. It is further believed that those explanations might instill a broader understanding of the model behavior. At this point, many efforts have been made to propose algorithms that generate counterfactual explanations to enhance transparency of the model’s decision process and by that increasing user understanding, trust, and acceptance of the models. However, only a few works conduct user evaluations to verify their claims that their models provide human-understandable output. In fact, only 7% of papers proposing methods to generate counterfactual explanations perform any type of user evaluation ( Delaney et al. 2023 ) , and hardly any of the conducted evaluations build on insights from the social sciences ( Rong et al. 2024 ) . With this work, we aim to fill this validation gap by empirically studying how different types of counterfactual explanations impact user understanding and trust in the AI model . In an online study environment, we ask users to estimate housing prices with the help of an AI model, and provide different explanations according to their assigned experimental arm. Participants receive either no explanation (control group), feature importance scores, point counterfactuals or interval counterfactuals (see Figure 1 ). By having participants go through multiple blocks with different tasks each, we aim to address our research question. We expect that participants receiving any type of explanation show higher scores for understanding and trust. We hypothesize that interval counterfactual explanations are most effective in promoting understanding and in increasing reliance on the AI model. We further expect that reliance will generally increase when participants do not have full access to all feature information. Figure 1: Schematic overview of the study concept and design. After filling out a survey to record demographics and experience level, each participant completes three blocks consisting of multiple trials each. Instructions within a block do not change. Participants are randomly assigned to one of the four explanation groups and one of the three different view modes. Results The experiment was successfully completed by 236 participants, of which 89 identify as female (38%) and 147 as male (62%). Participant ages ranged between 22 and 69 years, with a mean age of 34.28 (std=6.93) and a median age of 35. Participation took between 16 to 30 minutes and there was no significant difference in completion time of the experiment between groups (control: 1415.79s; feature importance scores: 1515.90s; point explanations: 1502.64s; interval explanations: 1485.02s; p=0.0699 by Kruskal-Wallis test). We refer to the supplementary material for detailed results of the models discussed in this section, such as regression coefficients and assumption checks. Impact of explanation type on estimating house price In the first block of trials, participants are provided with the values of features describing a house ( e.g. surface area, number of bathrooms, energy efficiency, etc.; see Section Methods ), as well as the AI’s prediction of the house price, and the corresponding explanation. Participants are then asked to give an estimate of the true house price given this information. The target variable is the absolute difference in price between their estimate and the true house price, on which we ran a mixed effects model with a random intercept for participants. Controlling for demographics ( gender , education , age ), participants receiving interval explanations score significantly better than the control group ( \beta_{interval}=-0.164,p_{interval}<0.05 ). Other explanation groups did not outperform the control group ( \beta_{featureImportance}=-0.097,p_{featureImportance}=0.163 ; \beta_{point}=-0.123,p_{point}=0.079 ), however we do see a trend