Towards Balancing Preference and Performance through Adaptive Personalized Explainability Conference: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction; March 11–14, 2024; Boulder, CO, USA Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Int
Andrew Silva, Pradyumna Tambwekar, Mariah Schrum, Matthew Gombolay · 2025
Read Towards Balancing Preference and Performance through Adaptive Personalized Explainability Conference: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction; March 11–14, 2024; Boulder, CO, USA Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Int by Andrew Silva, Pradyumna Tambwekar, Mariah Schrum, Matthew Gombolay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13856.txt.
Introduction
Andrew Silva, Pradyumna Tambwekar, Mariah Schrum, and Matthew Gombolay study how robots should explain their decisions to different users, in a 2025 paper. Explanations build trust and help human-robot teams, but most systems use one fixed style. Users differ in what they like and what helps them perform.
Two user studies in a simulated self-driving setting compared three explanation styles: text, feature-importance maps, and decision trees. Preference and performance both differed by style, and what people liked did not always match what helped them most. The team built an adaptive strategy that balances the two and gained clear performance improvements.
Catalog record
| CREATOR | Andrew Silva, Pradyumna Tambwekar, Mariah Schrum, Matthew Gombolay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 71 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13856
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Balancing Preference and Performance through Adaptive Personalized Explainability Conference: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction; March 11–14, 2024; Boulder, CO, USA Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’24), March 11–14, 2024, Boulder, CO, USA Price: 15.00 DOI: 10.1145/3610977.3635000 ISBN: 979-8-4007-0322-5/24/03 CCS: Human-centered computing Human computer interaction (HCI) CCS: Computing methodologies Artificial intelligence Andrew Silva, Pradyumna Tambwekar, Mariah Schrum, Matthew Gombolay Abstract . As robots and digital assistants are deployed in the real world, these agents must be able to communicate their decision-making criteria to build trust, improve human-robot teaming, and enable collaboration. While the field of explainable artificial intelligence (xAI) has made great strides to enable such communication, these advances often assume that one xAI approach is ideally suited to each problem (e.g., decision trees to explain how to triage patients in an emergency or feature-importance maps to explain radiology reports). This fails to recognize that users have diverse experiences or preferences for interaction modalities. In this work, we present two user-studies set in a simulated autonomous vehicle (AV) domain. We investigate (1) population-level preferences for xAI and (2) personalization strategies for providing robot explanations. We find significant differences between xAI modes (language explanations, feature-importance maps, and decision trees) in both preference ( p < 0.01 p<0.01 ) and performance ( p < 0.05 p<0.05 ). We also observe that a participant’s preferences do not always align with their performance, motivating our development of an adaptive personalization strategy to balance the two. We show that this strategy yields significant performance gains ( p < 0.05 p<0.05 ), and we conclude with a discussion of our findings and implications for xAI in human-robot interactions. 1. Introduction As robots and digital assistants are deployed to the real world, these agents must be able to communicate their decision-making criteria to build trust, improve human-robot teaming, and enable collaboration ( Boies et al. 2015 ; Paleja et al. 2021 ) . Researchers have identified explainability as a necessary component of high-quality human-robot interactions in many domains ( Doshi-Velez and Kim 2017 ; Rudin et al. 2021 ) . While several approaches for explainability are under active investigation (e.g., natural language explanations ( DeYoung et al. 2019 ) , decision-tree extraction ( Silva et al. 2020 ) , counterfactual presentation ( Karimi et al. 2021 ) , saliency-based explanations ( Suau et al. 2020 ; Ribeiro et al. 2016 ) , etc.), existing studies on human-use of explanations is almost entirely confined to treating explanation as a “one-size-fits-all” problem ( Madumal et al. 2020 ; Poursabzi-Sangdeh et al. 2021 ; Mullenbach et al. 2018 ) . However, explanations have different functional roles with respect to deployment context ( Arya et al. 2019 ; Gilpin et al. 2022 ) , suggesting that personalization and contextualization of explanations is an important and understudied avenue to bring explainability to the real world. If individual preferences and expertise affect the success of an explanation ( Tambwekar and Gombolay 2023 ) , a natural next step is to identify which xAI modalities should be shown to an individual user for any given decision. Within the field of xAI, simply measuring the accuracy or fidelity of an explanation (with respect to the underlying agent or algorithm) is not enough to know that an explanation was useful. If explanations do not carefully consider a user’s expertise or expectations, the simple act of showing an explanation can cause the user to blindly trust an agent’s advice, leading to adverse effects on performance and trust ( Poursabzi-Sangdeh et al. 2021 ; Silva et al. 2022c ) . This counter-intuitive result presents a key problem: explanations encourage inappropriate compliance. If users see explanations and defer to robots without critically examining the robot suggestion, then researchers must develop a deeper understanding of the relationship between explanations and compliance while also improving an xAI agent’s ability to expose faulty decision-making to human users ( Ehsan and Riedl 2021 ) . By improving our understanding of such relationships and by better calibrating to end-users, we can produce xAI systems that are not only easier and more enjoyable to use, but also improve outcomes and efficiency of human users ( Schlicker and Langer 2021 ; Ferrario and Loi 2022 ) . Ultimately, xAI research seeks to help humans understand when to rely on vs. override their AI assistants, using explanations to determine if decisions are sound and trustworthy ( Hoffman et al. 2018 ) . Such a dynamic exists when humans collaborate with fallible AI assistants– a scenario that we recreate in this work. Our work aims to understand the diverse preferences of untrained humans with potentially-faulty assistants that use xAI to support human decision making. We present a set of studies in which participants interact with a virtual AV to navigate through an unknown city with the assistance of a digital agent, replicating a common problem of navigating in a new place. Crucially, this assistive agent is not always correct, and incorrect advice is signalled with the inclusion of red-herring features (e.g., if the agent refers to “weather” in its explanation, the suggestion is wrong). Our work therefore simulates the use of xAI for explanatory debugging ( Das et al. 2021a ; Das et al. 2023b ) with concept-based explanations ( Das et al. 2023a ) , also called the “glitch detector task” ( Hoffman et al. 2001 ; Hoffman et al. 2023 ) . We investigate how xAI may improve people’s mental models for AI ( Anderson et al. 2020 ; Brachman et al. 2023 ) , and how personalized xAI will affect people’s ability to accurately identify when their assistant is correct or incorrect (i.e., if the agent adapts to the user, will the user make fewer mistakes?). Our contributions include: (1) We design two studies in which participants interact with xAI modalities randomly or using personalization. (2) We empirically study participants’ preferences for certain types of explanations, as well as their performance with such explanations, finding that language explanations are significantly preferred ( p<0.05 ) and lead to fewer mistakes ( p<0.05 ) relative to other modalities. (3) We develop a novel adaptive personalization approach to dynamically balance a participant’s preference- and performance-based needs depending on their progress in a task. (4) We find that adapting to a participant’s preferences while also maximizing their performance leads to fewer mistakes relative to naive, randomly-chosen explanations or a preference-maximization approach ( p<0.05 ), and leads to significantly greater perceptions of preference-accommodation relative to an agent that does not personalize ( p<0.05 ). Unlike prior work on personalizing xAI, which only considers user-preferences ( Conati et al. 2021 ; Lai et al. 2023 ; Li et al. 2023 ; Kouki et al. 2020 ) , our work is the first to directly account for the user’s task-performance when personalizing xAI. Our work takes a crucial first step towards understanding how future work should consider personalization in xAI as well as why such personalization matters. 2. Related work In this work, we investigate the effects of personalizing xAI mechanisms to users, focusing on three primary modalities for explanation: language generation and counterfactuals ( Karimi et al. 2020 ; Karimi