Explanation User Interfaces: A Systematic Literature Review DOI: XXXXXXX.XXXXXXX Journal: SUBMITTED 5 Price: 15.00 ISBN: 978-1-4503-XXXX-X/18/06 CCS: General and reference Surveys and overviews CCS: Human-centered computing User interface design CCS: Computing methodologies Artificial intelligence
Eleonora Cappuccio, Andrea Esposito, Francesco Greco, Giuseppe Desolda, Rosa Lanzilotti, Salvatore Rinzivillo · 2025
Read Explanation User Interfaces: A Systematic Literature Review DOI: XXXXXXX.XXXXXXX Journal: SUBMITTED 5 Price: 15.00 ISBN: 978-1-4503-XXXX-X/18/06 CCS: General and reference Surveys and overviews CCS: Human-centered computing User interface design CCS: Computing methodologies Artificial intelligence by Eleonora Cappuccio, Andrea Esposito, Francesco Greco, Giuseppe Desolda, Rosa Lanzilotti, Salvatore Rinzivillo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20085.txt.
Introduction
Eleonora Cappuccio and five colleagues review explanation user interfaces, the screens that show people why an artificial intelligence system decided as it did. Many models act as black boxes. Explanation methods exist, but how their output is presented often gets little design care, leaving end users poorly served.
The systematic review gathers the designs and guidelines used across the field so far. The authors also present HERMES, a platform meant to help scholars and builders design and test such interfaces with people at the center. Their aim is practical: explanations that make systems feel transparent, fair, and trustworthy in real use, not just in the lab.
Catalog record
| CREATOR | Eleonora Cappuccio, Andrea Esposito, Francesco Greco, Giuseppe Desolda, Rosa Lanzilotti, Salvatore Rinzivillo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20085
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Explanation User Interfaces: A Systematic Literature Review DOI: XXXXXXX.XXXXXXX Journal: SUBMITTED 5 Price: 15.00 ISBN: 978-1-4503-XXXX-X/18/06 CCS: General and reference Surveys and overviews CCS: Human-centered computing User interface design CCS: Computing methodologies Artificial intelligence Eleonora Cappuccio, Andrea Esposito, Francesco Greco, Giuseppe Desolda, Rosa Lanzilotti, Salvatore Rinzivillo Abstract . Artificial Intelligence (AI) is one of the major technological advancements of this century, bearing incredible potential for users through AI-powered applications and tools in numerous domains. Being often black-box (i.e., its decision-making process is unintelligible), developers typically resort to eXplainable Artificial Intelligence (XAI) techniques to interpret the behaviour of AI models to produce systems that are transparent, fair, reliable, and trustworthy. However, presenting explanations to the user is not trivial and is often left as a secondary aspect of the system’s design process, leading to AI systems that are not useful to end-users. This paper presents a Systematic Literature Review on Explanation User Interfaces (XUIs) to gain a deeper understanding of the solutions and design guidelines employed in the academic literature to effectively present explanations to users. To improve the contribution and real-world impact of this survey, we also present a framework for Human-cEnteRed developMent of Explainable user interfaceS (HERMES) to guide practitioners and academics in the design and evaluation of XUIs. HCXAI Human-Centered eXplainable Artificial Intelligence GDPR General Data Protection Regulations LLM Large Language Model UCD user-Centered Design ML machine learning XUI Explanation User Interface HCAI Human-Centered Artificial Intelligence XAI eXplainable Artificial Intelligence AI Artificial Intelligence HCD Human-Centered Design HCI Human-Computer Interaction UI User Interface SLR Systematic Literature Review EU European Union LIME Locally Interpretable Model-agnostic Explanations SHAP SHapley Additive exPlanation CAM Class Activation Mapping Grad-CAM Gradient-weighted Class Activation Mapping PDP Partial Dependence Plot DiCE Diverse Counterfactual Explanations DARPA Defense Advanced Research Projects Agency HERMES a framework for Human-cEnteRed developMent of Explainable user interfaceS 1. Introduction Recent years have witnessed a remarkable interest in the field of AI ( AI ), which has become a prominent topic in both academia and industry. The rapid advancements in AI technologies have led to their widespread adoption across various sectors, including healthcare, finance, and transportation. The ML subfield of AI has gained particular attention due to its ability to provide discriminative and predictive capabilities, enabling the development of sophisticated applications such as image recognition, natural language processing, and autonomous systems. The increasing complexity of ML models has raised concerns about their interpretability and transparency, in particular for those models that are based on deep learning architectures. These models often operate as “black boxes,” making it challenging for users to understand their decision-making processes ( Guidotti et al. 2019 ) . This has led to a growing demand for XAI ( XAI ) techniques that can elucidate the decision-making processes of these models. Black-box ML models pose various threats to their users. For example, the risk of unfair and unjust bias increases when the model’s inner workings are not clear ( Mehrabi et al. 2022 ) . Therefore, worldwide legislative bodies are attempting to regulate the use of such systems. For example, the EU ( EU ) recently released the AI act with this goal ( European Parliament and Council of the European Union 2024 ) . Most of these regulations advocate the need for AI systems to be accountable, fair, and explainable ( European Parliament and Council of the European Union 2024 ) . In this context, XAI may be beneficial. XAI is a sub-field of AI , initially formalised by the DARPA ( DARPA ) in 2017 ( Gunning and Aha 2019 ) , which focuses on “opening” black-box models, making their inner workings clear, explainable, and interpretable ( Guidotti et al. 2019 ; Gunning and Aha 2019 ) . Although different techniques for explaining black-box models exist (as will be detailed in Section 2.1 ), a common and precise definition of “good” explanations has yet to be established. Several studies have pointed out that the majority of the work on XAI is based on researchers’ intuition of what qualifies as a “good” explanation ( Miller 2019 ; Abdul et al. 2018 ; Liao and Varshney 2021 ; Lipton 2018 ; Cheng et al. 2019 ) , effectively framing XAI primarily as an algorithmic problem. As a result, there is a gap between XAI algorithms designed by researchers and their deployment in real-word scenarios ( Liao and Varshney 2021 ) . To fill this gap, HCI ( HCI ) research has recently begun to focus on AI-enabled applications and XAI , as testified by the birth of the research field of HCAI ( HCAI ), which reframes AI-development in a human-centred perspective by focusing on users’ needs ( Shneiderman 2022 ) . Since the first conceptualisation of XAI by DARPA , the design of XAI systems followed two different stages: the design of the algorithmic model itself, and the design of the UI ( UI ) ( Chromik and Butz 2021 ) . Such UI are usually referred to as XUI ( Chromik and Butz 2021 ) . As with other types of UI , XUI must be designed following a human-centred approach to be successful and usable ( ISO 2018 ) . In this study, we present a SLR ( SLR ), following Kitchenham 2004 ’s procedure ( Kitchenham 2004 ) , that aims at providing a comprehensive overview of the design of XUI . Our overall goal is to identify design trends among human-centred XUI and XAI algorithms to provide a cohesive understanding of the HCXAI ( HCXAI ) landscape. More precisely, we contribute to the state-of-the-art systematising the knowledge on four main interrelated aspects of XUI design; (i) The XAI models used to generate the explanations, (ii) The solutions adopted to provide the explanations, (iii) The user studies and human-centred techniques used to evaluate the designs, (iv) The design guidelines and frameworks that can help designers and practitioners. This article is structured as follows: Section 2 presents key background concepts; Section 3 presents the methodology used to conduct the SLR and the four research questions that guided it; Sections 4 , 5 , 6 and 7 report the results to answer each of the four research questions; Section 8 presents a set of conclusions drawn by analyzing the dimensions of each RQ; moreover, it presents HERMES, a framework to guide practitioners in the design and evaluation of XUIs; Section 9 reports open research challenges; Section 10 presents the limitations of our study, highlighting any potential threat to its validity; finally, Section 11 concludes the article. 2. Background This section introduces the background of our work. It provides an overview of the field of XAI , reprising a classification of methods already available in the literature that can guide in understanding the results of this SLR . Furthermore, it explores the need for a human-centred process and its implications for XUI . 2.1. Explainable Artificial Intelligence The increasing use of complex and opaque AI models, such as deep learning architectures, has resulted in the emergence of “black-box” systems. The decision-making processes of such systems are often difficult to interpret, even for AI engineers. XAI confronts this issue by offering techniques that enhance the understandability of AI model predictions for humans (