Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictions
Courtney Ford, Mark T. Keane · 2025
Read Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictions by Courtney Ford, Mark T. Keane (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-06029.txt.
Introduction
Courtney Ford and Mark Keane present a method to help non-experts judge model predictions, in this 2025 paper. Explanation tools often baffle lay users. Their method picks example neighbors that best represent each class, using both local and global feature weight.
In a study with 98 users sorting Kannada script letters, the method helped people spot model errors more often and decide faster. Agreement with correct predictions stayed sound. The chosen neighbors hugged class centers more tightly than distance-only picks and varied less from run to run. A gap remained between explanation quality and user trust, which the authors flag for later work.
Catalog record
| CREATOR | Courtney Ford, Mark T. Keane |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-06029
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictions Courtney Ford, Mark T. Keane Abstract Explainable AI (XAI) methods often struggle to generate clear, interpretable outputs for users without domain expertise. We introduce Feature-Guided Neighbor Selection (FGNS), a post hoc method that enhances interpretability by selecting class-representative examples using both local and global feature importance. In a user study (N = 98) evaluating Kannada script classifications, FGNS significantly improved non-experts’ ability to identify model errors while maintaining appropriate agreement with correct predictions. Participants made faster and more accurate decisions compared to those given traditional k-NN explanations. Quantitative analysis shows that FGNS selects neighbors that better reflect class characteristics rather than merely minimizing feature-space distance, leading to more consistent selection and tighter clustering around class prototypes. These results support FGNS as a step toward more human-aligned model assessment, although further work is needed to address the gap between explanation quality and perceived trust 1 1 1 Accepted at IJCAI 2025 Workshop on User-Aligned Assessment of Adaptive AI Systems . 1 Introduction Explainable Artificial Intelligence (XAI) is crucial in high-stakes domains such as healthcare, finance, and autonomous systems, where AI model decisions can have profound real-world impacts. While methods like Local Interpretable Model-agnostic Explanations (LIME) Ribeiro et al. 2016 and Shapley Additive Explanations (SHAP) Lundberg and Lee 2017 have made strides in improving transparency, recent work has revealed fundamental limitations in these approaches Huang and Marques-Silva 2023 ; Letoffe et al. 2024 . These challenges are particularly evident in misclassification cases, where non-expert users struggle to interpret and validate model decisions Ford and Keane 2023 ; Knapič et al. 2021 . The gap between domain experts and non-experts is a key challenge in real-world AI deployments Jiang and Senge 2021 . Cognitive science research shows that experts develop superior perceptual processes that aid in analyzing domain-familiar data Chase and Simon 1973 , an advantage that disappears in unfamiliar domains Searston and Tangen 2017 . This expertise gap can significantly impact how users interpret AI decisions, especially in image-based systems, with non-experts often misinterpreting model predictions and failing to recognize subtle class boundaries Ford and Keane 2023 . Existing model-agnostic approaches often assume users can intuitively validate outputs, which can disadvantage those lacking domain expertise Szymanski et al. 2021 . While recent human-centered XAI approaches have begun addressing this gap Ehsan and Riedl 2020 , there remains a critical need for methods that support users with varying levels of domain expertise in understanding and validating AI decisions Mueller et al. 2019 ; Buçinca et al. 2020 . This is especially important in settings where human-in-the-loop assessments are required, but the user may not possess deep knowledge of the underlying task or decision context Rudin 2019 . To address these challenges, we introduce Feature-Guided Neighbor Selection (FGNS), a novel XAI method that builds on the twin-system approach Kenny et al. 2021 , which combines feature contributions from a convolutional neural network with a k-nearest neighbor (k-NN) classifier for example-based explanations. While this approach has shown promise for familiar domains such as MNIST, non-experts have been shown to struggle when applying these explanations in less familiar contexts Ford and Keane 2023 . FGNS addresses this limitation by integrating local and global feature importance to select class-representative neighbors. It combines LIME’s local explanations with Shapley Additive Global ImportancE (SAGE) Covert et al. 2020 global feature importance to identify neighbors that effectively represent class characteristics while remaining relevant to the query instance. We evaluate FGNS using experiments with Kannada-MNIST, examining its effectiveness in helping non-experts identify model errors and assess prediction confidence compared to traditional example-based explanations. This work contributes to the broader goal of making XAI more accessible to users with varying levels of domain knowledge, particularly in high-stakes domains where accurate validation of AI decisions is critical Buçinca et al. 2020 ; Severes et al. 2023 . Our investigation focuses on both user performance metrics and quantitative measures of explanation quality to understand how feature-guided approaches can help bridge the gap between model complexity and user comprehension. 2 Related Work Explainable AI has made significant strides in increasing the transparency of complex models, yet supporting non-expert users remains a key challenge Ridley 2025 ; Severes et al. 2023 . This section reviews recent advances in XAI and highlights the limitations of existing methods in supporting users with varying levels of domain expertise. 2.1 Feature Attribution and Model Transparency Despite their popularity, traditional feature attribution approaches face substantial limitations. While LIME Ribeiro et al. 2016 and SHAP Lundberg and Lee 2017 have become foundational tools in XAI, Huang and Marques-Silva 2023 highlight key inadequacies in Shapley-based explanations, particularly when applied to complex models. Letoffe et al. 2024 propose refinements to better reflect true feature contributions, while SAGE Covert et al. 2020 extends these methods by providing global importance scores that account for feature interactions across datasets. The challenge of making such methods accessible to non-technical users has motivated new directions in human-centered XAI Wang et al. 2019 . Studies show that technical feature importance scores alone are often insufficient for non-experts seeking to understand model behavior Knapič et al. 2021 , especially in high-stakes settings where reliable interpretation is critical Buçinca et al. 2020 . 2.2 Prototype and Example-Based Interpretability Example-based explanations have emerged as a promising strategy for bridging the gap between model complexity and user understanding. Prototype-based approaches like ProtoPNet Chen et al. 2019 and Prototype Wrapper Networks Kenny et al. 2023 enhance interpretability through human-friendly examples, though they frequently assume a degree of domain familiarity that non-experts may not have. This limitation is particularly apparent in unfamiliar classification tasks such as script recognition, where users may not be able to intuitively validate outputs. Earlier research in case-based reasoning has shown that examples near class boundaries often offer more insight than those selected purely based on proximity to the query Doyle et al. 2004 . This insight has influenced recent hybrid approaches that combine example selection with feature attribution. For example, L. Alfeo and Cimino 2024 bridge local counterfactuals with global importance, while Löfström et al. 2024 incorporate uncertainty to generate more calibrated explanations. Recent work by Delaney et al. 2023 critiques the assumption that plausible counterfactual explanations must be minimally different from the original input. In a user study, they found that humans often produce more substantial edits that align better with class prototypes when asked to explain misclassifications. These findings support the idea that example-based explanations should prioritize semantic clarity and class-representativeness over solely proximity, a principle that FGNS operationalizes through its prototype-guided neighbor selection. 2.3 Twin Systems for Enhanced Interpretability The twin-system framework Kenny and Keane 2019 offers an alternative route to interpretability by pairing convolutional neural networks with k-nearest neighbor classifiers. One implementation, COLE-HP Kenny et al. 2021 , uses the Hadamard product between CNN features and classification weights to select relevant neighbors, making the system’s internal reasoning more transparent. However, this approach has been shown to present difficulties for non-expert users in less familiar domains Ford and Keane 2023 . These limitations become particularly apparent in misclassification scenarios, where users need to understand decision boundaries to make informed judgments. Such findings underscore the need for methods that better support