Minimizing False-Positive Attributions in Explanations of Non-Linear Models

Anders Gjølbye, Stefan Haufe, Lars Kai Hansen · 2025

Read Minimizing False-Positive Attributions in Explanations of Non-Linear Models by Anders Gjølbye, Stefan Haufe, Lars Kai Hansen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-11210.txt.

Introduction

Anders Gjølbye, Stefan Haufe, and Lars Kai Hansen tackle a flaw in explanations of AI decisions in a 2025 paper. Some inputs sway a model's output without being truly linked to the answer, and explanation tools may credit them anyway. Those false credits mislead users about what the model relies on.

Their method, PatternLocal, starts from a simple local explanation, such as those from common tools, then recasts the weights into a generative form that quiets those misleading variables. In tuned tests on a standard benchmark, it beat rival explainers and cut false credits. Faithful explanations matter where models guide medical or legal choices.

Catalog record

CREATORAnders Gjølbye, Stefan Haufe, Lars Kai Hansen
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-11210

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Minimizing False-Positive Attributions in Explanations of Non-Linear Models Anders Gjølbye, Stefan Haufe, Lars Kai Hansen Abstract Suppressor variables can influence model predictions without being dependent on the target outcome, and they pose a significant challenge for Explainable AI (XAI) methods. These variables may cause false-positive feature attributions, undermining the utility of explanations. Although effective remedies exist for linear models, their extension to non-linear models and instance-based explanations has remained limited. We introduce PatternLocal, a novel XAI technique that addresses this gap. PatternLocal begins with a locally linear surrogate, e.g., LIME, KernelSHAP, or gradient-based methods, and transforms the resulting discriminative model weights into a generative representation, thereby suppressing the influence of suppressor variables while preserving local fidelity. In extensive hyperparameter optimization on the XAI-TRIS benchmark, PatternLocal consistently outperformed other XAI methods and reduced false-positive attributions when explaining non-linear tasks, thereby enabling more reliable and actionable insights. We further evaluate PatternLocal on an EEG motor imagery dataset, demonstrating physiologically plausible explanations. 1 Introduction In recent years, the growing demand for transparent, accountable, and ethical AI systems across various industries and research communities has driven substantial advancements in explainable artificial intelligence (XAI). The need to understand machine learning models’ decision-making processes is well-established, particularly in high-stakes domains such as healthcare, finance, and criminal justice. While researchers have made promising progress with so-called self-explainable models such as prototypical networks, these approaches often fail to achieve the same downstream performance as conventional models and typically require complete retraining [ 22 ] . Consequently, researchers focus significant attention on post-hoc XAI methods that apply to existing models without retraining. Figure 1 : Conceptual step-by-step overview of how the PatternLocal method generates local explanations. A classifier is trained on the dataset (a), with both the dataset and the model’s decision boundary visualized in a 2D projection. Many XAI methods produce importance maps (c) through local linearization (e.g., LIME, KernelSHAP). PatternLocal enhances this process by transforming the local discriminative surrogate into a generative explanation (d), thereby reducing the influence of irrelevant or misleading features. The resulting importance maps (c–d) can be directly compared with the ground-truth attributions (b). Post-hoc XAI methods encompass a diverse array of approaches, including architecture-specific techniques such as Layer-wise Relevance Propagation (LRP) [ 1 ] , concept-based explanations like TCAV [ 14 ] , and model-agnostic frameworks such as SHAP [ 17 ] and LIME [ 21 ] . These methods promise to support model validation, data quality checks, and actionable intervention suggestions [ 23 ] . To provide evidence, researchers evaluate XAI methods against criteria such as faithfulness to the model, robustness to perturbations, and human interpretability [ 32 ] . However, none of these criteria guarantee that the features highlighted as important are discriminative for the target variable, an assumption behind many uses of XAI [ 13 ] . Several studies [ 34 , 35 ] have shown that widely used XAI methods often assign importance to suppressor variables [ 9 , 33 ] , features that affect model predictions without having a direct statistical dependency with the target. Suppressors may capture correlations in noise or other auxiliary signals that models exploit to improve predictions without offering causal or correlational insights. For instance, a model might predict epilepsy from seizure activity in a specific brain region, but to optimally do so, it also relies on a noise probe from an unaffected region; common XAI methods may then highlight both regions, mistakenly assigning significance to the irrelevant area. To support interpretations that rely on statistical relevance, a feature should depend on the target variable [ 34 , 13 ] . The Pattern method [ 12 ] formalizes this idea by separating prediction from explanation: a discriminative model makes predictions, and a corresponding generative model provides explanations. Extensions such as PatternNet and PatternAttribution generalize this idea to deep neural networks [ 15 ] but have shown weaknesses on non-linear suppressor benchmarks like XAI-TRIS [ 8 ] . Unlike human-centered evaluations, benchmarks like XAI-TRIS provide controlled environments with objective ground-truth explanations to evaluate the robustness of XAI methods against suppressors and other uninformative variables. Also, model-agnostic methods such as SHAP, LIME, and LRP largely ignore training data structure, making them prone to suppressor bias [ 34 , 35 , 8 ] . In this work, we introduce PatternLocal , a data-driven, model-agnostic XAI method that extends the Pattern approach to non-linear cases. PatternLocal converts locally discriminative explanations, produced by methods such as LIME or KernelSHAP, into generative representations that more accurately reflect statistical relevance; see Figure 1 . This transformation helps reduce the impact of suppressor and other non-informative variables while maintaining local fidelity to the model’s decision behavior. PatternLocal assumes access to representative training data and a meaningful input simplification (e.g., superpixels or feature masks) shared across samples. We evaluate PatternLocal on the XAI-TRIS benchmark, artificial lesion MRI benchmark, and an EEG Motor imagery dataset, and compare it with a range of established XAI methods. The results show that PatternLocal consistently provides more reliable and interpretable explanations without changing the underlying model. 2 Preliminary 2.1 Notation Let \mathcal{X}\subset\mathbb{R}^{D} and \mathcal{Y}\subset\mathbb{R} denote the input and output spaces, respectively. Consider a model f:\mathcal{X}\rightarrow\mathcal{Y} . For an instance \mathbf{x_{\star}}\in\mathcal{X} we wish to explain the prediction f(\mathbf{x_{\star}}) . Because highly complex models such as deep networks are notoriously hard to interpret, they are approximated locally by a simpler, intrinsically interpretable model g . We therefore restrict attention to local explanations for the specific instance \mathbf{x_{\star}} rather than to the behavior of f on \mathcal{X} as a whole. Many XAI methods first replace \mathbf{x_{\star}} with a lower-dimensional or more semantically meaningful representation \mathbf{x^{\prime}_{\star}}=h_{\mathbf{x_{\star}}}(\mathbf{x_{\star}})\in\mathbb{R}^{D^{\prime}} . In image analysis, for example, \mathbf{x^{\prime}_{\star}} may encode superpixels; in other settings, it may correspond to a low-rank projection or simply a subset of raw features. Such simplification often makes for easier interpretation and, when D^{\prime}\ll D , also reduces the computational cost. See Appendix A for a summary of the notation. Remark. In general, models that predict y from \mathbf{x} are termed backward or discriminative , whereas models that reconstruct \mathbf{x} (or its sources) from latent factors are called forward or generative . We adopt this convention to avoid terminological clashes. 2.2 Suppressor Variables Various studies [ 9 , 12 , e.g.,] demonstrated that linear models may need to assign significant non-zero weight to correlated noise variables (suppressors). Given \mathbf{x}\in\mathbb{R}^{D} , assume each instance is generated from a linear model of the form \mathbf{x}\;=\;\mathbf{A}\,\mathbf{m}\;+\;\bm{\varepsilon},\qquad\mathbf{m}=\bigl[m_{1},\dots,m_{K}\bigr]^{\!\top}\in\mathbb{R}^{K}, (1) where K is the number of latent factors, \mathbf{A}=[\mathbf{a}_{1},\dots,\mathbf{a}_{K}]\in\mathbb{R}^{D\times K} contains the activation patterns \mathbf{a}_{k} , and \bm{\varepsilon} denotes additive noise. A linear backward model attempts to recover the latent vector \mathbf{m} via \hat{\mathbf{m}}\;=\;\mathbf{W}^{\!\top}\mathbf{x},\qquad\mathbf{W}\in\mathbb{R}^{D\times K}. (2) Consider D=2 and K=1 with x_{1}=m+d and x_{2}=d , where the scalar m is the signal of interest and d is a suppressor . The signal is exactly recovered by \hat{m}=x_{1}-x_{2}=\mathbf{w}^{\!\top}\mathbf{x} with \mathbf{w}=[1,-1]^{\!\top} . Interpreting \mathbf{w} directly as feature importance, however, suggests x_{1} and x_{2} contribute equally, even though the second channel carries only the suppressor, which is removed entirely in \hat{m} by subtraction. This issue with suppressor variables extends to common explainable AI methods used to interpret machine learning models [ 35 , 8 ] . Remark. While suppressor variables can improve

Read the complete text