Feature Attribution from First Principles

Magamed Taimeskhanov, Damien Garreau · 2025

Read Feature Attribution from First Principles by Magamed Taimeskhanov, Damien Garreau (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-24729.txt.

Introduction

Magamed Taimeskhanov and Damien Garreau rebuild explanation scores for machine learning from scratch, in a 2025 paper. Feature attribution rates how much each input swayed a model's call, but judging the raters has relied on axiom lists the authors find too strict. They start instead with the simplest possible model, a yes-no indicator, and define its fair scores directly.

Stacking those atomic scores rebuilds several famous methods, each falling out from a different atomic choice. For deep rectifier networks they derive exact closed forms. The ground-up view turns a crowded shelf of heuristics into one family with a visible root, and points at fairer ways to grade explanations.

Catalog record

CREATORMagamed Taimeskhanov, Damien Garreau
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE104 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-24729

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Feature Attribution from First Principles Magamed Taimeskhanov, Damien Garreau Abstract Feature attribution methods are a popular approach to explain the behavior of machine learning models. They assign importance scores to each input feature, quantifying their influence on the model’s prediction. However, evaluating these methods empirically remains a significant challenge. To bypass this shortcoming, several prior works have proposed axiomatic frameworks that any feature attribution method should satisfy. In this work, we argue that such axioms are often too restrictive, and propose in response a new feature attribution framework, built from the ground up. Rather than imposing axioms, we start by defining attributions for the simplest possible models, i.e. , indicator functions, and use these as building blocks for more complex models. We then show that one recovers several existing attribution methods, depending on the choice of atomic attribution. Subsequently, we derive closed-form expressions for attribution of deep ReLU networks, and take a step toward the optimization of evaluation metrics with respect to feature attributions. 1 Introduction As machine learning models continue to improve, they are increasingly deployed in critical domains such as healthcare, self-driving, etc. Despite their impressive performances, deep learning models remain notoriously opaque due to their vast parameter spaces, complex designs, and reliance on a succession of non-linear transformations. These characteristics hinder human understanding of how specific outputs are produced, earning such models the label of “black boxes” ( 6 ) . In such settings, lack of transparency can undermine trust, hinder error diagnosis, and raise ethical or legal concerns. In response, the field of eXplainable AI (XAI) has emerged, aiming to bridge this gap by making model behavior more transparent and interpretable. A wide range of interpretability methods have emerged in recent years, such as LIME ( 36 ) , KernelSHAP ( 30 ) , Integrated Gradients ( 41 ) , Grad-CAM ( 39 ) and LRP ( 5 ) . These approaches typically fall under the classical feature attribution paradigm, which seeks to assign importance scores to input features based on their contribution to a model’s prediction. In computer vision, such feature attributions are often referred to as saliency maps. Despite their popularity, these methods produce inconsistent attributions for the same model and input point to explain ( 24 ) , and no widely accepted theoretical foundation has yet emerged. While some efforts ( 41 , 19 , 10 ) have attempted to formalize attribution through axiomatic frameworks, a consensus on what constitutes a “correct” explanation remains elusive. This work builds on the classical result that functions can be approximated by linear combinations of weighted indicator functions. We propose to begin by assigning atomic attributions to these indicator functions and then extend the attribution to more complex models through summation. Under mild regularity conditions, this construction yields a well-defined attribution method. This perspective leads naturally to a measure-theoretic framework for feature attribution, where explanations are represented as integrals. Our main contributions are threefold: • We analyze limitations of existing attribution axioms (Th. 2.1 ). • We propose a constructivist, measure-theoretic framework for feature attribution (Th. 3.2 ). • Within this framework, we derive closed-form attributions for deep ReLU networks (Cor. 4.1 ), recover existing methods (Table 1 ), and show how to optimize evaluation metrics as functions of attributions (Th. 4.1 ). Our implementation for computing these explanations is available online . All proofs can be found in Appendix D . 1.1 Related work Empirical analysis of XAI methods. Several empirical studies have revealed significant limitations of XAI methods, questioning in particular the reliability of saliency maps as faithful explanations ( 27 , 17 ) . Notably, ( 1 ) introduced a randomization-based sanity check, showing that some XAI methods for images are independent of both the model and the data. From a different perspective, ( 21 ) directly attacked the reliability of saliency-based attributions by adversarial model manipulation, fine-tuning a model with the purpose of making the saliency maps unreliable. Building upon these insights, several recent works have developed new evaluation frameworks. ( 24 ) and ( 37 ) proposed several evaluation strategies to assess the quality of feature attributions and developed ranking schemes to compare the methods. ( 2 ) and ( 20 ) provide practical toolkits for systematically assessing the quality of feature attribution methods, aiming to streamline and standardize the evaluation of XAI methods. Theoretical analysis of XAI methods. From the theoretical side, specific methods such as LIME ( 16 ) , attention-based explanations ( 29 ) , and KernelSHAP ( 7 ) have been analyzed, along with other work ( 4 ) covering additional methods such as DeepLIFT and \varepsilon -LRP. Some works have taken a more general approach, presenting impossibility results under certain assumptions, for instance, regarding the feasibility of counterfactual recourse and robustness ( 13 ) , or the soundness of feature attributions under specific axioms ( 8 ) . More precisely, ( 8 ) shows that, if the underlying assumptions are too general, one cannot conclude using the classical local feature attribution methods if a feature is important for the model’s prediction when its attribution is largely positive. ( 28 ) provides a decomposition for global feature attribution that disentangles a feature’s individual contribution from those arising through interactions with other features. Axiomatisation of feature attribution. Several works have proposed approaches to either unify or provide a solid foundation for XAI. ( 12 ) introduces explainability through a feature removal lens, aiming to measure how much each feature contributes to the model’s output. ( 41 ) proposes a set of axioms that any good explanation method should ideally satisfy. In the same spirit, ( 18 ) introduces two new axioms, inspired by principles from logic, that any saliency-based method should satisfy, along with evaluation metrics designed to capture both properties. ( 19 ) shows that many existing methods can be viewed as performing local approximations of the model being explained. Finally, ( 10 ) also adopts an approximation-based view, framing interpretability in terms of PAC-learning. 2 Limitations of existing axioms In this section, we explore the landscape of existing axioms for feature attribution methods and show formally that they tend to produce similar attributions, as observed by ( 4 ) . The main difference with ( 4 ) is that our result is valid for any feature attribution satisfying the axioms introduced in ( 41 ) . 2.1 Existing axioms The problem at hand is to explain the predictions of a function f:\mathcal{X}\to\mathbb{R} , with \mathcal{X} the input space, and \mathcal{F} is a vector space of functions to be explained. One common approach is to use a feature attribution method \phi , defined as follows: Definition 2.1 (Feature attribution ( 8 ) ) . A feature attribution method is any mapping \phi:\mathcal{X}\times\mathcal{F}\to\mathbb{R}^{d} , where \phi(\mathbf{x},f)_{j} represents the attribution of the j -th feature of the input \mathbf{x}\in\mathcal{X} in explaining the output f(\mathbf{x}) , with j\in\llbracket d\rrbracket\vcentcolon=\{1,2,\ldots,d\} , and f\in\mathcal{F} . Intuitively, if \phi(\mathbf{x},f)_{j}\gg 0 , then the feature \mathbf{x}_{j} is important for the model’s prediction f(\mathbf{x}) . We now list a handful of properties/axioms, found in the literature ( 41 ) , which are often taken as desirable for any attribution method \phi . Definition 2.2 (Completeness (

Read the complete text