Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretability
Shichang Zhang, Tessa Han, Usha Bhalla, Himabindu Lakkaraju · 2025
Read Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretability by Shichang Zhang, Tessa Han, Usha Bhalla, Himabindu Lakkaraju (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-18887.txt.
Introduction
Shichang Zhang and co-authors argue that three fields explaining AI are really doing the same thing under different names. Their 2025 position paper deals with attribution: crediting a model's behavior to something. One field credits input features. A second credits training data. A third credits inner components of the model. Each grew up apart, with its own terms and tools.
The authors compare popular methods from all three camps. The same tricks recur everywhere: perturbing inputs, following gradients, and linear approximations. What differs is the target of the credit, not the technique. That shared core, they argue, is a resource. It lets each field borrow checks and fixes from the others. It also points to new work in model editing, steering, and regulation. The paper is a call to unify a fragmented landscape.
Catalog record
| CREATOR | Shichang Zhang, Tessa Han, Usha Bhalla, Himabindu Lakkaraju |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-18887
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Towards Unified Attribution in Explainable AI, Data-Centric AI, and Mechanistic Interpretability Shichang Zhang, Tessa Han, Usha Bhalla, Himabindu Lakkaraju Abstract The increasing complexity of AI systems has made understanding their behavior critical. Numerous interpretability methods have been developed to attribute model behavior to three key aspects: input features, training data, and internal model components, which emerged from explainable AI , data-centric AI , and mechanistic interpretability , respectively. However, these attribution methods are studied and applied rather independently, resulting in a fragmented landscape of methods and terminology. This position paper argues that feature, data, and component attribution methods share fundamental similarities, and a unified view of them benefits both interpretability and broader AI research. To this end, we first analyze popular methods for these three types of attributions and present a unified view demonstrating that these seemingly distinct methods employ similar techniques (such as perturbations, gradients, and linear approximations) over different aspects and thus differ primarily in their perspectives rather than techniques. Then, we demonstrate how this unified view enhances understanding of existing attribution methods, highlights shared concepts and evaluation criteria among these methods, and leads to new research directions both in interpretability research, by addressing common challenges and facilitating cross-attribution innovation, and in AI more broadly, with applications in model editing, steering, and regulation. A Preprint 1 Introduction The escalating complexity of AI systems necessitates a deeper understanding of their behavior, giving rise to the field of AI interpretability . A central question in interpretability research is how to attribute an AI system’s behavior to its underlying factors. To address this question, researchers across different domains have developed various attribution methods. Feature attribution (FA) , which emerged from explainable AI , quantifies the influence of input features on a model’s output at test time to reveal critical decision drivers ( Zeiler and Fergus, 2014 ; Ribeiro et al., 2016 ; Horel and Giesecke, 2020 ; Horel and Giesecke, 2022 ; Lundberg and Lee, 2017 ; Smilkov et al., 2017 ) . Data attribution (DA) , a core approach in data-centric AI , analyzes how training data points shape overall model behavior during the training process ( Koh and Liang, 2017 ; Ghorbani and Zou, 2019 ; Ilyas et al., 2022 ) . Component attribution (CA) , a key branch of mechanistic interpretability , examines how internal model components, such as neurons or layers within a neural network (NN), contribute to model behavior ( Vig et al., 2020 ; Meng et al., 2022 ; Nanda, 2023 ; Shah et al., 2024 ) . Although numerous attribution methods and comprehensive survey papers have been published ( Guidotti et al., 2018 ; Covert et al., 2021 ; Wang et al., 2024 ; Hammoudeh and Lowd, 2024 ; Bereska and Gavves, 2024 ; Rai et al., 2024 ; Räuker et al., 2023 ; Vilas et al., 2024 ) , these methods have largely been studied and applied in isolation by their respective communities, resulting in a fragmented landscape of methods and terminology for similar concepts ( Saphra and Wiegreffe, 2024 ) . We take the position that 1) feature, data, and component attribution share core techniques despite their different perspectives, and that 2) this more unified view advances both interpretability, by overcoming common challenges and stimulating cross-attribution innovation, and broader AI research, with applications in model editing, steering, and regulation. In the following sections, we first define the attribution problem and formalize a unified framework that encompasses all three types of attributions (§ 2 ). Next, we examine each attribution type, analyzing its popular methods, and demonstrate how all three attribution types share fundamental techniques (i.e., perturbations, gradients, and linear approximations), key concepts, and evaluation criteria (§ 3 ). We summarize representative attribution methods based on their techniques in Table 1 . Then, we demonstrate how this unified view can advance both interpretability and broader AI research (§ 4 ). We also discuss alternative views of the paper’s position (§ 5 ). In summary, we believe that this unified view enhances our understanding of attribution methods, bridges the current fragmented landscape, and provides new insights and research directions. 2 The attribution problem Various attribution methods have been developed to analyze model behavior from different aspects. In this section, we formally introduce three types of attribution problems and show how they fall under a unified framework. Consider a learning problem with d -dimensional input features \mathbf{x}=[x_{1},x_{2},\ldots,x_{d}] . During training, a dataset of n data points: \mathcal{D}_{\text{train}}=\{\mathbf{x}^{(1)},\mathbf{x}^{(2)},\ldots,\mathbf{x}^{(n)}\} is used to train a model f_{\theta} with parameters \theta and components c=\{c_{1},c_{2},\ldots,c_{m}\} by optimizing the loss function \mathcal{L}(\theta) . At test (inference) time, the model generates an output f_{\theta}(\mathbf{x}^{\text{test}}) for a new input \mathbf{x}^{\text{test}} . For notational simplicity, we omit \theta and “test” and use f and \mathbf{x} when the context is unambiguous. A notation summary is provided in Appendix A . The core objective of all three problems is to attribute the model’s output f(\mathbf{x}) to different aspects and quantify their influence with attribution scores . Feature attribution (FA) quantifies how input features affect the model’s output. A features x_{i} may be pixels in images, tokens in text, or other domain-specific units with \phi_{i}(\mathbf{x}) as its attribution score. Data attribution (DA) analyzes how training data shape model behavior during training. We quantify the influence of each training point \mathbf{x}^{(j)}\in\mathcal{D}_{\text{train}} through its attribution score \psi_{j}(\mathbf{x}) . Component attribution (CA) studies the role of model components in producing model outputs. The component c_{k} can have various definitions, like a neuron in an NN, with \gamma_{k}(\mathbf{x}) as its attribution score. As illustrated in Figure 1 , these three attribution problems share a fundamental connection: they all seek an attribution function g that assigns scores to specific aspects (features x_{i} , training points \mathbf{x}^{(j)} , or components c_{k} ) for a given test output f(\mathbf{x}) , differing only in the choice of aspects. Figure 1: The three types of attribution: FA, DA, and CA. While each type seeks to attribute a model’s output to a different aspect (input features, training data, and model components) and provides complementary insight into model behavior, they all use shared techniques (perturbations, gradients, and linear approximations). Table 1: Representative FA, DA, and CA methods classified by core techniques into three categories showing a unified view that these three types differ mainly in perspective but share techniques. Technique Technique Details Feature Attribution Data Attribution Component Attribution Perturbation Direct Occlusions ( Zeiler and Fergus, 2014 ) LOO ( Cook and Weisberg, 1982 ) Causal Tracing ( Meng et al., 2022 ) RISE ( Petsiuk, 2018 ) Path Patching ( Wang et al., 2022 ) Vig et al. (2020) Bau et al. (2020) ACDC ( Conmy et al., 2023 ) Game-Theoretic SHAP ( Lundberg and Lee, 2017 ) Data Shapley ( Ghorbani and Zou, 2019 ) Neuron Shapley ( Ghorbani and Zou, 2020 ) (Shapley) TMC Shapley ( Ghorbani and Zou, 2019 ) KNN Shapley ( Jia et al., 2019 ) Beta Shapley ( Kwon and Zou, 2022 ) Game-Theoretic STII ( Dhamdhere et al., 2019 ) Data Banzhaf ( Wang and Jia, 2023 ) – (Others) BII ( Patel et al., 2021 ) Core Value ( Yan and Procaccia, 2021 ) Myerson Value ( Chen et al.,