On Measuring Intrinsic Causal Attributions in Deep Neural Networks Saptarshi was a Fulbright-Nehru Doctoral Research Fellow at the University at Buffalo during this work.
Saptarshi Saha, Dhruv Vansraj Rathore, Soumadeep Saha, Utpal Garain, David Doermann · 2025
Read On Measuring Intrinsic Causal Attributions in Deep Neural Networks Saptarshi was a Fulbright-Nehru Doctoral Research Fellow at the University at Buffalo during this work. by Saptarshi Saha, Dhruv Vansraj Rathore, Soumadeep Saha, Utpal Garain, David Doermann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-09660.txt.
Introduction
Saptarshi Saha and four co-authors develop a way to measure intrinsic causal influence inside deep neural networks. Standard tools report direct, indirect, and total effects of input features. Intrinsic effects capture what a feature contributes on its own path, a finer notion that past work left out.
The team treats networks as structural causal models and gives a post hoc framework that can identify intrinsic effects after training. They also link the new measure to Sobol indices, a classic tool for sensitivity analysis. Tests on synthetic and real data show that the new scores give more intuitive and more reliable explanations than current global methods. The work aids trust in model decisions.
Catalog record
| CREATOR | Saptarshi Saha, Dhruv Vansraj Rathore, Soumadeep Saha, Utpal Garain, David Doermann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-09660
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
On Measuring Intrinsic Causal Attributions in Deep Neural Networks Saptarshi was a Fulbright-Nehru Doctoral Research Fellow at the University at Buffalo during this work. Saptarshi Saha, Dhruv Vansraj Rathore, Soumadeep Saha, Utpal Garain, David Doermann Abstract Quantifying the causal influence of input features within neural networks has become a topic of increasing interest. Existing approaches typically assess direct, indirect, and total causal effects. This work treats NNs as structural causal models (SCMs) and extends our focus to include intrinsic causal contributions (ICC). We propose an identifiable generative post-hoc framework for quantifying ICC. We also draw a relationship between ICC and Sobol’ indices. Our experiments on synthetic and real-world datasets demonstrate that ICC generates more intuitive and reliable explanations compared to existing global explanation techniques. keywords Intrinsic Causal Contribution, Causal Normalizing Flow, Sobol Indices. 1 Introduction In recent years, there has been a significant surge of interest in incorporating causal principles into deep learning models ( Pawlowski et al. 2020 ; Saha and Garain 2022 ) . Much of the existing research has focused on post-hoc explanations of trained neural networks’ decisions using causal effect analysis ( Chattopadhyay et al. 2019 ; Alvarez-Melis and Jaakkola 2017 ) . Other studies have explored counterfactuals for explanations or data augmentation ( Dash et al. 2022 ; Goyal et al. 2019b ; Reddy et al. 2023b ; Pitis et al. 2020 ) , causal disentangled representation learning ( Yang et al. 2021 ; Schölkopf et al. 2021 ; Shen et al. 2022 ) , and causal discovery methods ( Zhu et al. 2020 ) . However, despite efforts ( Chattopadhyay et al. 2019 ; Reddy et al. 2023a ; Kancheti et al. 2021 ) to quantify the causal attributions learned by neural networks, there is presently no viable method for elucidating the “intrinsic causal contribution” (ICC) ( Janzing et al. 2024 ) in neural networks. In this paper, we present a new framework based on generative models—the first of its kind, to the best of our knowledge—that quantifies intrinsic causal contributions in neural network models. To illustrate this concept of ICC, imagine a relay race with three runners: A , B , and C . Runner A starts the race and passes the baton late to runner B , who then hands it off late to runner C , who ends up finishing late as well. To determine the “intrinsic contribution” of runner B to the delay of runner C , we compare the delay of C to a situation where B only contributes the delay it inherited from A without adding any additional delay of its own. This means we’re looking at how much delay B is responsible for beyond what it received from A . This concept helps differentiate between delays that B causes itself (intrinsic) and delays it simply passes on from A (inherited). This distinction is meaningful whether we analyze the delay in a single race, the average delay across many races, or the variation in delays across multiple races. To motivate the need for studying intrinsic causal contributions in neural network models, let’s consider the task of predicting a patient’s recovery time ( R ) using the features: treatment type ( T ), initial health condition ( H ), and post-treatment care ( P ). In the real world, H influences both T and P ; while T also influences P . H , T , and P all influence R . However, these relationships among the input features H , T , and P are often not explicitly modeled in a neural network model. Now, assume that patients with severe initial health conditions are assigned to more aggressive treatment. It is possible that a neural network model might misattribute the longer recovery times directly to aggressive treatments without considering the severity of the initial health condition. Usual causal effect estimates the expected change in Recovery Time R as the treatment T changes. It doesn’t account for the effect of upstream variable H on T (due to the do-intervention on T ). With intrinsic attribution analysis, the model aims to understand the part of T ’s impact on R that is inherited from H , and the part that represents T ’s intrinsic effect. Thus, learning intrinsic causal attributions can also find application in medicine. For example, medical practitioners can look at treatments that have shown intrinsic benefits and consider optimizing these treatments for broader patient use. To this end, the aim of our work is to identify the intrinsic causal contribution of an input on the output of a neural network. Our main contributions can be summarized as follows: We introduce an identifiable framework for computing intrinsic causal attributions in neural networks, a concept previously unexplored in neural network attribution to our knowledge. In addition to Shapley-based contributions, we advocate for asymmetric ICC. In Section 5 , we demonstrate that ICC meets several desirable properties for an attribution method. In Section 6 , we establish connections between the ICC and Sobol indices, offering a fresh perspective on global sensitivity analysis from a causal viewpoint. Finally, our experiments show that the ICC produces reliable global explanations. 2 Related works Explainability Several established methods for explaining neural network models quantify the influence of input features on model outputs. These methods include saliency maps ( Simonyan et al. 2014 ; Zeiler and Fergus 2014 ; Selvaraju et al. 2017 ) , Locally Interpretable Model-Agnostic Explanations (LIME) ( Ribeiro et al. 2016a ) , Integrated Gradients ( Sundararajan et al. 2017 ) , DeepLift ( Shrikumar et al. 2017 ) , Shapley values ( Lundberg and Lee 2017 ) among others. While some of these techniques are model-agnostic, they are local in nature, meaning that the explanations are limited to individual predictions. On the other hand, global attributions are a powerful tool for interpretability because they highlight the importance of features across an entire population. They often use interpretable surrogate models like decision trees or adjust the input space to assess overall predictive power ( Lakkaraju et al. 2016 ; Frosst and Hinton 2017 ; Yang et al. 2018 ) . Submodule pick LIME (SP-LIME) ( Ribeiro et al. 2016b ) uses submodular optimization to summarize local attributions, better capturing learned interactions. However, like surrogate models, it extracts useful and independent explanations from the LIME method, which may not effectively capture the non-linear feature interactions learned by neural networks. Ibrahim et al. 2019 proposed Global Attribution Mapping (GAM) to explain the non-linear representations learned by a neural network across different subpopulations. GAM clusters similar local feature importances to create human-interpretable global attributions, each tailored to explain a specific subpopulation. Additionally, GAM allows for adjustable granularity to capture varying numbers of subpopulations in its global explanations. Permutation Feature Importance (PFI) ( Breiman 2001 ; Strobl et al. 2008 ) is another comparable measure across model types, offering a global view of the model’s reliance on each feature. However, none of these methods account for causality in their explanations. Causal Explanations Frye et al. 2021 proposed Asymmetric Shapley Values to integrate real-world causal knowledge by restricting feature permutations to those that align with a (partial) causal ordering. Heskes et al. 2020 introduced causal Shapley