On Spectral Properties of Gradient-based Explanation Methods

Amir Mehrpanah, Erik Englesson, Hossein Azizpour · 2025

Read On Spectral Properties of Gradient-based Explanation Methods by Amir Mehrpanah, Erik Englesson, Hossein Azizpour (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10595.txt.

Introduction

Amir Mehrpanah, Erik Englesson, and Hossein Azizpour present a 2025 paper on why the heat maps used to explain neural networks can mislead. These maps lean on gradients, the model's sensitivity signals, and their quirks were mostly folklore. The authors give the methods a formal treatment with probability and spectrum tools.

They find a pervasive spectral bias baked into gradient-based explanations. The analysis explains popular tricks, like squaring the gradient or jittering the input, that practitioners found by trial and error. It also shows how jitter size in methods like SmoothGrad can make explanations inconsistent. Two fixes follow from the theory: a principled way to set the jitter scale, and a new aggregation called SpectralLens. Tests backed the theory with numbers.

Catalog record

CREATORAmir Mehrpanah, Erik Englesson, Hossein Azizpour
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10595

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Spectral Properties of Gradient-based Explanation Methods Amir Mehrpanah, Erik Englesson, Hossein Azizpour Abstract Understanding the behavior of deep networks is crucial to increase our confidence in their results. Despite an extensive body of work for explaining their predictions, researchers have faced reliability issues, which can be attributed to insufficient formalism. In our research, we adopt novel probabilistic and spectral perspectives to formally analyze explanation methods. Our study reveals a pervasive spectral bias stemming from the use of gradient, and sheds light on some common design choices that have been discovered experimentally, in particular, the use of squared gradient and input perturbation. We further characterize how the choice of perturbation hyperparameters in explanation methods, such as SmoothGrad, can lead to inconsistent explanations and introduce two remedies based on our proposed formalism: (i) a mechanism to determine a standard perturbation scale, and (ii) an aggregation method which we call SpectralLens. Finally, we substantiate our theoretical results through quantitative evaluations. 1 Introduction Ensuring the interpretability of machine learning models is crucial for user’s trust and understanding [ 45 , 2 , 6 ] . While considerable efforts have been dedicated to advancing explainability in the field [ 62 , 47 , 37 , 57 , 49 , 51 ] , there is a notable gap in systematically formalizing and consolidating existing knowledge [ 5 , 9 , 16 , 19 ] . Insufficient formal analysis leaves too much room for different interpretations of the explanations, making them unreliable [ 27 ] which is detrimental to the very use case of explanations, i.e., human-critical applications. Therefore, more attention is essential to retrospectively analyze and structure the wealth of work done, creating a cohesive foundation for explainability [ 39 , 61 ] . One main reason for the unreliability of explanation methods is their susceptibility to hyperparameters, leading to inconsistent explanations [ 33 , 4 , 13 , 48 , 36 ] , see Fig. 1 . Among the several works that formally study explanation methods [ 57 , 19 , 37 ] , to our knowledge, none shed light on such inconsistencies. Therefore, in this work, we adopt a novel probabilistic and spectral perspective to characterize the observed inconsistencies and propose preliminary solutions. Our Contribution. We iterate our contributions in three parts: • Analysis: We introduce a probabilistic representation for post-hoc gradient-based explanation methods in Section 2.1 , and perform a spectral analysis of them in Section 3.1 . • Identified Issues: Our examination reveals a shared spectral bias stemming from the fundamental assumption of utilizing gradients to measure feature contributions. We show that the combined spectral effects of gradient and perturbation lead to band-pass filters modulating the explanations, which could lead to inconsistencies in the explanations, discussed in Section 3.2 . Moreover, we conduct a spectral analysis of a few specific perturbation distributions that have been proposed in the literature in Section 3.3 . • Solutions and Evaluations: To address inconsistencies, we investigate two straightforward solutions that are finding an optimal perturbation scale based on the cosine similarity of the perturbation and the classifier in the frequency domain in Section 4.1 , and an aggregation mechanism named SpectralLens in Section 4.2 . Finally, we evaluate the proposed methods using the pixel removal strategy [ 48 ] , acknowledging their effectiveness and limitations under Section 5 . (a) (b) (c) (d) Figure 1 : The figure illustrates sample explanations generated by varying hyperparameters of SmoothGrad-Squared [ 30 , 55 ] for ResNet-50 on ImageNet (see Fig. 16 in the Appendix for more examples). It is evident from the figure that using different hyperparameters results in diverse explanations. Such variations have been demonstrated in recent studies for other explanation techniques as well [ 33 , 4 , 13 , 48 ] . 2 Background Notation. We consider a classification setting, that is, modeling p(y|\boldsymbol{x}) for labels y\in\{1,\dots,k\} and \boldsymbol{x}\in\mathbb{R}^{n} the set of input images. We denote the \log p(y|\boldsymbol{x}) by f . We denote standard Fourier bases by \{\psi(\boldsymbol{\omega},\boldsymbol{x})\}_{\boldsymbol{\omega}\in\mathbb{R}^{n}} , according to which, under some regularity conditions [ 43 ] , f can be decomposed as f(\boldsymbol{x})=\int\widehat{f}(\boldsymbol{\omega})\psi(\boldsymbol{\omega},\boldsymbol{x})d\boldsymbol{\omega} . In this representation, \widehat{f} denotes the Fourier transform (FT) of f . We denote the power spectral density (PSD) of f by \operatorname{S}_{f}(\boldsymbol{\omega})=|\widehat{f}(\boldsymbol{\omega})|^{2} . 2.1 A Probabilistic Representation of Explanations In this view, each explanation method consists of three components: (1) a distribution for perturbation of the input, (2) an explainer, and (3) a summary statistic. We use p(\tilde{\boldsymbol{x}}|\boldsymbol{x},\sigma) to denote the distribution of perturbations of the input sample parameterized by \sigma , referred to as the perturbation scale. A definition of “explanation” is recovering the contribution of features to a prediction, which emanates from human’s natural counterfactual reasoning [ 19 ] . Therefore, the explainer is usually the gradient with respect to input \nabla_{\boldsymbol{x}} , or approximations of it, e.g . the finite difference, given by \Delta_{\boldsymbol{x}}f(\boldsymbol{x})=\frac{f(\tilde{\boldsymbol{x}})-f(\boldsymbol{x})}{\tilde{\boldsymbol{x}}-\boldsymbol{x}} (see Appendix 0.D ). Finally, we assume that the summary statistic is the expectation of the outputs, or the squared outputs, of the explainer, e.g . \left(\nabla f(\tilde{\boldsymbol{x}})\right)^{2} , over the chosen perturbation distribution, p(\tilde{\boldsymbol{x}}|\boldsymbol{x},\sigma) , denoted as \mathbb{E}_{p(\tilde{\boldsymbol{x}})}\left[(\nabla f(\tilde{\boldsymbol{x}}))^{2}\right] , where the operations are carried out pixelwise. This is referred to as the explanation of f at \boldsymbol{x} with a hyperparameter \sigma . This probabilistic view can, in fact, unify different explanation methods into a single form (see Appendix 0.C ). We provide theoretical justifications for why using squared gradients is preferable to gradients, explaining the empirical findings [ 30 ] , and will discuss both cases in Section 3.1 . 3 Spectral Analysis of Explanations This section demonstrates the spectral property in explanations when using the gradient explainer and its interaction with perturbations, where a significant difference in the frequency of learned features leads to a selective attribution of features. In the following sections, we draw on insights from spectral analysis of signals and systems theory, utilizing this viewpoint to gain a deeper understanding of explanation methods. We demonstrate a spectral bias related to gradient-based explanation approaches, identify its source, and interpret various methods in the literature as efforts to alleviate it through the incorporation of low-pass filters, either explicitly [ 55 , 57 ] or implicitly [ 62 , 42 ] . We illustrate that the inconsistencies in explanation techniques stem from different strategies for mitigating the spectral bias. Finally, we suggest two straightforward approaches to tackle these inconsistencies: determining an optimal perturbation scale in Section 4.1 and aggregating the explanations in Section 4.2 . Writing a spectral representation for prediction, as shown in Section 0.A.1 , the model output with a perturbation distribution can be written as a convolution between the function and the perturbation distribution f*p , therefore we may use “perturbation kernel” and “perturbation distribution” interchangeably. 3.1 Spectral Representation of Explainers Here, by drawing a connection between explainability techniques and the literature on spectral signal processing [ 12 , 38 , 56 ] , we take a novel avenue for our analysis. This connection enables us to have a spectral representation of the explanations, as discussed in the next sections (see Appendix 0.A for the derivations). It is important to note that the Fourier bases in our work is 1D and per

Read the complete text