REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability
Kristoffer K. Wickstrøm, Thea Brüsch, Michael C. Kampffmeyer, Robert Jenssen · 2024
Read REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability by Kristoffer K. Wickstrøm, Thea Brüsch, Michael C. Kampffmeyer, Robert Jenssen (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-08513.txt.
Introduction
Kristoffer Wickstrøm, Thea Brüsch, Michael Kampffmeyer, and Robert Jenssen improve how explanation tools report their own certainty, in a research paper from 2024. These tools highlight which pixels drive a model's view of an image. Knowing whether a pixel is surely important matters for trust, but older tools only show how much scores vary.
Their method, REPEAT, runs an explanation tool many times and treats each pixel as a coin flip between important and not. That yields a direct estimate of importance plus a certainty score. In their tests, the certainty scores were easier to read than older measures and better at flagging images unlike the training data.
Catalog record
| CREATOR | Kristoffer K. Wickstrøm, Thea Brüsch, Michael C. Kampffmeyer, Robert Jenssen |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-08513
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
REPEAT: Improving Uncertainty Estimation in Representation Learning Explainability Kristoffer K. Wickstrøm, Thea Brüsch, Michael C. Kampffmeyer, Robert Jenssen Abstract Incorporating uncertainty is crucial to provide trustworthy explanations of deep learning models. Recent works have demonstrated how uncertainty modeling can be particularly important in the unsupervised field of representation learning explainable artificial intelligence (R-XAI). Current R-XAI methods provide uncertainty by measuring variability in the importance score. However, they fail to provide meaningful estimates of whether a pixel is certainly important or not. In this work, we propose a new R-XAI method called REPEAT that addresses the key question of whether or not a pixel is certainly important. REPEAT leverages the stochasticity of current R-XAI methods to produce multiple estimates of importance, thus considering each pixel in an image as a Bernoulli random variable that is either important or unimportant. From these Bernoulli random variables we can directly estimate the importance of a pixel and its associated certainty, thus enabling users to determine certainty in pixel importance. Our extensive evaluation shows that REPEAT gives certainty estimates that are more intuitive, better at detecting out-of-distribution data, and more concise. Code is available at https://github.com/Wickstrom/REPEAT . Introduction Representation learning through self-supervision is the cornerstone of recent improvements in the computer vision domain ( Caron et al. 2018 ; He et al. 2022 ; Assran et al. 2023 ; Bardes, Ponce, and LeCun 2022 ) . Transforming images into a new representation has been shown to improve performance in a wide range of unsupervised tasks ( Trosten et al. 2023 ; Wickstrøm et al. 2023 ; Sehwag, Chiang, and Mittal 2021 ) . Despite the benefits, unsupervised representation learning also suffers from some significant drawbacks, particularly a lack of explainability. Current methods in explainable artificial intelligence (XAI) are designed with supervised learning in mind, where a scalar model output is explained in relation to its input ( Petsiuk, Das, and Saenko 2018 ; Bach et al. 2015 ; Sundararajan, Taly, and Yan 2017 ) . Using these methods to explain representations is either not possible or requires major modifications of the underlying algorithms ( Crabbé and van der Schaar 2022 ) . Tackling this drawback has lead to a new direction within XAI, namely representation learning XAI (R-XAI). Methods within R-XAI solve the problem of explaining representations by either making adaptations of existing XAI methods ( Crabbé and van der Schaar 2022 ) or by designing new methods that are particularly designed to tackle the representation learning setting ( Wickstrøm et al. 2023 ; Lin et al. 2023 ; Bertolini, Clevert, and Montanari 2023 ; Møller et al. 2024 ) . A key ingredient in recent R-XAI research is uncertainty estimation ( Wickstrøm et al. 2023 ) , where importance is accompanied by a corresponding uncertainty estimate. Providing an indication of certainty is highly desirable, for instance in safety critical areas such as healthcare ( Tonekaboni et al. 2019 ; Kompa, Snoek, and Beam 2021 ) . However, existing frameworks are limited to measuring the variation in the importance scores. This only gives an indication of how the numerical importance scores spread out, not if we are certain of importance. A more critical aspect is how certain are we that a pixel is important . Consider an estimated importance map, where all pixels with importance scores higher than 2 are considered important. Now, take one pixel with importance value 5.6\pm 0.1 and another with importance value 5.6\pm 1.2 . Due to the higher variance of the second pixel, current R-XAI methods would assign high uncertainty to this pixel. However, since all values within the 95% confidence interval of the pixel would still be above the importance threshold. As such, we would still be certain that this pixel is important, despite the higher uncertainty of the exact value. Fig. 1 shows an example that illustrates the distinction between these two questions, where a prior methods is compared to our proposed solution. The example shows which pixels are important for the representation of this image and corresponding uncertainty estimates that indicate how certain the importance is. Both methods mostly agree on the important input pixels, but have vastly different estimates of uncertainty. This shows the effect of modeling certainty in importance, as opposed to variability in importance scores. Figure 1: Motivating example to show the difference between current uncertainty estimation techniques in R-XAI and our proposed approach. An image from Pascal-VOC ( Everingham et al. 2009 ) is encoded into a new representation using a ResNet18 feature extractor ( He et al. 2016 ) and important pixels are determined. The importance (red indicates high and blue indicates low importance) is accompanied by uncertainties that specify how confident the importance of a pixel is. The results show that while the importance estimates somewhat agree, the uncertainty estimates are very different, which is due to the different type of uncertainty being captured (variation in pixel importance vs. certainty of importance). With the aim of answering the question of whether we are certain that a pixel is important or not, we present a new R-XAI method called REPEAT. The key idea of REPEAT is to consider each input pixel as a Bernoulli random variable (RV) that indicates if the pixel is important for the representation of the input image. To generate samples of these Bernoulli RVs, we leverage the stochasticity of prior R-XAI methods combined with classic image thresholding techniques to threshold an image into important and non-important pixels. By repeating the thresholding process on numerous importance estimates, a set of Bernoulli samples can be generated to estimate the probability of a pixel being important to the representation of an image and a corresponding uncertainty estimate which indicates how certain we are of the pixel being important or not. Fig. 2 shows an overview of the proposed REPEAT framework. Our contributions are: 1. A new R-XAI framework called REPEAT that models the importance of a pixel for the representation of an image as a Bernoulli RV. This RV indicates the probability of the pixel being important or not and indicates the certainty of the pixel being important. 2. Extensive evaluation across numerous feature extractors and datasets and comparison with state-of-the-art baselines. Results show that REPEAT produces more intuitive uncertainty estimates that are better at detecting out-of-distribution data and has lower complexity, compared to other state-of-the-art methods. 3. Evaluation on a downstream task where uncertainty is used to detect poisoned data in the unsupervised representation learning setting ( He, Zha, and Katabi 2023 ) . Figure 2: Overview of REPEAT. An image is transformed into a new representation and pixel importance is estimated using stochastic R-XAI. A histogram is constructed from pixel importance and Bernoulli samples are generated by thresholding the importance scores into foreground and background. Then, importance and uncertainty is estimated using the Bernoulli samples. Related Work Existing R-XAI literature: There are two main approaches to extending the field of XAI to handle representations of data; adapt existing methods to handle the representation learning setting or design new methods designed for this particular use case. For adaptation approaches, Crabbé and van der Schaar 2022 proposed Label-Free Feature Importance, where an auxiliary scalar function allows standard XAI-methods to be used