Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning

Lisa Wimmer, Bernd Bischl & Ludwig Bothmann · 2025

Read Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning by Lisa Wimmer, Bernd Bischl & Ludwig Bothmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09137.txt.

Introduction

Lisa Wimmer and co-authors settle a dispute over what model uncertainty means in this 2025 paper. Neural networks report how unsure they are, but two camps read that signal differently. One sees ignorance; the other sees disagreement among plausible models. The authors argue both readings are right, in different learning setups. Shortcut learning, where a model leans on easy cues, proves decisive: with shortcuts present, the uncertainty signal shows up as disagreement.

Catalog record

CREATORLisa Wimmer, Bernd Bischl & Ludwig Bothmann
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09137

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Trust Me, I Know the Way: Predictive Uncertainty in the Presence of Shortcut Learning Lisa Wimmer, Bernd Bischl & Ludwig Bothmann Abstract The correct way to quantify predictive uncertainty in neural networks remains a topic of active discussion. In particular, it is unclear whether the state-of-the art entropy decomposition leads to a meaningful representation of model, or epistemic , uncertainty (EU) in the light of a debate that pits ignorance against disagreement perspectives. We aim to reconcile the conflicting viewpoints by arguing that both are valid but arise from different learning situations. Notably, we show that the presence of shortcuts is decisive for EU manifesting as disagreement. Figure 1: Examples of 3-class CMNIST3 data ( left ) with full-image color shortcut and PMNIST3 ( right ; cropped for better visibility) with 1\times 1 colored patches (shortcuts in 95% of images). 1 Introduction Safety-critical applications require models to report their uncertainty about predicted outcomes. The rise of deep learning, with its powerful-but-overconfident algorithms, has sparked rapid progress in uncertainty quantification ( Guo et al. 2017 ; Gawlikowski et al. 2023 , UQ,) . Yet, the problem of faithfully representing, measuring and communicating uncertainty remains far from solved ( Van Der Bles et al. 2019 ; Bickford Smith et al. 2024 ) . One disputed aspect is the correct approach to disaggregate uncertainty into its aleatoric (AU) and epistemic (EU) components ( Hüllermeier and Waegeman 2021 ; Gruber et al. 2023 ) . AU arises when the features X do not suffice to predict the target beyond doubt—e.g., due to omitted variables—even if we had access to infinite data. The ground-truth distribution over labels Y , \bm{\theta}^{\ast}=p(Y|X=\bm{x}) , will thus generally have positive dispersion (i.e., AU). EU, on the other hand, surrounds the approximation quality of the model h(\bm{x})=\hat{p}_{n}(Y|X=\bm{x}) based on n observations. EU is reducible in the sense that \hat{p}_{\infty} recovers p 1 1 1 We often assume a correctly specified model, equating hypotheses with parameterizations ( Draper 1995 ) . . Attributing predictive uncertainty to its sources helps understand learning dynamics and provides an inroad for targeted model improvement. The widely-used decomposition of Shannon entropy 2 2 2 Entropy is typically used with categorical target distributions. An analogous decomposition exists for variance-based UQ in regression tasks ( Depeweg et al. 2018 ; Sale et al. 2024 ) . ( Shannon 1948 ) as a measure of total uncertainty (TU) offers a neat mathematical expression but has attracted criticism for conflating distinct concepts ( Wimmer et al. 2023 ; Schweighofer et al. 2024 ; Bickford Smith et al. 2024 ) and being ineffective in practice ( Mucsányi et al. 2024 ; Fellaji and Pennerath 2024 ) . It is unclear, in particular, how well the resulting EU measure reflects lack of knowledge. Besides the uncertainty of a single model h\in\mathcal{H} about the predicted outcome (AU), there is uncertainty about which model even is the correct one (EU). Quantifying EU thus demands a bi-level framework that accounts for different hypotheses, uncertainty rising with the number of hypotheses deemed plausible ( Hofman et al. 2024 ) . The EU representation problem concerns this multiplicity and admits two competing viewpoints: Maximum uncertainty is attained when (V1) all hypotheses are equally likely or (V2) only hypotheses reflecting full confidence for one label each are assigned nonzero probability (in complete disagreement). Take the toy example of predicting a coin toss with EU concerning coin bias. Should EU be highest when the model deems any degree of bias equiprobable, or when it is sure the coin shows either heads or tails always but not which? Clarity about this worst-case scenario is necessary in uncertainty-based tasks like active learning ( Smith et al. 2023 ) and out-of-distribution (OOD) detection ( Azizmalayeri et al. 2024 ) . V1 fits a classical Bayesian view of associating uninformed beliefs with uniformity ( Dubois et al. 1996 ) , while EU as per the entropy decomposition embodies V2, becoming maximal when all hypotheses express full confidence in utter disagreement ( Shoja and Soofi 2017 ; Wimmer et al. 2023 ) . We do not aspire to resolve the philosophical dilemma but argue instead that the two EU manifestations might arise from different scenarios. More precisely, we postulate that conflicting hypotheses (V2) emerge in the presence of shortcut learning (SCL) : Since there is only one true mapping from features to targets, one hypothesis at most—or even none—can have recovered this relationship, while the others necessarily point to spurious patterns. Our experiments suggest that shortcuts in the data indeed prompt such disagreement. This observation bears important insights for learning dynamics and has not, to the best of our knowledge, been studied despite ample discussion about robust generalization ( Nagarajan et al. 2021 ; Wald et al. 2021 ; Richens and Everitt 2024 ) . We emphasize that we do not claim SCL is the sole cause of conflicting hypotheses (see Sec. 4 ); rather, our results should be taken as encouraging further research in this direction. 2 Background & Related Work Uncertainty Quantification Bayesian methods come with a natural bi-level uncertainty representation, posing a second-order probability distribution Q over first-order distributions \bm{\theta} induced by hypotheses h (for details, see App. A.1 ), and have therefore risen to gold standard in UQ ( Izmailov et al. 2021 ) . We consider the special case of finite ensembles, which can be viewed as approximately Bayesian ( Wilson and Izmailov 2020 ; Wild et al. 2023 ; Mlodozeniec et al. 2024 ) , but our argumentation holds for any approach expressing EU via multiple predictions ( Hofman et al. 2024 , induced by a distribution or set of hypotheses;) per instance. Following the entropy ( H(\cdot) ) decomposition popularized by Houlsby et al. 2011 and Kendall and Gal 2017 , EU in the Bayesian setting is measured via mutual information : \text{EU}=\underbrace{I(Y;\Theta)}_{\text{mutual information}}=\underbrace{H(\mathbb{E}_{Q}[Y|\Theta])}_{\text{entropy (TU)}}-\underbrace{\mathbb{E}_{Q}[H(Y|\Theta)]}_{\text{conditional entropy (AU)}}=\mathbb{E}_{Q}\left[D_{\text{KL}}\left(p(Y|\bm{\theta})~\|~\bar{\bm{\theta}}\right)\right], (1) where \Theta denotes the random variable of first-order probability distributions. Mutual information is equivalent to Jensen-Shannon divergence in the finite-ensemble case. The emphasis on disagreement is obvious from the Kullback-Leibler divergence term: D_{\text{KL}}(\cdot) increases with deviation between individual hypotheses p(Y|\bm{\theta}) and the consensus prediction \bar{\bm{\theta}} ( Shoja and Soofi 2017 , Eq. 3 , 5 - 6 ;) . Shortcut Learning SCL is fundamentally a problem of distribution shift and occurs when patterns picked up in training do not carry over to OOD scenarios (see Steinmann et al. 2024 , for a comprehensive discussion) . Relevant works discern stable and unstable features X_{s},X_{u}\subseteq X , with X_{u} non-causally correlated to Y , in varying terminology ( Chalupka et al. 2015 ; Eastwood et al. 2023 , e.g.,) . Those distinctions imply the desirability of predicting Y from X_{s} 3 3 3 Unstable features, while poor predictors on their own, can still boost performance ( Eastwood et al. 2023 ) . , which is not always possible 4 4 4 Jalaldoust et al. 2024 argue that situations of non-transportability , when X_{s} is not available and more (of the same) data cannot alleviate the shortcut problem, amount to a form of irreducible AU. . Relying on X_{u} instead induces shortcuts that work during training but break at deployment in OOD

Read the complete text