Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing

Fabian Denoodt, José Oramas · 2025

Read Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing by Fabian Denoodt, José Oramas (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15583.txt.

Introduction

Fabian Denoodt and José Oramas calibrate the uncertainty of neural networks with a cheap smoothing trick in this 2025 paper. Knowing when a model is unsure is as vital as its raw accuracy. The strongest uncertainty methods, though, often cost heavy computing.

Their setting covers tasks where each input splits into parts that carry the same answer. Voice clips split by time are one such case. The spread among part-level predictions turns out to mirror true uncertainty. Scaling by that spread yields estimates that rival costly methods. The method adds almost no computing burden to a trained model. It suits live systems where speed and honest doubt both matter.

Catalog record

CREATORFabian Denoodt, José Oramas
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-15583

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing Fabian Denoodt, José Oramas Abstract Since state-of-the-art uncertainty estimation methods are often computationally demanding, we investigate whether incorporating prior information can improve uncertainty estimates in conventional deep neural networks. Our focus is on machine learning tasks where meaningful predictions can be made from sub-parts of the input. For example, in speaker classification, the speech waveform can be divided into sequential patches, each containing information about the same speaker. We observe that the variance between sub-predictions serves as a reliable proxy for uncertainty in such settings. Our proposed variance-based scaling framework produces competitive uncertainty estimates in classification while being less computationally demanding and allowing for integration as a post-hoc calibration tool. This approach also leads to a simple extension of deep ensembles, improving the expressiveness of their predicted distributions. The code to replicate this work is available via GitHub 1 1 1 https://github.com/anonymoususerforpeerreview/Variance-Based-Softmax-Scaling . Figure 1: Overview of our method. We use a pre-trained classifier and variance between logits of sub-patches as an uncertainty proxy to recalibrate uncertainty estimates. 1 Introduction Deep neural networks (NNs) have achieved remarkable performance across a wide range of machine learning tasks and are increasingly used in critical applications [ Krizhevsky et al., 2012 , Ioffe, 2015 , Miotto et al., 2018 ] . However, while they excel in predictive accuracy, they often struggle to quantify uncertainty reliably, leading to overconfident predictions [ Lakshminarayanan et al., 2017 , Liu et al., 2023 , Guo et al., 2017 ] . This is a serious problem in high-stakes domains such as healthcare and autonomous systems, where well-calibrated confidence estimates are crucial. A perfectly calibrated model, for example, would predict “cat” with 70% confidence and be correct 70% of the time. State-of-the-art methods for improving uncertainty estimation, such as deep ensembles, Bayesian neural networks, and MC-dropout [ Lakshminarayanan et al., 2017 , Blundell et al., 2015 , Gal and Ghahramani, 2016 ] , have shown strong results. However, these methods come with substantial computational costs. Ensembles require training multiple models, increasing both memory usage and inference time. Bayesian neural networks, which learn distributions over weights, suffer from slow training and expensive inference. MC-dropout, while lighter than ensembles, still requires multiple stochastic forward passes during inference and keeping dropout active at test time, making it less suitable for real-time applications. These constraints limit the practicality of existing uncertainty quantification (UQ) methods in resource-constrained environments, such as autonomous drones or embedded systems. Interestingly, many real-world tasks involve input data with inherent redundancy, where the problem can be solved using substructures or subsets of the input. For instance, in speech classification, short audio patches often contain enough information to determine the speaker. Similarly, in radio signal identification, brief signal windows can be sufficient to identify the transmitting device. Despite this, existing UQ methods do not explicitly exploit such redundancy, leaving an opportunity for more efficient approaches. In this work, we propose a novel variance-based softmax scaling approach to recalibrate a model’s uncertainty estimates by exploiting tasks that follow what we refer to as the “informative sub-patches” assumption. Specifically, we introduce a method that uses the variance between predictions on sub-patches of the input as a proxy for uncertainty, achieving real-time inference speeds while improving the model’s reliability. We refer to this approach as Variance-based Smoothing (Fig. 1 ). Our contributions are threefold: 1. We propose a simple yet effective recalibration method that serves as a post-hoc uncertainty estimation technique for pre-trained models while preserving their original accuracy. 2. We demonstrate competitive uncertainty estimates on both clean and noisy data for diverse tasks, including speech, vision, and radio signal classification while being significantly more efficient. 3. Our proposal also leads to a simple extension of ensembles, improving upon the expressiveness of their predicted distributions in tasks with many classes while eliminating the need for the informative sub-patches assumption. 2 Exploiting Substructure for Uncertainty Estimation This section is structured as follows: we begin with a naive approach to exploiting this informative sub-patches assumption and discuss its limitations, followed by a refined method that preserves classification performance. Finally, we consider a generalization of this approach as an extension of ensembles, allowing us to omit this data requirement. 2.1 Ensemble Models as a Starting Point: A Naive Approach Ensemble-based methods train multiple models with different weight initializations. When the same input is passed through these models, their predictions tend to agree on in-distribution data similar to what they were trained on. However, for out-of-distribution data, predictions may vary due to differences in the learned model weights. The final classification output is computed as the mean of the softmax predictions: p(y\mid\mathbf{x})=\frac{1}{M}\sum_{m=1}^{M}p_{\theta_{m}}(y\mid\mathbf{x}), (1) where p(y\mid\mathbf{x}) is a probability vector over K classes, M is the number of models, and \theta_{m} denotes the parameters of the m -th model [ Lakshminarayanan et al., 2017 ] . The predictive distribution also serves as the uncertainty estimate. Based on these ensembles, a naive single-model alternative that exploits the informative sub-patches assumption is to split an input \mathbf{x} into T (non)overlapping sub-patches \mathbf{x}_{1},\mathbf{x}_{2}\dots\mathbf{x}_{T} and make separate predictions on each sub-patch individually. The final prediction is obtained by averaging the softmax distributions: p(y\mid\mathbf{x})=\frac{1}{T}\sum_{t=1}^{T}p(y\mid\mathbf{x}_{t}). (2) Unlike conventional NNs, where logits are pooled before applying softmax, this approach averages already-softmaxed outputs. While this can temper overconfident predictions, it also harms predictive accuracy, as each prediction is based on a sub-patch with a limited context window. Additionally, if a given sub-patch \mathbf{x}_{t} contains little information (e.g., a 500ms audio segment where the speaker is silent), the model may output unconfident logit scores where all class values are negative. After applying softmax, a peak is still “forced,” causing the prediction from that sub-patch to contribute equally to those from more informative patches. We found that this method tends to produce overly unconfident predictions for both in-distribution and out-of-distribution data, resulting in less informative output distributions. 2.2 Variance-based Smoothing To address these issues, we preserve the conventional pre-softmax classification output, denoted as p_{\text{logit}}(y\mid\mathbf{x}) , and adjust softmax confidence using temperature scaling. The scaling factor \tilde{\sigma}\geq 1 , derived from the variance of sub-predictions, adjusts the softmax function to better account for uncertainty: p(y\mid\mathbf{x})=\text{Softmax}\left(\frac{p_{\text{logit}}(y\mid\mathbf{x})}{\tilde{\sigma}}\right). (3) Since temperature scaling does not affect the predicted class ranking, classification accuracy remains unchanged. To obtain \tilde{\sigma} , the standard deviation \sigma_{k} is computed across sub-predictions for class k over different sub-patch-indices t : \sigma_{k}:=\sqrt{\frac{1}{T-1}\sum_{t=1}^{T}\left(\left[p_{\text{logit }}\left(y\mid\mathbf{x}_{t}\right)\right]_{k}-\mu_{k}\right)^{2}}, (4) where \mu_{k}=\frac{1}{T}\sum_{t=1}^{T}\left[p_{\text{logit }}\left(y\mid\mathbf{x}_{t}\right)\right]_{k} . We aggregate the per-class standard deviations {\sigma_{1},\dots,\sigma_{K}} into a single scalar \bar{\sigma} and compute \tilde{\sigma} as follows: \displaystyle\bar{\sigma}=\frac{1}{K}\sum_{k=1}^{K}\sigma_{k},\ \displaystyle\tilde{\sigma}=\max(\alpha(\bar{\sigma}+\beta),1). (5) The \max operation ensures that \tilde{\sigma} never falls below 1, preventing unintended confidence amplification. The hyperparameters \alpha and \beta control the sensitivity of scaling. We found empirically that reasonable values for \beta are between the negative 50th and 95th percentiles of \bar{\sigma} computed over a validation set, ensuring that smoothing activates only beyond a certain threshold. The parameter \alpha determines the strength of the adjustment, with \alpha=1 providing balanced uncertainty estimates. Higher values can be used to encourage more conservative or pessimistic predictions, with values between 1 and 5 being reasonable choices. Overall, this Variance-based Smoothing approach requires only a single model and a single forward pass over the entire input, significantly reducing computational costs compared

Read the complete text