Multi-level Monte Carlo Dropout for Efficient Uncertainty Quantification
Aaron Pim, Tristan Pryer · 2026
Read Multi-level Monte Carlo Dropout for Efficient Uncertainty Quantification by Aaron Pim, Tristan Pryer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13272.txt.
Introduction
Aaron Pim and Tristan Pryer cut the cost of doubt scores for neural nets. One stock way to score doubt is Monte Carlo dropout. The net is run many times with random parts switched off. The spread of the runs marks the doubt. That cost grows fast with each pass.
Their frame stacks runs by grade. Low grade runs are cheap and coarse. High grade runs are dear and fine. By mix of grades, in a multi level Monte Carlo plan, the same doubt score can be had for less cost. The dropout masks are treated as the chance source to be stacked. For teams that must ship doubt scores live, the cut in cost is the point.
Catalog record
| CREATOR | Aaron Pim, Tristan Pryer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13272
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multi-level Monte Carlo Dropout for Efficient Uncertainty Quantification Aaron Pim, Tristan Pryer Abstract . We develop a multilevel Monte Carlo (MLMC) framework for uncertainty quantification with Monte Carlo dropout. Treating dropout masks as a source of epistemic randomness, we define a fidelity hierarchy by the number of stochastic forward passes used to estimate predictive moments. We construct coupled coarse–fine estimators by reusing dropout masks across fidelities, yielding telescoping MLMC estimators for both predictive means and predictive variances that remain unbiased for the corresponding dropout-induced quantities while reducing sampling variance at fixed evaluation budget. We derive explicit bias, variance and effective cost expressions, together with sample-allocation rules across levels. Numerical experiments on forward and inverse PINNs–Uzawa benchmarks confirm the predicted variance rates and demonstrate efficiency gains over single-level MC-dropout at matched cost. 1. Introduction In many scientific computing and machine learning applications, neural networks are increasingly used as fast surrogate models for complex simulations [ RPK19 ] . Unlike traditional mechanistic models, these neural surrogates do not by default quantify prediction uncertainty. Their outputs are deterministic. In practice this is problematic: deep networks often interpolate training data well but can make overconfident, erroneous predictions when extrapolating [ Gaw+23 , HW21 ] . In applied surrogate pipelines this lack of uncertainty can be a practical barrier to reliable deployment, including settings where the surrogate is used for repeated evaluation and decision making [ PP25 ] . Consequently, there is a pressing need to accompany neural surrogates with uncertainty quantification (UQ). Bayesian deep learning methods address this by modelling a distribution over network outputs. For example, Monte Carlo dropout (MC-dropout) interprets dropout at test time as approximate Bayesian inference, yielding epistemic uncertainty estimates without changing network architecture [ GG16 ] . Similarly, methods like deep ensembles train multiple networks and use the ensemble variance for UQ [ LPB17 ] . In scientific machine learning, Physics-informed Neural Networks (PINNs) incorporate prior knowledge by training on physics constraints, but they too generally lack calibrated uncertainty outputs unless extended to Bayesian formulations. For inverse problems and constrained training setups this can be especially acute, since inference is driven by a combination of data mismatch and modelling assumptions [ Cox+24 , MPP24 ] . In such settings one may face both aleatoric uncertainty (inherent noise in data) and epistemic uncertainty (model uncertainty) [ KG17 ] . In this work we focus on reducing the cost of estimating epistemic uncertainty induced by MC-dropout. A standard approach (as in MC-dropout or ensemble sampling) is to run many stochastic forward passes to estimate predictive statistics. However, the computational cost grows linearly with the number of samples. Reducing estimator variance by increasing the number of samples can be prohibitive in large-scale settings. For example, MC-dropout often uses dozens of passes per input to estimate uncertainty, each pass incurring a full forward evaluation of the network. This renders uncertainty estimation expensive for large or real-time models and it is already a practical bottleneck in surrogate pipelines that require repeated uncertainty evaluation [ PP25 ] . More generally, naive Monte Carlo estimators are often too costly when each sample is expensive [ Gil15 ] . Multilevel Monte Carlo (MLMC) addresses precisely this issue by exploiting a hierarchy of fidelities and a telescoping decomposition [ Hei01 , Gil08 ] . Most samples are taken at low-fidelity (cheap) levels and few at high-fidelity (expensive) levels, reducing total cost for a given error [ Gil08 , Gil15 ] . In stochastic simulation (e.g. SDEs and PDEs) MLMC has proven very effective [ Cli+11 ] . Recent work has begun to bring MLMC ideas into machine learning; for instance, [ FS21 ] applies MLMC to variational inference in Bayesian models and [ Ger+21 ] proposes multilevel training strategies for deep networks. More closely related to our setting, [ Bla+23 ] develops an unbiased MLMC scheme to speed up dropout training of classifiers. However, to our knowledge MLMC has not been explored for inference-time uncertainty quantification in neural surrogates. The goal of this work is to bridge practical dropout-based uncertainty estimation with variance-reduction tools from multilevel Monte Carlo (MLMC). We formulate an MLMC framework in which Monte Carlo dropout provides the randomness, a single forward evaluation with dropout is treated as a randomised model evaluation and predictive statistics are estimated by sampling over dropout masks. Our primary notion of fidelity is the number T of stochastic forward passes used inside the predictive estimators. Higher fidelity corresponds to more dropout masks and hence higher computational cost. Building on this hierarchy, we construct multilevel estimators by coupling coarse and fine evaluations through shared dropout masks, so that level corrections have reduced variance. This yields multilevel estimators for both dropout-induced predictive expectations and dropout-induced predictive variances, while retaining unbiasedness for these dropout-induced targets. Other notions of fidelity, such as varying dropout probability or using a hierarchy of network resolutions, can also be viewed through a multilevel lens, but they require additional modelling choices and are not essential for the core development presented here. We provide a theoretical analysis of the resulting estimators. We derive explicit bias, variance and cost expressions, identify the quantities that control the variance of level corrections and use these to formulate sample allocation strategies across levels under a coupled cost model with mask reuse. The resulting allocations recover the classical MLMC structure M_{\ell}\propto\sqrt{V_{\ell}/C_{\ell}} , with problem-specific expressions for the level variances V_{\ell} induced by the dropout coupling. We also discuss practical ladder design and stopping criteria based on empirical level-variance estimates. We demonstrate the approach on benchmark problems drawn from scientific machine learning. First, we consider a forward PINNs–Uzawa problem for a boundary-layer ODE with a known closed-form solution and study the behaviour of single-fidelity and multilevel dropout estimators, including empirical verification of the predicted sampling-variance rates and a fixed-cost allocation test. Second, we consider an inverse PDE-constrained optimisation benchmark with a stochastic target generated by a scalar noise variable, for which the ground-truth target moments are available in closed form. In both settings we report how the estimator variances depend on the fidelity ladder and on the sample allocation, and we compare empirical best allocations with the continuous allocations predicted by the theory. Research on uncertainty for neural surrogates spans several strands that intersect in this work. Dropout, originally introduced for regularisation [ Sri+14 ] , can be kept active at test time and interpreted as approximate Bayesian inference, providing a practical route to epistemic uncertainty without changing architecture or training [ GG16 ] . A related theme is the distinction between aleatoric uncertainty (stemming from data noise) and epistemic uncertainty (stemming from model uncertainty), which is now standard in uncertainty-aware deep learning [ KG17 ] . Complementary strategies include Bayesian neural networks and deep ensembles, where independent predictors provide variance-based estimates that are often strong in practice [ LPB17 ] . In scientific surrogate pipelines, MC-dropout has also been used directly as a practical UQ tool when repeated uncertainty evaluation is required [ PP25 ] . Multilevel Monte Carlo provides a cost–accuracy tradeoff that is well suited to expensive stochastic evaluation. Foundational results establish telescoping estimators over coupled fidelity levels and show that, in canonical discretisation settings, complexity improvements such as O(\varepsilon^{-3}) to O(\varepsilon^{-2}) can