Uncertainty in Physics and AI:
Taxonomy, Quantification, and Validation · 2026
Read Uncertainty in Physics and AI: by Taxonomy, Quantification, and Validation (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10378.txt.
Introduction
Manuel Haußmann, Ramon Winterhalder, and Maria Ubiali offer a structured guide to doubt in machine learning for physics. Sound doubt estimates are vital where findings rest on validated chance statements, yet the field lacks a unified statistical view. Their 2026 overview introduces a unified taxonomy of doubt and clarifies predictive versus inference doubt across frequentist and Bayesian frameworks.
The authors discuss sound validation tools, including coverage, calibration, bias tests, and proper scoring rules, shown with simple regression and sorting examples. Coverage asks whether claimed intervals contain truth at the stated rate, while calibration checks whether stated confidences match observed rates. Readers get both a map of doubt types and practical tests for judging whether machine-learning doubt estimates in physics can be trusted.
Catalog record
| CREATOR | Taxonomy, Quantification, and Validation |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 102 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10378
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncertainty in Physics and AI: Taxonomy, Quantification, and Validation Abstract Manuel Haußmann \star ,1 , Ramon Winterhalder \star ,2 , and Maria Ubiali \circ ,3 1 Department of Mathematics and Computer Science, University of Southern Denmark 2 TIFLab, Università degli Studi di Milano & INFN Sezione di Milano, Italy 3 DAMTP, University of Cambridge, Wilberforce Road, Cambridge CB3 0WA, UK \star Leading authors \circ Advisor Abstract Reliable uncertainty quantification is essential for the use of machine learning in physics, where scientific discoveries depend on validated probabilistic statements. We provide a structured overview of uncertainty quantification in ML for physics, introducing a unified taxonomy of uncertainty and clarifying the interpretation of predictive and inference uncertainties across frequentist and Bayesian frameworks. We discuss principled validation tools, including coverage, calibration, bias tests, and proper scoring rules, and illustrate them with simple regression and classification examples. Contents 1 Introduction Uncertainty quantification is an important cornerstone of both physics [ 82 , 126 , 76 , 46 , 127 ] and machine learning (ML) [ 88 , 64 , 59 , 124 , 1 , 121 ] . In physics, reliable uncertainty estimates are indispensable: without them, a result is scientifically incomplete. They determine the credibility of measurements, underpin the combination of experimental and theoretical results, dictate the strength of discovery claims, and govern how tightly any physical parameter can be constrained. In particle physics, statistical significance thresholds benchmark the reporting of new particles or rare processes, and systematic uncertainties from detector effects or theoretical modelling often dominate the final error budget. In astrophysics and cosmology, uncertainty estimates play an equally central role in inferring cosmological parameters, characterising sources, and performing model comparisons. In machine learning, uncertainty estimates have become equally important. Modern AI systems are increasingly deployed in safety-critical applications ranging from medical diagnostics to autonomous driving, where trustworthy predictions cannot be reduced to a single point estimate [ 92 , 99 , 7 ] . Instead, decision-making depends on understanding how confident a model is in its outputs. Calibration of predictive probabilities, quantification of model uncertainty, and guarantees on generalization performance are therefore central to ensuring that ML methods can be applied responsibly [ 72 ] . When ML methods are used in physics, trustworthy uncertainty estimates are equally important: scientific conclusions must be accompanied by statistically validated uncertainty statements, and if ML-based estimates are unreliable, the conclusions drawn from them should be called into question. There already exist applications in physics where ML methods provide not only predictions but also associated uncertainty estimates, including examples in particle physics [ 61 , 16 , 58 , 30 , 37 , 110 , 86 , 8 , 21 , 68 , 67 , 18 , 17 , 12 , 39 , 34 , 60 , 74 , 11 , 93 , 23 , 80 , 28 , 14 , 25 , 89 , 15 , 24 , 9 , 44 , 43 , 20 , 22 , 19 , 13 ] , cosmology [ 111 , 117 , 75 , 55 , 123 , 97 , 48 ] and astrophysics [ 33 , 103 , 10 , 114 , 98 , 132 , 109 , 77 ] . While these developments demonstrate the practical relevance of ML-based uncertainty quantification in physics, a unified statistical interpretation of such uncertainty estimates – and clear guidelines for their validation – is often still lacking across disciplines. To clarify these issues, it is useful to begin by examining what we mean by the term uncertainty . One may consider uncertainty about inferred quantities such as model parameters or functions ( inference uncertainty ), or uncertainty about future observations ( predictive uncertainty ). While these notions are related – for instance through the posterior predictive distribution in Bayesian inference – they obey different validation criteria and correspond to different statistical questions. In particular, coverage, calibration, and bias diagnostics must always be interpreted with respect to the object under consideration. In this contribution we primarily focus on predictive uncertainties. This choice reflects that predictive uncertainty is the quantity most directly accessible to model-agnostic validation: given a new observation, one can empirically check whether it falls within the predicted interval, without reference to unobservable true parameters. Inference uncertainty – e.g. the uncertainty on quantities such as \alpha_{\mathrm{S}} , m_{\mathrm{t}} , or parton distribution functions in particle physics, or cosmological parameters such as H_{0} and \Omega – is equally central to physics but is the primary focus of a companion VERaiPHY contribution on parameter estimation [ 49 ] ; here it enters insofar as it propagates into predictive uncertainty via marginalisation. Despite the central role of uncertainty in both physics and statistics, the terminology used to describe it remains inconsistent across communities. Physicists traditionally distinguish between statistical and systematic uncertainties, while the statistics and ML literature more often refers to aleatoric and epistemic uncertainties. These vocabularies overlap but are not synonymous, and careless translation between them may cause confusion. For instance, statistical fluctuations from limited data can leave model parameters under-constrained (an epistemic effect), which may be further amplified by aleatoric randomness in the data. Likewise, systematic uncertainties may stem either from irreducible noise sources (aleatoric) or from potentially reducible modelling assumptions (epistemic). One of our goals in this article is therefore to provide a clear taxonomy that reconciles these perspectives and clarifies their relationships. A further challenge lies in the validation of uncertainty estimates. In classical statistics, well-defined procedures exist to validate uncertainties, such as confidence intervals and hypothesis tests in the frequentist framework, or posterior distributions and credible intervals in the Bayesian framework. Modern machine learning methods for uncertainty quantification vary in their theoretical grounding. Some approaches, such as conformal prediction, provide finite-sample coverage guarantees under minimal assumptions [ 131 , 120 ] . Others, such as deep ensembles or Bayesian neural networks with approximate inference, rely more heavily on empirical validation and heuristic calibration [ 95 ] . Assessing whether the uncertainties produced by such models are meaningful requires careful use of statistical diagnostics such as coverage tests, pull distributions, calibration curves, and proper scoring rules [ 69 ] . These validation concepts are inherently probabilistic statements whose interpretation depends on what is treated as random – for instance the training data, model parameters, or future observations. Different choices lead to distinct notions of coverage and calibration, such as marginal versus conditional guarantees, or inference versus predictive uncertainty. Understanding how to apply and interpret these diagnostics in physics contexts is therefore a key objective of this work. Our focus is not the accuracy of point predictions, but the validity of the uncertainty statements that accompany them: scientific applications require statistically meaningful uncertainty statements – for instance prediction intervals, credible regions, or calibrated probability distributions – whose consistency must be validated. This article contributes to VERaiPHY (Validation & Evaluation for Robust AI in PHYsics), a PHYSTAT review series establishing verification and validation standards for machine learning across particle physics, astrophysics, and cosmology [ 70 ] . Within this broader initiative, the present article aims to provide a structured overview of uncertainty quantification in physics-inspired machine learning workflows. Our scope is deliberately complementary to related VERaiPHY contributions focusing on statistical inference