Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results

Andrew Thompson, Miles McCrory · 2026

Read Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results by Andrew Thompson, Miles McCrory (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16830.txt.

Introduction

Andrew Thompson and Miles McCrory work out exact math for doubt in neural nets. A trained net gives a call for each input. When inputs blur, the call's blur counts too. Past work gauged that blur with rough series tricks. Their 2026 preprint gives exact forms for one class of nets.

The nets have one hidden layer and a ramp-style switch. Inputs are set as bell-shaped chance spreads. From those, exact mean and spread of the output fall out. No rough cuts are used in the proof. Exact doubt helps in care, flight, and risk work. It lets teams trust net calls with proof, not hope.

Catalog record

CREATORAndrew Thompson, Miles McCrory
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-16830

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Uncertainty propagation through trained multi-layer perceptrons: Exact analytical results Andrew Thompson, Miles McCrory Abstract We give analytical results for propagation of uncertainty through trained multi-layer perceptrons (MLPs) with a single hidden layer and ReLU activation functions. More precisely, we give expressions for the mean and variance of the output when the input is multivariate Gaussian. In contrast to previous results, we obtain exact expressions without resort to a series expansion. 1 Introduction The importance of assessing the trustworthiness of machine learning (ML) models through uncertainty quantification is now widely recognised – for example the EU AI Act indirectly mandates uncertainty quantification by requiring transparency and risk management [ 6 ] . Here we focus on the challenge of uncertainty propagation through trained machine learning regression models , in which known uncertainties concerning the input data are propagated through a fixed (trained) model to obtain uncertainties for the output prediction. Despite not taking into account contributions to the prediction uncertainty due to the training process, this approach is of interest for two reasons. Firstly, in some testing/validation scenarios the tester may only have access to the ML model as a ‘black box’, in which case uncertainty propagation through a fixed model is the only feasible approach that can be taken. Secondly, uncertainty propagation through a fixed model can be used to carry out sensitivity analysis of the model [ 3 ] . There are two main types of method for propagating uncertainties through models: analytical and sampling-based. In the analytical approach, a mathematical expression is derived either for the distribution of the output or some summary statistic such as variance. In the sampling-based approach, the model is evaluated on samples of the input data distribution, for example Monte Carlo sampling [ 2 ] , thereby obtaining an empirical distribution. If it is possible to derive analytical expressions characterising the output uncertainty, there are several reasons why it is usually preferable to use them. Firstly, analytical expressions are often more accurate, since they often precisely characterise the output uncertainty, in contrast to sampling-based approaches which only approximate the true distribution of the model output with accuracy increasing with the number of samples. Secondly, analytical expressions provide more transparency by providing mathematical insight and understanding into the sources of propagated uncertainty. Thirdly, analytical approaches are more easily reproducible, since all that is needed to reproduce the evaluation is the input data distribution and the analytical expression for its propagation, whereas sampling-based methods often require a record of the sampling process. In this paper, we focus on analytical characterisation of output uncertainty for fully-connected feed-forward regression neural networks, also known as multi-layer perceptrons (MLPs), with a single hidden layer and with the popular choice of a ReLU activation function. The usefulness of single-hidden layer networks is underlined by theoretical results showing that they are universal approximators if sufficiently wide [ 9 ] , and that they converge to Gaussian Processes in the infinite-width limit if the weights are initialised in a particular way [ 15 ] . More precisely, we derive analytical expressions for the mean and variance of model predictions under the assumption that the input data follows a multivariate Gaussian distribution. We demonstrate the practicality of our methods by illustrating their use upon a test problem involving the prediction of the state-of-health of lithium-ion cells based upon Electrical Impedance Spectroscopy (EIS) data. By comparing our methods with a Monte Carlo sampling approach [ 2 ] in this context, we validate the correctness of our expressions. 1.1 Paper structure The structure of the paper is as follows. We first review closely related work in Section 2 . In Section 3 , we define the single-hidden-layer MLP model that we are considering, and then we present our analytical results for uncertainty propagation in Section 4 . Proofs of all novel results can be found in the Appendix. We present our numerical experiments in Section 5 , before giving our conclusions in Section 6 . 2 Related work The closest related existing work is [ 19 ] , in which analytical expressions are given for the pairwise covariances after passing multivariate Gaussian inputs through general activation functions. The results apply to any elementwise independent and identical activation function whose mean function when applied to univariate Gaussian input is infinitely differentiable with respect to the mean of its input. This includes the ReLU activation function, and also others such as Heaviside and GELU. The expressions for the covariances take the form of infinite Taylor series, which the authors point out can be computed to arbitrary precision. However, the number of terms required to compute to a desired precision is dependent upon the covariances of the inputs, and each of the terms in the series have to be painstakingly calculated for a specific activation function. Our results, on the other hand, do not involve infinite series but are closed-form functions of standard (1D and 2D) Gaussian integrals. Our results therefore provide a simpler method for computing the output moments to arbitrary precision for the case of a ReLU activation function. To the authors’ best knowledge, all other analytical methods that have been proposed for propagating uncertainty through neural networks are not exact but involve approximation at some point. For neural networks with more than one hidden layer, the simplifying assumption is often made that the inputs to each activation step are multivariate Gaussian, which is an approximation except for the first hidden layer. Even for propagation through a single hidden layer, approximations are often employed, for example assuming the inputs are mean zero [ 1 ] , using asymptotic approximations [ 20 ] and linearising the activation function [ 7 ] . Finally, we note that various sampling-based methods have been proposed to approximately propagate uncertainty through neural networks, including Monte Carlo methods [ 13 ] , Kalman filtering [ 18 ] and the Unscented Transform [ 13 ] . 3 Neural network architecture The neural network architecture that we consider in this paper is shown in Figure 1 . We consider an MLP with a single hidden layer and a ReLU activation function. A vector of inputs V\in\mathbb{R}^{m} is passed through an affine convolution to give W=A^{T}V+c\in\mathbb{R}^{p} , where p is the number of hidden units. W is then passed through a componentwise ReLU to give X=W_{+}\in\mathbb{R}^{p} , where the ReLU function is defined in the usual way as \mathrm{ReLU}(x):=\max(x,0) . Finally, X is passed through an affine convolution to obtain the output Y=\beta^{T}X+d\in\mathbb{R} . v_{1} v_{2} ⋮ v_{m} w_{1} w_{2} ⋮ w_{p} x_{1} x_{2} ⋮ x_{p} y_{1} V\in\mathbb{R}^{m} W=A^{T}V+c\in\mathbb{R}^{p} X=W_{+}\in\mathbb{R}^{p} Y=\beta^{T}X+d\in\mathbb{R} Figure 1 : Architecture of an MLP with a single hidden layer and ReLU activation function. 4 Analytical uncertainty propagation results In this section we present our analytical expressions for the mean and variance of the output of the neural network defined in Section 3 when the input data follows a multivariate Gaussian distribution. Suppose then that the input data V\in\mathbb{R}^{m} is multivariate Gaussian with mean \lambda and covariance \Lambda . We will see that the main challenge is propagating uncertainty through the ReLU activation function as propagating uncertainty through the convolutional layers is relatively straightforward. For the first convolution, it is a standard

Read the complete text