Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles

Pablo Flores, Olga Graf, Pavlos Protopapas, Karim Pichara · 2025

Read Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles by Pablo Flores, Olga Graf, Pavlos Protopapas, Karim Pichara (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-06459.txt.

Introduction

Pablo Flores, Olga Graf, Pavlos Protopapas, and Karim Pichara improve how physics-informed neural networks report their own doubt, in a paper from 2025. These networks solve equations of motion and change, but they give no sense of error by default. A user cannot tell where the answer is firm and where it is a guess. The team add a two-step Bayesian layer on top.

Known error bounds for the networks shape a variance that shifts across the domain. That richer doubt model sharpens the uncertainty bands. They then solve forward problems and feed the doubt into inverse ones. A cosmology case shows the payoff: unknown parameters come with honest error bars. The method makes such networks safer for science use. It also helps flag where more data would count most.

Catalog record

CREATORPablo Flores, Olga Graf, Pavlos Protopapas, Karim Pichara
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-06459

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improved Uncertainty Quantification in Physics-Informed Neural Networks Using Error Bounds and Solution Bundles Pablo Flores, Olga Graf, Pavlos Protopapas, Karim Pichara Abstract Physics-Informed Neural Networks (PINNs) have been widely used to obtain solutions to various physical phenomena modeled as Differential Equations. As PINNs are not naturally equipped with mechanisms for Uncertainty Quantification, some work has been done to quantify the different uncertainties that arise when dealing with PINNs. In this paper, we use a two-step procedure to train Bayesian Neural Networks that provide uncertainties over the solutions to differential equation systems provided by PINNs. We use available error bounds over PINNs to formulate a heteroscedastic variance that improves the uncertainty estimation. Furthermore, we solve forward problems and utilize the obtained uncertainties when doing parameter estimation in inverse problems in cosmology. 1]Departamento de Ciencia de la Computación, Pontificia Universidad Católica de Chile, Santiago, Chile 2]Department of Computer Science, University of Tübingen, Germany 3]John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts 02138, USA 1 Introduction Physics-Informed Neural Networks (PINNs), first proposed by Lagaris et al. 1997 , solve differential equations (DEs) by embedding the physics of the problem into the network, eliminating the need for extra data. PINNs offer advantages over traditional solvers: they are continuous, differentiable, and parallelizable, allowing them to bypass the need for previous time steps. However, challenges remain in computing solution errors or error bounds, with ongoing research [ Liu et al. 2022 , Liu et al. 2023 , De Ryck and Mishra 2022b ] . Specific equations like Navier-Stokes and Elasticity are also under study [ De Ryck and Mishra 2022a , De Ryck et al. 2022 , Guo and Haghighat 2022 ] . Since their introduction, PINNs have rapidly gained attention, being applied to heat transfer [ Cai et al. 2021b ] , wave equations [ Rasht-Behesht et al. 2022 ] , and fluid mechanics [ Jin et al. 2021 , Cai et al. 2021a , Mao et al. 2020 ] . Efforts to address failure modes and optimization issues are ongoing [ Nabian et al. 2021 , Steger et al. 2022 , Krishnapriyan et al. 2021 , Daw et al. 2022 ] , with interest in Bayesian PINNs for uncertainty quantification (UQ) [ Yang et al. 2020 , Linka et al. 2022 , Graf et al. 2021 , Psaros et al. 2023 ] . The use of NNs for solving ODEs and PDEs was pioneered by Lagaris et al. 1997 and later advanced by Raissi et al. 2019 , who introduced PINNs for forward and inverse problems. Forward problems solve for the solution given the equation and boundary conditions, while inverse problems estimate unknown parameters. Inverse problems were implemented by Raissi et al. 2019 by treating these parameters as trainable variables. Although effective, this approach lacks a robust mechanism for uncertainty quantification, which Bayesian Neural Networks (BNN) address by introducing distributions over network weights. However, simply applying Bayesian methods does not fully leverage the available information about solution accuracy. This is where Solution Bundles [ Flamant et al. 2020 ] come into play. Solution Bundles enable statistical analysis over multiple possible solutions, providing a more comprehensive view of uncertainty in both the solution and the equation parameters. In cosmology, DEs aim to explain the universe’s expansion. Testing new models against observations typically involves statistical analysis to determine parameter bounds. By combining Solution Bundles with BNNs, we enhance uncertainty quantification, offering a more reliable method for parameter estimation. Our contributions include: • Introducing error-bound-based heteroscedastic variance for better uncertainty quality. • Solving forward problems for cosmological equations while quantifying uncertainties. • Applying Solution Bundles to solve inverse problems for parameter estimation. 2 Background 2.1 Problem Formulation We adopt a slightly different formulation from Psaros et al. 2022 . The DEs we will work with can be defined as follows: \displaystyle\mathcal{F}_{\lambda}[u(\mathbf{x})] \displaystyle=f(\mathbf{x}),\quad\mathbf{x}\in\Omega, (1) \displaystyle\mathcal{B}_{\lambda}[u(\mathbf{x})] \displaystyle=b(\mathbf{x}),\quad\mathbf{x}\in\Gamma, (2) where \mathbf{x} is the space-time coordinate, \Omega is a bounded domain with boundary \Gamma , f(\mathbf{x}) is the source term, u is the solution of the system, \mathcal{F}_{\lambda} is a differential operator, \mathcal{B}_{\lambda} and b(\mathbf{x}) are the boundary conditions (BCs) operator and term, respectively, and \lambda denotes the parameters of the system. In this paper, we will focus on problems where the operators \mathcal{F}_{\lambda} , \mathcal{B}_{\lambda} and terms f(\mathbf{x}) , b(\mathbf{x}) are known. If \lambda is assumed to be known, the goal is to find the solution u , referred to as the forward problem . Conversely, if u is known and the aim is to estimate \lambda , then this is known as the inverse problem . 2.2 Physics-Informed Neural Networks A Physics-Informed Neural Network uses a neural network, u_{\theta}(\mathbf{x}) , to approximate the true solution u(\mathbf{x}) of a differential equations system. As discussed by Psaros et al. 2023 , PINNs can be trained by minimizing a fitting dataset’s Mean Squared Error (MSE). A fitting dataset \mathcal{D}=\{\mathcal{D}_{f},\mathcal{D}_{b}\} is composed of noisy observations of f , \mathcal{D}_{f}=\{\mathbf{x}_{i},f_{i}\}_{i=0}^{N_{f}} , and noisy BCs data \mathcal{D}_{b}=\{\mathbf{x}_{i},b_{i}\}_{i=0}^{N_{b}} \mathcal{L}(\theta)=\frac{w_{f}}{N_{f}}\sum_{i=0}^{N_{f}}\lVert f_{\theta}(\mathbf{x}_{i})-f_{i}\rVert^{2}_{2}+\frac{w_{b}}{N_{b}}\sum_{i=0}^{N_{b}}\lVert b_{\theta}(\mathbf{x}_{i})-b_{i}\rVert^{2}_{2} (3) Where w_{f} and w_{b} are weighting constants. Psaros et al. 2023 call this setup the Forward Deterministic PDE Problem , they also describe Mixed Deterministic PDE Problem where the objective is to obtain solutions for u and \lambda , and the Mixed Stochastic PDE Problem which deals with stochastic PDEs. In this work, we adopt a different optimization problem to solve the forward deterministic problem and address the inverse problem separately rather than solving both forward and inverse problems simultaneously. 2.3 Solving Forward Problems We begin by defining the residual of a differential equation as r_{\theta}(\mathbf{x})=\mathcal{F}_{\lambda}[u_{\theta}(\mathbf{x})]-f(\mathbf{x}) (4) The computation of \mathcal{F}_{\lambda}[u_{\theta}(\mathbf{x})] is easy to implement thanks to automatic differentiation provided by Deep Learning frameworks such as PyTorch [ Paszke et al. 2019 ] . This formulation allows PINNs to be trained as a self-supervised network. Since any solution u^{*} to the differential equation satisfies \mathcal{F}_{\lambda}[u^{*}(\mathbf{x})]-f(\mathbf{x})=0 , we train the network to minimize the square of the residual \min_{\theta}\frac{1}{N_{r}}\sum_{i}^{N_{r}}r_{\theta}^{2}(\mathbf{x}_{i}). (5) Usually, \mathbf{x}_{i} are sampled from \Omega with a uniform distribution or by taking an equally-spaced subset. It is important to note that we do not deal with noisy data, unlike the work by Raissi et al. 2019 . 2.4 Enforcing Boundary Conditions While adding a term for BCs in the loss (see Eq. 3 ) when dealing with data is a good way to incorporate such knowledge, there is no guarantee that the conditions will be satisfied. When BCs are known rather than observed in the data, we can use the transformation introduced by Lagaris et al. 1997 that enforces the BCs to be always satisfied. This is achieved by writing the approximate solution as a sum of two terms: \tilde{u}_{\theta}(\mathbf{x}):=A(\mathbf{x})+F(\mathbf{x},u_{\theta}(\mathbf{x})) (6) where A does not depend on the network parameters \theta and it satisfies the BCs. Since we need \tilde{u}_{\theta} to satisfy BCs, F is constructed so it does not contribute to them. This transformation is also used by Graf et al. 2021 , Chen et al. 2020 . One-dimensional Initial Value Problem Given an initial condition u_{0}=u(t_{0}) , we consider a transformation \tilde{u}_{\theta}(t):=u_{0}+(1-e^{-(t-t_{0})})u_{\theta}(t) (7) In Appendix A of the Supplementary Material we show the enforcing of two-dimensional Dirichlet BCs. Similar transformations can be defined for Neumann and mixed

Read the complete text