Predictions with limited data: Bayesian (X)PINNs, entanglement surfaces and overconfidence

Filip Landgren, Marika Taylor · 2025

Read Predictions with limited data: Bayesian (X)PINNs, entanglement surfaces and overconfidence by Filip Landgren, Marika Taylor (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23784.txt.

Introduction

Neural networks that solve physics equations from sparse, boundary-only data can sound surer than they should. Filip Landgren and Marika Taylor explore Bayesian and extended physics-informed networks on a nonlinear equation from holography, tied to an entanglement surface. Analytic asymptotics supply training data from the boundary alone. The authors argue that standard overconfidence measures mislead here, since a model that obeys hard physics constraints may be confident for good reason.

They propose diagnostics instead: an information density and a local measure of how strongly the physics constraints collapse the posterior. The same tools are tried on Liouville-type and Van der Pol equations to probe apparent overconfidence further.

Catalog record

CREATORFilip Landgren, Marika Taylor
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-23784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Predictions with limited data: Bayesian (X)PINNs, entanglement surfaces and overconfidence Filip Landgren, Marika Taylor Abstract Solving differential equations from limited or noisy data remains a key challenge for physics-informed neural networks (PINNs), which are typically applied to already known and smooth solutions. In this work, we explore Bayesian PINNs and extended PINNs, (B-(X)PINNs), to solve non-linear second order differential equation typical for high energy theory, where data is only available from the boundary domain, to benchmark suitable approaches to PINNs in this category. In particular, we consider an entangling surface; a differential equation typical in holography. We perform asymptotic analysis to generate analytical training data from the boundary domain. We also explore the meaning of overconfidence in models that are constrained by physical priors and argue that standard overconfidence metrics are not suitable to consider when dealing with B-PINNs. Overconfidence can be a natural feature and not a bug in systems with soft or hard constraints on the loss function; one have to look at when the overconfidence is an artifact of the model adhering to the physics constraints. To diagnose this effect, we introduce an information density quantity, and a local physics-constraint coupling (PCC) metric, to capture locally to what extent the enforced physics collapses the posterior distribution. We also consider these quantities for a Liouville-type equation and the Van der Pol equation to probe apparent overconfidence further. 1 Introduction Machine learning and neural networks have been rapidly integrated into various domains in physics where data plays a crucial role rodrigues2023machinelearningphysicsshort . Neural networks are promising for solving differential equations where traditional numerical methods fail, such as in the cases with high non-linearity. Their expressive power stems from their capacity to model non-linear relationships between inputs and outputs. Neural networks are purely data driven and learn from examples making connections with weights and biasses between modes to represent a function approximating the solution to the problem at hand, as illustrated in figure 1 . Physics-Informed Neural Networks (PINNs) RAISSI2019686 , introduces a ”symbolic” element into the learning in terms of physical constraints in the loss function, typically in terms of penalizing deviations from boundary conditions and the residual. However, significant challenges remain when applying PINNs to problems where the solution is unknown or, for instance, where it is ill-behaved, non-unique or when training data is sparse zhang2024physicsinformedneuralnetworks . In such cases, a naive PINN, without further guidance, may converge to an arbitrary solution branch. Extended PINNs (XPINNs) augments ordinary PINNs by partitioning the domain into subdomains, each with its own separate network CiCP-28-2002 . This eases the learning in all sub-regions, and overall produces a better prediction, at the expense that the model is more prone to overfitting, due to potentially sparse data in the subdomains, and its inability to learn global features. In Hu_2022 , the authors investigate how well XPINNs generalize, and use Barron space theory to find a trade-off condition when XPINNs generalize better than ordinary PINNs. XPINN’s inability to learn global features is partly addressed by APINNs Hu_2023 , which allow flexible sharing of parameters between subnetworks, and by iPINNs dekhovich2023ipinns , which learn incrementally by training each subnetwork sequentially, pruning over all previous subnetworks, and merging them into a single network. Using XPINNs to solve ODEs and PDEs, with limited or noisy training data remains an active research area. In this work, we will focus on a complex ODE with two branches of solutions, with limited training data only near the domain boundaries. The data is multivalued, and we will do a mild partitioning and let the model train on the two branches separately, but not divide the domains for each branch further. Since we are working with limited data, we will explore Bayesian physics-informed learning Yang_2021 , a B-XPINN, that uses stochastic learning and replaces the fixed weights in the network with (Gaussian) distributions. Through Bayesian inference, the model learns a posterior distribution over the network weights, which in turn induces a distribution over solutions. B-PINNs have proven particularly advantageous when working with limited and or noisy data Yang_2021 ; hou2024improvementbayesianpinntraining ; Linka_2022 . Furthermore, the probabilistic treatment allows the model to quantify epistemic uncertainty from limited data, providing not just point estimates but also credible intervals for predictions. Such epistemic uncertainty estimates are crucial when working with sparse data, as they flag where the model is less certain and might benefit from additional data or refinement (see also e.g. arbel2023primerbayesianneuralnetworks for a review of Bayesian statistics in machine learning). We show that using domain decomposition and Bayesian inference, leads to more accurate and robust solutions compared to a standard PINN that lacks these features, when inferring the the solution from data only around the domain boundaries. Central to this work is also the study of overconfidence and what it means for Bayesian physics-informed learning. Accurate measures for uncertainties were explored in kuleshov2018accurateuncertaintiesdeeplearning and while the B-PINNs provides uncertainty estimates, interpreting and trusting these uncertainties requires care. An important question we investigate is how to ensure the model’s confidence is well-founded when it generalizes beyond the training region. In prediction tasks, a model is said to be overconfident if it estimates its uncertainty to be too low (or equivalently, is too certain in its predictions) in regions where it could actually be wrong. Overconfidence is a well-known issue in purely data-driven models, and often signals that the model is miscalibrated or overfit, failing to account for its lack of knowledge. In the context of physics-informed learning, however, the notion of overconfidence becomes more nuanced. A B-PINN heavily constrained by physical laws might appear overconfident even when it is correct, simply because the physical constraints eliminates degrees of freedom in the solution space. In other words, the model’s uncertainty can be very low not due to overconfidence in the usual sense, but because the physical prior confidently dictates the solution. It is thus crucial to distinguish between warranted confidence and misleading overconfidence in B-PINNs. In graf2022errorawareBPINNsimprovinguncertainty it was recognized that conventional B-PINNs merge measurement noise, parameter dispersion and equation error into a single posterior, masking the origin of the model’s certainty. They compensate by adding a pseudo-aleatoric variance term proportional to the PDE residual, which widens credible bands wherever the network violates the governing equation. Although this alleviates under-dispersion, it does not reveal why the model becomes confident, whether that confidence is earned from data or simply inherited from a physics prior. A parallel body of work has studied error propagation and coverage guarantees in PINNs shen2023picpropphysicsinformedconfidencepropagation ; Bajaj_2023 ; daw2023mitigatingpropagationfailuresphysicsinformed ; wu2025propinndemystifyingpropagationfailures . These approaches tighten or calibrate prediction intervals, but they likewise leave unexplored the explicit contribution of the physics constraints to overconfidence. Rather than treating all instances of high confidence as a flaw, regardless of origin, one should ask: when is the model’s confidence an artifact of limited data, and when is it a justified result of enforced physical laws? To diagnose overconfidence in physics-informed models, we introduce two metrics. The first is a gradient based information density measure ( 80 ), which assesses how much the observed data or physics constraints inform the posterior uncertainty of the model in different regions of the domain, by measuring sensitivity when varying the

Read the complete text