B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling

Kevin Innerebner 1 , Franz M. Rohrhofer 2 , and Bernhard C. Geiger 3 2 · 2025

Read B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling by Kevin Innerebner 1 , Franz M. Rohrhofer 2 , and Bernhard C. Geiger 3 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01714.txt.

Introduction

Kevin Innerebner, Franz Rohrhofer, and Bernhard Geiger steady a class of physics-taught nets in a 2025 paper. Such nets learn a field by its laws, not by data alone. They often stall: the known edge facts fail to spread to the core of the zone.

A past fix ran a crew of nets and grew the taught zone where the crew agreed. Their swap uses one net that knows its own doubt, in the Bayes way. Doubt stands in for crew votes, on a firm math base. Tests on bench tasks beat the crew way. Field solvers gain a steadier net with less kit.

Catalog record

CREATORKevin Innerebner 1 , Franz M. Rohrhofer 2 , and Bernhard C. Geiger 3 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-01714

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

B-PL-PINN: Stabilizing PINN Training with Bayesian Pseudo Labeling Kevin Innerebner 1 , Franz M. Rohrhofer 2 , and Bernhard C. Geiger 3 2 Abstract Training physics-informed neural networks (PINNs) for forward problems often suffers from severe convergence issues, hindering the propagation of information from regions where the desired solution is well-defined. Haitsiukevich and Ilin (2023) proposed an ensemble approach that extends the active training domain of each PINN based on i) ensemble consensus and ii) vicinity to (pseudo-)labeled points, thus ensuring that the information from the initial condition successfully propagates to the interior of the computational domain. In this work, we suggest replacing the ensemble by a Bayesian PINN, and consensus by an evaluation of the PINN’s posterior variance. Our experiments show that this mathematically principled approach outperforms the ensemble on a set of benchmark problems and is competitive with PINN ensembles trained with combinations of Adam and LBFGS. BPINN Bayesian Physics-Informed Neural Network PINN Physics-Informed Neural Network IC initial condition BC boundary condition BNN Bayesian Neural Network PDE partial differential equation NN Neural Network MCMC Markov chain Monte Carlo HMC Hamiltonian Monte Carlo NUTS No U-Turn Sampler PDF probability density function LHS latin hyper-cube sampling PSO particle swarm optimization MLE maximum likelihood estimation I Introduction PINN [ 1 ] have become one of the cornerstones and main representatives of physics-informed machine learning due to their ability to incorporate differential equations during training. They were shown to be capable of solving both forward problems—simulating a system of differential equations for a given computational domain—and inverse problems, which involve determining problem parameters or unobserved variables from measurements. A substantial portion of the scientific literature on PINN is concerned with forward problems, or with certain variations thereof. In this case, training is mostly guided by the system of (ordinary or partial) differential equations as well as by the provision of IC , BC , and potentially additional data in the interior of the computational domain. The task here is to find a solution to the system of differential equations that satisfies IC and BC and agrees with the provided data. This forward problem has been shown to be particularly difficult to solve using PINN , as PINN training in this setting is characterized by a peculiar loss landscape, cf. [ 2 ] . Indeed, it has been argued that learning with such a loss landscape often leads to failures in the propagation of information from IC , BC , and measurement data to the interior of the computational domain [ 3 ] . As a consequence, recent years have seen a surge in literature on stabilizing PINN training (see also Section II-B ). Among these approaches is an ensemble method proposed in [ 4 ] , in which training an ensemble of PINN is restricted to regions in the computational domain that are close to known, well-defined solution values (e.g., IC or otherwise labeled points). If the ensemble reaches consensus for a training point in this region, then this point is labeled and used as anchor for subsequent iterations. As such, the proposed approach ensures reliable propagation of information from the boundary to the interior of the computational domain. Fig. 1 : The solution provided by our B-PL-PINN for the convection system with \beta=30 . Data instances for the initial condition and pseudo labels are shown in dark and light blue, collocation points in the interior and the boundary are shown in gray and red. Points are filled if used for updating the posterior at the current iteration. See Section III for details. In this work, we replace the ensemble from [ 4 ] by a Bayesian PINN , effectively sampling from the posterior over solution functions and replacing ensemble consensus with a threshold on the posterior variance (Section III ). Thus, instead of training multiple PINN , our approach relies on sampling a Bayesian PINN . Our experiments with a set of benchmark problems (Section IV ) show that our approach in many cases outperforms the ensemble when the latter is trained with Adam, and is competitive even when a combination of Adam and LBFGS is used for ensemble training (Section V ). Although our method relies on computationally expensive MCMC ( MCMC ) sampling [ 5 , 6 ] , our analysis shows that its runtime is not more than 9\times the runtime of the ensemble approach by Haitsiukevich and Ilin [ 4 ] . Our proposed method 1 1 1 Code can be found at https://github.com/kev-inn/B-PL-PINN . is a mathematically principled alternative to ensemble methods in PINN and can be combined with a variety of complementary approaches to training stabilization, such as loss and collocation point weighting, domain decomposition, or curriculum learning (Section II-B ). As such, our work extends the toolbox of available training stabilization approaches and contributes towards successfully solving differential equations with PINN . II Background and Related Work In this work we are concerned with using PINN for solving systems of PDE , that are given by: \frac{\partial u}{\partial t}=\mathbf{\mathcal{N}}[u;\lambda],\quad x\in\Omega,t\in[0,T], (1) where u\equiv u(x,t) denotes the solution function, and \mathcal{N} is a non-linear differential operator acting on u and parameterized by \lambda\in\mathbb{R}^{P} . Furthermore, \Omega\subseteq\mathbb{R}^{D} is the spatial domain, and T the simulation time, defining the computational domain as \Omega\times[0,T] . In solving forward problems involving PDE , IC and BC must be specified to clearly define the solution at the boundaries of the computational domain and to ensure the uniqueness of the solution. The IC define the solution u at the beginning of the simulation, expressed as u(x,0)=u_{ic}(x) for all x\in\Omega . The BC employ a boundary operator \mathbf{\mathcal{B}} (Dirichlet, Neumann, periodic) to enforce properties of the solution u at the boundary \partial\Omega of the spatial domain \Omega , i.e., \mathbf{\mathcal{B}}[u]=0,\quad x\in\partial\Omega. (2) II-A Physics-Informed Neural Networks The idea of PINN [ 1 ] is to use a NN ( NN ) to approximate the true solution u(x,t) to ( 1 ) by a neural function u_{\theta}(x,t) with (network) parameters \theta . In order to infer the solution function as specified by the IC and BC , PINN are trained in a semi-supervised manner: labeled data \mathcal{D}_{l}=\{(x_{i},t_{i},u_{i})\} , encoding measurements or simulation results, IC , and/or fixed BC , contribute to the supervised (data) loss component, while unlabeled collocation points \mathcal{D}_{u}=\{(x_{i},t_{i})\} , which encode the governing PDE and periodic BC , are part of the unsupervised (physics) loss. The supervised (data) loss is given by the mean squared error between the labels and the results of the candidate solution u_{\theta} , \mathcal{L}_{\theta}^{l}(\mathcal{D}_{l})=\frac{1}{|\mathcal{D}_{l}|}\sum_{i=1}^{|\mathcal{D}_{l}|}\lvert u_{\theta}(x_{i},t_{i})-u_{i}\rvert^{2}. (3a) For example, a given initial condition u_{ic}(x) yields data points \{(x_{i},0,u_{ic}(x_{i}))\}=\mathcal{D}_{ic}\subseteq\mathcal{D}_{l} , while measurement data could provide a fully labeled dataset \mathcal{D}_{data} from the entire computational domain. The unsupervised (physics) loss evaluates a functional of the candidate solution u_{\theta} as follows:, \mathcal{L}_{\theta}^{u}(\mathcal{D}_{u})=\frac{1}{|\mathcal{D}_{u}|}\sum_{i=1}^{|\mathcal{D}_{u}|}\lvert\mathcal{G}[u_{\theta}](x_{i},t_{i})\rvert^{2}, (3b) where the collocation points \mathcal{D}_{pde}\subseteq\mathcal{D}_{u} are sampled from the interior of the computational domain. This functional is evaluated using automatic differentiation [ 7 ] on the candidate function u_{\theta} , based on the definition of the physics residuals f_{\theta}(x,t) (cf. ( 1 )): \mathcal{G}[u_{\theta}](x,t)\triangleq f_{\theta}(x,t)=\frac{\partial u_{\theta}}{\partial t}(x,t)-\mathbf{\mathcal{N}}[u_{\theta};\lambda](x,t). (3c) At the boundary of the computational domain, represented by the dataset \mathcal{D}_{bc}\subseteq\mathcal{D}_{u} , the unsupervised loss ( 3b

Read the complete text