PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks
Marius Almanstötter, Roman Vetter, Dagmar Iber · 2025
Read PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks by Marius Almanstötter, Roman Vetter, Dagmar Iber (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-05248.txt.
Introduction
Marius Almanstötter, Roman Vetter, and Dagmar Iber present a study from twenty twenty-five on reading off the constants of an equation from noisy data. Networks that learn with physics rules built in can do this task. They strain, though, when data is sparse: runs can stall, wobble, or overfit. Loss design is also a chore.
Their frame, PINNverse, recasts training as a bound optimization task. It keeps the data loss and the equation loss in live balance as training runs. A multiplier method lets it land on any point of the trade-off front. Tests on four classic models from physics and life science gave sound, true values from noisy data. The frame also helps when the forward run is costly.
Catalog record
| CREATOR | Marius Almanstötter, Roman Vetter, Dagmar Iber |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-05248
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
PINNverse: Accurate parameter estimation in differential equations from noisy data with constrained physics-informed neural networks Marius Almanstötter, Roman Vetter, Dagmar Iber Abstract Parameter estimation for differential equations from measured data is an inverse problem prevalent across quantitative sciences. Physics-Informed Neural Networks (PINNs) have emerged as effective tools for solving such problems, especially with sparse measurements and incomplete system information. However, PINNs face convergence issues, stability problems, overfitting, and complex loss function design. Here we introduce PINNverse, a training paradigm that addresses these limitations by reformulating the learning process as a constrained differential optimization problem. This approach achieves a dynamic balance between data loss and differential equation residual loss during training while preventing overfitting. PINNverse combines the advantages of PINNs with the Modified Differential Method of Multipliers to enable convergence on any point on the Pareto front. We demonstrate robust and accurate parameter estimation from noisy data in four classical ODE and PDE models from physics and biology. Our method enables accurate parameter inference also when the forward problem is expensive to solve. keywords Physics-Informed Neural Network, Differential equations, Inverse problem, Noisy data, Parameter estimation, Constraint Differential Optimization Introduction Accurate modeling of complex phenomena in science and engineering often necessitates solving differential equations whose parameters characterize essential physical properties. Directly measuring these parameters can be difficult or impractical, which has prompted the development of inverse methods designed to infer unknown parameters from observational data. Traditionally, inverse problems have been tackled using two main methodological frameworks. The frequentist approach seeks parameter estimates by maximizing the likelihood [ 1 , 2 ] of observed data given the model predictions [ 3 , 4 , 5 ] . However, likelihood optimization is complicated by loss landscapes that exhibit multiple modes and spurious local minima, resulting in solutions that strongly depend on initial guesses [ 6 ] . Consequently, multiple local minimizations from different starting points or global optimization strategies are typically employed, considerably increasing computational costs [ 7 ] . In contrast, Bayesian methods consider parameters as random variables [ 8 ] , using Markov chain Monte Carlo algorithms to estimate posterior distributions, which naturally quantify parameter uncertainties [ 9 , 10 ] . However, such approaches require extensive forward model evaluations, rely heavily on priors, and can face convergence challenges [ 11 , 12 ] . Conventionally, forward problems that are formulated as differential equations (DEs) are solved using numerical schemes such as Runge–Kutta, Finite Differences, Finite Volumes, or Finite Elements. Recently, Physics-Informed Neural Networks (PINNs) [ 13 , 14 ] have emerged as an attractive alternative. PINNs leverage deep neural networks in conjunction with automatic differentiation and gradient-based optimization to approximate the solutions of DEs. By incorporating DE constraints directly into the training objectives—including governing equations, initial and boundary conditions—PINNs learn to approximate the underlying physical laws. This strategy is unsupervised, mesh-free, and has successfully addressed a variety of forward and inverse problems as an alternative to classical numerical methods [ 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 ] . Nevertheless, employing PINNs can still present challenges, such as complex loss landscapes due to soft enforcement of constraints and difficulties encountered in scenarios involving shock waves [ 24 , 25 ] . Consequently, several enhancements have been proposed over recent years to enforce physical laws more effectively and to expand the applicability of PINNs. They include respecting temporal causality [ 26 ] , including Fourier features in the first layer [ 27 ] , curriculum-based training [ 25 ] , adaptive resampling strategies during training [ 28 , 29 ] , adaptive weights during training [ 30 , 31 , 32 ] and augmenting the training process with additional equations [ 33 ] . When training PINNs, a data loss term is introduced that competes with the physics-based loss component. This interplay has recently prompted increased interest in multi-objective optimization techniques and the exploration of Pareto fronts within the context of PINNs [ 34 , 35 , 36 , 37 , 38 , 39 ] . These studies typically emphasize on adaptive weighting schemes to manage the relative contributions of various loss terms, or apply evolutionary algorithms, such as NSGA-II [ 40 ] , explicitly targeting multi-objective optimization. However, these evolutionary methods often involve substantially greater computational expense compared to conventional PINN training. The adaptive weighting schemes introduce additional, often sensitive, hyperparameters that themselves require careful tuning, adding another layer of complexity to the problem. In inverse problem scenarios with substantial noise in the observational data, simultaneously minimizing data and physics losses to zero is unfeasible, necessitating a deliberate trade-off between these competing objectives. Furthermore, equal balancing of losses is typically undesirable, as it can result in overfitting the data noise instead of accurately capturing the underlying physics. To overcome these limitations, we propose a constrained optimization framework tailored to identify optimal PINN solutions that accurately fit observational data while strictly enforcing physical constraints. Previous investigations into constrained optimization approaches within PINNs have demonstrated networks structured to exactly fulfill initial and boundary conditions, while employing augmented Lagrangian methods to minimize physics and data losses [ 41 ] . Additionally, stochastic augmented Lagrangian methods have been effectively applied to train PINNs [ 42 ] . However, the augmented Lagrangian requires nested loops for training, which are typically not employed in deep learning, and increases computational complexity. To overcome this limitation, we propose employing the Modified Differential Method of Multipliers (MDMM) [ 43 ] , an optimization strategy that simultaneously updates the neural network parameters, differential equation parameters, and associated Lagrange multipliers in parallel. This method is fully compatible with modern state-of-the-art optimizers, such as Adam [ 44 ] or Adan [ 45 ] , and importantly, it does not incur additional computational costs beyond those of standard PINN training methods. We demonstrate on four classical examples that the proposed training paradigm outperforms standard PINNs, especially with high levels of noise in the data, and traditional optimizers such as the Nelder–Mead [ 46 ] when the initial parameter guess is inaccurate. Results Figure 1: Schematic representation of the difference between PINN and PINNverse. When approximating solutions to differential equations (DEs) in residual form \mathcal{F}=0 with a neural network (NN), the architecture utilizes hidden layers (parameter set \bm{\theta} ) to map input variables including spatial coordinates \bm{x} and time t to the solution space of state variables, represented by a k -dimensional vector of functions \bm{u}^{\bm{\theta}} (orange box). The training process of a PINN (purple box) minimizes a composite loss function incorporating terms penalizing deviations of the predicted solution \bm{u}^{\bm{\theta}} from observed noisy data points \bm{u}^{\textrm{data}}_{i} (blue dots) ( L_{\textrm{data}} ), as well as terms that penalize violations of the differential equations ( L_{\mathrm{de}} ) and of the initial and/or boundary conditions ( L_{\textrm{ic}} , L_{\textrm{bc}} ). The predicted NN solution typically suffers from overfitting and misses non-convex parts (solid red line) of the Pareto front (solid black line). Examples of trajectories starting from initial points within the feasible region (dashed area) and leading to this front are schematically visualized for a 2D subspace. With PINNverse (green box), the data, IC and BC losses are formulated as external constraints under which the optimization is carried