Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations

Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha · 2025

Read Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations by Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-03860.txt.

Introduction

Chandra Kanth Nagesh and four co-authors blended symbolic calculus with neural networks in twenty twenty-five. They model sets of differential equations when things are not known for sure. Physics-informed neural networks learn the solutions of an equation. They do so across parameters and first states. But they lose accuracy across a whole set of problems. The team reaches for a classic tool: Taylor series.

Their models expand solutions in symbols. They use higher-order Lie derivatives to do it. Just the Taylor remainder is left for a neural network to learn. That remainder itself obeys a first-order equation. This gives the learning a firm shape. On hard test equations the hybrid models beat standard physics-informed networks on accuracy. The team notes the models help with control of uncertain physical systems. Such systems are described by equations of this kind.

Catalog record

CREATORChandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-03860

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Taylor-Model Physics-Informed Neural Networks (PINNs) for Ordinary Differential Equations Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha Abstract We study the problem of learning neural network models for Ordinary Differential Equations (ODEs) with parametric uncertainties. Such neural network models capture the solution to the ODE over a given set of parameters, initial conditions, and range of times. Physics-Informed Neural Networks (PINNs) have emerged as a promising approach for learning such models that combine data-driven deep learning with symbolic physics models in a principled manner. However, the accuracy of PINNs degrade when they are used to solve an entire family of initial value problems characterized by varying parameters and initial conditions. In this paper, we combine symbolic differentiation and Taylor series methods to propose a class of higher-order models for capturing the solutions to ODEs. These models combine neural networks and symbolic terms: they use higher order Lie derivatives and a Taylor series expansion obtained symbolically, with the remainder term modeled as a neural network. The key insight is that the remainder term can itself be modeled as a solution to a first-order ODE. We show how the use of these higher order PINNs can improve accuracy using interesting, but challenging ODE benchmarks. We also show that the resulting model can be quite useful for situations such as controlling uncertain physical systems modeled as ODEs. \coltauthor and University of Colorado Boulder and , and SRI International keywords Physics-Informed Neural Networks, Initial Value Problems, Ordinary Differential Equations, Taylor Models 1 Introduction Finding closed-form analytic solutions to systems of Ordinary Differential Equations (ODEs) is challenging for all but the simplest class of systems. The problem is even more challenging for ODEs with parameters that can take on a set of possible values, unknown initial conditions and external inputs. Physics-Informed Neural Networks (PINNs) have emerged as a solution to finding approximate closed-forms modeled as neural networks Raissi et al. 2018 . They have been studied for solving PDEs, especially non-linear PDEs that are hard to solve numerically. The problem of PINNs for ODEs have received considerably less attention since numerical solvers are quite successful in finding solutions for many common ODEs Hairer et al. 1993 . However, the use of numerical solvers is distinctly problematic for applications that involve solving optimization problems involving ODEs with unknown parameters and inputs . These arise in machine learning, where one wishes to learn parameters from data or optimal control, wherein we seek control inputs that optimize a function across the trajectories of the system. For such applications, it is desirable to have a “surrogate model” that can capture the solution of the ODEs with high enough accuracy over a range of parameters, initial conditions and times. If each evaluation of the surrogate can be performed more efficiently than using a numerical solver, the overall optimization can be faster. We investigate the use of PINNs to build surrogate models that capture the solution to an initial value problem (IVP) given by a system of ODEs \dot{\vec{x}}=f(\vec{x},\vec{\theta},t) for parameters \vec{\theta}\in\Theta , initial conditions \vec{x}(0)\in\Omega and t\in[0,T] . In other words, our surrogate model \varphi(\vec{x}_{0},\vec{\theta},t) maps the inputs to a solution \vec{x}(t;\vec{x}_{0},\vec{\theta}) of the ODE at time t . The standard PINN approach of Raissi et al. 2019 a) uses a neural network to represent \varphi and b) a combination of two loss functions given by the initial condition loss \|\varphi(\vec{x}_{0},\vec{\theta},0)-\vec{x}_{0}\| and the gradient loss \|\dot{\varphi}-f(\varphi,\vec{\theta},t)\| averaged at various randomly chosen “collocation points”. In this paper, we first point out the inadequacy of the PINN approach to this problem by demonstrating its failure to approximate the solution when the sets \Theta,\Omega and [0,T] are large. We show that higher-order loss functions fail to address the issue. Therefore, we resort to a symbolic approach that uses successive Lie derivatives to compute the terms of a Taylor series expansion of the solution. We show that the remainder when carefully modeled can be written down as the solution to a derived ODE. Solving this derived ODE for the higher-order remainder using the “classic” PINN approach yields a solution that combines the best aspects of symbolic differentiation with neural network learning. We show that our error grows as O(t^{m+1}e^{Kt}) for an approach that uses derivatives up to order m\geq 1 , whereas, for PINNs, the error grows as O(te^{Kt}) . As a result, our approach provides high levels of accuracy at the initial times. We compare our approach, which we call “Taylor-Model PINNs” with PINNs, and the related approach of “Higher-Order PINNs” that extends the original PINN loss function with higher order derivative-based loss functions. We show that Taylor-Model PINNs provide higher accuracy. While our approach increases the complexity of the training process, the use of efficient symbolic differentiation tools offsets this process. 1.1 Related Work Machine learning approaches have found applications in diverse domains ranging from celestial object classification Angeloudi et al. 2024 , climate forecasting Iglesias-Suarez et al. 2024 , and tumor identification Li et al. 2023 . In such scenarios, it has been observed that using background knowledge in the form of mathematical models in the learning process can considerably speed up convergence and improve solution quality. Physics-Informed Neural Networks (PINNs) Raissi et al. 2019 represent a seminal contribution in this space. They have been effective in solving systems which involve partial differential equations (PDEs), where data is sparsely available. By utilizing differentiable loss functions, a neural network is trained on the PDE residual and boundary condition loss to learn a solution map to the PDE. These physics-inspired loss terms act as a regularizer against learning solution maps that do not involve the underlying dynamics of the system, thereby conforming well to how the system evolves over time. This promising approach has led to widespread use of the methodology in various applications Shukla et al. 2020 , Wang and Perdikaris 2021 , Yin et al. 2021 . Despite their contributions, PINN methodologies can often fail to learn physical dynamics in many cases Krishnapriyan et al. 2021 , Steger et al. 2022 . To tackle such issues, there have been efforts into PINNs with additional loss functions Son et al. 2023 , Wang et al. 2022 . However, their efficacy diminishes under two conditions (a) conflicting gradient updates between the two loss functions, leading to suboptimal gradient descent Hwang and Lim 2025 and (b) when applied to parametric PDE families, particularly those requiring simultaneous resolution across a range of initial conditions and parameters Xiang et al. 2025 . The Taylor series expansion represents a fundamental concept in mathematical analysis, providing a powerful framework for approximating functions through polynomials or power-series Apostol 1991 . Our approach of using Taylor series expansions has been heavily influenced by the work of Makino and Berz, who have applied so-called “Taylor-model calculus” to represent a set of complex and unknown functions by a finite Taylor series expansion with an interval remainder Berz and Makino 1998 ; Makino and Berz 2009 . This has led to popular approaches in the area of formal methods for proving properties of Cyber-Physical Systems Chen and Sankaranarayanan 2022 ; Althoff et al. 2021 ; Althoff 2015 ; Kong et al. 2015 . Here, we adapt Taylor models to represent

Read the complete text