Accelerating the Simulation of Ordinary Differential Equations Through Physics-Preserving Neural Networks

Andrew Tagg, Andrew Frandsen, Andrew Ning · 2026

Read Accelerating the Simulation of Ordinary Differential Equations Through Physics-Preserving Neural Networks by Andrew Tagg, Andrew Frandsen, Andrew Ning (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-06980.txt.

Introduction

Andrew Tagg, Andrew Frandsen, and Andrew Ning propose a way to speed up simulations of ordinary differential equations. Some systems change so fast that solvers must take tiny steps, which makes runs costly. Their method maps the system state into a larger learned space where motion is slower.

Unlike many neural methods, the new equations in that space are derived from the original equations, not learned from sample paths. That derivation should help the method generalize beyond its training cases. On several test problems, the authors solve the equations with far fewer function calls at the same accuracy. That saving matters across engineering and physics.

Catalog record

CREATORAndrew Tagg, Andrew Frandsen, Andrew Ning
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.06980

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Accelerating the Simulation of Ordinary Differential Equations Through Physics-Preserving Neural Networks Andrew Tagg, Andrew Frandsen, Andrew Ning Abstract Numerical simulation of ordinary differential equations (ODEs) can be challenging when the system exhibits high accelerations and rapidly changing dynamics. Under these conditions the ODE solver often needs to take very small time steps in order to resolve the solution accurately, resulting in increased computational cost. In order to accelerate the simulation of these ODEs we present a novel methodology that uses a pseudo-invertible neural network to map system states into a high-dimensional latent-space. The network is then trained so that the dynamics in this learned latent space are slow, and can be simulated with relatively few function calls. Unlike existing neural methods, the latent dynamic equations are not learned from trajectory data, but derived from the original system equations and the chain rule. This allows the method to generalize better than existing approaches because the derived equations are correct by construction. In this work, we derive latent state equations of motion for any general ODE, and describe the loss function used to enforce slow time evolution of the latent states. We then apply this technique to multiple example ODEs and show that these problems can be solved with 3 3 x to 20 20 x fewer function calls for the same accuracy when simulating in the learned latent space. This reduction in cost could decrease computational demands for scientific simulations across engineering and physics applications. 1 Introduction Ordinary differential equations (ODEs) are an essential mathematical tool for modeling how dynamic systems evolve. The use of ODEs for analyzing and optimizing systems is widespread throughout all fields of physics and engineering. Across all these applications, a common challenge when simulating ODEs occurs when a system exhibits rapidly changing dynamics. When these high accelerations occur, the numerical solver must take very small steps in order to resolve the trajectories accurately. This leads to increased computational cost, and simulations that can take hours or days to complete. The development of efficient numerical solution methods targeted toward fast changing dynamics is a well established field of research. State of the art ODE solvers include Runge-Kutta methods 10 ; 3 , Taylor methods 18 , and multi-step methods 1 , all of which construct a high-order approximation of the local trajectory before taking a step. For stiff systems, which have fast and slow dynamic modes, implicit methods are designed specifically to prevent numerical instability, but they often require more derivative evaluations to do so 16 . All of these ODE solvers usually benefit from adaptive time stepping schemes 11 . These adaptive methods vary the time step of the solver based on an approximation of the local error of the solution. In general, if the dynamics are smooth and slow changing, then the solver can safely take a larger time step. If the dynamics start to change more rapidly, then the solver must shrink its time step to maintain accuracy. Recently, deep learning has emerged as a viable approach for efficiently solving ODEs. Architectures like recurrent neural networks 22 , LSTMs 9 , and GRUs 21 naturally model sequential trajectory data without computing expensive derivatives. Neural ODEs 2 combine deep learning with advanced ODE solvers by representing the continuous time derivative with a neural network that can be simulated forward by any ODE solver. Other creative applications include learning mappings from coarse to fine discretizations 4 , improving accuracy for larger time steps 12 , and using autoencoders to map dynamic states to more desirable latent spaces 15 . Learning latent representations through autoencoders has led to improvements in dimensionality reduction 8 , Koopman linearization 20 , neural ODE learning 17 , and numerical stability 6 . Neural networks can be a powerful tool for modeling ODEs efficiently, but their main limitation is the amount of data required to learn the underlying dynamics. In order to learn the dynamics of the system, the network must have access to a sufficiently large set of training trajectories. In many applications, obtaining enough data for the model to generalize is a challenge because the data is expensive to generate and/or the trajectory space is very high-dimensional. This problem occurs in high-fidelity fluid simulations 19 , astrodynamic simulations 13 , financial markets 7 , and chemical reactions 14 . In these domains where the dynamic equations are known but expensive to evaluate, computational cost is a major bottleneck. Advanced numerical solvers have led to drastic improvements, but speedups are still limited when the underlying dynamics are fast, and induce very small solver time steps. Deep learning is a possible avenue to improve computational efficiency at run time, but the training process often requires a large data set that is prohibitively expensive to generate. This paper presents a novel method to address these challenges and accelerate the simulation of ODEs. The method employs a neural network architecture called the affine coupling layer 5 to define a pseudo-invertible mapping from the system’s dynamic states to a higher-dimensional latent-space. In this latent space, the dynamics are not learned from trajectory data, but derived directly from the original equations and the chain rule, making them automatically correct. The goal of network training in this framework is to learn a mapping such that the derived latent dynamics evolve smoothly and slowly. In this way, the simulation of the ODE is accelerated because the solver may safely take larger time steps. Because the dynamic equations are correct by construction, this method can generalize better than the traditional approach of learning unknown dynamics from data. We first develop the methodology of this approach, deriving the equations of motion in the learned latent-space, and describe the loss function used to enforce slow latent dynamics. Then we apply this technique to several benchmark examples, demonstrating accelerated ODE solutions with speedups ranging from 3 x to 20 x fewer function calls than the original simulations. To evaluate the generalizability compared to existing methods, we also simulate these systems with a physics-informed neural ODE (PINODE) 17 and show that while the PINODE performs well for trajectories similar to those provided during training, it struggles to extrapolate beyond the training data provided. In contrast, the new method can predict trajectories outside ts training data with arbitrary accuracy. 2 Methods 2.1 Pseudo-Invertible Architecture Let \mathbf{x}\in\mathbb{R}^{n} be the state vector of an ODE of interest. We will learn a latent representation of the state-space by defining a mapping \mathbf{z}(\mathbf{x}) , where \mathbf{z}\in\mathbb{R}^{m>n} . As we will see, constructing \mathbf{z} in a higher-dimensional space will allow more expressivity in the latent representation and improve the network’s ability to learn dynamics that are slow in \mathbf{z} -space. We desire this mapping to be pseudo-invertible, which is important not only for generalization of the mapping, but also for the validity of the derived equations of motion. To achieve this, we will define the following mapping \mathbf{z}=\phi(\mathbf{A}\mathbf{x}) (1) where \mathbf{A}\in\mathbb{R}^{m\times n} is a linear operation that lifts the original states to a higher dimension, and \phi is a nonlinear invertible map defined by a neural network. The pseudo-inverse of the mapping is \mathbf{x}=\mathbf{A}^{{\dagger}}\phi^{-1}(\mathbf{z}) (2) where \mathbf{A}^{{\dagger}} is the Moore-Penrose pseudo-inverse defined by \mathbf{A}^{{\dagger}}=(\mathbf{A}^{T}\mathbf{A})^{-1}\mathbf{A}^{T} . For the function \phi

Read the complete text