TI-DeepONet: Learnable Time Integration for Stable Long-Term Extrapolation

Dibyajyoti Nayak, Somdatta Goswami · 2025

Read TI-DeepONet: Learnable Time Integration for Stable Long-Term Extrapolation by Dibyajyoti Nayak, Somdatta Goswami (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17341.txt.

Introduction

Dibyajyoti Nayak and Somdatta Goswami present TI-DeepONet, a method to forecast how a physical system evolves over long times. Their twenty twenty-five paper uses neural operators, models that learn maps between functions. Standard versions either jump to a fixed horizon or step forward one frame at a time, piling up error.

Their method learns the rate of change of the state, not the state itself. A classic numerical solver then steps that rate forward, with steps that can grow finer at test time. A variant learns the stage weights of the solver too. On six test equations, errors fell sharply against both older styles, and forecasts stayed stable for near twice the training span.

Catalog record

CREATORDibyajyoti Nayak, Somdatta Goswami
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE107 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17341

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TI-DeepONet: Learnable Time Integration for Stable Long-Term Extrapolation Dibyajyoti Nayak, Somdatta Goswami Abstract Accurate temporal extrapolation presents a fundamental challenge for neural operators in modeling dynamical systems, where reliable predictions must extend significantly beyond the training time horizon. Conventional deep operator network (DeepONet) approaches rely on two inherently limited training paradigms: fixed-horizon rollouts, which predict complete spatiotemporal solutions while disregarding temporal causality, and autoregressive formulations, which accumulate errors through sequential predictions in time. We introduce TI-DeepONet (Time-Integrator-embedded Deep Operator Network) , a framework that integrates neural operators with adaptive numerical time-stepping techniques to preserve the underlying Markovian structure of dynamical systems while substantially mitigating error propagation in extended temporal forecasting. Our approach reformulates the learning objective from direct state prediction to the approximation of instantaneous time-derivative fields, which are subsequently integrated using established numerical schemes. This design naturally supports continuous-time prediction and enables the use of higher-precision integrators at inference than those employed during training, striking a balance between computational efficiency and predictive accuracy. We further develop TI(L)-DeepONet (Learnable Time-Integrator-embedded Deep Operator Network) , which incorporates learnable coefficients for intermediate stages in a multi-stage numerical integration scheme, thereby adapting to solution-specific variations and enhancing predictive fidelity. Rigorous evaluation across six canonical partial differential equations (PDEs) spanning diverse, high-dimensional, chaotic, dissipative, and dispersive dynamics shows that TI(L)-DeepONet marginally outperforms TI-DeepONet, with both frameworks achieving significant reductions in relative L 2 L_{2} extrapolation error: approximately 96.3% compared to autoregressive implementations and 83.6% compared to fixed-horizon approaches. Notably, both methods maintain stable predictions over temporal domains extending to nearly twice the training interval. This research establishes a physics-aware operator learning paradigm that bridges neural approximation with numerical analysis principles, preserving the causal structure of dynamical systems while addressing a critical gap in long-term forecasting of complex physical phenomena. 1 Introduction Time-dependent partial differential equations (PDEs) are typically solved using spatial discretization (e.g., finite element or volume) combined with time-stepping schemes such as Runge-Kutta or Adams-Bashforth. While accurate, such solvers can be prohibitively expensive for high-dimensional or many-query problems. Explicit integrators are constrained by stability conditions (e.g., CFL), while implicit schemes require solving large systems of equations at each step. Moreover, classical solvers generalize poorly: each new initial or boundary condition requires a fresh simulation. Neural operators (NOs) [ 1 , 2 , 3 , 4 , 5 , 6 , 7 ] offer an attractive alternative by learning mappings from input functions (e.g., initial conditions) to solutions, bypassing discretization and enabling fast inference. Models like DeepONet [ 1 , 8 , 9 , 10 , 11 ] and FNO [ 2 ] have shown promise in parametric PDE settings. However, extending NOs to time-evolving systems introduces new challenges. The standard approach is to train a model that maps the solution at time t to the solution at time t+\Delta t , and roll this forward autoregressively [ 12 ] (see Figure 1 ). Unfortunately, this approach suffers from severe error accumulation, as each prediction feeds into the next, compounding inaccuracies and often leading to instability. Alternatively, one can train a model to predict an entire trajectory (“full rollout”), which avoids stepwise feedback but tends to degrade in accuracy when extrapolating beyond the training time window. Crucially, most existing works [ 13 , 2 , 14 , 1 ] only evaluate models within the training time span, neglecting the need for reliable extrapolation, a capability that is essential for real-time engineering applications and control tasks. Moreover, existing work [ 15 , 16 , 17 ] on autoregressive and full rollout operator models tends to impose a discrete-time Markovian assumption, which misaligns with the underlying continuous-time dynamics of PDEs. These limitations often result in unstable or inaccurate long-term predictions. Thus, a key barrier remains: NOs struggle not only with long-horizon accuracy but also with extrapolating into unseen temporal domains, a prerequisite for trustworthy deployment. Several recent works attempt to address these limitations. For example, Diab et al. [ 18 ] augment DeepONet with an additional temporal branch to capture dependencies, while Michalowska et al. [ 19 ] employ recurrent networks (RNNs, LSTMs, GRUs) atop neural operator outputs to better handle sequences. Other approaches introduce memory modules [ 20 , 21 , 22 , 23 ] to compensate for the non-Markovian nature of rollout-based methods. We note that such memory-based approaches are conceptually related to the Mori-Zwanzig formalism [ 24 ] , which introduces memory terms to account for unresolved dynamics; however, our approach takes a different perspective by preserving the Markovian structure inherent in the full-order system through sequential time integration. However, these strategies largely rely on generic time-series architectures rather than tackling the issue from a dynamical systems perspective. In this work, we introduce a novel approach inspired by classical numerical analysis. Our proposed TI-DeepONet framework embeds time integration directly into the operator learning process for dynamical systems. Rather than predicting future solution states directly, we train a DeepONet to approximate the instantaneous time derivative \partial u/\partial t . During training, this derivative operator is coupled with a Runge-Kutta numerical integrator to evolve the state forward in time. The predicted future state is then compared against the ground truth in the loss function, enabling the network to learn dynamics that respect temporal causality and numerical stability. At inference time, we leverage high-order multistep integrators (such as Adams-Bashforth/Adams-Moulton predictor-corrector schemes) with refined timesteps to reliably propagate solutions based on the learned time derivative operator. This operator learning framework which maps between function spaces rather than from function spaces to vector spaces as in standard neural networks and neural ODEs [ 25 ] - provides the flexibility to modify time-stepping parameters between training and testing phases, thereby ensuring stable predictions. We further introduce TI(L)-DeepONet, an enhanced variant that incorporates learnable weighting coefficients for the intermediate stages conditioned on the system state at each timestep. This adaptation dynamically adjusts the integration process based on local solution characteristics. While TI-DeepONet shares conceptual similarities with neural ODEs in learning time derivatives and using numerical integration, our approach differs significantly in both scope and implementation. Neural ODEs typically operate on low-dimensional latent states, whereas TI-DeepONet is specifically designed for high-dimensional spatiotemporal fields and learns an explicit operator mapping between function spaces. Furthermore, unlike neural ODEs, which treat the integrator as fixed, we explicitly embed the integrator within the solution operator, allowing us to employ different integration schemes at inference time according to accuracy and efficiency requirements. We demonstrate the effectiveness of our approach on six benchmark PDEs: (1) 1D Burgers’, (2) 1D Korteweg-de Vries (KdV), (3) 1D Kuramuto-Sivashinsky (KS), (4) 2D Burgers’ , (5) 2D Scalar Rotation Advection-Diffusion , and (6) 3D Heat Conduction equations. We compare our methods against four standard neural operator (NO) baselines: (1) autoregressive DeepONet (DON AR), (2) full rollout DeepONet (DON FR), (3) autoregressive FNO (FNO AR), and (4) full rollout FNO (FNO FR). Across the majority of cases, TI-DeepONet and its adaptive variant TI(L)-DeepONet significantly outperform standard autoregressive and full rollout operator models, achieving stable long-term predictions and accurate extrapolation for time horizons approximately twice as long as the training interval. However, for problems involving rigid body rotational dynamics (e.g., the rotation advection-diffusion problem), FNO AR demonstrates superior performance, highlighting that problem characteristics should

Read the complete text