Deep Learning of the Evolution Operator Enables Forecasting of Out-of-Training Dynamics in Chaotic Systems
Ira J. S. Shokar, Peter H. Haynes, Rich R. Kerswell · 2025
Read Deep Learning of the Evolution Operator Enables Forecasting of Out-of-Training Dynamics in Chaotic Systems by Ira J. S. Shokar, Peter H. Haynes, Rich R. Kerswell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20603.txt.
Introduction
Ira Shokar, Peter Haynes, and Rich Kerswell show in 2025 that a deep learning model of chaotic flow can predict events it never saw in training. A common fear is that such models just copy past patterns. The team tests that fear on two classic chaotic systems.
Their model forecast a sudden calm state, started well from fresh chaotic states, and worked at settings outside the training range. It also kept true long-term traits when training data were cut in a planned way. The results suggest the model learned the math rules under the flow, not just the look of past runs. That opens a path to spotting rare events in complex systems.
Catalog record
| CREATOR | Ira J. S. Shokar, Peter H. Haynes, Rich R. Kerswell |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 27 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-20603
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Deep Learning of the Evolution Operator Enables Forecasting of Out-of-Training Dynamics in Chaotic Systems Ira J. S. Shokar, Peter H. Haynes, Rich R. Kerswell Abstract We demonstrate that a deep learning emulator for chaotic systems can forecast phenomena absent from training data. Using the Kuramoto-Sivashinsky and beta-plane turbulence models, we evaluate the emulator through scenarios probing the fundamental phenomena of both systems: forecasting spontaneous relaminarisation, capturing initialisation of arbitrary chaotic states, zero-shot prediction of dynamics with parameter values outside of the training range, and characterisation of dynamical statistics from artificially restricted training datasets. Our results show that deep learning emulators can uncover emergent behaviours and rare events in complex systems by learning underlying mathematical rules, rather than merely mimicking observed patterns. Chaotic systems are known to exhibit sensitivity to initial conditions, limiting long-term predictability. This unpredictability challenges the forecasting of large-scale behaviours and rare or extreme events, such as sudden regime shifts or extreme weather occurrences in the climate system, for example, which are critical for understanding the system’s dynamics pred_chaos . Recent advances in machine learning (ML) have shown significant promise in modelling chaotic dynamics pathek . However, a fundamental limitation remains unclear: do these models inherently fail to predict beyond what is seen during training? This can manifest in two main ways: an inability to predict unseen dynamics at the same parameter values of the training set, for a given dynamical system, or the inability to make predictions into new parameter regimes beyond the training data (the ’extrapolation’ problem physics_ML ). While the latter is unfortunate, the former is more critical as it calls into question whether an ML model can be used to predict extreme or rare events which are infrequent in or entirely absent in the training dataset fluids_ML_review ; pedram ; pred_lim ; cautionary ; extrap . Understanding whether these limitations can be circumvented is crucial for unlocking the full potential of ML as a tool for understanding chaotic systems. In this paper, we address both limitations by demonstrating that a deep learning emulator can successfully predict unobserved dynamics in two chaotic systems in and beyond the parameter regimes of the training data set. The chaotic systems considered are the Kuramoto-Sivashinsky (KS) equation unstableKS , a well-studied canonical model of spatiotemporal chaos, and beta-plane turbulence beta_turb , a useful simplest model for many aspects of flows in atmospheres and oceans. The emulator constructed accurately reproduces known behaviours and predicts phenomena not present in the training data, including relaminarisation events and initialisation dynamics. We believe that this is because the emulator constructs a sufficiently accurate approximation of the evolution operator {\cal F}_{\Delta t} that advances the solutions of the underlying partial differential equation (PDE), \partial_{t}u=F(u) , over time u_{t+\Delta t}={\cal F}_{\Delta t}(u_{t}) , without having to see all possible dynamical behaviours allowed in these systems, i.e. it is able to do more than just learn observed patterns. Figure 1: Prediction of out-of-training-distribution dynamics of the KS equation with L=56 , where training data excludes relaminarisation events or warm-up dynamics. (a) Relaminarisation event observed in direct numerical simulation (DNS). (b) Neural network (NN) emulation predicts the relaminarisation event from identical initial conditions. (c) Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) between the numerical simulation (a) and NN prediction (b). (d) Initialisation dynamics of KS flow from DNS. (e) NN emulation forecasts the correct dynamics from the same initial conditions. (f) MAE and RMSE between the initialisation dynamics (d) and NN prediction (e). The largest Lyapunov exponent for L=56 is \Lambda_{\text{max}}\approx 0.048 . Our first system is the Kuramoto-Sivashinsky (KS) equation, defined as F(u):=-u\partial_{y}u-\partial_{yy}^{2}u-\partial_{yyyy}^{4}u , where u(y,t) is a scalar field on the periodic domain [0,L) . As the domain size L increases, the system undergoes a series of bifurcations, leading to increasingly complex behaviour unstableKS . Previous work on emulating the KS equation has been limited to fixed domain sizes KS0 ; KS1 ; KS2 ; Chen_2021 or requires ab initio training for different L values pathek . Instead, we develop an emulator capable of handling varying spatial dimensions while learning parameter-dependent dynamics (see End Matter for details). This employs a transformer Attention with local attention SASA to capture spatial correlations across scales while efficiently handling inputs of any size. Restricting correlations to local windows imposes a physical locality bias natural to the KS equation and many other PDEs as well as reducing the memory overhead typically associated with transformers. The network is conditioned on parameter L via adaptive layer normalisation FILM , which modulates activations to learn parametric (here domain-size) dependence-facilitating generalisation across parameter regimes. Direct numerical simulation (DNS) of the KS equation was performed on a GPU using FourierFlows FourierFlows in Julia Julia , employing a pseudo-spectral method with a fourth-order exponential time differencing Runge-Kutta solver, with \delta t=2.5\times 10^{-2} , which is subsampled to \Delta t=1 for training. Initialised with small amplitude noise u_{\text{initial}}\sim\mathcal{N}(0,0.01) , only dynamics after a warm-up phase are used as training data. The network is pretrained on a weakly chaotic regime in a small domain ( L=22 ), where y\in\mathbb{R}^{56} (the domain is discretized by 56 uniformly-spaced grid points) using 5,000 snapshots, allowing the bulk of the computation to be done efficiently. The model is then fine-tuned, in the chaotic regime, on larger domains Lin\{22,36,48,64,98,128,200\} , with y\in\mathbb{R}^{\{56,90,120,160,246,320,500\}} respectively to maintain the same grid spacing, and using 500 snapshots each to learn dependence on the parameter L . The optimisation problem solved in training is to minimise the Mean Squared Error \text{MSE}=\frac{1}{L}\int_{0}^{L}\left|u_{t+\Delta t}-\mathcal{F}_{\Delta t}(u_{t})\right|^{2}dx over the one step prediction, to obtain an approximation for \mathcal{F}_{\Delta t} . The emulation is then auto-regressive, advancing the system in time according to the rule \tilde{u}_{t+\Delta t}=\mathcal{F}_{\Delta t}(\tilde{u}_{t}) . For evaluation we use both the Mean Absolute Error ( \text{MAE}(t)=\frac{1}{L}\int_{0}^{L}\left|u_{t}-\tilde{u}_{t}\right|dx ) and Root Mean Squared Error ( \text{RMSE}=\sqrt{\text{MSE}} ). We first start with relaminarisation—the process by which chaotic dynamics are attracted to an equilibrium state or a travelling wave solution back , see Fig.1 (a). Despite being trained exclusively on chaotic dynamics for L\neq 56 , the emulator accurately forecasts both the onset and progression of a transition to laminar dynamics at L=56 : see Fig.1 (b). It captures the precise timing of the transition and reproduces the final stable wave pattern, demonstrating capture of the system’s stability properties and the ability to predict the fixed point. Relaminarisation represents a qualitatively distinct flow regime from any represented in the training data, and capturing this regime requires the model to learn the fundamental mathematical rules describing the physics rather than relying on pattern matching. Fig.1 (c) shows the MAE and RMSE between the neural network (NN) emulation and DNS. These measures grow slowly due to a latitudinal translation drift in the laminar solution (an equally valid solution). The emulator also accurately captures the transient growth phase in the KS equation, where small-amplitude noise evolves into fully developed chaos, as shown in Fig.1(d,e). This transition is characterised by the formation of localised structures (‘kinks’)—steep gradients in the solution that separate distinct spatial regions, visualised here as red-blue pairs. Note that it is not just the development of finite amplitude chaotic patterns, but the precise spatial structure that is captured. This is particularly notable