True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics
Christoph Jürgen Hemmer, Daniel Durstewitz · 2025
Read True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics by Christoph Jürgen Hemmer, Daniel Durstewitz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13192.txt.
Introduction
Christoph Jürgen Hemmer and Daniel Durstewitz built an AI that can read a new changing system cold in a 2025 paper. Many fields, from weather to the brain, need stand in models that copy a system's long run mood. Past tools must be trained fresh for each new system, which wastes time and data when a fresh case lands.
Their model, DynaMix, learned from many systems ahead of time and works from a short sample of a new one, with no new training. It kept the long run shape of fresh systems where big time series models failed. It did so with a sliver of their size and far faster runs. Tests on real traffic and weather records backed the claim.
Catalog record
| CREATOR | Christoph Jürgen Hemmer, Daniel Durstewitz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13192
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics Christoph Jürgen Hemmer, Daniel Durstewitz Abstract Complex, temporally evolving phenomena, from climate to brain activity, are governed by dynamical systems (DS). DS reconstruction (DSR) seeks to infer generative surrogate models of these from observed data, reproducing their long-term behavior. Existing DSR approaches require purpose-training for any new system observed, lacking the zero-shot and in-context inference capabilities known from LLMs. Here we introduce DynaMix , a novel multivariate ALRNN-based mixture-of-experts architecture pre-trained for DSR, the first DSR model able to generalize zero-shot to out-of-domain DS. Just from a provided context signal, without any re-training, DynaMix faithfully forecasts the long-term evolution of novel DS where existing time series (TS) foundation models, like Chronos, fail – at a fraction of the number of parameters and orders of magnitude faster inference times. DynaMix outperforms TS foundation models in terms of long-term statistics, and often also short-term forecasts, even on real-world time series, like traffic or weather data, typically used for training and evaluating TS models, but not at all part of DynaMix’ training corpus . We illustrate some of the failure modes of TS models for DSR problems, and conclude that models built on DS principles may bear a huge potential also for advancing the TS prediction field. 1 Introduction Figure 1 : DynaMix achieves zero-shot DSR of attractor geometry and long-term temporal properties (power spectrum) from a short context signal while Chronos [ 2 ] fails. Most real-world processes, from atmospheric phenomena and stock markets to brain activity or ecological networks, can be described as dynamical systems (DS) [ 54 , 15 , 82 , 57 ] . Reconstructing these from observational data, called dynamical systems reconstruction (DSR), has been a long-standing challenge in scientific modeling [ 92 , 23 , 31 ] . DSR goes beyond conventional time series (TS) modeling, as we wish to have a generative model of the underlying process which exhibits long-term behavior with the same temporal and geometrical signatures (Fig. 1 ), i.e. the same invariant or ‘climate’ statistics [ 62 , 66 ] , as important in scientific applications. Achieving this usually requires special control-theoretic training techniques [ 56 , 38 ] or loss objectives [ 65 , 66 , 40 , 72 ] which accentuate the system’s long-term dynamics. Numerous deep learning approaches for DSR, based on recurrent neural networks (RNNs; [ 81 , 22 , 83 , 16 , 6 , 69 , 38 , 7 , 66 ] ), neural ODEs [ 17 , 43 , 1 , 45 ] , Koopman operators [ 14 , 53 , 60 , 3 , 58 , 28 , 86 ] , or library-based methods [ 12 , 52 , 41 , 19 , 55 ] , have been advanced over the years. However, all of these require purpose-training on the specific system observed and struggle to generalize beyond their training distribution [ 33 ] . Inspired by the strong in-context and zero-shot generalization abilities of LLMs [ 10 , 27 , 21 , 18 ] , there has been a push recently to develop models with likewise properties for the time series domain. Time series foundation models like Chronos [ 2 ] or Mamba4Cast [ 5 ] are pre-trained on a huge database of real-world and artificial time series, and then tasked to forecast novel time series from which snippets are presented in-context, without any parameter fine-tuning. They are not built for DSR, however, and – as we show here – typically fail to properly capture a system’s long-term behavior and the structure of its attractors (Fig. 1 ; [ 93 ] ). To address this gap, we introduce DynaMix , a novel mixture-of-experts model designed for DSR and zero-shot forecasting. By training across a diverse range of DS, DynaMix learns transferable representations, enabling it to forecast previously unseen systems without retraining, including their underlying attractor geometries and invariant statistics (Fig. 1 ). The core features of DynaMix are: • Accurate zero-shot DSR: DynaMix achieves strong generalization across diverse DS, eliminating the need for extensive fine-tuning while maintaining accuracy in reproducing attractor geometry and long-term statistics. • Multivariate information transfer: Due to its multivariate architecture, the model efficiently captures dependencies among multiple system dimensions, enabling accurate reconstruction of their coupled dynamics. Further, it is neither bound to a specific dimensionality, but can flexibly adapt to other dimensions through specific embeddings, nor to a specific context length. • Computational and parameter efficiency: The model achieves high performance with a very lightweight architecture, enabling orders of magnitude faster inference than other foundation models. • Interpretable dynamics composition: The model provides insight into the dynamical composition of reconstructed systems, elucidating similarities between different DS. • Time series forecasting: Beyond DSR, our model is applicable to general time series forecasting where we employ different embedding techniques to obtain data representations accessible to DSR. We evaluate our model on multiple benchmark DS and real-world time series, demonstrating superior zero-shot generalization on DSR problems compared to current TS foundation models. 2 Related work Dynamical systems reconstruction (DSR) In DSR we seek to learn generative models from time series data that represent the underlying system dynamics with all its topological, geometrical, and temporal properties [ 23 , 33 , 66 , 30 , 13 , 11 ] , in that sense providing an approximation to the underlying system’s governing equations. By definition, such a model should not only render viable short-term forecasts, but also reproduce the long-term evolution of the DS it has been trained on, both in state space and in the time domain. Methods approaching this goal have been founded on predefined function libraries, such as Sparse Identification of Nonlinear Dynamics (SINDy; [ 12 , 52 , 19 , 41 , 55 ] ), on reservoir computers [ 63 , 65 , 66 ] , neural ODEs [ 17 , 43 , 1 , 45 ] , Koopman operators [ 14 , 53 , 60 , 3 , 58 , 28 , 86 ] , or different types of RNNs [ 81 , 22 , 83 , 16 , 6 , 69 , 38 , 7 ] . More important than the architecture itself seems to be their proper training in order to ensure long-term (invariant) statistics of the underlying system are met: Methods like sparse [ 56 , 6 ] or generalized [ 38 ] teacher forcing allow RNN-generated trajectories to ‘explore the future’ whilst training, yet keep loss gradients in check. Alternatively, long-term statistics based on the observed system’s Lyapunov spectrum, fractal geometry, or invariant measures may be added to regularize the loss [ 65 , 66 , 40 , 72 ] , but require to compute these properties from the data first. Despite these advances, out-of-domain generalization remains a key challenge in DSR [ 33 ] . Meta- and hierarchical learning models, trained across many DS simultaneously, have recently been designed as steps toward foundation models for DSR [ 9 , 59 , 44 , 91 ] , but still require parameter fine-tuning and lack in-context inference capabilities. Time series foundation models Large language models (LLMs) exhibit an impressive ability to infer patterns from prompts (context) and generalize