In-context learning to predict critical transitions in dynamical systems
Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer, Carla Roesch, Tobias Weber, Vaios Laschos, Hang Fan, Gregor Ramien, Johannes Haux, Pierre Gentine, Benjamin Herdeanu · 2026
Read In-context learning to predict critical transitions in dynamical systems by Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer, Carla Roesch, Tobias Weber, Vaios Laschos, Hang Fan, Gregor Ramien, Johannes Haux, Pierre Gentine, Benjamin Herdeanu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12308.txt.
Introduction
Sevinchan and ten colleagues build an early-warning system for abrupt shifts in dynamical systems. Standard warning signs fail with scarce data and correlated noise. Deep classifiers rarely extrapolate beyond their training mix. Their TipPFN framework learns in context from a synthetic generator built on classic bifurcation patterns. It can judge closeness to a tipping point from contexts of many sizes, and detects unseen tipping regimes and real cases in both in-context and zero-shot modes.
Catalog record
| CREATOR | Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer, Carla Roesch, Tobias Weber, Vaios Laschos, Hang Fan, Gregor Ramien, Johannes Haux, Pierre Gentine, Benjamin Herdeanu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65794 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12308
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
In-context learning to predict critical transitions in dynamical systems Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer, Carla Roesch, Tobias Weber, Vaios Laschos, Hang Fan, Gregor Ramien, Johannes Haux, Pierre Gentine, Benjamin Herdeanu Abstract Critical transitions – abrupt, often irreversible changes in system dynamics – arise across human and natural systems, often with catastrophic consequences. Real-world observations of such shifts remain scarce, preventing the development of reliable early warning systems. Conventional statistical and spectral indicators, such as increasing variance, tend to fail under realistic conditions of limited data and correlated noise, whereas existing deep learning classifiers do not extrapolate beyond their training data distribution. In this work, we introduce TipPFN, an in-context learning (ICL) framework that uses a prior-data fitted network to infer a system’s proximity to a critical transition. Trained on our novel synthetic data generator, which is based on canonical bifurcation scenarios coupled to diverse, randomized stochastic dynamics, TipPFN flexibly capitalizes on contexts of various sizes, complexity and dimensionalities. We demonstrate robust, state-of-the-art early detection of critical transitions in previously unseen tipping regimes, sim-to-real examples, and real-world observations in both ICL and zero-shot settings. 1 Introduction Figure 1 : TipPFN detects critical transitions earlier and more accurately than classical early warning signals (EWS), state-of-the-art deep learning and ICL-based methods across 14 semi-real, sim-to-real, and real-world systems spanning climate, engineering, and biology. Lines show across-system mean and standard deviation. Tipping points occur when variation in forcing or system parameters triggers an abrupt, potentially irreversible transition to a qualitatively different dynamical regime [ 16 , 2 ] . Such transitions are central to climate [ 54 ] , ecological [ 62 ] , infrastructural [ 10 ] , and collective biological and social systems [ 18 , 52 , 57 ] , where local shifts can cascade through interacting components with far-reaching consequences. Classical theory has largely focused on bifurcation (b-tipping) classes of tipping mechanism. Here, the loss of stability of an attracting state produces critical slowing down (CSD) [ 13 ] , where recovery from perturbations becomes progressively weaker [ 65 , 61 ] . This has motivated widely used early warning signals (EWS), including increasing lag-1 autocorrelation (AR1) [ 12 ] , variance [ 9 ] , and skewness [ 20 ] . However, these indicators rely on restrictive assumptions, including near-equilibrium linearization, stationarity, sufficiently long records, and simple bifurcation structure [ 11 , 29 ] . They may therefore fail for rate-induced (r-tipping) and noise-induced (n-tipping), where transitions can occur without pronounced CSD [ 52 , 48 , 27 ] . From a machine learning (ML) perspective, characterizing critical transitions can be viewed as a form of out-of-distribution (OOD) task, where the system leaves one regime and enters into a dynamically distinct region [ 34 , 67 , 66 ] . This makes tipping prediction fundamentally different from standard regression tasks, since models must infer the onset and type of critical transitions from sparse, noisy data and partially observed trajectories, often without examples of the event itself. Recent ML approaches have shown promise by learning features of canonical tipping systems directly from data [ 8 , 33 , 52 , 68 , 46 ] . However, many of these methods are trained on particular classes, limiting their ability to generalize across observations and different critical mechanisms. A promising alternative has recently emerged in the form of prior-data fitted networks (PFNs), which are trained on large ensembles of synthetic datasets and learn to perform inference via in-context learning (ICL) [ 40 ] . Rather than fitting a model to a single dataset, PFNs are trained across millions of simulated tasks, enabling them to infer structure and make predictions directly from limited observations. Thus, PFNs have already transformed tabular ML through models such as TabPFN [ 23 , 24 , 50 ] , which, despite being trained exclusively on synthetic data, achieve state-of-the-art performance on real-world benchmarks. More broadly, PFNs have demonstrated remarkable generalisation across diverse tasks, including time series forecasting [ 25 ] , causal inference [ 53 , 3 ] , and robotic controls [ 56 ] . Our key contributions are (see Figure 2 ): • Model: TipPFN, a transformer-based architecture for joint prediction of proximity to a critical transition from short, noisy time series. • Training and Benchmark: TipBox, our benchmarking suite and training data generator of stochastic dynamical systems across diverse tipping regimes and complexities. • Generalization: Robust performance across unseen tipping regimes, sim-to-real cases, and real-world systems in both ICL and zero-shot settings (see Fig. 1 ). Figure 2 : Overview. TipPFN is trained on synthetic data, based on embedding canonical b-tipping systems into high-dimensional, randomized stochastic dynamics. Primary training target is the relative distance to criticality (RDTC), a measure of how close a system is to a critical transition. Although the synthetic dynamics were only driven by b-tipping systems, TipPFN successfully generalizes to other classes of tipping systems, and a manifold of real-world systems. 2 Background Critical transition in dynamical systems Critical transition, specifically tipping points in dynamical systems, can arise through multiple distinct mechanisms, broadly classified as bifurcation-induced (b-tipping), noise-induced (n-tipping), and rate-induced tipping (r-tipping) [ 2 ] . Consider a general nonautonomous stochastic differential equation (SDE) of the form: \frac{d\mathbf{x}}{dt}=f(\mathbf{x},\lambda(t))+\sigma\,\xi(t), (1) where \mathbf{x}\in\mathbb{R}^{d} denotes the system state, \lambda(t) is a time-dependent forcing parameter, \sigma>0 controls the noise amplitude, and \xi(t) represents Gaussian white noise. Bifurcation-induced tipping In b-tipping, a transition occurs when a quasi-static equilibrium \mathbf{x}^{\ast}(\lambda) loses stability at a critical parameter value \lambda_{\mathrm{crit}} . Linearizing the dynamics around \mathbf{x}^{\ast}(\lambda) yields the Jacobian J(\lambda)=\partial f/\partial\mathbf{x}\big|_{\mathbf{x}^{\ast}(\lambda)} , whose eigenvalues \{\mu_{i}(\lambda)\} govern local stability. A bifurcation occurs when the real part of the leading eigenvalue crosses zero, i.e., \max_{i}\,\mathrm{Re}(\mu_{i}(\lambda_{\mathrm{crit}}))=0. (2) Rate-induced tipping In r-tipping, a transition occurs when the rate of change of the forcing, given by \dot{\lambda} , is sufficiently large such that the system fails to track the moving equilibrium \mathbf{x}^{\ast}(\lambda(t)) , leading to a transition without any local loss of stability, i.e., the linearized system satisfy \max_{i}\mathrm{Re}(\mu_{i}(\lambda))<0 . Thus, these tipping mechanisms cannot, in general, be inferred solely from equilibrium stability analysis commonly used to check b-tipping (see Appendix A for more details). Noise-induced tipping In n-tipping, a transition occurs primarily through stochastic perturbations \sigma\,\xi(t) that induce escape from the basin of attraction of a stable equilibrium. This process can occur in conjunction with b-tipping and r-tipping in SDEs, a setup which form the basis for this work. Prior-data fitted networks PFNs are transformer-based neural networks trained on synthetically generated data drawn from a predefined prior, enabling approximate Bayesian inference via ICL [ 40 ] . In this framework, inference is amortized at the dataset level. The model learns a mapping from a dataset D to a posterior predictive distribution p(y|x,D) by training on large collections of simulated tasks (D_{i},x_{i},y_{i}) . Rather than fitting a model to each new dataset, PFNs directly infer predictions conditioned on the observed data, performing Bayesian inference in a single forward pass. Related work In addition to the classical EWS based on statistics (AR1, variance, skewness), there is an emergence of dynamics-based approaches extracting additional spectral structure of the systems, including tracking of dominant eigenvalues [ 38 , 19 ] . Recently, fully data-driven ML approaches,