ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations
Chang Liu, Bohao Zhao, Jingtao Ding & Yong Li · 2025
Read ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations by Chang Liu, Bohao Zhao, Jingtao Ding & Yong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-21802.txt.
Introduction
Forecasting chaotic systems like the weather is hard because small changes grow fast, and their patterns play out at many time scales at once. Chang Liu, Bohao Zhao, Jingtao Ding, and Yong Li present ChaosNexus in this 2025 paper. It is a foundation model, pre-trained once and then used on new systems. Earlier models of this kind often miss the mix of fast and slow structure.
ChaosNexus reads time in patches of several sizes, stacked in layers. That lets it track slow trends and quick wiggles together. Expert modules switch on for different kinds of systems, guided by a fingerprint of the system's frequencies. Tested on thousands of synthetic systems, it matched long-term behavior well. It also forecast real weather five days ahead with mean error under one degree.
Catalog record
| CREATOR | Chang Liu, Bohao Zhao, Jingtao Ding & Yong Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-21802
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ChaosNexus: A Foundation Model for Universal Chaotic System Forecasting with Multi-scale Representations Chang Liu, Bohao Zhao, Jingtao Ding & Yong Li Abstract Accurately forecasting chaotic systems, prevalent in domains such as weather prediction and fluid dynamics, remains a significant scientific challenge. The inherent sensitivity of these systems to initial conditions, coupled with a scarcity of observational data, severely constrains traditional modeling approaches. Since these models are typically trained for a specific system, they lack the generalization capacity necessary for real-world applications, which demand robust zero-shot or few-shot forecasting on novel or data-limited scenarios. To overcome this generalization barrier, we propose ChaosNexus, a foundation model pre-trained on a diverse corpus of chaotic dynamics. ChaosNexus employs a novel multi-scale architecture named ScaleFormer augmented with Mixture-of-Experts layers, to capture both universal patterns and system-specific behaviors. The model demonstrates state-of-the-art zero-shot generalization across both synthetic and real-world benchmarks. On a large-scale testbed comprising over 9,000 synthetic chaotic systems, it improves the fidelity of long-term attractor statistics by more than 40% compared to the leading baseline. This robust performance extends to real-world applications with exceptional data efficiency. For instance, in 5-day global weather forecasting, ChaosNexus achieves a competitive zero-shot mean error below 1°C—a result that further improves with few-shot fine-tuning. Moreover, experiments on the scaling behavior of ChaosNexus provide a guiding principle for scientific foundation models: cross-system generalization stems from the diversity of training systems, rather than sheer data volume. 1 Introduction Chaotic systems, characterized by their deterministic nature yet high sensitivity to initial conditions, are ubiquitous in the natural world and across diverse scientific and engineering disciplines, including weather forecasting ( Shukla 1998 ; Rind 1999 ) , fluid dynamics ( Yorke & Yorke 2005 ; Najm 2009 ) , and neural processes ( Jia et al. 2023 ; Vignesh et al. 2025 ) . The intrinsic complexity of such systems renders accurate forecasting both an essential and formidable task, particularly in real-world contexts where data acquisition is resource-intensive and observational records are sparse. While this sensitivity makes precise long-term point-wise prediction impossible, the system’s behavior is not entirely random; it is confined to a complex geometric structure known as a strange attractor ( Rössler 1976 ; Grassberger & Procaccia 1983 ) , which possesses unique and invariant statistical properties. An effective forecasting model should not only predict the short-term evolution but also reproduce the long-term geometry and statistics of the system’s attractor. The intrinsic difficulty of forecasting chaotic systems is further compounded by the challenge of data sparsity. Traditional system-specific models ( Srinivasan et al. 2022 ; Brenner et al. 2022 ; Hess et al. 2023 ) typically require extensive and high-quality observational data from a novel system to accurately infer its underlying dynamics and attractor geometry, creating a significant bottleneck in practical applications. Here, we propose a paradigm shift from system-specific modeling to the pretraining of a single foundation model for universal chaotic system forecasting. This approach is motivated by the proposition that a model exposed to a vast and heterogeneous collection of observational data spanning diverse dynamical systems and operating regimes can learn a rich repertoire of underlying patterns and principles common to chaotic behavior ( Liu et al. 2024c ; Woo et al. 2024 ; Shi et al. 2024 ; Ansari et al. 2024 ) . By leveraging the large-scale data during pretraining, the model can then be applied to a target system with little or no in-distribution data. This strategy is designed to exploit cross-system similarities to compensate for downstream data sparsity, thereby reducing the burden of data acquisition and enhancing the out-of-distribution forecasting performance. However, realizing such a foundation model for chaotic system forecasting is non-trivial, as training a single parameterization on a heterogeneous ensemble of chaotic systems introduces formidable challenges that preclude the straightforward application of standard time-series methods. First, the inherent multi-scale nature of chaotic systems poses a fundamental representation challenge. These systems exhibit broadband spectra where essential dynamical structures unfold across a continuum of time scales. A mono-scale representation fails to capture the full picture, either truncating long-range dependencies or aliasing distinct behaviors across scales, thereby obscuring the unique attractor geometries of each system. Second, beyond their differences, the attractors of these systems may share underlying geometric and statistical properties that enable generalization. Credible long-horizon forecasting demands that the model’s architecture is explicitly designed to capture these transferable principles, effectively learning a shared parametric structure for common behaviors while identifying system-specific regimes to ensure stable and accurate extended rollouts. To overcome these obstacles, we introduce ChaosNexus, a foundation model engineered for universal chaotic dynamics forecasting. At its core is our proposed ScaleFormer, a U-Net-inspired Transformer architecture designed to master the multi-scale nature of chaotic systems. Its encoder progressively models fine-grained to coarse temporal contexts through hierarchical patch merging, while the symmetric decoder, aided by skip connections, reconstructs fine-grained details via patch expansion. To facilitate robust cross-system generalization, each Transformer block is augmented with a Mixture-of-Experts (MoE) layer, enabling the model to disentangle diverse dynamics by allocating specialized parameters for distinct system regimes. Furthermore, we condition the model with a frequency fingerprint derived from a wavelet scattering transform, providing a stable spectral signature that captures the system’s intrinsic oscillatory and modulatory behaviors. ChaosNexus is pretrained on a vast and diverse corpus of approximately 20K simulated chaotic systems ( Lai et al. 2025 ) . The training is guided by a composite objective function designed to ensure both predictive accuracy and the preservation of long-term statistical properties. Through extensive experiments, we demonstrate that ChaosNexus establishes a new state-of-the-art in zero-shot forecasting, improving the fidelity of long-term attractor statistics by 40.55%. Its remarkable sample efficiency is further highlighted in real-world weather forecasting, achieving zero-shot MAE below 1°C on temperature, surpassing strong baselines even when they are fine-tuned on over 470K samples from the target system. Moreover, our scaling analysis reveals a key insight for future work: generalization is driven more by the diversity of systems in the pretraining corpus than by the sheer volume of trajectories per system. Our primary contributions are summarized as follows: • We explore a new paradigm for chaotic system forecasting: pre-training a single foundation model on diverse chaotic systems to overcome data sparsity in downstream forecasting tasks. • We propose ChaosNexus, a unified framework that effectively captures diverse chaotic dynamics and disentangles system-specific behaviors by integrating a novel multi-scale ScaleFormer backbone with two key augmentations: Mixture-of-Experts (MoE) layers for adaptive specialization and a wavelet-based frequency fingerprint to provide a distinct spectral signature. • We conduct extensive experiments to provide empirical evidence of the generalization capabilities of ChaosNexus, confirming that large-scale pre-training is a highly effective strategy for building powerful, data-efficient models in complex scientific domains. 2 Related Works Chaotic System Forecasting. Forecasting chaotic systems is a central challenge in science and engineering. Reservoir computing (RC)-based methods ( Srinivasan et al. 2022 ; Gauthier et al. 2021 ; Li et al. 2024 ) represent a key advance: they employ a fixed, randomly initialized reservoir to lift inputs into high-dimensional state spaces while training only a linear readout. Concurrently, deep learning models like recurrent neural networks (RNNs) have proven effective, though they often require techniques such as teacher forcing to counteract training instabilities like exploding