Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems

Jingwen Cheng, Ruikun Li, Huandong Wang, Yong Li · 2025

Read Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems by Jingwen Cheng, Ruikun Li, Huandong Wang, Yong Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17459.txt.

Introduction

Jingwen Cheng, Ruikun Li, Huandong Wang, and Yong Li wrote a 2025 paper on forecasting how complex systems change over time. Such systems, from weather to fluids, follow hidden rules that are hard to learn from raw data. Most models squeeze data into a compact code, then predict step by step from that code.

Their method, SparseDiff, fixes a flaw in that plan: flat codes lose where things sit in space. It updates its coding scheme at test time, while forecasts are running. This lets the model catch new patterns that emerge late in a run. The design aims for sharper long-range forecasts. Better forecasts help in fields like climate study and engineering design.

Catalog record

CREATORJingwen Cheng, Ruikun Li, Huandong Wang, Yong Li
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17459

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sparse Diffusion Autoencoder for Test-time Adapting Prediction of Complex Systems Jingwen Cheng, Ruikun Li, Huandong Wang, Yong Li Abstract Predicting the behavior of complex systems is critical in many scientific and engineering domains, and hinges on the model’s ability to capture their underlying dynamics. Existing methods encode the intrinsic dynamics of high-dimensional observations through latent representations and predict autoregressively. However, these latent representations lose the inherent spatial structure of spatiotemporal dynamics, leading to the predictor’s inability to effectively model spatial interactions and neglect emerging dynamics during long-term prediction. In this work, we propose SparseDiff, introducing a test-time adaptation strategy to dynamically update the encoding scheme to accommodate emergent spatiotemporal structures during the long-term evolution of the system. Specifically, we first design a codebook-based sparse encoder, which coarsens the continuous spatial domain into a sparse graph topology. Then, we employ a graph neural ordinary differential equation to model the dynamics and guide a diffusion decoder for reconstruction. SparseDiff autoregressively predicts the spatiotemporal evolution and adjust the sparse topological structure to adapt to emergent spatiotemporal patterns by adaptive re-encoding. Extensive evaluations on representative systems demonstrate that SparseDiff achieves an average prediction error reduction of 49.99% compared to baselines, requiring only 1% of the spatial resolution. 1 Introduction Figure 1: Illustration of adapting probe topologies. The dynamics of complex systems are driven by the nonlinear interactions and co-evolution of numerous components, giving rise to rich emergent spatiotemporal structures, as seen in fluid dynamics [ 1 ] , climate science [ 2 ] , and molecular dynamics [ 3 ] . Accurate long-term prediction of these systems is crucial for many real-world applications [ 4 , 5 ] . However, complex systems are typically observed in high-dimensional spaces with unknown intrinsic dynamics, making high-fidelity, long-term prediction a challenging and computationally expensive problem [ 6 , 7 , 8 ] . Data-driven methods offer an equation-free and computationally friendly approach. The core idea is to build reduced-order models of high-dimensional observations using encoder-decoders, which project observed states to a low-dimensional latent space for prediction, thereby improving computational efficiency [ 6 , 9 , 10 , 11 , 12 ] . However, most existing work employs parameterized neural network encoders that implicitly encode spatial structure by compressing observational spatial information into a latent representation vector. This conversion of spatially-correlated data into latent vectors hinders predictors from effectively understanding and modeling the system’s inherent spatial interactions and structural relationships [ 5 ] . For instance, in fluid convection, heat drives fluid to form rising plumes and sinking cold flows. These specific spatial structures, their relative positions, and interactions are critical for understanding and predicting convection patterns [ 13 ] . Considering that spatial interaction patterns and emergent structures continuously change during long-term evolution, losing such information progressively amplifies the misalignment between the encoder-decoder and predictor in long-term predictions. This leads to a core question: Can we construct a reduced-order model that preserves crucial spatial structure and continuously adapts to newly emerging dynamics patterns during prediction? This work is inspired by a recent finding that the system state in the entire spatial domain can be effectively reconstructed from sparse observation points [ 14 , 15 , 16 ] . The data disparity between sparse points and the full spatial grid strongly suggests a promising encoding paradigm: aggregating the full-space dynamic information onto a much smaller set of sparse probes [ 17 ] . These probes and their spatial topology constitute the dynamical skeleton of the spatiotemporal dynamics, as shown in Figure 1 . In this paradigm, the model is capable of adapting to the system’s latest spatiotemporal patterns during long-term predictions by dynamically adjusting probe positions and topology based on the re-encoded predicted states [ 6 , 18 ] . Nevertheless, achieving effective probe-based aggregation and adaptive updating in practice faces two key challenges: 1) A lack of theoretical understanding for aggregating rich spatiotemporal dynamics from a continuous domain into a small set of discrete probes; 2) Frequent re-encoding during long-term prediction introduces significant computational overhead and potential accumulation of prediction errors. To address these challenges, we propose a novel Sparse Diffusion Autoencoder, SparseDiff, consisting of a codebook-based sparse encoder and an unconditional diffusion decoder. The sparse encoder learns a representative pattern codebook from historical spatiotemporal trajectories, enabling it to dynamically aggregate and project full-space observational data onto a spatial topology formed by sparse probes, thus constructing a reduced-order model. On this probe topology, we model spatiotemporal dynamics using a graph neural ordinary differential equation and explicitly introduce a diffusion term to capture spatial interactions and information propagation among probes. Finally, we pad the probe predictions to the full spatial domain to serve as the initial state for the diffusion process, enabling rapid reconstruction of the full spatiotemporal field. SparseDiff’s key innovation and strength lie in its test-time adaptation: during long-term prediction, it utilizes the learned codebook to re-encode the latest prediction, dynamically adjusting and constructing a probe topology better matched to the current system state to continuously adapt to emerging spatiotemporal patterns. Experimental evaluation on simulated and real-world systems demonstrates that SparseDiff outperforms baselines by over 49.99% in long-term predictions, and reliably predicts full-space spatiotemporal dynamics using less than 1% of grid points as probes. The highlights of this work are summarized as follows: • We propose SparseDiff, a novel autoencoder that uses sparse probes to construct a reduced-order model that effectively preserves the spatial structure of spatiotemporal dynamics. • We introduce a test-time adaptation strategy via re-encoding, allowing the model to dynamically adapt its probe-based representation to emerging dynamics patterns, significantly improving long-term prediction accuracy and computational efficiency. • Experiments demonstrate that SparseDiff significantly outperforms baselines in long-term prediction accuracy and achieves high efficiency by reliably predicting full-space dynamics using only approximately 1% of grid points as probes 1 1 1 Code will be available after review. . 2 Preliminary 2.1 Problem Definition In this work, we focus on spatiotemporal dynamics \frac{du}{dt}=f(u,x,t,\frac{\partial u}{\partial x},\frac{\partial^{2}u}{\partial x^{2}},...) on a 2D regular spatial domain x\subset\mathbb{R}^{2} . Such systems can often be formalized as time-dependent partial differential equations, for example, the Navier-Stokes equations, which include a Laplacian operator term acting on the spatial domain [ 19 ] . Without loss of generality, we address the problem of data-driven modeling of system dynamics from historical evolving trajectories U=\{u(x,t)\}^{T} , thereby extending our focus to real-world systems like climate dynamics where explicit equations are unknown. In practice, the spatial domain is discretized into an h\times w grid. Consequently, the full-domain observation state at each timestamp is represented as a tensor u\in\mathbb{R}^{h\times w} . Given a historical observation trajectory of lookback steps, we predict multiple future steps in an autoregressive manner, forecasting one step at a time. 2.2 Guided Diffusion for Sparse Reconstruction Diffusion methods have recently been shown to reliably reconstruct the full spatial domain’s system state, guided by sparse observations [ 16 , 15 , 14 ] . Diffusion models aim to learn a probabilistic mapping from a simple prior distribution, such as a standard Gaussian to a complex target data distribution [ 20 , 21 , 22 ] . This is achieved by defining a forward diffusion process that gradually adds noise to data instances and training a reverse process

Read the complete text