Learning from History: A Retrieval-Augmented Framework for Spatiotemporal Prediction
Hao Jia 1 , Penghao Zhao 2 , Hao Wu 3 , Yuan Gao 3 , Yangyu Tao 2 , Bin Cui 1 , 1 Peking University, Beijing, China 2 Tencent Inc., China 3 Tsinghua University, Beijing, China [email protected], [email protected], [email protected], [email protected], brucetao@tencen · 2025
Read Learning from History: A Retrieval-Augmented Framework for Spatiotemporal Prediction by Hao Jia 1 , Penghao Zhao 2 , Hao Wu 3 , Yuan Gao 3 , Yangyu Tao 2 , Bin Cui 1 , 1 Peking University, Beijing, China 2 Tencent Inc., China 3 Tsinghua University, Beijing, China [email protected], [email protected], [email protected], [email protected], brucetao@tencen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-24049.txt.
Introduction
Hao Jia and five co-authors present a method for predicting how complex physical systems change over space and time. Deep learning models can do this task well at first, but errors build up over long runs. The result can look wrong for the real system and break basic physical rules.
Their method, called RAP, pairs a neural network with real past cases from history. For each new state, it looks up past patterns that show how the system moved before. That history acts as a check on the network's guess. The authors report more stable long term forecasts and fewer false patterns than models that rely on learned rules alone.
Catalog record
| CREATOR | Hao Jia 1 , Penghao Zhao 2 , Hao Wu 3 , Yuan Gao 3 , Yangyu Tao 2 , Bin Cui 1 , 1 Peking University, Beijing, China 2 Tencent Inc., China 3 Tsinghua University, Beijing, China [email protected], [email protected], [email protected], [email protected], brucetao@tencen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-24049
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Learning from History: A Retrieval-Augmented Framework for Spatiotemporal Prediction Hao Jia 1 , Penghao Zhao 2 , Hao Wu 3 , Yuan Gao 3 , Yangyu Tao 2 , Bin Cui 1 , 1 Peking University, Beijing, China 2 Tencent Inc., China 3 Tsinghua University, Beijing, China [email protected], [email protected], [email protected], [email protected], [email protected], [email protected] Abstract Accurate and long-term spatiotemporal prediction for complex physical systems remains a fundamental challenge in scientific computing. While deep learning models, as powerful parametric approximators, have shown remarkable success, they suffer from a critical limitation: the accumulation of errors during long-term autoregressive rollouts often leads to physically implausible artifacts. This deficiency arises from their purely parametric nature, which struggles to capture the full constraints of a system’s intrinsic dynamics. To address this, we introduce a novel Retrieval-Augmented Prediction (RAP) framework, a hybrid paradigm that synergizes the predictive power of deep networks with the grounded truth of historical data. The core philosophy of RAP is to leverage historical evolutionary exemplars as a non-parametric estimate of the system’s local dynamics. For any given state, RAP efficiently retrieves the most similar historical analog from a large-scale database. The true future evolution of this analog then serves as a reference target . Critically, this target is not a hard constraint in the loss function but rather a powerful conditional input to a specialized dual-stream architecture. It provides strong dynamic guidance , steering the model’s predictions towards physically viable trajectories. In extensive benchmarks across meteorology, turbulence, and fire simulation, RAP not only surpasses state-of-the-art methods but also significantly outperforms a strong analog-only forecasting baseline . More importantly, RAP generates predictions that are more physically realistic by effectively suppressing error divergence in long-term rollouts. Our work demonstrates that integrating non-parametric historical experience with parametric deep networks provides an effective pathway toward building the next generation of high-fidelity surrogate models for complex physical systems. 1 1 1 https://github.com/RAP-ANAO/Retrieval-Augmented-Prediction I Introduction Spatiotemporal sequence prediction is a cornerstone of many critical scientific and engineering domains, aiming to deduce the future states of complex systems from historical observations shi2015convolutional ; wang2022predrnn ; wu2024earthfarsser . In weather forecasting, accurate prediction underpins disaster mitigation and socio-economic activities wu2025triton ; bi2023accurate ; gao2025oneforecast . In computational fluid dynamics, it is crucial for revealing the intrinsic mechanisms of complex phenomena like turbulence and vortices wu2025turb ; wu2024neural ; wu2024pure . In fire simulation feiopen ; wu2024prometheus , it provides vital decision support for evacuation strategies and emergency response. However, these physical systems commonly exhibit highly nonlinear, multi-scale, and chaotic characteristics, which make long-term, high-fidelity prediction an exceptionally formidable scientific challenge. In recent years, data-driven methods, particularly those based on deep learning models leveraging convolutions (CNNs) gao2022simvp ; tan2022simvp ; wu2023pastnet , recurrence (RNNs) wang2022predrnn , and attention mechanisms (Transformers) wu2025triton ; gao2022earthformer ; wu2023earthfarseer , have achieved remarkable success. These methods automatically learn complex dependencies from massive datasets, demonstrating powerful fitting capabilities. However, these models are fundamentally purely parametric function approximators that learn physical laws implicitly. This lack of an explicit understanding of the system’s intrinsic dynamics becomes a critical bottleneck, causing their performance to degrade sharply when facing rare events or performing long-term autoregressive rollouts lam2023learning ; wu2025triton . Consequently, models tend to generate physically unrealistic artifacts and suffer from the rapid accumulation of errors, ultimately causing predictions to diverge and lose their value wu2025triton . In stark contrast to the implicit learning of machines, human experts, such as meteorologists, often learn from history . They search vast historical records for cases with high similarity to the current state known as historical analogs and use their true subsequent evolutions as a crucial reference. We argue that this classic idea contains a profound insight: every historical analog is an evolutionary exemplar validated by a real physical process, providing powerful non-parametric information. At a deeper level, since the evolutionary trajectory of any physical system is constrained to a low-dimensional attractor manifold, the path of an analog represents a real sample of the local dynamics on this manifold. This offers a powerful form of dynamic guidance, rather than a hard constraint, for the predictive model. This distinction raises a critical question: can a neural network learn to effectively fuse this guidance with the current state, or does it merely learn to copy the analog? Based on this philosophy, we propose a novel Retrieval-Augmented Prediction (RAP) framework. The RAP framework decomposes the prediction paradigm into three synergistic stages. First, in the Retrieve stage, for any given query, RAP employs an efficient similarity metric (e.g., Mean Squared Error) to retrieve the most similar historical analogs from a large-scale database wang2025beamvq . Second, in the Augment stage, the framework extracts the corresponding true subsequent evolutions, using them as a reference target. Finally, in the Predict stage, a specialized dual-stream network deeply fuses features from the current input and the reference target, generating a precise prediction informed by both the current state and historical precedent. To comprehensively validate our framework, we conduct systematic evaluations on challenging benchmarks spanning ❶. Global Weather Forecasting, ❷. 2D Turbulence Modeling, and ❸. Fire-spread Simulation. Our experiments are designed not only to compare RAP against state-of-the-art models but also against a strong ‘analog-only‘ baseline, where the prediction is simply the retrieved reference target. The results demonstrate that the RAP framework significantly outperforms both classes of methods. More importantly, our qualitative analysis reveals that RAP generates predictions with more physically realistic details and effectively suppresses error divergence, providing strong evidence that incorporating historical experience as an explicit form of guidance is a robust pathway to improving prediction fidelity. In summary, the main contributions of this paper are threefold: • We propose the RAP framework, a novel, general, and effective retrieval-augmented paradigm that organically integrates the classic idea of historical analogs with modern deep learning models. • We articulate and validate the core idea of using historical evolutionary exemplars, retrieved from data, as a form of non-parametric dynamic guidance to steer deep models toward more physically plausible predictions. • We conduct extensive experiments demonstrating the significant advantages of our method, showing its superiority over both purely parametric models and strong non-learning, analog-only baselines, thus offering a new direction for building the next generation of high-fidelity surrogate models. II Related Work Deep Learning for Spatiotemporal Prediction Deep learning has become the dominant paradigm for spatiotemporal prediction shi2015convolutional ; yu2018spatio ; wu2024earthfarsser , driving significant progress in diverse fields, from video prediction to climate science wu2023pastnet . Early works often employed architectures based on Convolutional Neural Networks (CNNs) he2016deep , such as U-Net ronneberger2015u , which excel at capturing local spatial features but are limited in their ability to explicitly model temporal dependencies. To better handle time-series data, researchers integrated Recurrent Neural Networks (RNNs) with CNNs, leading to models like ConvLSTM shi2015convolutional and PredRNN wang2022predrnn . These models leverage recurrent state units to propagate temporal information, improving the modeling of long-range dependencies. However, their sequential nature restricts parallel computation and they still face challenges like vanishing gradients in very long sequences. More recently, inspired by the success of Transformers in natural language processing and computer vision, a series of attention-based models, including FourCastNet kurth2023fourcastnet , Pangu-Weather bi2023accurate , and Triton wu2025triton , have achieved breakthrough performance