Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting
Jinjin Chi, Lei Feng, Lulu Zhang, Yongcheng Jing, Yiming Wang, Ximing Li, Jialie Shen, Leszek Rutkowski, Dacheng Tao · 2026
Read Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting by Jinjin Chi, Lei Feng, Lulu Zhang, Yongcheng Jing, Yiming Wang, Ximing Li, Jialie Shen, Leszek Rutkowski, Dacheng Tao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-24911.txt.
Introduction
Jinjin Chi leads ROAR, a method sharpening time series forecasts by making smarter use of past cases pulled in as guides. Even strong retrieved cases can fail to help a base forecast. ROAR weights training toward queries where the base forecast struggles and retrieved cases offer real gains. It first blends aligned past cases with a gate setting how strongly they correct the base forecast. Then it tunes forecast and gate jointly while staying anchored to the first-stage output, winning the lowest mean squared error on seven benchmarks.
Catalog record
| CREATOR | Jinjin Chi, Lei Feng, Lulu Zhang, Yongcheng Jing, Yiming Wang, Ximing Li, Jialie Shen, Leszek Rutkowski, Dacheng Tao |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39143 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.24911
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Factorize to Generalize: Retrieval-Guided Invariant-Dynamic Decomposition for Time Series Forecasting Jinjin Chi, Lei Feng, Lulu Zhang, Yongcheng Jing, Yiming Wang, Ximing Li, Jialie Shen, Leszek Rutkowski, Dacheng Tao Abstract Time series foundation models (TSFMs) have recently achieved strong zero-shot forecasting performance through large-scale pretraining and retrieval-augmented prediction. However, our empirical analysis reveals a non-trivial limitation of retrieval-based forecasting: retrieval tends to induce more oscillatory predictions, improving performance on highly fluctuating series while degrading accuracy on smoother, trend-dominated ones. This suggests that retrieved information may be fused into prediction without explicitly distinguishing stable temporal structure from instance-specific variations, which can reduce robustness under distribution shifts. We propose a R etrieval-guided I nvariant- D ynamic DE composition framework ( RIDDE ) for time series forecasting. Rather than using retrieval as auxiliary predictive context, we leverage retrieved sequences as implicit samples from related environments to guide representation decomposition. Specifically, we first construct a retrieval-aware representation via attention-based aggregation, and then introduce a retrieval-guided routing mechanism to decompose it into an invariant component capturing stable shared structure and a dynamic component modeling context-dependent variations. These two components are forecast separately and fused for final prediction, enabling the model to preserve transferable patterns while remaining adaptive to evolving dynamics. We further design training objectives that encourage invariant learning and disentanglement, and provide theoretical insight showing that retrieval aggregation reduces variance and approximates invariant representation learning without explicit environment supervision. Extensive experiments demonstrate that our method consistently improves robustness under distribution shifts and outperforms existing TSFMs and retrieval-based baselines in zero-shot forecasting settings. 1 Introduction Time series forecasting aims to predict future trends and dynamics from historical observations. It plays a fundamental role in a wide range of real-world applications, including weather prediction ( 25 ; 23 ) , energy consumption modeling ( 29 ) , healthcare ( 8 ; 28 ) , and financial analysis ( 21 ; 22 ) . Recently, Time Series Foundation Models (TSFMs) have emerged as a powerful paradigm, demonstrating strong capabilities in modeling complex temporal dynamics and capturing long-range dependencies in sequential data ( 2 ; 26 ; 1 ; 7 ) . Despite these advances, existing TSFMs often struggle under distribution shifts, where test-time data deviates from training distributions ( 9 ; 38 ; 30 ) . A key challenge lies in the intrinsic structure of real-world time series, which are typically governed by a combination of stable underlying mechanisms and time-varying factors, especially under non-stationary and distribution-shifted settings ( 17 ; 15 ) . The former capture invariant structure that generalizes across environments, while the latter introduce context-dependent fluctuations driven by temporal, spatial, or external influences ( 3 ; 34 ; 10 ) . Effective generalization under distribution shifts therefore requires distinguishing between these two sources of variation. However, standard foundation models tend to learn entangled representations in which stable and dynamic structures are mixed together, reducing both robustness and interpretability under unseen conditions. To mitigate distribution shifts, Retrieval-Augmented Generation (RAG) ( 13 ; 36 ; 33 ) has recently been explored to enhance forecasting by incorporating external time series as additional context. While RAG has shown promising empirical improvements, a fundamental question remains: how does retrieval influence the temporal structure learned by the model? To answer this, we conduct a simple empirical analysis comparing forecasting behavior with and without RAG (see Figure 1 ; details in Appendix C ). We find that RAG tends to induce more oscillatory predictions: it improves performance on highly fluctuating series, while degrading accuracy on smoother, trend-dominated ones. A finer-grained analysis further shows that RAG reduces errors on high-frequency (seasonal) components but increases errors on low-frequency (trend) components. Taken together, these results indicate that retrieval does not affect all temporal structures uniformly. Instead, it biases the model toward fitting rapidly varying patterns, while weakening its ability to preserve stable temporal structure. These observations suggest that retrieval does not affect all temporal structures uniformly. Instead, it appears to bias the model toward rapidly varying patterns while weakening its ability to preserve stable temporal structure. We hypothesize that this behavior arises because retrieved sequences are fused with the input in a largely undifferentiated manner, causing stable shared structure and instance-specific variations to be mixed in the learned representation. Motivated by this limitation, we propose a R etrieval-guided I nvariant- D ynamic DE composition framework ( RIDDE ) for time series forecasting. Unlike existing retrieval-based forecasting methods that primarily use retrieval to directly enhance prediction, RIDDE uses retrieval as a structural signal for representation decomposition. Our key intuition is that retrieved sequences can be viewed as implicit samples from related environments: while they may share similar underlying mechanisms with the query sequence, their dynamic variations can differ. This makes retrieval aggregation a natural source of shared structural information. Based on this intuition, we first construct a retrieval-aware consensus representation via attention-based aggregation over retrieved samples. We then introduce a retrieval-guided routing mechanism that decomposes the fused representation into an invariant component and a dynamic component. The invariant branch is encouraged to preserve stable patterns supported across retrieved samples, while the dynamic branch is encouraged to capture query-specific deviations. These components are forecast separately and fused for final prediction, enabling the model to better balance robustness and adaptability under distribution shifts. Our contributions are summarized as follows: • We identify a previously underexplored limitation of retrieval-augmented forecasting, showing that retrieval can introduce a structural bias toward high-frequency fluctuations while weakening stable trend modeling. • We propose RIDDE, a retrieval-guided invariant–dynamic decomposition framework that uses retrieval as a structural signal for representation decomposition rather than direct predictive enhancement. • We develop a routing-based mechanism that decomposes retrieval-aware representations into invariant and dynamic components, together with a regularizer that encourages branch separation. • We conduct extensive experiments across diverse benchmarks and settings, showing that RIDDE consistently improves robustness and zero-shot forecasting performance under distribution shifts. Figure 1 : Forecasted series of training w/o RAG (Base) and w/ RAG against the ground truth on (a) smoother series, (b) more fluctuating series, (c) seasonal series components, (d) trend series components. Training with RAG tends to generate more oscillatory predictions, suggesting that RAG introduces biases towards modeling high-frequency variations. 2 Related Work Time Series Foundation Models (TSFMs). Time Series Foundation Models (TSFMs) have emerged as a scalable paradigm for time series forecasting, aiming to learn transferable temporal patterns from large and heterogeneous corpora ( 14 ) . A representative line of work adapts the modeling principles of large language models to temporal data. For instance, Chronos ( 1 ) tokenizes numerical values into discrete tokens and trains transformer-based models with language-modeling-style objectives, demonstrating strong zero-shot forecasting performance. In contrast, TimesFM ( 2 ) represents time series as sequences of continuous patches and leverages large-scale pretraining to improve generalization across unseen domains. Another line of research focuses on handling the heterogeneity of real-world temporal data. Moirai ( 31 ) proposes a unified modeling framework for variations in frequency, dimensionality, and marginal distributions, while its mixture-of-experts variants ( 16 ) further introduce adaptive routing mechanisms to capture diverse temporal patterns across domains. More recent efforts extend TSFMs beyond forecasting-specific architectures toward more general-purpose pretrained temporal models. For example, MOMENT ( 5 ) emphasizes