TimeRecipe : A Time-Series Forecasting Recipe via Benchmarking Module Level Effectiveness

Zhiyuan Zhao, Juntong Ni, Shangqing Xu, Haoxin Liu, Wei Jin, B. Aditya Prakash · 2025

Read TimeRecipe : A Time-Series Forecasting Recipe via Benchmarking Module Level Effectiveness by Zhiyuan Zhao, Juntong Ni, Shangqing Xu, Haoxin Liu, Wei Jin, B. Aditya Prakash (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06482.txt.

Introduction

Zhiyuan Zhao and co-authors present, in twenty twenty-five, a benchmark for time-series forecasting at the level of parts, not whole models. Forecasting future values from past data is key in many fields. Deep learning models now do it well, but debate rages over which design parts help most. Past benchmarks compare full models, which hides why a design wins. Their framework, TimeRecipe, runs more than ten thousand tests. Each test varies one part, such as how a series is split or scaled, across many data sets and forecast spans.

Full search of the design space yielded models that beat the best known methods, they report. The work also gives a toolkit that suggests a model shape for a new task.

Catalog record

CREATORZhiyuan Zhao, Juntong Ni, Shangqing Xu, Haoxin Liu, Wei Jin, B. Aditya Prakash
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE129 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06482

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TimeRecipe : A Time-Series Forecasting Recipe via Benchmarking Module Level Effectiveness Zhiyuan Zhao, Juntong Ni, Shangqing Xu, Haoxin Liu, Wei Jin, B. Aditya Prakash Abstract Time-series forecasting is an essential task with wide real-world applications across domains. While recent advances in deep learning have enabled time-series forecasting models with accurate predictions, there remains considerable debate over which architectures and design components, such as series decomposition or normalization, are most effective under varying conditions. Existing benchmarks primarily evaluate models at a high level, offering limited insight into why certain designs work better. To mitigate this gap, we propose TimeRecipe , a unified benchmarking framework that systematically evaluates time-series forecasting methods at the module level. TimeRecipe conducts over 10,000 experiments to assess the effectiveness of individual components across a diverse range of datasets, forecasting horizons, and task settings. Our results reveal that exhaustive exploration of the design space can yield models that outperform existing state-of-the-art methods and uncover meaningful intuitions linking specific design choices to forecasting scenarios. Furthermore, we release a practical toolkit within TimeRecipe that recommends suitable model architectures based on these empirical insights. The benchmark is available at: https://github.com/AdityaLab/TimeRecipe . 1 Introduction Time-series forecasting plays a critical role in a wide range of real-world domains, including economics, urban computing, and epidemiology zhu2002statstream ; zheng2014urban ; deb2017review ; mathis2024evaluation ; liu2025cape . These applications focus on predicting future trends or events based on patterns observed in historical time-series data. Recently, the emergence of deep learning has significantly advanced the field, leading to the development of numerous forecasting models lai2018modeling ; torres2021deep ; salinas2020deepar ; Yuqietal-2023-PatchTST ; zhou2021informer . These models have demonstrated strong predictive accuracy and generalization capability across diverse datasets and domains, particularly within the supervised time-series forecasting paradigm. Despite recent successes, particularly at the model level (i.e., end-to-end forecasting architectures and pipelines), ongoing debates persist regarding the most effective deep learning strategies at the module level, referring to the internal design components within forecasting models. For example, while Transformer-based architectures are known for their ability to capture long-range temporal dependencies, they tend to struggle to generalize well on highly irregular time-series patterns, such as ETT time series. This has led to the reflection in reconsidering the effectiveness of MLP-based designs zeng2023transformers ; xu2023fits ; wang2024timemixer ; ni2025timedistill . Beyond architectural debates, further divergence arises in the design of specific modules and components. For instance, wang2024timemixer++ highlights the importance of separately modeling trend and seasonal components via series decomposition, which is overlooked in other contemporary works shi2024time . Additionally, zhao2023performative raises concerns for the effectiveness of instance normalization in short-term forecasting tasks when capturing rapid trend changes, despite it being a commonly effective practice primarily associated with long-term forecasting scenarios. However, despite ongoing debates surrounding divergent design choices at module level, existing benchmarks only emphasize evaluation at the model level qiu2024tfb ; aksu2024gift ; li2024foundts ; du2024tsi , overlooking the importance of benchmarking the effectiveness of specific module-level components. These studies typically conclude with case-specific best-performing models, which can be less practical when applying benchmarked results to real-world forecasting scenarios that fall outside the benchmarked settings brigato2025position . Moreover, while these benchmarks are comprehensive and empirically informative, they offer limited intuitive insights into why certain models outperform others in specific forecasting scenarios, leaving important questions about model behavior and design choices underexplored. To bridge this gap and gain a deeper understanding of the underlying factors driving model performance, we propose TimeRecipe . While existing benchmarks focus primarily on model-level evaluation and offer limited interpretability, TimeRecipe takes a step further by benchmarking time-series forecasting methods at a finer-grained, module level. Specifically, TimeRecipe aims to assess the effectiveness of individual components commonly used in state-of-the-art forecasting models. This enables us to answer a key research question: Which modules and model designs are most effective under specific time-series forecasting scenarios? Guided by this question, we summarize the main contributions of this work as follows: • Novel Benchmarking Scope: Unlike existing time-series forecasting benchmarks that focus primarily on holistic evaluation of entire state-of-the-art (SOTA) models, TimeRecipe focuses on assessing the effectiveness of individual modules commonly used in model construction. To the best of our knowledge, TimeRecipe is the first benchmark to systematically explore the design space of forecasting models for supervised time-series forecasting tasks at the modular level. • Comprehensive Evaluations: TimeRecipe evaluates hundreds of module combinations across diverse forecasting scenarios, spanning univariate and multivariate settings, as well as short- and long-term horizons. The benchmark encompasses dozens of datasets and involves over 10,000 distinct experiments, offering a robust and exhaustive evaluation framework. • Insightful Findings: The TimeRecipe benchmark reveals that exhaustive exploration of the modular design space can yield forecasting models that outperform existing SOTA approaches. Moreover, it uncovers meaningful correlations between module effectiveness and specific characteristics of time-series data and forecasting tasks, offering insights beyond raw accuracy. • Actionable Toolkit: Building on the above insights, we develop a training-free toolkit that makes model architecture selections within TimeRecipe . We demonstrate its effectiveness by comparing the selected architectures against those discovered via exhaustive search. Problem Formulation. We define the time-series forecasting problem following popular existing formulations zhou2021informer ; liu2023itransformer ; liu2024time : Given historical observations \text{X}_{t}=\{\bm{x}_{t-L},\ldots,\bm{x}_{t}\}\in\mathbb{R}^{L\times d_{\text{X}}} consisting of L past time steps and d_{\text{X}} variables at time step t , the goal is to predict the future H steps \text{Y}_{t}=\{\bm{x}_{t+1},\ldots,\bm{x}_{t+H}\}\in\mathbb{R}^{H\times d_{\text{X}}} . For convenience, we denote X as the collection of all \text{X}_{t} over the full time series of length T , and similarly, Y as the collection of all corresponding \text{Y}_{t} . The time-series forecasting task aims to learn a model parameterized by \theta through empirical risk minimization (ERM) to obtain f_{\theta}:\text{X}\rightarrow\text{Y} for all time steps t\in T . Additional Related Work. Due to page limitations, we provide an extended discussion of related works in Appendix A , including reviews of model designs for supervised learning and foundation models in time-series forecasting, as well as existing time-series forecasting benchmarks. 2 TimeRecipe Framework Since the introduction of Transformer-based architectures to time-series forecasting, particularly following the release of Informer zhou2021informer , most modern approaches have converged toward a common design paradigm, referred to here as the Canonical Architecture , as illustrated in Figure 1 . This canonical architecture consists of five major components: pre-processing, embedding, feed-forward modeling, projection, and post-processing (which is paired with pre-processing operations). The canonical architecture serves as the foundation of TimeRecipe and captures the typical structure adopted by many state-of-the-art (SOTA) forecasting methods. Figure 1: The proposed canonical architecture in TimeRecipe for constructing general time-series forecasting models. The canonical architecture comprises five key components: pre-processing, embedding, feed-forward modeling, projection, and post-processing. Following this structure, TimeRecipe is designed to offer a systematic evaluation with in-depth intuitions of alternative module choices across key components of the canonical architecture. Specifically, TimeRecipe benchmarks common module designs for pre-processing, embedding, and feed-forward modeling, with the aim of understanding their relative effectiveness across diverse forecasting scenarios and in conjunction with other design choices. For example, one example question to answer by our benchmark could be: how do different embedding strategies perform under varying task settings, and how do they interact with other designs from other components of the canonical architecture? We omit the benchmarking of the

Read the complete text