TSGym: Design Choices for Deep Multivariate Time-Series Forecasting
Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Minqi Jiang, Songqiao Han, Hailiang Huang, · 2025
Read TSGym: Design Choices for Deep Multivariate Time-Series Forecasting by Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Minqi Jiang, Songqiao Han, Hailiang Huang, (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-17063.txt.
Introduction
Shuang Liang and co-authors dissect design choices in time-series forecasting in a 2025 paper. Deep models now lead at predicting multivariate series, from sales to sensors. Most studies judge a whole model at once, which hides which choices matter. Their testbed, TSGym, varies parts systematically so each choice can be credited or blamed.
Extensive tests show the gym beats top forecasting and automated machine learning methods. All code is shared in public. A shared gym for design choices could end wasteful folklore in forecasting, where teams copy full recipes without knowing which ingredients earn their keep. New models can be probed piece by piece.
Catalog record
| CREATOR | Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Minqi Jiang, Songqiao Han, Hailiang Huang, |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 126 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-17063
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TSGym: Design Choices for Deep Multivariate Time-Series Forecasting Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Minqi Jiang, Songqiao Han, Hailiang Huang, Abstract Recently, deep learning has driven significant advancements in multivariate time series forecasting (MTSF) tasks. However, much of the current research in MTSF tends to evaluate models from a holistic perspective, which obscures the individual contributions and leaves critical issues unaddressed. Adhering to the current modeling paradigms, this work bridges these gaps by systematically decomposing deep MTSF methods into their core, fine-grained components like series-patching tokenization, channel-independent strategy, attention modules, or even Large Language Models and Time-series Foundation Models. Through extensive experiments and component-level analysis, our work offers more profound insights than previous benchmarks that typically discuss models as a whole. Furthermore, we propose a novel automated solution called TSGym for MTSF tasks. Unlike traditional hyperparameter tuning, neural architecture searching or fixed model selection, TSGym performs fine-grained component selection and automated model construction, which enables the creation of more effective solutions tailored to diverse time series data, therefore enhancing model transferability across different data sources and robustness against distribution shifts. Extensive experiments indicate that TSGym significantly outperforms existing state-of-the-art MTSF and AutoML methods. All code is publicly available on https://github.com/SUFE-AILAB/TSGym . 1 Introduction Multivariate time series refer to time series data involving multiple interdependent variables, which are widely present in various fields such as finance sezer2020financial , energy alvarez2010energy ; deb2017review , traffic cirstea2022towards ; yin2016forecasting , and health bui2018time ; kaushik2020ai . Among the numerous analysis tasks, multivariate time series forecasting (MTSF) attracts substantial attention from the research community due to its significant practical applications. Traditional approaches to MTSF are largely based on statistical methods abraham2009statistical ; zhang2003time and machine learning techniques hartanto2023stock ; masini2023machine . In recent years, deep learning (DL) has become the most active area of research for MTSF, driven by its ability to handle complex patterns and large-scale datasets effectively wang2024deep . Early academic efforts of deep MTSF methods like RNN-type methods yamak2019comparison are reported to struggle with capturing long-term temporal dependencies due to their inherent limitations of gradient vanishing or exploding problem zhou2021informer ; zhou2022fedformer . More recently, Transformer vaswani2017attention shows significant potential, largely due to the effectiveness of its attention mechanisms in modeling temporal correlation vaswani2017attention ; wen2022transformers . Consequently, attention mechanism has continuously been studied in MTSF, with a focus on adapting them to time series data, for instance, by exploiting sparsity inductive bias li2019enhancing ; zhou2021informer , transforming time and frequency domains zhou2022fedformer , and fusing multi-scale series liu2022pyraformer . While simpler MLP-based structures emerged Zeng2022AreTE offering alternatives to the established Transformer architecture in MTSF, notable modeling strategies like series-patching and channel-independent nie2023PatchTST , significantly enhanced the performance of Transformer-based methods, thereby sustaining research interest in them. Building upon these developments, large time-series models including large language models (LLMs) jin2024position ; zhou2023one ; jin2023time and time series foundation models (TSFMs) jin2023lm4ts ; liu2024timer have recently been introduced, achieving promising results and fostering new research directions for MTSF. Alongside these advancements in model architectures, active research within the deep MTSF community also focuses on other critical topics, such as variable (channels) dependency modeling nie2023PatchTST ; liu2024itransformer ; zhang2023crossformer , series normalization methods Liu2022NonstationaryTR ; fan2023dish , and trend-seasonal decomposition Zeng2022AreTE ; liu2023koopa . As the field of MTSF continues to diversify, existing studies typically address critical concerns about methodological effectiveness, either by conducting large-scale benchmarks wang2024deep ; shao2024exploring ; qiu2024tfb or performing model selection via AutoML abdallah2022autoforecast ; fischer2024autoxpcr . However, we identify three main challenges with these prevailing approaches: First, the granularity of existing studies is insufficient . Current benchmarking works evaluate or select models as a whole, which hinders a deeper understanding of the mechanisms that drive model performance. In AutoML, this lack of granularity prevents breakthroughs beyond the limits of existing models. Second, the scope of existing studies is limited . Current benchmarking and automated selection efforts are often confined to restricted model architectures or hyperparameters, without covering a broad range of data processing methods or feature modeling techniques. Third, the range of existing studies is narrow . Existing studies tend to cover only a subset of network architectures and often lack discussions on more diverse models, such as LLMs and TSFMs. Figure 1 : The Pipeline of Existing Multivariate Time Series Forecasting Methods. To bridge these gaps, we propose TSGym—a framework designed for the Large-scale Evaluation , Analysis , and Automated Model Construction in deep MTSF tasks. Rather than viewing models as unified entities, TSGym systematically deconstructs popular deep MTSF methods by organizing them into distinct design dimensions that cover the entire time series modeling pipeline (see Fig. 1 and Table 1 ). Through extensive experiments, TSGym conducts fine-grained, isolated evaluations of core components, thereby identifying key design dimensions/choices and valuable insights from the vast MTSF methods. Leveraging a large-scale benchmark, TSGym systematically validates or refutes a series of prevailing claims within the MTSF community, including comparisons between Transformer and MLP architectures, as well as the adaptability of channel-independent approaches. Moreover, TSGym proposes the first component-level model construction in MTSF tasks, which effectively overcomes limitations in the previous automation methods by enabling more flexible and customized model designs tailored to data characteristics. Extensive experimental results indicate that the proposed TSGym generally outperforms existing SOTA methods. We summarize the key contributions of TSGym as follows: Component-level evaluation of MTSF methods . We propose TSGym, the first large-scale benchmark that systematically decouples deep MTSF methods. By evaluating 16 design dimensions across 10 benchmark datasets, TSGym elucidates contested issues in the current community and offers key insights to inform future development for MTSF. Automated MTSF model construction . Leveraging meta-learning, TSGym develops models that outperform current SOTA methods, offering the MTSF community an effective, automated, and data-adaptive solution for model design. Discussion on emerging large time-series models . TSGym broadens current MTSF scope by applying systematic evaluation and automated combination not only to well-established models like MLP and Transformer, but also to novel large time-series models like LLMs and TSFMs. 2 Related Work 2.1 Deep Learning-based MTSF MTSF evolves from traditional statistical methods like ARIMA and Gaussian processes to modern deep learning approaches. Recurrent Neural Networks (RNNs) introduce memory mechanisms for sequential data but struggle with long-term dependencies. Temporal Convolutional Networks (TCNs) improve this by capturing multi-scale patterns, though their fixed window sizes limit global context. Transformers, using self-attention, enable long-range forecasting but introduce high computational complexity, leading to efficient variants like sparse attention wu2021autoformer and patch-based models nie2023PatchTST . Multilayer Perceptrons (MLPs) regain attention as simple yet effective models zeng2023dlinear , with numerous variants offering competitive performance chen2023tsmixer ; yi2023frets ; das2023TiDE ; liu2023koopa . Leveraging NLP foundation models, LLM adaptation approaches use frozen backbones and prompt engineering jin2024timellm ; zhou2023one or fine-tuning chang2023llm4ts to transfer pretrained knowledge. Simultaneously, pure TSFMs trained on large datasets achieve zero-shot generalization liu2024timer ; goswami2024moment , though constrained by Transformers’ complexity. Our TSGym framework modularizes six core backbones—RNNs, CNNs, Transformers, MLPs, LLMs, and TSFMs—offering flexible, hybrid integration based on temporal dependencies and resource needs. In recent advancements in MTSF, we summarize the design paradigm through a unified pipeline (Fig. 1 ), consisting of four stages: Series Preprocessing \rightarrow Series Encoding \rightarrow Network