Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 9–13, 2026; Jeju Island, Republic of Korea. Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD 2026), August 9–13, 2026, Jeju Island, Republic of Korea ISBN: 979-8-4007-2259-2/2026/08 DOI: 10.1145/3770855.3817551 v2dtb413 CCS: Computing methodologies Machine learning
Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Hailiang Huang, Songqiao Han, Minqi Jiang · 2026
Read Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 9–13, 2026; Jeju Island, Republic of Korea. Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD 2026), August 9–13, 2026, Jeju Island, Republic of Korea ISBN: 979-8-4007-2259-2/2026/08 DOI: 10.1145/3770855.3817551 v2dtb413 CCS: Computing methodologies Machine learning by Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Hailiang Huang, Songqiao Han, Minqi Jiang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-26562.txt.
Introduction
Shuang Liang and six colleagues shift multivariate forecasting from whole models to part-level view with TSCOMP. The team breaks deep forecasting methods into fine parts: preprocessing, encoding plans, network designs, and optimization methods. Constrained test design and wide tests show part value across backbones and data traits. The benchmark offers a performance corpus over 20,000 model-dataset runs supporting automated part selection and zero-shot building. Corpus-driven selection beats top methods, showing systematic part selection beats manual complex designs.
Catalog record
| CREATOR | Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Hailiang Huang, Songqiao Han, Minqi Jiang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 103022 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.26562
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Holistic Models: Systematic Component-level Benchmarking of Deep Multivariate Time-Series Forecasting Conference: Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2; August 9–13, 2026; Jeju Island, Republic of Korea. Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD 2026), August 9–13, 2026, Jeju Island, Republic of Korea ISBN: 979-8-4007-2259-2/2026/08 DOI: 10.1145/3770855.3817551 v2dtb413 CCS: Computing methodologies Machine learning Shuang Liang, Chaochuan Hou, Xu Yao, Shiping Wang, Hailiang Huang, Songqiao Han, Minqi Jiang Abstract . While previous research in multivariate time series forecasting has focused on developing complex holistic models, this work advocates for a shift toward a granular, component-level understanding of their impacts. We propose TSCOMP, the first large-scale benchmark that systematically deconstructs deep forecasting methods into their core, fine-grained components—spanning series preprocessing, encoding strategies, network architectures including specific and large time-series models, and optimization methods. Using constrained orthogonal experimental design and extensive evaluations, we conduct multi-view analyses that reveal component effectiveness across different backbones, data characteristics, and their interactions. Beyond providing insights, this benchmark establishes a fine-grained performance corpus comprising over 20,000 model-dataset evaluations, which supports the learning of automated component selection, enabling zero-shot model construction on new datasets. Our experiments demonstrate that the corpus-driven approach, despite its simplicity, consistently outperforms state-of-the-art methods, validating the soundness of our evaluation design and confirming that systematic component selection surpasses manually designed complex architectures. All code and the performance corpus are publicly available at https://github.com/SUFE-AILAB/TSCOMP . † † cc-license: by 1. Introduction Figure 1 . Overview of the proposed TSCOMP framework. TSCOMP deconstructs existing SOTA models into a modular component pool. Through large-scale experimental analysis, TSCOMP conducts bottom-up evaluation from component-level comparisons to dimension-level and pipeline-level importance ranking. The resulting performance corpus enables automated model construction via a pre-trained meta-predictor that delivers zero-shot, data-adaptive component selection. Multivariate time series refers to time series data involving multiple interdependent variables, which are widely present in various fields such as finance ( 44 ) , energy ( 3 ; 13 ) , traffic ( 12 ; 62 ) , and health ( 7 ; 23 ) . Among the numerous analysis tasks, multivariate time series forecasting (MTSF) attracts substantial attention from the research community due to its significant practical applications. Traditional approaches to MTSF are largely based on statistical methods ( 2 ; 65 ) and machine learning techniques ( 20 ; 39 ) . In recent years, deep learning (DL) has become the most active area of research for MTSF, driven by its ability to handle complex patterns and large-scale datasets effectively ( 53 ) . Early academic efforts of deep MTSF methods like RNN-type methods ( 60 ) struggle with capturing long-term temporal dependencies due to their inherent limitations of gradient vanishing or exploding problems ( 68 ; 70 ) . To address these issues, Transformer shows significant potential by effectively modeling temporal correlations via attention variants ( 29 ; 70 ) . Although simpler MLP-based structures ( 64 ) later challenged this paradigm, innovations like patching and channel-independence strategies ( 40 ) further enhanced its performance. Alongside these architectural advances, critical modular studies have emerged, focusing on variable dependency ( 67 ) , normalization ( 36 ) , and decomposition ( 34 ) . This evolution has recently expanded to Large Language Models (LLMs) ( 22 ; 71 ) and Time Series Foundation Models (TSFMs) ( 37 ; 17 ) . As the field of MTSF continues to diversify, existing studies typically address concerns about methodological effectiveness by conducting large-scale benchmarks ( 53 ; 45 ; 41 ) . These studies consistently indicate that no single approach—whether a specific deep forecasting model (e.g., MLP, Transformer) or large time-series models—dominates across all scenarios ( 38 ) . This variability suggests the need to investigate effective MTSF design at finer granularities. Specifically, the MTSF formulation involves a multi-stage modeling pipeline, where each stage (e.g., series preprocessing) comprises distinct component dimensions (e.g., series normalization) instantiated by specific components (e.g., RevIN). However, existing benchmarks typically evaluate models holistically, failing to analyze this multi-level hierarchy. Consequently, contributions of internal mechanisms remain obscured. This ambiguity isolates effective designs within specific methods, hindering the combination of these strengths into superior solutions. To bridge these gaps, we propose TSCOMP, a comprehensive framework designed to systematically deconstruct and benchmark deep MTSF methods. Instead of viewing models as indivisible black boxes, TSCOMP performs a hierarchical deconstruction across three levels: the Pipeline , Component Dimensions , and Deconstructed Components (see Fig. 1 ). To ensure rigorous evaluation, we employ a constrained orthogonal experimental protocol that isolates the contribution of individual components. This enables a multi-view analysis that extends beyond general performance rankings: we investigate component efficacy under different backbones, their distinct adaptability to diverse data characteristics and domains, and the complex interactions between deconstructed components. Building upon the proposed benchmark, we establish a fine-grained performance corpus that not only validates prevailing claims but also serves as a robust foundation for automated model construction. Based on this valuable corpus, TSCOMP learns how components adapt to different data characteristics and adaptively assembles optimal components tailored to specific datasets. This approach consistently surpasses state-of-the-art methods. We summarize the key contributions of TSCOMP as follows: • Comprehensive Benchmark via Hierarchical Deconstruction . We propose TSCOMP, the first large-scale benchmark that systematically deconstructs deep MTSF methods. TSCOMP examines the MTSF workflow through a hierarchical design space, spanning from the overall modeling pipeline to fine-grained specific components. To rigorously assess these elements, we design a constrained orthogonal evaluation protocol that isolates the core mechanisms driving forecasting performance. • Multi-View Analysis and Insights . We conduct a large-scale analysis that provides both overall and conditional insights. Beyond evaluating general component effectiveness, we extensively investigate performance variations across different backbones (including specific models and emerging LLMs/TSFMs), diverse data domains, and data characteristics. Furthermore, we explore the intricate interaction effects among deconstructed components, verifying community claims with rigorous experimental evidence. • Open-Sourced Corpus and Automated Construction . We open-source the resulting fine-grained performance corpus and validate its utility for model design. This corpus facilitates automated construction of MTSF methods that are adaptively tailored to different forecasting scenarios, consistently achieving better results than state-of-the-art methods. 2. Related Work 2.1. Deep Learning-based MTSF MTSF evolves from traditional statistical methods like ARIMA and Gaussian processes to modern deep learning approaches. While legacy RNNs struggle with long-term dependencies, Transformers revolutionize temporal modeling through attention mechanisms ( 59 ; 40 ) . Alternatively, MLP-based models regain prominence for their simplicity and effectiveness. DLinear ( 64 ) demonstrates that linear mappings with decomposition often surpass complex Transformers. Advanced variants like TimeMixer ( 52 ) and OLinear ( 63 ) leverage multi-scale analysis and orthogonal decomposition. Leveraging foundation models represents a paradigm shift. Adaptation methods like GPT4TS ( 71 ) transfer language model knowledge via prompt engineering ( 22 ) or fine-tuning ( 9 ) . Native TSFMs like Timer ( 37 ) and Time-MOE ( 47 ) target zero-shot generalization. In this work, TSCOMP systematically deconstructs these diverse methodologies into atomic components across the entire forecasting pipeline. Convergent design directions emerge across these approaches. Preprocessing addresses non-stationarity via adaptive normalization ( 25 ; 14 ) or decomposition (