InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling
Yuhang Wang · 2025
Read InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling by Yuhang Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-20302.txt.
Introduction
Yuhang Wang builds InvDec, a net for forecasts of many time series at once. It keeps time traits and cross-series traits in split parts. PatchTST is strong on time traits but skips links from one series to the next. iTransformer flips that flaw. InvDec pairs a patch-based time coder with a flipped decoder that attends across the series side.
Late series tags enrich each series code only after time work is done. That guard keeps time traits clean. A tuned mix step weighs time facts and series facts for each set. On seven test sets, the PatchTST form cuts error by near 21 percent on a 321-series power set. Gains hold on most sets, and the edge grows as series count grows.
Catalog record
| CREATOR | Yuhang Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-20302
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
InvDec: Inverted Decoder for Multivariate Time Series Forecasting with Separated Temporal and Variate Modeling Yuhang Wang Abstract Multivariate time series forecasting requires simultaneously modeling temporal patterns and cross-variate dependencies. Channel-independent methods such as PatchTST excel at temporal modeling but ignore variable correlations, while pure variate-attention approaches such as iTransformer sacrifice temporal encoding. We propose InvDec (Inverted Decoder), a hybrid architecture that achieves principled separation between temporal encoding and variate-level decoding. InvDec combines a patch-based temporal encoder with an inverted decoder operating on the variate dimension through variate-wise self-attention. We introduce delayed variate embeddings that enrich variable-specific representations only after temporal encoding, preserving temporal feature integrity. An adaptive residual fusion mechanism dynamically balances temporal and variate information across datasets of varying dimensions. Instantiating InvDec with PatchTST yields InvDec-PatchTST. Extensive experiments on seven benchmarks demonstrate significant gains on high-dimensional datasets: 20.9% MSE reduction on Electricity (321 variables), 4.3% improvement on Weather, and 2.7% gain on Traffic compared to PatchTST, while maintaining competitive performance on low-dimensional ETT datasets. Ablation studies validate each component, and analysis reveals that InvDec’s advantage grows with dataset dimensionality, confirming that cross-variate modeling becomes critical as the number of variables increases. 1 Introduction Time series forecasting is a fundamental problem with widespread applications across diverse domains, including energy management, transportation planning, climate science, healthcare monitoring, and financial market analysis [ 1 , 2 ] . The ability to accurately predict future trends and patterns from historical observations is critical for decision-making, resource allocation, and risk management in these fields [ 3 ] . Traditional statistical approaches such as ARIMA [ 4 ] have long served as the foundation for time series analysis, offering interpretable models based on stationarity assumptions and linear relationships. However, as real-world time series data have grown increasingly complex—characterized by high dimensionality, non-stationarity, and intricate temporal dependencies—the limitations of classical methods have become apparent [ 1 ] . The advent of deep learning has revolutionized time series forecasting, enabling models to capture non-linear patterns and long-range dependencies that elude traditional techniques [ 2 ] . Among deep learning architectures, Transformers [ 5 ] have emerged as particularly powerful, leveraging self-attention mechanisms to model temporal dynamics across extended horizons. Early Transformer-based forecasting models such as Informer [ 6 ] , Autoformer [ 7 ] , and FEDformer [ 8 ] focused primarily on improving computational efficiency and capturing temporal patterns through innovations such as sparse attention and decomposition strategies. Despite their success, these models largely adopt a channel-independent paradigm, treating each variable in a multivariate time series as an isolated entity [ 3 ] . This design choice, while computationally efficient, overlooks the rich correlations and interactions among variables—a critical limitation in high-dimensional forecasting scenarios where cross-variate dependencies often encode essential information about underlying system dynamics [ 3 ] . Recent research has begun to address this gap by exploring architectures that explicitly model relationships across the variate dimension. Notably, methods such as iTransformer [ 9 ] invert the traditional Transformer design by applying attention mechanisms on variates rather than time steps, while others such as Crossformer [ 10 ] interleave temporal and variate modeling throughout the network. Although these approaches demonstrate the importance of cross-variate dependencies, they often involve trade-offs: some sacrifice fine-grained temporal modeling, while others introduce architectural complexity that complicates interpretation and increases computational overhead. Furthermore, specialized methods designed for exogenous forecasting [ 11 ] or spatial grid data [ 12 ] address domain-specific challenges but do not generalize to arbitrary multivariate time series where variables lack explicit spatial topology or external covariates. In this paper, we propose InvDec ( Inv erted Dec oder), a novel hybrid architecture that achieves a principled separation between temporal encoding and variate decoding. InvDec builds upon the strengths of patch-based temporal modeling [ 13 ] while introducing an inverted decoder that operates on the variate dimension to capture cross-variate dependencies. Unlike prior methods that entangle temporal and variate modeling or sacrifice one for the other, InvDec explicitly decouples these objectives: the encoder focuses solely on extracting temporal features from each variable’s sequence, while the decoder performs variate-level attention to model inter-variable correlations. To further enrich variate representations, we incorporate learnable variate embeddings [ 11 ] in the decoder, but crucially delay their introduction until after temporal encoding to avoid distorting temporal patterns. Additionally, an adaptive residual fusion mechanism dynamically balances contributions from temporal and variate pathways, enabling the model to adapt to datasets of varying dimensions and correlation structures. Figure 1: Performance overview: InvDec-PatchTST (red) consistently outperforms baselines on high-dimensional datasets (Electricity, Weather, Traffic) while remaining competitive on low-dimensional ETT benchmarks. Radial distance represents normalized MSE. Our contributions in this work are threefold. First, we introduce InvDec, a hybrid encoder-inverted decoder architecture that, to the best of our knowledge, represents the first work to combine a temporal encoder with a variate-level decoder in this manner, achieving superior performance on multivariate forecasting tasks through clear separation of modeling concerns. Second, we propose a delayed variate embedding strategy that enriches variate-specific representations without interfering with temporal encoding, alongside an adaptive residual fusion mechanism that enables dynamic balancing of temporal and variate information across diverse datasets. Third, we conduct extensive experiments on seven widely-used benchmarks, demonstrating that InvDec achieves significant performance gains on high-dimensional datasets—including a 20.9% MSE reduction on Electricity (321 variables) compared to PatchTST—while maintaining competitive performance on low-dimensional datasets (Figure 1 ), with comprehensive ablation studies validating the effectiveness of each architectural component. 2 Related Work 2.1 Deep Learning Architectures for Time Series Forecasting Prior to the widespread adoption of Transformers, deep learning approaches to time series forecasting relied primarily on recurrent neural networks (RNNs), convolutional neural networks (CNNs), and multi-layer perceptrons (MLPs) [ 1 , 2 ] . RNN-based architectures, including LSTMs and GRUs, were designed to capture sequential dependencies through recurrent connections, enabling them to model temporal dynamics in univariate and multivariate settings [ 14 ] . However, these methods suffer from training instability, vanishing gradients, and limited capacity to capture long-range dependencies [ 1 ] . CNN-based approaches such as Temporal Convolutional Networks (TCN) [ 15 ] leverage dilated convolutions to expand receptive fields, achieving competitive performance while maintaining computational efficiency. Recent works such as TimesNet [ 16 ] and ModernTCN [ 17 ] further demonstrate the effectiveness of convolutional architectures by exploiting multi-scale temporal patterns and hierarchical feature extraction. Graph Neural Networks (GNNs) have also been explored for time series forecasting, particularly in scenarios where variables exhibit explicit spatial or relational structures [ 18 ] . Methods such as spatial-temporal graph convolutional networks [ 19 ] and temporal graph networks [ 20 ] model interactions among variables through graph attention mechanisms [ 21 ] . While effective for spatially-structured data (e.g., traffic networks, sensor grids), these approaches require predefined graph topologies and may not generalize to arbitrary multivariate time series where relationships among variables are latent and dynamic. In parallel, MLP-based architectures have re-emerged as surprisingly competitive alternatives to more complex models. DLinear [ 22 ] challenges the necessity of Transformers by demonstrating that simple linear mappings can outperform sophisticated architectures on certain benchmarks. Building on this insight, methods such as N-BEATS [ 23 ] , N-HiTS [ 24 ] , TSMixer [ 25