Channel, Trend and Periodic-Wise Representation Learning for Multivariate Long-term Time Series Forecasting
2025
Read Channel, Trend and Periodic-Wise Representation Learning for Multivariate Long-term Time Series Forecasting (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-23583.txt.
Introduction
Forecasting many linked series far ahead fails when a model sees each channel and each chunk in isolation. Zhangyao Song and 4 co-authors propose CTPNet, which learns from three angles at once. Attention across channels tracks how series move together. A transformer inside each downsampled chunk follows trend changes. Reusing the encoder across chunks then picks up global periodic patterns that link the chunks.
Downsample-based rivals mostly miss those cross-chunk and cross-channel ties, the authors argue. By joining all three views, CTPNet forms a fuller picture of the dynamics. Wide experiments show the method ahead of prior approaches on long-term forecasting tasks.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-23583
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Channel, Trend and Periodic-Wise Representation Learning for Multivariate Long-term Time Series Forecasting Abstract Downsampling-based methods for time series forecasting have attracted increasing attention due to their superiority in capturing sequence trends. However, this approaches mainly capture dependencies within subsequences but neglect inter-subsequence and inter-channel interactions, which limits forecasting accuracy. To address these limitations, we propose CTPNet, a novel framework that explicitly learns representations from three perspectives: i) inter-channel dependencies, captured by a temporal query-based multi-head attention mechanism; ii) intra-subsequence dependencies, modeled via a Transformer to characterize trend variations; and iii) inter-subsequence dependencies, extracted by reusing the encoder with residual connections to capture global periodic patterns. By jointly integrating these levels, proposed method provides a more holistic representation of temporal dynamics. Extensive experiments demonstrate the superiority of the proposed method. document 1 Introduction Time series forecasting, aims to predict future values based on historical observed time series data, has holds significant applications in fields such as energy management [ 1 ] , weather forecasting [ 2 ] , and traffic monitoring [ 3 , 4 ] . Furthermor, multivariate long-term time series forecasting has received increasing attention, due to it can provide more future information and utilize the dependencies between multiple variables [ 5 , 6 , 7 ] . Thanks to advances in deep learning, a variety of methods have been proposed for multivariate long-term time series forecasting, such as Recurrent Neural Network (RNN)-based [ 8 , 9 ] , Convolutional Neural Network (CNN)-based [ 10 , 7 , 11 , 12 ] , Multi-layer Perceptron (MLP)-based [ 13 , 14 ] , and Transformer-based [ 4 , 15 , 16 ] methods. However, the complexity of time series in real-world scenarios limits their capabilities. In fact, the foundation of accurate long-term time series forecasting lies in the inherent inter-channel dependencies and time dependencies within the data, where time dependencies are further divided into trend and periodic. For inter-channel dependencies, recent studies [ 6 , 5 , 15 ] have employed attention mechanisms, owing to their strong ability to extract multi-level representations. For temporal dependencies, a common strategy in recent works [ 17 , 18 , 19 ] is to downsample (see 1(b) ) the time series into subsequences according to its periodicity, which has demonstrated strong predictive capability. At this stage, each subsequence exhibits the trend variations of the time series, while the relationships across subsequences capture periodic dependencies. Specifically, [ 17 ] employs a weight-shared MLP to extract intra-subsequence dependencies. Since time series often follow fixed periodic patterns (e.g., daily period), each downsampled subsequence tends to exhibit highly similar or consistent trends. However, this approach mainly focuses on dependencies within individual subsequences, while neglecting inter-subsequence dependencies, which results in the loss of valuable temporal information. Moreover, it also overlooks the modeling of inter-channel dependencies [ 20 ] . These limitations highlight a fundamental question: how can we design a forecasting framework that effectively learns comprehensive temporal representations across, inter-channel, intra-subsequence, and inter-subsequence dependencies? In this paper, we address the above limitations by proposing CTPNet ( C hannel, T rend and P eriod-wise representation learning Net work), a novel framework that performs representation learning on three complementary levels of dependencies: i) I_{1} inter-channel dependencies, which capture interactions across variables; ii) I_{2} intra-subsequence dependencies, which model local temporal structures within each subsequence; and iii) I_{3} inter-subsequence dependencies, which uncover global temporal dynamics across subsequences. Specifically, for inter-channel dependencies, we employ a temporal query-based [ 5 ] Multi-Head Attention (MHA, [ 21 ] ). By using a learnable query matrix Q , the MHA mechanism effectively captures cross-channel dependencies. Secondly, for intra-subsequence dependencies, we adopt a Transformer with linear complexity [ 22 ] . The residual connection [ 23 ] between the inter-channel dependencies and the original input, followed by linear encoding [ 13 ] , is used as the Query, Key, and Value of the attention mechanism. In this way, the Transformer can effectively capture the complex trend variations (i.e., intra-subsequence dependencies) mapped in the high-dimensional hidden space. Finally, for inter-subsequence dependencies, we use the same network as before. By inputting the transposed encoder output with a residual connection to I_{2} , the network can model the inter-subsesquence dependencies. Using these three perspectives together, CTPNet offers a more holistic representation of time series data, overcoming the limitations of previous methods. Extensive experiments conducted on multiple benchmark datasets demonstrate the superiority of our approach. In a nutshell, our contributions are summarized in three folds: • Unlike previous methods that only exploit intra-subsequence dependencies within downsampled subsequences, we explicitly model inter-subsequence dependencies. • We propose CTPNet, a unified framework that simultaneously learns inter-channel, intra-subsequence, and inter-subsequence dependencies for comprehensive representation. • Through extensive experiments, we establish new state-of-the-art performance across multiple benchmarks, demonstrating the robustness and generality of our framework. ((a)) g_{1} : CRL ((b)) s : DownSample( \cdot ) ((c)) g_{2} (Top): TRL and g_{3} (Buttom): PRL X^{\text{in}} g_{1} I_{1} + s Encoder g_{2} I_{2} + g_{3} I_{3} Decoder s_{\text{De}} \hat{X}^{\text{out}} Figure 1 : Overview of CTPNet . Here g_{1} , g_{2} and g_{3} represent the representation learning networks for inter-channel dependencies, intra-subsequence dependencies, and inter-subsequence dependencies, respectively. s denotes \text{Downsample}(\cdot) , and s_{\text{De}} denotes it’s inverse process. 2 Preliminaries 2.1 Problem Statement Given a historical series X^{\text{in}}=[x_{t}^{(1:N_{c})},\cdots,x_{t-L_{\text{in}}-1}^{(1:N_{c})}]\in\mathbb{R}^{N_{c}\times L_{\text{in}}} of L_{\text{in}} time steps with N_{c} channels (e.g., the number of variables), the goal of time series forecasting (TSF) is to predict the series \hat{X}^{\text{out}}=[\hat{x}_{t+L_{\text{out}}}^{(1:N_{c})},\cdots,\hat{x}_{t+1}^{(1:N_{c})}]\in\mathbb{R}^{N_{c}\times L_{\text{out}}} of the next L_{\text{out}} time steps based on X^{\text{in}} , and to minimize the difference between \hat{X}^{\text{out}} and the target X^{\text{target}} . It can be formulated as [\hat{x}_{t+L_{\text{out}}}^{(1:N_{c})},\cdots,\hat{x}_{t+1}^{(1:N_{c})}]=f\left([x_{t}^{(1:N_{c})},\cdots,x_{t-L_{\text{in}}-1}^{(1:N_{c})}]\right), (1) where f represents a trainable neural network. 2.2 Time Series Downsampling Given a time series X=[x_{t}^{(n)},\cdots,x_{t-L_{\text{in}}-1}^{(n)}] with periodicity P , the operation of downsampling [ 17 ] decomposes the sequence into P downsampled subsequences of length N_{\text{pin}}=\lfloor\frac{L_{\text{in}}}{P}\rfloor . As shown in 1(b) , it can be formulated as [\mathbf{x}_{1},\cdots,\mathbf{x}_{P}]=\text{DownSample}(X), (2) where \mathbf{x}_{p}=[x_{t+p-1+0\times P},\cdots,x_{t+p-1+(N_{\text{pin}}-1)\times P}]\in\mathbb{R}^{N_{\text{pin}}} denotes the p -th subsequence. Meanwhile, we denote the inverse process of \text{DownSample}(\cdot) as \text{De-DownSample}(\cdot) . 3 Method: CTPNET 3.1 Structure Overview As illustrated in 1 , the proposed CTPNet consists of seven key components: ① Channel-wise Representation Learning (CRL) captures inter-channel dependencies; ② \text{Downsample}(\cdot) downsamples the residual connection of I_{1} and the original input to generate multiple subsequences; ③ Encoder maps each subsequence of length N_{\text{pin}} into a D -dimensional representation; ④ Trend-wise Representation Learning (TRL) captures intra-subsequence dependencies; ⑤ Periodic-wise Representation Learning (PRL) captures inter-subsequence dependencies; ⑥ Decoder converts the subsequence length from D back to N_{\text{pout}} , enabling the next operation to yield a prediction horizon of length L_{\text{out}} ; ⑦ \text{De-Downsample}(\cdot) recombinesthe multiple subsequences into a single sequence. Table 1 : Multivariate time series forecasting results. Model CTPNet TQNet [ 5 ] Twins* [ 24 ] Leddam [ 20 ] iTrans* [ 15 ] PatchTST [ 16 ] Cross* [ 6 ] FED* [ 1 ] (Ours) (2025) (2025) (2024) (2024) (2023) (2023) (2022) Metric MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE ETTh1 96 0.363 0.388 0.371 0.393 0.385 0.401 0.377 0.394 0.386 0.405 0.414 0.419 0.423 0.448 0.376 0.419 192 0.414 0.416 0.428 0.426 0.439 0.431 0.424 0.422 0.441