DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting

Qianyang Li, Xingjun Zhang, Shaoxun Wang, Wei Jia · 2025

Read DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting by Qianyang Li, Xingjun Zhang, Shaoxun Wang, Wei Jia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14868.txt.

Introduction

Qianyang Li, Xingjun Zhang, Shaoxun Wang, and Wei Jia propose DPANet for long-term time series forecasting in a 2025 machine learning paper. Long-range forecasts are hard because patterns repeat at several time scales and frequencies at once. Earlier Transformer and multilayer perceptron models often fail to capture that mix in one structured design.

DPANet builds two pyramids side by side. A temporal pyramid progressively downsamples the series to catch slow and fast dynamics, while a frequency pyramid splits the signal by band-pass filtering to catch cycles of different lengths. A cross-pyramid fusion block exchanges information between matching levels through cross-attention, working from coarse to fine. Experiments on public benchmarks report state-of-the-art accuracy, ahead of earlier models across the tested forecasting tasks.

Catalog record

CREATORQianyang Li, Xingjun Zhang, Shaoxun Wang, Wei Jia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE20 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-14868

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DPANet: Dual Pyramid Attention Network for Multivariate Time Series Forecasting Qianyang Li, Xingjun Zhang, Shaoxun Wang, Wei Jia Abstract Long-term time series forecasting (LTSF) is hampered by the challenge of modeling complex dependencies that span multiple temporal scales and frequency resolutions. Existing methods, including Transformer and MLP-based models, often struggle to capture these intertwined characteristics in a unified and structured manner. We propose the Dual Pyramid Attention Network (DPANet), a novel architecture that explicitly decouples and concurrently models temporal multi-scale dynamics and spectral multi-resolution periodicities. DPANet constructs two parallel pyramids: a Temporal Pyramid built on progressive downsampling, and a Frequency Pyramid built on band-pass filtering. The core of our model is the Cross-Pyramid Fusion Block, which facilitates deep, interactive information exchange between corresponding pyramid levels via cross-attention. This fusion proceeds in a coarse-to-fine hierarchy, enabling global context to guide local representation learning. Extensive experiments on public benchmarks show that DPANet achieves state-of-the-art performance, significantly outperforming prior models. Code is available at https://github.com/hit636/DPANet. † † address: 1 School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China 2 Department of Computer Science and Technology, Tsinghua University, Beijing, China {liqianyang,shaoxunwang}@stu.xjtu.edu.cn, [email protected], [email protected] 1 Introduction Time series forecasting is a critical task in numerous real-world applications, driven by its broad practical utility and complex theoretical questions [ 1 ] . The field offers significant impacts across numerous critical sectors, including financial market analysis [ 2 ] , energy demand management [ 3 ] , meteorological forecasting [ 4 ] , and transportation system optimization [ 5 ] .While recent advances in deep learning, particularly with Transformer and MLP-based architectures [ 6 ] , have significantly improved performance, a fundamental challenge remains: effectively modeling the hierarchical structure inherent in time series data. Real-world time series are compositions of multi-scale temporal patterns (e.g., daily vs. yearly trends) and multi-resolution frequency components (e.g., high-frequency fluctuations vs. low-frequency seasonalities). These properties are orthogonal yet deeply coupled; for instance, a long-term seasonal trend modulates high-frequency daily patterns. However, existing models like PatchTST [ 7 ] , while excelling at long-range dependencies, lack an explicit structure for this intricate multi-resolution analysis. This paper argues that addressing the challenges of Long-Term Time Series Forecasting (LTSF) requires a model architecture that mirrors the inherent multi-level structure of time series data. Drawing inspiration from pyramid networks in signal and image processing [ 8 ] , and building on recent advances in multi-scale fusion models [ 9 ] , we propose a fusion approach that decouples temporal and frequency hierarchies and integrates them through interactive fusion. We introduce the Dual Pyramid Attention Network (DPANet), which employs a dual-stream architecture with two pyramids: a Temporal Pyramid that captures multi-scale hierarchical features via downsampling, and a Frequency Pyramid that disentangles periodic patterns using band-pass filtering. The core innovation is the Cross-Pyramid Fusion Block, enabling deep information exchange between corresponding pyramid levels through cross-attention. Our contributions are threefold: (1) We propose a novel dual-pyramid architecture for parallel time and frequency domain analysis. (2) We design a cross-pyramid fusion mechanism for a deep hierarchical feature interaction. (3) We empirically establish the superiority of DPANet as a new strong baseline for LTSF. 2 Related Work 2.1 Transformer-based Time Series Forecasting Transformer-based models have set the state of the art by leveraging attention to capture long-range dependencies. Innovations have focused on improving efficiency and adapting the mechanism for temporal data. Notable examples include Informer [ 10 ] with its ProbSparse attention, Autoformer [ 11 ] with its auto-correlation mechanism, and FEDformer [ 12 ] which operates in the frequency domain. More recently, PatchTST [ 7 ] , inspired by Vision Transformers (ViT), has shown great success by processing time series in patches. While powerful, these methods typically operate within a single domain (either time or frequency). DPANet distinguishes itself by explicitly modeling both domains in parallel and facilitating deep, hierarchical interaction between them. 2.2 MLP-based Time Series Forecasting Recent works have demonstrated that lightweight MLP-based models can be surprisingly effective. N-BEATS [ 13 ] used a deep stack of MLP blocks with a basis expansion principle. DLinear [ 6 ] further simplified this, showing that a single linear layer applied to decomposed trend and seasonality can outperform complex Transformers. Following this trend, models like TSMixer [ 14 ] have successfully adapted the MLP-Mixer architecture for efficient temporal and channel-wise feature extraction. DPANet learns from this paradigm by using efficient computational blocks but embeds them within a structured, hierarchical framework to explicitly capture the multi-scale and multi-resolution properties that simple MLPs overlook. 2.3 Multi-Scale and Multi-Resolution Methods Multi-scale and multi-resolution analysis is a classic approach to time series that has been integrated into various deep learning models. Pyraformer [ 15 ] used a pyramidal attention structure to reduce complexity, while TimesNet [ 16 ] reshapes 1D series into 2D tensors based on periodicities discovered via FFT. Recent mixing models like TimeMixer [ 17 ] leverage multi-scale decomposition. DPANet advances this line of research by constructing two distinct, parallel pyramids for the temporal and spectral domains and introduces a novel mechanism for their interactive fusion, enabling a more comprehensive and principled hierarchical analysis than prior works. 3 Method 3.1 DPANet Architecture Given a historical multivariate time series \mathbf{X}\in\mathbb{R}^{L_{\text{in}}\times C} , where L_{\text{in}} is the look-back window length and C is the number of variates, our objective is to predict the future sequence \mathbf{Y}\in\mathbb{R}^{L_{\text{pred}}\times C} of length L_{\text{pred}} . As illustrated in Figure 1 , the DPANet architecture systematically decouples and fuses multi-scale temporal and multi-resolution information. The input is first normalized via a RevIN layer [ 18 ] . The model’s core is a stack of novel Cross-Pyramid Fusion Blocks operating on parallel temporal and frequency pyramids. The fusion adheres to a coarse-to-fine hierarchical principle, where high-level semantics guide low-level feature learning. A final prediction head maps the fused representation to the forecast, which is then de-normalized by the inverse RevIN transformation. Figure 1: The overall architecture of DPANet. The model decouples a time series into parallel (a) Temporal and (b) Frequency Pyramids. These representations are then fused through a stack of (c) Cross-Pyramid Fusion Blocks in a coarse-to-fine hierarchy. 3.2 Dual-Pyramid Representation To simultaneously capture dynamics across different time granularities and frequency bands, we construct two parallel feature pyramids from the normalized input \mathbf{X}^{\prime} . Temporal Pyramid: The temporal pyramid, \mathcal{P}_{t}=\{\mathbf{T}^{(0)},\dots,\mathbf{T}^{(S-1)}\} , is created by progressively downsampling the input series \mathbf{X}^{\prime} in the time domain. Let \mathbf{T}^{(0)}=\mathbf{X}^{\prime} . Each subsequent level is obtained by: \mathbf{T}^{(s)}=\text{AvgPool1d}(\mathbf{T}^{(s-1)})\in\mathbb{R}^{(L_{\text{in}}/2^{s})\times C} (1) where s\in\{1,\dots,S-1\} is the scale level. This pyramid effectively captures the series’ structure from fine-grained details to coarse-grained trends. Frequency Pyramid: The frequency pyramid P_{f}=\{F^{(0)},\dots,F^{(S-1)}\} is created by decomposing the input series into different frequency resolutions. We first compute the Real Fourier Transform of the input \mathcal{F}=\text{RFFT}(X^{\prime}) . The spectrum is then partitioned into S disjoint, logarithmically-spaced bands. This logarithmic partitioning is crucial: it allocates finer resolution to the low-frequency components, which often correspond to the dominant, long-term seasonalities, while grouping high-frequency noise more coarsely. For each band s , we create a binary mask M^{(s)} to isolate its range of frequencies. The corresponding pyramid level F^{(s)} is reconstructed by applying this mask and transforming the result back into the time domain

Read the complete text