MDMLP-EIA: Multi-domain Dynamic MLPs with Energy Invariant Attention for Time Series Forecasting

Hu Zhang, Zhien Dai, Zhaohui Tang, Yongfang Xie · 2025

Read MDMLP-EIA: Multi-domain Dynamic MLPs with Energy Invariant Attention for Time Series Forecasting by Hu Zhang, Zhien Dai, Zhaohui Tang, Yongfang Xie (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-09924.txt.

Introduction

Hu Zhang, Zhien Dai, Zhaohui Tang, and Yongfang Xie propose MDMLP-EIA, a forecasting model for time series built from multilayer perceptrons rather than attention. Perceptron models are lean and robust, but they can lose weak seasonal signals, run short of capacity when channels multiply, and fuse channels poorly. The design answers each weakness with a named mechanism.

An adaptive dual-domain seasonal module separates strong and weak seasonal parts and fuses channels with little noise. An energy invariant attention focuses on different feature channels while keeping total signal energy constant, which fits the decompose, predict, and rebuild pipeline. A dynamic rule scales neuron counts with channel count. Across nine benchmark datasets, the model reached state-of-the-art accuracy and efficiency.

Catalog record

CREATORHu Zhang, Zhien Dai, Zhaohui Tang, Yongfang Xie
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-09924

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MDMLP-EIA: Multi-domain Dynamic MLPs with Energy Invariant Attention for Time Series Forecasting Hu Zhang, Zhien Dai, Zhaohui Tang, Yongfang Xie Abstract Time series forecasting is essential across diverse domains. While MLP-based methods have gained attention for achieving Transformer-comparable performance with fewer parameters and better robustness, they face critical limitations including loss of weak seasonal signals, capacity constraints in weight-sharing MLPs, and insufficient channel fusion in channel-independent strategies. To address these challenges, we propose MDMLP-EIA (Multi-domain Dynamic MLPs with Energy Invariant Attention) with three key innovations. First, we develop an adaptive fused dual-domain seasonal MLP that categorizes seasonal signals into strong and weak components. It employs an adaptive zero-initialized channel fusion strategy to minimize noise interference while effectively integrating predictions. Second, we introduce an energy invariant attention mechanism that adaptively focuses on different feature channels within trend and seasonal predictions across time steps. This mechanism maintains constant total signal energy to align with the decomposition-prediction-reconstruction framework and enhance robustness against disturbances. Third, we propose a dynamic capacity adjustment mechanism for channel-independent MLPs. This mechanism scales neuron count with the square root of channel count, ensuring sufficient capacity as channels increase. Extensive experiments across nine benchmark datasets demonstrate that MDMLP-EIA achieves state-of-the-art performance in both prediction accuracy and computational efficiency. Code — https://github.com/zh1985csuccsu/MDMLP-EIA Introduction Time series forecasting (TSF) has extensive applications in air quality monitoring ( 41 ) , traffic flow prediction ( 35 ) , energy management ( 1 ) , economic analysis ( 38 ) , and industrial indicator forecasting ( 37 ) . In recent years, deep learning methods for time series prediction have received widespread research attention ( 23 ) . These methods include RNN-based approaches (e.g., LSTNet ( 11 ) , Witran ( 10 ) , DAN ( 12 ) ), CNN-based methods (e.g., SciNet ( 15 ) , TimesNet ( 26 ) , Moderntcn ( 17 ) ), GNN-based methods (e.g., StemGNN ( 3 ) , FourierGNN ( 33 ) , Msgnet ( 2 ) ), and Transformer-based methods (e.g., Informer ( 42 ) , PatchTST ( 18 ) , Crossformer ( 40 ) , Ada-MSHyper ( 22 ) ). Figure 1: Limitations of current MLP-based methods. (a) Loss of weak seasonal signals: Weak seasonal signals closely resemble noise signals and are difficult to distinguish from them. When applying frequency domain amplitude restrictions to reduce noise, some weak seasonal signals are inevitably lost. (b) Different capacity requirements of channel-independent MLPs for prediction tasks with varying numbers of channels: For prediction tasks with larger numbers of channels (such as Traffic), MLPs require greater capacity to meet the predictive demands of each channel. Conversely, for prediction tasks with fewer channels (such as ETTh1), MLPs require smaller capacity to prevent model overfitting. (c) Reduced Mean Squared Errors (MSEs) achieved by the proposed energy invariant attention at varied prediction lengths in some datasets. These deep learning models achieve good prediction results in certain scenarios, but typically suffer from complex structures, slow training/inference speeds, and reduced robustness due to numerous parameters, particularly with limited training data ( 42 ) . DLinear ( 36 ) addresses these issues by introducing a simple single-layer linear model that achieves Transformer-comparable prediction accuracy with fewer parameters. This success has sparked extensive research into MLP-based time series prediction methods, including FreTS ( 34 ) , CycleNet ( 13 ) , FilterNet ( 32 ) , and Amplifier ( 7 ) . These methods enhance temporal feature representation and improve MLP performance through trend-seasonal decomposition and frequency domain signal processing. However, several limitations still remain: 1) Loss of weak seasonal signal : Softshrink is commonly employed to reduce noise in the frequency domain ( 34 ) . However, this approach may inadvertently remove weak seasonal signals that have low energy but contain valuable cyclical information for time series prediction (see Figure 1 (a)). Although Amplifier ( 7 ) attempts to address the loss of weak seasonal signals by transferring high-energy components from low-frequency to high-frequency regions through spectrum flipping, it simultaneously amplifies unwanted noise along with useful weak seasonal signals. 2) Capacity limitation of weight-sharing MLPs and insufficient channel fusion for channel-independent strategy : The channel-independent strategy employs weight sharing to merge values from different time steps across each channel dimension, and it is considered as a robust prediction strategy with fewer model parameters ( 9 ) . As the channel-independent MLPs need to handle temporal fusion of features across multiple different channels, their capacity is closely related to the fusion performance. However, current channel-independent MLPs usually use fixed capacity, which limits their performance when processing tasks with varying numbers of channels, as shown in Figure 1 (b). Furthermore, the channel-independent strategy may result in insufficient channel fusion for TSF, particularly for popular forecasting methods via seasonal-trend decomposition ( 7 ) ( 25 ) ( 24 ) ( 29 ) ( 43 ) . These methods directly sum trend and seasonal predictions to generate final outputs, potentially limiting the model’s ability to represent different channels across time steps, as shown in Figure 1 (c). To address these issues, we propose MDMLP-EIA with the following main contributions: 1) To effectively capture weak seasonal signals while minimizing noise interference, we design an adaptive fused dual-domain seasonal MLP . We partition seasonal signals into strong and weak components, processing them through frequency-domain MLP and standard MLP respectively. Especially, since both noise and weak seasonal signals exhibit similar low energy characteristics in the frequency domain, we develop an Adaptive Zero-initialized Channel Fusion (AZCF) strategy that selectively integrates strong and weak seasonal predictions while suppressing noise contamination. The idea is different with the spectrum flipping in Amplifier ( 7 ) , which transfers high-energy components from low-frequency to high-frequency regions and unavoidably amplifies unwanted noise along with useful weak seasonal signals. 2) We propose a dynamic capacity adjustment (DCA) mechanism for channel-independent MLPs. This mechanism adjusts neuron count based on the square root of channel numbers, ensuring adequate MLP capacity scaling. For small channel counts, minimal inter-channel redundancy requires rapid capacity growth; for large channel counts, increased redundancy permits slower growth, preventing parameter explosion and overfitting. 3) To address insufficient channel fusion, we introduce an energy invariant attention mechanism (EIA) that adaptively weights trend and seasonal predictions while maintaining theoretical consistency with decomposition frameworks. Unlike conventional weighted fusion that may suffer from energy inconsistency and signal distortion, EIA preserves total signal energy, ensuring stable fusion and enhanced robustness. 4) Nine benchmark experiments achieve state-of-the-art accuracy and efficiency. Related works MLP-based forecasting methods N-BEATS ( 19 ) stacks multiple fully connected layers with forward and backward residuals for univariate TSF. LightTS ( 39 ) applies an MLP-based structure on top of two delicate down-sampling strategies for fast TSF. DEPTS ( 6 ) employs local and seasonal MLP modules to capture local trends and global periodicity in time series, respectively. NHITS ( 4 ) introduces multi-rate data sampling and hierarchical interpolation to focus on time series components of varying frequencies. TimeMixer ( 5 ) uses feature-dimension and time-dimension weight-sharing MLPs for efficient time series prediction with minimal parameters. DLinear ( 36 ) designs a simple single-layer linear model to establish dynamic relationships between input and output time series. SOFTS ( 8 ) employs a

Read the complete text