Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting

Jingru Fei, Kun Yi, Wei Fan, Qi Zhang, Zhendong Niu · 2025

Read Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting by Jingru Fei, Kun Yi, Wei Fan, Qi Zhang, Zhendong Niu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-17216.txt.

Introduction

Jingru Fei and co-authors present Amplifier in a 2025 paper, a time series forecaster that boosts faint parts of the signal. Standard models tend to skip low-energy parts of a series, though those parts can carry key detail.

One block raises the energy of faint parts so the model learns them well. A second block returns the energy to its true level for output. Boosted data show two energy peaks, modeled by a seasonal-trend forecaster that handles each peak apart. An extra block sharpens links across channels. On eight benchmark sets, Amplifier beat top rivals on both accuracy and speed. Forecasters get a fix for a quiet blind spot.

Catalog record

CREATORJingru Fei, Kun Yi, Wei Fan, Qi Zhang, Zhendong Niu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-17216

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Amplifier: Bringing Attention to Neglected Low-Energy Components in Time Series Forecasting Jingru Fei, Kun Yi, Wei Fan, Qi Zhang, Zhendong Niu Abstract We propose an energy amplification technique to address the issue that existing models easily overlook low-energy components in time series forecasting. This technique comprises an energy amplification block and an energy restoration block . The energy amplification block enhances the energy of low-energy components to improve the model’s learning efficiency for these components, while the energy restoration block returns the energy to its original level. Moreover, considering that the energy-amplified data typically displays two distinct energy peaks in the frequency spectrum, we integrate the energy amplification technique with a seasonal-trend forecaster to model the temporal relationships of these two peaks independently, serving as the backbone for our proposed model, Amplifier . Additionally, we propose a semi-channel interaction temporal relationship enhancement block for Amplifier, which enhances the model’s ability to capture temporal relationships from the perspective of the commonality and specificity of each channel in the data. Extensive experiments on eight time series forecasting benchmarks consistently demonstrate our model’s superiority in both effectiveness and efficiency compared to state-of-the-art methods. 1 Beijing Institute of Technology 2 State Information Center of China 3 University of Oxford 4 Tongji University {jingrufei, zniu}@bit.edu.cn, [email protected], [email protected], [email protected] Code — https://github.com/aikunyi/Amplifier 1 Introduction Time series forecasting holds significant importance in real-life applications, encompassing various fields such as financial markets ( Yi et al. 2024d ) , weather forecasting ( Yi et al. 2024a ) , traffic flow prediction ( Yu, Yin, and Zhu 2018 ; Fan et al. 2022 ) , energy planning ( Yi et al. 2024b ) . Recently, the rapid advancement of deep learning has given rise to various models for time series forecasting, including RNN-based methods (e.g., LSTNet ( Lai et al. 2018 ) , DeepAR ( Salinas et al. 2020 ) ), TCN-based methods (e.g., SCINet ( Liu et al. 2022a ) , TimesNet ( Wu et al. 2023 ) ), Transformer-based methods (e.g., PatchTST ( Nie et al. 2023 ) , iTransformer ( Liu et al. 2024 ) ), and Linear-based methods (e.g., DLinear ( Zeng et al. 2023 ) , RLinear ( Li et al. 2023 ) ), etc. Although these deep learning methods have demonstrated competitive performance across various scenarios, they possess several inherent drawbacks in their ability to learn from different energy components, and tend to focus on learning high-energy components while neglecting low-energy components , leading to an incomplete utilization of the informative aspects crucial for time series forecasting. What are low-energy components? In signal processing and analysis, the energy of a signal refers to the total amount of signal strength or power ( Lathi and Green 1998 ) . Low-energy components in a signal refer to frequency components with smaller amplitudes within the frequency spectrum ( Oppenheim 1999 ) . Figure 1: Analysis about low-energy components in time series forecasting. (a) Indispensability : Discarding low-energy components results in an increased MSE value. (b) Dependence on energy magnitude : The components that are ignored are consistently those with low energy, irrespective of their position within the frequency bands. Low-energy components are indispensable. In contexts like signal compression or model simplification, low-energy components might be dismissed as unimportant or treated as noise. However, in scenarios where subtle changes or background patterns are significant, low-energy components can hold critical details. For example, in weather forecasting, small meteorological shifts can accumulate over time, leading to dramatic changes in weather patterns. In financial markets, minor fluctuations can trigger larger trends or volatility, with high-frequency trading systems often capitalizing on these small changes to execute profitable trades. We also have empirically validated the indispensability of low-energy components in time series forecasting. As illustrated in Figure 1 (a), directly filtering out low-energy components leads to increased MSE values across different types of models (PatchTST, RLinear, DLinear, FreTS), highlighting the crucial role of low-energy components in enhancing prediction accuracy. Low-energy components are often overlooked. We generate two signals with different energy distributions (corresponding to the left and right of Figure 1 (b)), and conduct iTransformer ( Liu et al. 2024 ) , PatchTST ( Nie et al. 2023 ) , and DLinear ( Zeng et al. 2023 ) on them respectively. As detailed in Figure 1 (b), the phenomenon of ignorance (rf. small green circles) always occurs with low-energy components, regardless of the frequency band in which they are located. In this paper, to address the issue of low-energy components being neglected, we propose an Energy Amplification Technique , which comprises an energy amplification block and an energy restoration block . The energy amplification block is designed to boost the energy of low-energy components, enhancing the model’s learning efficiency for these components, while the energy restoration block brings the energy back to its original level. This technique can be applied as a general approach in other time series forecasting models to enhance their performance. To better harness the potential of the energy amplification technique, we design a new model, Amplifier , based on the characteristics of the data after energy amplification. Specifically, since the data processed by the energy amplification block exhibits two energy peaks in the spectrum, it is crucial to separate these two peaks to prevent them from simultaneously dispersing the attention of a single module or layer. Thankfully, the trend and seasonal components obtained from the seasonal-trend decomposition (abbreviated as STD) correspond directly to these two energy peaks. And the widespread use of STD reflects its strong acceptance among researchers. Thus we integrate the energy amplification technique with a seasonal-trend forecaster (based on STD) to model the temporal relationships for these two peaks separately, forming the backbone of Amplifier. To further improve information utilization, we also develop a semi-channel interaction temporal relationship enhancement block (abbreviated as SCI block) as an optional built-in component for Amplifier to enhance the performance from the perspective of the commonality and specificity of each channel in time series data. Our contributions can be summarized as follows: • We identify a common issue in existing time series forecasting models, i.e., the neglect of low-energy components, which can lead to performance degradation. • We propose an energy amplification technique to address the above issue, serving as a general method to enhance the performance of other foundational models. • To better leverage the energy amplification technique, we combine this technique with a temporal relationship enhancement and a seasonal-trend forecaster to propose a novel model for time series forecasting, called Amplifier. • Extensive experiments on 8 real-world datasets demonstrate that our Amplifier consistently outperforms state-of-the-art methods while offering higher efficiency. 2 Related Work Deep Time Series Forecasting With the rapid advancement of deep learning technology and the increasing importance of time series forecasting, various models have emerged in a flourishing and competitive landscape ( Lim and Zohren 2021 ) . Recent studies have proposed a series of upgraded transformer-based models for time series forecasting, such as LogTrans ( Li et al. 2019 ) , Informer ( Zhou et al. 2021 ) , and Pyraformer ( Liu et al. 2021 ) . Meanwhile, some models focus on functional improvements to the Transformer architecture, making it more suitable for time

Read the complete text