FilterNet: Harnessing Frequency Filters for Time Series Forecasting
Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, Wei Fan · 2024
Read FilterNet: Harnessing Frequency Filters for Time Series Forecasting by Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, Wei Fan (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01623.txt.
Description
Abstract Given the ubiquitous presence of time series data across various domains, precise forecasting of time series holds significant importance and finds widespread real-world applications such as energy, weather, healthcare, etc. While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting.
Catalog record
| CREATOR | Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, Wei Fan |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01623
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FilterNet: Harnessing Frequency Filters for Time Series Forecasting Kun Yi, Jingru Fei, Qi Zhang, Hui He, Shufeng Hao, Defu Lian, Wei Fan Abstract Given the ubiquitous presence of time series data across various domains, precise forecasting of time series holds significant importance and finds widespread real-world applications such as energy, weather, healthcare, etc. While numerous forecasters have been proposed using different network architectures, the Transformer-based models have state-of-the-art performance in time series forecasting. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series with thousands of points. In this paper, we explore a novel perspective of enlightening signal processing for deep time series forecasting. Inspired by the filtering process, we introduce one simple yet effective network, namely FilterNet , built upon our proposed learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals. Concretely, we propose two kinds of learnable filters in the FilterNet: (i) Plain shaping filter, that adopts a universal frequency kernel for signal filtering and temporal modeling; (ii) Contextual shaping filter, that utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. Equipped with the two filters, FilterNet can approximately surrogate the linear and attention mappings widely adopted in time series literature, while enjoying superb abilities in handling high-frequency noises and utilizing the whole frequency spectrum that is beneficial for forecasting. Finally, we conduct extensive experiments on eight time series forecasting benchmarks, and experimental results have demonstrated our superior performance in terms of both effectiveness and efficiency compared with state-of-the-art methods. Our code is available at 1 1 1 https://github.com/aikunyi/FilterNet . 1 Introduction Time series forecasting has been playing a pivotal role across a multitude of contemporary applications, spanning diverse domains such as climate analysis WuZLW23 , energy production deepcn_2024 , traffic flow patterns catn_2022 , financial markets ariyo2014stock , and various industrial systems mehdiyev2017time . The ubiquity and profound significance of time series data has recently garnered substantial research efforts, culminating in a plethora of deep learning forecasting models lim2021time that have significantly enhanced the domain of time series forecasting. Previously, leveraging different kinds of deep neural networks derives a series of time series forecasting methods, such as Recurrent Neural Network-based methods including DeepAR salinas2020deepar , LSTNet lstnet_2018 , Convolution Neural Network-based methods including TCN bai2018 , SCINet liu2022scinet , etc. Recently, however, with the continuing advancement of deep learning, two branches of methods that received particularly more attention have been steering the development of time series forecasting, i.e., Multilayer Perceptron (MLP)-based methods, such as N-BEATS oreshkin2019n , DLinear dlinear_2023 , and FreTS frets_23 , and Transformer-based methods, such as Informer informer_2021 , Autoformer autoformer_21 , PatchTST patchtst_2023 , and iTransformer itransformer_2024 . While MLP-based models are capable of providing accurate forecasts, Transformer-based models continue to achieve state-of-the-art time series forecasting performance. However, forecasters based on Transformers are still suffering from vulnerability to high-frequency signals, efficiency in computation, and bottleneck in full-spectrum utilization, which essentially are the cornerstones for accurately predicting time series composed of thousands of timesteps. In designing a very simple simulation experiment on the synthetic data only composed of a low-, middle- and high-frequency signal respectively (see Figure 1(a) ), we find the state-of-the-art iTransformer itransformer_2024 model performs much worse in forecasting (Figure 1(b) and Figure 1(c) ). This observation shows that state-of-the-art Transformer-based model cannot utilize the full spectrum information, even for a naive signal of three different frequency components. In contrast, in the field of signal processing , a frequency filter enjoys many good properties such as frequency selectivity, signal conditioning, and multi-rate processing. These could have great potential in advancing the model’s ability to extract key informative frequency patterns in time series forecasting. Thus, inspired by the filtering process asselin1972frequency in signal processing, in this paper, we introduce one simple yet effective framework, namely FilterNet , for effective time series forecasting. Specifically, we start by proposing two kinds of learnable filters as the key units in our framework: (i) Plain shaping filter, which makes the naive but universal frequency kernel learnable for signal filtering and temporal modeling, and (ii) Contextual shaping filter, which utilizes filtered frequencies examined in terms of its compatibility with input signals for dependency learning. The plain shaping filter is more likely to be adopted in predefined conditions and efficient in handling simple time series structures, while the contextual filter can adaptively weight the filtering process based on the changing conditions of input and thus have more flexibility in facing more complex situations. Besides, these two filters as the built-in functions of the FilterNet can also approximately surrogate the widely adopted linear mappings and attention mappings in time series literature dlinear_2023 ; informer_2021 ; itransformer_2024 . This also illustrates the effectiveness of our FilterNet in forecasting by selectively passing or attenuating certain signal components while capturing the core time series structure with adequate learning expressiveness. Moreover, since filters are better fit in the stationary frequency domain, we let filters wrapped by two reversible transformations, i.e., instance normalization revin_2021 and fast Fourier transform duhamel1990fast to reduce the influence of non-stationarity and accomplish the domain transformation of time series respectively. In summary, our contributions can be listed as follows: • In studying state-of-the-art deep Transformer-based time series forecasting models, an interesting observation from a simple simulation experiment motivates us to explore a novel perspective of enlightening signal processing techniques for deep time series forecasting. • Inspired by the filtering process in signal processing, we introduce a simple yet effective network, FilterNet , built upon our proposed two learnable frequency filters to extract key informative temporal patterns by selectively passing or attenuating certain components of time series signals, thereby enhancing the forecasting performance. • We conduct extensive experiments on eight time series forecasting benchmarks, and the results have demonstrated that our model achieves superior performance compared with state-of-the-art forecasting algorithms in terms of effectiveness and efficiency. (a) The spectrum of input signal (b) iTransformer with MSE=1.1e-01 (c) FilterNet with MSE=2.7e-05 Figure 1: Performance of Mean Squared Error (MSE) on a simple synthetic multi-frequency signal. More details about the experimental settings can be found in Appendix C.4 . 2 Related Work Deep Learning-based Time Series Forecasting In recent years, deep learning-based methods have gained prominence in time series forecasting due to their ability to capture nonlinear and complex correlations Lim_2021 . These methods have employed various network architectures to learn temporal dependencies, such as RNN lstnet_2018 ; salinas2020deepar , TCN bai2018 ; liu2022scinet , etc. Notably, MLP- and Transformer-based methods have achieved competitive performance, emerging as state-of-the-art approaches. N-HiTS nhits_2022 integrates multi-rate input sampling and hierarchical interpolation with MLPs to enhance univariate forecasting. DLinear dlinear_2023 introduces a simple approach using a single-layer linear model to capture temporal relationships between input and output time series data. RLinear rlinear_2023 utilizes linear mapping to model periodic features, demonstrating robustness across diverse periods with increasing input length. In contrast to the simple structure of MLPs, Transformer’s advanced attention mechanisms VaswaniSPUJGKP17 empower the models informer_2021 ; autoformer_21 ; fedformer_22 ; liu2021pyraformer to capture intricate dependencies and long-range interactions. PatchTST patchtst_2023