Efficient Time Series Forecasting via Hyper-Complex Models and Frequency Aggregation Eyal Yakir, Dor Tsur, Haim Permuter Abstract Time series forecasting is a long-standing problem in statistics and machine learning. One of the key challenges is processing sequences with long-range dependencies. To that end, a recent line of work applied the short-time Fourier transform (STFT), which partitions the sequence into multiple subsequences and applies a Fourier transform to each separately. We propose the Frequency Information Aggregation (FIA)-Net, which is based on a novel complex-valued MLP architecture that aggregates adjacent window information in the frequency domain. To further increase the receptive field of the FIA-Net, we treat the set of windows as hyper-complex (HC) valued vectors and employ HC algebra to efficiently combine information from all STFT windows altogether. Using the HC-MLP backbone allows for improved handling of sequences with long-term dependence. Furthermore, due to the nature of HC operations, the HC-MLP uses up to three times fewer parameters than the equivalent standard window aggregation method. We evaluate the FIA-Net on various time-series benchmarks and show that the proposed methodologies outperform existing state of the art methods in terms of both accuracy and efficiency. Our code is publicly available on https://anonymous.4open.science/r/research-1803/ . 1 Introduction Time series forecasting (TSF) is a long standing challenge, which plays a key role in various domains, such as energy management [ 1 ] , traffic prediction [ 2 ] and financial analysis [ 3 ] . With the development of deep learning, myriad neural network (NN) architectures had been proposed, and have gradually improved the accuracy on the TSF problem. Two key architectures has been used for TSF are recurrent NNs (RNNs) [ 4 , 5 , 6 ] and transformers [ 7 , 8 , 9 , 10 ] , each of which aims to capture long-term dependencies through a different functional feature extraction procedure. While both methods were proven useful, RNNs struggled with long-term dependencies [ 11 ] or non-stationary data patterns, While transformer architectures may overlook important temporal information due to permutation invariance [ 12 ] , they require many parameters and may suffer from long runtime. Additional NN-based approaches for TSF consider graph NNs (GNNs) [ 13 ] and decomposition models [ 14 ] . Recent advancements have demonstrated promising results in processing and extracting features from the frequency domain [ 15 ] . Techniques leveraging frequency-based transformations have been in various contexts, ranging from computational efficiency improvements [ 9 ] to seasonal-trend decomposition [ 8 ] . To better process the frequency domain data, [ 16 ] developed a complex-valued MLP, which demonstrated superior capability in capturing both temporal and cross-channel dependencies. To better handle nonstationarities in the data, [ 17 ] substituted the standard FFT with the Short-Time Fourier Transform (STFT) [ 18 ] , which divides the sequence into separate windows and transforms each window individually into the frequency domain. While showing better suitability for non-stationary time series data, the STFT yields a set of windows, each of which represents exclusive information on the sequence. However, in practice, adjacent windows are highly correlated, albeit processed separately by current STFT-based models. To incorporate the overlooked shared information, we propose the FIA-Net, a novel TSF model that is designed to handle long-term dependencies in the data by aggregating information from subsets of STFT windows. The FIA-Net has an MLP backbone that processes the STFT windows in the frequency domain. We propose two novel MLP architectures. The first, is termed window-mixing MLP (WM-MLP), which mixes each STFT windows with its neighboring bands. The second is the HC-MLP. The HC-MLP leverages HC algebra to efficiently combine information from all STFT together. By using HC algebra, the STFT is implements with three times less parameters than the equivalent WM-MLP. The main contributions of this paper are as follows • We construct the FIA-Net and the WM-MLP backbone. The resulting TSF model captures inter-window dependencies in the frequency domain and benefits from a forward pass complexity of O(L\log L/p) operations, where L is the lookback window length and p is the number of STFT windows. • We propose a novel HC-MLP backbone that expands the receptive field of the WM-MLP, while requiring a fraction of total parameters. • To reduce the model size and complexity, we filter the STFT windows, leaving only the top- M frequency components. We show that accuracy is maintained even when M is significantly smaller than the total number of components. • We provide an array of experiments that demonstrate the performance of the model and its efficiency. We show that the FeeqShiftNet improves upon existing models accuracy by up to 20\% . • We provide an ablation study, in which explores the effect of operating over the complex plane and compare the performance of the two considered MLP backbones. 2 Related Work Time-Series Forecasting The first notable works on TSF utilize classical statistical linear models such as ARIMA [ 19 , 20 ] which consider series decomposition. Those were then generalized to a non-linear setting in [ 21 ] . To overcome the limitations posed by the classical models, deep learning was incorporated, where initially, sequential deep learning was performed by RNN-based models. Two key RNN models are long-short term memory networks [ 5 ] which introduce a sophisticated gating mechanism and the DeepAR model [ 4 ] that connected the RNN model with AR modeling. Despite their expressive power for sequential modeling, RNN demonstrated low efficiency and introduced high runtimes in both the forward and backward pass [ 11 ] . Two popular architectures were proposed to improve upon RNNs; transformers and GNNs. Notable transformer-based methods are Informer [ 22 ] , Reformer [ 23 ] , and PatchTST [ 24 ] , each leveraging the attention mechanism to capture temporal dependencies, while proposing sophisticated methods to reduce the attention operation complexity. GNNs, however, allowed for better modeling of dependencies between time series variables by treating them as graph nodes, making them particularly suitable for capturing spatio-temporal patterns. For example, AGCRN [ 25 ] introduced an adaptive graph convolution mechanism to dynamically adjust the graph structure based on inter-series relationships, while MTGNN [ 13 ] combined graph convolutions with temporal convolutional layers to jointly learn spatial-temporal dependencies. Frequency Domain Models for Time Series Forecasting A recent line of work attempts to solve the TFS problem in the frequency domain [ 15 ] , with the purpose of revealing patterns that may be hidden in the time domain. The FEDformer [ 8 ] uses a Fourier-based framework to separate trend and seasonal components by leveraging the Fourier Transform on sub-sequences, allowing it to isolate periodic patterns more effectively. ETSformer [ 26 ] combines exponential smoothing and applies attention in the frequency domain to enhance seasonality modeling by capturing both short- and long-term dependencies. In FiLM [ 27 ] , Fourier projections are used to reduce noise and emphasize relevant features. Additionally, SFM [ 28 ] and StemGNN [ 29 ] utilize frequency decomposition and Graph Fourier Transforms to handle complex temporal dependencies in multivariate time series. FRETS [ 16 ] extends this approach by proposing frequency-domain MLPs to learn complex relationships between real and imaginary components of the FFT. FREQTSF [ 17 ] uses STFT with attention mechanisms to capture temporal patterns across overlapping time windows. While frequency models, and specifically the recent use of STFT, have shown significant improvement in TFS performance, each STFT window is often processed separately, ignoring the strong correlations between adjacent windows. Hyper-complex Numbers HC numbers extend the complex number system to higher dimensions [ 30 ] . Base- 4 HC numbers, have been widely used in computer graphics to model 3D rotations [ 31 ] . Base- 8 HC numbers have been explored in image classification and compression [ 31 , 32 ] , developing an HC network that showed favorable performance on popular datasets. The merit of HC numbers to extract relevant information in time-series was explored in [ 33 ] , in which an HC-net was used to analyse brain-wave data, and in [ 34 ] , which explored HC-network for financial data. In this work, we explore the utility of HC architectures for the efficient processing of STFT windows in the frequency domain. Figure 1 : Window Mixing mechanism. An input X is transformed into a set of p STFT windows which are transformed to the frequency domain and are then fed into the WM-MLP, which aggregates adjacent windows. The WM-MLP outputs are then transformed back to the time domain via a real STFT, from which the prediction (red) is obtained. 3 Proposed Model : FIA-Net In this section, we describe FIA-Net, a TSF model that leverages shared information between STFT windows. We begin by discussing the existing gap in current frequency domain TSF methods, followed by a brief introduction to frequency domain MLPs [ 35 ] . We then outline the FIA-Net components, presenting the novel complex MLP backbone, discussing a simple frequency compression step that reduces the MLP input dimension, and outline the complete model. Motivation Even though most real-world time-series data is nonstationary, it may adhere to a piecewise stationary structure, as observed in speech signals [ 36 ] and financial data [ 37 ] . This local stationarity allows us to partition the series into stationary correlated STFT subsequences that can be transformed in the frequency domain. The correlation between the STFT sequences has been efficiently utilized in recent works, even though, as we later show, it affects the downstream model accuracy in the task of time prediction. Figure 2 : FD-MLP architecture. Frequency Domain MLPs As we handle complex-valued data, we adopt the frequency domain MLP (FD-MLP) unit from [ 16 ] . The FD-MLP generalizes the simple neuron to operate with complex-valued weights and biases. Incorporating complex MLPs has been shown to improve the model performance as it aligns better with the geometrical structure induced by the complex plane. The FD-MLP unit is visualized in Figure 2 . In Section 4 , we will discuss the expansion of the FD-MLP for hyper-complex numbers. 3.1 Adjacent Information Aggregation Consider a sequence X=\{x_{1},\ldots,x_{L}\}\in\mathbb{R}^{D\times L} where x_{i}\in\mathbb{R}^{D} , L is the sequence length, which we refer to as the lookback size, and D is the latent space dimension. Our objective is to predict the next T elements of the sequence \hat{X}=\{\hat{x}_{L+1},\ldots,\hat{x}_{L+T}\}\in\mathbb{R}^{D\times T} , where T is a predetermined prediction horizon. We are interested in processing X in the frequency domain. We utilize the STFT, which partitions X into p windows and applies the FFT separately to each window. In addition, we exploit the real-valued inputs to perform a Real STFT, which results in half the frequency coefficients. The STFT for the i -th window is defined as: \mathsf{STFT}\{X\}(\omega,\tau_{i})=\sum_{t=1}^{L}x_{t}w(t-\tau_{i})e^{-j\omega t}, (1) Where, w(t-\tau_{i}) is the window function centered at the location of the i -th window ( i\in\{1,\dots,p }), \omega represents the angular frequency, and j satisfies j^{2}=-1 . Each window is defined by its center \tau_{i} and has a size of \frac{N_{\mathsf{FFT}}}{2}+1 . The output of the STFT consists of p windows, each producing a spectrum of length \frac{N_{\mathsf{FFT}}}{2}+1 . We propose the window mixing MLP (WM-MLP), which adapts the FD-MLP to properly aggregate neighboring STFT windows to incorporate shared information. Given a set of complex transformed windows \{C_{1},\dots,C_{p}\} , the WM-MLP operates on the i th window C^{\mathsf{in}}_{i} as follows: C^{\mathsf{out}}_{i}=\sigma\left(C^{\mathsf{in}}_{i}W_{i\to i}+C^{\mathsf{in}}_{i-1}\overline{W}_{(i-1)\to i}+C^{\mathsf{in}}_{i+1}\overline{W}_{(i+1)\to i}+B_{i}\right) (2) where \sigma(\cdot) is an activation function, (W_{(i-1)\to i},W_{i\to i},W_{(i+1)\to i})_{i=1}^{p} are the WM-MLP weight matrices with C_{j} being a matrix of zeros for j\notin\{1,\dots,p\} , and (B_{i})_{i=1}^{p} are the WM-MLP bias vectors, and \overline{W} is the elementwise complex conjugate of W . The outputs of the WM-MLP are transformed back to the time domain using the element-wise inverse STFT, which is given by: \mathsf{iSTFT}\{X^{F}(w,\tau_{i})\}(t)=\sum_{\omega}X^{F}(\omega,\tau)e^{j\omega t}w(t-\tau_{i}) (3) The STFT, WM-MLP operation, and inverse transform are depicted by Figure 1 . In highly nonstationary data, energy transition between adjacent windows can be sharp. To that end, we introduce a minor overlap between adjacent windows of N_{FFT}-\frac{L-N_{FFT}}{p-1} , which implicitly adjusts their statistics prior to processing by the TSF model by increasing the inter-window correlations. Figure 3 : FIA-Net Model: The input, denoted X , is first fed into the embedding layer, resulting in X_{E} , which is transformed to the frequency domain via the STFT. We then extract the top- M components of each STFT window and feed the compressed windows through the WM-MLP. The MLP outputs are then passed through position-aware zero padding, whose outputs are transformed back to the time domain and summed with X_{E} via skip connection. The model output \hat{X} is then given by applying a linear transformation. 3.2 Implementation Details and Complete System Selective Frequency Compression To reduce the input dimensionality to the WM-MLP, we compress each transformed window C_{i}\in\mathbb{C}^{N_{\mathsf{FFT}}\times D} along the frequency axis. Specifically, we select the top M frequency components based on their real and imaginary values across each dimension and denote the compressed window with C_{i}^{M} . Then, (C_{1}^{M},\dots,C_{p}^{M}) is fed into the WM-MLP layer. The top- M procedure is given by C^{M}_{i}=\underset{j=1,\dots,M}{\text{Top-M}}|C_{i,j}|_{\mathbb{C}} (4) where C_{i,j} is the j th component of C_{i} and |z|_{\mathbb{C}} is the magnitude of z\in\mathbb{C} . Additionally, we store the top component indices of ( 4 ) in a list \mathcal{I}(i) , which encodes the band from which the information came. To transform the WM-MLP output C^{\mathsf{out}}_{i} back to the time domain, we perform a position-aware zero padding, which adds N_{\mathsf{FFT}}-M zeros while placing the nonzero components in their original indices, which correspond to the original frequency bands, i.e., C_{i,j}^{\text{padded}}=\begin{cases}C^{\mathsf{out}}_{i,j},\quad j\in\mathcal{I}(i)\\ 0,\quad\qquad\text{else.}\end{cases} In Section 5 , we demonstrate that, in addition to improving computational efficiency, this frequency compression procedure enhances the performance of downstream TSF tasks. The selection of top- M components allows us to reduce the model’s complexity while maintaining the most relevant frequency information. Complete Model The complete FIA-Net, as shown in Figure 3 , operates as follows: Given an input X\in\mathbb{R}^{B\times L\times D} , the dimension of X is expanded through a learned embedding layer, resulting in X_{E}\in\mathbb{R}^{B\times L\times D\times E} . This expanded representation is then fed into an STFT block that uses the real input to perform \mathsf{R-STFT} on X_{E} . The transformed signal is passed through the SM block, whose output is further processed by the WM-MLP. The WM-MLP outputs are subsequently padded and transformed back to the temporal axis, where they are integrated with X_{E} via a skip connection and resized to the desired output sequence shape using a two-layer MLP decomposition. Model Complexity The forward pass complexity of the WM-MLP is primarily determined by the STFT complexity, which is O(L\log(\frac{L}{p})) . This represents a significant reduction in complexity compared to transformer-based methods, which employ intricate mechanisms to reduce their O(L^{2}) attention complexity to O(L\log L) . Additionally, the application of top- M frequency selection further optimizes the forward pass in the frequency domain, reducing both computational demands and the corresponding MLP size. A detailed analysis of these complexities is provided in Table 11 . 4 Window Aggregation via Hyper-Complex Models Even though the WM-MLP backbone integrates valuable information that benefits the FIA-Net’s accuracy, information is not only shared between two adjacent STFT windows. In fact, the stronger the dependencies on the long-term past, the more information is shared between two distant windows on the frequency axis. Ideally, we would like to aggregate information between all p STFT windows. Unfortunately, a straightforward extension of the WM-MLP requires O(p^{2}) weight matrices, which may impair the training procedure and increase model complexity. To address that, we interpret the set of windows as an HC vector and propose an HC-based MLP that efficiently processes the set of STFT windows. We begin with a short introduction on HC-algebras, followed by the construction of the proposed MLP backbone for the FIA-Net. 4.1 Hyper-complex Numbers HC numbers generalize the complex field by introducing additional dimensions while maintaining algebraic properties. HC number systems are defined by a parameter q that determines the number of components in the number system. Complex numbers can thus be viewed as an HC number with q=2 , and an HC number of base q can be represented with p=q/2 complex numbers. In what follows, we focus on HC numbers with p=4 , termed Octonions \mathbb{O} , whose elements are denoted o=(\alpha_{1},\alpha_{2},\alpha_{3},\alpha_{4})\in\mathbb{O} , with \alpha_{i}\in\mathbb{C} for i=1,\dots,4 . Additional discussion on p\neq 4 is given in Appendix C . The addition of two Octonions, o_{1}=(\alpha_{1},\dots,\alpha_{4}) and o_{2}=(\beta_{1},\dots,\beta_{4}) , is given by their componentwise sum, while their multiplication follows the Cayley-Dickson construction [ 38 ] . The product o_{3}=o_{1}\cdot o_{2}=(\gamma_{1},\gamma_{2},\gamma_{3},\gamma_{4}) is given by: \displaystyle\gamma_{1}=\alpha_{1}\beta_{1}-\alpha_{2}\overline{\beta_{2}}-\alpha_{3}\overline{\beta_{3}}-\alpha_{4}\overline{\beta_{4}} (5) \displaystyle\gamma_{2}=\alpha_{2}\overline{\beta}_{1}+\alpha_{1}\beta_{2}+\alpha_{3}\overline{\beta_{4}}-\alpha_{4}\overline{\beta_{3}} \displaystyle\gamma_{3}=\alpha_{3}\overline{\beta}_{1}+\alpha_{4}\overline{\beta}_{2}+\alpha_{1}\beta_{3}-\alpha_{2}\overline{\beta}_{4} \displaystyle\gamma_{4}=\alpha_{4}\overline{\beta}_{1}+\alpha_{2}\overline{\beta}_{3}+\alpha_{1}\beta_{4}-\alpha_{3}\overline{\beta}_{2} Hyper-complex numbers exhibit additional properties such as closed-form expressions for norm calculations and norm preservation for specific bases. For completeness, we provide additional information on HC-numbers in Appendix C , where the proposed MLP is presented under specific bases. 4.2 Hyper-Complex MLP Figure 4 : HC-MLP operating on C^{\mathsf{in}}=(C^{\mathsf{in}}_{1},C^{\mathsf{in}}_{2},C^{\mathsf{in}}_{3},C^{\mathsf{in}}_{4}) , implementing the HC multiplication (( 6 )). Each output unit is the sum of the corresponding inner blocks of the same color, where a \bigoplus symbol denotes complex addition and a \bigotimes denotes complex multiplication. A red outline denotes minus multiplication, and a blue input arrow denotes complex conjugation. The longer the range of temporal dependencies in the data, the more shared information there is between gathered windows. In such cases, the WM-MLP, which incorporates short-term information in the frequency domain, might fail to capture long-term dependencies. To that end, our goal is to increase the extent to which information is shared across the STFT windows. To derive a parameter-efficient solution, we incorporate HC algebra into the frequency domain learning procedure. Assume that we are given p=4 complex-valued STFT windows (C^{\mathsf{in}}_{i}\in\mathbb{C}^{B\times M\times E})_{i=1}^{4} , where the second axis is the transformed frequency domain after top- M frequency component selection. We treat the set of windows as a single Octonion tensor (C^{\mathsf{in}}_{1},C^{\mathsf{in}}_{2},C^{\mathsf{in}}_{3},C^{\mathsf{in}}_{4})\in\mathbb{O}^{B\times M\times E} and feed it through an HC-valued MLP, whose output is C^{\mathsf{out}}=\sigma(C^{\mathsf{in}}\cdot W+B) . For C^{\mathsf{out}}=(C^{\mathsf{out}}_{1},C^{\mathsf{out}}_{2},C^{\mathsf{out}}_{3},C^{\mathsf{out}}_{4}) , it is given by: \displaystyle C^{\mathsf{out}}_{1} \displaystyle=\sigma(C^{\mathsf{in}}_{1}W_{1}-C^{\mathsf{in}}_{2}\overline{W}_{2}-C^{\mathsf{in}}_{3}\overline{W}_{3}-C^{\mathsf{in}}_{4}\overline{W}_{4}+{B}_{1}), (6) \displaystyle C^{\mathsf{out}}_{2} \displaystyle=\sigma(C^{\mathsf{in}}_{2}\overline{W}_{1}+C^{\mathsf{in}}_{1}W_{2}-C^{\mathsf{in}}_{4}\overline{W}_{3}+C^{\mathsf{in}}_{3}\overline{W}_{4}+{B}_{2}), \displaystyle C^{\mathsf{out}}_{3} \displaystyle=\sigma(C^{\mathsf{in}}_{3}W_{1}+C^{\mathsf{in}}_{1}\overline{W}_{3}-C^{\mathsf{in}}_{2}\overline{W}_{4}+C^{\mathsf{in}}_{4}\overline{W}_{2}+{B}_{3}), \displaystyle C^{\mathsf{out}}_{4} \displaystyle=\sigma(C^{\mathsf{in}}_{4}\overline{W}_{1}+C^{\mathsf{in}}_{1}W_{4}-C^{\mathsf{in}}_{3}\overline{W}_{2}+C^{\mathsf{in}}_{2}\overline{W}_{3}+{B}_{4}). where W=(W_{1},\dots,W_{4})\in\mathbb{O}^{E\times E} , B=(B_{1},\dots,B_{4})\in\mathbb{O}^{E\times 1} are the HC-MLP weights and bias, respectively, and \sigma is a standard activation function, e.g., ReLU. We stress that, as considered in the complex MLP from [ 16 ] , the HC-MLP is implemented with real-valued operations, which allows it to plug into every existing automatic differentiation scheme over standard GPUs. The HC-MLP unit is depicted in Figure 4 . The WM-MLP demonstrates distinct advantages depending on the prediction horizon. For shorter prediction lengths, it achieves better performance by effectively leveraging all available information from adjacent and nearby windows. In contrast, for longer horizons, where only closer temporal information remains relevant, the WM-MLP’s ability to aggregate adjusted windows proves to be more effective. This behavior is clearly demonstrated in Section 5.2 . Moreover, the HC perspective offers a significant advantage in terms of parameter efficiency. It allows for an implementation with only p weight matrices, whereas the corresponding WM-MLP would require 3p-2 weight matrices (and even p^{2} weight matrices for a generalization of the WM-MLP), all while preserving performance. This reduction in parameters becomes increasingly dramatic as p>4 , as further detailed in Appendix C . 5 Results and Discussion 5.1 Experimental Setting Datasets Following [ 8 , 16 ] , we consider the following representative real-world datasets: 1) WTH (Weather), 2) Exchange (Finance), 3) Traffic , 4) ECL (Electricity), 5) ETTh1 (Electricity transformer temperature hourly), and 6) ETTm1 (Electricity transformer temperature minutely). The train/validation/test split is 70%, 15%, and 15%, respectively. Baselines In this research, we followed the TSF SoTA baselines: 1) FedFormer [ 8 ] , 2) Reformer [ 23 ] , 3) FreTS [ 16 ] , 4) PatchTST [ 24 ] , 5) Informer [ 22 ] , 6) Autoformer [ 9 ] and 7) LSTF-Linear [ 39 ] . Experiments setup All experiments were conducted using PyTorch [ 40 ] on a single RTX 3090, utilizing mean squared error (MSE) loss and the Adam optimizer [ 41 ] . We established an initial learning rate of 10^{-3} with an exponential decay scheduler. Hyperparameters were optimized individually for each dataset (see Appendix B.3 for specific details). We report performance metrics under both root mean squared error (RMSE) and mean absolute error (MAE). Additional information on the Normalization B.5 , datasets B.1 , and baseline models B.2 can be found in the appendix. 5.2 Main results Table 1 compares the FIA-Net performance under both the WM-MLP and the HC-MLP backbones with the SoTA baselines. It is evident that the FIA-Net consistently outperforms the baselines on most considered values of prediction horizon T , with an average improvement of 5.4\% in MAE and 3.8\% in RMSE over SoTA models. We note that the performance of the HC-MLP-based network, which is implemented with significantly fewer parameters, achieves comparable results with the corresponding WM-MLP and attains the best results over several settings. We can deduce that the HC-MLP is more suitable for shorter-term prediction, while the WM-MLP backbone is more suitable for longer ranges. Table 1 : Forecasting performance comparison across datasets and prediction horizons using RMSE and MAE. Lower values indicate better performance. Bold denotes the best results, and underlined indicates the second-best. Weather Exchange Traffic Electricity ETTh1 ETTm1 Metric 96 192 336 720 96 192 336 720 96 192 336 720 96 192 336 720 96 192 336 720 96 192 336 720 HC-MLP (Ours) RMSE 0.069 0.079 0.090 0.098 0.050 0.062 0.078 0.112 0.032 0.034 0.035 0.036 0.070 0.068 0.071 0.077 0.083 0.085 0.094 0.101 0.074 0.082 0.089 0.096 MAE 0.030 0.039 0.043 0.054 0.035 0.049 0.061 0.089 0.016 0.017 0.017 0.018 0.040 0.041 0.044 0.049 0.057 0.064 0.068 0.075 0.049 0.056 0.060 0.067 WM-MLP (Ours) RMSE 0.071 0.081 0.089 0.097 0.048 0.060 0.076 0.107 0.033 0.033 0.034 0.036 0.067 0.068 0.070 0.076 0.084 0.088 0.097 0.102 0.076 0.082 0.089 0.094 MAE 0.031 0.041 0.045 0.053 0.034 0.047 0.058 0.086 0.016 0.016 0.016 0.018 0.039 0.041 0.044 0.049 0.057 0.066 0.071 0.075 0.052 0.055 0.058 0.064 FreTS RMSE 0.071 0.081 0.090 0.099 0.051 0.067 0.082 0.110 0.036 0.038 0.038 0.039 0.065 0.064 0.072 0.079 0.087 0.091 0.096 0.108 0.077 0.083 0.089 0.096 MAE 0.032 0.040 0.046 0.055 0.037 0.050 0.062 0.088 0.018 0.020 0.019 0.020 0.039 0.040 0.046 0.052 0.061 0.065 0.07 0.082 0.052 0.057 0.062 0.069 PatchTST RMSE 0.074 0.084 0.094 0.102 0.052 0.074 0.093 0.166 0.032 0.035 0.039 0.040 0.067 0.066 0.067 0.081 0.091 0.094 0.099 0.113 0.082 0.085 0.091 0.097 MAE 0.034 0.042 0.049 0.056 0.039 0.055 0.071 0.132 0.016 0.018 0.020 0.021 0.041 0.042 0.043 0.055 0.065 0.069 0.073 0.087 0.055 0.059 0.064 0.070 LTSF-Linear RMSE 0.081 0.089 0.098 0.106 0.052 0.069 0.085 0.116 0.039 0.042 0.040 0.041 0.075 0.070 0.071 0.080 0.089 0.094 0.097 0.108 0.080 0.087 0.093 0.099 MAE 0.040 0.048 0.056 0.065 0.038 0.053 0.064 0.092 0.020 0.022 0.020 0.021 0.045 0.043 0.044 0.054 0.063 0.067 0.070 0.082 0.055 0.060 0.065 0.072 FEDformer RMSE 0.088 0.092 0.101 0.109 0.067 0.082 0.105 0.183 0.036 0.042 0.042 0.042 0.072 0.072 0.075 0.077 0.096 0.100 0.105 0.116 0.087 0.093 0.102 0.108 MAE 0.050 0.051 0.057 0.064 0.050 0.064 0.080 0.151 0.022 0.023 0.022 0.022 0.049 0.049 0.051 0.055 0.072 0.076 0.080 0.090 0.063 0.068 0.075 0.081 Autoformer RMSE 0.104 0.103 0.101 0.110 0.066 0.083 0.101 0.181 0.042 0.050 0.053 0.050 0.075 0.099 0.115 0.119 0.105 0.114 0.119 0.136 0.109 0.112 0.125 0.126 MAE 0.064 0.061 0.059 0.065 0.050 0.063 0.075 0.150 0.026 0.033 0.034 0.035 0.051 0.051 0.088 0.116 0.079 0.086 0.088 0.102 0.081 0.083 0.091 0.093 Informer RMSE 0.139 0.134 0.115 0.132 0.084 0.088 0.127 0.170 0.039 0.047 0.053 0.054 0.124 0.138 0.144 0.148 0.121 0.137 0.145 0.157 0.096 0.107 0.119 0.149 MAE 0.101 0.097 0.101 0.132 0.066 0.068 0.093 0.117 0.023 0.030 0.034 0.035 0.094 0.105 0.112 0.116 0.093 0.103 0.112 0.125 0.070 0.082 0.090 0.115 Reformer RMSE 0.152 0.201 0.203 0.228 0.146 0.169 0.189 0.201 0.053 0.054 0.053 0.054 0.125 0.138 0.144 0.148 0.143 0.148 0.155 0.155 0.089 0.108 0.128 0.163 MAE 0.108 0.147 0.154 0.173 0.126 0.147 0.157 0.166 0.035 0.035 0.035 0.035 0.095 0.121 0.122 0.120 0.113 0.120 0.124 0.126 0.065 0.081 0.100 0.132 The WM-MLP backbone results reported in Table 1 consider an optimization with respect to p , the number of windows, while the HC-MLP considers a fixed size of p=4 windows. Thus, for a more suitable comparison, Table 2 shows a comparison of the FIA-Net performance under both backbones with p=4 . We note that when p is similar for both models, the FIA-Net attains similar results under both backbones, while the HC-MLP requires significantly fewer parameters. Consequently, when the number of windows allows for an HC-MLP version (e.g., p=2^{\ell} as we further explain in Appendix C ), an HC-MLP backbone is preferable. Table 2 : Performance comparison between WM-MLP and HC-MLP with a fixed number of STFT windows (p = 4). Results demonstrate that HC-MLP achieves comparable accuracy while significantly reducing model parameters, making it preferable for efficient implementations. Traffic ETTh1 ETTm1 Metric 96 192 336 720 96 192 336 720 96 192 336 720 WM-MLP (p=4) RMSE 0.033 0.034 0.035 0.036 0.088 0.094 0.100 0.103 0.074 0.082 0.089 0.096 MAE 0.016 0.016 0.017 0.018 0.058 0.064 0.068 0.075 0.049 0.056 0.060 0.067 HC-MLP RMSE 0.032 0.034 0.035 0.036 0.083 0.085 0.094 0.101 0.072 0.082 0.089 0.096 MAE 0.016 0.017 0.017 0.018 0.049 0.057 0.064 0.068 0.049 0.056 0.060 0.067 5.3 Ablation Studies We consider three ablation studies that best demonstrate the key aspects of the proposed work. We focus on the effect of frequency selection, the size of the lookback window, and the omission of real/imaginary components in the training procedure. We show that, in various cases, the total amount of parameters can be decreased by up to 60%. Due to space limitations, the results are demonstrated on a single dataset, while a full discussion and additional results are given in Appendix D.4 . 5.3.1 Frequency Dimension Compression Figure 5 : Accuracy vs. M We study the effect of the parameter M in the top- M frequency component selection process on the ETTh dataset. As seen in figure 5 , even though the model performance varies over different datasets and forecasting horizon sizes, in most cases, M=4 attains the best accuracy. Furthermore, note that taking M