A Review of the Long Horizon Forecasting Problem in Time Series Analysis Journal: International Journal of Forecasting

Hans Krupakar, Kandappan V A · 2025

Read A Review of the Long Horizon Forecasting Problem in Time Series Analysis Journal: International Journal of Forecasting by Hans Krupakar, Kandappan V A (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-12809.txt.

Introduction

Hans Krupakar and Kandappan V A review long range forecasting in 2025. Guessing values far ahead in a time series has challenged the field for decades. The review covers how deep learning took on the problem. It sorts the tricks used, from trend splits and wave transforms to filters that cut bias.

The authors describe model families from simple layered nets to attention based designs. They stress how the input features are built and how data windows are cut. Tests on a standard energy data set probe what each trick adds. Readers get a map of what helps when forecasts must reach far into the future.

Catalog record

CREATORHans Krupakar, Kandappan V A
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-12809

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Review of the Long Horizon Forecasting Problem in Time Series Analysis Journal: International Journal of Forecasting Hans Krupakar, Kandappan V A Abstract The long horizon forecasting (LHF) problem has come up in the time series literature for over the last 35 years or so. This review covers aspects of LHF in this period and how deep learning has incorporated variants of trend, seasonality, fourier and wavelet transforms, misspecification bias reduction and bandpass filters while contributing using convolutions, residual connections, sparsity reduction, strided convolutions, attention masks, SSMs, normalization methods, low-rank approximations and gating mechanisms. We highlight time series decomposition techniques, input data preprocessing and dataset windowing schemes that improve performance. Multi-layer perceptron models, recurrent neural network hybrids, self-attention models that improve and/or address the performances of the LHF problem are described, with an emphasis on the feature space construction. Ablation studies are conducted over the ETTm2 dataset in the multivariate and univariate high useful load (HUFL) forecasting contexts, evaluated over the last 4 months of the dataset. The heatmaps of MSE averages per time step over test set series in the horizon show that there is a steady increase in the error proportionate to its length except with xLSTM and Triformer models and motivate LHF as an error propagation problem. The trained models are available here: https://bit.ly/LHFModelZoo . 1 Introduction Time series forecasting has applications in risk management, disease propagation studies, sensor network monitoring, power plant maintenance, disaster management and prediction, construction, economics, energy and road traffic-related applications etc. ( Challu et al. 2023 ; Wu et al. 2021 ) Most of these applications would find benefits in being able to define a horizon size that is longer and this has been shown to be true for EEG, ECG time series data, predicting clinical events, climate studies, inflation data and the financial analysis of stock markets due to the inherent nature of these datasets ( Saneesh Cleatus Thundiyil 2024 ). These benefits would come from being able to find long-term dependencies in the data and inherent co-variate analysis that isn’t restricted to a short horizon. However, the presence of entangled representations increases in the context of longer horizons and this makes multi-variate correlation analysis harder than in the case of shorter horizon forecasting ( Wu et al. 2021 ). Also, the computational complexity and memory in long-horizon forecasting models are typically dependent on the horizon size which makes longer horizon forecasting slower without GPUs ( Challu et al. 2023 ). Statistical methods like Auto-Regressive Integrated Moving Average ( ARIMA ), Exponential Smoothing ( ETS ), Vector Auto-Regression ( VAR ) models have been used for time series analysis for around the last 50 years but they have consistently reported a decline in performance over longer horizons ( Nelson 1998 ; Cronan 1975 ; Akram et al. 2009 ). In the 1990’s, as summarized by Stock 1995 , it was widely believed that "If the variables are co-integrated, imposing this information can provide substantial improvements in forecasts over long horizons". While co-integrated systems find improvements in long horizon forecasting (LHF), most systems don’t have co-integrated variables be a part of the variables considered in multi-variate time series analysis ( Christoffersen and Diebold 1998 ). In the early 2000s, researchers in economics used long horizon forecasts to find out definitively whether one variable influences the predictions of another ( Clark and McCracken 2001 ). Forecast rationality questions the functional form of the multivariate loss functions given that errors could co-occur across the variables, potentially needing larger error representations for them than the independence assumption a metric like MSE introduces ( Komunjer and Owyang 2012 ). In their paper, Patton and Timmermann 2012 introduce bounds to test for forecast rationality and validate them on simulated and empirical data over longer horizons, up to a horizon size of 8. Long Horizon forecasting (LHF) finds problems in computational complexity and forecast volatility in the modeling aspects of the problem ( Challu et al. 2023 ). In the early 21st century, forecasting methods broadly fell into Iterative Multi-Step (IMS) forecasting and Direct Multi-Step (DMS) ( Chevillon 2007 ) forecasting approaches wherein IMS predicted the forecast horizon one time step at a time and DMS predicted the forecast horizon altogether. Even though IMS models have a smaller variance, the error propagates through time and accumulates more and so isn’t preferred for long horizon forecasting (LHF) ( Zeng et al. 2023 ). Recurrent Neural Network models overcome this error propagation using backpropagation through time (BPTT) and teacher forcing approaches. Galbraith 2003 shows using simulated data that the forecast content decreases asymptotically with respect to an increase in the forecast horizon. In the case of univariate time series forecasting using statistical methods, stationarity and ergodicity properties need to be satisfied ( Lara-Benítez et al. 2021 ). The need for ergodicity is also visible in neural network based time series models in which the right subset of time series data taken for training gives better performance instead of the entire data. Based on findings from Lara-Benítez et al. 2021 over tourism data, traditional models often work well in the context of shorter forecast horizons while neural network models perform well over longer horizons. This enables the use of hybrid models for the problem. Given such findings and empirical evidence, over the more recent years, time series forecasting and analysis in general in research literature has shifted from the use of traditional methods like ARIMA , VAR , ETS etc. to neural forecasting approaches (and hybrids) involving multi-layer perceptrons (MLPs), recurrent neural networks (RNNs) and self-attention based approaches and the long horizon forecasting (LHF) problem has been studied extensively in the context of the use of deep learning over the past 7 years or so. Temporal CNNs are used in some works, one of which is Wan et al. 2019 , where they use dilations in TCNs for multivariate forecasting and show results better than vanilla RNN LSTMs and convLSTMs. The self-attention mechanism and transformer models for forecasting will be covered extensively in the upcoming sections of this paper. The LogTrans model uses convolutional self-attention using causal convolutions on the key and query embeddings to leverage local context whereas traditional self-attention without local context awareness could not differentiate between anomalies, change points or parts of patterns ( Li et al. 2019a ). Improvements to the self-attention mechanism that improve its memory and computational complexity like the use of locality sensitive hashing (LSH) in attention and causal masking to introduce sparsity show improvements in the context of extremely long sequences in language models ( Kitaev et al. 2020 ; Zhou et al. 2021 ) but surprisingly doesn’t as much with time series forecasting ( Challu et al. 2023 ). Also, Zhou et al. 2021 find similarities between the spatial aspects of the dilation operation in convolutions, de-gridding to remove dilation artifacts and length of the convolutional feature spaces ( Yu et al. 2017 ) to lengths of horizons in time series forecasting. Given that transformer models are able to capture long-term dependencies by construction ( Saneesh Cleatus Thundiyil 2024 ), it is important to consider the problem of long horizon forecasting (LHF) as an independent problem as well and Challu et al.

Read the complete text