Forecasting High Dimensional Time Series with Dynamic Dimension Reduction

Daniel Peña · 2025

Read Forecasting High Dimensional Time Series with Dynamic Dimension Reduction by Daniel Peña (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-19418.txt.

Introduction

Forecasting hundreds of linked time series at once swamps methods that treat each series on its own. Daniel Peña and one co-author propose a shared frame that shrinks the data while it forecasts. Many known techniques, they show, are just special cases of this one frame. Dynamic principal components and reduced-rank autoregression both fit inside it. Two more members are new here: dynamic forms of two classic tools that link two sets of series.

Because the methods share one derivation, their meanings line up and fair comparisons get easier. The frame also builds in shrinkage, which steadies the fits when data are noisy or short. A simulated case and a real economic forecast task show the gains. The authors aim to give forecasters one language for a crowded family of shrinkage methods for time series.

Catalog record

CREATORDaniel Peña
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-19418

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Forecasting High Dimensional Time Series with Dynamic Dimension Reduction Thanks: Partially supported by Grants 20020170100330BA from the Universidad de Buenos Aires and PICT 1302 from ANPYCT. We are grateful to Z. Gao and R. S. Tsay for providing the code to run their procedure Daniel Peña Abstract Many dimension reduction techniques have been developed for independent data, and most have also been extended to time series. However, these methods often fail to account for the dynamic dependencies both within and across series. In this work, we propose a general framework for forecasting high-dimensional time series that integrates dynamic dimension reduction with regularization techniques. The effectiveness of the proposed approach is illustrated through a simulated example and a forecasting application using an economic dataset. We show that several specific methods are encompassed within this framework, including Dynamic Principal Components and Reduced Rank Autoregressive Models. Furthermore, time-domain formulations of Dynamic Canonical Correlation and Dynamic Redundancy Analysis are introduced here for the first time as particular instances of the proposed methodology. All of these techniques are analyzed as special cases of a unified procedure, enabling a coherent derivation and interpretation across methods. Forecasting High Dimensional Time Series with Dynamic Dimension Reduction Keywords: Dynamic Principal Components; Dynamic Canonical Correlation Analysis; Dynamic Redundancy analysis; Reduced rank autorregressive models; Regularization. 1 Introduction Building prediction rules for vector time series variables as a function of a large set of other explanatory variables is an important objective in Statistics, Machine Learning and Artificial Intelligence. In this paper we will study linear prediction functions that can be used by themselves, or as a starting point for more sophisticated non linear approaches. For instance, instead of training a neural network directly on the observed data it will be faster, and with better interpretation, to fit first a linear model and use its residuals for searching for non linear effects with a recurrent neural network (RNN). Then, both parts can be put together in a RNN with linear part followed by a non linear structure, to have a joint estimation of the forecasting rule. See, for instance, Babu & Reddy, 2014 . A standard procedure to build linear prediction rules is by the multivariate regression model, where the parameters are estimated by least squares. However, as it is well known, least squares estimation is not efficient when the ratio between the number of explanatory variables and the sample size is large, and several alternatives have been proposed for this situation. Let us consider first the static case, in which we have a sample of independent observations and we want to explain a response variable as a function of a set of explanatory variables. One of the first approaches proposed for high dimensional regression is variable selection, see Mallows, 1973 , Bühlmann & Yu, 2003 , Efron et al., 2004 , Wasserman & Roeder, 2009 , Ing & Lai, 2011 , Bertsimas et al., 2016 , and Hazimeh & Mazumder, 2020 .A second approach is dimension reduction, where the original explanatory variables are replaced by a small set of linear combinations of them, that summarize the relevant information. This method includes principal component regression, partial least squares, redundancy analysis, projection pursuit regression, reduced rank regression, factor analysis and structural equation models, among other less popular methods. The third approach is regularization (see Tibshirani, 1996 and Hastie et al., 2009 ), where the estimation function to be minimized is modified by introducing a penalization for the size of the regression parameters, so that the effect of the variables with small explanatory power is reduced, or even eliminated. In this way, a kind of variable selection is made ( Huang et al., 2008 ) . These three approaches are related ( Hastie et al., 2020 ). If the relevant information is included in a subspace of small dimension, we can take as explanatory variables a basis of this central subspace. In the most interesting case in which the response is a vector of many correlated variables, the central subspace includes the linear combinations that are correlated with all the dependent variables, reducing, some times enormously, the number of parameter to be estimated. Regularization and variable selection achieve a similar objective, by searching for a subspace formed by the most important original variables. In this article we will use both dimension reduction and regularization for forecasting vector time series response variables. Suppose we are interested in forecasting y_{t+h} , where y_{t} is a single time series, using the values of a large vector time series of explanatory variables, \mathbf{x_{t}} . Stock & Watson, 2002 proposed to assume that ( y_{t+h},\mathbf{x_{t}} ) follow a factor model representation and estimate the factors using the leading principal components of the \mathbf{x_{t}} variables. Then, the forecast of y_{t+h} is made by using these factors and the lag values of the y_{t} as explanatory variables and estimating the factor loadings and autorregressive coefficients by least squares. They showed that, under general conditions, these estimates are shown to be consistent. As some of the leading principal components may not be the most useful for forecasting, several improvements of this procedure have been proposed in order to select the best factors for prediction. For instance, Bai & Ng, 2009 applied boosting and thresholding rules to select them and Stock & Watson, 2012 used regularization to down-weight non important factors. When instead of a single time series we want to forecast a set of q dependent time series and, assuming that the set of explanatory variable is common to all of them, use the same factors, \widehat{\mathbf{f}}_{t}, for each of the series. Also, specific factors for each of the series can be computed from different sets of explanatory variables. Other approaches are based on other dimension reduction methods, as for instance, sliced inverse regression, see Barbarino & Bura, 2024 , Matilainen et al., 2019 , partial least squares, see Fuentes et al., 2015 , and other related methods, see Nordhausen et al., 2021 and Fan et al., 2017 . Huang et al., 2022 proposed to scale the PC by regressing the variable to be predicted on each of the p predictors, \widehat{y}_{t+h}=\widehat{\alpha}_{i}+\widehat{\gamma}_{i}x_{it}=\widehat{\alpha}_{i}+\widehat{x}_{it}, (1) and instead of computing the principal components of the original variables use those of the scaled variables \widehat{x}_{it}=\widehat{\gamma}_{i}x_{it} . The method is called scaled PC, as the explanatory variables are scaled by their predictive power. Then, calling \widehat{\mathbf{f}}_{t}^{sc} to the principal components corresponding to the s largest eigenvalues of the scaled variables \widehat{x}_{it}, the forecast is made using \widehat{\mathbf{f}}_{t}^{sc} as factors. They showed that this procedure improves the forecasting performance with respect to using the standard principal components. Note that if we want to forecast a vector of time series this method always computes specific factors for each of the series. An extension of this idea was proposed by Gao & Tsay, 2024 as the Supervised Dynamic Principal Component Analysis (sdPCA). First, for i=1,...,p, they run the regressions including also lags in the relationship of the explanatory variable considered \widehat{y}_{t+h}=\widehat{\alpha}_{i}+\widehat{\gamma}_{i,0}x_{t,i}+...+\widehat{\gamma}_{i,q_{i}-1}x_{t-q_{i}+1,i}=\widehat{\alpha}_{i}+\widehat{x}_{t,q_{i},i}, (2) and, second, the main principal components of the variables \widehat{x}_{t,q_{i},i} are obtained. The forecast is made by using the regression of the variables {y}_{t+h} on the selected principal components, {p}_{i,t} . They showed the good practical performance of his

Read the complete text