Bayesian Models for Joint Selection of Features and Auto-Regressive Lags: Theory and Applications in Environmental and Financial Forecasting
Alokesh Manna, Sujit K. Ghosh · 2025
Read Bayesian Models for Joint Selection of Features and Auto-Regressive Lags: Theory and Applications in Environmental and Financial Forecasting by Alokesh Manna, Sujit K. Ghosh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10055.txt.
Introduction
Manna and Ghosh build Bayesian models that pick forecasting variables and time lags at the same time. Their setting is linear regression where errors stay correlated over time, a common problem in finance and environmental data. The method uses spike-and-slab priors inside a two-stage sampling algorithm.
Theory in the study shows the selection stays consistent even when candidate predictors grow very fast. Tests on groundwater depth and S&P 500 returns beat older methods on prediction error. The models also found the true variables more often and held up better under correlated noise. The authors aim the tools at analysts who need models they can interpret.
Catalog record
| CREATOR | Alokesh Manna, Sujit K. Ghosh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-10055
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bayesian Models for Joint Selection of Features and Auto-Regressive Lags: Theory and Applications in Environmental and Financial Forecasting Alokesh Manna, Sujit K. Ghosh Abstract We develop a Bayesian framework for variable selection in linear regression models with autocorrelated errors that accommodate both lagged covariates and autoregressive error structures. This setting arises naturally in many time-series applications where the response is influenced not only by contemporaneous and past values of explanatory variables but also by persistent stochastic shocks. Examples include financial modeling (e.g., logarithmic returns of asset prices), hydrological forecasting (e.g., groundwater dynamics), and meteorological applications (e.g., temperature or precipitation patterns), where capturing temporal dependencies is critical for accurate inference and prediction. Our methodology employs a hierarchical Bayesian model with spike-and-slab priors to simultaneously select relevant covariates and lagged error terms. To address the computational challenges of high-dimensional model spaces, we propose an efficient two-stage Markov Chain Monte Carlo (MCMC) algorithm that separates the sampling of variable inclusion indicators and model parameters. Theoretical analysis establishes posterior selection consistency under mild regularity conditions, even when the number of candidate predictors grows exponentially with the sample size, a common feature in modern time-series applications involving many potential lagged variables. Through extensive simulation studies and two real-data applications, the prediction of groundwater depth and the modeling of S&P 500 index log-returns—we demonstrate that the proposed approach offers substantial gains in variable selection accuracy and predictive performance. Compared to existing methods, our framework achieves a lower mean squared prediction error (MSPE), improved identification of true model components, and greater robustness in the presence of autocorrelated noise. These results underscore the practical utility of our Bayesian procedure for both model interpretation and forecasting in autoregressive settings. K eywords Autoregressive, Hydrology, Spike and slab, Stock market prediction, Time series modeling, Variable selection. 1 Introduction Vector Autoregression with Exogenous Variables (VARX) is a prevalent approach for forecasting in multivariate time series analysis, leveraging multiple lags of both endogenous and exogenous variables to capture intricate temporal dependencies and interactions. This structure enables a more precise prediction of current values by accounting for the influence of past values of both types of variables (see [ 16 ] , [ 15 ] ). Due to the high dimensionality inherent in VARX models, effective variable selection is essential to enhance model interpretability and mitigate overfitting. [ 6 ] , [ 5 ] discussed change point detection in a very high-dimensional low number of time points data to account for non-stationarity. In many real-world data scenarios, the time points may be large with low-dimensional, relevant features that can change stochastically. Regularization techniques, notably the lasso introduced by [ 18 ] , are commonly employed to enforce sparsity by “shrinking” irrelevant coefficients towards zero, thus improving computational efficiency and preserving only the most influential variables. The “glmnet” package by [ 7 ] provides a computational tool that integrates sparsity within a linear regression framework. Building on this, [ 15 ] proposed structured penalties to facilitate the simultaneous estimation of high-dimensional time series within both VAR and VARX frameworks. In the Bayesian framework, variable selection is typically performed through sparse priors on the regression coefficients, which are designed to promote sparsity. The spike-and-slab prior, as discussed by [ 10 ] , combines a point mass at zero (the “spike”) with a continuous distribution (the “slab”) over the remaining coefficient values, enabling selective shrinkage. Alternatively, the horseshoe prior, explored by [ 3 ] , applies global and local shrinkage parameters to the coefficients, providing a flexible approach for managing sparsity while accommodating signals of varying strengths across the coefficients. These Bayesian sparsity-inducing priors offer robust alternatives for variable selection in high-dimensional VARX settings, allowing for improved model accuracy and interpretability. [ 9 ] investigated variable selection using Gaussian and diffused-gamma prior with l_{0} norm penalization. [ 12 ] explored water table depth prediction using the “BigVAR” package using a frequentist elastic net penalized regression method for variable selection. [ 11 ] explored interval estimation of penalized regression in a complicated Tweedie model framework in the Generalized Linear Model (GLM). Conformal inference for quantifying uncertainty in prediction intervals using GLM-type residuals is also developed in [ 13 ] . While the above are frequentist methods, this paper presents a Bayesian approach that effectively quantifies uncertainty for non-IID data, such as time series, while achieving high predictive accuracy. While variable selection and prediction within an autoregressive framework, especially the simultaneous selection of coefficients for both endogenous and exogenous variables and their lags have been explored, there remains a gap in the Bayesian framework. To address this the strong posterior consistency of the coefficients for both exogenous and endogenous variables within an autoregressive model was investigated, providing a Bayesian approach to variable selection and prediction. [ 4 ] considered the model \mathbf{y}=\mathbf{X}\boldsymbol{\beta}+\boldsymbol{\epsilon}, where \mathbf{y} is a n\times 1 vector of response variables, \mathbf{X} is a n\times p matrix of predictor variables, \boldsymbol{\beta} is a p\times 1 vector of coefficients, \sigma^{2} is an unknown variance term, and \boldsymbol{\epsilon} is a n\times 1 vector of zero-mean errors that are potentially auto-correlated. The columns of the \mathbf{X} may contain the lagged values of the feature or predictor variables. Thus, the total predictors p can be larger than the time points n . In this work, it is assumed that these errors follow an AR(q) model described more briefly as follows: \epsilon_{t}=\sum_{l=1}^{q}\phi_{l}\epsilon_{t-l}+u_{t},\quad|\phi_{j}|<1,\quad j=1,2,\dots,q; where u_{t} are assumed to be independently identically distributed (i.i.d.) errors satisfying E(u_{t})=0 , \text{Var}(u_{t})=\sigma^{2} , and they are independent of the predictor variables (see [ 19 ] ). A similar model with auto-regressive error has also been explored in [ 20 ] where l_{1} penalty was introduced over the coefficients for variable selection. Thus, the model incorporating lag structures for both errors and covariates is formulated as follows: \displaystyle y_{t}=\mathbf{\nu}+\sum_{j=1}^{r}\mathbf{x}_{t-j}^{\prime}\boldsymbol{\beta}_{j}+\epsilon_{t}\;\text{ for }\;t=1,\ldots,n, (1) \displaystyle\epsilon_{t}=\sum_{l=1}^{q}\phi_{l}\epsilon_{t-l}+u_{t},\quad|\phi_{j}|<1,\quad j=1,2,\dots,q. (2) In this model, predictors are represented as \{\mathbf{x}_{t}\}_{t=1}^{n} and the dependent variable is represented as \{y_{t}\}_{t=1}^{n} . \boldsymbol{\beta}_{j} is the coefficient vector of \mathbf{x}_{t-j} , j^{th} lag of the predictors and \phi_{\ell} is the l^{th} coefficient for the lag structure of y_{t} . We use “ r ” for the number of lags used for the covariates. An additive random Gaussian noise u_{t}\stackrel{{\scriptstyle\text{wn}}}{{\sim}}(0,\sigma_{u}^{2}) is considered in the model. For This model, the variable selection was examined incorporating elastic net penalty over the coefficients [\mathbf{\Phi},\boldsymbol{\beta}] in [ 15 ] . Define \mathbf{\Phi}:=\left(\phi_{1},\phi_{2},\cdots,\phi_{q}\right) as the vector of all coefficients of the lag structure of y_{t} with order “q” and \boldsymbol{\beta}:=\left(\boldsymbol{\beta}_{1},\boldsymbol{\beta}_{2},\cdots,\boldsymbol{\beta}_{p}\right) be the vector of coefficients of the predictors with their lag order “p”. [ 8 ] proposed model is based on a pseudo-likelihood-based Bayesian approach for variable selection, and then examined model selection consistency for the VAR model under the assumption of sparsity in \mathbf{\Phi} for the model \mathbf{y}=\mathbf{X}\mathbf{\Phi}+\boldsymbol{\epsilon} , where the response vector \mathbf{y} is n\times 1 , and \boldsymbol{\varepsilon}\sim\mathcal{N}_{n}(0,\mathbf{\Sigma}) . In the pseudo-likelihood approach, \mathbf{\Sigma} was assumed to be diagonal, and \mathbf{\Phi} was estimated using a spike and slab prior. One of the primary goals of this work is the simultaneous selection of the number of feature variables p and the number of the lags q of the AR errors process. It may happen that p+q>n but the true model is assumed to