High-dimensional Penalized Linear IV Estimation & Inference using BRIDGE and Adaptive LASSO
Eleftheria Kelekidou · 2025
Read High-dimensional Penalized Linear IV Estimation & Inference using BRIDGE and Adaptive LASSO by Eleftheria Kelekidou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-00265.txt.
Introduction
Estimating cause and effect with instrumental variables gets hard when there are more parameters than observations. Eleftheria Kelekidou writes an exposition in 2025 on high-dimensional penalized linear instrumental variable estimation. The methods covered are BRIDGE and adaptive LASSO, used inside two-stage least squares.
Both stages of the procedure can have parameter counts that grow with the sample size. Penalization keeps such problems solvable by shrinking weak coefficients toward zero. The exposition helps applied researchers in economics and health choose and understand tools for settings where traditional instrumental variable methods break down. Two-stage least squares is the classic estimator the penalized methods extend. Growing parameter counts are now routine in genetic and economic instruments. For folks who work with scant data and many loose ends, the gain is a set of tools that stay put. They hold up when the old rules break down at scale.
Catalog record
| CREATOR | Eleftheria Kelekidou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 124 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-00265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
High-dimensional Penalized Linear IV Estimation & Inference using BRIDGE and Adaptive LASSO Eleftheria Kelekidou Abstract This paper is an exposition of how BRIDGE and adaptive LASSO can be used in a two-stage least squares problem, to estimate the 2nd stage coefficients when the number of parameters p p in both stages is growing with the sample size n n . Facing a larger class of problems compared to the usual analysis in the literature, i.e. replacing the assumption of normal with sub-gaussian errors, I prove that both methods are model selection consistent and oracle efficient even when the number of instruments and covariates exceeds the sample size. For BRIDGE, I also prove that if the former is lower than the latter, the same properties hold without sub-gaussian errors. When p > n p>n , BRIDGE requires a slightly weaker set of assumptions to have the desirable properties, as adaptive LASSO requires a good initial estimator of the relevant weights. However, adaptive LASSO is expected to be much faster computationally, so the methods are competitive on different fronts and the one that is recommended depends on the researcher’s resources. 1 Introduction Using increasingly large datasets, economists and social scientists in general face new challenges that are connected with the high-dimensional nature of the problems they study. A particular instance of such a scenario is the class of applications that involve high-dimensional instruments. Valid instruments are often the holy grail of applied research; however, recently there is increased availability in rich datasets such as the GWAS Catalog MacArthur et al. 2017 , Sollis et al. 2023 , which includes information on how phenotypic variations affect biological traits and disease traits give new paths for research. This type of variables can provide a number of instrument alternatives for applications related to productivity, risk, welfare, actuarial science, etc Gui and Li 2005 , Ma and Huang 2007 , Wang et al. 2008 . In order to be able to do estimation and inference in environments like this, the researcher has to address the statistical difficulties that occur. I will focus on the usual linear model Y=X^{\prime}\beta_{0}+u where the usual exogeneity restriction, i.e. E[u|X]=0 fails. Instead, I will assume that there exist observable variables Z that can be used as instruments, such that E[u|Z]=0 , {\rm Cov}[ZX] is full rank, and the conditional expectation E[X|Z] is linear. I am allowing the number of both the endogenous covariates and the instruments to grow and I am making the high level assumption that the number of instruments is sufficient for the identification of the model. Due to the dimensionality problem, if the number of covariates in each stage exceeds the sample size, the OLS estimator is not identified. In this case, one can obtain valid estimators by minimizing the penalized loss function, i.e. L_{n}(b)=\argmin_{b}\frac{1}{2}\sum_{i=1}^{n}(Y_{i}-X_{i}^{\prime}b)^{2}+\lambda_{n}\sum_{j}^{p}p(b_{j}) where p(b_{j}) is a penalty function. Due to the presence of endogeneity, instead of X_{i} , I will use an estimator of E[X|Z] . Under sparsity, the latter can be obtained by using any proper penalization function. Then, I will estimate the model choosing the following popular penalty functions: BRIDGE , with p(b)=|b|^{\gamma} , where \gamma\in(0,1) and adaptive LASSO with p(b)=w_{b}|b| , where w_{b} is a set of weights. Related literature: The standard practice to estimate high-dimensional, sparse, linear models includes popular penalization functions, such as, for \gamma=1 , LASSO Tibshirani 1996 , adaptive LASSO Zou 2006 , BRIDGE Fu and Knight 2000 , the smoothly clipped absolute deviation (SCAD) Fan and Li 2001 , the minimax concave pentaly (MCP) Zhang 2010 etc. However, inference in these setups often remains a difficult problem even with the sole existence of exogenous covariates van de Geer et al. 2014 . In environments that endogeneity is also an issue, the econometrician faces three distinct scenarios; the first is having a growing number of instruments, yet a small number of covariates of interest Belloni et al. 2012 , Caner and Fan 2015 , Fan and Zhong 2018 , Hansen and Kozbur 2014 . The second is allowing for a growing number of covariates but a low-dimensional set of them being endogenous, which permits for a low dimensional estimation in the first stage (eg. Fan and Liao 2014 ). Finally, there is the case with high-dimensionality in both stages with a growing number of instruments and regressors. Lin et al. 2015 allow for this setting and they prove model selection consistency and oracle efficiency using LASSO, SCAD, and MCP, under the restrictive assumption of normality. Caner and Zhang 2014 propose a GMM estimator using adaptive elastic nets as a penalty function, allowing for a growing number of covariates but without exceeding the sample size. Gold et al. 2020 follows a similar direction, providing general assumptions under which a penalty function would have the desirable properties but restricting the nature of the instruments to sub-Gaussian. Also, for LASSO, the required properties imply the presence of strong compatibility conditions to achieve the oracle property. To address the need for a more flexible framework, a natural path would be to turn to a penalty function that ensures model selection consistency with minimal assumptions in the simple OLS case, and consider the extension to the endogeneity framework. BRIDGE and adaptive LASSO are natural candidates for this setup. There have been some attempts Bahador et al. 2024 to apply BRIDGE in two-stage penalized least squares using a control function approach, yet there is no formal proof of model selection and they require normality for the error terms. Contribution: The main contribution of this paper is showing that the estimator for \beta_{0} using both penalty functions is model selection consistent and oracle efficient. For both methods, if the number of parameters grows faster than the sample size, the result is based only on sub-Gaussian tails of the error term and standard rate conditions. The argument for BRIDGE is an extension to the partial orthogonality condition in Huang et al. 2008a . For adaptive LASSO, the proof is based on the requirement of a consistent estimator for the weights in the penalty function as stated for the exogenous case in Huang et al. 2008b . Moreover, I show that for a growing number of parameters but not higher than the sample size, the result for BRIDGE holds with just homoskedastic, mean 0 errors and the corresponding rate assumptions. Lastly, for BRIDGE, I provide a proposal of a consistent estimator for the standard errors as well as some computational evidence on the performance of the method. Corresponding exercises on adaptive LASSO are ongoing and available upon request. Structure: The rest of the paper is organized as follows. Section 2 introduces the notation and the requirements for the 1st stage of the problem, regardless of the 2nd stage method. In Section 3, I present the results for BRIDGE when p<n . In Sections 4 and 5, I present the results for p>n for BRIDGE and adaptive LASSO correspondingly. Finally, Section 6 contains some computational results comparing how BRIDGE and LASSO perform in this problem. 2 Notation & Setup Consider the vector (Y,X,Z) , where Y is the outcome random variable, X is the random vector of observable covariates - possibly endogenous, and Z is a fixed vector