Wild Bootstrap Inference for Linear Regressions with Many Covariates
Wenze Li · 2025
Read Wild Bootstrap Inference for Linear Regressions with Many Covariates by Wenze Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20972.txt.
Introduction
Wenze Li tweaks a resampling method so it stays valid for regressions stuffed with control variables. The 2025 paper works on the wild bootstrap, a tool that rebuilds data noise to gauge uncertainty in a fit. Wide models with uneven errors can break the standard version of the tool.
Li proposes a simple change to the resampling step and proves it holds in the long run. The proof covers cases where control counts rival the sample size itself. Fake-data trials show the changed method works very well in practice. It beats methods that rely on neat bell-curve cutoffs, most of all with small samples. The gain matters for studies that must control for many factors at once.
Catalog record
| CREATOR | Wenze Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 31 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-20972
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Wild Bootstrap Inference for Linear Regressions with Many Covariates Wenze Li Abstract We propose a simple modification to the wild bootstrap procedure and establish its asymptotic validity for linear regression models with many covariates and heteroskedastic errors. Monte Carlo simulations show that the modified wild bootstrap has excellent finite sample performance compared with alternative methods that are based on standard normal critical values, especially when the sample size is small and/or the number of controls is of the same order of magnitude as the sample size. Keywords: Wild Bootstrap, Many Covariates, Heteroskedasticity. JEL codes: C12, C21 1 Introduction An important econometric method for estimating the causal effect of a certain variable is based on the linear regression model with the assumption that the variable of interest is exogenous after controlling for a sufficiently large set of covariates (e.g., panel data models with fixed effects or partial linear models). However, inference with many covariates can be challenging under (conditional) heteroskedasticity. Cattaneo et al. 2018a provide a unified framework for nonstandard distributional approximation, which accommodates a variety of asymptotics studied in the literature, including the small bandwidth, many instruments, and many covariates asymptotics. Furthermore, as shown by Cattaneo et al. 2018b , under the asymptotic framework where the number of covariates, q_{n} , grows with the sample size, n , the consistency of the traditional heteroskedasticity-robust variance estimators would require q_{n}/n\rightarrow 0 . In their seminal studies, Cattaneo et al. 2018b and Jochmans 2022 propose new variance estimators that remain consistent even when q_{n}/n\nrightarrow 0 , and provide asymptotically valid inferences that are based on normal critical values. However, it is well known that asymptotic normal approximations may have finite sample distortions, especially when the number of observations is small. Bootstrap is found in the econometric literature to provide an improvement over asymptotic approximations in many situations; e.g., see Hall and Horowitz 1996 , Horowitz 2001 , Davidson and MacKinnon 2010 , Djogbenou et al. 2019 , and the references therein. However, it may fail in high-dimensional settings; e.g., see El Karoui and Purdom 2018 and the references therein. For linear regressions with heteroskedastic errors, Mammen 1993 established that the validity of nonparametric bootstrap and wild bootstrap requires q^{1+\delta}_{n}/n\rightarrow 0,\delta>0 . Similarly, for linear instrumental variable (IV) models with many IVs and homoskedastic errors, Wang and Kaffo 2016 show bootstrap inconsistency for the standard residual-based bootstrap when the number of IVs is of the same order of magnitude as the sample size n , and propose a valid alternative bootstrap procedure. However, their procedure cannot be directly extended to the case with both many IVs and heteroskedasticity, even with a fixed number of controls. To our knowledge, there is no existing proven-valid bootstrap method for linear regression models under both q_{n}/n\nrightarrow 0 and heteroskedasticity. Simulations in Cattaneo et al. 2018b also suggest that the nonparametric bootstrap seems to fail in their settings (e.g., see Remark 2 in their paper). In this paper, following the construction of the (approximate) cross-fit variance estimator in Jochmans 2022 , we propose a simple modification to the standard wild bootstrap and show its asymptotic validity when q_{n}/n\nrightarrow 0 . Monte Carlo experiments illustrate good finite sample performance of our procedure. 2 Model and Setup Consider the linear regression model \displaystyle y_{i}=x_{i}\beta+w^{\prime}_{i}\gamma+u_{i},\;i=1,...,n, (1) where y_{i} is a scalar outcome variable, x_{i} is a scalar explanatory variable such as a certain treatment, w_{i} is a q_{n}\times 1 vector of covariates, and u_{i} is an error term. We allow q_{n} to be a non-negligible fraction of the sample size n . The ordinary least squares (OLS) estimator of \beta is then defined as \displaystyle\hat{\beta}_{n}=\left(\sum_{i=1}^{n}\hat{v}^{2}_{i}\right)^{-1}\left(\sum_{i=1}^{n}\hat{v}_{i}y_{i}\right), where \hat{v}_{i}=\sum_{j=1}^{n}(M_{n})_{ij}x_{j} , (M_{n})_{ij}=\{i=j\}-w_{i}^{\prime}(\sum_{k=1}^{n}w_{k}w_{k}^{\prime})^{-1}w_{j} , and \{\cdot\} denotes the indicator function. Recently, Cattaneo et al. 2018b propose a new heteroskedasticity-robust variance estimator, which is consistent when \limsup_{n}q_{n}/n<1/2 . Furthermore, Jochmans 2022 proposes an alternative variance estimator, which remains consistent as long as \limsup_{n}q_{n}/n<1 , and its corresponding t -ratio can be defined as t_{n}=(\hat{\beta}_{n}-\beta_{0})/\sqrt{\acute{\Omega}_{n}} , where \displaystyle\acute{\Omega}_{n}=\left(\sum_{i=1}^{n}\hat{v}_{i}^{2}\right)^{-1}\left(\sum_{i=1}^{n}\hat{v}_{i}^{2}(y_{i}\acute{u}_{i})\right)\left(\sum_{i=1}^{n}\hat{v}_{i}^{2}\right)^{-1}, (2) with \acute{u}_{i}=\hat{u}_{i}/(M_{n})_{ii} and \hat{u}_{i}=\sum_{j=1}^{n}(M_{n})_{ij}(y_{j}-x_{j}\hat{\beta}_{n}) . Then, t -tests based on normal critical values have correct size in large samples. However, simulations suggest that asymptotic normal approximation may not have satisfactory finite sample performance (Section 5 ), especially when the sample size is small and/or q_{n} equals a large fraction of n . In Section 3 , we propose a modified wild bootstrap procedure, which follows the (approximate) cross-fit variance estimator of Jochmans 2022 . 3 Modified Wild Bootstrap Procedure Our procedure is defined as follows: Step 1: Given the null hypothesis H_{0}:\beta=\beta_{0} , generate the residuals \{\tilde{u}_{i}(\beta_{0})\}_{i=1}^{n} , where \tilde{u}_{i}(\beta_{0})=y_{i}-x_{i}\beta_{0}-w_{i}^{\prime}\tilde{\gamma}_{n}, and \tilde{\gamma}_{n} is the null-restricted OLS estimator of \gamma . Step 2: Generate \{u_{i}^{*}\}_{i=1}^{n} , where u_{i}^{*}=a_{n}(\beta_{0})\omega^{*}_{i}\tilde{u}_{i}(\beta_{0}) , \{\omega^{*}_{i}\}_{i=1}^{n} is an i.i.d. sample of random weights that are independent of the data with zero mean and unit variance while \displaystyle a_{n}(\beta_{0})=\sqrt{\max\{\acute{\Sigma}_{n}(\beta_{0}),1/n\}/\hat{\Sigma}_{n}(\beta_{0})} (3) is an adjustment factor, which takes into account the high dimensionality of the covariates. Specifically, \acute{\Sigma}_{n}(\beta_{0})=\sum_{i=1}^{n}\hat{v}_{i}^{2}y_{i}\acute{u}_{i}(\beta_{0}) , \hat{\Sigma}_{n}(\beta_{0})=\sum_{i=1}^{n}\hat{v}_{i}^{2}\tilde{u}^{2}_{i}(\beta_{0}) , \tilde{u}_{i}(\beta_{0}) is defined in Step 1, and \acute{u}_{i}(\beta_{0})=\tilde{u}_{i}(\beta_{0})/(M_{n})_{ii} . Notice that \acute{\Sigma}_{n}(\beta_{0}) is similar to the “meat” part of \acute{\Omega}_{n} in ( 2 ), but here we impose H_{0} , as we generated the bootstrap samples with the null imposed in Step 1. Additionally, max\{\cdot,1/n\} guards against the case that \acute{\Sigma}_{n}(\beta_{0}) could take a non-positive value in finite samples. 1 1 1 This may also occur for the variance estimators of Cattaneo et al. 2018b and Jochmans 2022 . Step 3: Generate y^{*}_{i}=x_{i}\beta_{0}+w_{i}^{\prime}\tilde{\gamma}_{n}+u_{i}^{*},i=1,...,n . Then, compute the bootstrap OLS estimator \hat{\beta}^{*}_{n}=\left(\sum_{i=1}^{n}\hat{v}^{2}_{i}\right)^{-1}\left(\sum_{i=1}^{n}\hat{v}_{i}y^{*}_{i}\right) , and the bootstrap residual \hat{u}_{i}^{*}=\sum_{j=1}^{n}(M_{n})_{ij}(y^{*}_{j}-x_{j}\hat{\beta}^{*}) . Step 4: Compute t_{n}^{*}=(\hat{\beta}^{*}_{n}-\beta_{0})/\sqrt{\acute{\Omega}^{*}_{n}} , where \acute{\Omega}^{*}_{n}=(\sum_{i=1}^{n}\hat{v}_{i}^{2})^{-1}(\sum_{i=1}^{n}\hat{v}_{i}^{2}(y^{*}_{i}\acute{u}^{*}_{i}))(\sum_{i=1}^{n}\hat{v}_{i}^{2})^{-1} , with \acute{u}_{i}^{*}=\hat{u}_{i}^{*}/(M_{n})_{ii} , i.e., \acute{\Omega}^{*}_{n} has the same formula as \acute{\Omega}_{n} but uses bootstrap samples. Step 5: Repeat Steps 2-4 B times, and compute the bootstrap p -value p_{n}^{*}=B^{-1}\sum_{b=1}^{B}1\{|t_{n}|>|t_{n}^{*(b)}|\} . We reject H_{0} if p_{n}^{*} is smaller than the nominal level \alpha . Several comments are in order. Comment 1. In Step 1, we impose null when computing the residuals, which is advocated by Cameron et al. 2008 , Davidson and Flachaire 2008 , Roodman et al. 2019 , among others. As a consequence, we also impose null for the adjustment factor a_{n}(\beta_{0}) in Step 2. When q_{n}/n\rightarrow 0 , \acute{\Sigma}_{n}(\beta_{0}) and \hat{\Sigma}_{n}(\beta_{0}) are asymptotically equivalent so that our procedure reduces to the standard (null-imposed) wild bootstrap. Comment 2. Instead of the above percentile- t type procedure, one may consider a percentile type procedure, whose bootstrap p -value equals B^{-1}\sum_{b=1}^{B}1\{|\sqrt{n}(\hat{\beta}_{n}-\beta_{0})|>|\sqrt{n}(\hat{\beta}^{*(b)}_{n}-\beta_{0})|\} . However, in line with the recommendation in the literature (e.g., see the papers cited above), we find in simulations that the percentile- t has better behaviours and thus also recommend using it. Comment 3. Here we focus on the case where x_{i} is a scalar variable. It is possible to extend the analysis to the case where x_{i} is a (fixed) d_{x} -dimensional vector by considering a modification to the score bootstrap (e.g., Kline and Santos 2012 ). Specifically, the procedure for testing H_{\lambda}:c^{\prime}\beta=\lambda , where c\in R^{d_{x}} and \lambda\in R , can be defined as follows: (1) Obtain the null-restricted OLS estimator \tilde{\beta}_{n} and \tilde{\gamma}_{n} , and employ it to the score contributions \{S_{i}(\tilde{\beta}_{n})\}_{i=1}^{n} , where S_{i}(\tilde{\beta}_{n})=\hat{v}_{i}\tilde{u}_{i}(\tilde{\beta}_{n}) and \tilde{u}_{i}(\tilde{\beta}_{n})=y_{i}-x^{\prime}_{i}\tilde{\beta}_{n}-w_{i}^{\prime}\tilde{\gamma}_{n} ; (2) Compute the perturbed score contributions \{S_{i}^{*}(\tilde{\beta}_{n})\}_{i=1}^{n} , where