A statistician’s guide to weak-instrument-robust inference in instrumental variables regression with illustrations in Python

Malte Londschien · 2025

Read A statistician’s guide to weak-instrument-robust inference in instrumental variables regression with illustrations in Python by Malte Londschien (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-12474.txt.

Introduction

Malte Londschien surveys how to draw reliable conclusions from instrumental variables regressions when the instruments are weak, in a 2025 guide. Weak instruments can badly distort standard estimates and tests in this workhorse method of causal inference. The guide gathers the main results on estimation and on inference methods that stay valid despite weak instruments. Worked examples run in Python through the author's ivmodels package. Readers get both the theory and runnable illustrations of each method.

Catalog record

CREATORMalte Londschien
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE131 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-12474

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A statistician’s guide to weak-instrument-robust inference in instrumental variables regression with illustrations in Python Malte Londschien Abstract We provide an overview of results relating to estimation and weak-instrument-robust inference in instrumental variables regression. Methods are implemented in the ivmodels software package for Python, which we use to illustrate results. 1 Introduction Instrumental variables regression is essential for empirical economics. It allows for causal effect estimation in the presence of unobserved confounding, also called endogeneity. Instruments are unconfounded variables that affect the outcome only through the treatment variable. They encode variation in the treatment variable, akin to the placebo or treatment assignment in randomized controlled trials. Instrumental variables regression uses this variation to estimate the causal effect of the treatment on the outcome. One popular estimator in instrumental variables regression is the two-stage least-squares (TSLS) estimator. It is computed in two steps. First, the endogenous treatment variables are regressed on the instruments (the first stage). Second, the outcome is regressed on the first regression’s fitted values (the second stage). It has desirable properties: If the instruments are strong enough, that is, they encode sufficient variation in the confounded treatment variables, it is consistent and asymptotically Gaussian. The limiting Gaussian distribution’s covariance can be estimated from the data, and the resulting Wald test is often used to construct confidence sets and p -values for the causal effect. Such uncertainty quantification is essential for causal effect estimation to influence policy. However, as noted by Staiger and Stock 1997 , in many applications, the signal-to-noise ratio in the first stage is low. This leads to the two-stage least-squares estimator being biased towards the biased ordinary least-squares estimator and having a distribution that is not well-approximated by a Gaussian. Staiger and Stock 1997 show that, for a single instrument and endogenous variable, if the expected first-stage F -statistic exceeds 10, asymptotically, the two-stage least squares estimator has at most 10% bias relative to the ordinary least squares estimator and the Wald-based 95% confidence sets have at least 85% coverage. This resulted in the following widely adopted heuristic: If the first-stage F -statistic is greater than 10, the instrument is considered strong enough, and Wald-based confidence sets are considered valid. An alternative to such pre-testing is to use weak-instrument-robust inference. This relies on tests that have the correct asymptotic distribution, even under weak instrument asymptotics ( Staiger and Stock 1997 ) where the first stage’s signal-to-noise ratio decreases such that the expected first-stage F -statistic is of constant order. Prominent weak-instrument-robust tests include the Anderson-Rubin test ( Anderson 1951 ) , the conditional likelihood-ratio test ( Moreira 2003 ) , and the Lagrange multiplier test ( Kleibergen 2002 ) . In some applications ( Card 1995 ; Tanaka et al. 2010 ; Thams et al. 2024 , e.g.,) there are multiple endogenous variables. Then, p -values and multivariate confidence sets for the full causal parameter are difficult to interpret. Partial (or subvector) inference for individual coefficients of the causal parameter, similar to t -tests in linear regression, is more useful. Subvector variants of the Anderson-Rubin test, the conditional likelihood-ratio test, and the Lagrange multiplier test have been proposed by Guggenberger et al. 2012 ; Kleibergen 2021 ; Londschien and Bühlmann 2024 . In the following, we introduce k-class estimators, a family of instrumental variables regression estimators that includes the ordinary least-squares, two-stage least-squares, and the limited information maximum likelihood estimator. We introduce the Wald, Anderson-Rubin, conditional likelihood-ratio, and Lagrange multiplier tests, their subvector variants, and the confidence sets resulting from their inversion. We also introduce auxiliary tests such as the J-statistic ( Sargan 1958 ; Hansen 1982 ) and its LIML variant ( Guggenberger et al. 2012 ) of overidentifying restrictions, the Cragg-Donald test ( Cragg and Donald 1997 ) , related to Anderson 1951 ’s ( Anderson 1951 ) likelihood-ratio test of reduced rank, a multivariate extension of the first-stage F-test, and Scheidegger et al. 2025 ’s ( Scheidegger et al. 2025 ) residual prediction test of misspecification. We provide proofs or proof sketches for all results in appendix C . These are mostly not novel and we reference to the original literature. All estimators and tests introduced in this manuscript are implemented in the ivmodels software package for Python, available on PyPI and conda-forge . See the GitHub repository at github.com/mlondschien/ivmodels and the documentation at ivmodels.readthedocs.io for more details. We use Card 1995 ’s ( Card 1995 ) application estimating the causal effect of education on log-wages as a running example throughout this manuscript and apply the estimators and tests introduced in this manuscript to their dataset. 2 Model setup and assumptions We visualize a basic instrumental variables regression model in figure 1 . Here, X are the endogenous treatment variables, Z are instruments, y is the outcome, and U are unobserved confounders. We are interested in the causal effect of X on y , encoded by the causal parameter \beta_{0} . The first stage parameter \Pi is a nuisance parameter. Z X Y U \Pi \beta_{0} Figure 1 : Causal graph visualizing 1 . review-pratenddefaultcategory.tex Model 1 . Let y_{i}=X_{i}^{T}\beta_{0}+\varepsilon_{i}\in{\mathbb{R}} with X_{i}=Z_{i}^{T}\Pi+V_{X,i}\in{\mathbb{R}}^{{m_{x}}} for random vectors Z_{i}\in{\mathbb{R}}^{k},V_{X,i}\in{\mathbb{R}}^{{m_{x}}} , and \varepsilon_{i}\in{\mathbb{R}} for i=1\ldots,n and parameters \Pi\in{\mathbb{R}}^{k\times{{m_{x}}}} , and \beta_{0}\in{\mathbb{R}}^{{m_{x}}} . We call the Z_{i} instruments , the X_{i} endogenous covariates , and the y_{i} outcomes . Let Z\in{\mathbb{R}}^{n\times k},X\in{\mathbb{R}}^{n\times{{m_{x}}}}, and y\in{\mathbb{R}}^{n} be the matrices of stacked observations. In strong instrument asymptotics , we assume that \Pi is fixed for all n and of full column rank {{m_{x}}} . In weak instrument asymptotics ( Staiger and Stock 1997 ) , we assume that \sqrt{n}\Pi is fixed and of full column rank m_{X} . Thus, \Pi={\cal O}(\frac{1}{\sqrt{n}}) . Both asymptotics imply that k\geqslant{{m_{x}}} . review-pratenddefaultcategory.tex In standard asymptotics, the expected first-stage F -statistic (or its multivariate extension, see section 5.2 and Anderson 1951 , Anderson 1951 , Cragg and Donald 1997 , Cragg and Donald 1997 ) grows with the sample size. Staiger and Stock 1997 note that in many applications, even though the sample size n is large, the F -statistic for regressing X on Z is small. In these settings, standard asymptotics no longer apply, motivating weak instrument asymptotics, where the first-stage F -statistic is of constant order. A setup comparable to weak instrument asymptotics is high-dimensional statistics ( Bühlmann and Van De Geer 2011 ) . In standard asymptotics, the number of covariates p is fixed and the sample size n grows to infinity. However, in application areas such as genomics, the number of covariates p can be as large or even larger than the sample size n . In these settings, standard asymptotics no longer apply, and one needs to use high-dimensional asymptotics, where p grows with n . We assume that a central limit theorem applies to the sums Z^{T}\varepsilon and Z^{T}V_{X} . Assumption 1 . Let \Psi:=\begin{pmatrix}\Psi_{\varepsilon}&\Psi_{V_{X}}\end{pmatrix}:=(Z^{T}Z)^{-1/2}Z^{T}\begin{pmatrix}\varepsilon&V_{X}\end{pmatrix}\in{\mathbb{R}}^{k\times(1+{{m_{x}}})}. Assume there exist \Omega\in{\mathbb{R}}^{(1+{{m_{x}}})\times(1+{{m_{x}}})} and Q\in{\mathbb{R}}^{k\times k} positive definite such that, as n\to\infty , \displaystyle\mathrm{(a)}\ \ \frac{1}{n}\begin{pmatrix}\varepsilon&V_{X}\end{pmatrix}^{T}\begin{pmatrix}\varepsilon&V_{X}\end{pmatrix}\overset{{\mathbb{P}}}{\to}\Omega=\begin{pmatrix}\sigma^{2}_{\varepsilon}&\Omega_{\varepsilon,V_{X}}\\ \Omega_{V_{X},\varepsilon}&\Omega_{V_{X}}\\ \end{pmatrix}, \displaystyle\mathrm{(b)}\ \ \vecop(\Psi)\overset{d}{\to}{\cal N}(0,\Omega\otimes\mathrm{Id}_{k}),\text{ and } \displaystyle\mathrm{(c)}\ \ \frac{1}{n}Z^{T}Z\overset{{\mathbb{P}}}{\to}Q, where \Cov(\vecop(\Psi))=\Omega\otimes\mathrm{Id}_{k} means \Cov(\Psi_{i,j},\Psi_{i^{\prime},j^{\prime}})=1_{i=i^{\prime}}\cdot\Omega_{j,j^{\prime}} . review-pratenddefaultcategory.tex While standard in the literature, 1 might be difficult to interpret. A sufficient condition for 1 is that (i) the error terms \varepsilon_{i} and V_{X,i} are homoscedastic and uncorrelated with

Read the complete text