Interpretational errors with instrumental variables
Luca Locher, Mats Stensrud, Aaron Sarvet · 2025
Read Interpretational errors with instrumental variables by Luca Locher, Mats Stensrud, Aaron Sarvet (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02045.txt.
Introduction
Luca Locher, Mats J Stensrud, and Aaron L Sarvet study interpretational errors in instrumental variable studies across top journals. Instrumental variables estimate causal effects when direct experiments fail, under stated assumptions. They typically identify a local average effect for a narrow subgroup, not the original target effect. Many authors pragmatically swap targets, then misread results as if the original question were answered.
A systematic review covers hundreds of unique studies in economics, politics, epidemiology, and clinical medicine. Most targeted the local effect, yet few showed specific interest in that parameter. A majority of those studies made claims implying a different parameter was estimated. Since values can differ or even flip sign, conclusions often rest on shaky ground. The authors urge sharper reporting of target parameters.
Catalog record
| CREATOR | Luca Locher, Mats Stensrud, Aaron Sarvet |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-02045
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Interpretational errors with instrumental variables Luca Locher, Mats Stensrud, Aaron Sarvet Abstract Instrumental variables (IV) are often used to identify causal effects in observational settings and experiments subject to non-compliance. Under canonical assumptions, IVs allow us to identify a so-called local average treatment effect (LATE). The use of IVs is often accompanied by a pragmatic decision to abandon the identification of the causal parameter that corresponds to the original research question and target the LATE instead. This pragmatic decision presents a potential source of error: an investigator mistakenly interprets findings as if they had made inference on their original causal parameter of interest. We conducted a systematic review and meta-analysis of patterns of pragmatism and interpretational errors in the applied IV literature published in leading journals of economics, political science, epidemiology, and clinical medicine (n = 309 unique studies). We found that a large fraction of studies targeted the LATE, although specific interest in this parameter was rare. Of these studies, 61% contained claims that mistakenly suggested that another parameter was targeted—one whose value likely differs, and could even have the opposite sign, from the parameter actually estimated. Our findings suggest that the validity of conclusions drawn from IV applications is often compromised by interpretational errors. 1 Introduction Strategies for identifying causal effects from experimental and quasi-experimental data have undoubtedly enriched the applied statistician’s toolbox and their popularization has likely increased the internal validity of empirical investigations. Many methods associated with this “credibility revolution” ( Angrist and Pischke 2010 ) draw attention to causal estimands that arise as side effects of the identification strategy, not from deliberate choice. One such estimand is the local (L) average treatment effect (ATE), a parameter commonly targeted in instrumental variable (IV) analyses. Imbens 2022 summarizes the pragmatic rationale behind the LATE: “The standard approach to identification was to first state what the target estimand was, and then to articulate the identification strategy through assumptions that would allow one to identify that estimand. Angrist and I turned this strategy around and introduced an alternative way to study identification questions. Rather than start with an estimand and ask if and how we could identify that, we started by asking what we could identify under reasonable assumptions.” Pragmatism is a vital component in the application of statistics, distinguishes it from pure mathematical disciplines, and reinforces its status as a methodological discipline in close alliance with many applied domains. However, one line of criticism raises the concern that in the case of LATE estimation, the pragmatic pursuit of identification compromises relevance—yielding an identified quantity of diminished informative value ( Deaton 2010 , Heckman and Urzúa 2010 , Swanson and Hernán 2018 ). Here, we focus on a different, albeit related, problem arising from distinctly pragmatic approaches to causal inference. Suppose an investigator is interested in a causal parameter, say the ATE. Concerned that the ATE may not be identified under plausible assumptions, the investigator instead targets the LATE. We argue that the shift to the LATE presents a source of interpretational error: the investigator may interpret findings as if they pertain to the original parameter of interest. We refer to this type of error as identity slippage . In this article, we study patterns of pragmatism and identity slippage , and present empirical evidence that both phenomena are widespread in the applied literature. We also discuss the practical consequences of these phenomena. Our investigation is structured as follows. First, we revisit assumptions for the identification of causal effects with IV. Then, we proceed to address three central questions: (1) Why is identity slippage relevant? It may create the false impression that inference pertains to another parameter—one with a value different from the parameter under analysis and therefore corresponding to different policy questions. Thus, identity slippage poses a risk to the translation of scientific findings into actionable implications, with perhaps the same impact as other, more well-known errors arising from, for example, model misspecification or p-hacking. (2) How frequently does it occur in applied research? We studied an extensive sample of IV applications published in leading journals of disciplines where IV applications are popular (economics, epidemiology, medicine, and political science). We found that identity slippage was common across all of these domains; 61% of the studied articles exhibited such instances. (3) What factors contribute to its prevalence? We identify pragmatism as a cause of identity slippage and argue that the subtle interpretation of the LATE might increase the chance of making such an error. 2 Identification of causal effects with IV IVs are a popular tool for the identification of causal effects in observational settings and in experiments subject to non-compliance. In this section, we revisit assumptions for the identification of causal effects with IVs, focusing specifically on the (L)ATE. Although IV methods can, in principle, be used to target other estimands, such as marginal treatment effects (MTEs) ( Heckman and Vytlacil 2007 ) , these are beyond the scope of this article. We use capital letters to denote random variables and lower-case letters to denote their possible realizations. We consider an instrument Z , a treatment A and an outcome Y . We use parentheses to denote potential outcome variables. For example, Y(z,a) denotes the potential outcome under an intervention that sets Z=z and A=a , and A(z) denotes the potential treatment under an intervention that sets Z=z . Angrist et al. 1996 show that a valid IV allows for identification of a so called LATE, here denoted as \Psi_{LATE} . According to Angrist et al. 1996 , an IV is valid if the following conditions hold: (Stable unit treatment value) (1) \displaystyle\quad\text{(a)}\quad\text{If }Z=z,\text{ then }A=A(z),\text{ for all }z,\text{ with probability 1}, \displaystyle\quad\text{(b)}\quad\text{If }Z=z\text{ and }A=a,\text{ then }Y=Y(z,a),\text{ for all }z,a,\text{ with probability 1}, \displaystyle\text{(Relevance)}\quad Z\not\!\perp\!\!\!\perp A, (2) \displaystyle\text{(Independence)}\quad Z\perp\!\!\!\perp Y(a), (3) \displaystyle\text{(Exclusion restriction)}\quad Y(z,a)=Y(a)\text{ for all }z,a,\text{ with probability 1}, (4) \displaystyle\text{(Monotonicity)}\quad A(z)\geq A(z^{\prime})\text{ for all }z\geq z^{\prime},\text{ with probability 1}. (5) In a setting with binary Z and A , where Z is a causal instrument, i.e. Z and A are not merely associated via a non-causal path, \Psi_{LATE} can be expressed as: \displaystyle\Psi_{LATE}=E[Y(1)-Y(0)|A(1)>A(0)]. (6) The interpretation of this quantity is relatively straightforward: It corresponds to the ATE in the sub-population of so-called compliers, i.e. the units that are exposed to A if and only if they are exposed to Z ( Angrist et al. 1996 ) . The population of compliers is in general not identifiable, but the size and observed characteristics of this subgroup can be estimated ( Swanson and Hernán 2018 ) . In a generalized setting where either A or Z are multi-valued or a set of control covariates is introduced, the targeted quantity represents a non-negatively weighted average of local effects and the interpretation becomes more nuanced. Angrist and Pischke 2009 provide a comprehensive overview of the interpretation of \Psi_{LATE} in these settings. Further complications are added if instruments are non-causal ( Swanson and Hernán 2018 ) 1 1 1 As an example of a non-causal instrument, consider judge-leniency designs. The quantity used as instrument, e.g. a judge’s denial rate in other cases is not a direct cause of the studied treatment. Rather, the treatment and the instrument share a common