Sharp Bounds on the Variance of General Regression Adjustment in Randomized Experiments
Jonas M. Mikhaeil, Donald P. Green · 2024
Read Sharp Bounds on the Variance of General Regression Adjustment in Randomized Experiments by Jonas M. Mikhaeil, Donald P. Green (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-00191.txt.
Description
Abstract A growing statistical literature focuses on causal inference in the context of experiments where the target of inference is the average treatment effect in a finite population and random assignment determines which subjects are allocated to one of the experimental conditions.
Catalog record
| CREATOR | Jonas M. Mikhaeil, Donald P. Green |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 90 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-00191
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Sharp Bounds on the Variance of General Regression Adjustment in Randomized Experiments Jonas M. Mikhaeil, Donald P. Green Abstract A growing statistical literature focuses on causal inference in the context of experiments where the target of inference is the average treatment effect in a finite population and random assignment determines which subjects are allocated to one of the experimental conditions. In this framework, variances of average treatment effect estimators remain unidentified because they depend on the covariance between treated and untreated potential outcomes, which are never jointly observed. Conventional variance estimators are upwardly biased. Recent work provides an estimator for the variance of the difference-in-means estimator that is asymptotically sharp. The practical relevance of this result, however, remains limited, as researchers often rely on some form of covariate adjustment, such as linear regression, when estimating the average treatment effect. Adapting propositions from empirical process theory, we provide asymptotically sharp variance estimators for general regression adjustment. We apply these results to linear regression adjustment and show benefits both in a simulation and in three empirical applications drawn from different disciplines. Keywords— Causal Inference, Finite Populations, General Regression Adjustment, Randomized Experiments, Variance Estimation 1 Introduction Although a century has passed since the publication of Neyman (1923) , the finite population framework for analyzing randomized experiments remains a vibrant topic, perhaps because it cleanly differentiates between the analysis of an experiment on a given set of subjects and extrapolations to other populations. Recent scholarship has made important contributions to the understanding of the finite population setting (cf. central limit theorems in ( Li and Ding, 2017 ; Schochet et al., 2022 ; Liu et al., 2024 ) , yet variance estimation remains a lingering concern. Neyman’s original result noted the fact that the variance of the difference-in-means estimator is not point-identified except under special conditions, such as homogeneous treatment effects among all subjects. The widely-used classical variance estimator, described below, is unbiased under homogeneous treatment effects but positively biased under heterogeneous treatment effects. The classical variance estimator is recommended in textbook treatments ( Ding, 2024 ; Imbens and Rubin, 2015 ; Gerber and Green, 2012 ) on the grounds that it offers a “conservative” assessment of variance, which in turn implies that confidence intervals will have nominal coverage or greater. Well aware of this problem of the unidentified variance of the average treatment effect estimator, Neyman (1923) derived an alternative variance estimator based on the Cauchy-Schwarz inequality that provides a less conservative upper bound on the variance. The Cauchy-Schwarz inequality, however, may be unnecessarily loose. Aronow et al. (2014) , generalizing a result in ( Robins, 1988 ) , derive a variance estimator based on sharp bounds on the estimator’s variance. Aronow et al. show that this estimator renders a less conservative variance estimate than the classic formula as well as the bounds proposed by Neyman (1923) . The practical utility of these sharp bounds, however, remains limited. As proposed by Aronow et al. (2014) , they are only available for the difference-in-means estimator. This drastically limits their applicability, as randomized controlled trials often use some form of covariate adjustment, such as weighting ( Schochet, 2024 ; Zubizarreta et al., 2023 ) , linear regression ( Lin, 2013 ) or machine learning methods ( Bloniarz et al., 2016 ; Wu and Gagnon-Bartsch, 2018 ; Su et al., 2023 ) in order to improve precision. To fill this gap, the present paper derives a sharp variance bound and an accompanying variance estimator for general regression adjustment. To illustrate our results, we explicitly apply it to the variance of linear regression adjustment ( Lin, 2013 ) , see Section 4.2 . In Appendix D , we apply our results in the context of a decorrelation method for general regression adjustment ( Su et al., 2023 ) . We provide an R package to make our sharp variance estimators accessible to practitioners. The sharp variance estimators proposed here are guaranteed to be no larger than conventionally used variance estimators, and they potentially offer improvements under heterogeneous treatment effects. As such, they are well-suited to randomized trials in which treatment effects are suspected to be heterogeneous ex ante, e.g., when pre-analysis plans propose to investigate heterogeneous effects across subgroups. Given the many literature reviews in domains such as biomedical research ( Kent et al., 2018 ) , policy analysis ( Ferraro and Miranda, 2013 ) , political science ( Kertzer, 2022 ) , evaluation research ( Smith, 2022 ) , psychology ( Hickin et al., 2021 ) , and behavioral science more generally ( Bryan et al., 2021 ) that offer theoretical and empirical grounds for expecting heterogeneous effects, it appears that many studies would benefit from sharp variance estimation. Even if the gains in precision from the sharp bounds estimator are often modest – we report benefits of up to 15 percent compared to conventional variance estimates – the cumulative gains are sizable given the vast number of studies that could potentially benefit from more accurate variance estimation. The operating characteristics of our sharp variance estimator for linear regression adjustment are demonstrated via simulations. We also show the potential practical benefits of this approach by reanalyzing three experiments from a range of different fields. The first tests the “anchoring effect” that has long figured prominently in behavioral economics ( Lee and Morewedge, 2022 ) ; the second assesses the effects of exposure to the TV sitcom Little Mosque on the Prairie on attitudes toward Arabs ( Murrar and Brauer, 2018 ) ; the third revisits a biomedical trial that evaluates a treatment for acute lung injury ( The National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network, 2006 ) . We conclude by discussing the advantages and limitations of sharp bounds for variance estimation in the context of regression-adjusted estimation of average treatment effects. Related work: The design-based, or finite population, perspective on causal inference that our paper takes is widely adopted in the literature ( Rosenbaum, 2002 ; Imbens and Rubin, 2015 ; Li and Ding, 2017 ; Lin, 2013 ) . Abadie et al. (2020) discuss the benefits of design-based inference in the context of regression analysis. As noted above, the history of design-based inference for randomized experiments goes back to Neyman (1923) , who derives variance bounds based on the Cauchy-Schwarz inequality to allow for conservative inference despite the unidentified nature of the variance of treatment effect estimators. Generalizing ( Robins, 1988 ) , Aronow et al. (2014) derive sharp variance bounds for the difference-in-means estimator for a randomized experiment with two arms. Drawing on the sharp bounds result in Aronow et al. (2014) , Imbens and Menzel (2021) consider the least-favorable coupling of treated and untreated potential outcomes given the marginals of these potential outcomes to design a “causal bootstrap” procedure. Yu et al. (2025) extend sharp variance estimators to stratified experiments. Moving beyond Fréchet-Hoeffding bounds, which offer a solution to the optimization problem defining sharp variance bounds for two-armed experiments, multi-marginal optimal transport can be used to define sharp variance bounds when there are more than two potential outcomes per unit ( Gao et al., 2025 ) , as in the case of multi-arm trials or