Inference with weights: Residualization produces short, valid intervals for varying estimands and varying resampling processes
Erin Hartman Chad Hazlett Arisa Sadeghpour UC Berkeley UCLA UC Berkeley · 2025
Read Inference with weights: Residualization produces short, valid intervals for varying estimands and varying resampling processes by Erin Hartman Chad Hazlett Arisa Sadeghpour UC Berkeley UCLA UC Berkeley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-19607.txt.
Introduction
Erin Hartman, Chad Hazlett, and Arisa Sadeghpour study how to get honest uncertainty bounds when weighting survey or study data, in a 2025 paper. Weighting is used in causal studies to balance groups on background traits. The usual robust errors from a weighted regression can be far too wide, because weighting itself inflates the variance estimate.
The authors recommend adding the balancing traits, and their interaction with treatment, to the weighted regression before taking errors. Those errors are sharper and still correct in the long run, for weights that balance exactly, under several sampling frames. Gains can be large when the traits predict the outcome well. For weights that balance only roughly, the added model still cuts leftover error. Tests by simulation and re-analysis of past studies support the advice.
Catalog record
| CREATOR | Erin Hartman Chad Hazlett Arisa Sadeghpour UC Berkeley UCLA UC Berkeley |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-19607
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Inference with weights: Residualization produces short, valid intervals for varying estimands and varying resampling processes Erin Hartman Chad Hazlett Arisa Sadeghpour UC Berkeley UCLA UC Berkeley Abstract Weighting procedures are used in observational causal inference to adjust for covariate imbalance within the sample. Common practice for inference is to estimate robust standard errors from a weighted regression of outcome on treatment. However, it is well known that weighting can inflate variance estimates, sometimes significantly, leading to standard errors and confidence intervals that are overly conservative. We instead examine and recommend the use of robust standard errors from a weighted regression that additionally includes the balancing covariates and their interactions with treatment. We show that these standard errors are more precise and asymptotically correct for weights that achieve exact balance under multiple common resampling frameworks, including design-based and model-based inference, as well as superpopulation sampling with a finite sample correction. Gains to precision can be quite significant when the balancing weights adjust for prognostic covariates. For procedures that balance only approximately or in expectation, such as inverse propensity weighting or approximate balancing weights, our proposed method improves precision by reducing residuals through augmentation with the parametric model. We demonstrate our approach through simulation and re-analysis of multiple empirical studies. 1 Introduction Weighting procedures of many kinds are popular in observational causal inference research. These weights may be derived from a modeled probability of treatment (the propensity score), may represent matching or stratification adjustments, or may be chosen to achieve exact or approximate balance on (functions of) the covariates. Our question begins by asking how investigators should estimate the variance of effect estimates obtained using these weighting approaches, especially (but not only) those involving exact balancing weights. One common practice for inference after weighting is to conceptualize the weights as a pre-processing step, then simply estimate (robust) standard errors from a weighted difference in means, or equivalently a from a weighted regression of outcome on treatment (see e.g. Hernán and Robins, 2020 , Reifeis and Hudgens, 2022 ), with a similar approach sometimes advocated with matching ( Ho et al., 2007 ; Greifer and Stuart, 2021 ) . However, investigators may be concerned that these standard errors are inappropriate for several reasons. First, what should be done to incorporate uncertainty in the weights themselves? And, as we have come to emphasize here, how does this choice relate to the choice of population vs. sample-based estimands and the investigator’s chosen resampling framework (i.e. model-based versus designed-based)? Second, weighting can inflate variance estimates relative to the unweighted sample, sometimes significantly, and investigators may worry whether this is appropriate or problematic. Third, such a standard error may seem not to recognize special properties achieved by the weights, such as the degree to which differences between groups have already been minimized by the weights. Further, the identical point estimates one obtains from weighted regression with or without the covariates that have been exactly balanced raises questions about the need for including these variables in a (weighted) regression step. Consequently, investigators may be uncertain as to the appropriate way to incorporate these weighting procedures into the uncertainty quantification of subsequent causal effect estimates, taking guidance from notions of “standard practice” or habit in their discipline rather than from clearly justified practices and principles appropriate to their inferential questions. In this paper, we hope to offer clarity regarding this question. We first find that the question itself is ill-posed without clearly defining the types of uncertainty the investigator is hoping to account for in their variance estimate. We thus consider both the design-based and model-based philosophies, as well as questions of inference for sample- and population-based target estimands. A close look at these frameworks appears to complicate matters. For example, the the “usual” estimands entertained in causal inference can have different meanings in these different settings, and these approaches apparently imply different challenges for variance estimation. Second, we find that despite these definitional and meaningful differences between these approaches, there is a common, reasoned solution with desirable properties. The resulting standard error estimate are well-known as “linearized” standard errors in the survey literature. We refer to them as “residualized” standard errors, which provides a more general and intuitively descriptive name. Approaching from either the design- or model-based settings, these can be estimated simply and in the same way. For the average treatment effect on all units (ATE), this can be obtained using robust standard errors from a weighted regression that additionally includes the (centered) balancing covariates, and their interactions with treatment. With adjustments to the weights and form of the estimator, these can be adapted to obtain variance estimates for the average treatment effect on the treated (ATT). These can further be extended for “superpopulation” inference using an adjustment to account for uncertainty in the covariate means due to Berk et al., 2013 ; Negi and Wooldridge, 2021 and Ding, 2023 . We show that these standard errors are smaller, but still conservative or correct under each resampling framework, producing nominal or higher coverage. Gains to precision compared to non-residualizing estimators (e.g. variance estimators built directly on the weighted difference in means or weighted outcome regression without covariates) can be quite significant, especially when the weights adjust for prognostic covariates. Our empirical application and simulations demonstrate reductions of 10-30% in the estimated standard errors, bringing them closer to empirical standard errors. The approach remains appropriate, though with different justifications, for weights that achieve exact balance (e.g. on covariate means) but also to inverse propensity score weighting, approximate balancing weights, or any other weighting procedure. 2 Background We focus on an observational setting, where researchers cannot control treatment assignment, and where it is important that they address potential bias induced by selection into treatment when estimating causal effects. Weighting estimators—in which researchers construct a set of weights that adjust the data, or a subset of the data, to look “representative” of a specified target distribution in varying senses described below—are a popular method for covariate adjustment. Our objective is to clarify what are appropriate variance estimators after weighting. To justify these choices, we find it necessary to clarify the target estimands and sources of sampling uncertainty relevant to researchers under different data-generating paradigms. We introduce some common notation here, with details specific to the data-generating process outlined in subsequent sections. We then introduce our primary weighted estimator for causal effects. Notation and assumptions We consider a binary treatment Z_{i}\in\{0,1\} and observed outcome Y_{i} with potential outcomes Y_{i}(1) and Y_{i}(0) , and covariates \textbf{X}_{i} . We assume a single version of treatment and no interference, collectively referred to as the Stable Unit Treatment Value assumption (SUTVA), with consistency holding ( Y_{i}(z)=Y_{i} for unit i taking treatment Z_{i}=z ) ( Rubin, 1980 ) . We will let n denote the sample size of our realized data, with n_{t}=\sum_{i}Z_{i} and n_{c}=\sum_{i}(1-Z_{i}) . Finally, we take causal quantities such as that average treatment effect to be the target of inference after weighting, which applies where investigators will argue for an absence of unobserved confounders, i.e. conditional ignorability or selection on observables (e.g. Rubin, 1974 ), \{Y(1),Y(0)\}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 5.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 5.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 5.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 5.0mu{\scriptscriptstyle\perp}}}Z\mid\textbf{X}\quad\quad\text{(conditional ignorability)} We require conditional ignorability in our analysis below