Efficient Estimation of Causal Effects Under Two-Phase Sampling with Error-Prone Outcome and Treatment Measurements
Keith Barnatchez, Kevin P. Josey, Nima S. Hejazi, Bryan E. Shepherd, Giovanni Parmigiani, Rachel C. Nethery · 2025
Read Efficient Estimation of Causal Effects Under Two-Phase Sampling with Error-Prone Outcome and Treatment Measurements by Keith Barnatchez, Kevin P. Josey, Nima S. Hejazi, Bryan E. Shepherd, Giovanni Parmigiani, Rachel C. Nethery (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-21777.txt.
Introduction
Causal effects measured with error can still be estimated efficiently, Keith Barnatchez and co-authors show in a 2025 paper. Electronic health record data often mismeasure clinical outcomes and treatments. Researchers validate a subset by manual chart review, a form of two-phase sampling. Phase-two samples are often collected in a biased manner that depends on the initial, error-prone measurements, which complicates inference.
The team develops solutions for this specific problem and for the broader problem of causal inference with two-phase samples. Two asymptotically equivalent doubly-robust estimators of the average effect are constructed for the measurement error setting. Doubly-robust methods stay valid if one of two models is right. Readers in biostatistics get efficiency gains from a validation design they already use.
Catalog record
| CREATOR | Keith Barnatchez, Kevin P. Josey, Nima S. Hejazi, Bryan E. Shepherd, Giovanni Parmigiani, Rachel C. Nethery |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-21777
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Efficient Estimation of Causal Effects Under Two-Phase Sampling with Error-Prone Outcome and Treatment Measurements Keith Barnatchez, Kevin P. Josey, Nima S. Hejazi, Bryan E. Shepherd, Giovanni Parmigiani, Rachel C. Nethery Abstract Measurement error is a common challenge for causal inference studies using electronic health record (EHR) data, where clinical outcomes and treatments are frequently mismeasured. Researchers often address measurement error by conducting manual chart reviews to validate measurements in a subset of the full EHR data—a form of two-phase sampling. To improve efficiency, phase-two samples are often collected in a biased manner dependent on the patients’ initial, error-prone measurements. In this work, motivated by our aim of performing causal inference with error-prone outcome and treatment measurements under two-phase sampling, we develop solutions applicable to both this specific problem and the broader problem of causal inference with two-phase samples. For our specific measurement error problem, we construct two asymptotically equivalent doubly-robust estimators of the average treatment effect and demonstrate how these estimators arise from two previously disconnected approaches to constructing efficient estimators in general two-phase sampling settings. We document various sources of instability affecting estimators from each approach and propose modifications that can considerably improve finite sample performance in any two-phase sampling context. We demonstrate the utility of our proposed methods through simulation studies and an illustrative example assessing effects of antiretroviral therapy on occurrence of AIDS-defining events in patients with HIV from the Vanderbilt Comprehensive Care Clinic. Keywords— Doubly-robust, Influence function, HIV, Measurement error, Two-phase sampling 1 Introduction The use of large, administrative electronic health record databases to assess causal relationships has surged over the past few decades. These databases typically contain a vast number of subjects and covariates, offering the potential to address a wide range of scientific questions. However, despite their growing popularity, EHR databases present unique challenges for researchers focused on causal inference questions. Among these challenges is the presence of measurement error in key clinical variables, particularly in exposures and outcomes of interest, the two key components of any causal analysis. Statistical methods for addressing measurement error have a well-established literature in the parametric modeling tradition ( Carroll et al., 2006 ) and have gained attention more recently in causal inference ( Valeri, 2021 ; Barnatchez et al., 2024 ). A long-established design-based approach to addressing measurement error is to employ a two-phase sampling procedure ( Neyman, 1938 ) . In two-phase sampling designs, gold-standard measurements for error-prone variables are obtained for a small subset of the full data, effectively reframing the measurement error issue as a missing data problem ( Lotspeich et al., 2022 ) . This small validated subset is often referred to as the phase-two sample, while the full dataset is typically termed the phase-one sample. There is a growing body of work at the intersection of causal inference, measurement error, and missing data that accommodates sampling schemes of this nature ( Kallus and Mao, 2024 ; Kennedy, 2020 ; Levis et al., 2024a , e.g.,) . Most closely related to our work, recent developments from the semi-supervised learning literature explore variations of the measurement error problem, when the phase-two validation data is obtained completely at random but at a rate that decays to zero as the size of the overall dataset grows ( Hou et al., 2025 ) . While these methods serve as important advances in the careful use of EHR data for drawing causal inferences, several practical challenges remain unaddressed. We highlight two key challenges in this study. First, specific to our applied science motivation, there is a need for semi-parametric efficient methods in measurement error settings that explicitly account for biased phase-two sampling schemes. Biased sampling occurs when phase-two sampling probabilities depend on measured covariates and the error-prone measurements of the outcome and treatments, potentially resulting in a validation sample that is not representative of the target population from which the EHR data is drawn. Although such sampling requires careful statistical methodology, biased sampling rules can offer significant efficiency gains for downstream estimation tasks compared to simple random validation sampling ( Breslow and Chatterjee, 1999 ) , especially in scenarios where the outcome and exposure of interest are rare. In practice, budgetary constraints often necessitate that only a small share of EHR data can be validated, meaning biased sampling rules can, in terms of efficiency of the resulting estimator, drastically increase the effective sample size of the phase-two data. The use of biased sampling schemes to address measurement error is well-studied in the context of parametric and semi-parametric models that aim to capture associations between variables ( Shepherd et al., 2023 ) , but is relatively underexplored in settings where interest lies in semi-parametric efficient estimation of causal effects defined non-parametrically. The second challenge applies more generally to the problem of performing semi-parametric efficient estimation under two-phase sampling schemes, of which our specific problem of interest is a special case. We document that there are two general approaches to performing causal inference under two-phase sampling schemes. In the first approach, the researcher uses a combination of unconfoundedness and missing-at-random assumptions to identify the causal quantity of interest as a statistical functional of the observed data distribution. They then use standard tools from semi-parametric theory to derive efficient estimators of this functional. This is undoubtedly the approach most commonly taken to derive asymptotically efficient estimators in causal inference applications and has been taken to address several problems arising at the intersection of causal inference and missing data ( Kennedy, 2020 ; Levis et al., 2024b ) Alternatively, the second approach leverages links between the observed data distribution, and the underlying complete data distribution one would have access to if it were possible to observe gold-standard measurements for every subject ( Rose and van der Laan, 2011 ; Hejazi et al., 2021 ; Wang et al., 2023 ) . We document that the construction of semi-parametric efficient estimators tends to be drastically simplified under the second approach. While the estimators arising from both approaches will be asymptotically equivalent in general two-phase sampling problems, there is no consensus on how their behavior differs in finite samples. This issue is further compounded by the fact that in practice, researchers tend to implement one of the two approaches without carefully considering the merits of the other. Further understanding of the finite-sample behavior of estimators arising from each approach is crucial, since validated sample sizes will tend to be small in practice. In this paper, we address these two challenges by deriving semi-parametric efficient estimators of counterfactual mean outcomes under settings where (1) the outcome and exposure are measured with error, and (2) the researcher is able to collect gold-standard measurements for the outcome and exposure for a small but biased subsample of the overall dataset. We present two classes of estimators, one arising from each of the two general approaches to performing semi-parametric causal inference under two-phase sampling described above. Through simulations and a study based on data from the Vanderbilt Comprehensive Care Clinic (VCCC) we show that while these estimators are asymptotically equivalent, their behavior can yield meaningfully different results in finite samples and in settings where the phase-two sample is relatively small. To address challenges that typically arise with small amounts of