Cross-Validated Causal Inference: a Modern Method to Combine Experimental and Observational Data
Xuelin Yang Department of Statistics, UC Berkeley Licong Lin Department of Statistics, UC Berkeley Susan Athey Graduate School of Business, Stanford University Michael I. Jordan Department of Statistics, UC Berkeley Guido W. Imbens Graduate School of Business, Stanford University · 2025
Read Cross-Validated Causal Inference: a Modern Method to Combine Experimental and Observational Data by Xuelin Yang Department of Statistics, UC Berkeley Licong Lin Department of Statistics, UC Berkeley Susan Athey Graduate School of Business, Stanford University Michael I. Jordan Department of Statistics, UC Berkeley Guido W. Imbens Graduate School of Business, Stanford University (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00727.txt.
Introduction
Xuelin Yang, Licong Lin, Susan Athey, Michael Jordan, and Guido Imbens develop a way to combine randomized experiments with observational data when estimating cause and effect. Trials are trustworthy but small and costly. Observational data are larger and cheaper, yet hidden factors can bias them.
Their framework treats estimation as risk minimization, fitting one model that balances a loss from the experiment against a loss from the observational data. Cross validation across the experimental folds chooses how much weight each source receives, judged by the causal parameter itself. Tests on real and synthetic data show the method works reliably, and the authors prove non-asymptotic error bounds for the estimates.
Catalog record
| CREATOR | Xuelin Yang Department of Statistics, UC Berkeley Licong Lin Department of Statistics, UC Berkeley Susan Athey Graduate School of Business, Stanford University Michael I. Jordan Department of Statistics, UC Berkeley Guido W. Imbens Graduate School of Business, Stanford University |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 153 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-00727
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Cross-Validated Causal Inference: a Modern Method to Combine Experimental and Observational Data Xuelin Yang Department of Statistics, UC Berkeley Licong Lin Department of Statistics, UC Berkeley Susan Athey Graduate School of Business, Stanford University Michael I. Jordan Department of Statistics, UC Berkeley Guido W. Imbens Graduate School of Business, Stanford University Abstract We develop new methods to integrate experimental and observational data in causal inference. While randomized controlled trials offer strong internal validity, they are often costly and therefore limited in sample size. Observational data, though cheaper and often with larger sample sizes, are prone to biases due to unmeasured confounders. To harness their complementary strengths, we propose a systematic framework that formulates causal estimation as an empirical risk minimization (ERM) problem. A full model containing the causal parameter is obtained by minimizing a weighted combination of experimental and observational losses—capturing the causal parameter’s validity and the full model’s fit, respectively. The weight is chosen through cross-validation on the causal parameter across experimental folds. Our experiments on real and synthetic data show the efficacy and reliability of our method. We also provide theoretical non-asymptotic error bounds. Keywords: Causal inference; treatment effect; data integration; unobserved confounders, randomized experiments, observational data. 1 Introduction We focus on the problem of estimating the average treatment effect (ATE) in causal inference. Over the past decades, a wide range of statistical methods have been developed to draw causal conclusions from either experimental or observational studies. Experimental data, collected from randomized controlled trials (RCTs), offer high internal validity. However, such data can be costly to obtain. In contrast, observational data are often cheaper, but their internal validity is suspect. Specifically, ATE estimates based on observational data, assuming unconfoundedness, may suffer from biases due to unobserved confounders. In this paper we consider the combination of experimental and observational data, with the goal of producing robust (to the presence of unobserved confounders) and precise (by including observational data) causal conclusions. We propose a framework that minimizes a weighted combination of losses: the experimental loss, which assesses the causal parameter’s validity; the observational loss, which measures the full model’s fit; and their relative weighting, chosen adaptively via cross-validating the causal parameter. To illustrate the basic ideas, consider a setting with no covariates. We have an experimental sample where we observe both treated and control units, and an observational sample where we observe only control units, based on the widely used LaLonde data ( lalonde1986evaluating ; dehejia1999causal ) . Because in this setting there is no question about estimating the average outcome for the treated, for which we only have the experimental data, the question is how to estimate the average control outcome for the experimental population, \mathbb{E}[Y_{i}^{\mathrm{exp}}(C)] . The average control outcome in the experimental sample, \overline{Y}_{C}^{\mathrm{exp}} , is unbiased for this expectation (but possibly imprecise due to limited data size). The average of the control outcome in the observational sample, \overline{Y}_{C}^{\mathrm{obs}} , may be biased for the experimental population’s average control outcome. We consider a weighted average of the average control outcome in the observational sample and the average of the control outcome in the experimental sample, with weights \lambda\in[0,1] and 1-\lambda respectively: \displaystyle\widehat{\theta}(\lambda)=(1-\lambda)\overline{Y}_{C}^{\mathrm{exp}}+\lambda\overline{Y}_{C}^{\mathrm{obs}}. (1) What properties would we like \lambda to have? If the experimental sample is large, then even if the bias in the observational sample is very small, as long as there is some bias we would like \lambda to be close to zero. If on the other hand the bias in the observational sample is negligible, then we would like to choose \lambda close to one. In other words, we would like to shrink our experimental estimate towards the observational data, but do so in a data-adaptive fashion, that is, with a data-driven \lambda . In this simple no-covariate case where the focus is on the expected control outcome in the experimental population, we implement this objective by selecting \lambda through cross-validating on the experimental data: \widehat{\lambda}=\arg\min_{\lambda\in[0,1]}\underbrace{\frac{1}{K}\sum_{k=1}^{K}\Big(\overline{Y}_{C,B_{k}}^{\mathrm{exp}}-\Big((1-\lambda)\overline{Y}^{\mathrm{exp}}_{C,-B_{k}}+\lambda\overline{Y}_{C}^{\mathrm{obs}}\Big)\Big)^{2}}_{\mathsf{CV}(\lambda),\text{ the cross-validation objective}}, where the subscripts \{B_{k},-B_{k}\}_{k\in[K]} denote the complementary subsets in K -fold cross-validation. In the paper we extend this to the case with more general models for the observational data involving covariates. In Figure 1 , we present some results for this example based on the LaLonde data ( lalonde1986evaluating ; dehejia1999causal ) . In the bottom two panels we present two sets of three estimates of the ATE. First, in both panels, results based on the experimental data alone (corresponding to \lambda=0 ). Second, again in both panels, results based on the observational data alone (corresponding to \lambda=1 ). Both are intended to set the stage for our preferred results based on the cross-validated \hat{\lambda} . The cross-validation is based on five fold splits, leading to a unique \hat{\lambda} . We repeat this many times to get a distribution of selected \hat{\lambda}. In the case without covariates, we find that the selected \hat{\lambda} is always close to or exactly equal to 0, corresponding to the experimental estimates. The cross-validation makes clear that the the data can tell us that the observational data are of little value in this case. For a covariate-adjusted version of the observational data estimator, the cross-validated \hat{\lambda} is much closer to 1, with the average value for \hat{\lambda} over many choices of five folds equal to 0.77. Here the data imply that the observational data are valuable. The combination of the two sets of results shows that in this case our proposed method can detect when the observational sample is valuable, and when it is not, in a fully data-driven way. (a) (b) Figure 1 : Cross-Validated Causal Inference (CVCI) using \lambda . Top panel: selection of \lambda via the cross-validation objective \mathsf{CV}(\lambda) . The curve shows the average of \mathsf{CV}(\lambda) over 5000 runs, and the blue dashed line shows the average selected \widehat{\lambda} . Bottom panel: ATE estimates for different \lambda . PSID control group. We provide the setup, a discussion, and results for the CPS control group in Section 10 . Our contributions are three-fold: First, we introduce a novel method to systematically combine experimental and observational data. The methodological advantages include: (i) we do not require additional model specifications or identification assumptions; (ii) the method allows for the setting where observational and experimental samples have different covariates; (iii) the method allows the treatment to have a different effect in two populations. Second, we conduct experiments on both synthetic and real data to show the effectiveness of our method. For synthetic data, we address the most common cases (no-covariate and covariate-adjusted linear setting). For real data, we use the LaLonde-Dehejia-Wahba dataset lalonde1986evaluating ; dehejia1999causal . Third, we develop supporting non-asymptotic theories for the robustness of our method. Under regularity conditions, we show that our method achieves an \mathcal{O}(1/{N^{{\mathrm{exp}}}}) error rate regardless of the level of bias in the observational data, where {N^{{\mathrm{exp}}}} is the experimental sample size (Corollary 2 ). This is known to be optimal for estimators that are based solely on experimental data. Moreover, in the no-covariate setting, we show that the \mathcal{O}(1/{N^{{\mathrm{exp}}}}) -rate is minimax optimal (when the observational data are unbiased) over a class of robust estimators that combine experimental and observational data (Theorem 3 ). 2 Related Work We choose the weight for the