Does Residuals-on-Residuals Regression Produce Representative Estimates of Causal Effects?
Apoorva Lal, Winston Chou · 2025
Read Does Residuals-on-Residuals Regression Produce Representative Estimates of Causal Effects? by Apoorva Lal, Winston Chou (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07462.txt.
Introduction
Apoorva Lal and Winston Chou question a popular method for estimating cause and effect in a twenty twenty-five paper. The method is double machine learning, and its simplest form regresses residuals on residuals. It is fast and easy, so it is widely used on observational data.
The pair shows a catch. When treatment comes in doses, not just yes or no, the method averages effects with odd weights, at dose levels that may not even appear in the data. So its number need not match the true average effect. They propose a new estimator for that average and test it on toy and Netflix data. Their method is now the default in Netflix's causal platform.
Catalog record
| CREATOR | Apoorva Lal, Winston Chou |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07462
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Does Residuals-on-Residuals Regression Produce Representative Estimates of Causal Effects? Apoorva Lal, Winston Chou Abstract . Double Machine Learning is commonly used to estimate causal effects in large observational datasets. The “residuals-on-residuals” regression estimator (RORR) is especially popular for its simplicity and computational tractability. However, when treatment effects are heterogeneous, the proper interpretation of RORR may not be well understood. We show that, for many-valued treatments with continuous dose-response functions, RORR converges to a conditional variance-weighted average of derivatives evaluated at points not in the observed dataset, which generally differs from the Average Causal Derivative (ACD). Hence, even if all units share the same dose-response function, RORR does not in general converge to an average treatment effect in the population represented by the sample. We propose an alternative estimator suitable for large datasets. We demonstrate the pitfalls of RORR and the favorable properties of the proposed estimator in both an illustrative numerical example and an application to real-world data from Netflix. 1. Introduction Double Machine Learning (DML) ( Bickel et al. 1993 ; Robins et al. 1994 ; Chernozhukov et al. [n. d.] ) is fast becoming the standard for estimating causal effects in large, high-dimensional datasets under conditional ignorability assumptions, which stipulate that the treatment is as good as randomly assigned given observed covariates ( Imbens [n. d.] ) . To strengthen such assumptions, researchers in a wide variety of fields use the DML method to condition on many covariates without strong commitments to functional form ( Jares and Malhotra 2023 ; Chernozhukov et al. 2020 ; Holtz et al. 2020 ) . While DML encompasses a large family of methods, this paper focuses on the Partially Linear Model (PLM), which relates an outcome Y_{i} to a (continuous- or discrete-valued) treatment T_{i} conditional on pretreatment covariates X_{i} as follows: Y_{i}=\theta T_{i}+g(X_{i})+e_{i}\quad\text{and}\quad T_{i}=h(X_{i})+u_{i}. The PLM imposes very weak assumptions on how the treatment and outcome relate to the covariates. It also motivates an intuitive two-step DML estimator of \theta , the residuals-on-residuals regression (RORR). RORR, a natural extension of the Frisch-Waugh-Lovell theorem, involves first “partialing out” the effect of X_{i} using flexible machine learning methods, then forming the residuals: \widetilde{Y}_{i}=Y_{i}-\widehat{g}(X_{i})\quad\text{and}\quad\widetilde{T}_{i}=T_{i}-\widehat{h}(X_{i}), and lastly regressing \widetilde{Y}_{i} on \widetilde{T}_{i} to obtain the estimate \hat{\theta} ( Robinson [n. d.] ) . When the treatment effect \theta is the same for all units in the population, it is also the Average Treatment Effect (ATE) for binary treatments, the Average Causal Derivative (ACD) for continuous treatments, and the Average Incremental Effect (AIE) for integer-valued treatments. Alternatively, we study the probability limit (plim) and interpretation of \hat{\theta} when treatment effects are heterogeneous. Under such heterogeneity, \hat{\theta} converges to a conditional variance-weighted average of causal effects, which puts greater weight on units whose treatment values are less predictable. For example, when X is discrete, the RORR estimand places the most weight on the treatment effects in the strata where the treatment has the most variance ( Angrist [n. d.] ; Aronow and Samii [n. d.] ; Słoczyński [n. d.] ) . When treatments are many-valued (for example, continuous), RORR may be subject to other, more subtle biases. In this paper, we provide a general analysis of RORR with binary and many-valued treatments. We demonstrate the empirical relevance of these biases both in a stylized numerical example and with real-world data from Netflix. Lastly, we propose an alternative estimator that utilizes Augmented Inverse Propensity Weighting (AIPW) along with binning the treatment ( Cattaneo 2010 ) and demonstrate its favorable properties theoretically and in our empirical application. 2. Residuals-on-Residuals Regression with Treatment Effect Heterogeneity RORR has many favorable attributes that make it attractive in empirical applications. As mentioned above, it enables highly flexible estimation of the nuisance functions. Furthermore, RORR has the doubly-robust property, converging to the true \theta if either the treatment model or the outcome model is correct. When both models are correct, it is also the efficient estimator ( Dukes et al. 2024 ) . Recent applications of RORR include studies in economics ( Baiardi and Naghi 2024 ) , ecology ( Fink et al. 2023 ) , and public health ( Wei et al. 2024 ) . In commercial settings – characterized by large datasets, short timelines, and stakeholders with diverse technical backgrounds – RORR is also used for practical reasons. For example, the final regression step is computationally efficient, as only a small number of statistics is needed to compute the final estimate of \theta . Moreover, the recipe of (1) removing variation explainable by pretreatment covariates and (2) estimating the effect of the remaining exogenous variation in T_{i} on Y_{i} is intuitive and easy to explain to non-experts. However, this simplicity can come at a cost: The interpretation of \theta as an “average” treatment effect (whether the ATE, ACD, or AIE) depends on the assumption of a homogeneous treatment effect embedded in the PLM, which may not hold in applications. In this section, we discuss the interpretation of \hat{\theta} under two common violations of this assumption: binary treatments with heterogeneous effects across individuals and many-valued treatments with continuous dose-response functions. 2.1. Binary Treatments with Heterogeneous Treatment Effects We begin by studying the plim of the RORR estimator for binary treatments with heterogeneous treatment effects. Letting T_{i}\in\{0,1\} denote the binary treatment indicator, we consider the model: (1) \displaystyle Y_{i} \displaystyle= \displaystyle\theta_{i}T_{i}+g(X_{i})+e_{i} \displaystyle T_{i} \displaystyle= \displaystyle h(X_{i})+u_{i}, where \theta_{i} is a individual treatment effect. We further assume that the errors are conditionally exogenous and uncorrelated: E[e_{i}|X_{i}]=0 , E[u_{i}|X_{i}]=0 , and E[e_{i}u_{i}|X_{i}]=0 . This is a linear instantiation of conditional ignorability. Note that this assumption implies that g(X_{i})=E[Y_{i}-\theta_{i}T_{i}|X_{i}] and h(X_{i})=E[T_{i}|X_{i}] . We also assume that \theta_{i} is conditionally independent of T_{i} given X_{i} , \theta_{i}\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern 5.0mu{\displaystyle\perp}}}{\mathrel{\hbox to0.0pt{$\textstyle\perp$\hss}\mkern 5.0mu{\textstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptstyle\perp$\hss}\mkern 5.0mu{\scriptstyle\perp}}}{\mathrel{\hbox to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern 5.0mu{\scriptscriptstyle\perp}}}T_{i}|X_{i} . We consider the plim of the OLS regression of Y_{i}-g(X_{i}) on T_{i}-h(X_{i}) with observations O_{i}=(Y_{i},T_{i},X_{i}) , i=1,\ldots N . We assume that the observations O_{1},\ldots,O_{N}\sim O are independent and identically distributed. While g and h must be estimated in practice, for ease of exposition, we assume consistent estimators for these and focus on the (true) limiting g and h . 1 1 1 Researchers typically use extremely flexible function classes for g and h (e.g., gradient boosted trees or deep neural networks) that are able to approximate the true nuisance functions arbitrarily closely. First observe that: (2) \displaystyle\hat{\theta} \displaystyle\overset{p}{\to} \displaystyle\frac{E[(T_{i}-h(X_{i}))(Y_{i}-g(X_{i}))]}{E[(T_{i}-h(X_{i}))^{2}]} \displaystyle= \displaystyle\frac{E[\theta_{i}(T_{i}^{2}-T_{i}h(X_{i}))]}{E[(T_{i}-h(X_{i}))^{2}]}. Using the fact that T_{i} is binary and applying the law of iterated expectations, we can rewrite this as: (3) \displaystyle\frac{E[\theta_{i}(T_{i}^{2}-T_{i}h(X_{i}))]}{E[(T_{i}-h(X_{i}))^{2}]} \displaystyle= \displaystyle\frac{E[\theta_{i}E[T_{i}(1-h(X_{i}))|X_{i}]]}{E[(T_{i}-h(X_{i}))^{2}]} (4) \displaystyle= \displaystyle\frac{E[\theta_{i}(T_{i}-h(X_{i}))^{2}]}{E[(T_{i}-h(X_{i}))^{2}]}, where Equation ( 3 ) follows from \theta_{i} being conditionally independent of T_{i} . This demonstrates the known result that, with a binary treatment, linear regression converges to a conditional variance-weighted average of individual treatment effects ( Angrist [n. d.] ; Aronow and Samii [n. d.] ) . For an intuitive restatement, denote the conditional variance weights by \omega_{i}:=\frac{(T_{i}-h(X_{i}))^{2}}{E[(T_{i}-h(X_{i}))^{2}]} and note that E[\omega_{i}]=1 by construction. Then the bias of the RORR plim (which we will denote by \tilde{\theta} ) with respect to the ATE can be written as: (5) \displaystyle\tilde{\theta}-E[\theta_{i}] \displaystyle= \displaystyle E[\omega_{i}\theta_{i}]-E[\theta_{i}] (6) \displaystyle= \displaystyle\operatorname{Cov}(\omega_{i},\theta_{i}). In other words, the bias of RORR for binary treatments when treatment effects are heterogeneous is equal to the covariance