Debiased Ill-Posed Regression
AmirEmad Ghassami, James M. Robins, Andrea Rotnitzky · 2025
Read Debiased Ill-Posed Regression by AmirEmad Ghassami, James M. Robins, Andrea Rotnitzky (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-20787.txt.
Introduction
AmirEmad Ghassami, James Robins and Andrea Rotnitzky wrote a 2025 statistics paper on estimating a function known only through a moment condition. Such problems arise when teasing out cause and effect with instruments or proxy data. The usual route minimizes a projected error measure. That route is fragile when helper estimates are off or slow to settle.
Their debiased method builds on the influence function of a tweaked error. Its bias is second order in the helper errors, and it tolerates one helper being wrong. A tuning value is picked by cross-checks. The team bounds the cost of that choice and studies plain parameter cases.
Catalog record
| CREATOR | AmirEmad Ghassami, James M. Robins, Andrea Rotnitzky |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 138 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-20787
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Debiased Ill-Posed Regression AmirEmad Ghassami, James M. Robins, Andrea Rotnitzky Abstract In various statistical settings, the goal is to estimate a function which is restricted by the statistical model only through a conditional moment restriction. Prominent examples include the nonparametric instrumental variable framework for estimating the structural function of the outcome variable, and the proximal causal inference framework for estimating the bridge functions. A common strategy in the literature is to find the minimizer of the projected mean squared error. However, this approach can be sensitive to misspecification or slow convergence rate of the estimators of the involved nuisance components. In this work, we propose a debiased estimation strategy based on the influence function of a modification of the projected error and demonstrate its finite-sample convergence rate. Our proposed estimator possesses a second-order bias with respect to the involved nuisance functions and a desirable robustness property with respect to the misspecification of one of the nuisance functions. The proposed estimator involves a hyper-parameter, for which the optimal value depends on potentially unknown features of the underlying data-generating process. Hence, we further propose a hyper-parameter selection approach based on cross-validation and derive an error bound for the resulting estimator. This analysis highlights the potential rate loss due to hyper-parameter selection and underscore the importance and advantages of incorporating debiasing in this setting. We also study the application of our approach to the estimation of regular parameters in a specific parameter class, which are linear functionals of the solutions to the conditional moment restrictions and provide sufficient conditions for achieving root-n consistency using our debiased estimator. Keywords: Ill-Posedness; Debiased Estimator; Influence Function-Based Estimation; Hyper-Parameter Selection; Nonparametric Instrumental Variable; Proximal Causal Inference 1 Introduction In many applications in statistics, particularly in causal inference, given a random sample drawn from the unknown distribution of a random vector V, we are interested in estimating a function h satisfying a conditional moment restriction of the form \mathbb{E}\left[g_{1}(V)h(V_{h})-g_{0}(V)\mid V_{q}\right]=0, (1) where V_{h},V_{q}\subseteq V , and g_{0} and g_{1} are known functions. If V_{h}=V_{q} , Equation ( 1 ) can be reformulated as a standard regression problem. Yet, we consider the more challenging case where V_{h}\neq V_{q} . In this case, Equation ( 1 ) is an ill-posed integral equation ( Kress,, 2013 ) . We refer to the task of estimating a function h that solves this equation as ill-posed regression . Two of the most prominent applications of ill-posed regression in the filed of causal inference are found in instrumental variable (IV) and in proximal causal inference methodologies. In both cases, the goal is to perform causal inference in settings where unobserved confounders in the treatment-outcome relationship cannot be ruled out. IV methods require the existence of an auxiliary variable, called an instrumental variable, which only affects the outcome variable through the treatment variable and does not share an unobserved confounder with the outcome variable. A particularly popular IV-based model is the nonparametric IV (NPIV) model ( Newey and Powell,, 2003 ; Hall and Horowitz,, 2005 ; Darolles et al.,, 2011 ; Florens et al.,, 2011 ; Chen and Pouzo,, 2012 ) . This model formalizes the aforementioned IV conditions as follows. Let W\in\mathcal{W} and Y\in\mathcal{Y} be the treatment and the outcome variables, respectively. The NPIV model assumes that the outcome variable Y is generated from a structural function of the treatment variable, h_{0}(W) , with additive noise. That is, the outcome is generated as Y=h_{0}(W)+\epsilon, where \epsilon is a mean-zero error term. Interest is in estimation of the unknown function h_{0} based on random sample from the target population, when W is not assumed to be exogenous, and hence \mathbb{E}[\epsilon\mid W] is not assumed to be equal to zero. Therefore, \mathbb{E}[Y\mid W=\cdot] is not necessarily equal to the structural function h_{0}\left(\cdot\right) . Instead, in the NPIV model, we assume existence of an instrumental variable Z\in\mathcal{Z} , for which we have \mathbb{E}[\epsilon\mid Z]=0 . That is, h_{0} satisfies the conditional moment restriction \mathbb{E}[h(W)-Y\mid Z]=0. Estimation of h_{0} is a special case of the introduced ill-posed regression with V_{q}=\{Z\} , V_{h}=\{W\} , g_{1}(V)=1 , and g_{0}(V)=Y . In words, this conditional moment restriction requires that after projecting onto the space of functions of Z , random variables Y and h_{0}(W) have the same projection. This requirement implies that the IV affects the outcome variable only through the treatment variable. Proximal causal inference methods ( Miao et al.,, 2018 ; Tchetgen Tchetgen et al.,, 2024 ; Cui et al.,, 2023 ) allow for the presence of unobserved confounders but require access to two proxy variables of the confounder, which satisfy certain statistical conditions. Consider, for example, the goal of estimating the mean of an outcome of interest in the counterfactual world in which all subjects in the population receive the same treatment, based on a random sample of \left(A,X,Y,Z,W\right) , where A=1 if the subject received the treatment of interest and A=0 otherwise, X is a vector of observed baseline covariates, Z and W are the treatment and outcome proxy variables (defined in Section 4.1 ), and Y is the outcome variable. Under the proximal causal inference assumptions (detailed in Section 4.1 ), the counterfactual mean under treatment A=a , for a\in\{0,1\} , is equal to \mathbb{E}\left[h_{0}\left(W,X\right)\right] where h_{0} solves the conditional moment equation \mathbb{E}[I(A=a)h(W,X)-I(A=a)Y\mid Z,A,X]=0. Estimation of h_{0} is another special case of ill-posed regression with V_{q}=\{Z,A,X\} , V_{h}=\{W,X\} , g_{1}(V)=I(A=a) , and g_{0}(V)=I(A=a)Y . Under slightly different assumptions, Cui et al., (2023) , showed that the counterfactual mean is equal to the mean of YI(A=a)q_{0}\left(Z,X\right) , where q_{0}(Z,X) solves the conditional moment equations \mathbb{E}\left[I(A=a)q(Z,A,X)-1\mid W,X\right]=0, which is another special case of ill-posed regression with V_{q}=\{W,X\} , V_{h}=\{Z,A,X\} , g_{1}(V)=I(A=a) , and g_{0}(V)=1 . The preceding examples illustrate the important role of ill-posed regression in causal inference, and motivates the development of solutions for this task. For a function h , define \mathbb{E}[\mathbb{E}[g_{1}(V)h(V_{h})-g_{0}(V)\mid V_{q}]^{2}] as the projected error of h . Because the solution h_{0} to Equation ( 1 ) minimizes the projected error over all functions h , a common approach in the literature for ill-posed regression is to estimate h_{0} with a minimizer of an estimate of (a regularized version of) the projected error. Therefore, the more precise the estimator of the projected error is, the better the quality of the estimated structural function, and hence, improving the estimator should be considered. Formalizing this idea is one of the main goals in this work. In this paper, we propose a novel strategy for ill-posed regression which is based on minimizing a debiased estimator of the projection error. We perform the debiasing step using techniques from the modern semiparametric theory, specifically, by leveraging the influence function of the projected error. The technique of influence function-based debiasing has recently gained significant attention in the literature of causal inference and missing data ( van der Laan and Robins,, 2003 ; Van der Laan and Rose,, 2011 ; Chernozhukov et al.,, 2018 ) , starting from the seminal work of Robins et al., (1994) . For the instrumental variable setting, compared to existing work, our proposed method requires the estimation of an extra nuisance function, which is used for debiasing the original estimator of the projected error. We demonstrate