Residual Balancing for Non-Linear Outcome Models in High Dimensions

Isaac Meza Lopez Department of Economics, Harvard University [email protected] · 2025

Read Residual Balancing for Non-Linear Outcome Models in High Dimensions by Isaac Meza Lopez Department of Economics, Harvard University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00324.txt.

Introduction

Isaac Meza extends a causal inference method to a harder setting. Approximate residual balancing estimates average treatment effects by weighting cases so treated and control groups compare fairly. The original framework handled linear outcome models, leaving open the problem posed by Athey and co-authors in 2018. What happens when outcomes follow a nonlinear model, as in generalized linear models for counts or binary results?

Meza derives a new bias breakdown for the nonlinear case and shows a second-order correction is needed, to account for the curve of the link function. New balancing weights control both first- and second-order bias sources. He proves the estimator stays consistent at the standard root-n rate and settles into a normal distribution, under standard high-dimensional assumptions.

Catalog record

CREATORIsaac Meza Lopez Department of Economics, Harvard University [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00324

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Residual Balancing for Non-Linear Outcome Models in High Dimensions Isaac Meza Lopez Department of Economics, Harvard University [email protected] Abstract We extend the approximate residual balancing (ARB) framework to nonlinear models, answering an open problem posed by ( Athey et al., 2018 ) . Our approach addresses the challenge of estimating average treatment effects in high-dimensional settings where the outcome follows a generalized linear model. We derive a new bias decomposition for nonlinear models that reveals the need for a second-order correction to account for the curvature of the link function. Based on this insight, we construct balancing weights through an optimization problem that controls for both first and second-order sources of bias. We provide theoretical guarantees for our estimator, establishing its n \sqrt{n} -consistency and asymptotic normality under standard high-dimensional assumptions. 1 Introduction In observational studies, practitioners often rely on the unconfoundedness assumption, which posits that treatment assignments are as good as random given observed covariates. This assumption is at the core of many popular methods for causal inference, including regression adjustment, matching, propensity score weighting, and doubly robust methods ( Rosenbaum & Rubin, 1983 ; Imbens & Rubin, 2015 ; Hirano et al., 2003 , e.g.,) . These methods, when applied in low-dimensional settings, allow for the estimation of Average Treatment Effects (ATEs) with attractive statistical properties, including asymptotic normality and efficiency. However, modern datasets often feature a high-dimensional structure, where the number of covariates p can exceed the sample size n . High-dimensional covariates arise both from richer data collection processes and deliberate expansions of low-dimensional features (e.g., interactions, polynomial expansions, fixed effects). While high-dimensional data can mitigate issues of unmeasured confounding by capturing more covariate variation, they also pose significant challenges for causal inference. In particular, classical approaches that rely on p -fixed asymptotics may no longer yield valid inferences, and new techniques are required to handle the biases introduced by high-dimensional regularization methods. This has motivated a new line of research focused on developing robust methods for causal inference in high-dimensional environments. Recent developments in high-dimensional causal inference have adapted classical methods to this regime. One prominent approach is the Approximate Residual Balancing (ARB) framework introduced by ( Athey et al., 2018 ) , which combines regularized regression adjustments with balancing weights. ARB demonstrates that, in high-dimensional linear models, it is possible to achieve \sqrt{n} -consistent and asymptotically normal inference for the ATE by constructing approximately balancing weights and correcting for residual confounding. This framework eliminates the need for consistent propensity score estimation and focuses directly on addressing linear biases in the outcome model. While ARB provides strong theoretical guarantees and practical utility for linear models, many real-world applications involve nonlinear relationships between covariates and outcomes. Generalized Linear Models (GLMs) are a natural extension, allowing for nonlinear link functions \psi(\cdot) that capture more complex outcome structures. In such settings, the standard ARB framework is insufficient, as it does not account for the non-linearities introduced by the link function and its derivatives. Recognizing this limitation, ( Athey et al., 2018 ) noted the extension to GLMs as a crucial direction for future research. This paper directly addresses that challenge. We develop a principled extension of the ARB methodology to nonlinear outcome models. We show that a naive application of ARB fails in the nonlinear case because it ignores the curvature of the link function. Our key insight is that controlling for estimation bias requires a second-order correction. This leads to a new optimization problem for constructing balancing weights that simultaneously controls for imbalances in both the covariates and their second moments, weighted by the first and second derivatives of the link function, respectively. Our main contributions are: 1. A bias decomposition for nonlinear models: We derive a fundamental bias-variance decomposition for the treatment effect estimator in the presence of a nonlinear link function. This decomposition explicitly characterizes the bias in terms of first and second-order residual imbalances. 2. A second-order balancing procedure: Based on our decomposition, we propose a new optimization framework for constructing balancing weights that effectively eliminates asymptotic bias without requiring a correctly specified propensity score model. 3. Complete theoretical guarantees: We prove that under standard sparsity and regularity conditions, our proposed estimator is \sqrt{n} -consistent and asymptotically normal, providing a valid basis for statistical inference on treatment effects in high-dimensional nonlinear models. 1.1 Related work Our work builds on several streams of literature in high-dimensional statistics and causal inference. High-Dimensional Causal Inference. The primary challenge in high-dimensional causal inference is obtaining estimators that are not only consistent but also asymptotically normal, enabling the construction of valid confidence intervals. Two main paradigms have emerged to address this. The first is the rate double robustness paradigm, where an estimator relies on both sparsity in the outcome regression and propensity score model, remaining consistent if at least one is correctly specified. The debiased or double machine learning (DML) framework is a leading example, providing a general recipe for constructing estimators insensitive to first-order errors in these nuisance components ( Belloni et al., 2014 ; Farrell, 2015 ; Chernozhukov et al., 2017 ) . While methods relying on sparsity double robustness like those in ( Bradic et al., 2019 ; Ning et al., 2020 ) can accommodate a nonlinear propensity score model (e.g., a GLM for a binary treatment), their framework is fundamentally different from ours. Theirs is an approach that requires estimating both an outcome model and a propensity score model. By contrast, we purposely avoid propensity estimation and achieve orthogonality only with respect to the outcome bias via link-sensitive balancing. In nonlinear links, the Taylor remainder contains a \psi^{\prime\prime} weighted quadratic term that does not vanish under first-order balance alone; consequently, we introduce a second-order guardrail (or impose a beta-min regime with support recovery) to render the remainder o_{p}\left(n^{-1/2}\right) . Methodologically, hdCBPS enforces balance through PS estimating equations, whereas our weights directly balance cross-group moments tied to the link’s derivatives (e.g., X^{\top}\psi^{\prime}\left(X\hat{\beta}_{c}\right) ), reflecting our outcome-centric residual-balancing perspective. Our work is located in the second paradigm, which achieves consistency by focusing on sparsity in a single nuisance component ( model double robustness ) but under stronger sparsity restrictions. We provide valid inference under a sparse outcome model without requiring any specification or estimation of the propensity score model. Balancing and Calibration Weights. A parallel line of research constructs weights to directly balance covariates, without relying on a correctly specified propensity model. Entropy balancing calibrates weights to exactly match prespecified moments via a KL projection ( Hainmueller, 2012 ) . Stable balancing weights minimize dispersion subject to exact balance constraints, controlling variance while enforcing mean balance ( Zubizarreta, 2015 ) . Empirical balancing calibration weighting achieves semiparametric efficiency within a calibration family by solving a global moment - calibration problem ( Chan et al., 2016 ) . Kernel balancing extends balance to rich function classes by matching RKHS means ( Hazlett, 2020 ) . Generalized optimal matching (and KOM) unifies matching and weighting as optimization problems that trade off imbalance against variance ( Kallus, 2020 ) - finding weights that minimize more general measures of distributional discrepancy. Propensity - based balancing approaches estimate the score to satisfy balance moments, e.g., CBPS ( Imai & Ratkovic, 2014 )

Read the complete text