Fast and robust invariant generalized linear models
Parker Knight, Ndey Isatou Jobe, Rui Duan · 2025
Read Fast and robust invariant generalized linear models by Parker Knight, Ndey Isatou Jobe, Rui Duan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02611.txt.
Introduction
Parker Knight, Ndey Isatou Jobe, and Rui Duan published a 2025 statistics paper on finding effects that stay the same across different datasets. Prediction models often fail when moved between hospitals or populations, because most patterns shift. A few influences, though, may act identically everywhere. Models built on those invariant features could travel safely. Finding them has been slow or fragile in earlier methods.
The authors present a framework that estimates invariant effects quickly and under flexible assumptions. A robust extension guards against a corrupted or misspecified data source, so one bad dataset cannot poison the rest. Simulations confirm the protection works. They then build a transferable prediction model for a health endpoint as a demonstration. The work targets precision health, where a model trained in one clinic must serve another. Invariant structure, they argue, is the part of the data worth trusting abroad.
Catalog record
| CREATOR | Parker Knight, Ndey Isatou Jobe, Rui Duan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-02611
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fast and robust invariant generalized linear models Parker Knight, Ndey Isatou Jobe, Rui Duan Abstract Statistical integration of diverse data sources is an essential step in the building of generalizable prediction tools, especially in precision health. The invariant features model is a new paradigm for multi-source data integration which posits that a small number of covariates affect the outcome identically across all possible environments. Existing methods for estimating invariant effects suffer from immense computational costs or only offer good statistical performance under strict assumptions. In this work, we provide a general framework for estimation under the invariant features model that is computationally efficient and statistically flexible. We also provide a robust extension of our proposed method to protect against possibly corrupted or misspecified data sources. We demonstrate the robust properties of our method via simulations, and use it to build a transferable prediction model for end stage renal disease using electronic health records from the All of Us research program. 1 Introduction Data integration has become a central challenge of contemporary biomedical research. As research networks spread across institutions and even geographical boundaries, practitioners need to think carefully about how to handle heterogeneity and similarity between diverse data sources. This problem is especially pertinent in precision medicine [ 1 ] . Distributional shifts between patient subgroups, manifesting themselves in the training and testing data, may severely impact downstream model performance, which in turn can adversely affect decision making at the patient level [ 2 ] . If statistical machine learning systems are to play a role in the clinic, statistical researchers must develop models and methods that enable domain experts to better understand and, in some cases, protect against discrepancies between sources of medical data. One such model that is growing in popularity is the invariant prediction or features model proposed by [ 3 ] . Here, the authors presume that the analyst is given data as outcome-feature pairs from a discrete set of sources, referred to as environments, that may correspond to different sub-populations of observations, experimental conditions, or any other source of heterogeneity. Letting \mathcal{E} denote this set of environments, the invariant features model assumes that there exists a subset S of features such that for any two environments e,f\in\mathcal{E} , we have \mathbb{E}[Y^{e}|X_{S}^{e}=x]=\mathbb{E}[Y^{f}|X_{S}^{f}=x] (1) where (Y^{e},X^{e})\in\mathbb{R}\otimes\mathbb{R}^{p} represents the outcome and feature vector drawn from environment e . The features in S are invariant with respect to the environments \mathcal{E} , and knowledge of S can be used to build prediction tools that generalize well to previously unseen environments satisfying ( 1 ). This model is particularly amenable to large-scale electronic health records data. Due to differences across healthcare systems such as in clinical workflows, patient demographics, as well as EHR systems implementations, many features derived from EHR may admit effects that are idiosyncratic to their source institution, and hence disease prediction models built using these features will be useless if applied to data from a new clinical setting [ 4 , 5 , 6 ] . Knowledge of a set of features satisfying ( 1 ) would greatly improve EHR-based phenotyping and disease screening tools across institutions, and lead researchers to a better understanding of disease symptoms. The interpretability and potential applicability of the invariant features model has inspired a litany of works exploring methods for estimating the conditional mean \mathbb{E}[Y^{e}|X_{S}^{e}] and the set S [ 7 , 8 , 9 , 10 , 11 , 12 ] . While the community has made fundamental strides in building tools for estimation under ( 1 ), existing methods are routinely hampered by computational infeasibility. For instance, the original Invariant Causal Prediction algorithm of [ 3 ] requires running a hypothesis test for each S\subset[p] , and the Environment Invariant Linear Least Squares estimator of [ 9 ] is defined as the solution to novel subset selection problem which even recent advancements in integer optimization [ 13 ] cannot solve efficiently. This computational cost totally prohibits these methods from being useful for reasonable data analysis tasks. Recent works such as [ 11 ] and [ 12 ] attempt to design computationally faster methods, but do so by imposing stricter assumptions on the data generating process or by requiring a-priori knowledge of the set S . When these assumptions are not satisfied or such information is unavailable, the analyst has no option but to use an exponentially slow method. We make two contributions in the present work. First, we provide a computationally efficient procedure for estimating \mathbb{E}[Y^{e}|X_{S}^{e}] that is valid for any generalized linear model satisfying ( 1 ). Our approach uses a linear relaxation of the estimator proposed in [ 9 ] , and we propose an alternating minimization scheme to compute our estimator quickly and accurately. We then propose a robust version of our method that is able to maintain good statistical performance even when some environments fail to satisfy the condition in ( 1 ). Robustness is invaluable in real-world settings, as data sources may suffer from measurement error, corruption, or outright misspecification. We demonstrate through an analysis of synthetic data that our proposed method and its robust extension perform well under a variety of settings. Then, we use a real EHR dataset from the All of Us program [ 14 ] to build a risk prediction model for end stage renal disease. We show that our proposed methods give the best predictive performance on previously unseen environments even when some of the training data is corrupted. 1.1 Related literature Our work directly builds on the invariant features model first proposed by the seminal work of [ 3 ] . The authors of [ 3 ] propose a multiple testing based procedure for recovering the invariant features S under a linear assumption on ( 1 ). In [ 7 ] and [ 10 ] , these authors extend this idea to nonlinear and time series models respectively. The first theoretical guarantees on estimating the conditional mean \mathbb{E}[Y^{e}|X_{S}^{e}] under a linear model are given in [ 9 ] , and these authors extend their results to nonlinear models in [ 15 ] . The work of [ 11 ] also aims to estimate \mathbb{E}[Y^{e}|X_{S}^{e}] under a linear model and does so using a relaxation similar to the one proposed in the present work. They provide theoretical results showing that their CoCo procedure can identify the invariant effects if the analyst has prior knowledge on the support S . The recent work of [ 12 ] uses a non-convex optimization program to estimate \mathbb{E}[Y^{e}|X_{S}^{e}] , requiring that distinct environments are generated from additive interventions. Finally, [ 16 ] studies an invariant features-type model under fairness constraints, and provides algorithms for estimation and feature screening under this model. This line of work is situated within the broader statistical data integration literature. The invariant features model is closely related to the multi-task learning problem [ 17 ] , as distinct environments can be thought of as different tasks that are modeled simultaneously. The papers [ 18 , 19 ] initiate the contemporary theoretical study of multi-task learning using classical tools from empirical process theory. The more recent work [ 20 ] gives sharper results in the multi-task linear model under a