Domain Generalization and Adaptation in Intensive Care with Anchor Regression
Malte Londschien, Manuel Burger, Gunnar Rätsch, Peter Bühlmann · 2025
Read Domain Generalization and Adaptation in Intensive Care with Anchor Regression by Malte Londschien, Manuel Burger, Gunnar Rätsch, Peter Bühlmann (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-21783.txt.
Introduction
Malte Londschien, Manuel Burger, Gunnar Rätsch, and Peter Bühlmann present a 2025 study of hospital models that must travel. A risk model tuned at one ICU often sags at the next, as patient mixes shift. The team apply anchor regression, a causality-flavored method that leans on stable links, plus a new tree-based kin, anchor boosting. Data held four hundred thousand patients across nine ICU stores.
Anchors lifted out-of-home scores most where the new home differed most. The methods held up even when their theory terms bent. A three-zone frame then prices outside data: use it raw, refit it, or drop it, by how much local data sits on hand. Traveling models are a safety question, not a stats nicety.
Catalog record
| CREATOR | Malte Londschien, Manuel Burger, Gunnar Rätsch, Peter Bühlmann |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-21783
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Domain Generalization and Adaptation in Intensive Care with Anchor Regression Malte Londschien, Manuel Burger, Gunnar Rätsch, Peter Bühlmann Abstract The performance of predictive models in clinical settings often degrades when deployed in new hospitals due to distribution shifts. This paper presents a large-scale study of causality-inspired domain generalization on heterogeneous multi-center intensive care unit (ICU) data. We apply anchor regression and introduce anchor boosting, a novel, tree-based nonlinear extension, to a large dataset comprising 400,000 patients from nine distinct ICU databases. The anchor regularization consistently improves out-of-distribution performance, particularly for the most dissimilar target domains. The methods appear robust to violations of theoretical assumptions, such as anchor exogeneity. Furthermore, we propose a novel conceptual framework to quantify the utility of large external data datasets. By evaluating performance as a function of available target-domain data, we identify three regimes: (i) a domain generalization regime, where only the external model should be used, (ii) a domain adaptation regime, where refitting the external model is optimal, and (iii) a data-rich regime, where external data provides no additional value. Keywords: distributional robustness, intensive care unit (ICU) data, invariance generalization, multi-source data, tree-boosting 1 Introduction A standard assumption in predictive modeling is that training and test data come from the same distribution. This assumption often fails in real-world scenarios. For example, in clinical applications, test data may originate from a different time period or hospital than the training data. When these distribution shifts occur, model performance tends to drops significantly ( Barak-Corren et al. 2021 ; Guo et al. 2021 ; Roland et al. 2022 ; Yang et al. 2022 ; van de Water et al. 2024 ; Hüser et al. 2024 ) . The field of distributional robustness has emerged to address this problem, but its successes have been largely demonstrated on simulated, semi-synthetic (for example, colored MNIST of Arjovsky et al. 2019 and ImageNet-C of Hendrycks and Dietterich 2019 ), or curated (for example, PACS of Li et al. 2017 and waterbirds of Sagawa et al. 2020 ) datasets. In contrast, large-scale empirical studies show mixed or negative results, with domain generalization models often failing to outperform simple baselines ( Gulrajani and Lopez-Paz 2021 ; Guo et al. 2022 ; Rockenschaub et al. 2024 ) . An alternative approach to achieve generalization is to scale data and model capacity, a strategy proven successful for large language models ( Brown et al. 2020 ) . This is the focus of prior work by Burger et al. 2025 , who develop a foundation model on large ICU data. They establish a “square-root” scaling law for domain generalization, where quadrupling the external data provides a similar performance gain to doubling the locally available data. In contrast, we focus here on methods that use existing heterogeneity to improve robustness by exploiting causal models. In particular, we consider anchor regression ( Rothenhäusler et al. 2021 ) . Intuitively, we expect causal relationships (vasopressor drugs raise blood pressure) to be stable, whereas relationships induced through hidden confounding (clinicians prescribe vasopressor drugs to severely ill patients, and thus vasopressor drug use correlates with increased mortality) can shift with varying treatment policies. Anchor regression promotes stability or invariance to such shifts by penalizing dependencies that vary with the so-called anchor variable. It achieves this by interpolating between ordinary least-squares and instrumental variables regression. In this paper, we apply and extend anchor regression for medical prediction. Recognizing that linear models may be insufficient to capture the complex feature interactions in clinical data, we first propose a novel, nonlinear extension to anchor regression based on gradient boosting trees ( Friedman 2001 ) , popular in clinical predictive modeling ( Hyland et al. 2020 ; Lyu et al. 2024 ; Hüser et al. 2024 ) . We then conduct a large-scale empirical study, applying linear anchor regression and our nonlinear extension to predict adverse events in intensive care units (ICUs). We assess the method’s ability to improve distributional robustness on a strongly heterogeneous dataset aggregating nine ICU databases ( Burger et al. 2024 ; Burger et al. 2025 ) . Finally, we propose a conceptual framework for quantifying the utility of large external datasets, particularly when target domain data is scarce. 1.1 Our contribution Our contributions are threefold: A novel nonlinear extension to anchor regression We introduce anchor boosting , a novel nonlinear extension of anchor regression ( Rothenhäusler et al. 2021 ) based on boosted tree learners. While the concept of anchor boosting had been proposed before ( Bühlmann 2020 ) , our implementation extends to classification tasks ( Kook et al. 2022 ) and incorporates second-order optimization to update tree leaf node values. These extensions prove to be crucial for our application. A large-scale application of causality-inspired regularization We conduct the largest-scale application of anchor regression ( Rothenhäusler et al. 2021 ) and its variants to date, using a dataset of 400,000 patients and 10 million observations from nine distinct ICU databases. To our knowledge, this work is also the first application of anchor regression and the largest application of a causality-inspired method to a medical prediction problem. In a setting where many other domain generalization methods have been shown to provide little benefit over simple baselines ( Rockenschaub et al. 2024 ) , we show that the anchor methods yield significant performance improvements, particularly for the most out-of-distribution target domains. A framework to quantify the value of external data We propose and empirically validate a framework, with thematic similarities to Desautels et al. 2017 , to assess the utility of external data for a given target domain and task. By expressing performance as a function of target sample size, we identify three regimes: (i) a domain generalization regime where only external data should be used, (ii) a domain adaptation regime, where refitting an external model is optimal, and (iii) a data-rich regime, where training on target data is best and the external data provides no additional value. This taxonomy provides a practical methodology to quantify the information value of large external datasets in terms of equivalent number of in-domain samples. 2 Data description We use data from Burger et al. 2024 ; Burger et al. 2025 who build upon the R-package ricu ( Bennett et al. 2023 ) to harmonize and aggregate ICU data from different sources. We will describe this dataset in further detail next. 2.1 ICU datasets included We focus on the following 9 ICU datasets: The eICU Collaborative Research Database ( Pollard et al. 2018 ) is a multi-center critical care database of 207 hospitals throughout the USA. It contains data from 188’257 patient stays from 2015 and 2016. The Medical Information Mart for Intensive Care (MIMIC) III ( Johnson et al. 2016 ) and IV ( Johnson et al. 2023 ) contain data from the Beth Israel Deaconess Medical Center (BIDMC) in Boston (USA). The BIDMC switched critical care information systems in 2008 from Philips CareVue to iMDsoft Metavision. We consider only the CareVue subset of MIMIC-III to avoid an overlap with MIMIC-IV. This MIMIC-III CareVue subset contains data from 34’154 patient stays from 2001–2008 and includes neonatal patient stays. MIMIC-IV contains data from 93’679 patient stays from 2008–2022. The Northwestern ICU database (NWICU) ( Moukheiber et