Observing the unobserved confounding through its effects: toward randomized trial-like estimates from real-world survival data Journal: The Lancet Digital Health

Vasiliki Stoumpou 1 , Dimitris Bertsimas 1,2 , Samuel Singer 3 , Georgios Antonios Margonis 2,3,4 1 Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA 2 Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA 3 Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA 4 Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany [email protected] · 2026

Read Observing the unobserved confounding through its effects: toward randomized trial-like estimates from real-world survival data Journal: The Lancet Digital Health by Vasiliki Stoumpou 1 , Dimitris Bertsimas 1,2 , Samuel Singer 3 , Georgios Antonios Margonis 2,3,4 1 Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA 2 Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA 3 Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA 4 Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12137.txt.

Introduction

Stoumpou and three colleagues infer a hidden prognostic factor from survival-time gaps between similar patients who received the same treatment but fared differently, then balance it with observed traits. Across three observational cohorts with trial benchmarks, balancing the hidden factor improved agreement with trial hazard ratios in every comparison, cutting log-error roughly tenfold at best. The approach also narrowed cross-center differences in multicenter cohorts.

Catalog record

CREATORVasiliki Stoumpou 1 , Dimitris Bertsimas 1,2 , Samuel Singer 3 , Georgios Antonios Margonis 2,3,4 1 Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA 2 Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA 3 Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA 4 Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83648
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.12137

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Observing the unobserved confounding through its effects: toward randomized trial-like estimates from real-world survival data Journal: The Lancet Digital Health Vasiliki Stoumpou 1 , Dimitris Bertsimas 1,2 , Samuel Singer 3 , Georgios Antonios Margonis 2,3,4 1 Operations Research Center, Massachusetts Institute of Technology, Cambridge, MA, USA 2 Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA 3 Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA 4 Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany [email protected] Abstract Background: Randomized controlled trials (RCTs) are costly, time-consuming and often infeasible, whereas treatment-effect estimation from observational data is limited by unobserved confounding. Methods: We developed a three-step framework to address unobserved confounding in observational survival data. First, we infer a latent prognostic factor ( U ~ \tilde{U} ) from restricted mean survival time (RMST) discrepancies between patients with similar observed prognostic factors, the same treatment assignment, and divergent outcomes, based on the premise that the aggregate effect of unmeasured prognostic factors can be inferred even when individual factors cannot be observed. Second, we balance U ~ \tilde{U} together with observed baseline prognostic factors between treatment groups using prognostic matching, entropy balancing, or inverse probability of treatment weighting. Third, we apply multivariable survival analysis to estimate hazard ratios (HRs). We evaluated the framework in three settings: three observational cohorts with RCT benchmark HRs, two RCT cohorts, and six multicenter observational cohorts. Results: In three observational cohorts (nine cohort-method comparisons), balancing U ~ \tilde{U} improved agreement with benchmark trial HRs in all comparisons; in the strongest settings, it reduced absolute log-HR error by approximately ten-fold relative to adjustment using observed covariates alone (mean reduction, 0.344; p=0.001). In the two RCT cohorts, U ~ \tilde{U} was balanced across treatment arms (most SMDs <0.1) and adjusting for it had little effect on estimated treatment log-HRs (mean absolute change ≃ \simeq 0.08). Across six multicenter cohorts, balancing U ~ \tilde{U} reduced cross-center dispersion in chemotherapy log-HR estimates for overall- and recurrence-free survival (mean reduction 0.147; p = 0.016 p=0.016 ) when balancing was performed within centers to address residual confounding, and narrowed cross-center survival differences in 75% to 100% of informative comparisons when populations were directly balanced across centers to account for differences in patient case mix. Conclusions: Inferring and balancing a latent prognostic signal may reduce unobserved confounding in observational survival analyses and support more reliable treatment-effect estimation from real-world data. 1 Introduction Observational real-world data are increasingly used to evaluate treatment effects, particularly when randomized controlled trials (RCTs) are infeasible, underpowered, outdated, or not fully representative of contemporary practice. However, such analyses remain vulnerable to bias, including immortal time bias, selection bias, and confounding by indication. Although recent methodological advances have improved our ability to address some of these problems, and standard regression, matching, and weighting approaches can mitigate confounding from measured covariates, unobserved confounding remains fundamentally difficult to resolve. By definition, it is not possible to adjust for variables that are not measured; one cannot regress or match on what is not observed. As a result, even after careful adjustment for measured covariates, treatment groups may still differ in important unmeasured aspects of prognosis, potentially leading observational studies to produce biased or even misleading treatment-effect estimates. This challenge is especially relevant in surgical oncology, where decisions regarding surgery and perioperative treatment often depend on nuanced aspects of disease severity, patient fitness, and clinician judgment that are not fully captured in routine datasets. In practice, this problem is often addressed not by solving it, but by assuming that unobserved confounding is negligible for the clinical question at hand. Yet that assumption is frequently implausible, as illustrated by settings in which treatment-effect estimates from observational data were later contradicted by randomized evidence [ 13 , 5 ] . These discrepancies highlight the need for methods that can account, at least indirectly, for latent prognostic differences that remain outside the recorded data. Research in context Evidence before this study Unobserved confounding arising from unknown, unmeasured, or unrecorded prognostic factors remains a fundamental challenge in clinical outcomes research. When such factors are imbalanced across treatment groups, treatment effect estimates can be biased even after adjustment for observed covariates. Existing approaches generally do not directly address this problem; instead, they rely on the assumption that unobserved confounding is limited or negligible in the clinical setting under study. As a result, the validity of many observational analyses depends heavily on design restrictions and unverifiable assumptions. Added value of this study Existing methods for treatment-effect estimation in real-world data, including inverse probability of treatment weighting, entropy balancing, and prognostic matching, adjust only for observed covariates and therefore remain fundamentally limited by unobserved confounding. The added value of this study is that it extends these standard frameworks by incorporating a patient-level latent quantity, \tilde{U} , that indirectly captures the aggregate effects of unobserved prognostic factors. Thus, rather than only balancing what is measured, the proposed approach seeks to also balance part of what is unmeasured. Empirically, this added value was supported by improved agreement with benchmark randomized trials, by preservation of balance of \tilde{U} across randomized trial arms, and by findings in multicenter colorectal liver metastasis cohorts showing that \tilde{U} captured both hidden factors contributing to treatment-selection bias within cohorts and hidden differences in case mix across centers. These are sources of heterogeneity that conventional observed-covariate methods cannot directly address. Implications of all the available evidence This framework provides a practical strategy to strengthen causal inference from real-world data in the presence of likely unobserved confounding. By augmenting standard adjustment methods with a patient-level estimate of latent prognostic burden, it may help observational analyses better approximate trial-like treatment-effect estimates. In doing so, it could be particularly useful for revisiting unresolved clinical debates in settings where randomized evidence is unavailable, limited, outdated, or difficult to generalize to contemporary practice. In this study, we propose a framework to address this problem by inferring and balancing a latent prognostic signal that reflects the aggregate effect of otherwise unobserved prognostic factors. Rather than attempting to recover each missing factor individually, the framework aims to capture their combined prognostic influence before estimating treatment effects as hazard ratios (HRs). We evaluate the framework in three complementary settings. First, in three observational surgical oncology cohorts, we test whether it moves treatment-effect estimates closer to corresponding RCT benchmarks. Second, in two oncology RCT datasets, where confounding should be absent by design, we test whether it leaves the trial-reported HRs largely unchanged. Third, in multicenter colorectal liver metastasis (CRLM) data, we test whether the framework reduces two distinct forms of between-center heterogeneity: hidden bias in who receives adjuvant chemotherapy within a center, reflected in variation in chemotherapy HR estimates across centers, and hidden differences in patient case mix across center populations, reflected in residual cross-center survival differences after balancing recorded covariates. 2 Methods We first provide an overview of the proposed methodology and then describe each component in detail. Our approach aims to transform an observational cohort to a population that closely mimics the conditions of an RCT, namely balance in both observed and unobserved covariates. The ultimate goal is to enable reliable treatment effect estimation within a survival analysis

Read the complete text