Counterfactual Survival Q-Learning for Longitudinal Randomized Trials via Buckley–James Boosting
Jeongjin Lee a and Jong-Min Kim b · 2025
Read Counterfactual Survival Q-Learning for Longitudinal Randomized Trials via Buckley–James Boosting by Jeongjin Lee a and Jong-Min Kim b (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-11060.txt.
Introduction
Jeongjin Lee and Jong-Min Kim propose in 2025 a method to learn the best order of treatments when the outcome is how long a patient lives. Trial data often end before all patients are followed, and effects may bend in complex ways. Older methods lean on hazard models that can mislead when their core premise fails.
Their method models survival time head on and boosts the fit step by step with flexible learners. It then works backward through treatment stages to build personal rules. Tests on two real trials, one for HIV and one for leukemia, plus made up data, showed sharper treatment choices than the older route.
Catalog record
| CREATOR | Jeongjin Lee a and Jong-Min Kim b |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-11060
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Counterfactual Survival Q-Learning for Longitudinal Randomized Trials via Buckley–James Boosting Jeongjin Lee a and Jong-Min Kim b Abstract We propose a Buckley–James (BJ) Boost Q-learning framework for estimating optimal dynamic treatment regimes under right-censored survival data, tailored for longitudinal randomized clinical trial settings. The method integrates accelerated failure time models with iterative boosting techniques—componentwise least squares and regression trees—within a counterfactual Q-learning framework. By directly modeling conditional survival time, BJ Boost Q-learning avoids the restrictive proportional hazards assumption and enables unbiased estimation of stage-specific Q-functions. Grounded in potential outcomes, this framework ensures identifiability of the optimal treatment regime under standard causal assumptions. Compared to Cox-based Q-learning, which relies on hazard modeling and may suffer from bias under misspecification, our approach provides robust and flexible estimation. Simulation studies and analysis of the ACTG175 HIV trial demonstrate that BJ Boost Q-learning yields higher accuracy in treatment decision-making, especially in multi-stage settings where bias can accumulate. keywords Boosting; Reinforcement Learning; Imputation; Survival Analysis † † articletype: ARTICLE TEMPLATE † † affiliation: a Division of Biostatistics, The Ohio State University, 281 W Lane Ave, Columbus, OH 43210, U.S.A.; b Division of Science and Mathematics, University of Minnesota-Morris, Morris, MN 56267, U.S.A. 1 Introduction In the evolving field of contemporary healthcare, individualized treatment strategies ( Moodie et al. 2012 ; Wahed & Thall 2013 ; Chakraborty & Murphy 2014 ; Kosorok & Moodie 2015 ; Song et al. 2015 ; Simoneau et al. 2020 ; Cho et al. 2023 ) have become a central approach for tailoring interventions to optimize patient outcomes. This approach is especially important in the presence of complex censored data, where event times may be partially unobserved due to issues such as patient dropout or loss to follow-up. Such challenges have motivated the development of methodological frameworks capable of handling incomplete outcome information. Survival analysis, which focuses on time-to-event data, is crucial in medical research. A major challenge in this field is the frequent occurrence of censoring, where the event time for certain observations is unknown, making it difficult to make well-informed decisions from these datasets. Cox regression ( Cox 1972 ) is commonly used for analyzing time-to-event data with censoring. However, it has limitations, including the assumption of proportional hazards and a less intuitive interpretation compared to linear regression. Moreover, hazard ratios (HRs) derived from the Cox model are relative measures that compare the hazard rates of two groups and depend on the underlying hazard function, which may not always be straightforward to interpret. The Cox PH model assumes that the hazard function for an individual is a product of a baseline hazard function and an exponential function of a linear combination of the covariates. This linearity assumption may not hold in all situations, especially when the relationships between covariates and the hazard are complex and non-linear. An alternative approach is the accelerated failure time (AFT) model ( Buckley & James 1979 ; Wei 1992 ; Jin et al. 2003 ; Jin et al. 2006 ; Zeng & Lin 2007 ; Choi et al. 2021 ) , which offers several advantages. The AFT model directly models survival time using a linear regression form, providing more interpretable effects than hazard ratios in Cox models. Moreover, the AFT model is robust to violations of the proportional hazards assumption. The Buckley–James (BJ) method ( Buckley & James 1979 ; Jin et al. 2006 ) , a semiparametric estimator for the AFT model, accommodates arbitrary censoring mechanisms and remains consistent under mild conditions. Due to these advantages, several penalized extensions of BJ estimation have been developed to address high-dimensional data ( Johnson et al. 2008 ; Wang et al. 2008 ; Johnson 2009 ; Li et al. 2014 ; Lee et al. 2024 ) . To extend the Buckley–James (BJ) framework to non-linear and high-dimensional settings, Wang & Wang 2010 proposed the BJ Boosting algorithm, which iteratively updates the predictive function using flexible base learners such as componentwise least squares or regression trees. Unlike Cox-based machine learning approaches that model hazard functions, BJ Boosting directly targets the log-transformed survival time under the accelerated failure time (AFT) model, enabling more interpretable and robust inference. It effectively accommodates right-censoring and tied event times while capturing complex, non-linear relationships. In terms of computational efficiency and estimation accuracy, BJ Boosting has shown superior performance over classical BJ and Cox models. Its iterative structure adaptively selects relevant covariates and mitigates overfitting, making it particularly well-suited for biomarker-driven survival analysis and dynamic treatment regime estimation. Reinforcement learning, particularly Q-Learning ( Watkins & Dayan 1992 ) , has shown great potential in personalized treatment optimization due to its ability to adapt and learn from sequential decision-making processes. In healthcare, treatment decisions are often made in stages, considering the evolving state of a patient’s health. Q-Learning can model this dynamic process by learning optimal policies that maximize long-term health outcomes based on cumulative rewards. This adaptive capability makes Q-Learning especially promising for personalized treatment, where the goal is to tailor interventions to individual patient needs over time. Despite its promise in personalized treatment optimization, directly applying Q-Learning to censored survival data proves challenging due to severe censoring. To address these challenges, we propose the BJ Boost Q-Learning under a counterfactual framework, which integrates Q-learning with the Buckley–James (BJ) boosting algorithm to accommodate non-linear relationships under right-censored survival data. This approach is motivated by real-world clinical studies such as the ACTG 175 trial, a randomized clinical trial designed to compare monotherapy with zidovudine or didanosine against combination therapies involving zidovudine and didanosine or zidovudine and zalcitabine, in HIV-positive adults with CD4 T cell counts between 200 and 500 cells/mm 3 . In such trials, each patient is assigned to one treatment arm, but to evaluate optimal dynamic treatment regimes, it is necessary to estimate potential outcomes under all possible treatment strategies, requiring counterfactual reasoning. While Lee & Kim 2025 introduced a linear BJ Q-learning approach under a counterfactual framework, its reliance on linear associations between covariates and survival outcomes limits its applicability in complex, real-world clinical contexts. Furthermore, the linear Buckley-James estimator ( Buckley & James 1979 ; Jin et al. 2006 ) is known to exhibit convergence issues when the true data-generating mechanism is non-linear. By employing flexible base learners such as componentwise least squares or regression trees, the BJ boosting approach enables stable and accurate imputation of censored survival times even in non-linear settings. Coupled with the recursive structure of Q-learning, this iterative boosting mechanism adaptively refines treatment value estimation at each stage, thereby improving the reliability of learned dynamic treatment regimes and ultimately enhancing patient-specific clinical decision-making. 2 Method 2.1 Preliminary We consider a longitudinal (or sequentially) randomized clinical trial consisting of K decision stages, indexed by k=1,\ldots,K , where each stage corresponds to the interval between two consecutive patient visits. These visits may be scheduled regularly (e.g., every 30 days) or triggered by clinical events (e.g., symptom onset or adverse reaction). At each stage k , clinicians observe patient-specific, time-varying covariates X_{i,k} (e.g., biomarker levels, symptom scores), and randomly assign a treatment A_{i,k}\in\mathcal{A} according to the trial protocol. The treatment set \mathcal{A} may consist of binary (e.g., treatment vs. control), ordinal (e.g., low vs. high dose), or categorical options.