Bayesian Machine Learning for Estimating Optimal Dynamic Treatment Regimes with Ordinal Outcomes
Xinru Wang, Tanujit Chakraborty, Bibhas Chakraborty · 2025
Read Bayesian Machine Learning for Estimating Optimal Dynamic Treatment Regimes with Ordinal Outcomes by Xinru Wang, Tanujit Chakraborty, Bibhas Chakraborty (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-06199.txt.
Introduction
Xinru Wang and co-authors present a 2025 method for planning treatment that adapts over time. Care plans often change as a patient state evolves. Outcomes in many trials fall into ordered levels, not just yes or no. Building best plans for such ordered outcomes has had little study so far.
The team build a Bayesian machine learning frame for this task. A new tree model handles ordered outcomes by way of a hidden score. Tests on simulated data compare the method with older tools. A smoking cessation trial shows the method at work. Code and an R package come with the paper for reuse.
Catalog record
| CREATOR | Xinru Wang, Tanujit Chakraborty, Bibhas Chakraborty |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-06199
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bayesian Machine Learning for Estimating Optimal Dynamic Treatment Regimes with Ordinal Outcomes Xinru Wang, Tanujit Chakraborty, Bibhas Chakraborty Abstract Dynamic treatment regimes (DTRs) are sequences of decision rules designed to tailor treatment based on patients’ treatment history and evolving disease status. Ordinal outcomes frequently serve as primary endpoints in clinical trials and observational studies. However, constructing optimal DTRs for ordinal outcomes has been underexplored. This paper introduces a Bayesian machine learning (BML) framework to handle ordinal outcomes in the DTR setting. To deal with potential nonlinear associations between outcomes and predictors, we first introduce ordinal Bayesian additive regression trees (OBART), a new model that integrates the latent variable framework within the traditional Bayesian additive regression trees (BART). We then incorporate OBART into the BML to estimate optimal DTRs based on ordinal data and quantify the associated uncertainties. Extensive simulation studies are conducted to evaluate the performance of the proposed approach against existing methods. We demonstrate the application of the proposed approach using data from a smoking cessation trial and provide the OBART R package along with R code for implementation. Keywords: Dynamic treatment regimes (DTRs); Ordinal outcomes; Bayesian machine learning; Bayesian Additive Regression Trees (BART). 1 Introduction In real-life clinical practice dealing with chronic and relapsing diseases, such as mental illness [ 1 ] and human immunodeficiency virus (HIV) infection [ 2 ] , a single treatment may show heterogeneous effects for patients with different characteristics. To minimize unnecessary side effects and improve treatment efficacy, it is often preferable to offer treatments in multiple stages, adjusting dosage and intensity based on patients’ evolving disease conditions. Dynamic treatment regimes (DTRs) are sequences of decision rules that formalize the multi-stage personalized treatment by taking the treatment histories and covariates as input and the recommended treatments as output [ 3 ] . In recent years, there are increasing methodological innovations for estimating optimal DTRs using sequential multiple assignment randomized trials (SMARTs) [ 3 , 4 ] or observational studies [ 5 , 6 , 7 , 8 ] . For example, Wallace and Moodie (2015) [ 6 ] presented a dynamic weighted ordinary least squares regression (dWOLS) approach for estimating optimal DTRs for continuous data, offering both the doubly robust property of the G-estimation and the ease of implementation property of the Q-learning. Simoneau et al. (2020) [ 9 ] extended the work of Wallace and Moodie (2015) [ 6 ] to survival endpoints subject to right-censoring. In addition to parametric or semiparametric models, Tao et al. (2018) [ 8 ] proposed more flexible and interpretable tree-based methods for estimating optimal DTRs. To the best of our knowledge, the majority of these methodologies focus on continuous, binary, or time-to-event outcomes. However, in many clinical trials, the primary endpoint is formalized as ordinal outcomes. This is true for many SMARTs as well. One example is a two-stage smoking cessation trial [ 10 ] conducted by the Center for Health Communications Research (CHCR) at the University of Michigan. This smoking cessation trial, designed to help people quit smoking, consists of two sub-studies that together form a SMART. The first sub-study is the ProjectQuit (PQ) study conducted to find an optimal multi-component behavioral intervention to help adult smokers quit smoking and the second is the ForeverFree (FF) study conducted to help those who already quit stay quit and help those who failed at the initial stage with a second chance of receiving a beneficial treatment. Although the original studies used the binary quit status in the last seven days prior to the final survey as the primary outcome to evaluate the treatment, this outcome can be criticized as not measuring the long-term behavioral changes for an addiction like smoking; as such, many other secondary outcomes, e.g., the number of months not smoked by a subject over the study period, ranging from 0 to 6, were also collected [ 11 ] . To handle near-zero frequencies of some of the months in the distribution, a collapsed version of this variable, re-coded to three levels such as 0, 1, and 2, was used. Researchers may be interested in estimating the optimal decision rules based on such ordinal outcomes using six-month data instead of the binary quit status in the last seven days. In such cases, the existing methods for constructing optimal DTRs can only be applied by transforming the ordinal data into continuous ones using scores for each category provided by relevant experts or dichotomizing the ordinal outcomes with more than two categories into two categories. Despite the simplicity of not requiring new statistical methodology, the potential drawback of the former approach is that the results may be sensitive to the specified scores, thus leading to biased estimates of the optimal DTRs, which will be more problematic when there is no consensus on the suitable numeric scores for each outcome category. Additionally, useful information may be wasted for the latter approach when dichotomizing the ordinal outcomes into binary outcomes. Various methodologies have been proposed to analyze ordinal data in non-DTR settings. Johnson and Albert (1999) [ 12 ] presented a comprehensive overview of regression models for ordinary data from both the frequentist and the Bayesian perspectives. However, limited literature focuses on ordinal data in the field of DTRs. Within the DTR literature, Ghosh et al. (2023) [ 13 ] proposed an easy-to-interpret measure of association, named generalized odds ratio (GOR), for comparing two embedded regimes in a SMART with ordinal outcomes. However, the GOR cannot be utilized when the research goal is to develop more tailored DTRs among a large set of potential DTRs. In addition, Jiang et al. (2024) [ 14 ] proposed a doubly robust dWOLS approach for constructing optimal DTRs for ordinal outcomes with household interference, i.e., one household member’s treatment may affect another’s outcome. However, as pointed out by Wallace and Moodie (2015) [ 6 ] , the standard bootstrap approach for variance estimation may be subject to non-regularity issues. Recently, Murray et al. (2018) [ 7 ] proposed a Bayesian machine learning (BML) approach, which innovatively combines Bayesian inferences with dynamic programming, to estimate an optimal DTR and quantify the uncertainties about the estimated parameters. Built upon their work, the current paper aims to propose a BML approach to make valid inferences about the optimal DTRs using data with ordinal outcomes. Our main contributions include presenting both frequentist and Bayesian perspectives for constructing optimal DTRs when dealing with ordinal outcomes, which are commonly used in medical settings but often overlooked. Specifically, we introduce an easy-to-implement Q-learning approach for ordinal data. To alleviate the non-regularity issues when quantifying the uncertainties of the parameters of interest in Q-learning [ 15 ] , we propose to use the BML approach [ 7 ] , utilizing a data augmentation approach to account for all sources of uncertainties from stage-2 estimation when estimating optimal decision rules at stage 1. Built upon Chipman et al. (2010) [ 16 ] and Tan and Roy (2019) [ 17 ] , we propose the Ordinal Bayesian Additive Regression Tree (OBART), and then incorporate it into the BML approach to capture nonlinear interactions between covariates and treatments and name it BML-OBART. We conduct extensive simulation studies to compare the candidate approaches mentioned above.