Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules

Johannes Hruza, Erin Gabriel, Arvid Sjölander, Samir Bhatt ∗, Michael Sachs · 2025

Read Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules by Johannes Hruza, Erin Gabriel, Arvid Sjölander, Samir Bhatt ∗, Michael Sachs (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-14653.txt.

Introduction

Johannes Hruza and four co-authors bound, rather than pinpoint, the value of a proposed treatment rule when hidden factors may bias the data. Doctors want to know the average outcome if everyone followed a rule based on patient traits. Observational data tempt strong assumptions about confounders nobody measured. Their 2025 paper derives nonparametric bounds: an honest range for the outcome that holds even with unobserved confounding. Two derivation strategies are developed and compared. The techniques then extend to instrumental variable settings.

A range that cannot be narrowed without assumptions is more truthful than a false point. Policy built on such bounds knows its own ignorance. The authors aim at methodologists and trial-minded clinicians. Their work continues a line that trades precision for credibility. Bounds thinking is spreading wherever experiments are impossible or unethical.

Catalog record

CREATORJohannes Hruza, Erin Gabriel, Arvid Sjölander, Samir Bhatt ∗, Michael Sachs
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-14653

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Nonparametric Bounds for Evaluating the Clinical Utility of Treatment Rules Johannes Hruza, Erin Gabriel, Arvid Sjölander, Samir Bhatt ∗, Michael Sachs Abstract Evaluating the value of new clinical treatment rules based on patient characteristics is important but often complicated by hidden confounding factors in observational studies. Standard methods for estimating the average patient outcome if a new rule were universally adopted typically rely on strong, untestable assumptions about these hidden factors. This paper tackles this challenge by developing nonparametric bounds—a range of plausible values—for the expected outcome under a new rule, even with unobserved confounders present. We propose and investigate two main strategies for derivation of these bounds. We extend these techniques to incorporate Instrumental Variables (IVs), which can help narrow the bounds, and to directly estimate bounds on the difference in expected outcomes between the new rule and an existing clinical guideline. In simulation studies we compare the performance and width of bounds generated by the reduction and conditioning strategies in different scenarios. The methods are illustrated with a real-data example about prevention of peanut allergy in children. Our bounding frameworks provide robust tools for assessing the potential impact of new clinical treatment rules when unmeasured confounding is a concern. Keywords: Causal Inference, Partial Identification, Treatment Rules 1 Introduction While numerous risk scores are available in clinical practice, many have not been effectively evaluated for their role in guiding medical decision-making. One formal approach to evaluating such scores is by assessing the expected outcome if treatment decisions were based on the score, often framed within the context of dynamic treatment regimes (DTRs) or individualized treatment rules (ITRs). A DTR (or ITR, for a single treatment point) is a sequence of treatment rules, mapping patient covariates to recommended treatments ( Murphy 2003 ; Chakraborty and Moodie 2013 ; Tsiatis et al. 2019 ) . The value of a specific rule, f , derived from a risk score based on covariates \boldsymbol{X} , can be quantified by the average outcome observed if all subjects in the population of interest were treated according to that rule. Existing methods for estimating and optimizing this quantity often rely on the strong assumption of no unmeasured confounding given the observed covariates ( Zhao et al. 2012 ) . However, in many real-world observational settings, particularly when evaluating a new rule f(\boldsymbol{X}) , it is plausible that unmeasured factors \boldsymbol{U} influence both the treatment A actually received and the outcome Y . These unmeasured factors (for example visual appearance to a clinician) might already be implicitly considered in the current Standard of Care (SOC), leading to confounding bias in later analysis if ignored. Assessing the clinical utility ( Sachs et al. 2020 ; Hruza et al. 2025 ; Brand et al. 2023 ) of a rule f(\boldsymbol{X}) , defined as the improvement over the current standard of care, CU(f)=E[Y(A=f(\boldsymbol{X}))]-E[Y] , ideally accounts for such complexities. Here, f(X) varies across subjects, and Y(A=f(X)) denotes the outcome had each subject’s treatment A been set to their specific rule-based value. While the SOC E[Y] is typically observable, estimating the counterfactual outcome under the treatment rule E[Y(A=f(\boldsymbol{X}))] remains challenging under unmeasured confounding. Partial identification provides a framework for deriving valid bounds on causal estimands without relying on the common assumption of no unmeasured confounding ( Manski 1990 ; Manski 2003 ) . Researchers have also investigated the use of mild, interpretable assumptions—often through sensitivity analysis—to further narrow these bounds ( Yadlowsky et al. 2022 ; Dorn et al. 2021 ; Ding and VanderWeele 2016 ; Peña 2022 ) . Several authors have applied bounding techniques, often using Instrumental Variables (IVs), to estimate treatment effects or optimize DTRs ( Balke and Pearl 1997 ; Zhang et al. 2012 ; Qiu et al. 2020 ; Cui 2021 ) . For instance, Balke and Pearl 1997 pioneered the use of linear programming (LP) to derive sharp bounds for the Average Treatment Effect (ATE) in a binary IV setting. This LP approach has been generalized by Sachs et al. 2022 for a broader class of causal queries and DAGs. Recent work has also explored bounds for super-optimal treatment rules ( Laurendeau et al. 2024 ) . Despite these advances, bounding the specific quantity E[Y(A=f(\boldsymbol{X}))] relevant for clinical utility in the presence of unmeasured confounders and potentially an IV remains an area needing further exploration. Our goal is to establish nonparametric bounds on E[Y(A=f(\boldsymbol{X}))] under such conditions, as directly applying LP methods to the problem is computationally infeasible and grows complexity with the levels of X . We investigate two primary strategies: (i) Reduction: ignoring the information in the covarites \boldsymbol{X} given the information contained in f(\boldsymbol{X}) , thereby simplifying the causal graph and applying existing bounding machinery. (ii) Conditioning: Explicitly conditioning on \boldsymbol{X} , deriving bounds within strata defined by \boldsymbol{X}=\boldsymbol{x} , and then marginalizing these bounds over the distribution of \boldsymbol{X} . We compare these approaches, analyze their theoretical justification, and explore their extension to settings with IVs and additional observed covariates W as well under the influence of existing guidelines g that also map the covariates \boldsymbol{X} to a treatment A . We aim to understand conditions under which these methods yield valid and potentially sharp bounds for clinical utility assessment. We structure the paper as follows: In Section 2 we introduce the notation and causal settings. In Section 3 we describe the DAG reduction approach for bounding E[Y(A=f(\boldsymbol{X}))] . In Section 4 we describe the conditioning approach. In Section 5 we extend these methods to scenarios involving instrumental variables and additional covariates W as well as guidelines g . In Section 7 we present simulation results comparing the two approaches. In Section 8 we apply the techniques to real-world data in the field of immunology. Finally, in Section 9 we discuss the findings, challenges, and implications. 2 Setting and Notation We consider an observational setting where we are interested in the impact of applying a treatment rule f based on observed covariates \boldsymbol{X} . Let A be the treatment variable, which we assume to be categorical with levels \mathcal{A}=\{a_{0},a_{1},\dots,a_{k-1}\} . Let Y be the outcome variable, assumed binary ( Y\in\{0,1\} ) for simplicity, representing an event like recovery or mortality. Let \boldsymbol{X} be a vector of observed covariates, taking values in a set \mathcal{X} . Let \boldsymbol{U} represent a set of unmeasured variables that confound the relationship between A and Y . We make no distributional assumption on \boldsymbol{U} . We are given a deterministic function f:\mathcal{X}\to\mathcal{A} , representing the treatment rule or risk score of interest. This rule maps the covariates \boldsymbol{X} of an individual to a recommended treatment level f(\boldsymbol{X})\in\mathcal{A} . Here, f(\boldsymbol{X}) is a random variable representing the treatment recommendation dictated by the rule f . Our primary causal estimand is the expected counterfactual outcome under rule f : \displaystyle\theta_{f}=E[Y(A=f(\boldsymbol{X}))]=P(Y(A=f(\boldsymbol{X}))=1), where Y(a) denotes the potential outcome if treatment a were assigned. The clinical utility can then be expressed as CU(f)=\theta_{f}-E[Y] . Initially we assume E[Y] is point identifiable from the observed data distribution P(Y) , and our focus is on bounding \theta_{f} . We can expand \theta_{f} using the law of total probability over the recommendations f(\boldsymbol{X}) : \displaystyle\theta_{f}=\sum_{a_{j}\in\mathcal{A}}P(Y(a_{j})=1,f(\boldsymbol{X})=a_{j}). We will analyze this quantity and the clinical utility

Read the complete text