Conditional Inference for Secondary Outcomes Based on the Testing Result for the Primary Outcome in Clinical Trials

2025

Read Conditional Inference for Secondary Outcomes Based on the Testing Result for the Primary Outcome in Clinical Trials (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-09009.txt.

Introduction

Tianyu Pan, Vivek Charu, Ying Lu, and Lu Tian tackle a subtle problem in clinical trials. Teams often study secondary outcomes only when the primary outcome shows a clear effect. They may also change course at interim looks. Standard checks control error rates. But they do not ensure that inferences made after such a choice keep their pledged traits.

A plain confidence interval, for one, can cover the truth at a rate far from its stated level once the choice process is counted. The team argues that validity given the choice is what uses in fact need. They build confidence intervals with correct coverage given that choice. The intervals rest on a pivotal value from the truncated normal distribution the choice creates. Theory and broad tests support the method.

Catalog record

YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-09009

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Conditional Inference for Secondary Outcomes Based on the Testing Result for the Primary Outcome in Clinical Trials Abstract In clinical trials, inferences on clinical outcomes are often made conditional on specific selective processes . For instance, only when a treatment demonstrates a significant effect on the primary outcome, further analysis is conducted to investigate its efficacy on selected secondary outcomes. Similarly, inferences may also depend on whether a trial is terminated early at interim stage. While conventional approaches primarily aim to control the family-wise error rate through multiplicity adjustments, they do not necessarily ensure the desired statistical property of the inference result, when the analysis is conducted according to a selective process . For example, the conditional coverage level of a regular confidence interval under a selective processes can be very different from its nominal level even after adjustment for multiple testing. In this paper, we argue that the validity of the inference procedure conditional on selective process is important in many applications. In addition, we propose to construct confidence intervals with correct conditional coverage probability by accounting for related selective process . Specifically, our approach involves a pivotal quantity constructed by inversing the cumulative distribution function of a truncated normal distribution induced by the selective process . Theoretical justification and comprehensive simulations illustrate the effectiveness of this method in realistic settings. We also apply our method to analyze data from the SPRINT, resulting in more conservative but conditionally valid confidence intervals for the average treatment effect than those originally published. Tianyu Pan 1,2 , Vivek Charu 2,3 , Ying Lu 1 , Lu Tian 1∗ 1 Department of Biomedical Data Science, Stanford University School of Medicine 2 Department of Pathology, Stanford University School of Medicine 3 Quantitative Sciences Unit, Department of Medicine, Stanford University School of Medicine Keywords : Conditional confidence interval; Family-wise error rate; Selection bias; Selective inference. 1 Introduction 1.1 Review It is widely acknowledged in clinical trials that the analysis of multiple outcomes, particularly secondary outcomes, should be conditioned on the significance of the analysis result for the primary outcome ( O’Neill, 1997 ) , as the primary outcome serves as the main efficacy indicator for a treatment. While confidence intervals (CIs) for secondary outcomes are often reported, their importance is frequently downplayed, especially if the primary outcome is nonsignificant. For example, these results are typically deferred to the discussion section per the policies of major medical journals such as the New England Journal of Medicine (NEJM) and the Journal of the American Medical Association (JAMA) (see Bangalore et al. (2015) , Kato et al. (2019) , McMurray et al. (2019) , and Pocock et al. (2021) for examples). As a consequence, their usage and significance is not entirely clear to the research community other than the vague statement that they should be interpreted cautiously. Although reported inference results on secondary outcome analyses are “marginally” valid, it raises concerns about potential bias when they are only used/reported selectively. For example, if a researcher only care about secondary outcomes when the results for primary outcomes are significant, then these CIs for secondary outcomes may fail to cover true parameters at the nominal level for this researcher. This constitutes a selective process , potentially introducing bias in inferences, especially when primary and secondary outcomes are correlated. In an extreme case, if the correlation is close to one, secondary outcomes would almost always appear significant when the primary outcome is significant. A strong correlation is particularly likely when outcomes are defined based on related clinical endpoints, such as the overall survival as the primary endpoint and progression-free survival as a secondary endpoint. This selective process can also occur in group sequential designs, where trials can be terminated early at the interim analysis for efficacy or futility, leading to conditional inferences on related parameters. Additional examples could also be explored within our framework when selective processes are present. In the presence of a selective process , the inferences on secondary outcomes become inherently conditional. At a broader level, these inferences condition on specific decisions, which are derived from pre-defined clinical endpoints, such as thresholds established for primary outcome estimator. The relationship between these conditional and marginal inferences - the common practices in recent clinical studies - can be expressed as follows, \begin{split}&\text{pr}(\text{rejection}\mid\theta_{2}=0)\\ =&~\text{pr}(\text{rejection}\mid\theta_{2}=0,\text{positive decision })\times\text{pr}(\text{positive decision})+\\ &~\text{pr}(\text{rejection}\mid\theta_{2}=0,\text{negative decision})\times\text{pr}(\text{negative decision}),\end{split} (1) where the probability on the left side of the equation corresponds to the marginal type I error under the null that \theta_{2}=0 , while the two conditional probabilities on the right side represent the conditional type I errors. These conditional probabilities depend on whether specific decisions, such as “if the primary outcome is significant” or “if the trial was stopped at an interim stage” were met in the context of earlier examples. Although adjusting for multiple testing across endpoints can control the family-wise error rate (FWER) and, by extension, the marginal type I error on the left, the conditional probability \text{pr}(\text{rejection}\mid\theta_{2}=0,\text{decision=positive }) is not inherently controlled and often inflated due to the selective process . This is troublesome, when researchers report and use the test result for \theta_{2} only when the decision from the selective process is positive. Among those cases, actual type I error rate can be substantially higher than the nominal level. More generally, this problem raises when constructed CIs are treated, interpreted and used differently according to a set of statistics even without an explicit selective process. We will further provide a concrete examples to illustrate this concern in Section 1.2 . This inflation indicates a need for additional adjustments, which form the primary focus of this study. Inferences associated with conditional probabilities conceptually align with gatekeeping procedures ( Dmitrienko et al., 2003 ) , where primary and secondary outcomes (or families) are tested sequentially in a pre-determined hierarchy, generally ordered by their statistical power and clinical significance. The initial concept of such sequential testing can be tracked back to Holm (1979) . Unlike the primary focus in multiple testing ( Hochberg, 1987 ) , where controlling FWER at a desired level through multiplicity adjustments is essential ( Hochberg, 1987 ; Bauer, 1991 ; Koch and Gansky, 1996 ; Bauer et al., 1998 ; Westfall and Krishen, 2001 ) , the hierarchical nature of gatekeeping procedures can automatically maintain FWER control ( Bretz et al., 2009 ) . Over the past several decades, researchers have explored FWER control within more complex trial designs, such as group sequential designs ( Glimm et al., 2010 ; Tamhane et al., 2010 ; Tamhane et al., 2012a ; Tamhane et al., 2012b ; Ye et al., 2013 ; Maurer and Bretz, 2013 ; Alosh et al., 2014 ; Tamhane et al., 2018 ) . Furthermore, gatekeeping procedures have been studied under stronger error control frameworks, such as controlling the false coverage rate (FCR; Benjamini and Yekutieli (2005) ). For a thorough review of gatekeeping procedures, readers may refer to Dmitrienko and Tamhane (2007) ; Dmitrienko and Tamhane (2009) ; Ristl et al. (2019) ; Proschan and Follmann (2023) and references therein for extensive discussions. However, while these methods focus on marginal error probabilities, they do not address controlling the conditional error probabilities defined in ( 1 ), nor do they provide

Read the complete text