Weighted Asymptotically Optimal Sequential Testing
Soumyabrata Bose, Jay Bartroff · 2025
Read Weighted Asymptotically Optimal Sequential Testing by Soumyabrata Bose, Jay Bartroff (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-07588.txt.
Introduction
Soumyabrata Bose and Jay Bartroff improve sequential testing, where data arrive over time and many hypotheses compete. Adding prior knowledge should speed decisions, but careless weighting can break error control. Their framework folds weights into a modified log-likelihood ratio statistic.
Two procedures, Weighted Gap and Weighted Gap-Intersection, keep strong family-wise error control. Both reach the theoretical lower bound on expected stopping time as error chances shrink, proving asymptotic optimality. The optimality survives the weighting rather than depending on lucky priors. Faster valid testing matters in trials and screening where each extra observation costs money, time, or risk.
Catalog record
| CREATOR | Soumyabrata Bose, Jay Bartroff |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-07588
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Weighted Asymptotically Optimal Sequential Testing Soumyabrata Bose, Jay Bartroff Abstract This paper develops a framework for incorporating prior information into sequential multiple testing procedures while maintaining asymptotic optimality. We define a weighted log-likelihood ratio (WLLR) as an additive modification of the standard LLR and use it to construct two new sequential tests: the Weighted Gap and Weighted Gap-Intersection procedures. We prove that both procedures provide strong control of the family-wise error rate. Our main theoretical contribution is to show that these weighted procedures are asymptotically optimal; their expected stopping times achieve the theoretical lower bound as the error probabilities vanish. This first-order optimality is shown to be robust, holding in high-dimensional regimes where the number of null hypotheses grows and in settings with random weights, provided that mild, interpretable conditions on the weight distribution are met. 1 Introduction The sequential acquisition of data is a defining feature of modern large-scale inference. In domains ranging from clinical trials ( Bartroff et al., 2013 ) to genomics ( Lin et al., 2022 ) and online A/B testing ( Kohavi et al., 2020 ) , data are not collected as a single batch but accrue over time. This paradigm has fueled the development of sequential multiple testing procedures, which aim to satisfy the dual objectives of controlling a statistical error rate, such as the family-wise error rate (FWER), while minimizing the expected sample size or stopping time. A key theoretical benchmark in this area is the concept of asymptotic optimality, where a procedure’s expected stopping time attains a fundamental information-theoretic lower bound in the limit of vanishing error probabilities ( Song and Fellouris, 2017 ) . While this sequential optimality framework provides a powerful theoretical foundation, it has largely been developed under the assumption that, a-priori , there is no differential importance, priority, or weighting of the hypotheses; in other words, that they are interchangeable. This assumption is often misaligned with practice in applications, where substantial scientific structure or external information is available to prioritize certain hypotheses over others. As Genovese et al., 2006 write, “All null hypotheses are not created equal.” Examples of scientific situations where naturally-occurring structure, priority, or prior information motivates weights among the hypotheses include the following. • In genomics, researchers conducting RNA-seq experiments to identify differentially expressed genes often have access to prior information from genome-wide association studies (GWAS) or pathway databases like KEGG. This information can be used to assign higher weights to genes with known links to the phenotype of interest, thereby prioritizing biologically plausible candidates ( Korthauer et al., 2019 ) . • In platform clinical trials, which evaluate multiple treatments simultaneously, prior evidence from pre-clinical studies or Phase I/II trials may suggest that certain drug candidates are more likely to be effective. Weighting these hypotheses can accelerate decisions for the most promising treatments, an approach with clear ethical and economic benefits ( Berry et al., 2015 ) . • In neuroscience, studies using fMRI to map brain activity may prioritize specific regions of interest (ROIs) based on known anatomical brain structure from pilot data or previous studies, assigning higher weights to voxels within those ROIs ( Ignatiadis et al., 2016 ) . In these examples, weights are naturally motivated as “ positive constants indicating the importance of the hypothesis, ” as described by Holm, 1979 who incorporated weights into his multiple testing procedure. Larger values of Holm’s weights increase the chance of rejecting null hypotheses in his procedure. We adopt this is our working definition of weights, and consider both deterministic and random regimes for weights below in Sections 4 - 6 . The examples above all have explicit weights which the researchers would like to incorporate into a testing procedure. In other settings researchers may not have explicit weights in mind but implicitly end up using a weighted procedure. For example, commonly-used Bonferroni-type multiple testing procedures compare each p -value P_{1},\ldots,P_{J} with a threshold and the j th null hypothesis is rejected if P_{j}\leq\alpha_{j} where \sum_{j=1}^{J}\alpha_{j}=\alpha , the desired overall familywise error rate. Taking weight W_{j}=\alpha_{j}/(\alpha/J) , this comparison is of course equivalent to comparing each |\log P_{j}|+\log W_{j} with the same threshold |\log(\alpha/J)| . Chen et al., 2014 give an example of this selective “ \alpha allocation” in biomarker clinical trials where more biologically plausible hypotheses are assigned larger \alpha_{j} . Another example of using implicit weights is in Bayesian testing where, for example in testing simple hypotheses X\sim f_{0} vs. X\sim f_{1} , it is well known that the posterior probability P(X\sim f_{1}|X) is an increasing function of \log(f_{1}(X)/f_{0}(X))+\log W , where W is the prior odds of f_{1} . This weighted log-likelihood ratio is the statistic we utilize in our approach below (see ( 2 )). Although primarily frequentist, in this sense our approach can be considered a way to incorporate Bayesian prior information without changing the error measure. Regardless of how weights are motivated, in the fixed-sample setting, methods for incorporating such information through hypothesis-specific weights are well-established and have been shown to increase statistical power ( Benjamini and Hochberg, 1997 ; Genovese et al., 2006 ) . However, the implications of weighting for asymptotically optimal sequential procedures are unknown. It is unclear whether the benefits of weighting can be achieved without compromising the guarantee of asymptotic efficiency. This paper addresses this gap by developing a formal framework for weighted asymptotically optimal sequential testing. We introduce a modified statistic, the weighted log-likelihood ratio (WLLR), to integrate prior information directly into the evidence accumulation process. Building on this, we propose two procedures: the Weighted Gap Procedure for a known signal count and the Weighted Gap-Intersection Procedure for a bounded signal count. We prove that both methods provide strong FWE control. The central contribution of this work is to show that they are asymptotically optimal: their expected stopping times converge to the theoretical lower bound, demonstrating that weights affect only lower-order terms. We further analyze the high-dimensional regime where the number of hypotheses J grows, and a setting with random weights, characterizing the conditions under which optimality is preserved. Our results provide a principled methodology for leveraging prior information to enhance the efficiency of sequential experiments. The remainder of this paper is organized as follows. Section 2 formalizes the problem setup and notation. Section 3 provides a brief overview of related work. In Section 4 , we introduce the weighted sequential procedures and establish their FWE control. Section 5 contains the core results on asymptotic optimality. Section 6 extends the analysis to the random weights regime, and Section 7 presents a simulation study to evaluate the finite-sample performance of our methods. 2 Setup and Notations We consider the problem of simultaneously testing J simple null hypotheses against J simple alternatives. For each hypothesis j\in[J]:=\{1,\dots,J\} , we observe an independent stream of data, X^{j}=\{X_{i}^{j},i=1,2,\dots\} , generated sequentially in time. The probability measure governing the data stream j , denoted P^{j} , is assumed to be one of two distinct possibilities: P_{0}^{j} under the null or P_{1}^{j} under the alternative. The component testing problem for each stream is thus H_{0}^{j}:P^{j}=P_{0}^{j}\quad\text{vs.}\quad H_{1}^{j}:P^{j}=P_{1}^{j}. The complete state of nature is described by the set of true alternatives (signals), denoted by A\subseteq[J] . The complementary set, A^{c}=[J]\setminus