Weighted Holm Procedures: Theory, Properties, and Recommendations

Beibei Li, Wenge Guo · 2026

Read Weighted Holm Procedures: Theory, Properties, and Recommendations by Beibei Li, Wenge Guo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20016.txt.

Introduction

Li and Guo compare two weighted Holm methods for multiple testing with tests of unequal importance. The weighted Holm method orders weighted p-values, while the weighted alternative orders raw p-values. Their closed testing analysis shows the first is always more powerful, and further results show the second, though consonant, lacks monotonicity. Graphs aid general readers, and adjusted p-values follow for both. Optimality favors the weighted Holm method, while the second is optimal only under certain conditions.

Catalog record

CREATORBeibei Li, Wenge Guo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51877
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.20016

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Weighted Holm Procedures: Theory, Properties, and Recommendations Beibei Li, Wenge Guo Abstract In many statistical applications, particularly in clinical studies, hypotheses may carry different levels of importance, motivating the use of weighted multiple testing procedures (wMTPs) to control the familywise error rate (FWER). Among these approaches, two weighted Holm procedures are commonly used: the weighted Holm procedure (WHP), which is based on ordered weighted p p -values, and the weighted alternative Holm procedure (WAP), which relies on ordered raw p p -values. This paper provides a systematic comparison of these two procedures, along with practical recommendations for their use. We first examine their corresponding closed testing procedures (CTPs) and show that WHP is uniformly more powerful than WAP. We further investigate their structural properties, demonstrating that WAP, while consonant, lacks monotonicity. To facilitate communication with non-statisticians, we introduce graphical representations of both procedures using a common initial graph and distinct updating strategies. In addition, we derive adjusted p p -values and adjusted weighted p p -values for both methods. Finally, we establish an optimality result: WHP cannot be improved by enlarging any of its critical values without violating FWER control, whereas WAP is optimal only under specific conditions. Simulation studies support these theoretical findings and highlight the superior FWER control and average power of WHP. KEY WORDS: weighted Holm procedures; closed testing; graphical methods; adjusted p -values; optimality; FWER. 1 Introduction In many clinical applications, hypotheses differ in importance, this motivates the use of weighted multiple testing procedures (wMTPs). Several such methods have been developed, including weighted Bonferroni, Simes, and resampling-based approaches ( 25 ; 17 ; 2 ; 27 ; 31 ) . A weighted Bonferroni procedure, for example, can increase power for more important hypotheses. Among stepwise methods, two weighted Holm procedures are widely used: the weighted Holm procedure (WHP; 17 , 17 ), which is based on ordered weighted p -values, and the weighted alternative Holm procedure (WAP; 2 , 2 ), which relies on ordered raw p -values. WAP is often viewed as more objective since its ordering is independent of weights, but this comes at the cost of losing monotonicity, a key property in stepwise testing. Prior studies have examined weighted procedures under pre-ordered hypotheses ( 23 ; 33 ; 32 ) and weighted Simes-type approaches ( 27 ) , but the relative merits of ordering by raw versus weighted p -values remain unresolved. This motivates a careful comparison of WHP and WAP to clarify their statistical properties, evaluate their performance in terms of familywise error rate (FWER) control and power, and provide practical guidance. We show that WHP is uniformly more powerful than WAP, establish new optimality results, and demonstrate the advantages of WHP through theory and simulation. Bonferroni-type weighted procedures can also be formulated as closed testing procedures (CTPs), which offer a rigorous framework for FWER control ( 22 ; 3 ; 16 ) . However, as the number of hypotheses grows, the number of intersection tests increases rapidly, making CTPs difficult to apply in practice. Graphical methods provide a more accessible alternative: they can represent a wide range of multiple testing procedures (MTPs), are easier to communicate to clinical teams than lengthy decision tables, and reduce programming effort. They are now widely used in clinical research ( 4 ; 6 ; 5 ) . Weighted procedures also underpin key gatekeeping strategies, including serial, parallel, and mixture approaches ( 23 ; 29 ; 11 ; 9 ; 10 ) . Developing simple and powerful weighted procedures is therefore essential for advancing these frameworks. Moreover, their optimality—whether they can be improved without sacrificing error control—remains largely unexplored, especially for WHP and WAP. Although this paper does not address the choice of weights, their importance should be emphasized. Weights may be assigned based on the a priori importance of hypotheses or prior knowledge ( 29 ) , or chosen adaptively from data to improve power while preserving significance levels. Several approaches have been proposed for data-dependent weighting that still maintain familywise or generalized familywise error control ( 13 ; 19 ; 7 ; 30 ; 28 ) . In addition, weighted parametric procedures that exploit the joint distribution of test statistics have been studied ( 35 ; 34 ) . In summary, we compare WHP and WAP through their closed testing formulations, graphical representations, adjusted p -values, and optimality, and provide practical recommendations. We establish that WHP is uniformly more powerful than WAP. Although WAP lacks monotonicity—both with respect to raw p -values and within the closed testing framework—it remains consonant, which enables computational shortcuts, and its graphical formulation extends beyond the settings of 18 and 4 . Regarding optimality, WHP cannot be improved without sacrificing FWER control, whereas WAP is optimal only under specific weight-ratio conditions. Simulation studies and clinical examples further demonstrate that WHP consistently outperforms WAP, and that judiciously chosen weights enhance the power of both procedures, particularly WAP. The remainder of the paper is organized as follows. Section 2 introduces the notation. Section 3 presents the CTPs and associated monotonicity results. Section 4 develops graphical representations. Section 5 derives adjusted p -values. Section 6 establishes optimality properties. Section 7 presents simulation results. Section 8 illustrates applications with clinical examples. Section 9 concludes. All proofs are collected in Section 10 . 2 Preliminaries In this section, we introduce notation and describe the two weighted Holm procedures studied in this paper. 2.1 Basic Notation We consider the simultaneous testing of m hypotheses H_{1},\ldots,H_{m} with corresponding p -values P_{1},\ldots,P_{m} and positive weights w_{1},\ldots,w_{m} . The weighted p -values are defined as \tilde{P}_{i}=\frac{P_{i}}{w_{i}},\quad i=1,\ldots,m. Let P_{(1)}\leq\cdots\leq P_{(m)} denote the ordered raw p -values with associated hypotheses H_{(1)},\ldots,H_{(m)} and weights w_{(1)},\ldots,w_{(m)} . Similarly, let \tilde{P}_{(1)}\leq\cdots\leq\tilde{P}_{(m)} denote the ordered weighted p -values, with corresponding hypotheses H^{*}_{(1)},\ldots,H^{*}_{(m)} and weights w^{*}_{(1)},\ldots,w^{*}_{(m)} . Suppose that m_{0} of the hypotheses are true nulls and m_{1}=m-m_{0} are false nulls, and let I_{0} be the index set of the true null hypotheses. Denote by R the total number of rejections and by V the number of true nulls rejected. The FWER is defined as \text{FWER}=\Pr(V\geq 1), i.e., the probability of making at least one type I error ( 26 ) . Our goal is to control the FWER at a pre-specified significance level \alpha\in(0,1) . Throughout, we assume that for each i\in I_{0} , the corresponding p -value satisfies the standard super-uniform condition \Pr(P_{i}\leq x)\leq x,\quad\forall\,x\in(0,1), without imposing any restrictions on their joint distribution. 2.2 Two weighted Holm procedures We now describe two weighted Holm procedures that are widely used in practice. The key difference lies in how the hypotheses are ordered: WHP orders by weighted p -values, while WAP orders by raw p -values. This choice affects both their properties and performance. Procedure 2.1 . ( 17 ) . WHP. Reject H^{*}_{(i)} if \tilde{P}_{(j)}\leq\frac{\alpha}{\sum_{k=j}^{m}w^{*}_{(k)}},\quad j=1,\ldots,i. (1) Here, the hypotheses are ordered according to the weighted p -values. By incorporating weights into the ordering, WHP gives priority to hypotheses deemed more important, extending the idea of the weighted Bonferroni test. Procedure 2.2 . ( 2 ) . WAP. Reject H_{(i)} if P_{(j)}\leq\frac{w_{(j)}}{\sum_{k=j}^{m}w_{(k)}}\alpha,\quad j=1,\ldots,i. (2) In contrast, WAP orders hypotheses by their raw p -values, regardless of their assigned weights. This makes the procedure

Read the complete text