Anytime-valid testing with e-values and confirmatory adaptive designs

Werner Brannath, Lasse Fischer · 2026

Read Anytime-valid testing with e-values and confirmatory adaptive designs by Werner Brannath, Lasse Fischer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00878.txt.

Introduction

Brannath and Fischer compare two schools of flexible trial design in this 2026 paper. Confirmatory adaptive designs are over 30 years old. They let a clinical trial resize its sample or pick treatments, endpoints, and subgroups mid-course. E-value tests are newer: they promise anytime-valid inference, safe to check at any point.

The pair show the two schools rest on the same math. Tools like conditional error functions and combination tests are formally equal to e-value sequential tests. The aims still differ in stress. Adaptive tests tend to spend the full type-I error budget the freedom allows. E-value tests prize optional continuation, a level picked on the fly, and control of set loss functions. The authors sketch cross-gains both ways. Adaptive craft could enrich e-value tests. E-value wins could refresh trial method in turn.

Catalog record

CREATORWerner Brannath, Lasse Fischer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00878

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Anytime-valid testing with e-values and confirmatory adaptive designs Werner Brannath, Lasse Fischer Abstract Confirmatory adaptive designs were introduced more than 30 years ago and enable for example sample size re-assessments and the selection of treatments, endpoints as well as subpopulations during the course of a clinical trial. Recently, sequential tests based on e-values for an anytime-valid inference have been developed, promising seemingly similar or even more flexibility and utility. In this note, we compare these two independently developed concepts, shedding light on their formal and methodological connections and differences. Specifically, we show that adaptive design tools like conditional error functions and combination tests are formally equivalent to e-value based, anytime-valid sequential tests. However, in spite of their common fundamental intention to bring flexibility into statistical inference, they have quite different emphases: While hypothesis testing with combination tests and conditional error function usually intent to exhaust type I error rates under the offered flexibility, e-value based testing aims on the additional flexibility with regard to optional continuation, the chosen level and, in recent extensions, in the loss functions to be controlled. We also indicate how recent e-value achievements could enrich clinical trial methodology and adaptive design methodology could inspire and improve e-value based testing. 1 Introduction The powerful e-value theory for anytime-valid inference, which emerged in the recent years [ 13 , 31 , 37 , 40 ] , permits to stop and test (or continue) with accumulating data at any time during the data accumulation process. An important characteristics is the fact that, in order to control the probability for false rejections, the data driven stopping time does not need to be prespecified. The flexibility with regard to the sample size reminds one to the flexibility of confirmatory adaptive designs (in the following, we just write adaptive designs for brevity) [ 3 , 30 , 45 ] , in which (among others) data-driven adaptations of sample size and number of stages without a prespecification of the adaption rule are permitted as well. Adaptive designs are primarily developed for clinical trials, in particular for phase II and phase III clinical trials and combinations thereof. With the conditional rejection probability principle [ 25 , 26 ] and recursive combination tests [ 8 ] such adaptions can be done at any time during the data accumulation process. This leads to the natural question of how these seemingly different approaches are related. This question is addressed in this note. We will show that concepts of adaptive designs, like combination tests and conditional error functions, are formally equivalent to anytime-valid test based on sequential e-values. We also illustrate a fundamental difference of the two approaches in their emphasis: While combination tests and conditional error functions intent to exhaust type I error rate levels, meaning to have a type I error probability of exactly \alpha , e-values intend to introduce flexibility with regard to optional continuation, choice of type I error rate levels and, more general, the expected losses to be controlled. These two emphases are complementary in the sense that the first cannot be achieved without violating the second and vice versa. Moreover, the e-value based perspective has the potential to provide solutions where the adaptive design approach may fail because the latter requires more information on the null distribution of the test statistics than required for e-value based tests. It should be noted that we do not claim to derive substantially new methodology in this paper but show the yet unexplored connections and differences between the closely related approaches of e-values and adaptive designs. By pointing out connections, we aim to build a bridge between the yet rather weakly connected scientific communities and literature. In addition, recent achievements of e-values are highlighted in the discussion that may also be valuable to the adaptive design community. On the other hand, achievements in the field of adaptive designs may inspire new achievements and improvements for e-value based sequential tests, like the uniform improvement via recursive combinations tests presented in the Appendix. Anytime-valid inference via e-values. We start with a brief and basic introduction to anytime-valid inference via the e-value methodology. Let us assume that we are interested in testing a null hypothesis H_{0} with a possibly infinite stream of data. From this stream of data we calculate over time a sequence of non-negative statistics E_{1},E_{2},\ldots , where each E_{t} can either be calculated based on one data point or a batch of data, with the property that at any time point t \displaystyle\mathbb{E}_{\mathbb{P}}\left[E_{t}|E_{1},\ldots,E_{t-1}\right]\leq 1\text{ for all distributions $\mathbb{P}\in H_{0}$}. (1) This particularly allows to choose each E_{t} based on the data (and external information) of the stages 1,\ldots,t-1 , as long as ( 1 ) is satisfied. A sequence of statistics satisfying ( 1 ) is often denoted as “sequential e-values” [ 41 ] , whereby an “e-value” is understood to just be a nonnegative random variable with expected value at most 1 under H_{0} . Sequential e-values can be used for anytime-valid inference in the following way: Defining the test statistics M_{0}=1 , \displaystyle M_{t}=\prod_{i=1}^{t}E_{i},\quad t=1,2,\ldots, (2) we control the type I error at level \alpha when rejecting H_{0} if M_{t}\geq 1/\alpha at any time t . This follows from Ville’s inequality [ 38 ] (a sequential generalization of Markov’s inequality), since (M_{t})_{t\in\mathbb{N}_{0}} is a nonnegative supermartingale under H_{0} with M_{0}=1 . Such a process (M_{t})_{t\in\mathbb{N}_{0}} is also called test supermartingale [ 39 , 36 ] . Furthermore, by the optional stopping theorem \mathbb{E}_{\mathbb{P}}[M_{\tau}]\leq 1 for any stopping time \tau and \mathbb{P}\in H_{0} , showing that every stopped test supermartingale is an e-value. Example 1 (Likelihood ratios) . Suppose we have an i.i.d. sequence of data X_{1},X_{2},\ldots and are testing a simple null hypothesis against a simple alternative. Let p_{0} be the density of the null distribution and p_{1} the alternative density. Then the likelihood ratios \lambda_{t}=\frac{p_{1}(X_{t})}{p_{0}(X_{t})},t\in\mathbb{N}, define sequential e-values, since \mathbb{E}_{\mathbb{P}_{0}}[\lambda_{t}|\lambda_{1},\ldots,\lambda_{t-1}]=\int_{\text{supp}(p_{0})}\frac{p_{1}(x)}{p_{0}(x)}p_{0}(x)\ \mu(dx)=\int_{\text{supp}(p_{0})}p_{1}(x)\ \mu(dx)\leq 1. The resulting anytime-valid test is equivalent to the sequential probability ratio test (SPRT) by 42 , if no lower bound to accept H_{0} is specified. Likelihood ratios play an important role in the e-value theory, since log-optimal e-values always take the form of a likelihood ratio, even if the null hypothesis is composite [ 13 , 22 ] . In Section 5 we will give a brief description of log-optimality. It is valuable to note that p_{0} in \lambda_{t} can be the maximum over the densities of a composite null hypothesis H_{0} without violating the e-value property of \lambda_{t} . Moreover, the numerator p_{1} could be the maximum of the densities from some composite alternative determined with an independent training sample. These ideas have been introduced in [ 44 ] and are known as “universal inference”. Remark 1 . There are sequential tests that cannot be constructed by multiplying sequential e-values in the way described above [ 32 ] . In general, one would need to define a sequential e-value E_{t}^{\mathbb{P}} for each null distribution \mathbb{P}\in H_{0} , multiply those sequential e-values M_{t}^{\mathbb{P}}=\prod_{i=1}^{t}E_{i}^{\mathbb{P}} and then define (M_{t})_{t\in\mathbb{N}_{0}} such that M_{t}\leq M_{t}^{\mathbb{P}} \mathbb{P} -almost surely for all \mathbb{P}\in H_{0} and t\in\mathbb{N}_{0} [ 33 , 31 ] . Although (M_{t})_{t\in\mathbb{N}_{0}} is not necessarily a test supermartingale in this case, it is easy

Read the complete text