Debiasing hazard-based, time-varying vaccine effects using vaccine-irrelevant infections: An observational extension of a pivotal Phase 3 COVID-19 vaccine efficacy trial
Ethan Ashby, Dean Follmann, Holly Janes, Peter B. Gilbert, Ting Ye, Lindsey R Baden, Hana M El Sahly, Bo Zhang · 2025
Read Debiasing hazard-based, time-varying vaccine effects using vaccine-irrelevant infections: An observational extension of a pivotal Phase 3 COVID-19 vaccine efficacy trial by Ethan Ashby, Dean Follmann, Holly Janes, Peter B. Gilbert, Ting Ye, Lindsey R Baden, Hana M El Sahly, Bo Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-15099.txt.
Introduction
Ethan Ashby, Dean Follmann, and six co-authors fix a subtle bias in measuring how vaccine protection fades over time. Standard time-varying estimates from Cox regression can be distorted by hidden differences between vaccinated and unvaccinated people and by selection effects. The team instead uses infections that the vaccine cannot affect as a built-in control signal.
Under stated assumptions, those irrelevant infections reveal the hidden bias and let the true hazard-based effect be identified. Two estimator styles are developed, with options for monotone shapes and for several viral variants at once. As a case study, the authors revisit data modeled on a pivotal phase three COVID-19 vaccine trial.
Catalog record
| CREATOR | Ethan Ashby, Dean Follmann, Holly Janes, Peter B. Gilbert, Ting Ye, Lindsey R Baden, Hana M El Sahly, Bo Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-15099
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Debiasing hazard-based, time-varying vaccine effects using vaccine-irrelevant infections: An observational extension of a pivotal Phase 3 COVID-19 vaccine efficacy trial Ethan Ashby, Dean Follmann, Holly Janes, Peter B. Gilbert, Ting Ye, Lindsey R Baden, Hana M El Sahly, Bo Zhang Abstract Understanding how vaccine effectiveness (VE) changes over time can provide evidence-based guidance for public health decision making. While commonly reported by practitioners, time-varying VE estimates obtained using Cox regression are vulnerable to hidden biases. To address these limitations, we describe how to leverage vaccine-irrelevant infections to identify hazard-based, time-varying VE in the presence of unmeasured confounding and selection bias. We articulate assumptions under which our approach identifies a causal effect of an intervention deferring vaccination and interaction with the community in which infections circulate. We develop sieve and efficient influence curve-based estimators and discuss imposing monotone shape constraints and estimating VE against multiple variants. As a case study, we examine the observational booster phase of the Coronavirus Vaccine Efficacy (COVE) trial of the Moderna mRNA-1273 COVID-19 vaccine which used symptom-triggered multiplex PCR testing to identify acute respiratory illnesses (ARIs) caused by SARS-CoV-2 and 20 off-target pathogens previously identified as compelling negative controls for COVID-19. Accounting for vaccine-irrelevant ARIs supported that the mRNA-1273 booster was more effective and durable against Omicron COVID-19 than suggested by Cox regression. Our work offers an approach to mitigate bias in hazard-based, time-varying treatment effects in randomized and non-randomized studies using negative controls. Keywords : Negative control outcome; Observational studies; Vaccine effectiveness 1 Introduction 1.1 The promise and perils of hazard-based time-varying vaccine effects A key goal of vaccine research is to developing novel vaccines or adapt existing vaccines to be long-lasting and broadly protective against many pathogen strains. Additionally, understanding the breadth and durability of vaccine effectiveness (VE) can inform decisions regarding whether and when to offer vaccines to the public. For example, evidence of waning influenza vaccine effectiveness supports that vaccinating high-risk groups closer to the flu season may offer greater protection than vaccination well in advance of the flu season ( 5 ; 14 ) . Hence, understanding time-varying VE , or vaccine effectiveness in preventing disease X days after vaccination, is of interest to both vaccine science and public health. A common approach to estimate time-varying VE is to fit a Cox proportional hazards model with a coefficient that varies in time-since-vaccination ( 35 ; 36 ) . However, hazard-based VE estimates are vulnerable to subtle biases that compromise their reliability in both randomized and observational studies. We describe these sources of bias below. Even in randomized trials – the gold-standard study design for evaluating vaccines – hazard-based, time-varying vaccine effects can suffer from a built-in selection bias ( 24 ) commonly referred to as “depletion of susceptibles bias” ( 38 ) . In Figure 1 , we illustrate how depletion of susceptibles bias arises in a randomized trial of a moderately effective vaccine conducted in a population with heterogeneous infection risk. For illustrative purposes, suppose the study is conducted in a mixed population of “low-risk” (blue) and “high-risk” (orange) individuals. By virtue of randomization, the proportions of low and high risk individuals are balanced at baseline. Suppose we are interested in estimating the vaccine’s efficacy during the late period (T1 to T2) and comparing it to the early period (0 to T1) to determine if VE is waning. One approach is to compare the hazard ratios of infection between the early and late periods. By the definition of the hazard function, the late period hazard ratio compares infection rates between vaccinated and unvaccinated persons who were uninfected during the early period . Conditioning on no early period infection selects for a vaccinated risk set with higher average infection risk (i.e., higher proportion of orange participants) than the unvaccinated risk set because high-risk participants from the unvaccinated group were infected more rapidly during the early period. This example exhibits how hazard-based, time-varying vaccine effects can become entangled with time-varying imbalances in infection risk between vaccinated and unvaccinated risk sets. The result of depletion of susceptibles bias is exaggerated estimates of VE waning. Previous efforts to mitigate depletion of susceptibles bias included serological testing to detect prior natural immunity ( 30 ) , exploiting seasonal variation in infections to approximate a trial where exposure is deferred until all participants are vaccinated ( 38 ; 46 ) , and proposing a causal VE waning parameter where exposure is deferred until an analyst-chosen time point ( 29 ) . Figure 1 : Illustration of depletion of susceptibles in a randomized trial comparing placebo (top) to an effective vaccine (bottom) in the presence of heterogeneous infection risk. Conducting long, placebo-controlled vaccine trials may not be feasible in some circumstances due to resource limitations, lack of clinical equipoise, or pathogen evolution in the population. Observational studies contain rich data on how vaccine effectiveness varies across different settings, variants, and over time. However, VE estimates from observational studies are particularly vulnerable to bias. World Health Organization guidance documents ( 64 ; 65 ) highlight the myriad sources of confounding that could impact observational vaccine studies including differences in behaviors (such as mask-wearing and social distancing) influencing contact rates with infected community members ( 22 ; 54 ) , immune function influencing per-contact susceptibility to infection ( 22 ; 30 ) , and healthcare seeking behaviors impacting case ascertainment ( 27 ; 28 ) . These concerns are not theoretical: several observational studies in winter 2021 reported low and negative VE estimates, which are hypothesized to be the result of hidden bias ( 7 ) . 1.2 Negative control outcomes in vaccine studies Common strategies for bias mitigation like observed confounder adjustment are ill-suited to vaccine studies because the sources of bias are hard to measure, overlapping, and potentially time-varying. For example, a strong predictor of an individual’s infection risk — and therefore a potential source of bias — is their rate of contact with infected individuals in the broader community ( 22 ) . Accurate measurement of contact rates is nearly impossible for several reasons ( 56 ) . Contact rates change over time and measuring contacts requires knowledge about which members of the broader community are infected. Moreover, it is nearly impossible to establish a standard definition of “contact” for community-transmitted respiratory pathogens. In practice, analysts typically measure and adjust for demographic factors (e.g., age, employment as a healthcare worker, etc.) with the hope that they are correlated with the true, unmeasured predictors of infection. Another strategy for bias mitigation is to measure and adjust for a negative control outcome (NCO), or an auxiliary endpoint unaffected by the intervention but that shares unmeasured causes with the primary outcome ( 48 ; 39 ; 49 ) . A common choice of NCO in vaccine effectiveness studies is infection/illness caused by a pathogen/serotype antigenically distinct enough from the pathogen of interest such that the vaccine is assumed to have no effect – we refer to such infections as “off-target” or “vaccine-irrelevant” . NCOs have been used to detect ( 33 ; 25 ) and eliminate ( 9 ; 11 ) confounding bias in time-invariant VE estimates from observational studies. However, prior negative control approaches do not address selection