Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2026 Sep 3.
Published in final edited form as: Vaccine. 2025 Sep 3;64:127645. doi: 10.1016/j.vaccine.2025.127645

Adopting the Estimand Framework in Prophylactic Vaccine Trials

Francois Beckers 1, Naveen Karkada 2, Ye Yang 3, John Scott 4, Lei Huang 5, Florian Klinglmüller 6, Michael P Fay 7, Stefan Englert 8, Bart Spiessens 9, Ilse Van Dromme 10, Tulin Shekar 11, Honghong Zhou 12, Qiqi Deng 13, Daniela Casula 14, Holly Janes 15, Lawrence H Moulton 16
PMCID: PMC12433591  NIHMSID: NIHMS2110086  PMID: 40907068

Abstract

The estimand framework as outlined in ICH E9(R1) has been extensively discussed and implemented in clinical trials of therapeutic products. However, there is limited literature on the application of the framework in preventive vaccine trials, which has many unique characteristics, including emphasis on estimating the per-protocol or “biological” effect. We provide a comprehensive review of the application of the framework to preventive vaccine trials evaluating clinical outcome and immunogenicity, focusing on commonly encountered intercurrent events including but not limited to: noncompliance with vaccination schedule and blood sampling window, infection not meeting protocol definition, death, and use of prohibited products. We discuss various considerations in choosing strategies to handle intercurrent events in terms of their utility in addressing the scientific questions. Finally, we provide considerations and examples for summarizing study estimands and data handling which may be incorporated into the protocol and statistical analysis plan.

Keywords: Estimand, ICH E9(R1), Vaccine

1. Introduction

The goal of most clinical trials is to investigate whether a new intervention is efficacious and safe compared to a control. Generally, participants are randomly assigned between intervention arms to control both known and unknown confounders so that the observed differences between arms could be inferred as the causal effect.

However, events may arise after randomization that affect the interpretation of the target effect and its estimate. Events occurring after study intervention, referred to as intercurrent events, could be related or unrelated to the intervention and affect either the interpretation or the existence of the measurements associated with the clinical question. Randomization does not ensure that the nature and frequency of intercurrent events will be balanced between trial arms, as their occurrences may be associated with the intervention. Thus, the observed differences in the measurements might not reflect the true causal effect in question. Therefore, it is necessary to address intercurrent events to properly define the effect that is to be estimated, the estimand (1).

The ICH E9 (R1) guidance provides a structured framework for translating trial objectives into the design, conduct and analysis of a clinical trial, to concisely articulate information on the intervention effects to be estimated (1, 2). The guidance has been adopted and extensively discussed in various areas such as oncology (3–6), metabolic diseases (7, 8) and others (9). In prophylactic vaccines, few publications exist (10, 11), with no joint effort and consensus on implementation, leading to heterogeneity and inconsistency across vaccine study protocols and analysis plans (12). Thus, it is valuable to further explore the interpretation of ICH E9(R1) in vaccine trials. This work focuses on the specifics of prophylactic vaccines development (Section 2), the concept of estimands for clinical outcome and immunogenicity related objectives (Section 3), and finally considerations for the implementation of the estimand framework in study protocols and analysis plans.

2. Specifics of prophylactic vaccine clinical development

A fundamental element of prophylactic vaccine trials is that they are generally conducted among healthy individuals with the goal of evaluating a vaccine’s protective effect based on a clinical outcome. Endpoints include the occurrence of the specific clinical outcome or a surrogate such as an immunological response. While for many therapeutics, the therapeutic effect of the intervention (or lack thereof) may be perceived by an individual and in turn influence the individual’s ability or willingness to adhere to the protocol, the protective effect of the vaccine generally cannot be perceived prior to the primary endpoint evaluation. As such, one may reason that many intercurrent events associated with protocol deviations or the participant’s medical state are unlikely to be directly related to the vaccine efficacy and thus are unlikely to complicate its interpretation. In most cases, rescue treatments for the disease of interest do not apply prior to onset of the endpoint and are therefore irrelevant in the vaccine effect estimation. Exceptions may include, for example, use of antibiotics, antivirals, or post-exposure vaccination which can alter disease progression or detection, including longer-term outcomes such as hospitalization or ICU admission.

Vaccine administration consists of limited number of doses, fewer than most therapeutics, and so it is reasonable to expect that intervention compliance will be higher in vaccine trials. The level of compliance is often as high as 90–95%. Vaccine toxicity is low compared to many therapeutics, and dropout rate due to toxicity is low in vaccine trials (13).

3. Estimands in prophylactic vaccine trials

ICH E9(R1) provides a structured approach to precisely describe the treatment effect to be estimated. The process starts by considering the background and clinical setting.

3.1. Questions of clinical interest in vaccine trials

The vaccine clinical effect to be estimated is usually the relative disease reduction following vaccination, with a focus on disease incidence after the individual is expected to have achieved adequate immune response and after the disease incubation period (14). The objective might be to evaluate the vaccine efficacy among those who would be able to fully comply with the intended regimen, sometimes referred to as the “biological effect” (13), or to estimate a vaccine effect that may be expected in a setting closer to “routine use” (i.e., effectiveness) among participants receiving at least one dose of the intervention regardless of vaccination compliance and occurrence of other protocol deviations, despite their potential impact on the endpoint(s).

Vaccine trials aim to address specific questions in terms of clinical outcome or immunogenicity:

  • Does the vaccine confer some meaningful level of protection (15)?

  • What protection is maintained over a certain time-period?

  • What protection is conferred against specific disease subtypes (i.e., severity, strains)?

  • What protection is conferred to individuals from specific populations (e.g., immuno-compromised, immunologic naivety, etc.)?

  • What is the direct and indirect vaccine effectiveness following a vaccination campaign?

3.2. Definition of estimands

Description of the targeted vaccine effect consists of the following five attributes (1):

  • Treatment Condition: The intervention is usually the experimental vaccine and a control, either placebo or alternative approved vaccine for the same or another indication.

  • Population: This is the group of individuals targeted by the clinical question. Often, vaccine trials focus on a broad population of healthy individuals or special populations (e.g. children, immuno-compromised, pregnant women).

  • Variable or Endpoint: This refers to the individual data level measure to address the clinical question. This is typically a measure of immune response for immunogenicity related questions (i.e., antibody titers or concentrations), the occurrence of the disease of interest for efficacy or effectiveness related questions or the occurrence of solicited symptoms and adverse events for safety objectives. In practice, each of these endpoints encompasses a broad spectrum of measurements. For example, immunogenicity may be assessed through both humoral and cellular responses, and clinical endpoints may span a continuum from asymptomatic infection to severe disease outcomes, such as hospitalization or death. The definition should be precise and ideally includes the evaluation period for each participant. Hudgens et al. (16) provide further discussion of various vaccine endpoints and their analysis.

  • Population-Level Summary: This provides the basis for comparison between interventions. For immunogenicity objectives, this may be the serological response, serological conversion rate difference, or a geometric mean antibody concentration/titer ratio. For efficacy or effectiveness objectives, the vaccine effect is normally the population-level summary, which may be defined based on the relative risk, hazard ratio, odds ratio or incidence ratio of the clinical endpoint (17). Different vaccine effect definitions represent different estimands while the differences between the estimates are typically small when the incidence rate is small (18).

  • Intercurrent Events: These are events occurring after study intervention that affect the interpretation or existence of measurements associated with the clinical question.

The estimand is driven by the specific clinical question being investigated and different estimands will be required to tackle different clinical questions. A clearly articulated estimand supports developing the appropriate trial design and data collection. Specifying relevant intercurrent events allows for implementation of measures to minimize their occurrence, evaluating their impact towards the study objectives, planning appropriate sensitivity and/or supplemental analyses, and providing clarity on vaccine effects for decision making.

3.3. Intercurrent event handling strategies

The different strategies described in ICH E9(R1) are summarized as follows:

  • Treatment Policy strategy, building on the intent-to-treat (ITT) principle, considers the intercurrent event irrelevant to the treatment effect. This strategy requires data to be collected after event occurrence or an appropriate missing data strategy for data that are not collected. The observed values are used regardless of whether the intercurrent event occurs.

  • Hypothetical strategies concern the hypothetical values that would have been observed under a specific scenario in which the intercurrent event in question would not have occurred. Under this strategy, values observed for the endpoint after the occurrence of the event are not of interest, though they may be relevant to inform the target hypothetical value. As hypothetical values are unobservable, they must be imputed under (possibly unverifiable) assumptions. It is important to note that the hypothetical strategy is not a single, fixed approach but a general framework within which different hypothetical scenarios can be defined each corresponding to the absence of a specific intercurrent event. These scenario-specific strategies should be clearly defined and justified depending on the clinical question of interest.

  • While-on-Treatment strategy restricts the observation time to the period before occurrence of the intercurrent event. Similar to the hypothetical strategy, values observed after the event are not of interest. The main difference is that the while-on-treatment strategy ignores the data after the intercurrent event, while the hypothetical strategy imputes post event data with hypothetical values. Of note, the original randomization may not be maintained for data observed after the intercurrent event.

    Terminology for this strategy will depend on the intercurrent event (e.g. “while-alive” when considering death as an intercurrent event, “while-on-condition” considers, for instance, expected medical conditions).

  • Composite Variable strategy relates to the variable of interest. Under this strategy, the intercurrent event is incorporated into the definition of the variable. This strategy is applied when the event itself is clinically relevant.

  • Principal Stratum strategy relates to the population of interest. The target population might be taken to be the principal stratum in which an intercurrent event would occur, or the principal stratum in which an intercurrent event would not occur. The objective is to estimate a treatment effect in a population whose status with respect to the intercurrent event(s) would be identical under either treatment assignment (19). In situations where the occurrence of an intercurrent event is not expected to be influenced by the intervention arm (e.g. based on past experience or a priori belief), it may be appropriate to implement the principal stratum strategy by merely excluding participants who experience these events for that specific objective (20, 21). This is based on the belief that if the participant was assigned to a different arm, the intercurrent event would still occur. However, this belief is unlikely to be true if the intercurrent event is correlated with the vaccine arm.

For the hypothetical, principal stratum, and while-on-treatment strategies, causal inferences are subject to underlying assumptions that need to be evaluated. Therefore, the context of use and the likelihood of occurrence must also be considered.

The following sections will review commonly encountered intercurrent events in vaccine trials. We discuss in detail different strategies for addressing them and the potential context for their use.

Non-compliance with vaccination schedule (for multi-dose vaccines) or vaccine administration protocol

Deviation from the planned vaccination window(s) or from expected vaccine administration procedure (e.g., missed dosing, incorrect vaccination route, receiving the wrong vaccine) is a set of intercurrent events which may modify the vaccine effect, as the magnitude of the immune response and protection depends on compliance with the intended vaccine administration schedule. These deviations may be considered as different intercurrent events and addressed via different strategies, or combined as a single event as their potential impact is generally similar.

Studies have conventionally targeted vaccine effects in individuals who would be able to comply with the administration schedule, leveraging a principal stratum strategy, as individuals and providers will interpret effects as applying only to the full regimen, and full compliance with the administration schedule that is expected in routine use, particularly for vaccines that are required as a condition of employment or school attendance.

While interest lies firstly in understanding the effect of the vaccine in a population that adheres fully to the recommended administration schedule, as this is crucial for ensuring optimal individual and public health outcome, causal inference of the effect targeted by the principal stratum strategy may become impractical or difficult to interpret if: (i) the level of compliance is low (e.g. <90%); (ii) non-compliance is differential between arms and is plausibly associated with the individual’s risk of disease in a given arm. Examples include failure to comply with the intended regimen due to adverse events that are associated with the magnitude of immune response, or with certain characteristics (e.g. age) that are in turn associated with the risk of disease with a differential impact between arms. These conditions can be examined using the same principles in handling missing data.

In cases such as these where the principal stratum strategy is difficult to implement or interpret, it may be of greater interest to apply the treatment policy strategy to such intercurrent events, estimating the clinical effect among the entire randomized population. Of note, as the compliance level in a study is usually much higher than vaccine uptake in the real-world due to the fact that study participants have already consented to vaccination, estimands based on the treatment policy strategy may not reflect real-world compliance.

A hypothetical strategy seeks to estimate the vaccine effect under a hypothetical scenario where the deviation would not have happened, in contrast to the principal stratum strategy which seeks the effect in a target population of individuals who would comply. In practice, the hypothetical strategy has limited interpretability and practical value, as few stakeholders would present or use the vaccine for its “hypothetical” effect, and thus this strategy is best to be avoided (22).

Use of a protocol-prohibited therapy or concomitant vaccination

Relevant intercurrent events include the use of certain medications and vaccines categorized as prohibited immune modifiers in the protocol hampering an individual’s immune response.

A treatment policy strategy considers immune modifiers as irrelevant for the clinical question and is generally the most appropriate strategy to evaluate the clinical effect if use of the medication resembles real-world clinical practice. In contrast, receipt of a non-study vaccine intended for the same indication is usually not expected in routine practice, and thus if interest is in estimating the protective effect solely due to the study vaccine, a while-on-condition strategy (i.e., using data until occurrence of the intercurrent event) for clinical effect may be adopted to maximize information use.

For immunogenicity, if the objective is to quantify vaccine-induced immune responses in an otherwise healthy population, then the principal stratum strategy excluding recipients of any prohibited immune modifiers prior to the immunogenicity assessment may be the most appropriate strategy. Of note, use of certain prohibited immune modifiers expected in routine practice often signals a special population (e.g., immunocompromised) that may be of clinical interest to study in a separate trial.

As discussed previously, caution should be exercised when adopting the while-on-condition and principal stratum strategies, which should carefully consider, in the context of the scientific question, 1) the nature and frequency of prohibited immune modifier use, 2) plausibility that use may be related to an individual’s risk or immune response, and 3) representativeness of the target estimand to that expected in routine use.

Death

Deaths may or may not be related to the pathogen or interventions under study. The cause of death must be carefully considered in terms of its relevance to the objective. If death is attributed to the pathogen under study, the composite variable strategy is the most sensible strategy to address this intercurrent event. Death would be included as part of the clinical endpoint definition. In an immunogenicity objective, one may reason that death was due to inadequate protection, and thus under the composite variable strategy one may impute an immune response value of below the lower level of quantification (<LLOQ), or a value based on a known disease-risk model for post-death response time points. When death is not expected to be attributed to the disease or pathogen under study, it is irrelevant to the question of interest, and a while-alive approach can be justified assuming that death occurred completely at random.

Individual Unblinding and/or Crossover

A specific situation arises when a new vaccine, targeting the same indication and population as the investigational study vaccine, becomes available in the country. If individual unblinding/crossover is considered to occur completely at random, a while-participating strategy may be considered. If it is expected to be related to other factors, then the while-participating strategy may be challenging to justify. In this scenario, the hypothetical approach may be adopted targeting a scenario of what would have been observed had the subject continued participation. In this case, however, a justifiable imputation strategy will be needed, which may require untestable assumptions. The treatment policy strategy would not be relevant as only placebo recipients are expected to crossover and the interpretation of clinical data observed thereafter would substantially change. A designed blinded crossover is possible and under certain assumptions may be used to estimate a longer-term vaccine effect (23, 24).

Infection not fulfilling variable definition

Infection with the target pathogen after randomization is a major intercurrent event that can significantly impact the assessment of the vaccine effect. Such infections can trigger an immune response independently of vaccination, thereby complicating the interpretation of the observed vaccine effect.

For a clinical effect objective, a first consideration relates to the endpoint evaluation period of onset after a prespecified number of days post-vaccination, allowing the immune system to be effectively stimulated. Any case occurring before the evaluation period would not be considered as an endpoint but may be considered as an intercurrent event. In most cases, a treatment policy strategy ignoring the early event may be the more interpretable (i.e., easier to communicate) estimand. This strategy targets a vaccine effect in individuals regardless of infection during induction of immune responses. The principal stratum strategy may also be relevant in targeting a vaccine effect in individuals not infected during immune response induction. Special consideration is needed on the potential impact of early events on the ability to perform causal inference which depends heavily on the length of time between first vaccination and the beginning of the evaluation period. If the vaccine is expected to confer some level of protection against early infection, one may consider adjusting the targeted effect by moving the evaluation period to immediately after vaccination.

The other consideration relates to the infection symptomatology. The simplest approach may be to apply a composite variable strategy by defining the primary endpoint as any infection regardless of symptomatology. However, less symptomatic infections may be difficult to detect through passive surveillance and might not be the most relevant to public health. More importantly, if the objective is to assess the vaccine’s effect in preventing symptomatic or severe disease, it would not be appropriate to apply this strategy to infections that do not rise to a predefined severity. The treatment policy strategy is conventional and intuitive: individuals either meet the endpoint definition, or they do not. A principal stratum strategy targeting never-infected individuals, or a hypothetical strategy targeting the hypothetical scenario had the individual not been infected are less meaningful and interpretable. Depending on the nature of disease, some infections may be considered a competing risk where an individual is censored at the time of infection via a while-at-risk strategy, assuming that the individual is no longer at-risk post infection. In most cases, however, the treatment policy strategy is appropriate as it would maximize the use of information and account for the effect of routinely occurring natural infections.

When comparing immunogenicity across groups, both the treatment policy strategy and the principal stratum strategy targeting non-infected individuals can lead to overestimation of the immune response, particularly if infection is associated with a lower response to the vaccine. However, the principal stratum strategy may result in a smaller degree of inflation compared to the treatment policy approach, which includes all participants regardless of infection status. Similarly, if a higher proportion of subjects receiving the less effective vaccine(s) is expected to be infected, both strategies can lead to an underestimate of the difference in immune response between vaccines, with the latter likely to a lesser extent. An alternative, but less common, approach to address this issue is a composite variable strategy. This may be implemented, for example, by setting a value of LLOQ for the infected individuals, assuming that infection is due to a lack of an adequate immune response.

Two strategies for handling infections in assessing immunogenicity persistence are most relevant: the treatment policy strategy addresses the clinical question of how antibody levels in a vaccinated population change over time in a setting where the population is exposed to the pathogen; the while-at-risk strategy address the question of how antibody levels in response to initial vaccination change over time, which may be useful in informing the need for and timing of booster doses. In the latter setting, systematic detection of infection onsets is essential.

Non-compliance with blood sampling window

As antibody levels rise and wane over time after vaccination, the timing of the blood collection to evaluate immunogenicity is critical. As such, the variable definition would ideally include the time(s) window for blood collection, with any measurement outside the window as irrelevant for that endpoint.

It is usually not anticipated that an out-of-window sample collection would be related to the immune response itself, the intervention or other confounding factors. As such, out-of-window collection is typically a missing data issue and can best be approached with appropriate missing data methods.

However, it can be possible that the reason for noncompliance with the assessment schedule is an intercurrent event (i.e., an event that changes the existence or interpretation of the endpoint of immune response at a particular point in time). In such cases, depending on the specific intercurrent event, one may consider a principal stratum strategy targeting the population of individuals who would be able to comply with the collection window. A treatment policy strategy in which the within-window immune response is imputed by the value from the sample collected closest in time to the window would allow a larger number of samples to be considered but may come at the expense of precision and interpretability from samples collected substantially out of window. A hypothetical strategy targeting within-window values assuming the noncompliance reason did not occur is likely of limited interest given the associated challenges with interpretability.

4. From estimand to estimates: data handling

As described by Ratich et al. (19), the estimand should be driven by the clinical question of interest. The next step is to choose the appropriate estimator and specify the analysis to obtain the estimates (Figure 1). The approach in handling trial data should be aligned with the defined estimand.

Figure 1.

Figure 1

Data handling strategies

While presented as strictly sequential, in practice, it is an iterative process as the estimand and estimator need to bridge the question to the estimates. The selection of the estimator is tied to the population-level summary attribute of the estimand (see Section 3.2). However, it requires specifying further the actual statistical method. For example, vaccine clinical effect might use an exact binomial approach conditional to the total number of cases (25, 26), possibly adjusted for surveillance time, or a time-to-event method with a log-rank test or a Cox model with proportional hazards assumption. Alternative estimators allowing vaccine efficacy or effectiveness to vary over time and possibly involving delayed vaccination in the placebo arm have been proposed (27–30). Although more advanced estimation methods can be used as causal inference methods such as the Structural Distribution Model (18, 31, 32), they often lead to similar estimates as the traditional methods (18).

Next, the analysis set on which the methods will be applied needs to be specified. While the definition and use of analysis set might appear at first sight as inconsistent or redundant with the estimand framework, it remains very relevant:

  • The population targeted by the clinical question might be either the entire trial population or a subgroup defined by particular baseline characteristics.

  • Certain protocol deviations/intercurrent events might categorically exclude participants to the extent that no data contributed by those participants are of primary clinical interest.

  • A defined analysis set is critical to meaningfully describe the demographics, baseline characteristics and disposition for the population(s) of interest, facilitating readers’ understanding of the population(s) analyzed and reproducibility and comparison between trials for similar objectives.

  • Different estimators targeting the same estimand may require different analysis sets.

In the next sections, we discuss considerations in defining the analysis set and handling of missing data.

4.1. Analysis set of interest

The analysis set consists of all participants in the study population who contributed information to the analysis. The primary analysis set is ideally representative of the target population to minimize bias and maximize generalizability. Several analysis sets could be used to evaluate study objective(s), with each estimand having its own analysis set. For simplicity, we consider two analysis sets:

  • The Full Analysis Set (FAS) closely represents the study population and the intent-to-treat principle. It consists of all randomized participants with minimal exclusion (33). The FAS would include those who received at least one dose of the study intervention regardless of protocol deviations.

  • The “Evaluable Subjects Set” (ESS), also referred to as the “Per-Protocol Set” in ICH E9, restricts the set to participants who have adequately received the randomized intervention in the specified time window and do not have intercurrent events that are deemed to have major impact on the clinical assessment that may be handled reasonably with a focus on the strata of individuals who would adhere/comply. The ESS terminology with reference to the objective (e.g. Efficacy ESS) may be considered as a more general and comprehensive term under the estimand framework.

When defining the analysis set with different exclusion criteria, it is important to indicate the rationale including: (i) target population definition; (ii) intercurrent event strategy according to principal stratum; and (iii) missing data handling.

4.2. Missing data handling

As the estimand is defined at the planning stage, neither study withdrawal nor other reasons for missing data (e.g., administrative censoring in trials with survival outcomes) are in themselves intercurrent events while the reasons for them might be. For instance, participants who withdraw from the trial may have experienced an intercurrent event before withdrawal, such as an adverse event, or exposure to a newly available, non-study vaccine. Thus, in the absence of an intercurrent event associated with the discontinuation, data that would have been observed after discontinuation may be regarded as missing.

Missing data may be a source of bias and increase the variance of the estimate. While every effort should be taken to minimize their occurrence, in practice it is very hard to avoid them completely. The reason for missing data and their handling may critically affect inference (34). Many publications and guidelines discuss methods to address them (34–38). Missing data are those that would be meaningful for the analysis of a given estimand but were not collected (39). They should be distinguished from those that do not exist or not considered meaningful due to an intercurrent event.

Handling missing data is part of estimation process and would be addressed after establishing appropriate intercurrent events strategies. While the method of handling missing data is not intended to directly manage intercurrent events, certain imputation techniques or model-based approaches, developed for the handling of missing data, are useful in estimating the hypothetical values of the variable of interest (19, 40)

5. Implementation in prophylactic vaccine protocol

As the estimand framework is targeted for statistical inference, this is expected to be implemented for the primary and key secondary objectives in pivotal trials. However, it is advisable to implement the thinking process for early phase trials as well.

Supplemental Table 1 proposes a way to present the primary estimand associated with each study objective. Supplemental Table 2 provides a more exhaustive overview of both the estimand and estimator, adding elements related to the analysis for obtaining the most relevant estimate, providing a clear understanding on how the primary and key secondary objectives will be answered. As in most studies, if the population definition is common to all study objectives, there is no value in repeating the definition for each of the objectives.

In Supplemental Table 2, one column describes the Population Level Summary to address the objective, being explicit about the estimator and including the decision criteria where relevant. The last column refers to intercurrent events strategies and Data Handling that would encompass: (i) intercurrent events strategies with reference to the analysis set; (ii) any aspect related to the handling of missing data for the particular objective.

When it is desirable to strictly dissociate the elements of the estimand from the estimator and estimates, one might adopt a more discriminating layout as proposed by Kang et al. (41).

Aligned with each estimand, a main estimator is established. In addition, sensitivity analyses may be planned in the statistical section to evaluate the robustness of the estimate when the validity of underlying assumptions are uncertain.

6. Conclusion

The estimand framework provides a valuable structure to communicate clearly the vaccine effect that is being targeted and maximize the use of relevant data. In this paper, we have discussed the specificities of prophylactic vaccine clinical trials and general considerations in constructing estimands according to ICH E9(R1), including detailed descriptions of common intercurrent events and potential strategies for addressing them. The choice of strategy depends not only on the specific trial objective(s) but also on the medical and intervention context, the frequency and timing of intercurrent events, and their likelihood to be related to the intervention or outcome. In addition, we have provided general considerations for defining analysis sets that will be used to obtain estimates of each estimand.

If the goal is to evaluate the vaccine’s “biological effect”, the principal stratum, hypothetical, and while-on-condition strategies will be relevant in addressing many intercurrent events. These strategies are similar in the sense that data collected in the presence of the intercurrent event are not of primary interest but nonetheless have important differences.

Per ICH E9(R1), the principal stratum is the subpopulation in which the intercurrent event would not (or would) occur, while the hypothetical strategy concerns the entire study population in the scenario in which the event would not occur. Similarly, the while-on-condition strategy considers information from the entire study population prior to occurrence of the intercurrent event. The principal stratum approach differs from the hypothetical and while-on-condition approaches in their target population. The while-on-condition approach differs from the hypothetical approach in the sense that, in the former scenario, data occurring after the intercurrent event are not of interest and thus are not part of the estimand or constitute a missing data problem, while in the latter scenario, such data are of interest but are impossible to observe, and instead values need to be imputed.

Alternatively, although not necessarily of primary interest, one may reconsider the scientific question at hand and adopt a treatment policy strategy, which avoids the potential causal inference problem inherent in the three aforementioned strategies and, for certain events, more accurately reflects routine use.

The framework may also be applied to safety objectives. Except for certain intercurrent events such as failure to receive any intervention or certain Good Clinical Practice-related deviations, treatment policy is the common and appropriate approach for addressing intercurrent events.

The choice of the strategy for addressing an intercurrent event ultimately depends on the scientific question that is the most sensible to the stakeholders, considering also the ability of the trial to adequately answer that question. Ultimately, the fundamental question clinicians and statisticians should ask themselves is, “Why do we think our primary estimand and associated estimator and estimates best allow to answer the study objective and generalize to the overall target population?”

Supplementary Material

1

Acknowledgements

We thank the following reviewers for their contributions to this paper (listed in alphabetical order): Sang Ahnn (FDA/CBER), Brigitte Cheuvart (GSK), Leroy Jide Ovbude (GSK), Tsai-Lien Lin (FDA/CBER), EFSPI/EFPIA Estimands Implementation Working Group (EIWG, www.eiwg.info).

Funding:

This study did not receive any funding

Declaration of Interest Statement

Francois Beckers is an employee of Sanofi and reports administrative support from the same. Naveen Karkada is an employee of GSK. Stefan Englert is an employee of Janssen-Cilag GmbH and holds equity or stocks in the company. Bart Spiessens reports employment by Janssen Pharmaceutica NV and GlaxoSmithKline Biologicals SA, and holds equity or stocks in both. Ilse Van Dromme is employed by Janssen Pharmaceutica NV and holds equity or stocks in the company. Tulin Shekar is an employee of Merck & Co., Inc., Rahway, NJ, USA. Honghong Zhou and Qiqi Deng are employees of Moderna Inc., own equity or stocks in the company, and declare financial support provided by Moderna Inc. Daniela Casula is employed by CSL Seqirus. Holly Janes declares financial support provided by the National Institutes of Health. Lawrence Moulton reports board membership and consulting or advisory roles with Merck Research Laboratories, Pfizer Inc., Kentucky BioProcessing, and Medicago Inc., and is employed by Pfizer Canada ULC. Ye Yang, John Scott, Lei Huang, Florian Klinglmüller, and Michael P. Fay declare no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Disclaimer

The views and opinions expressed represent those of the authors and should not be attributed to the respective organizations.

Contributor Information

Francois Beckers, Sanofi, Marcy-l’Étoile, France.

Naveen Karkada, GlaxoSmithKline, Rixensart, Belgium.

Ye Yang, Center for Biologics Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA.

John Scott, Center for Biologics Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA.

Lei Huang, Center for Biologics Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, USA.

Florian Klinglmüller, Austrian Agency for Health and Food Safety, Vienna, Austria.

Michael P. Fay, National Institute of Allergy and Infectious Diseases, NIH, Bethesda, MD, USA

Stefan Englert, Johnson & Johnson, Neuss, Germany.

Bart Spiessens, Johnson & Johnson, Beerse, Belgium.

Ilse Van Dromme, Johnson & Johnson, Beerse, Belgium.

Tulin Shekar, Merck & Co. Inc, Rahway, NJ, USA.

Honghong Zhou, Moderna Inc, Cambridge, MA, USA.

Qiqi Deng, Moderna Inc, Cambridge, MA, USA.

Daniela Casula, CSL Seqirus, Siena, Tuscany, Italy.

Holly Janes, Fred Hutchinson Cancer Center, Seattle, Washington.

Lawrence H. Moulton, Pfizer Canada, Kirkland, Quebec, Canada

References

  • 1.ICH Harmonised Guideline. ICH E9(R1) addendum on estimands and sensitivity analysis in clinical trials to the guideline on statistical principles for clinical trials. : International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH); 2019. [Available from: https://database.ich.org/sites/default/files/E9-R1_Step4_Guideline_2019_1203.pdf. [Google Scholar]
  • 2.Fletcher C, Hefting N, Wright M, Bell J, Anzures-Cabrera J, Wright D, et al. Marking 2-Years of New Thinking in Clinical Trials: The Estimand Journey. Ther Innov Regul Sci. 2022;56(4):637–50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Rufibach K. Treatment effect quantification for time-to-event endpoints–Estimands, analysis strategies, and beyond. Pharm Stat. 2019;18(2):145–65. [DOI] [PubMed] [Google Scholar]
  • 4.Fiero MH, Pe M, Weinstock C, King-Kallimanis BL, Komo S, Klepin HD, et al. Demystifying the estimand framework: a case study using patient-reported outcomes in oncology. The Lancet Oncology. 2020;21(10):e488–e94. [DOI] [PubMed] [Google Scholar]
  • 5.Manitz J, Kan-Dobrosky N, Buchner H, Casadebaig M-L, Degtyarev E, Dey J, et al. Estimands for overall survival in clinical trials with treatment switching in oncology. Pharm Stat. 2022;21(1):150–62. [DOI] [PubMed] [Google Scholar]
  • 6.Englert S, Mercier F, Pilling EA, Homer V, Habermehl C, Zimmermann S, Kan-Dobrosky N. Defining estimands for efficacy assessment in single arm phase 1b or phase 2 clinical trials in oncology early development. Pharm Stat. 2023;22(5):921–37. [DOI] [PubMed] [Google Scholar]
  • 7.Min T, Bain SC. Estimands in diabetes clinical trials. The Lancet Diabetes & Endocrinology. 2020;8(3):181–3. [DOI] [PubMed] [Google Scholar]
  • 8.Little RJ, Lewis RJ. Estimands, Estimators, and Estimates. JAMA. 2021;326(10):967–8. [DOI] [PubMed] [Google Scholar]
  • 9.Cai X, Gewandter JS, He H, Turk DC, Dworkin RH, McDermott MP. Estimands and missing data in clinical trials of chronic pain treatments: advances in design and analysis. Pain. 2020;161(10):2308–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Fu R, Li H, Wang X, Shou Q, Wang WWB. Application of estimand framework in ICH E9 (R1) to vaccine trials. J Biopharm Stat. 2023;33(4):502–13. [DOI] [PubMed] [Google Scholar]
  • 11.Nauta J Estimand for non-inferiority influenza vaccine immunogenicity trials. Vaccine. 2024;42(4):840–3. [DOI] [PubMed] [Google Scholar]
  • 12.Hofner B, Asikanius E, Jacquet W, Framke T, Oude Rengerink K, Aguirre Dávila L, et al. Vaccine Development during a Pandemic: General Lessons for Clinical Trial Design. Stat Biopharm Res. 2024;16(2):158–70. [Google Scholar]
  • 13.Horne AD, Lachenbruch PA, Goldenthal KL. Intent-to-treat analysis and preventive vaccine efficacy. Vaccine. 2000;19(2):319–26. [DOI] [PubMed] [Google Scholar]
  • 14.Dean NE, Gsell P-S, Brookmeyer R, De Gruttola V, Donnelly CA, Halloran ME, et al. Design of vaccine efficacy trials during public health emergencies. Sci Transl Med. 2019;11(499):eaat0360. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Patterson S, Fu B, Meng Y, Bailleux F, Chen J. Statistical Observations on Vaccine Clinical Development for Pandemic Diseases. Stat Biopharm Res. 2022;14(1):28–32. [Google Scholar]
  • 16.Hudgens MG, Gilbert PB, Self SG. Endpoints in vaccine trials. Stat Methods Med Res. 2004;13(2):89–114. [DOI] [PubMed] [Google Scholar]
  • 17.Halloran ME, Struchiner CJ, Longini IM Jr. Study Designs for Evaluating Different Efficacy and Effectiveness Aspects of Vaccines. Am J Epidemiol. 1997;146(10):789–803. [DOI] [PubMed] [Google Scholar]
  • 18.Michiels H, Vandebosch A, Vansteelandt S. Estimation and interpretation of vaccine efficacy in COVID-19 randomized clinical trials. Stat Commun Infect Dis. 2022;14(1). [Google Scholar]
  • 19.Ratitch B, Bell J, Mallinckrodt C, Bartlett JW, Goel N, Molenberghs G, et al. Choosing Estimands in Clinical Trials: Putting the ICH E9(R1) Into Practice. Ther Innov Regul Sci. 2020;54(2):324–41. [DOI] [PubMed] [Google Scholar]
  • 20.Kahan BC, White IR, Edwards M, Harhay MO. Using modified intention-to-treat as a principal stratum estimator for failure to initiate treatment. Clinical Trials. 2023;20(3):269–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Bornkamp B, Rufibach K, Lin J, Liu Y, Mehrotra DV, Roychoudhury S, et al. Principal stratum strategy: Potential role in drug development. Pharm Stat. 2021;20(4):737–51. [DOI] [PubMed] [Google Scholar]
  • 22.Keene ON. Strategies for composite estimands in confirmatory clinical trials: Examples from trials in nasal polyps and steroid reduction. Pharm Stat. 2019;18(1):78–84. [DOI] [PubMed] [Google Scholar]
  • 23.Follmann D, Fay M, Magaret C. Estimation of vaccine efficacy for variants that emerge after the placebo group is vaccinated. Stat Med. 2022;41(16):3076–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Follmann D, Fintzi J, Fay MP, Janes HE, Baden LR, El Sahly HM, et al. A Deferred-Vaccination Design to Assess Durability of COVID-19 Vaccine Effect After the Placebo Group Is Vaccinated. Ann Intern Med. 2021;174(8):1118–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Dragalin V, Fedorov V, Cheuvart B. Statistical approaches to establishing vaccine safety. Stat Med. 2002;21(6):877–93. [DOI] [PubMed] [Google Scholar]
  • 26.Nauta J Statistics in clinical vaccine trials: Springer Science & Business Media; 2010. [Google Scholar]
  • 27.Fintzi J, Follmann D. Assessing vaccine durability in randomized trials following placebo crossover. Stat Med. 2021;40(27):5983–6007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Tsiatis AA, Davidian M. Estimating Vaccine Efficacy over Time After a Randomized Study is Unblinded. Biometrics. 2021;78(3):825–38. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Lin D-Y, Zeng D, Gilbert PB. Evaluating the Long-term Efficacy of Coronavirus Disease 2019 (COVID-19) Vaccines. Clin Infect Dis. 2021;73(10):1927–39. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Ewell M Comparing methods for calculating confidence intervals for vaccine efficacy. Stat Med. 1996;15(21–22):2379–92. [DOI] [PubMed] [Google Scholar]
  • 31.Robins JM. Correcting for non-compliance in randomized trials using structural nested mean models. Communications in Statistics - Theory and Methods. 1994;23(8):2379–412. [Google Scholar]
  • 32.Vansteelandt S, Joffe M. Structural Nested Models and G-estimation: The Partially Realized Promise. Statist Sci. 2014;29(4):707–31, 25. [Google Scholar]
  • 33.ICH Harmonised Guideline. ICH E9: Statistical Principles for Clinical Trials: International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH); 1998. [Available from: https://database.ich.org/sites/default/files/E9_Guideline.pdf. [Google Scholar]
  • 34.Committee for Medicinal Products for Human Use (CHMP). Guideline on Missing Data in Confirmatory Clinical Trials.: European Medicines Agency; 2011. [Available from: https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-missing-data-confirmatory-clinical-trials_en.pdf. [Google Scholar]
  • 35.Little RJA, Rubin DB. Statistical Analysis with Missing Data. 1st ed: John Wiley & Sons, Inc.; 2002. Available from: https://onlinelibrary.wiley.com/doi/book/10.1002/9781119013563. [Google Scholar]
  • 36.Molenberghs G, Kenward MG. Missing Data in Clinical Studies: John Wiley & Sons, Ltd; 2007. Available from: https://onlinelibrary.wiley.com/doi/book/10.1002/9780470510445. [Google Scholar]
  • 37.Molenberghs G, Fitzmaurice G, Kenward MG, Tsiatis A, Verbeke G. Handbook of missing data methodology: CRC Press; 2014. Available from: https://www.taylorfrancis.com/books/9781439854624. [Google Scholar]
  • 38.Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The Prevention and Treatment of Missing Data in Clinical Trials. N Engl J Med. 2012;367(14):1355–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Rehal S, Cro S, Phillips PP, Fielding K, Carpenter JR. Handling intercurrent events and missing data in non-inferiority trials using the estimand framework: A tuberculosis case study. Clinical Trials. 2023;20(5):497–506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Carpenter JR, Roger JH, Kenward MG. Analysis of Longitudinal Trials with Protocol Deviation: A Framework for Relevant, Accessible Assumptions, and Inference via Multiple Imputation. J Biopharm Stat. 2013;23(6):1352–71. [DOI] [PubMed] [Google Scholar]
  • 41.Kang M, Kendall MA, Ribaudo H, Tierney C, Zheng L, Smeaton L, Lindsey JC. Incorporating estimands into clinical trial statistical analysis plans. Clinical Trials. 2022;19(3):285–91. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1

RESOURCES