Skip to main content
PMC home page
American College of Physicians - PMC COVID-19 Collection logoLink to American College of Physicians - PMC COVID-19 Collection
. 2020 Oct 22:M20-6169. doi: 10.7326/M20-6169

Clinical Endpoints for Evaluating Efficacy in COVID-19 Vaccine Trials

Devan V Mehrotra 1,2, Holly E Janes 2,2, Thomas R Fleming 3,2, Paula W Annunziato 4,2, Kathleen M Neuzil 5,2, Lindsay N Carpp 2,2, David Benkeser 6,2, Elizabeth R Brown 3,2, Marco Carone 3,2, Iksung Cho 7,2, Deborah Donnell 2,2, Michael P Fay 8,2, Youyi Fong 2,2, Shu Han 9,2, Ian Hirsch 10,2, Ying Huang 2,2, Yunda Huang 2,2, Ollivier Hyrien 2,2, Michal Juraska 2,2, Alex Luedtke 11,2, Martha Nason 8,2, An Vandebosch 12,2, Honghong Zhou 9,2, Myron S Cohen 13,2, Lawrence Corey 3,2, Jonathan Hartzel 1,2, Dean Follmann 8,2, Peter B Gilbert 3,2
PMCID: PMC7596738  PMID: 33090877

Safe and effective vaccines are a critical component in the control of COVID-19. A group of industry, government, and academic researchers discuss pragmatic issues in the choice and interpretation of clinical endpoints for evaluating efficacy in COVID-19 vaccine trials.

Abstract

Several vaccine candidates to protect against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection or coronavirus disease 2019 (COVID-19) have entered or will soon enter large-scale, phase 3, placebo-controlled randomized clinical trials. To facilitate harmonized evaluation and comparison of the efficacy of these vaccines, a general set of clinical endpoints is proposed, along with considerations to guide the selection of the primary endpoints on the basis of clinical and statistical reasoning. The plausibility that vaccine protection against symptomatic COVID-19 could be accompanied by a shift toward more SARS-CoV-2 infections that are asymptomatic is highlighted, as well as the potential implications of such a shift.

Widespread use of safe and durably effective vaccines for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), especially in combination with multiple concomitant prevention strategies (1), would curtail the coronavirus disease 2019 (COVID-19) pandemic. Several vaccine candidates have entered or will soon enter phase 3 clinical trial testing (2–4). Regulatory approval of a SARS-CoV-2 vaccine will require demonstration of safety and clinical benefit in a placebo-controlled efficacy trial. Guidance from the U.S. Food and Drug Administration (FDA) recommends minimal phase 3 success criteria for approval of a vaccine: an estimated reduction in the primary endpoint of at least 50% in the vaccine group versus the placebo group, with the 95% CI providing assurance of at least a 30% reduction (5)—a benchmark consistent with the World Health Organization's Solidarity Vaccines Trial design (6). The FDA guidance also indicates that acceptable primary endpoints for approval could include SARS-CoV-2 infection, symptomatic infection (COVID-19), severe COVID-19 (5), or some combination of these.

We address 4 salient issues on study endpoints in COVID-19 vaccine efficacy trials. First, we propose a general set of clinical endpoints to facilitate a harmonized evaluation and comparison of the efficacy of vaccine candidates, overall and across relevant subgroups. Second, we consider the pros and cons of various endpoints for use as primary endpoints. Third, we recommend adequate follow-up of all participants to enable enhanced sensitivity regarding effects on severe COVID-19 as well as assessment of the longer-term vaccine effect on the set of endpoints, including an assessment of durability of protection. Fourth, we recommend including asymptomatic infection as a study endpoint, given that vaccine protection against COVID-19 could be accompanied by a shift toward more asymptomatic SARS-CoV-2 infections, a plausible outcome if the vaccine does not confer sterilizing immunity.

Set of Clinical Endpoints to Facilitate Harmonized Vaccine Efficacy Evaluation and Comparison

The interconnections and definitions of our proposed set of study endpoints in COVID-19 vaccine efficacy trials are described in Figure 1 (top and middle, respectively). Two of the endpoints—virologically confirmed symptomatic SARS-CoV-2 infection regardless of the severity of symptoms (COVID-19) and virologically confirmed SARS-CoV-2 infection with symptoms classified as severe (severe COVID-19)—will likely be universally used because they fit standard endpoints used in virtually all vaccine efficacy trials (7). Coronavirus disease 2019 is ascertained at presentation, whereas severe COVID-19 is ascertained at presentation and through a fixed schedule of postdiagnosis follow-up (Figure 1, bottom), which continues through the resolution of all symptoms, ensuring that all severe COVID-19 endpoints are distinguished from nonsevere COVID-19 endpoints. For the COVID-19 endpoint, per FDA guidance, outcomes can include fever or chills, cough, shortness of breath or difficulty breathing, fatigue, muscle or body aches, headache, new loss of taste or smell, sore throat, congestion or runny nose, nausea or vomiting, and diarrhea. Corresponding outcomes for severe COVID-19 include clinical signs at rest indicative of severe illness; respiratory failure; evidence of shock (on the basis of specific blood pressure thresholds); clinically significant acute renal, hepatic, or neurologic dysfunction; admission to an intensive care unit; and death. The FDA guidance acknowledges the need for adaptation of symptom-based case definitions for pediatric patients and for persons with respiratory comorbid conditions.

Figure 1. Clinical endpoint relationships, definitions, and example sampling scheme for diagnosed COVID-19 cases.

Figure 1. Clinical endpoint relationships, definitions, and example sampling scheme for diagnosed COVID-19 cases. BOD = burden of disease; COVID-19 = coronavirus disease 2019; NAAT = nucleic acid amplification test; PCR = polymerase chain reaction; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2. Top. Structural relationships among study endpoints in a COVID-19 vaccine efficacy trial. Middle. Study endpoint definitions. Bottom. Example schedule of disease severity monitoring and virologic sampling for COVID-19 cases, with data or sample collection beginning at COVID-19 diagnosis and extending past COVID-19 diagnosis, in a setting where frequent follow-up of confirmed cases can be assured. Participants diagnosed with virologically confirmed, symptomatic SARS-CoV-2 infection (COVID-19) enter a postdiagnosis sampling schedule to monitor viral load and COVID-19–related symptoms (types, severity levels, and durations). This follow-up continues through resolution of all symptoms, enabling distinction of the nonsevere and severe COVID-19 endpoints. * Seroconversion is assessed via a validated assay that distinguishes natural vs. vaccine-induced SARS-CoV-2 antibodies. † Alternatively, the asymptomatic infection endpoint can also include an RNA PCR positive test result obtained through testing regardless of symptoms (e.g., as a requirement for travel, return to school or work, or elective medical procedures) and follow-up to confirm that the participant remains asymptomatic. ‡ Timed to be as close to day 28 after symptom onset as possible.

Open in a new tab

BOD = burden of disease; COVID-19 = coronavirus disease 2019; NAAT = nucleic acid amplification test; PCR = polymerase chain reaction; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2. Top. Structural relationships among study endpoints in a COVID-19 vaccine efficacy trial. Middle. Study endpoint definitions. Bottom. Example schedule of disease severity monitoring and virologic sampling for COVID-19 cases, with data or sample collection beginning at COVID-19 diagnosis and extending past COVID-19 diagnosis, in a setting where frequent follow-up of confirmed cases can be assured. Participants diagnosed with virologically confirmed, symptomatic SARS-CoV-2 infection (COVID-19) enter a postdiagnosis sampling schedule to monitor viral load and COVID-19–related symptoms (types, severity levels, and durations). This follow-up continues through resolution of all symptoms, enabling distinction of the nonsevere and severe COVID-19 endpoints.

* Seroconversion is assessed via a validated assay that distinguishes natural vs. vaccine-induced SARS-CoV-2 antibodies.

† Alternatively, the asymptomatic infection endpoint can also include an RNA PCR positive test result obtained through testing regardless of symptoms (e.g., as a requirement for travel, return to school or work, or elective medical procedures) and follow-up to confirm that the participant remains asymptomatic.

‡ Timed to be as close to day 28 after symptom onset as possible.

Serologic assays to accurately detect anti–SARS-CoV-2 nucleocapsid antibodies, which would be elicited by naturally acquired infection but not by SARS-CoV-2 spike protein–based vaccination, have been developed and validated (8), providing the technology required to enable evaluation of efficacy against the infection endpoint. The nesting of endpoints and their partitioning into mutually exclusive and exhaustive categories aid in the interpretation of results (Figure 1 , top). Every infection endpoint is either a COVID-19 endpoint or an asymptomatic infection endpoint, and a harmonized analysis of these 3 endpoints can assess the overall vaccine effect on infection and the proportion of this effect on each component endpoint. Similarly, every COVID-19 endpoint is either a nonsevere or severe COVID-19 endpoint, and a harmonized analysis of these 3 endpoints can elucidate the proportion of the vaccine effect on each component endpoint.

An additional endpoint includes all COVID-19 cases and quantitatively differentiates severe from nonsevere COVID-19, which we call the burden of disease (BOD) endpoint (9). To understand the difference between the BOD and COVID-19 endpoints, consider that vaccine efficacy is commonly expressed as a relative reduction (vs. placebo) in the risk for becoming an endpoint “case.” This is mathematically equivalent to assigning a binary score to each trial participant based on their case status and expressing vaccine efficacy as the relative reduction in the mean endpoint score. The COVID-19 endpoint is scored as 0 for no disease and 1 for disease. The BOD endpoint score extends this by using 0 for no COVID-19, 1 for nonsevere COVID-19, and 2 for severe COVID-19.

Given the anticipated heterogeneity among phase 3 trial participants, consistent with established practices in clinical trials (10), it would be important to explore whether vaccine efficacy is generally consistent across subgroups with differing levels of pretrial risk for SARS-CoV-2 infection or COVID-19. Accordingly, for each of the core endpoints described in Figure 1, we advocate reporting point estimates and 95% CIs for vaccine efficacy for prespecified subgroups defined by factors that include sex assigned at birth; age; geographic location; race/ethnicity; and presence or absence of preexisting health problems, such as heart or lung conditions, severe obesity, or diabetes. Because the phase 3 trial designs are generally powered for assessing overall but not subgroup-specific vaccine efficacy, such summaries should be interpreted as hypothesis-generating rather than hypothesis-confirming. Moreover, for specific population subgroups of interest, such as women who become pregnant while participating in a phase 3 trial, the number of endpoint cases will likely be too small to support reliable assessments of vaccine efficacy from a single trial; meta-analyses of relevant phase 3 trials and postapproval studies may be warranted.

Pros and Cons of Different Endpoints for Use as the Primary Endpoints

From both a public health perspective and an individual perspective, prevention of severe COVID-19 is perhaps the most important clinical benefit expected from an effective vaccine. There is precedent (for example, dengue [11], influenza [12–14], pertussis [15], pneumococcal bacteremia [16], rotavirus [17], and varicella [18]) that many vaccines confer greater efficacy against severe disease than milder disease. However, severe COVID-19 constitutes a relatively small portion of COVID-19 cases, and incidence varies widely by age, underlying risk, and ethnicity (19–21), implying that statistical power to demonstrate adequate vaccine efficacy against the severe COVID-19 endpoint may be lower than that for an endpoint that includes reduction in nonsevere COVID-19. For that reason, the broader-encompassing endpoint of COVID-19 symptomatic disease is deemed an appropriate primary endpoint and has been selected as such for all 6 ongoing phase 3 trials (4) and for the Solidarity Vaccines Trial. Moreover, there is consensus to assess severe COVID-19 as a key secondary endpoint.

Given that detection of safety problems with vaccines is critically important, the statistical analysis plans of the trials use 2-sided 95% CIs for vaccine efficacy for each study endpoint, so the data analyses can detect evidence for a higher rate of any endpoint in the vaccine versus the placebo group. Vaccine efficacy is 1 minus the relative risk (vaccine to placebo) of an endpoint, such that data analysis results (for example, Figure 2) can be equivalently reported as point estimates and 95% CIs for the relative risk for each endpoint, with evidence of vaccine harm deriving from an inference of a relative risk greater than 1. Given the precedent that vaccines can increase the risk for severe disease in certain populations (22–25), analysis of the severe COVID-19 endpoint is of particular interest. In addition to the standard analysis that includes all randomly assigned participants and thus provides a valid answer with high confidence, supportive analyses that compare rates of the severe COVID-19 endpoint between vaccine and placebo COVID-19 endpoint cases are also recommended. These latter analyses compare groups that were not randomly assigned and thus are susceptible to potential postrandomization selection bias (26); hence, the data analysis should adjust for baseline prognostic factors of severe COVID-19 and include sensitivity analyses to assess the robustness of results to potential postrandomization selection bias.

Figure 2. Hypothetical example of results of a COVID-19 vaccine efficacy trial with 2:1 (vaccine–placebo ratio) randomization, with the analysis done for 147 total COVID-19 cases.

Figure 2. Hypothetical example of results of a COVID-19 vaccine efficacy trial with 2:1 (vaccine–placebo ratio) randomization, with the analysis done for 147 total COVID-19 cases.BOD = burden of disease; COVID-19 = coronavirus disease 2019; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2. Top. Number of uninfected and infected participants in each group, along with breakdown by endpoint for infected trial participants. Bottom. Vaccine efficacy point estimates and 95% CIs against 6 clinical endpoints. The black, dashed vertical line in the forest plot marks the lower 95% confidence bound of 30% given in guidance from the U.S. Food and Drug Administration.

Open in a new tab

BOD = burden of disease; COVID-19 = coronavirus disease 2019; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2. Top. Number of uninfected and infected participants in each group, along with breakdown by endpoint for infected trial participants. Bottom. Vaccine efficacy point estimates and 95% CIs against 6 clinical endpoints. The black, dashed vertical line in the forest plot marks the lower 95% confidence bound of 30% given in guidance from the U.S. Food and Drug Administration.

The BOD endpoint, which encodes severe disease as being worse than nonsevere disease, may be viewed by some as more informative than the COVID-19 endpoint. Moreover, it can provide increased statistical power if the vaccine provides greater protection against severe than nonsevere COVID-19. Table 1 shows this for a 2:1 (vaccine to placebo ratio) randomization trial design, with 147 cases required to ensure 90% power for rejecting a null hypothesis of vaccine efficacy against COVID-19 being at most 30% assuming the true vaccine efficacy is at least 60% and the simple conditional binomial method described in the Supplement (available at Annals.org) is used for analysis. (A corresponding 1:1 randomization design could also be considered, with 160 COVID-19 cases required to ensure 90% power.) The results show that if vaccine efficacy against COVID-19 and severe COVID-19 is at least 60%, then both the COVID-19 and BOD endpoints provide about 90% or better power to meet FDA success criteria. However, if vaccine efficacy against COVID-19 is slightly lower at 55%, power decreases to 74% for the COVID-19 endpoint, whereas power for the BOD endpoint is between 75% and 92% if the vaccine confers a 60% to 90% relative reduction in severe COVID-19. Of note, if vaccine efficacy against severe COVID-19 is at or below 30%, BOD will be notably less powerful than COVID-19. A potential option is to use BOD and COVID-19 as dual primary endpoints with a statistical adjustment for associated multiple testing. Although the power of this strategy can be about as good as or notably better than that achieved with COVID-19 as the sole primary endpoint across all of the scenarios in Table 1, the clinical relevance of statistical success for a vaccine associated with moderate protection against nonsevere COVID-19 but poor protection against severe COVID-19 is unclear. Statistical details on methods used for the illustrative power calculations and for the multiplicity adjustment that leverages the correlation between the COVID-19 and BOD endpoints are provided in the Supplement.

Table 1. Comparison of Statistical Power for Different Efficacy Endpoints*.

Table 1. Comparison of Statistical Power for Different Efficacy Endpoints*

Open in a new tab

We believe that the BOD endpoint adheres to regulatory guidelines for suitability as a primary or key secondary endpoint for phase 3 trials, including clinical relevance in upweighting severe disease endpoints and sensitivity to detect a meaningful intervention effect validly assessed on the basis of the randomization principle (27). However, the BOD endpoint has 3 potential limitations. First, unlike for the COVID-19 endpoint, vaccine efficacy against BOD cannot be expressed as a percentage reduction in the risk for becoming an endpoint “case,” hampering interpretability for some. Second, consensus is lacking about the best way to score the degree of severity of COVID-19 events; specifically, there is a question about how each unit increment in the score relates to the corresponding increment in clinical significance. A third potential limitation, shared with the severe COVID-19 endpoint, is that follow-up after COVID-19 diagnosis is needed to ascertain some severe COVID-19 endpoints, whereas COVID-19 can be more rapidly and easily assessed, as defined at presentation. Regardless of whether BOD is used as a primary or key secondary endpoint, a separate efficacy evaluation for each of the COVID-19 incidence and severity components embedded in the composite BOD endpoint is necessary for clinical interpretability.

Given that asymptomatic infection is relatively common (28), infection cases would be expected to occur at a notably higher frequency than symptomatic disease. However, although prevention from acquiring SARS-CoV-2 infection (whether that means true sterilizing immunity or absence of detectable antibodies to the invading virus) has its advocates, this is considered a high bar to achieve (29, 30). A vaccine that protects against disease and confers limited protection against infection would still provide great clinical benefit and overall utility.

Importance of Longer-Term Follow-up of All Participants

We recommend longer-term follow-up of all participants (that is, after occurrence of 147 COVID-19 endpoints for a 2:1 randomization trial design) in a double-blind manner, if feasible, for at least a year after randomization or until deployment of a vaccine that has been proven effective in each participant's geographic region (6) for the following reasons. First, as described earlier, severe COVID-19 constitutes a relatively small portion of COVID-19 cases (19–21), suggesting that an efficacy evaluation for severe COVID-19 will likely be underpowered in an analysis based on 147 COVID-19 events. The data shown in Table 1 confirm this expectation. Thus, longer-term follow-up would enhance the sensitivity of detection of vaccine effects on severe COVID-19. Second, if a candidate vaccine is proven to be safe and found to have efficacy against 1 or more clinical endpoints, effective planning of mass immunization campaigns and strategies will require knowledge of the duration of such protection. Waning vaccine efficacy against a clinical endpoint has been documented in randomized controlled trials of several vaccines, including the RTS,S/AS01 malaria vaccine (31) and 2 similar killed whole-cell oral cholera vaccines (32). Waning vaccine efficacy against a clinical endpoint has also been suggested by observational case–control studies of several vaccines, including the influenza vaccine (33). In addition to improving insights about durability of effects, extended follow-up provides needed insights about whether a vaccine could make COVID-19 more hazardous, referred to as disease enhancement. Thus, longer-term follow-up as defined earlier would also provide important information on duration of the vaccine effect on each of the endpoints defined in Figure 1.

Inclusion of Asymptomatic Infections as a Study Endpoint

Because each infection endpoint is also either an asymptomatic infection endpoint or a COVID-19 endpoint (Figure 1), vaccine efficacy levels for infection and for COVID-19, together with an assumption of the expected percentage of infections in the placebo group that will be asymptomatic, determine the vaccine efficacy for asymptomatic infection. Table 2 shows this mathematical calculation if 40% of the infections in placebo recipients are expected to be asymptomatic (28). Under this expectation, if a vaccine reduces SARS-CoV-2 infections by a modest amount, say 20%, but reduces symptomatic infections by an impressive 70%, the net result will be a 55% increase in asymptomatic infections. In such a scenario, although the vaccine would likely meet success criteria if COVID-19 (or BOD) were the sole primary endpoint and would provide direct clinical benefit to individual vaccine recipients, it may have different population-level effects on the number of secondary transmissions (34). If vaccine recipients acquiring asymptomatic infection have low viral infectivity and are weak transmitters, then the vaccine in the aforementioned scenario would likely confer public health benefit on transmission. However, viral shedding data (35, 36) and epidemiologic and modeling studies (37) suggest that unvaccinated asymptomatic persons can transmit the virus (38, 39), raising questions about the transmission potential of vaccine recipients who acquire asymptomatic infection. A vaccine that essentially converts symptomatic to asymptomatic infections, without also decreasing viral shedding and transmission potential, may paradoxically increase transmissions given that viral testing and isolation are presumably less likely in asymptomatic persons. Conversely, if the vaccine decreases both symptomatic infections and transmission potential, then the population benefits may be magnified. Acknowledging the tenuous link in extrapolating results from nonhuman primate studies to humans, we note that challenge studies of rhesus macaques vaccinated against SARS-CoV-2 (vs. unvaccinated controls) offer evidence that either possibility may occur ([40] and [41], respectively). Designing phase 3 trials to evaluate a vaccine effect on asymptomatic infections, which requires collecting blood from all participants periodically for serology testing or swabs for virology testing (along with incorporating symptom-triggered testing for virologically confirmed SARS-CoV-2 infection to ensure that any presymptomatic persons who have no symptoms at the time of a positive virologic test result, but eventually do develop symptoms, will be correctly classified as having a COVID-19 endpoint rather than an asymptomatic infection endpoint) will be an important way to carefully explore this issue.

Table 2. Interconnection Among VE Against SARS-CoV-2 Infection, Symptomatic SARS-CoV-2 Infection (COVID-19), and Asymptomatic SARS-CoV-2 Infection.

Table 2. Interconnection Among VE Against SARS-CoV-2 Infection, Symptomatic SARS-CoV-2 Infection (COVID-19), and Asymptomatic SARS-CoV-2 Infection

Open in a new tab

Figure 2 shows recommended forest plot reporting of vaccine efficacy results for the proposed core study endpoints for a hypothetical phase 3 clinical trial of a COVID-19 vaccine (see the Supplement for calculation details). Here, success criteria would be met if COVID-19, severe COVID-19, or BOD were the sole primary endpoint. Vaccine efficacy is low for preventing infection. The negative vaccine efficacy against asymptomatic infection can be reasonably interpreted as the vaccine having shifted infections that would have been symptomatic without vaccination to asymptomatic infections. Although an alternative interpretation that the vaccine at least partially caused more asymptomatic infections that would not have occurred without vaccination cannot be ruled out, it would be deemed less biologically plausible given the promising vaccine efficacy results for the COVID-19, severe COVID-19, and BOD endpoints.

The fact that a vaccine with low efficacy against acquisition of SARS-CoV-2 infection and moderate to high efficacy against symptomatic infection could result in a net increase in asymptomatic infections highlights the need to understand the effect of asymptomatic infections on transmission of COVID-19 (42). The contribution of asymptomatic carriers to transmission dynamics has been modeled for many infectious diseases, including influenza (43) and Bordetella pertussis (causative agent of whooping cough). For the latter, modeling analyses have considered the plausible situation where vaccination with the acellular B pertussis vaccine prevents symptomatic disease but does not prevent asymptomatic transmission, and concluded that such a scenario could explain the observed increase in B pertussis incidence during the past few decades (44). For SARS-CoV-2, modeling of the effect of a vaccine that reduces or prevents COVID-19 but increases asymptomatic infections entails biological issues (for example, the relative rates of asymptomatic vs. symptomatic infection, the relative transmissibility of asymptomatic vs. symptomatic infection, the relative vaccine effect on secondary transmission for asymptomatic vs. symptomatic infection, how this relative transmissibility differs in unvaccinated vs. previously vaccinated infected persons, and how well this relative transmissibility can be measured by a viral load putative surrogate marker) and behavioral issues (for example, the rapidity and completeness of isolation after COVID-19 diagnosis, the coverage of SARS-CoV-2 testing in communities after vaccine approval, and the coverage of vaccination). Under very high rates of vaccine coverage, increased numbers of asymptomatic infections may be innocuous because any excess transmissions will tend to be toward vaccinated persons who are well protected against severe disease. In contrast, in communities with low vaccination coverage, excess asymptomatic infections with transmission potential could increase the number of severe COVID-19 cases because the excess transmissions will tend to be to unvaccinated persons who are fully vulnerable to severe COVID-19. We anticipate that recommendations about how a vaccine should be used in communities will likely be informed not only by the vaccine efficacy against infection or disease estimated in the clinical trials but also by estimates of community effect predicted by transmission models. Duration of vaccine efficacy against the core set of study endpoints will be a key parameter influencing the community impact.

Conclusion

A primary endpoint should be clinically meaningful (27), sensitive, and specific. A simple designation, such as reducing COVID-19 disease severity, seems uncomplicated, but it masks potential interpretability and misclassification concerns. The COVID-19, severe COVID-19, and BOD endpoints depend on sets of prescribed symptoms, and myriad definitions could be used to specify these sets. Although the FDA recommends defining the COVID-19 endpoint as virologically confirmed SARS-CoV-2 infection accompanied by 1 or more of 11 symptoms (5), trialists have latitude to select particular symptoms and severities needed to trigger virologic testing. It is important to define a common COVID-19 endpoint that can be used consistently across trials, both for interpretation of results and for facilitation of meta-analyses of trials.

Because several candidate COVID-19 vaccines are entering phase 3 testing, it is important to adopt a standard set of clinical endpoints for vaccine efficacy evaluation across all of the trials to provide uniform, comprehensive evaluations of benefit and risk and to support pooling data for analyses of immunologic surrogate endpoints. We have proposed a core set of primary and secondary clinical endpoints. The designation of endpoints as primary versus secondary may differ by vaccine candidate because some may have a greater preevaluation probability of blocking SARS-CoV-2 acquisition, which could justify including infection as a primary endpoint. Regardless, we propose that COVID-19 and severe COVID-19 should be important standalone clinical endpoints to assess in every vaccine efficacy trial, with adequate follow-up of all participants to accumulate meaningful data for assessment of longer-term protection against both endpoints, especially severe COVID-19, which needs greater endpoint counts for a reliable quantification of vaccine efficacy. Moreover, we have noted reasons for considering the BOD endpoint as a (dual) primary or key secondary endpoint if the vaccine candidate is expected to work best against severe COVID-19 and at an intermediate level against nonsevere COVID-19. In such scenarios, the BOD endpoint will provide improved power relative to COVID-19, although this gain may be offset by a potential detriment in clinical interpretability associated with this composite endpoint. Finally, given that a desired, vaccine-induced decrease in the incidence of symptomatic SARS-CoV-2 infections may be accompanied by a shift toward more asymptomatic infections, we recommend including the ability to evaluate vaccine efficacy against the asymptomatic infection endpoint in trial designs.

Supplementary Material

Click here for additional data file. (909.9KB, pdf)

Footnotes

This article was published at Annals.org on 22 October 2020

* Drs. Mehrotra and Gilbert contributed equally to this work.

References

  • 1. Cohen MS, Corey L. Combination prevention for COVID-19 [Editorial]. Science. 2020;368:551. [PMID: 32381692] doi:10.1126/science.abc5798 [DOI] [PubMed]
  • 2. Corey L, Mascola JR, Fauci AS, et al. A strategic approach to COVID-19 vaccine R&D. Science. 2020;368:948-950. [PMID: 32393526] doi:10.1126/science.abc5312 [DOI] [PubMed]
  • 3. World Health Organization. An international randomised trial of candidate vaccines against COVID-19. Accessed at www.who.int/publications-detail/an-international-randomised-trial-of-candidate-vaccines-against-covid-19 on 26 July 2020.
  • 4. World Health Organization. Draft landscape of COVID-19 candidate vaccines. Accessed at www.who.int/publications/m/item/draft-landscape-of-covid-19-candidate-vaccines on 2 August 2020.
  • 5. U.S. Department of Health and Human Services, U.S. Food and Drug Administration, Center for Biologics Evaluation and Research. Development and Licensure of Vaccines to Prevent COVID-19: Guidance for Industry. Accessed at www.fda.gov/media/139638/download on 24 July 2020.
  • 6. Krause P, Fleming TR, Longini I, et al; World Health Organization Solidarity Vaccines Trial Expert Group. COVID-19 vaccine trials should seek worthwhile efficacy. Lancet. 2020;396:741-743. [PMID: 32861315] doi:10.1016/S0140-6736(20)31821-3 [DOI] [PMC free article] [PubMed]
  • 7. Clements-Mann ML. Lessons for AIDS vaccine development from non-AIDS vaccines. AIDS Res Hum Retroviruses. 1998;14 Suppl 3:S197-203. [PMID: 9814944] [PubMed]
  • 8. Bryan A, Pepper G, Wener MH, et al. Performance characteristics of the Abbott Architect SARS-CoV-2 IgG assay and seroprevalence in Boise, Idaho. J Clin Microbiol. 2020;58. [PMID: 32381641] doi:10.1128/JCM.00941-20 [DOI] [PMC free article] [PubMed]
  • 9. Chang MN, Guess HA, Heyse JF. Reduction in burden of illness: a new efficacy measure for prevention trials. Stat Med. 1994;13:1807-14. [PMID: 7997714] [DOI] [PubMed]
  • 10. Wang R, Lagakos SW, Ware JH, et al. Statistics in medicine—reporting of subgroup analyses in clinical trials. N Engl J Med. 2007;357:2189-94. [PMID: 18032770] [DOI] [PubMed]
  • 11. Hadinegoro SR, Arredondo-García JL, Capeding MR, et al; CYD-TDV Dengue Vaccine Working Group. Efficacy and long-term safety of a dengue vaccine in regions of endemic disease. N Engl J Med. 2015;373:1195-206. [PMID: 26214039] doi:10.1056/NEJMoa1506223 [DOI] [PubMed]
  • 12. Thompson MG, Pierse N, Sue Huang Q, et al; SHIVERS Investigation Team. Influenza vaccine effectiveness in preventing influenza-associated intensive care admissions and attenuating severe disease among adults in New Zealand 2012-2015. Vaccine. 2018;36:5916-5925. [PMID: 30077480] doi:10.1016/j.vaccine.2018.07.028 [DOI] [PubMed]
  • 13. Nichols MK, Andrew MK, Hatchette TF, et al; Serious Outcomes Surveillance Network of the Canadian Immunization Research Network (CIRN), the Toronto Invasive Bacterial Diseases Network (TIBDN). Influenza vaccine effectiveness to prevent influenza-related hospitalizations and serious outcomes in Canadian adults over the 2011/12 through 2013/14 influenza seasons: a pooled analysis from the Canadian Immunization Research Network (CIRN) Serious Outcomes Surveillance (SOS Network). Vaccine. 2018;36:2166-2175. [PMID: 29548608] doi:10.1016/j.vaccine.2018.02.093 [DOI] [PubMed]
  • 14. Dbaibo G, Amanullah A, Claeys C, et al; Flu4VEC Study Group. Quadrivalent influenza vaccine prevents illness and reduces healthcare utilization across diverse geographic regions during five influenza seasons: a randomized clinical trial. Pediatr Infect Dis J. 2020;39:e1-e10. [PMID: 31725115] doi:10.1097/INF.0000000000002504 [DOI] [PMC free article] [PubMed]
  • 15. Onorato IM, Wassilak SG, Meade B. Efficacy of whole-cell pertussis vaccine in preschool children in the United States. JAMA. 1992;267:2745-9. [PMID: 1578592] [PubMed]
  • 16. Jackson LA, Neuzil KM, Yu O, et al; Vaccine Safety Datalink. Effectiveness of pneumococcal polysaccharide vaccine in older adults. N Engl J Med. 2003;348:1747-55. [PMID: 12724480] [DOI] [PubMed]
  • 17. Steele AD, Patel M, Parashar UD, et al. Rotavirus vaccines for infants in developing countries in Africa and Asia: considerations from a World Health Organization-sponsored consultation. J Infect Dis. 2009;200 Suppl 1:S63-9. [PMID: 19817616] doi:10.1086/605042 [DOI] [PubMed]
  • 18. Seward JF, Marin M, Vázquez M. Varicella vaccine effectiveness in the US vaccination program: a review. J Infect Dis. 2008;197 Suppl 2:S82-9. [PMID: 18419415] doi:10.1086/522145 [DOI] [PubMed]
  • 19. Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72 314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020;323:1239-1242. [PMID: 32091533] doi:10.1001/jama.2020.2648 [DOI] [PubMed]
  • 20. Stokes EK, Zambrano LD, Anderson KN, et al. Coronavirus disease 2019 case surveillance—United States, January 22-May 30, 2020. MMWR Morb Mortal Wkly Rep. 2020;69:759-765. [PMID: 32555134] doi:10.15585/mmwr.mm6924e2 [DOI] [PMC free article] [PubMed]
  • 21. Clark A, Jit M, Warren-Gash C, et al; Centre for the Mathematical Modelling of Infectious Diseases COVID-19 working group. Global, regional, and national estimates of the population at increased risk of severe COVID-19 due to underlying health conditions in 2020: a modelling study. Lancet Glob Health. 2020;8:e1003-e1017. [PMID: 32553130] doi:10.1016/S2214-109X(20)30264-3 [DOI] [PMC free article] [PubMed]
  • 22. Chin J, Magoffin RL, Shearer LA, et al. Field evaluation of a respiratory syncytial virus vaccine and a trivalent parainfluenza virus vaccine in a pediatric population. Am J Epidemiol. 1969;89:449-63. [PMID: 4305200] [DOI] [PubMed]
  • 23. Fulginiti VA, Eller JJ, Sieber OF, et al. Respiratory virus immunization. I. A field trial of two inactivated respiratory virus vaccines; an aqueous trivalent parainfluenza virus vaccine and an alum-precipitated respiratory syncytial virus vaccine. Am J Epidemiol. 1969;89:435-48. [PMID: 4305199] [DOI] [PubMed]
  • 24. Kim HW, Canchola JG, Brandt CD, et al. Respiratory syncytial virus disease in infants despite prior administration of antigenic inactivated vaccine. Am J Epidemiol. 1969;89:422-34. [PMID: 4305198] [DOI] [PubMed]
  • 25. Sridhar S, Luedtke A, Langevin E, et al. Effect of dengue serostatus on dengue vaccine safety and efficacy. N Engl J Med. 2018;379:327-340. [PMID: 29897841] doi:10.1056/NEJMoa1800820 [DOI] [PubMed]
  • 26. Lu X, Mehrotra DV, Shepherd BE. Rank-based principal stratum sensitivity analyses. Stat Med. 2013;32:4526-39. [PMID: 23686390] doi:10.1002/sim.5849 [DOI] [PMC free article] [PubMed]
  • 27. Fleming TR, Powers JH. Biomarkers and surrogate endpoints in clinical trials. Stat Med. 2012;31:2973-84. [PMID: 22711298] doi:10.1002/sim.5403 [DOI] [PMC free article] [PubMed]
  • 28. Oran DP, Topol EJ. Prevalence of asymptomatic SARS-CoV-2 infection: a narrative review. Ann Intern Med. 2020;173:362-367. [PMID: 32491919] doi:10.7326/M20-3012 [DOI] [PMC free article] [PubMed]
  • 29. Branswell H. The world needs Covid-19 vaccines. It may also be overestimating their power. STAT. 22 May 2020. Accessed at www.statnews.com/2020/05/22/the-world-needs-covid-19-vaccines-it-may-also-be-overestimating-their-power on 26 August 2020.
  • 30. Branswell H. Four scenarios on how we might develop immunity to Covid-19. STAT. 25 August 2020. Accessed at www.statnews.com/2020/08/25/four-scenarios-on-how-we-might-develop-immunity-to-covid-19 on 26 August 2020.
  • 31. RTS,S Clinical Trials Partnership. Efficacy and safety of RTS,S/AS01 malaria vaccine with or without a booster dose in infants and children in Africa: final results of a phase 3, individually randomised, controlled trial. Lancet. 2015;386:31-45. [PMID: 25913272] doi:10.1016/S0140-6736(15)60721-8 [DOI] [PMC free article] [PubMed]
  • 32. Gilbert PB, Wei LJ, Kosorok MR, et al. Simultaneous inferences on the contrast of two hazard functions with censored observations. Biometrics. 2002;58:773-80. [PMID: 12495131] [DOI] [PubMed]
  • 33. Ferdinands JM, Fry AM, Reynolds S, et al. Intraseason waning of influenza vaccine protection: evidence from the US influenza vaccine effectiveness network, 2011-12 through 2014-15. Clin Infect Dis. 2017;64:544-550. [PMID: 28039340] doi:10.1093/cid/ciw816 [DOI] [PubMed]
  • 34. Halloran ME, Struchiner CJ, Longini IM Jr. Study designs for evaluating different efficacy and effectiveness aspects of vaccines. Am J Epidemiol. 1997;146:789-803. [PMID: 9384199] [DOI] [PubMed]
  • 35. Long QX, Tang XJ, Shi QL, et al. Clinical and immunological assessment of asymptomatic SARS-CoV-2 infections. Nat Med. 2020;26:1200-1204. [PMID: 32555424] doi:10.1038/s41591-020-0965-6 [DOI] [PubMed]
  • 36. Lee S, Kim T, Lee E, et al. Clinical course and molecular viral shedding among asymptomatic and symptomatic patients with SARS-CoV-2 infection in a community treatment center in the Republic of Korea. JAMA Intern Med. 2020. [PMID: 32780793] doi:10.1001/jamainternmed.2020.3862 [DOI] [PMC free article] [PubMed]
  • 37. Furukawa NW, Brooks JT, Sobel J. Evidence supporting transmission of severe acute respiratory syndrome coronavirus 2 while presymptomatic or asymptomatic. Emerg Infect Dis. 2020;26. [PMID: 32364890] doi:10.3201/eid2607.201595 [DOI] [PMC free article] [PubMed]
  • 38. Gandhi M, Yokoe DS, Havlir DV. Asymptomatic transmission, the Achilles' heel of current strategies to control Covid-19 [Editorial]. N Engl J Med. 2020;382:2158-2160. [PMID: 32329972] doi:10.1056/NEJMe2009758 [DOI] [PMC free article] [PubMed]
  • 39. World Health Organization. Transmission of SARS-CoV-2: implications for infection prevention precautions. Accessed at www.who.int/news-room/commentaries/detail/transmission-of-sars-cov-2-implications-for-infection-prevention-precautions on 3 August 2020.
  • 40. van Doremalen N, Lambe T, Spencer A, et al. ChAdOx1 nCoV-19 vaccine prevents SARS-CoV-2 pneumonia in rhesus macaques. Nature. 2020. [PMID: 32731258] doi:10.1038/s41586-020-2608-y [DOI] [PMC free article] [PubMed]
  • 41. Corbett KS, Flynn B, Foulds KE, et al. Evaluation of the mRNA-1273 vaccine against SARS-CoV-2 in nonhuman primates. N Engl J Med. 2020;383:1544-1555. [PMID: 32722908] doi:10.1056/NEJMoa2024671 [DOI] [PMC free article] [PubMed]
  • 42. Fox SJ, Pasco R, Tec M, et al. The impact of asymptomatic COVID-19 infections on future pandemic waves. medRxiv. Preprint posted online 23 June 2020. doi:10.1101/2020.06.22.20137489
  • 43. Ip DK, Lau LL, Leung NH, et al. Viral shedding and transmission potential of asymptomatic and paucisymptomatic influenza virus infections in the community. Clin Infect Dis. 2017;64:736-742. [PMID: 28011603] doi:10.1093/cid/ciw841 [DOI] [PMC free article] [PubMed]
  • 44. Althouse BM, Scarpino SV. Asymptomatic transmission and the resurgence of Bordetella pertussis. BMC Med. 2015;13:146. [PMID: 26103968] doi:10.1186/s12916-015-0382-8 [DOI] [PMC free article] [PubMed]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Click here for additional data file. (909.9KB, pdf)

Articles from Annals of Internal Medicine are provided here courtesy of American College of Physicians

RESOURCES