INTRODUCTION
The US Food and Drug Administration (FDA) has thus far granted Emergency Use Authorization (EUA) to three vaccines for the prevention of coronavirus disease 2019 (COVID-19), authorizing use—contingent upon further evaluation—based on less evidence than required for traditional licensure.1,2 While these were FDA’s first uses of EUA for novel vaccines, the “accelerated approval” pathway has for decades permitted FDA to approve vaccines based on limited preapproval evidence, specifically surrogate measures (e.g., antibody levels) reasonably likely to predict clinical benefit, while requiring completion of postapproval trials to verify clinical benefit. To inform future regulatory decisions, including the EUA or accelerated approval of vaccines for COVID-19 and other diseases, we evaluated all novel vaccines granted accelerated approval, characterizing the evidence from preapproval and postapproval trials, including how often postapproval studies confirmed clinical benefit.
METHODS
Using FDA approval letters, we identified all novel vaccines granted accelerated approval from the pathway’s inception in 1992 through 2017, allowing 3 years minimum for completion of postapproval trials. Following previously described approaches,3,4 we identified pivotal efficacy trials (i.e., those serving as the basis of FDA approval) and extracted trial characteristics from FDA’s clinical reviews, as well as the total number of studies supporting approval and the prelicensure safety population. Next, we identified postapproval trials FDA required for accelerated approval, including ClinicalTrials.gov registrations and corresponding publications on PubMed, and extracted trial characteristics and determined study status.
For each preapproval and postapproval trial, we determined use of randomization, blinding, number of treated patients (overall and intervention group), completion rate, trial duration, duration of follow-up for serious adverse events (SAEs), type of comparator (active, placebo, or none), and primary endpoint (clinical outcome or surrogate measure). Lastly, we determined vaccine efficacy and assessed whether clinical benefit was confirmed. Fisher exact and Mann-Whitney U tests were conducted in R, version 3.4.0 (R Foundation for Statistical Computing) (2-sided P<0.05).
RESULTS
Between 1992 and 2017, FDA granted accelerated approval for 8 novel vaccines for seasonal influenza (n=5; 62.5%), meningococcus (n=2; 25.0%), and Haemophilus influenzae (n=1; 12.5%) based on a median of 9 (IQR, 7–19) total studies, including 1.5 (IQR, 1–3) pivotal efficacy trials and a total safety population of 4711 (IQR, 3718–10,968) participants. Overall, FDA required 15 postapproval trials, a median of 2 (IQR, 1–2.3) per vaccine. Within 3 and 6 years after approval, 13 (86.7%) and 14 (93.3%) of the postapproval trials were completed, respectively, while 1 (6.7%) remains delayed.
Most of the completed pivotal efficacy (n=18) and postapproval (n=14) trials were randomized (16/18 [88.9%] vs 14/14 [100%]; P=0.49) (Table 1). Compared with pivotal trials, postapproval trials were larger (median enrollment, 976 [IQR, 162–1689] vs 4586 [IQR, 2207–7406]; P<0.001), but there was no statistically significant difference in follow-up duration for SAEs (median days, 52 [IQR, 30–183] vs 183 [IQR, 180–183]; P=0.08) or in use of double-blinding (11/18 [61.1%] vs 13/14 [92.9%]; P=0.05). Postapproval trials were more likely to use clinical outcomes as primary endpoints (0/18 [0%] vs 7/14 [50.0%]; P=0.001); among these, median vaccine efficacy was 46.3% (IQR, 32.2–63.1%) and 3 (42.9%) confirmed benefit (Table 2).
Table 1.
Preapproval and Postapproval Study Characteristics of Vaccines Receiving Accelerated Approval, 1992–2017
| Study characteristics | Pivotal efficacy trials (n=18) | Postapproval trials (n=14) | P value |
|---|---|---|---|
| Overall enrollment, median (IQR) | 976 (162–1689) | 4586 (2207–7406) | <0.001 |
| Intervention group enrollment, median (IQR) | 741 (162–1219) | 2828 (1310–3756) | 0.002 |
| Overall completion rate, median (IQR) | 97.4 (90.5–99.2) | 95.7* (90.5–97.9) | 0.51 |
| Duration of follow-up for SAEs, median (IQR), days | 52 (30–183) | 183 (180–183) | 0.08 |
| Randomized (%) | 16† (88.9) | 14 (100) | 0.49 |
| Double-blinded (%) | 11 (61.1) | 13 (92.9) | 0.05 |
| Comparator (%) | 0.02 | ||
| Active | 5 (27.8) | 5 (35.7) | |
| Placebo | 6 (33.3) | 9 (64.3) | |
| None | 7 (38.9) | 0 | |
| Primary endpoint (%) | 0.001 | ||
| Clinical outcome | 0 | 7 (50.0) | |
| Surrogate measure | 14 (77.8) | 7 (50.0) | |
| Safety | 2 (11.1) | 0 | |
| Lot consistency | 2 (11.1) | 0 |
IQR, interquartile range; SAE, serious adverse event
*Includes data from only 13 postapproval studies. The number of patients completing one of Flulaval’s postapproval studies could not be identified
†Includes 5 trials in which the vaccine was randomized to different doses, formulations, lots, or schedules
Table 2.
Characteristics and Findings of 14 Completed Postapproval Trials of Vaccines Receiving Accelerated Approval, 1992–2017
| Vaccine* (approval year; indication) | Design | Comparators | Overall enrollment, no. (intervention group) | Primary endpoint | Vaccine efficacy† | Confirmed clinical benefit† (FDA status‡) |
|---|---|---|---|---|---|---|
| Fluarix (2005; seasonal influenza) | Randomized, double-blind active-controlled trial |
Group 1: Fluarix Group 2: Fluzone |
1845 (923) |
Antibody titer | -- |
-- (fulfilled) |
| Randomized, double-blind placebo-controlled trial |
Group 1: Fluarix Group 2: placebo |
6203 (4137) |
Culture-confirmed influenza | 22.3% (95% CI: −49.1 to 58.5%) |
No (fulfilled||) |
|
| Randomized, double-blind placebo-controlled trial |
Group 1: Fluarix Group 2: placebo |
7652 (5103) |
Culture-confirmed influenza | 66.9% (95% CI: 51.9 to 77.4%) |
Yes (fulfilled) |
|
| Flulaval (2006; seasonal influenza) | Randomized, double-blind active-controlled trial |
Group 1: Flulaval Group 2: Fluzone |
1225 (610) |
Antibody titer | -- |
-- (fulfilled) |
| Randomized, double-blind placebo-controlled trial |
Group 1: Flulaval Group 2: placebo |
7611 (3783) |
Culture-confirmed influenza | 46.3% (97.5% CI: lower limit, 9.8%) |
No (fulfilled||) |
|
| Randomized, double-blind placebo-controlled trial |
Group 1: Flulaval quadrivalent Group 2: Havrix§ |
5168 (2584) |
Real-time-PCR confirmed influenza | 59.3% (95% CI: 45.2 to 69.7%) |
Yes (fulfilled) |
|
| Afluria (2007; seasonal influenza) | Randomized, double-blind placebo-controlled trial |
Group 1: Afluria Group 2: placebo |
15,044 (10033) |
Laboratory-confirmed influenza | 42.0% (95% CI: 22.9 to 52.0%) |
No (fulfilled) |
| Randomized, single-blind active-controlled trial |
Group 1: Afluria Group 2: Fluzone |
1268 (631) |
Antibody titer | -- |
-- (fulfilled) |
|
| Agriflu (2009; seasonal influenza) | Randomized, double-blind placebo-controlled trial¶ |
Group 1: Agriflu Group 2: Flucelvax Group 3: placebo |
11,404 (3676) |
Culture-confirmed influenza | 78.4% (97.5% CI: lower limit, 52.1%) |
Yes (fulfilled) |
| Hiberix (2009; Haemophilus influenzae) | Randomized, double-blind active-controlled trial |
Group 1: Hiberix# Group 2: ActHIB Group 3: Pentacel |
4003 (2963) |
Antibody titer | -- |
-- (fulfilled) |
| Trumenba (2014; meningococcus) | Randomized, double-blind placebo-controlled trial |
Group 1: Trumenba# Group 2: Havrix§ + placebo |
3590 (2693) |
Complement mediated bactericidal activity | -- |
-- (fulfilled) |
| Randomized, double-blind placebo-controlled trial |
Group 1: Trumenba Group 2: placebo |
3293 (2471) |
Complement mediated bactericidal activity | -- |
-- (fulfilled) |
|
| Bexsero (2015; meningococcus) | Randomized, double-blind active-controlled trial |
Group 1: Bexsero Group 2: Menveo + placebo |
305 (154) |
Complement mediated bactericidal activity | -- |
-- (submitted) |
| Fluad (2015; seasonal influenza) | Randomized, double-blind placebo-controlled trial |
Group 1: Fluad quadrivalent Group 2: Boostrix§ |
6790 (3394) |
RT-PCR-confirmed influenza | 19.8% (97.5% CI: −5.3 to 38.9%) |
No (submitted) |
CI, confidence interval; FDA, US Food and Drug Administration; RT-PCR, reverse transcription-polymerase chain reaction
*Fluarix, Flulaval, Afluria, Agriflu, and Fluad are inactivated vaccines; Hiberix is a conjugate vaccine; Trumenba and Bexsero are recombinant protein vaccines
†Vaccine efficacy and assessment of clinical benefit were determined only for postapproval trials with clinical outcomes as primary endpoints. Clinical benefit was assessed in relation to each trial’s predefined criteria for establishing vaccine efficacy
‡FDA status as of May 4, 2021. FDA status categories for postapproval studies include pending, ongoing, delayed, terminated, submitted, fulfilled, and released
§Active placebo control group
||While the FDA confirmed fulfillment of this study, the accelerated approval commitment to verify clinical benefit was not considered fulfilled and the FDA required an additional clinical endpoint study
¶Vaccine efficacy was assessed for significance versus placebo
#To evaluate lot consistency, participants were randomized into 3 groups each receiving a different manufacturing lot of the vaccine. Group 1 represents all 3 of the groups
DISCUSSION
Since 1992, FDA has granted 8 novel vaccines accelerated approval based on evidence from a median of 9 clinical studies, including 1–2 pivotal efficacy trials, while requiring 2 postapproval trials. Nearly 90% of FDA-required postapproval trials were completed within 3 years after approval, a higher rate than observed for drug approvals.5 However, only 3 of the 8 vaccines had benefit confirmed by a postapproval trial using clinical outcomes. Our study was limited to FDA-required postapproval trials; other studies may have confirmed vaccine benefit.
Given the ongoing pandemic, FDA’s consideration of EUA or accelerated approval for COVID-19 vaccines is clearly justified.6 Moreover, Pfizer-BioNTech’s, Janssen’s, and Moderna’s pivotal efficacy trials each enrolled over 30,000 participants, used a clinical outcome as a primary efficacy endpoint, and demonstrated efficacy of 95%, 66%, and 94% in preventing COVID-19, respectively. These trials provide substantially more robust evidence than the studies used to support accelerated approval of vaccines by FDA, which had far smaller sample sizes and used surrogate measures. Going forward, FDA should acknowledge areas of evidentiary uncertainty associated with accelerated approval of vaccines and require completion of large, rigorously designed, and timely postapproval trials to assess long-term safety and clinical benefit.
Author Contribution
All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. ACE and JSR contributed to study concept and design; ACE abstracted the data; JDW validated the data and conducted the statistical analyses; all authors contributed to the analysis and interpretation of the data; ACE drafted the manuscript; all authors contributed to the critical revision of the manuscript; and JSR provided study supervision.
Funding
This project was not supported by any external grants or funds. Dr. Wallach is supported by the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under the award K01AA028258.
Data Availability
The datasets generated during the current study are available from the corresponding author on reasonable request.
Declarations
Conflict of Interest
All authors have completed the ICMJE uniform disclosure form at www.icmje.org/coi_disclosure.pdf and declare the following: In the past 36 months, Mr. Egilman and Drs. Ross, Wallach, and Zhang received research support through Yale University from the Laura and John Arnold Foundation for the Collaboration for Research Integrity and Transparency (CRIT) at Yale; Mr. Egilman and Drs. Ross and Wallach currently receive and Dr. Zhang has received support from the Food and Drug Administration for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938); Dr. Ross received research support through Yale University from Medtronic, Inc. and the Food and Drug Administration (FDA) to develop methods for postmarket surveillance of medical devices (U01FD004585) and from the Centers of Medicare and Medicaid Services (CMS) to develop and maintain performance measures that are used for public reporting (HHSM-500-2013-13018I); Dr. Ross currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology (NEST), from the Agency for Healthcare Research and Quality (R01HS022882), from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164, R01HL144644), and from the Laura and John Arnold Foundation to establish the Good Pharma Scorecard at Bioethics International. Drs. Puthumana and Schwartz have no competing interests to disclose.
Disclaimer
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Footnotes
Prior presentations:
None.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.FDA Center for Biologics Evaluation and Research. Emergency Use Authorization for Vaccines to Prevent COVID-19: Guidance for Industry. October 2020. Available at https://www.fda.gov/regulatory-information/search-fda-guidance-documents/emergency-use-authorization-vaccines-prevent-covid-19. Accessed May 4, 2021.
- 2.Lynch HF, Bateman-House A, Joffe S. Emergency approvals for COVID-19: evolving impact on obligations to patients in clinical care and research. Ann Intern Med. 2020;174(2):256–7. doi: 10.7326/M20-6703. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Downing NS, Aminawung JA, Shah ND, Krumholz HM, Ross JS. Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012. JAMA. 2014;311(4):368–77. doi: 10.1001/jama.2013.282034. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Puthumana J, Egilman AC, Zhang AD, Schwartz JL, Ross JS. Speed, evidence, and safety characteristics of vaccine approvals by the US Food and Drug Administration. JAMA Intern Med. 2021;181(4):559. doi: 10.1001/jamainternmed.2020.7472. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Naci H, Smalley KR, Kesselheim AS. Characteristics of preapproval and postapproval studies for drugs granted accelerated approval by the US Food and Drug Administration. JAMA. 2017;318(7):626–36. doi: 10.1001/jama.2017.9415. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Dal-Ré R, Caplan AL, Gluud C, Porcher R. Ethical and scientific considerations regarding the early approval and deployment of a COVID-19 vaccine. Ann Intern Med. 2020;174(2):258–60. doi: 10.7326/M20-7357. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets generated during the current study are available from the corresponding author on reasonable request.