A comprehensive analysis of the efficacy and safety of COVID-19 vaccines

Abstract

The numbers of cases and deaths from coronavirus disease 2019 (COVID-19) are continuously increasing. Many people are concerned about the efficacy and safety of the COVID-19 vaccines. We performed a comprehensive analysis of the published trials of COVID-19 vaccines and the real-world data from the Vaccine Adverse Event Reporting System. Globally, our research found that the efficacy of all vaccines exceeded 70%, and RNA-based vaccines had the highest efficacy of 94.29%; moreover, Black or African American people, young people, and males may experience greater vaccine efficacy. The spectrum of vaccine-related adverse drug reactions (ADRs) is extremely broad, and the most frequent ADRs are pain, fatigue, and headache. Most ADRs are tolerable and are mainly grade 1 or 2 in severity. Some severe ADRs have been identified (thromboembolic events, 21–75 cases per million doses; myocarditis/pericarditis, 2–3 cases per million doses). In summary, vaccines are a powerful tool that can be used to control the COVID-19 pandemic, with high efficacy and tolerable ADRs. In addition, the spectrum of ADRs associated with the vaccines is broad, and most of the reactions appear within a week, although some may be delayed. Therefore, ADRs after vaccination need to be identified and addressed in a timely manner.

Graphical abstract

The numbers of cases and deaths from COVID-19 are continuously increasing. Cai et al. are the first to comprehensively analyze the efficacy of the existing COVID-19 vaccines and the incidence, spectrum, timing, and clinical features of adverse reactions associated with the COVID-19 vaccines, which can provide reference for general public.

Introduction

As of April 5, 2021, there were more than 131 million confirmed cases and more than 2.8 million deaths due to coronavirus disease 2019 (COVID-19) worldwide.¹ COVID-19 has posed a serious threat to public health worldwide. There is no cure for COVID-19, and only vaccines can stop the spread of the COVID-19 pandemic. According to the World Health Organization (WHO), as of April 5, 2021, 184 vaccines were being evaluated in the preclinical development stage, 85 were in the clinical evaluation stage, and some had partially passed through phase III clinical trials.² Vaccination against COVID-19 has now started in 161 locations, covering 91% of the global population.³ However, the vaccination rates are still low; as of April 5, 2021, the highest rate of full vaccination was 56.2% in Israel, while those in other countries were all lower than 20%, and those in some countries were 0%.⁴ A previous study pointed out that 53%–84% of the population needs to be vaccinated against COVID-19 to achieve herd immunity.⁵ However, as various mutations of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have been reported, herd immunity is becoming more and more unrealistic, unless a vaccine to protect against different variants of SARS-CoV-2 can be developed. Other than protection, vaccination can reduce the severity of COVID-19 infection and be life saving. One of the key reasons for the low vaccination rate is that many people are concerned about the safety and efficacy of the COVID-19 vaccines.

However, no reports have addressed this issue satisfactorily. It is important to perform an analysis of the safety and efficacy of the COVID-19 vaccines. Therefore, we performed a comprehensive analysis to determine the incidence, spectrum, timing, and clinical features of adverse drug reactions (ADRs) and the efficacy of the COVID-19 vaccines.

First, we performed a meta-analysis of the published trials of the COVID-19 vaccines. Furthermore, we retrospectively obtained real-world data from the Vaccine Adverse Event Reporting System (VAERS), which is comanaged by the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA) of the United States of America.⁶ In our research, we provided a complete overview of the COVID-19 vaccines in terms of the incidence, spectrum, timing, and clinical features of ADRs and efficacy. We do hope this study will provide a guideline for clinicians managing ADRs associated with the COVID-19 vaccines and increase the confidence of the general public in the COVID-19 vaccines.

Results

Efficacy of COVID-19 vaccines

To estimate the efficacy of the COVID-19 vaccines, we evaluated all the COVID-19 vaccine data that have been published from phase III clinical trials; a total of 194,015 cases were included. The overall efficacy was highly heterogeneous (>90%); therefore, we performed subgroup analyses with stratification by vaccine type, sex, and age, which effectively reduced the heterogeneity. The analysis of different types of vaccines showed that the efficacy of inactivated vaccines was 73.11% (95% confidence interval [CI], 34.23; 89.03), the efficacy of protein subunit vaccines was 89.33% (95% CI, 81.44; 93.10), and the efficacy of RNA-based vaccines was 94.29% (95% CI, 93.65; 95.40). The efficacy of the viral vector (non-replicating) vaccines was 79.56% (95% CI, 60.00; 89.92; Table1; Figure S2; Table S1).

Table 1.

The efficacy of COVID-19 vaccines

Group	The incidence of COVID-19 infection		Vaccine efficacy (95% CI)
	Vaccine group (95% CI)	Placebo group (95% CI)	VE = 100∗(1–RR) %
Vaccine type
Inactivated vaccine (n = 5,705 versus 5,440)	0.0096 (0.0034– 0.0269)	0.0357 (0.0310– 0.0409)	73.11% (34.23–89.03)
Protein subunit (n = 7,500 versus 7,500)	0.0008 (0.0004– 0.0018)	0.0075 (0.0058– 0.0097)	89.33% (81.44–93.10)
RNA-based vaccine (n = 48,578 versus 48,732)	0.0006 (0.0004– 0.0008)	0.0105 (0.0087– 0.0126)	94.29% (93.65–95.40)
Viral vector (non-replicating; n = 40,285 versus 30,275)	0.0037 (0.0013– 0.0102)	0.0181 (0.0129– 0.0255)	79.56% (60.00–89.92)
Gender
Male (n = 36,245 versus 30,071)	0.0010 (0.0003– 0.0038)	0.0137 (0.0094- 0.0200)	92.70% (81.00–96.81)
Female (n = 29,774 versus 25,955)	0.0018 (0.0006– 0.0054)	0.0148 (0.0098– 0.0223)	87.84% (75.78–93.88)
Age
16 to 55 years old (n = 47,958 versus 39,139)	0.0018 (0.0006– 0.0055)	0.0162 (0.0117– 0.0224)	88.89% (75.45–94.87)
Over 55 years old (n = 19,458 versus 18,390)	0.0013 (0.0005– 0.0038)	0.0105 (0.0067– 0.0164)	87.62% (76.83–92.54)
Race
Black or African American (n = 9,952 versus 9,979)	0.0005 (0.0000– 0.0138)	0.0108 (0.0043– 0.0265)	95.37% (47.92–100.00)
White (n = 35,650 versus 35,719)	0.0016 (0.0004– 0.0063)	0.0157 (0.0104– 0.0234)	89.81% (73.08–96.15)

Since inactivated vaccines and protein subunit vaccines lacked subgroup data, including age and sex, only RNA-based vaccines and viral vector (non-replicating) vaccines were included in the subsequent subgroup analyses. Vaccine efficacy (VE) among male and female participants was 92.70% (95% CI, 81.00; 96.81) and 87.84% (95% CI, 75.78; 93.88), respectively. At the same time, the efficacy of vaccine among 16 to 55 years old recipients was 88.89% (95% CI, 75.45; 94.87) and that among those over 55 years old was 87.62% (95% CI, 76.83; 92.54). Only RNA-based vaccines and viral vector (non-replicating) vaccines provided the data of different races. In the subgroup analysis, VE among Black or African American and White participants was 95.37% (95% CI, 47.92; 100.00) and 89.81% (95% CI, 73.08; 96.15), respectively. We found that all vaccines achieved good efficacy, among which RNA-based vaccines had the highest, whereas inactivated vaccines had the lowest, although they were more than 70% effective. In addition, Black or African American people, males, and the 16- to-55-year-old subgroup experienced greater VE (Table 1; Figure S2; Table S1).

Incidence of ADRs related to the COVID-19 vaccines

Safety is another important factor when considering vaccines. Therefore, we first performed a meta-analysis of the clinical trial data and then collected real-world data from the VAERS maintained by the CDC in the United States. In the clinical trials analysis, we evaluated a total of 6 phase III clinical trials and 6 phase I/II clinical trials and official reports of phase III results of COVID-19 vaccines, and 56,310 cases were included. Meanwhile, the data of 86,506,742 doses from 5 reports about the thromboembolic events were included, while 603,862,888 doses from 3 reports about the myocarditis/pericarditis events were included. In the real-world analysis, we included 11,936 participants. The results are as follows.

Incidence of ADRs in the meta-analysis of clinical trials

We observed 36 types of ADRs in the clinical trials, among which 8 were observed after vaccination with more than 50% of the vaccines (Table S2), including pain, swelling, fever, fatigue, chills, muscle pain (myalgia), joint pain (arthralgia), and headache. We further conducted a meta-analysis of these 8 ADRs. Similarly, to minimize heterogeneity, we performed subgroup analyses stratified by dose, vaccine type, and age.

Since inactivated vaccines lacked ADRs data of dose 1, only RNA-based vaccines, viral vector (non-replicating) vaccines, and protein subunit vaccines were included in the analyses. The results showed that the most frequently reported ADR was pain (at the injection site) after dose 1 in protein subunit vaccines (38.46%) and RNA-based vaccines (80.97%). Pain was reported more frequently in younger vaccine recipients (16 to 55 years old) than in older vaccine recipients (over 55 years old; 80.00% versus 59.35%). Fatigue was the second most frequent ADR after dose 1 (30.77% of those receiving the protein subunit vaccines and 39.27% of those receiving the RNA-based vaccines). The incidence in the 16- to 55-year-old subgroup was significantly higher than that in the over 55-year-old subgroup (52.72% versus 33.73%). The incidences of other ADRs were below 50%. Headache ranked third, followed by muscle pain (myalgia), joint pain (arthralgia), chills, swelling, and fever. The incidence of ADRs after vaccination with RNA-based vaccines was high, and further analysis of age subgroups indicated that the results were generally consistent with those observed in the overall analysis. Meanwhile, unlike the two vaccines above, in viral vector (non-replicating) vaccines, the most frequently reported ADR was fatigue (56.25%). Headache ranked second, followed by pain, muscle pain (myalgia), joint pain (arthralgia), chills, fever, and swelling (Table 2; Figures S3–S5; Table S2).

Table 2.

The incidence of ADRs associated with COVID-19 vaccines via meta-analysis

Group	Pain	Swelling	Fever	Fatigue	Chills	Muscle pain (myalgia)	Joint pain (arthralgia)	Headache
Dose 1 (subgroups by vaccine type, n = 23,505)
Protein subunit (n = 26)	0.3846 (0.2210–0.5793)	0.0000 (0.0000–1.0000)	0.0000 (0.0000–1.0000)	0.3077 (0.1620–0.5055)	0.0000 (0.0000–1.0000)	0.2308 (0.1075–0.4276)	0.0385 (0.0054–0.2279)	0.2308 (0.1075–0.4276)
RNA-based vaccine (n = 23,351)	0.8097 (0.7667–0.8463)	0.0624 (0.0594–0.0656)	0.0145 (0.0059–0.0350)	0.3927 (0.3630–0.4233)	0.0936 (0.0785–0.1112)	0.2029 (0.1726–0.2370)	0.1303 (0.0920–0.1813)	0.3366 (0.3218–0.3517)
Viral vector (non-replicating; n = 128)	0.4141 (0.3321–0.5011)	0.0156 (0.0039–0.0603)	0.0938 (0.0540–0.1578)	0.5625 (0.4755–0.6458)	0.1719 (0.1159–0.2473)	0.3594 (0.2811–0.4460)	0.2188 (0.1555–0.2986)	0.5234 (0.4371–0.6084)
Dose 1 (subgroups by age, n = 23,479)
16 to 55 years old (n = 16,177)	0.8000 (0.6778–0.8838)	0.0649 (0.0612–0.0688)	0.0458 (0.0094–0.1949)	0.5272 (0.3499–0.6979)	0.1615 (0.0842–0.2875)	0.2984 (0.1765–0.4578)	0.1733 (0.1043–0.2739)	0.4549 (0.3267–0.5892)
Over 55 years old (n = 7,302)	0.5935 (0.3553–0.7945)	0.0525 (0.0374–0.0731)	0.0049 (0.0019–0.0123)	0.3373 (0.3265–0.3482)	0.0568 (0.0518–0.0624)	0.1805 (0.1388–0.2313)	0.1275 (0.0897–0.1782)	0.2919 (0.2130–0.3857)
Dose 1 ≥ grade 3 (subgroups by vaccine type, n = 23,505)
Protein subunit	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000	0.0000
(n = 26)	(0.0000– 1.0000)	(0.0000– 1.0000)	(0.0000– 1.0000)	(0.0000–1.0000)	(0.0000–1.0000)	(0.0000–1.0000)	(0.0000–1.0000)	(0.0000–1.0000)
RNA-based vaccine	0.0138	0.0065	0.0022	0.0104	0.0030	0.0064	0.0048	0.0142
(n = 23,351)	(0.0052– 0.0361)	(0.0049–0.0086)	(0.0007– 0.0063)	(0.0092– 0.0118)	(0.0012– 0.0074)	(0.0054–0.0075)	(0.0037–0.0062)	(0.0101–0.0199)
Viral vector (non-replicating)	0.0000	0.0000	0.0078	0.0313	0.0234	0.0156	0.0078	0.0156
(n = 128)	(0.0000– 1.0000)	(0.0000–1.0000)	(0.0011–0.0533)	(0.0118– 0.0803)	(0.0076–0.0701)	(0.0039–0.0603)	(0.0011–0.0533)	(0.0039–0.0603)
RNA-based vaccine dose 1 (subgroups by age, n = 23,351)
16 to 55 years old (n = 16,128)	0.8507 (0.8219–0.8755)	0.0647 (0.0596–0.0702)	0.0193 (0.0069–0.0528)	0.4263 (0.3682–0.4865)	0.1137 (0.0847–0.1510)	0.2239 (0.2060–0.2429)	0.1358 (0.1014–0.1796)	0.3857 (0.3406–0.4328)
Over 55 years old (n = 7,223)	0.7250 (0.704–0.7451)	0.0555 (0.0401–0.0765)	0.0053 (0.0021–0.0135)	0.3361 (0.3253–0.3471)	0.0568 (0.0517–0.0623)	0.1669 (0.1310–0.2103)	0.1224 (0.0796–0.1836)	0.2473 (0.2375–0.2573)
Dose 2 (subgroups by vaccine type, n = 32,805)
Inactivated vaccine (n = 6,286)	0.3175 (0.0781–0.7188)	0.0576 (0.0521–0.0636)	0.0060 (0.0007–0.0515)	0.0533 (0.0072–0.3041)	0.0161 (0.0008–0.2528)	0.0152 (0.0002–0.5744)	0.0156 (0.0006–0.2925)	0.0778 (0.0050–0.5877)
Protein subunit (n = 26)	0.5769 (0.3851–0.7480)	0.0385 (0.0054–0.2279)	0.0000 (0.0000–1.0000)	0.4615 (0.2839–0.6495)	0.0000 (0.0000–1.0000)	0.4615 (0.2839–0.6495)	0.2692 (0.1341–0.4671)	0.4615 (0.2839–0.6495)
RNA-based vaccine (n = 22,860)	0.8176 (0.6887–0.9009)	0.0890 (0.0565–0.1374)	0.1473 (0.1364–0.1590)	0.6057 (0.5369–0.6705)	0.3677 (0.2750–0.4713)	0.4553 (0.2945–0.6260)	0.3069 (0.1760–0.4785)	0.5260 (0.4418–0.6088)
Viral vector (non-replicating; n = 3,633)	0.4475 (0.3455–0.5540)	0.0534 (0.0465–0.0612)	0.1796 (0.0007–0.9846)	0.3922 (0.3765–0.4082)	0.0020 (0.0000–0.2194)	0.2622 (0.1949–0.3429)	0.0197 (0.0005–0.4453)	0.3473 (0.2742–0.4283)
Dose 2 (subgroups by age, n = 26,344)
16 to 55 years old (n = 17,988)	0.7240 (0.5312–0.8586)	0.0699 (0.0413–0.1158)	0.1160 (0.0499–0.2471)	0.5681 (0.4737–0.6577)	0.0440 (0.0008–0.7399)	0.4368 (0.3307–0.5490)	0.0295 (0.0006–0.6011)	0.4901 (0.3842–0.5969)
Over 55 years old (n = 8,356)	0.5106 (0.2473–0.7682)	0.0528 (0.0279–0.0975)	0.0478 (0.0167–0.1291)	0.4367 (0.3235–0.5569)	0.0291 (0.0010–0.4605)	0.2992 (0.2108–0.4055)	0.0396 (0.0014–0.5424)	0.3600 (0.2885–0.4384)
Dose 2 ≥ grade 3 (subgroups by vaccine type, n = 32,592)
Inactivated virus	0.0000	0.0000	0.0000	0.0000	0.0001	0.0000	0.0001	0.0000
(n = 6,202)	(0.0000–1.0000)	(0.0000–1.0000)	(0.0000– 1.0000)	(0.0000–1.0000)	(0.0000– 0.0013)	(0.0000–1.0000)	(0.0000– 0.0013)	(0.0000– 1.0000)
Protein subunit	0.0000	0.0000	0.0000	0.0385	0.0189	0.0385	0.0385	0.0000
(n = 26)	(0.0000– 1.0000)	(0.0000–1.0000)	(0.0000–1.0000)	(0.0054– 0.2279)	(0.0012–0.2938)	(0.0054–0.2279)	(0.0056–0.2628)	(0.0000– 1.0000)
RNA-based vaccine	0.0212	0.0072	0.0135	0.0634	0.0164	0.0425	0.0245	0.0344
(n = 22,860)	(0.0083–0.0533)	(0.0021– 0.0248)	(0.0117– 0.0156)	(0.0343– 0.1145)	(0.0104–0.0258)	(0.0144– 0.1188)	(0.0055– 0.1097)	(0.0235– 0.0501)
Viral vector (non-replicating)	0.0031	0.0017	0.0023	0.0103	0.0024	0.0094	0.0024	0.0071
(n = 3,504)	(0.0017–0.0057)	(0.0008– 0.0038)	(0.0011–0.0046)	(0.0074– 0.0142)	(0.0001–0.0452)	(0.0067–0.0132)	(0.0001– 0.0452)	(0.0048– 0.0105)
RNA-based vaccine dose 2 (subgroups by age, n = 22,860)
16 to 55 years old (n = 15,707)	0.8488 (0.7482–0.9138)	0.0879 (0.0517–0.1454)	0.1681 (0.1588–0.1778)	0.6346 (0.5728–0.6922)	0.4168 (0.3259–0.5136)	0.4928 (0.3261–0.6611)	0.3267 (0.1863–0.5069)	0.5751 (0.4988–0.6480)
Over 55 years old (n = 7,153)	0.7557 (0.6183–0.8553)	0.0874 (0.0642–0.1179)	0.1050 (0.0981–0.1123)	0.5467 (0.4958–0.5967)	0.2678 (0.2166–0.3261)	0.3763 (0.2602–0.5087)	0.2622 (0.1667–0.3870)	0.4255 (0.3768–0.4757)
Viral vector vaccine dose 2 (subgroups by age, n = 3,484)
16 to 55 years old	0.5831	0.0299	0.0257	0.4419	0.0010	0.3906	0.0008	0.4445
(n = 2,281)	(0.5627–0.6032)	(0.0019–0.3302)	(0.0006– 0.5449)	(0.4216– 0.4624)	(0.0000– 0.9134)	(0.3708–0.4108)	(0.0000–0.6305)	(0.4243– 0.4650)
over 55 years old	0.2736	0.0266	0.0291	0.3009	0.0013	0.2377	0.0017	0.3026
(n = 1,203)	(0.1816– 0.3898)	(0.0189– 0.0374)	(0.0210– 0.0403)	(0.2756– 0.3274)	(0.0000– 0.3578)	(0.2145– 0.2626)	(0.0000– 0.8062)	(0.2773– 0.3291)

Among participants who received dose 2, the overall incidence of ADRs was higher than that after dose 1. As observed for dose 1, pain was the most frequent ADR. The incidences of pain in patients administered different types of vaccines were as follows: inactivated vaccines (31.75%), protein subunit vaccines (57.69%), RNA-based vaccines (81.76%), and viral vector (non-replicating) vaccines (44.75%). The incidences of pain in different age groups were as follows: 16 to 55 years old (72.40%) and over 55 years old (51.06%). The incidences of ADRs other than pain differed among the various types of vaccines. In particular, the incidence of ADRs was lowest for inactivated vaccines, with incidences of all ADRs less than 10%. In descending order of frequency, the ADRs were headache, swelling, fatigue, chills, joint pain (arthralgia), muscle pain (myalgia), and fever. The ADRs associated with the other three types of vaccines were similar to those after dose 1, with fatigue and headache ranking second and third, respectively. However, more than 50% of recipients experienced headache after dose 2, unlike after dose 1. Moreover, among the other ADRs with incidences less than 50%, chills ranked fifth after dose 2, while it had ranked sixth after dose 1, and the other ADRs in order were joint pain (arthralgia), swelling, and fever. For RNA-based vaccines and viral vector (non-replicating) vaccines, consistent results were obtained among subgroups stratified by age (Table 2; Figures S7–S10; Table S2).

To assess the severity of vaccine-related ADRs, we calculated the proportions (the ADRs over grade 3/all ADRs) and conducted a meta-analysis according to severity grade. The results showed that the ADRs associated with the RNA-based vaccine (Moderna, BNT162b2) were the most severe, and instances of grade 3 reactions were reported for all 8 ADRs. Fortunately, the proportions were low, and even the largest was less than 20%. Grade 3 ADRs also occurred after vaccination with viral vector (non-replicating) vaccines (AZD1222, Sputnik V); however, the proportions were low (less than 10%). Most ADRs after vaccination with inactivated vaccines (BBIBP) and protein subunit vaccines (NVX-COV2373) were grades 1–2. Importantly, among the participants who received the first dose of an RNA-based vaccine, grade 4 fever was noted, but the proportion was less than 5%. Meanwhile, we also found that younger participants were more likely to report higher-grade ADRs than older participants. For the RNA-based vaccine (Moderna) and protein subunit vaccine (NVX-COV2373), the ADR grades were higher after the second dose than the first dose (Figure 1; Table S2).

Figure 1.

The severity of vaccine-related ADRs in clinical trials

Stacked bar chart showing the percentage of four ADRs grade after dose 1 or dose 2 of COVID-19 vaccines. (A) pain, (B) swelling, (C) fever, (D) fatigue, (E) chills, (F) muscle pain (myalgia), (G) joint pain (arthralgia), and (H) headache. Grade 1 (dark blue), grade 2 (light blue), grade 3 (red), and grade 4 (brown).

In the analysis of ADRs over grade 3, the incidences were all less than 10%, among which the most frequently reported ADR was fatigue (6.34%) in RNA-based vaccines after dose 2. The ADR grades were higher after the second dose than the first dose in RNA-based vaccines, contrary to the viral vector (non-replicating) vaccines (AZD1222, Sputnik V). What’s more, the incidences of the ADRs over grade 3 in viral vector (non-replicating) vaccines were higher than those in RNA-based vaccines after dose 1 (Table 2; Figures S6 and S11; Table S2).

The severe and rare ADRs of COVID-19 vaccines

Besides the ones that have been reported in the clinical trials, there are some severe and rare ADRs, such as thromboembolic events and myocarditis/pericarditis events, which may result in death. Our results showed that thromboembolic events were only found in viral vector (non-replicating) vaccines (Ad26.COV2.S and AZD1222), while myocarditis/pericarditis events were reported in both viral vector (non-replicating) vaccines (Ad26.COV2.S and AZD1222) and RNA-based vaccines (BNT162b2 and Moderna). The incidence of thromboembolic events in Ad26.COV2.S (75 cases per million doses) was higher than that in AZD1222 (21 cases per million doses; Figure 2A; Table S3). The incidence of myocarditis/pericarditis events was similar in viral vector (non-replicating) vaccines and RNA-based vaccines (2 versus 3 cases per million doses; Figure 2B; Table S3).

Figure 2.

Forest plot of the incidence of thromboembolic events and myocarditis/pericarditis events

Meta-analysis was performed using R statistical software. Event rates and their corresponding 95% confidence intervals were estimated using both a fixed-effects model and a random-effects model. (A) Thromboembolic events and (B) myocarditis/pericarditis events.

Incidence of ADRs associated with RNA-based vaccines in the real world (VAERS)

To evaluate the safety of the COVID-19 vaccines more comprehensively, we retrospectively obtained real-world data pertaining to ADRs associated with RNA-based vaccines from VAERS. A total of 11,936 participants were included in the study, among whom 4,990 were vaccinated with the Moderna vaccine and 6,946 were vaccinated with the Pfizer-BioNTech vaccine (Table S4).

Our research revealed an unexpected phenomenon. The incidence of ADRs in the real world was far lower than that in clinical trials. The ADR with the highest incidence is headache (16.53%), but the spectrum of ADRs is significantly wider than that in clinical trials. We identified more than 700 ADRs, but the incidence of most ADRs (more than 90%) was lower than 1% (Figure 3D). To evaluate the tolerance of the vaccine in different populations, we conducted subgroup analyses stratified by age, sex, and vaccine manufacturer. All ADRs with incidences higher than 5% were included. After stratification by the vaccine manufacturer (Moderna and Pfizer-BioNTech), the results showed that there were no significant differences in the incidences of headache, pain, myalgia, and nausea, but the incidences of chills, pyrexia, injection site pain, injection site erythema, pain in the extremities, and injection site swelling were higher among patients vaccinated with the Moderna vaccine than among those vaccinated with the Pfizer-BioNTech vaccine. In contrast, fatigue, dizziness, and dyspnea occurred more frequently in patients vaccinated with the Pfizer-BioNTech vaccine (Figure 4; Table S6). The details of the incidences of all ADRs associated with the different vaccines are shown in Table S6. Headache was still the most frequent ADR after the subgroup analysis was performed with stratification by age. Meanwhile, among those vaccinated with the Moderna vaccine, all ADRs were reported more often in older participants than young participants. The result was the opposite for the Pfizer-BioNTech vaccine (Figures 3D and 4; Tables S7 and S8). The details of the incidences of all ADRs in different age groups are shown in Tables S7 and S8. In the analysis stratified by sex, we found that regardless of whether the Moderna or Pfizer-BioNTech vaccine was administered, pyrexia ranked first, which was different from the results of the other subgroup analyses. Other than pyrexia and chills, which were more common in males, the incidences of other ADRs were higher in females than in males (Figures 3D and 4; Tables S9 and S10). The details of the incidences of all ADRs stratified by sex are shown in Tables S9 and S10.

Figure 3.

The incidence of ADRs of RNA-based vaccine from real-world data (VAERS)

Log-rank test of ADRs onset time stratified by (A) vaccine type, (B) age, and (C) gender. (D) Heatmap showing the incidence of ADRs. (∗ADRs Spectrum: due to the limitation of figure size, the details are shown in Table S5.)

Figure 4.

The subgroup analyses of ADRs in RNA-based vaccine from real-world data (VAERS)

To evaluate the tolerance of the vaccine in different populations, we conducted subgroup analyses stratified by age, sex, and vaccine manufacturer. All ADRs with incidences higher than 5% were included. (∗No application: the incidences of ADRs under 5% in the subgroups were defined as “no application,” which were not tested by χ².)

We also further explored the timing of the onset of ADRs. Most participants developed symptoms within a week after vaccination, but the longest interval was 60 days. The median symptom onset time for the Moderna and Pfizer-BioNTech vaccines were 2 days and 1 day, respectively, but the difference was not statistically significant (Figure 3A, p = 0.07). Symptoms appeared earlier in young participants, and the median interval was 1 day, while in older people, it was 2 days (Figure 3B, p < 0.0001). Symptoms appeared earlier in females, with a median interval of 1 day, while in males it was 2 days (Figure 3C, p < 0.0001).

Discussion

COVID-19 remains a global public health threat, although it has been more than a year since the first case was diagnosed. The number of cases and deaths from COVID-19 continues to increase. Undoubtedly, vaccines are the most promising means to control the COVID-19 pandemic. As of April 5, 2021, several vaccines had been approved for public use, including RNA-based vaccines (Moderna and Pfizer-BioNTech), inactivated vaccines (Sinopharm [BBIBP], CoronaVac, Covaxin, Sinopharm [WIBP], and CoviVac), viral vector vaccines (Oxford-AstraZeneca, Sputnik V, Johnson & Johnson, and Convidecia), and protein subunit vaccines (EpiVacCorona, RBD-Dimer).³ Although vaccinations are continuing to be administered, the vaccinated population only accounts for a small proportion of the entire population, and safety and efficacy are the issues about which many people are concerned.

This is the first study on the efficacy and safety of COVID-19 vaccines using published clinical trial data and real-world data. We comprehensively analyzed the efficacy of the existing COVID-19 vaccines and their incidence, spectrum, timing, and clinical features of ADRs after vaccination. Our research indicated that the efficacy of all vaccines exceeded 70% and that RNA-based vaccines had the highest efficacy of 94.29%; moreover, young people, Black or African American people, and males may experience greater vaccine efficacy. The spectrum of vaccine-related ADRs is extremely broad, involving multiple systems. The most common ADRs are pain, fatigue, and headache. Most ADRs are tolerable and mainly in grade 1 or 2 in severity; only grade 4 fever has been observed. Some severe ADRs have been identified, though the incidences were low (thromboembolic events, 21–75 cases per million doses; myocarditis/pericarditis, 2–3 cases per million doses). Most symptoms appear soon after vaccination, and many people recover without any medication.

In terms of efficacy, RNA-based vaccines ranked first, reaching greater than 94%, due to their strong immunogenicity and effective presentation of SARS-CoV-2 antigens to the immune system.⁷ Currently, mutant virus strains are also attracting attention. RNA-based vaccines may be more effective against these mutant strains owing to their use of the full immunogenicity of SARS-CoV-2. However, the incidence of ADRs is high after vaccination with RNA-based vaccines, reaching over 80% based on the clinical trial data, with the incidences of grade 3 or 4 ADRs accounting for a small proportion. Although the real-world incidence of ADRs was lower than that in the clinical trials, the spectrum was broader, and a large portion of types of ADRs were not observed in clinical trials, suggesting that attention should be given to the identification and treatment of rare ADRs. Meanwhile, myocarditis/pericarditis have been identified in RNA-based vaccines; fortunately, the incidence was low. Protein subunit vaccines had an efficacy of 89%, while the highest incidence of ADRs was only 57%, and highest incidence of the ADRs over grade 3 was 3.85%, significantly lower than that associated with RNA-based vaccines; therefore, it may be a promising candidate. However, because real-world data regarding protein subunit vaccines are lacking and the sample of published data is small, further analysis is needed. Moreover, viral vector (non-replicating) vaccines have an efficacy of 79%, while the highest incidence of ADRs is 40%. In addition, the incidence of ADRs above grade 3 is significantly lower than that associated with RNA-based vaccines. However, some thromboembolic events and myocarditis/pericarditis events have been reported after vaccination with viral vector (non-replicating) vaccines (Ad26.COV2.S and AZD1222), which are very severe. Fortunately, the incidences of thromboembolic events and myocarditis/pericarditis events were low. Inactivated vaccines, in particular, are very safe and easy to preserve and transport, although their efficacy is relatively lower.

In the subgroup analysis, the ADRs after dose 1 of viral vector (non-replicating) vaccine (AZD1222, Sputnik V) occurred more often than dose 2. In contrast, the incidence of ADRs was higher after dose 2 of the RNA-based vaccine produced by Moderna and the protein subunit vaccine called NVX-COV2373. The results suggest that there are differences among the vaccines, and the monitoring of ADRs cannot be taken lightly even if no adverse reaction occurs following dose 1, especially among those receiving RNA-based vaccines (e.g., Moderna) and protein subunit vaccines (e.g., NVX-COV2373). The second dose should not be avoided because of ADRs after dose 1. The process of building tolerance to viral vector (non-replicating) vaccines is gradual in vaccinated recipients. We also found that young people seem to be relatively more prone to higher grade ADRs. We speculate that the relatively stronger immune systems in young people lead to both a higher incidence of ADRs and greater vaccine efficacy.⁸ This finding also reduces concerns about vaccinating elderly people. The higher incidence of ADRs among female participants than male participants is puzzling, because it suggests that a stronger immune response was elicited in females, but the efficacy is lower in females than in males. This is inconsistent with the results of previous studies on sex differences.⁹ The specific reasons need to be explored further. Furthermore, in the analysis of the timing of the onset of ADRs, we found that young people and females developed symptoms earlier, which may be related to the higher incidence of ADRs and their stronger immune systems.⁹ In addition, the interval between vaccination and the development of ADRs in some patients can be up to 60 days, suggesting that the vaccination history should be actively reported when symptoms develop after vaccination and clinicians should pay attention to the lag between vaccination and the development of ADRs.

In the ADR analysis, the real-world data from the VAERS and clinical trial data were compared. We found that there are differences in the spectrum of ADRs, with a wider spectrum of ADRs identified in the real-world data. One plausible explanation is that the data in VAERS are continuously and openly collected. However, only ADRs that occurred within 1 week were counted in most clinical trials, and those that appeared after 1 week were omitted. In addition, the VAERS system lacks a standardized description of symptoms, with multiple different descriptions referring to the same ADR, falsely increasing the spectrum of ADRs. Another surprising finding is that the incidence of ADRs in the real world is far lower than that in clinical trials. Real-world data are only available for RNA-based vaccines, and the sample size is not yet large enough. Additionally, the VAERS is a self-reporting system with reporting bias,¹⁰ and a large number of participants who were vaccinated did not report their ADRs, resulting in a lower incidence rate than in clinical trials.

We also found that few cases of mortality were reported to VAERS, and there was not enough evidence to indicate that the death was related to vaccination after carefully assessing each case. Therefore, a large-scale real-world study is needed for further confirmation.

In addition to the possible bias in VAERS, our study also has other deficiencies. The heterogeneity of several subgroups was large in the meta-analysis. To minimize heterogeneity, we used a total of 5 transformation methods (PFT, PAS, PRAW, PLN, PLOGIT) and chose the method by which the lowest heterogeneity was achieved.¹¹

In addition, we also conducted sensitivity analyses and multiple subgroup analyses to minimize heterogeneity. Both fixed-effect model and random-effect model were performed. When I² was less than 50% and p > 0.1, the fixed-effect model was chosen; otherwise, the random-effect model was chosen.12, 13, 14

The Begg’s and Egger’s tests were not used because there were not more than 10 subjects in each group.¹⁵ Although some subgroups were heterogeneous, we determined that the heterogeneity was derived from the data itself after sufficient statistical correction and analysis, possibly due to factors such as the area in which the study was conducted, the risk of exposure to SARS-CoV-2, and other factors that were beyond our control. Therefore, our research comprehensively demonstrated the efficacy and safety of the COVID-19 vaccines to the greatest extent possible, providing a credible reference for clinical practice and the general public.

In summary, vaccines are a powerful tool against the COVID-19 pandemic, with high efficacy and tolerable adverse reactions. Each vaccine has its own advantages and shortcomings, and every citizen should choose to be vaccinated as soon as possible. In addition, the spectrum of ADRs associated with the vaccines is broad, and most of the reactions appear within a week, although a delay sometimes occurs. Some severe ADRs have been identified, though the incidences were low (thromboembolic events and myocarditis/pericarditis). Therefore, ADRs should be identified and addressed in a timely manner after vaccination. We hope that our research can eliminate fear of the vaccines among the general public and provide guidance regarding the management of vaccine-related side effects in a timely manner.

Materials and methods

Meta-analysis

Part 1: The landscape of efficacy and safety of COVID-19 vaccines

Inclusion criteria

The study was registered in PROSPERO (CRD42021234481). We identified records by searching PubMed, Medline, EMBASE, and the Cochrane Central Register of Controlled Trials (CENTRAL) for “(COVID-19 OR 2019-nCoV OR SARS-CoV-2) AND vaccine” on March 7, 2021. English-language clinical trials were included.

Exclusion criteria

All 8,215 initially identified studies were screened; those that were clinical trials were included (n = 53), and those in which a vaccine against SARS-CoV-2 was not used were excluded (n = 29). Trials without adverse effect or efficacy data (n = 1) and those with only the clinical trial protocol (n = 1) were excluded.

The remaining trials (n = 22) included 17 phase I/phase II clinical trials of 12 vaccines, and 4 of these vaccines had published phase III clinical trial results (n = 5). We further searched for the remaining 8 vaccines on Google using the following keywords: “(candidate vaccine name or manufacturer) AND (COVID-19 OR 2019-nCoV OR SARS-CoV-2).” Phase I/phase II trials of vaccines that did not have official results from phase III clinical trials were excluded (n = 11).

The remaining trials (n = 11) included 8 different vaccines, phase III clinical trials were updated on June 17, 2021, and a new trial of Ad26.COV2.S vaccine was included (n = 1). Finally, 12 clinical trials were assessed individually, and a total of 194,015 cases were included.16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27 The total number of patients treated, the number and type of adverse effects, the VE were compared, and the PRISMA diagram of articles selected for meta-analysis was shown in Figure S1 (Figure S1A; Table 3).

Table 3.

Clinical trials and the characteristics of COVID-19 vaccines

Vaccine name	Vaccine type	Developer	Participant age (years)	Dose schedule	Reference
BNT162b2	RNA-based vaccine	Pfizer/BioNTech; Fosun Pharma	16–55 >55	0 days: 30 μg 21 days: 30 μg	Polack et al.16
mRNA-1273	RNA-based vaccine	Moderna; National Institute of Allergy and Infectious Diseases (NIAID)	18–65 >65	0 days, 100 μg 28 days, 100 μg	Baden et al.17
AZD1222	viral vector (non-replicating)	AstraZeneca; University of Oxford	18–55 56–69 ≥70	0 days: low dose (2.2 × 1010 virus particles)/standard dose (3.5–6.5 × 1010 virus particles) 28 days: standard dose (3.5–6.5 × 1010 virus particles)	Voysey et al.18 and Ramasamy et al.19
Sputnik V	viral vector (non-replicating)	Gamaleya Research Institute; Health Ministry of the Russian Federation	18–30 31–40 41–50 51–60 >60	0 days: rAd26 1011 viral particles 21 days: rAd5 1011 viral particles	Logunov et al.20,21
CoronaVac	inactivated virus	Sinovac Research and Development	18–59 ≥60	0 days: 3 μg 14 days: 3 μg	Bureau25
NVX-CoV2373	protein subunit	Novavax	18–84	0 days: 5 μg, with Matrix-M1 adjuvant 21 days: 5 μg, with Matrix-M1 adjuvant	Keech et al.22 and Novavax I24
BBIBP-CorV	inactivated virus	Sinopharm, China National Biotec Group, and Beijing Institute of Biological Products	18–80	0 days: 4 μg 21 days: 4 μg	Xia et al.23
Ad5-nCoV	viral vector (non-replicating)	Cansino Biological/Beijing Institute of Biotechnology	≥18	5 × 1011 virus particles, one dose	Zhu et al.26
Ad26.COV2.S	viral vector (non-replicating)	Janssen/Johnson & Johnson	18–59 ≥60	single dose: 5 × 1010 viral particles	Sadoff et al.27

Part 2: The severe and rare ADRs of COVID-19 vaccines

Inclusion criteria

We identified records by searching PubMed, Medline, EMBASE, Google, and the CENTRAL for “(Thromboembolic OR Myocarditis OR Pericarditis) AND COVID-19 vaccine” on June 17, 2021. English-language clinical trials and official reports were included.

Exclusion criteria

All 2,910,000 results initially identified studies were screened; those that were clinical trials (n = 1), cohort study (n = 3), case reports (n = 12), and official reports (n = 8) were included. Those studies without the data of the exact total number and exact number of patients with thromboembolic or myocarditis or pericarditis were excluded (n = 14). Those official reports that were outdated or without the data of the exact vaccine type (n = 3) were excluded.

The remaining trial (n = 1),²⁷ cohort study (n = 1),²⁸ and official reports (n = 5)29, 30, 31, 32, 33 were assessed individually, and a total of 86,506,742 doses with thromboembolic events and 603,862,888 doses with myocarditis/pericarditis events were included. The total number of doses, the number and type of adverse effects, and the vaccine types were compared, and the PRISMA diagram of articles selected for meta-analysis is shown in Figure S2 (Figure S1B; Table S3).

VAERS

The study is based on data downloaded from the VAERS (https://vaers.hhs.gov/data.html). The VAERS is comanaged by the CDC and the FDA and has been used to detect possible safety problems in U.S.-licensed vaccines since 1990. Healthcare providers, vaccine manufacturers, and the public can submit reports to the system.⁶

We accessed the VAERS on March 5, 2021 and downloaded data from 2020 and 2021. We included all entries in which the patient had been injected with the Moderna or Pfizer COVID-19 vaccine. Patients injected with COVID-19 vaccines manufactured by unknown developers or vaccines against other pathogens were excluded.

Statistics

VE was calculated as 1-relative risk (RR):^34,35

$$VE=\frac{\text{Risk among unvaccinated group }-\text{ risk among vaccinated group}}{\text{Risk among unvaccinated group}}=1-\text{RR}$$

The incidence of ADRs was extracted by “Engauge Digitizer” from histograms if the raw data were not displayed.³⁶ The incidences of ADRs were compared with χ2 tests. Other clinical variables of interest were evaluated descriptively. Statistical analyses were performed in GraphPad Prism (version 7, GraphPad Software); the meta-analysis was performed using R statistical software (packages metafor and meta, R Foundation). Event rates and their corresponding 95% confidence intervals were estimated using both a fixed-effects model and a random-effects model. Forest plots were constructed to summarize the data for each analytical group according to the incidence rate and to provide a visual analysis of fatal drug-related events.