Safety and effectiveness of SARS‐CoV‐2 vaccines: A systematic review and meta‐analysis

Abstract

Objective: To systematically evaluate the effectiveness and safety of the SARS‐CoV‐2 vaccines currently undergoing clinical trials. Methods: PubMed, EMBASE, and Cochrane Library databases were searched to collect open human COVID‐19 vaccines randomized controlled trials, without limiting the search time and language. The research papers collected in the above‐mentioned databases were initially screened according to the title and abstract content and merged, and the repeated ones were removed. After reading the full text of the remaining research, the studies that did not meet the inclusion criteria were excluded, and finally, nine studies were obtained. After extracting the statistical data of adverse events in the study, load them into Review Manager for heterogeneity analysis. Results: The incidence of adverse reactions of inactivated virus vaccines, RNA vaccines, and adenovirus vector vaccines was higher than that of placebo. Common adverse reactions included pain, swelling, and fever at the injection site. Conclusion: From the perspective of effectiveness, RNA vaccine > adenovirus vector vaccine > inactivated virus vaccine. From the perspective of safety, the incidence of adverse reactions of the three vaccines is higher than that of a placebo, and the incidence of adverse reactions of the adenovirus vector vaccine is higher.

Highlights

• Compare the three mainstream COVID‐19 vaccines on the market.

1. INTRODUCTION

Statistics from the World Health Organization show that as of 14:44 on March 10, 2021, there were 117 332 262 confirmed cases of COVID‐19 worldwide, including 2 605 356 deaths (https://covid19.who.int/). COVID‐19 is a serious threat to human life. Although various countries have implemented measures against COVID‐19, including wearing masks, reducing crowd gathering, conducting travel surveys, and quarantining people with fever symptoms, the number of COVID‐19 infections in some regions is still rising. Therefore, there is an urgent need for a vaccine to provide some protection against highly infectious COVID‐19. With the efforts of scientific researchers from various countries, a total of 81 vaccines are currently in clinical trials, and another 182 are registered for clinical trials. ¹ Generally speaking, vaccines typically take 5 to 15 years to develop and this emergency has greatly shortened the time from clinical trials to marketing. The safety of the COVID‐19 vaccine in humans and its effectiveness against SARS‐CoV‐2 have become hot issues.

There are several vaccine types available for COVID‐19: protein subunit vaccines, nonreplicating virus vector vaccines, DNA vaccines, inactivated virus vaccines, and RNA vaccines. Among them, inactivated virus vaccines from Beijing Institute of Biological Products, ² Wuhan Institute of Biological Products, ³  and Sinovac Biotech ⁴ have been granted conditional market approval in China. Pfizer has received conditional marketing authorization for its RNA vaccine in the European Union. ⁵ CanSino has received conditional marketing authorization for its adenovirus vector vaccine in China. ⁶ Most of the vaccines that have received conditional marketing authorization are inactivated virus vaccines, RNA vaccines, and viral vector vaccines. For this reason, this article will make a systematic review of the three types of vaccines mentioned above.

As far as we know, although several basic research and clinical trial results on the COVID‐19 vaccine have been published, there is still a lack of comprehensive meta‐analysis on the safety and effectiveness of the COVID‐19 vaccine. We will summarize and analyze the effectiveness and adverse reactions of the current COVID‐19 vaccine in clinical trials to provide important guidance for subsequent clinical applications.

2. METHODS

2.1. Search strategy and eligibility criteria

This study was designed based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta‐Analyses) statement. ⁷ Systematic literature searches were performed on PubMed, EMBASE, and Cochrane Library databases, respectively. The deadline for searching is March 9, 2021, and no language restriction was applied. Medical subject headings and free words, such as “COVID‐19 Vaccines,” “SARS‐CoV‐2 vaccine,” “COVID‐19 Virus Vaccines,” “2019‐nCoV vaccine,” were used for search in the above databases. The search was limited to randomized controlled clinical trials in humans.

The following conditions were required to be included in the study: (1) the type of trial was randomized controlled blinded trial; (2) the subjects were healthy people who had not been infected with SARS‐CoV‐2; (3) normal saline or adjuvant were used as controls, and no other vaccines were used as controls; (4) the study provides statistical information on adverse events following vaccination, including any adverse events, local adverse events, and systemic adverse events; (5) the intervention was administering COVID‐19 vaccine. The retrieval results were screened with the help of Endnote and duplicate literature was eliminated. According to the research title and abstract, studies and reviews unrelated to this study were eliminated, and studies that did not meet the inclusion criteria were eliminated after reading the full text.

2.2. Data extraction

Bias evaluation was performed on the included studies based on the Cochrane risk bias assessment tool ⁸ to assess the degree of bias risk in the randomization process, allocation concealment, blind application of subjects and investigators, integrity of study data, selective outcome reporting, and other bias risks. The first author of all studies, publication year, group information, phase of clinical trials, name and type of vaccine (Table 1) as well as any adverse events after vaccination, local adverse events, systemic adverse events, common adverse events that occur after vaccination (pain, erythema, swelling, fever, headache, and fatigue), combined with the antibody geometric average concentration, combined with the geometric mean titer of the antibody, statistics on vaccine efficacy was extracted into a predesigned table.

Table 1.

Related information of included studies

Study	Year	Clinical trials registration	Phase	Vaccine Name	Vaccine type	Male (%)	Female (%)	Age range
Baden 9	2021	NCT04470427	III	mRNA‐1273	RNA vaccines	52.7	47.3	18–95
Mulligan 10	2020	NCT04368728	I/II	BNT162b1	RNA vaccines	51.1	48.9	18–55
Polack 11	2020	NCT04368728	III	BNT162b2	RNA vaccines	50.6	49.4	16–91
Sadoff 12	2021	NCT04436276	1‐2a	Ad26.COV2.S	Adenovirus vaccines	48.6	51.4	18–88
Wu 13	2021	NCT04383574	I/II	CoronaVac	Inactivated vaccines	48.4	51.6	≥60
Xia 14	2020	ChiCTR2000031809	I/II	‐	Inactivated vaccines	37.3	62.7	18–59
Xia 15	2021	ChiCTR2000032459	I/II	BBIBP‐CorV	Inactivated vaccines	45.8	54.2	18–80
Zhang 16	2021	NCT04352608	I/II	CoronaVac	Inactivated vaccines	46.6	53.4	18–59
Zhu 17	2020	NCT04341389	II	adenovirus type‐5 (Ad5)‐vectored COVID‐19 vaccine	Adenovirus vaccines	50	50	18–55, ≥55

2.3. Statistical analysis

Based on the Review Manager 5.3 software, the odds ratio (OR) of the binary variable results was analyzed. When I ² < 50%, the fixed effects model was used, and when I ² > 50%, the random effects model was used. Q statistic test and I ² test were used to evaluate the heterogeneity between studies. Statistically, no heterogeneity was found with p value > 0.1, and vice versa.

3. RESULTS

3.1. Study selection

The flowchart for the study selection is shown in Figure 1. After searching the database, 91 different kinds of literature were identified, among which 25 were duplicated, 40 irrelevant ones were screened out according to the title and abstract content, 15 were excluded after full‐text reading, and 9 were obtained after the final screening. A total of 105 801 participants were included in 9 clinical trials. Participants were randomly assigned to the COVID‐19 vaccine group or the placebo group. These studies included more detailed post‐vaccination adverse events with a low risk of bias (Figure 2), so these data were included in this meta‐analysis. The included studies were randomized controlled trials, including two adenovirus vector vaccines, four inactivated virus vaccines, and three RNA vaccines.

Figure 1.

Flowchart of literature inclusion

Figure 2.

Bias risk assessment diagram. (A) Bias risk diagram and (B) summary chart of risk bias

3.2. Overall analysis

3.2.1. Adverse events after vaccination

The incidence of any adverse events after injection was higher in the vaccine group than in the placebo group (OR: 2.97, 95% confidence interval [CI]: 1.49–5.92, I ² = 100%) (Figure 3A). Among them, the incidence of local adverse events was higher in the vaccine group (OR: 6.07, 95% CI: 2.13–17.28, I ² = 96%); the incidence of systemic adverse events was lower than that of local adverse events (OR: 1.80, 95% CI: 1.02–3.17, I ² = 87%). Among the common adverse events after injection, the vaccine group had a higher incidence than the placebo group (Figure 3B), local pain after injection (OR: 4.36, 95% CI: 1.01–18.86, I ² = 97%), fatigue (OR: 3.06, 95% CI: 2.96–3.17, I ² = 0%), fever (OR: 2.95, 95% CI: 0.66–13.21, I ² = 94%), headache (OR: 2.46, 95% CI: 2.1–2.89, I ² = 4%), swelling (OR: 2.3, 95% CI: 0.3–17.81, I ² = 85%), and erythema (OR: 1.42, 95% CI: 0.24–8.52, I ² = 83%), the incidence increased in turn.

Figure 3.

Adverse events. (A) Overall and (B) common

3.3. Subgroup analysis

3.3.1. Inactivated virus vaccine

Inactivated virus vaccine destroys the nucleic acid components of the virus by physical or chemical methods to make it non‐infectious, retains the antigenic virus shell for the body to perform immune recognition, induces the body's immune response to produce antibodies that can neutralize the virus to prevent the development of disease caused by the virus when the individual is exposed to. The inactivated virus vaccines currently approved for marketing are mainly from China. Among them, CoronaVac of Sinovac Life Sciences selected SARS‐CoV‐2 (CN02) strain; BBIBP‐CorV developed by Beijing Institute of Biological Products Co., Ltd. used 19 nCoV‐CdC‐TAN‐HB02 strain (HB02) isolated from bronchoalveolar lavage samples or throat swabs of patients; and the SARS‐CoV‐2 (WIV04) strain was selected by Wuhan Institute of Biological Products Co., Ltd., they were all cultured and propagated in Vero cells, using β‐propiolactone for inactivation. Among the three, the former two selected the aluminum hydroxide adjuvant adsorption vaccine, while the latter selected the alum adjuvant adsorption vaccine.

Any adverse events were significantly higher in the inactivated virus vaccine group than in the placebo group (OR: 2.44, 95% CI: 0.76–7.87, I ² = 83%), and local adverse events were higher than those in the placebo group (OR: 1.22, 95% CI: 0.87–1.71, I ² = 0%), systemic adverse events were less different from those in the placebo group (OR: 0.92, 95% CI: 0.59–1.41, I ² = 0%) (Figure 4A). Common adverse events after vaccine injection include mild to moderate pain at the injection site, fever, fatigue, and headache. Among them, local pain (OR: 1.63, 95% CI: 0.77–3.43, I ² = 53%), fatigue (OR: 1.52, 95% CI: 0.56–4.08, I ² = 0%), and fever (OR: 1.32, 95% CI: 0.57–3.09, I ² = 0%) tended to occur more frequently in the vaccine group, while headache (OR: 0.83, 95% CI: 0.2–3.4, I ² = 0%), local swelling (OR: 0.69, 95% CI: 0.19–2.5, I ² = 0%), and erythema (OR: 0.62, 95% CI: 0.16–2.51, I ² = 0%) tended to occur more frequently in the placebo group (Figure 4B). It is noteworthy that the OR value of headache in the three studies is >1, only one OR < 1, and the overall OR < 1.

Figure 4.

Adverse events of inactivated virus vaccines. (A) Overall, (B) common, and (C) dose

Inactivated virus vaccine adverse events are dose‐dependent. Compared with the medium dose (OR: 1.1, 95% CI: 0.7–1.73, I ² = 0%), low dose (OR: 1.33, 95% CI: 0.74–2.4, I ² = 0%) and high dose (OR: 1.32, 95% CI: 0.85–2.06, I ² = 0%) had a higher incidence of adverse events (Figure 4C).

Phase II results of the CoronaVac vaccine of Sinovac Life Sciences showed that 28 days after vaccination, the GMT of the medium‐dose group was 1783.6 (1519.3, 2093.8) (p < 0.0001), high‐dose group was 2287.5 (2038.2, 2567.3) (p < 0.0001), and placebo group was 87.9 (79.7, 96.9) (p < 0.0001), respectively. Phase II clinical data of the BBIBP‐CorV vaccine from the Beijing Institute of Biological Products showed that the GMT of the middle‐dose group was 218 (181.8–261.3) on the 28th day after vaccination. The Phase II clinical data of the Wuhan Institute of Biological Products showed that the GMT on the 21st day of vaccination in the middle‐dose group was 247 (95% CI, 176–345).

3.3.2. RNA vaccine

After RNA vaccine is injected into host cells, the encoded mRNA is first translated into viral proteins in the ribosome. The translated viral proteins are recognized as antigens and cause the immune response of the body to achieve the effect of disease prevention. Currently, clinically completed Phase III RNA vaccines include Moderna's mRNA‐1273 and Pfizer's BNT162b2. Among them, the mRNA‐1273 vaccine encodes the full‐length pre‐fused stable spike protein of SARS‐CoV‐2; the BNT162b1 vaccine encodes the trimeric receptor‐binding domain (RBD) of the Spike protein of SARS‐CoV‐2; BNT162b2 vaccine encodes a full‐length, membrane‐anchored SARS‐CoV‐2 spike protein that is stable before fusion. All three were mRNA‐based vaccines encapsulated by lipid nanoparticles.

The RNA vaccine group caused much more adverse events than the placebo group (OR: 5.84, 95% CI: 1.67–20.41), severe adverse events were slightly higher in the placebo group (OR: 1.14, 95% CI: 0.88–1.47) (Figure 5A). There was only one paper with detailed statistics on local adverse events, systemic adverse events, and common adverse events (Figure 5B), so further analysis could not be made. The occurrence of adverse reactions to RNA vaccines is dose‐dependent. The incidence of adverse events with RNA vaccines at medium doses (OR: 2.62, 95% CI: 2.49–2.76, I ² = 0%) was lower than the low dose (OR: 8, 95% CI: 0.75–85.31) and high dose (OR: 10.76, 95% CI: 10.3–11.25, I ² = 0%) (Figure 5C).

Figure 5.

Adverse events of RNA vaccines. (A) Overall, (B) common, and (C) dose

Moderna's mRNA‐1273 vaccine showed 94.1% (95% CI: 89.3–96.8%, p < 0.001) efficacy in a phase III clinical trial; Phase III clinical trial results of Pfizer's BNT162b2 vaccine showed that the vaccine was 95% effective (95% CI: 90.3–97.6).

3.3.3. Adenovirus vector vaccine

The principle of the viral vector vaccine is similar to that of an attenuated vaccine. Genes encoding pathogen antigens are cloned into nonreplicating virus vectors, such as adenovirus. The antigens are produced by transduced host cells after immunization. The advantage of vectored vaccine is that it does not replicate in host cells, thereby reducing safety concerns as those for live vaccines. Johnson & Johnson's Ad26.COV2.S is a recombinant, non‐replicating adenovirus serotype 26 (Ad26) vector encoding a full‐length and stable spike protein called SARS‐CoV‐2. CanSino's Ad5 vector COVID‐19 vaccine contains a replication‐deficient Ad5 vector, which is based on Wuhan‐Hu‐1 expressing a full‐length spike gene.

The any adverse events were significantly higher in the vaccine group than in the placebo group (OR: 5.64, 95% CI: 4.09–7.77, I ² = 22%), and local adverse events were higher than those in the placebo group (OR: 13.89, 95% CI: 6.64–29.05, I ² = 71%) (Figure 6A). There was only one paper with detailed statistics on common adverse events (Figure 6B), so further analysis could not be made. The occurrence of vaccine adverse events is dose‐dependent. On the whole, the high‐dose adenovirus vector vaccine group (OR: 5.1, 95% CI: 3.68–7.06, I ² = 0%) has a lower incidence of adverse events than the low‐dose group (OR: 6.31, 95% CI: 3.35–11.89, I ² = 76%) (Figure 6C). However, it is interesting that the adverse events in the low‐dose group were significantly stronger than those in the high‐dose group in Sadoff's study. And in Zhu's study, the adverse events of the low‐dose group were slightly lower than that of the high‐dose group.

Figure 6.

Adverse events of adenovirus vector vaccines. (A) Overall, (B) common, and (C) dose

Phase II clinical data from Johnson & Johnson's Ad26.COV2.S vaccine showed geometric mean titers (GMT) of binding antibodies measured at 29 days after the first dose ranged from 224 to 354, rising to 827 to 1266 after the second dose. Phase II data from CanSino's Ad5‐vector COVID‐19 vaccine showed a GMT of 656.5 (95% CI: 575.2–749.2) on day 28 after inoculation in the low dose group and GMT of 571.0 (95% CI: 467.6–697.3) in the high‐dose group.

4. DISCUSSION

Vaccination is an effective measure to prevent and control the COVID‐19 pandemic. Under the urgent need to fight against COVID‐19, researchers from various countries are developing a variety of SARS‐CoV‐2 vaccines, including the inactivated virus vaccine, nucleic acid vaccine, adenovirus vector vaccine, and viral subunit vaccine. Some of the inactivated virus vaccines, RNA vaccines, and adenovirus vaccines have received conditional marketing authorization through phase III clinical trials. Different types of vaccines have different advantages and limitations.

By means of physical or chemical methods, the inactivated virus loses its virulence to the original target organ and retains the immunogenicity of its corresponding antigen. Inactivated virus vaccines have stable conformation‐dependent epitope expression and are easy to mass‐produce. ¹⁸ Inactivated viruses are commonly used in the development of traditional vaccines, such as influenza vaccine and pertussis vaccine. Experience has proved that the inactivated virus vaccine has good preventive efficacy, good stability, and easy storage and transportation. However, since the inactivated virus vaccine only induced antibodies, the immune effect is not ideal. The inactivated virus vaccine often requires multiple doses to maintain serum antibody levels. According to the results of the meta‐analysis of the adverse events of the inactivated virus, the incidence of the vaccine group and the placebo group are similar. The adverse events are mostly pain, fatigue, and fever. According to reports, the adverse events can alleviate spontaneously within 7 days. The result shows that the incidence of headache in the placebo group is higher than the vaccine group should cause concern. In the four clinical studies on inactivated virus vaccines, there were three with OR > 1, only one (Xia 2020) OR < 1, and the final overall result was OR < 1. Due to the limited sample size (n = 4), the results are easily affected by extreme values. This shows that when concluding in the case of limited sample size, not only the overall analysis results should be considered, but also the local influence on the overall results should be considered.

The RNA vaccine has received widespread attention in the development of the COVID‐19 vaccine. The process includes screening and optimizing the mRNA sequence encoded by the core antigen from the target pathogen, selecting the appropriate carrier material to encapsulate the mRNA, and delivering the vaccine into the body with a suitable administration method. After the RNA vaccine reaches the host cell, it first translates the encoded mRNA into the modified target antigen in the ribosome and then triggers the host immune response. ¹⁹ RNA vaccine is currently a relatively new vaccine development technology. Its advantage lies in its high versatility, rapid adaptation, and production when pathogens appear. And because RNA vaccine can induce both antibody and cytotoxic T cell response, its efficacy is stronger than conventional inactivated virus vaccines. RNA vaccines must first complete the translation in the host cell, and then the immune response takes place. Two steps are required, and the differences in immune responses between individuals may be amplified. In addition, RNA vaccines have poor stability and require stringent conditions for large‐scale production and storage.

Adenovirus vaccines use adenovirus as a carrier to recombine the modified target antigen gene into the adenovirus genome to express the target antigen and induce humoral and cellular immune responses. ²⁰ However, its disadvantage is that both the adenovirus vector and the target antigen are antigens to the human body, and the body will produce an immune response to the adenovirus vector, which may affect the expected immune effect. The results of Section 2.3.3 showed that adverse events were dose‐related in both Sadoff and Zhu studies, but their results were the opposite. This may be related to the difference in the number of trials included in the vaccine group between the two studies. For different doses of vaccine groups, the number of people in the placebo group was the same. In Sadoff's study, the number of people in the low‐dose group was similar to that in the high‐dose group, while the number of people in Zhu's study was about twice that in the low‐dose group. The difference in experimental design between the two studies led to the opposite result. As a result, the adverse events of Ad26.COV2.S vaccine reported by Sadoff in the low‐dose group (OR: 8.68) and high‐dose group (OR: 5.12) were higher than those reported by Zhu in adenovirus type‐5 (Ad5)‐vectored COVID‐19 vaccine, low‐dose group (OR: 4.55) and high‐dose group (OR: 5.06). Moreover, adenovirus vaccines have the strongest adverse reactions among the three types of vaccines.

From the point of view of effectiveness, RNA vaccine > adenovirus vector vaccine > inactivated virus vaccine; from the perspective of safety, the incidence of adverse events of the three vaccines is higher than that of a placebo, and the incidence of adverse events of the adenovirus vector vaccine is higher. Moreover, this study has certain limitations. This article does not include randomized clinical trials of non‐placebo‐controlled vaccines. The number of samples included in the clinical trials varies greatly. The age range and ethnicity of the volunteers involved in the clinical trials are not comprehensive enough. It is difficult to properly judge whether the vaccine is suitable for people of different races and ages around the world based on the existing results. The time point for testing the effectiveness of the vaccine is relatively short, and it is temporarily impossible to know how long the effect of the vaccine against SARS‐CoV‐2 can last.

CONFLICT OF INTERESTS

The authors declare that there are no conflict of interests.

Ling Y, Zhong J, Luo J. Safety and effectiveness of SARS‐CoV‐2 vaccines: A systematic review and meta‐analysis. J Med Virol. 2021;93:6486‐6495. 10.1002/jmv.27203