An umbrella review of Lianhua Qingwen combined with Western medicine for the treatment of coronavirus disease 2019

Objective:

Lianhua Qingwen combined with Western medicine (LHQW+WM) has been proposed as a viable treatment for coronavirus disease 2019 (COVID-19). Interestingly, umbrella reviews of systematic reviews (SRs), which provide the most comprehensive evidence, are the best evidence in evidence-based medicine. Therefore, an umbrella review of SRs that summarizes and evaluates the efficacy of LHQW+WM for COVID-19 is urgently required.

Methods:

Overall, 6 databases were used to conduct a comprehensive literature search from inception to January 22, 2022. The corrected covered area (CCA) was used to analyze the overlapping between SRs. Meta-analysis was conducted when that of the included SRs was inappropriate. A MeaSurement Tool to Assess Systematic Reviews (AMSTAR-2) was also employed to assess the quality of the included SRs.

Results:

In total, 12 SRs were identified, which included 12 unique primary studies. The included SRs ranged in quality from moderate to critically low and had an extremely high CCA (36.4%). Compared to conventional treatment, LHQW+WM showed efficacy concerning fatigue recovery [risk ratio (RR) = 1.69, 95% confidence interval (CI): 1.04–2.73, n = 2, I² = 0%], cough recovery (RR = 1.65, 95% CI: 1.09–2.51, n = 3, I² = 39.1%), and overall effective rates (RR = 1.17, 95% CI: 1.07–1.28, n = 3, I² = 17.5%).

Conclusion:

LHQW+WM may improve the clinical symptoms of patients with COVID-19; however, the results should be interpreted cautiously because of the rigorous processes in the included SRs.

Graphical abstract:

http://links.lww.com/AHM/A32.

Introduction

According to the World Health Organization, it has been more than two and a half years since the coronavirus disease 2019 (COVID-19) was first reported in December 2019, causing 529 million confirmed cases, including 6 million deaths^[1]. The long-term epidemic has severely impacted socioeconomic activities and prevented access to available medical treatment for the saturation of hospital beds^[2–5].

Unfortunately, no antiviral drugs have been officially proven effective in treating patients with COVID-19. However, patients with asymptomatic or mild illnesses, including antipyretics, analgesics, and anti-tussives, could benefit from supportive care and symptomatic treatment. Moreover, some patients may deteriorate rapidly, and the early use of antiviral and monoclonal antibodies improves patient outcomes^[6–7]. However, the United States Food and Drug Administration (FDA) has approved only two antiviral medications for emergency use: nirmatrelvir with ritonavir (Paxlovid) and remdesivir (Veklury)^[8]. Therefore, further investigation into any therapy that may improve clinical outcomes is warranted.

Traditional Chinese medicine (TCM) is one of the essential treatments applied to 92.4% of confirmed patients in China. It has effectively prevented the transition into severe and critical states from mild and moderate states and improved critical patients’ conditions in multiple aspects^[9–13]. Notably, Lianhua Qingwen (LHQW) is a typical TCM formulation recognized and validated to show therapeutic effects through clinical research and observation^[14]. Furthermore, systematic reviews (SRs) are the source of high-quality evidence that can provide appropriate clinical decision-making conclusions^[15]. Despite the growing literature reports on treating COVID-19 with LHQW, the evidence from these SRs has not been systematically assessed. Therefore, umbrella reviews can provide comprehensive evidence at one of the highest levels available^[16], and identify potential gaps in the evidence, thus informing which issues new SRs should prioritize.

This umbrella review aims to evaluate the efficacy of LHQW + Western medicine (LHQW+WM) for COVID-19 and conduct a detailed review of recent SRs, which identifies the known and unknown, to provide clinical decision-makers and researchers with a consolidated source of currently available studies.

Method

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed during this umbrella review^[17].

Inclusion and exclusion criteria

SRs that fulfilled the following criteria were included: (1) patients with confirmed or suspected cases of COVID-19; (2) intervention: LHQW or LHQW combined with other drugs; and (3) study type: SRs that used explicit, systematic techniques to identify, appraise, and synthesize all evidence that fulfills the pre-specified eligibility criteria in answering a specific research question^[18]. Additionally, guidelines, narrative reviews, and SRs that did not focus on LHQW were excluded.

Search strategy

A comprehensive literature search was conducted using the following databases: China National Knowledge Infrastructure (CNKI), Chinese Biomedical Literature Database (CBM), WanFang database (WF), Chinese Scientific Journal Database (VIP), PubMed, and Embase from inception to January 22, 2022. We used search terms regarding SARS-CoV-2, COVID-19, Lianhua Qingwen, meta-analysis, and systematic review. Furthermore, we manually searched the reference lists of the included reviews to identify additional SRs.

Study selection and data extraction

Two reviewers (Zhao L and Cheng LY) independently screened the titles and abstracts of the records after removing duplicates using Endnote X8 [Thomson Reuters (Scientific) LLC Philadelphia, PA, USA] software and also performed full-text screening to identify eligible SRs. Therefore, the following data were extracted: first author, year of publication, patients, mean age, sample size, search details, and meta-analysis results. All disagreements were resolved through discussion in all research teams.

Quality assessment

The risk of bias of SRs was assessed using the A Measurement Tool to Assess Systematic Reviews checklist^[19], comprising 16 items, of which 7 are critical items in 10 domains. One of the following options was answered for each item: “Yes” “Partial Yes” “No” or “No meta-analysis conducted”. The overall confidence can be categorized into four levels based on critical items as follows: “High” “Moderate” “Low” and “Critically low”. Furthermore, two reviewers (Li YY and Kang YC) conducted quality assessments independently, and all disagreements were resolved by discussion.

Overlapping assessment

The degree of overlap between SRs was assessed by calculating the corrected covered area (CCA)^[20], and a detailed five-step procedure^[21] was taken to discuss the evidence more thoroughly and identify critical defects. CCA is calculated as (N − r)/(rc − r), where N is the number of publications included in the evidence synthesis, r is the number of rows (the number of index publications), and c is the number of columns (the number of reviews). Furthermore, the degree of overlap was classified as follows: very high (CCA, > 15%), high (CCA, 11%–15%), moderate (CCA, 6%–10%), or slight (CCA, 0%–5%).

Data synthesis

First, a narrative synthesis of the fundamental characteristics and findings of the included SRs were performed. Next, descriptive statistical analyses, such as frequencies and percentages and mean and standard deviation, were used to present the study results and quality assessments. Meta-analysis was also conducted when the meta-analysis of included SRs was inappropriate or when missing primary studies were found. Stata 14.0 was employed using a random-effects model to estimate the difference between the LHQW+WM and control groups for eligible studies. Statistical significance was considered at P <0.05. The pooled risk ratio (RR) or odds ratio (OR) and the mean difference (MD), both with a 95% confidence interval (CI), were used for the dichotomous and continuous variables, respectively. Finally, I² statistics were used to assess heterogeneity, which was graded as low, moderate, and high at 25%, 50%, and 75%, respectively.

Results

Literature selection

Overall, 108 records were retrieved from the databases, and 53 duplicates were removed. Finally, we included 12 SRs^[22–33] after the title/abstract and full-text screenings. In addition, 3 studies were excluded from the full-text screening since they did not focus on the treatment effectiveness of LHQW+WM in patients with COVID-19. Figure 1 illustrates the literature selection process.

Figure 1.

PRISMA Flow diagram for searched, screened, and included studies. CNKI: China National Knowledge Infrastructure;WF: WanFang Database;CBM: Chinese Biomedical Literature Database; VIP: Chinese Scientific Journal Database.

Basic characteristics of included SRs

Half of the included SRs were published in Chinese^[28–33] (n = 6), and the others were all in English^[22–27] (n = 6) from 2020 to 2021. The number of included primary studies using SRs varied from 2 to 8, with sample sizes ranging from 154 to 924. Among the 12 SRs, 5 SRs focused on common or mild/moderate COVID-19^{[23,25,28–29,33]}. Regarding LHQW dose, granules and capsules were the main dosages of LHQW described in the included SRs, with LHQW decoction also found in one SR^[27]. Additionally, the control groups in the SRs received WM as conventional care. The meta-analysis of the included SRs yielded 26 different outcomes, with fatigue recovery rate (n = 9), cough recovery rate (n = 9), overall effective rate (n = 8), fever recovery rate (n = 8), aggravation rate (n = 7), and duration of fever (n = 7) recorded in more than half of the included SRs. Regarding the risk of bias assessment, excluding Qi et al^[28]. that failed to report the risk of bias assessment tool they used in the article, Liu et al^[22]. and Wang et al^[26]. used the Newcastle-Ottawa Scale (NOS) and Cochrane risk of bias tool. In contrast, others used only the Cochrane risk of bias tool. Further details can be found in Table 1.

Table 1.

Basic characteristics of included systematic reviews

Systematic reviews	Patients	Intervention	Control	Diagnostic criteria	Included study design	Number of the Included studies (sample size)	Risk of bias tool	Outcomes
Liu et al[22].	COVID-19	LHQW granules/capsules+WM	WM	Laboratory test	RCTs; CCSs; Case series	8 (924)	Cochrane ROB; NOS	1– 3
Zhuang et al[25].	Common COVID-19 including suspected cases	LHQW granules/capsules+WM	WM	NCP-D and T program (4th edition and 5th edition)	RCTs; Case-control study	3 (245)	Cochrane ROB	3–9
Wang et al[26].	COVID-19	LHQW granules/capsules+WM	WM	NCP-D and T program (7th edition)	RCTs; Retrospective study	6 (856)	Cochrane ROB; NOS	1, 4–7, 10, 11
Fan et al[23].	Mild/Moderate COVID-19 including suspected cases and excluding severe cases	LHQW granules/capsules+WM	WM	Laboratory tests	RCTs, RTs	5 (824)	Cochrane ROB	1–3, 11
Zeng et al[27].	COVID-19	LHQW capsules/granules/decoctions+WM	WM	NCP-D and T program (7th edition)	RCTs	2 (154)	Cochrane ROB	4–10, 12–16
Hu et al[24].	COVID-19	LHQW granules/capsules+WM	WM	Laboratory tests	RCTs	7 (942)	Cochrane ROB	1, 3, 5–10,13–22
Tang et al[31].	COVID-19	LHQW granules/capsules+WM	WM	NCP-D and T program	RCTs	5 (824)	Cochrane ROB	1–4, 6–7, 23–24
Zhang et al[29].	COVID-19, excluding severe cases	LHQW granules/capsules+WM	WM	NCP-D and T program	RCTs	5 (600)	Cochrane ROB	1, 4–7, 10–11
Zhang et al[32].	COVID-19 including suspected cases	LHQW granules/capsules+WM	WM	NCP-D and T program	RCTs	5 (597)	Cochrane ROB	3, 5–7
Yang et al[33].	COVID-19, excluding severe cases	LHQW granules/capsules+WM	WM	NCP-D and T program	RCTs; Before-after study in the same patient	3 (138)	NR	5–8,14,16
Wang et al[30].	COVID-19	LHQW granules/capsules+WM	WM	NCP-D and T program	RCTs	7 (665)	Cochrane ROB	1–4, 25–26
Qi et al[28].	COVID-19, excluding severe cases	LHQW granules+WM	WM	NCP-D and T program	RCTs; Single arm studies	4 (181)	Cochrane ROB	1, 5–7, 9, 14, 16

1: overall effective rate; 2: Computed Tomography (CT) improvement rate; 3: aggravation rate; 4: duration of fever; 5: cough recovery rate; 6: fever recovery rate; 7: fatigue recovery rate; 8: disappearance rate of anhelation; 9: disappearance rate of expectoration; 10: shortness of breath recovery rate; 11: adverse reaction rate; 12: Disappearance rate of other symptoms; 13: improvement rate of muscle pain; 14: improvement rate of chest tightness; 15: improvement rate of nausea; 16: improvement rate of loss of appetite; 17: flu-like symptoms; 18: improvement rate of sore throat; 19: improvement rate of diarrhea; 20: improvement rate of headache; 21: improvement rate of pulmonary imaging; 22: healing period; 23: duration of cough recovery; 24: duration of fatigue recovery; 25: duration of clinical symptom recovery; 26: duration of hospitalization; CCS: Case-control study; COVID-19: Coronavirus disease 2019; LHQW: Lianhua Qingwen; NCP-D and T program: New Coronavirus Pneumonia Diagnosis and Treatment Program; NOS: Newcastle-Ottawa Scale; RCT: Randomized controlled trial; ROB: Risk of bias; WM: Western medicine (including antibacterial, antiviral, hormone therapy, and respiratory support, among others).

Quality assessment results

Two SRs were rated as moderate in quality^[22,25], 3 SRs were low in quality^[24,26,30], and the remaining 7 were critically low in quality^{[23,27–29,31–33]}. Most of the critically low-quality SRs did not meet 2 of the 7 domains considered critical: they did not state that the review methods were established before conducting the review; they used inappropriate methods for the statistical combination of results; they did not discuss or interpret the potential impact of risk of bias on the results. Moreover, none of the included SRs assessed the possible impact of the risk of bias of individual primary studies on the meta-analysis results or explained the study design selection. Table 2 presents the details of the quality assessment.

Table 2.

Quality assessment of included systematic reviews

Systematic reviews	I1	I2	I3	I4	I5	I6	I7	I8	I9	I10	I11	I12	I13	I14	I15	I16	Overall
Liu et al[22].	Yes	Yes	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	Yes	Yes	Yes	Yes	Moderate
Zhuang et al[25].	Yes	Yes	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	Yes	Yes	Yes	Yes	Moderate
Wang et al[26].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	Yes	Yes	No	Yes	Low
Fan et al[23].	yes	No	No	Yes	Yes	Yes	Partial yes	Partial yes	Yes	No	No	No	Yes	No	Yes	Yes	Critical low
Zeng et al[27].	yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	No	No	Yes	No	Yes	Yes	Critical low
Hu et al[24].	yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	Yes	Yes	Yes	No	Low
Tang et al[31].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	No	No	Yes	No	Yes	No	Critical low
Zhang et al[29].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	No	Yes	No	No	Critical low
Zhang et al[32].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	No	Yes	Yes	No	Critical low
Yang et al[33].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	No	No	No	No	No	No	Yes	No	Critical low
Wang et al[30].	Yes	No	No	Partial yes	Yes	Yes	Partial yes	Partial yes	Yes	No	Yes	No	Yes	Yes	Yes	No	Low
Qi et al[28].	Yes	No	No	Partial yes	Yes	Yes	No	Partial yes	Yes	No	No	No	No	No	No	No	Critical low

I 1: Do the research questions and inclusion criteria for the review include components of the PICO? I 2: Did the review report explicitly state that the review methods were established before the review, and did the report justify any significant deviations from the protocol? I 3: Did the review authors describe their selection of study designs for inclusion in the review? I 4: Do review authors use a detailed literature search strategy? I 5: Did the review authors conduct study selection in duplicate? I 6: Did the review authors perform duplicate data extractions? I 7: Did the review authors provide a list of excluded studies and justify the exclusions? I 8: Did the authors describe the included studies in adequate detail? I 9: Did the review authors use a satisfactory technique to assess the ROB in individual studies included in the review? I 10: Did the review authors report the funding sources for the studies included in the review? I 11: When meta-analysis was conducted, did the review authors adopt appropriate methods for the statistical combination of results? I 12: When a meta-analysis was performed, did the review authors evaluate the potential effect of ROB in individual studies on the meta-analysis results or other evidence synthesis? I 13: Did the review authors consider ROB in individual studies when interpreting and discussing the review results? I 14: Did the review authors provide an adequate explanation for and discuss any heterogeneity observed in the review results? I 15: When they performed quantitative synthesis, did the review authors conduct an adequate investigation of publication bias (small study bias) and discuss its probable impact on the review results? I 16: Did the review authors report any potential sources of conflicts of interest, which includes any funding they received for conducting the review?

I: Item; PICO: participants; interventions; comparisons; outcomes; ROB: risk of bias.

Overlapping associations

In total, 12 primary studies comprising 1,132 patients were identified from the included SRs^[34–45]. Additionally, 10 of 12 studies restricted the patients with COVID-19 as common/mild/moderate COVID-19^{[34–37,39–42,44–45]}, excluding two studies that covered suspected^[38] and severe cases^[43]. Therefore, eight and four primary studies were retrospective^{[34–35,37–39,41,43–44]} and prospective^{[36,40,42,45]} studies, respectively.

The degree of overlap was calculated, and the CCAs for all patients were very high. For example, the CCA for all included SRs for SRs concerning common/mild/moderate COVID-19 and those with all COVID-19 cases were 36.4%, 37.5%, and 38.9%, respectively. Supplementary Table 1, http://links.lww.com/AHM/A25, presents the citation metrics. Additionally, for SRs with almost complete overlap of the primary studies, there was only one primary study^[37] in Wang et al^[26]. compared to Fan et al^[23]., and Lyu et al^[38]. was identified in Zhuang et al^[25]. but not in Zeng et al^[27].

Summary findings

Multiple sources of the high overlap were examined since the overall CCA was extremely high.

Definitions of topics

The research objectives of the included SRs aiming to assess the efficacy and safety of LHQW+WM in treating COVID-19 were similar. The similarity in the inclusion criteria was examined across the included SRs. Besides the information on population and intervention described in the basic characteristics, four, two, and one SRs included adult patients only^{[23,26,28,32]}, suspected cases^[23,25], and only Chinese patients^[27], respectively. Furthermore, the study types also varied among the included SRs, with three SRs including randomized controlled trials (RCTs) only^[36,40,45] and the remaining reviews containing other study types such as retrospective and non-randomized studies.

Search and study selection strategies

Overall, 8 databases were identified, of which 3 were retrieved from all included SRs (CNKI, WF, and Medline via PubMed or Ovid). Additionally, six, three, two, and one SRs searched seven^{[23–25,27,31,33]}, six^[22,26,30], five^[29,32], and four^[28] databases, respectively. The search terms were mainly associated with COVID-19 and LHQW; however, a clear difference was observed in the number of synonyms used in each SR. We retrieved search items based on the search items listed in the Methods section of the manuscript because search strategies were only supplied in three SRs^[23,27,33]. Furthermore, 3 SRs applied only 2 items about COVID-19 as search terms in the Chinese and English databases^[28,29,32]. Comparatively, the search strategy of Fan et al^[23]. comprised 8 items concerning COVID-19. Supplementary Table 2, http://links.lww.com/AHM/A26, presents additional search details.

Outcomes and results

All SRs obtained similar conclusions as an effective and promising therapy regarding the efficacy and safety of LHQW+WM in treating COVID-19. Among the 26 outcomes, two related to fever, fatigue, and cough were involved in fever/fatigue/cough recovery rate and the duration of fever/fatigue/cough.

Therefore, we extracted the primary studies that generated combined results in a meta-analysis on fatigue recovery, cough recovery, and overall effective rates, which were the most overlapping outcomes encompassing eight primary studies, to explain the results on different outcomes. Generally, the meta-analysis pooled results showed the therapeutic efficacy of LHQW+WM, despite the various effect sizes or different included primary studies.

For fatigue and cough recovery rates, Wang et al^[41]. and Cheng et al. (54 cases)^[35] did not meet the inclusion criteria of most meta-analyses since they were before and after the studies. Furthermore, Sun et al^[40]. was not found in Wang et al^[26]., Zeng et al^[27]., and Tang et al^[31]., considering the publication date of the primary studies and SRs. In addition, several errors were observed in the effect size retrieved from each meta-analysis; for fatigue recovery rate, incorrect data on Cheng et al. (51 cases)^[34] were detected in 3 SRs about fatigue recovery rate^[25,27,29] and one regarding cough recovery rate^[26], and incorrect data from Yao et al^[44] .were found in one SR for both outcomes^[31]. Tables 3 and 4 present the additional details.

Table 3.

Outcome comparison of fatigue recovery rate

Primary studies	Systematic reviews
	Zhuang et al[25].	Wang et al[26].	Zeng et al[27].	Tang et al[31].	Zhang et al[29].	Zhang et al[32].	Yang et al[33].	Qi et al[28].
Effect measure	RR [95% CI]	OR [95% CI]	OR [95% CI]	RR [95% CI]	OR [95% CI]	RR [95% CI]	RR [95% CI]	RD [95% CI]
Synthesis model	Fix	Fix	Fix	Fix	Fix	Fix	Fix	Random
I2 (%)	0	0	0	0	0	0	0	68
Software	RevMan	RevMan	RevMan	RevMan	RevMan	RevMan	Stata	RevMan
Result (favor)	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW
Cheng et al[34].	1.73 [1.06, 2.82]	3.03 [1.11, 8.29]	2.93 [1.15, 7.45]	1.73 [1.01, 2.97]	Not estimable	1.79 [1.04, 3.06]	Not published	0.61 [0.44, 0.78]
Yao et al[44].	1.35 [0.47, 3.89]	1.61 [0.31, 8.32]	1.61 [0.31, 8.32]	8.40 [1.16, 60.84]	1.61 [0.31, 8.32]	1.35 [0.47, 3.89]	Unable to extract	0.42 [0.14, 0.70]
Wang et al[41].	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	0.85 [0.71, 0.99]
Cheng et al[35].	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	Unable to extract	0.76 [0.62, 0.90]
Lyu et al[38].	1.41 [1.00, 1.97]	3.33 [1.11, 10.0]	DMIC	1.41 [1.00, 1.97]	3.33 [1.11, 10.0]	1.41 [1.00, 1.97]	Unable to extract	DMIC
Sun et al[40].	Not published	Missing	Missing	Missing	Not published	1.25 [0.90, 1.75]	Not published	Not published
Overall	1.52 [1.15, 2.01]	2.82 [1.44, 5.53]	2.53 [1.13, 5.68]	1.72 [1.28, 2.32]	2.65 [1.07, 6.58]	1.46 [1.16, 1.86]	1.77 [1.36, 2.30]	0.69 [0.54, 0.84]

RR: Risk ratio; OR: Odds ratio; RD: Risk difference; CI: confidence interval; LHQW: Lianhua Qingwen; DMIC: Did not meet inclusion criteria.

Table 4.

Outcome comparison of cough recovery rate

Primary studies	Systematic reviews
	Zhuang et al[25].	Wang et al[26].	Zeng et al[27].	Tang et al[31].	Zhang et al[29].	Zhang et al[32].	Yang et al[33].	Qi et al[28].
Effect size	RR [95% CI]	OR [95% CI]	OR [95% CI]	RR [95% CI]	OR [95% CI]	RR [95% CI]	RR [95% CI]	RD [95% CI]
Synthesis model	Fix	Fix	Fix	Fix	Fix	Fix	Fix	Random
I2 (%)	18	0	37.9	0	0	30	24	77
Software	RevMan	RevMan	RevMan	RevMan	RevMan	RevMan	Stata	RevMan
Result (favor)	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW
Cheng et al[34].	1.73 [1.06, 2.82]	2.93 [1.15, 7.45]	14.88 [1.56, 142.2]	1.73 [1.06, 2.82]	3.03 [1.11, 8.29]	1.73 [1.06, 2.82]	Not published	0.62 [0.47, 0.78]
Yao et al[44].	8.40 [1.16, 60.84]	14.88 [1.56, 142.2]	3.03 [1.11, 8.29]	1.35 [0.47, 3.89]	14.88 [1.56, 142.2]	8.40 [1.16, 60.84]	Unable to extract	0.47 [0.21, 0.72]
Wang et al[41].	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	0.72 [0.60, 0.85]
Cheng et al[35].	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	Unable to extract	0.77 [0.62, 0.92]
Lyu et al[38].	1.82 [1.05, 3.14]	2.84 [1.17, 6.91]	DMIC	1.82 [1.05, 3.14]	2.84 [1.17, 6.91]	1.82 [1.05, 3.14]	Unable to extract	DMIC
Sun et al[40].	Not published	Missing	Missing	Missing	Not published	1.42 [1.03, 1.94]	Not published	Not published
Overall	1.99 [1.39, 2.86]	3.39 [1.85, 6.23]	4.22 [1.73, 10.26]	1.72 [1.22, 2.43]	3.46 [1.85, 6.49]	1.77 [1.36, 2.28]	1.96 [1.43, 2.68]	0.67 [0.57, 0.78

RR: Risk ratio;OR: Odds ratio; RD: Risk difference; CI: confidence interval; LHQW: Lianhua Qingwen;DMIC:Did not meet inclusion criteria.

For the overall efficacy rate, 3 primary studies were erroneously included in a meta-analysis^[34,38,42]. Wang et al^[42]. aimed to investigate the efficacy of lopinavir/ritonavir, but not LHQW, which was included in 2 SRs^[29–30]. Cheng et al. (51 cases)^[34] was a retrospective study mistakenly included in four SRs^{[23,28,29,31]}. However, Lyu et al^[38]. Should not have been included in Wang et al^[26]. because it contains suspected cases. Notably, Shi et al^[39]. and Xia et al^[43]. were not detected in Wang et al^[26]. Because of the ambiguity and inconsistent definition of the overall effective rate, Liu et al^[22]. retrieved the symptom recovery rate on day 14 from Hu et al^[36]., whereas others extracted the overall rate of recovery on day 14. Compared with the overall recovery rate definition, the symptom recovery rate on day 14 from Hu et al^[36]. Should be adopted, which was clearly defined and was also the primary endpoint of this study. Further details are presented in Table 5.

Table 5.

Outcome comparison of overall effective rate

Primary studies	Systematic reviews
	Liu et al[22].	Wang et al[26].	Fan et al[23].	Hu et al[24].	Tang et al[31].	Zhang et al[29].	Wang et al[30].	Qi et al[28].
Effect size	RR[95% CI]	OR[95% CI]	OR[95% CI]	OR[95% CI]	RR[95% CI]	OR[95% CI]	RR[95% CI]	RD[95% CI]
Synthesis model	Random	Fixed	Fixed	Fixed	Fixed	Fixed	Fixed	Random
I2 (%)	46	0	0	29.8	0	0	0	77
Software	RevMan	RevMan	RevMan	Stata	RevMan	RevMan	Stata	RevMan
Result (favor)	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW	LHQW
Yu et al[45].	1.24 [1.08, 1.43]	2.30 [1.35, 3.92]	2.30 [1.35, 3.92]	2.48 [1.44, 4.26]	1.25 [1.08, 1.44]	2.30 [1.35, 3.32]	Unable to extract	Not published
Hu et al[36].	1.11 [1.01, 1.22]	2.31 [1.11, 4.81]	2.31 [1.11, 4.81]	1.91 [1.12, 3.25]	1.19 [1.03, 1.38]	Not published	Not published	Not published
Wang et al[42].	–	–	–	–	–	3.76 [1.24, 11.38]	Unable to extract	–
Cheng et al[34].	Not included in the meta-analysis	2.87 [1.06, 7.76]	2.87 [1.06, 7.76]	4.10 [1.29, 13.09]	1.26 [1.01, 1.56]	2.87 [1.06, 7.76]	Unable to extract	0.80 [0.69, 0.91]
Wang et al[41].	Not included in the meta-analysis	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	0.93 [0.86, 1.00]
Cheng et al[35].	Not included in the meta-analysis	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC	0.74 [0.62, 0.86]
Lyu et al[38].	DMIC	3.76 [1.24, 11.38]	DMIC	DMIC	DMIC	DMIC	DMIC	DMIC
Shi et al[39].	DMIC	Missing	DMIC	DMIC	DMIC	DMIC	Unable to extract	DMIC
Xia et al[43].	DMIC	Missing	DMIC	10.18 [1.83, 56.54]	DMIC	DMIC	Unable to extract	DMIC
Overall	1.16 [1.04, 1.30]	2.51 [1.73, 3.64]	2.39 [1.61, 3.55]	2.49 [1.76, 3.53]	1.23 [1.12, 1.34]	2.59 [1.68, 3.98]	1.24 [1.12, 1.38]	0.83 [0.72, 0.95]

RR: Risk ratio;OR: Odds ratio; RD: Risk difference; CI: confidence interval; LHQW: Lianhua Qingwen; DMIC: Did not meet inclusion criteria.

Meta-analysis

The pooled results of all eligible studies showed significant differences in the three most overlapping outcomes. Compared with confirmed patients treated with conventional treatment, patients treated with LHQW combined with conventional treatment showed better efficacy concerning fatigue recovery rate (RR = 1.69, 95% CI: 1.04–2.73, n = 2, I² = 0%), cough recovery rate (RR = 1.65, 95% CI: 1.09–2.51, n = 3, I² = 39.1%), and overall effective rate (RR = 1.17, 95% CI: 1.07–1.28, n = 3, I² = 17.5%) (Figure 2).

Figure 2.

Results of meta-analysis on high overlapping outcomes. Right: favors LHQW+WM; Left: favors WM. LHQW+WM: Lianhua Qingwen + Western medicine; WM: Western medicine.

Discussion

This umbrella review summarizes the existing SRs concerning LHQW+WM therapy for COVID-19, combined with an in-depth quantity analysis, to provide an overview of current evidence in a single article. Therefore, an extremely high overlap was identified in 12 SRs that involved 12 unique primary studies, and the efficacy of LHQW+WM in treating COVID-19 was validated regarding fatigue recovery, cough recovery, and overall effective rates.

Additionally, high overlapping was inevitable considering the specific context of COVID-19. The CCA value was considered an index to measure the overlap among SRs, and a very high overlap was recorded across the included SRs. However, this high-level overlap was unavoidable since 12 primary studies were identified from the same number of SRs. Furthermore, despite the urgent need for evidence in treating COVID-19, it was clear that the number of SRs appeared extremely high since only 12 primary studies were identified. Therefore, conducting prospective studies in a minimal time frame was difficult, under the specific context of COVID-19, especially in the early period of the pandemic. However, multiple SRs under a particular research topic leads to redundancy and are also considered a waste of scientific resources^[46]. Here, registering and checking the registration website before conducting a systematic review can efficiently prevent this redundancy. Unfortunately, only two included SRs^[22,25] pre-registered the protocol with the International Prospective Register of Systematic Reviews (PROSPERO).

Furthermore, examining the overlapping sources showed that the inconsistency in the included primary studies regarding similar research topics of all included SRs may be due to the search and study selection. Moreover, studies involving severe or suspected cases and different study types were not integrated into the eligibility criteria of many included SRs, which could explain most of the inconsistency. However, studies that did not fulfill the inclusion criteria were erroneously included in the SRs, causing significant changes in the pooled results. The selection process that lacked scientific rigor would undoubtedly produce an undependable SR, a high probability of which would have been prevented if the actual “back-to-back” screening process was performed.

Regarding the most overlapping outcomes, none of the meta-analyses had a similar pooled result as another, even when similar primary studies were included; however, the same effect direction was found in all meta-analyses. The SRs recorded different estimates of the effect measures and/or their 95% CIs for comparable primary studies, as shown in the tables. After comparing the data in the primary studies, incorrect data extraction, different effect measures, and different programs used for data synthesis contributed to the inconsistency of the pooled results. Additionally, variation in the manner of reporting fever, fatigue, and cough was observed, which could indicate the existence of selective reporting or publication bias^[21,47]. Finally, non-comprehensive outcomes were reported, possibly because only outcomes with significant differences were reported.

We conducted a meta-analysis of the three most overlapping outcomes according to the accurate data and all eligible primary studies. Although significant differences were observed between the LHQW+WM and control groups, pooled results should be cautiously interpreted because of the low methodological quality of primary studies, which was a consistent conclusion among the included SRs.

A comprehensive comparison of current SRs may present not only multiple papers in one document but also provide practical directions and information for future research. Additionally, conducting comparisons based on a framework is more feasible and logical and could ascertain an extensive and exhaustive review. Moreover, we included SRs and also considered the primary individual studies.

This study had some limitations. First, the included SRs and primary studies in the SRs had moderate to low quality, possibly decreasing the certainty of the pooled results. Second, most of the primary studies were retrospective, thus may further weaken the confidence of the pooled results. Last, interpreting the pooled results in clinical practice still requires more caution, considering the combined intervention of the LHQW and different settings.

Conclusions

Our umbrella review indicated that LHQW+WM might improve the clinical symptoms of patients with COVID-19; however, the results should be interpreted cautiously. Additionally, the high overlap and inconsistency of existing SRs showed that more attention should be given to the rigorous process of conducting SRs and pre-registration.

Conflict of interest statement

The authors declare no conflict of interest.

Funding

None.

Author Contribution

Kelu Yang, Jiaoyan zhang, and Yingying Kang conceived and designed the manuscript idea. Kelu Yang and Jiaoyan zhang drafted and revised the manuscript. Kelu Yang and Jiaoyan zhang analyzed the data and interpreted the results of the analysis. Liang Zhao and Luying Cheng filtered the articles, the performed data extraction. Yuanyuan Li and Yuchen Kang performed the quality assessment. Xiangyu Zhang designed the search strategy. Yingying Kang provided a critical version of the manuscript. All authors have read and approved the final manuscript.

Ethical approval of studies and informed consent

Not applicable.

Acknowledgments

None.