Graphical abstract
Keywords: Herbal medicine, Traditional Chinese medicine, Systematic review, Meta-analysis, Evidence
Abbreviations: AMSTAR, Assessment Tool to Assess Systematic Studies; COVID-19, Coronavirus disease 2019; GRADE, Grading of Recommendations Assessment, Development and Evaluation; MeSH, Medical Subject Heading; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-analysis, RoB, risk of bias, RT–PCR, reverse transcription-polymerase chain reaction, SARS-CoV-2, severe acute respiratory syndrome coronavirus 2, SR, systematic review
Abstract
Background
As coronavirus disease 2019 (COVID-19) continues to spread throughout countries, researchers and scientific groups have published a large number of scientific papers examining effective treatments and prevention strategies for COVID-19, including herbal medicine. It has become difficult to navigate the increasing volume of scientific material on the pandemic, and critical appraisal of these outcomes is needed. This overview of systematic reviews (SRs) aims to synthesize evidence from SRs and summarize the effects of herbal medicine interventions in the treatment of COVID-19.
Methods
Four databases were searched from inception up to October 20, 2021. SRs analyzing primary studies of the efficacy of herbal medications for treating COVID-19 were included. Two reviewers selected the studies and retrieved the data independently. The AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) was used to assess the methodological quality of the included SRs.
Results
A total of 21 SRs on herbal medicine treatments for COVID-19 were included. All SRs were published between May 2020 and September 2021. Thirteen of the SRs included only randomized controlled trials (RCTs), whereas the remaining eight included evidence from nonrandomized trials in addition to RCTs, with a significant overlap identified across the RCTs. Twelve SRs concluded that existing evidence was insufficient to form a definite judgment, nine found that herbal therapy was useful, and none indicated that herbal medicine had no benefit. The AMSTAR 2 tool revealed that the methodological quality of the included SRs was generally low.
Conclusion
In this overview of SRs, we reviewed herbal medicine-related evidence from 21 SRs that were published after the outbreak of COVID-19. This study shows that while there is considerable evidence demonstrating the advantages of herbal medicine interventions, the quality of the evidence is inadequate to provide solid and accurate judgments about the effectiveness of herbal medicine therapies for COVID-19. Despite the crisis caused by the pandemic, clinical studies and SRs should comply with established methodological standards.
Introduction
During epidemics and pandemics, having access to credible sources of information is critical for informing public health measures. However, during the coronavirus disease 2019 (COVID-19) pandemic, an overabundance of information has hampered the execution of appropriate public health measures. At the beginning of 2020, the world was informed about the new disease, and by mid-March 2020, more than 2000 publications on COVID-19 had been published in scholarly journals (Fidahic et al., 2020). Therefore, more systematic reviews (SRs) on COVID-19 are anticipated to be published than reviews on any other disorders.
SRs play an important role in evidence-based decision-making processes, as they represent current scientific understanding by summarizing all available evidence on a specific issue. With such a large number of SRs published, overviews of SRs that assess their methodological quality are crucial (Hunt et al., 2018). Overviews are known by a diversity of names, each of which may represent distinct aspects and purposes of the syntheses (Smith et al., 2011). Some of the terminology used includes overview, umbrella review, meta-review, review of reviews, synthesis of SRs, and summary of SRs.
As COVID-19 continues to spread throughout countries, the use of herbal medicine in the management of COVID-19 began to gain interest and consideration from researchers and scientific groups (Dai et al., 2021; Lee et al., 2020; Shankar et al., 2020; Teo et al., 2021). Herbal medicine has been widely used in improving symptoms of COVID-19, improving laboratory indicators, and increasing the clinical cure rate. However, herbal medicine may cause certain adverse effects due to the complicated compositions of herbal medicine and possible herb/drug interactions among individuals, which requires a more thorough assessment. The evidence for the efficacy and safety of herbal treatments has also been thoroughly summarized in SRs (Ang et al., 2020; Li et al., 2021; Liu et al., 2021). However, the quality, emphasis, and intervention utilized in these SRs have impeded the accurate interpretation of data about the effect of herbal medicines.
This overview of SRs aimed to analyze and summarize herbal medicine-related SRs that were available for COVID-19, hoping to provide an objective and comprehensive evaluation of the efficacy and safety of herbal medicines in treating COVID-19.
Methods
Review registration
This overview of SRs was conducted according to the Cochrane Handbook for Systematic Reviews of Intervention (Higgins et al., 2019), and the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) 2020 statement was used to conduct this review (Page et al., 2021), as the Preferred Reporting Items for Overviews of Reviews (PRIOR) have yet to be fully developed (Hunt et al., 2018). This overview of SRs was registered in the Research Registry (unique ID: reviewregistry1250).
Data source and search strategy
A comprehensive search of the literature was performed from inception to October 20, 2021, including but not limited to the following databases: PubMed, Embase, Allied and Complementary Medicine Database, and Cochrane Register of Controlled Trials. The databases were searched using Medical Subject Headings (MeSH) terms and other relevant non-MeSH terms, including ‘severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)’, ‘COVID-19′, ‘herbal medicine’, and ‘traditional medicine’, along with the filter ‘systematic review’. No restrictions were imposed regarding the publication time or language.
Inclusion and exclusion criteria
The inclusion and exclusion criteria for the SRs were as follows:
Types of studies: All Cochrane SRs and non-Cochrane SRs of randomized controlled trials (RCTs) and non-RCTs that covered topics related to herbal medicine and COVID-19 were included, with or without meta-analyses. There were no restrictions on the design and methodology of the primary studies included within the SRs.
Participants: SRs evaluating patients diagnosed with COVID-19, independent of sex or ethnicity, were eligible. SRs that included people with disorders other than COVID-19 were not included.
Intervention: All herbal medicines used either alone or as an add-on therapy to standard treatment (integrative medicine) were included.
Comparator: No restrictions were imposed regarding the type of comparators.
Outcomes measured: SRs that reported on at least one outcome interest on clinical effectiveness and clinical symptoms were included. Safety outcomes were evaluated in this overview but were not a criterion for inclusion in SRs.
Study selection and data extraction
Two reviewers separately assessed the citations retrieved during the search, and full-text publications from potentially relevant SRs were retrieved and appraised for inclusion. One reviewer extracted the data using a standardized form. A second reviewer independently evaluated the retrieved data, and any differences were addressed through discussions with a third reviewer. The following data were extracted: authors, publication years, total sample size, outcome measures, SR conclusions, number and type of primary studies, type of herbal intervention, type of comparator, and quality assessment method.
Methodological quality assessment
The AMSTAR 2 (Assessment Tool to Assess Systematic Studies) was used to assess the methodological quality of the included SRs. The AMSTAR 2 has 16 domain-specific items. When all of the requirements for each item were met, the item received a “Yes” rating. If no information was provided or if one of the stated requirements was lacking, the item received a “No” rating without the benefit of the doubt. Items receiving “Partial Yes” indicated that partial adherence to the requirements was acceptable. Two reviewers applied the AMSTAR 2 instrument independently to all included SRs, and a third reviewer was consulted to resolve disagreements. As the AMSTAR 2 does not generate an overall ‘score,’ and several critical weaknesses of an SR may be disguised, a process of considered judgment was used to interpret the results obtained by the AMSTAR 2 until a consensus was reached on the overall quality assessment of each SR.
Data synthesis and analysis
The characteristics, direction of effects, and methodological quality of the included SRs were summarized in a table and narratively. The results of the herbal medicine intervention were also narratively summarized.
Results
Literature search
As of October 20, 2021, the literature search yielded 674 results. After removing duplicates and excluding papers based on title and abstract screening, 21 SRs (Ang et al., 2020; Du et al., 2021; Fan et al., 2020, 2021; Jiang et al., 2021; Li et al., 2021; Liang et al., 2021; Liu et al., 2021, 2020; Luo et al., 2021; Pang et al., 2020; Sun et al., 2020; Wang et al., 2021a, 2021b; Wu et al., 2021; Xiong et al., 2020; Yan et al., 2021; Yin et al., 2021; Zeng et al., 2020; Zhou et al., 2020; Zhuang et al., 2021) matching the inclusion criteria were identified (Fig. 1 ).
Fig. 1.
Study selection process. SR, systematic review.
Study characteristics
The summary features of 21 SRs are presented in Table 1 . All SRs were non-Cochrane and were published between May 2020 and September 2021 in English. The first authors originated from China (n = 19), the Republic of Korea (n = 1), and the United States (n = 1). All 21 SRs included meta-analysis, nine out of 21 SRs included subgroup analysis, and only six SRs included sensitivity analysis. The outcomes of SRs vary widely but are mainly focused on the overall effective rate and clinical symptoms. Sixteen SRs assessed the adverse effects of herbal medicine interventions in treating COVID-19. Nine SRs arrived at a clearly positive conclusion. Among 12 SRs, the conclusions were neither positive nor negative. None of the SRs drew negative conclusions.
Table 1.
Systematic reviews of herbal medicine used for the treatment of COVID-19.
| Study ID | Included study design (n) | Trials (sample size, n) | Methodology quality assessment | Quality of primary studies | Meta-analysis | Subgroup analysis | Sensitivity analysis | Safety assessments | Direction of results | Country | Quality of SR |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Liu (a)(Liu et al., 2020) | RCTs (4) + CCTs (7) | 11 (982) | RoB 1 and NOS | Variable | Yes | Yes | No | Yes | + | China | Poor |
| Ang (Ang et al., 2020) | RCTs (7) | 7 (855) | RoB 1 | Poor | Yes | No | No | Yes | +/− | Korea | Moderate |
| Sun (Sun et al., 2020) | RCTs (7) | 7 (681) | RoB 1 | Poor | Yes | No | No | Yes | + | China | Poor |
| Xiong (a)(Xiong et al., 2020) | RCTs (18) | 18 (2275) | RoB 1 | Poor | Yes | No | No | Yes | + | China | Poor |
| Fan (Fan et al., 2020) | RCTs (7) | 7 (732) | RoB 1 | Poor | Yes | No | No | Yes | + | United States | Poor |
| Pang (Pang et al., 2020) | RCTs (11) | 11 (1259) | RoB 1 | Poor | Yes | No | No | Yes | +/− | China | Poor |
| Luo (Luo et al., 2021) | RCTs (6) + CCTs (13) | 19 (1474) | RoB 1 and NOS | Variable | Yes | Yes | Yes | Yes | +/− | China | Poor |
| Liu (Liu et al., 2021) | RCTs (3) + CCTs (3) + CS (2) | 8 (924) | RoB 1, NOS, and AHRQ | Variable | Yes | No | No | Yes | +/− | China | Poor |
| Zeng (Zeng et al., 2020) | RCTs (2) | 2 (142) | RoB 1 | High* | Yes | No | No | No | + | China | Poor |
| Yan (Yan et al., 2021) | RCTs (3) + cohorts (5) + case series (4) + pre–post study (1) | 13 (1326) | RoB 1 and IHE | Poor | Yes | No | No | No | + | China | Poor |
| Zhou (Zhou et al., 2020) | RCTs (10) | 10 (1293) | RoB 1 | Poor | Yes | Yes | Yes | Yes | +/− | China | Moderate |
| Li (Li et al., 2021) | RCTs (8) | 8 (750) | Modified Jadad score and NOS | Poor | Yes | No | No | No | + | China | Poor |
| Fan (Fan et al., 2021) | RCTs (5) | 5 (824) | RoB 1 | Poor | Yes | No | No | Yes | +/− | China | Poor |
| Liang (Liang et al., 2021) | RCTs (7) | 7 (1079) | RoB 2 | Poor | Yes | Yes | No | Yes | +/− | China | Poor |
| Jiang (Jiang et al., 2021) | RCTs (19) + CCTs (14)+ cohort studies (2) | 35 (3808) | RoB 2 and NOS | Variable | Yes | Yes | Yes | Yes | +/− | China | Moderate |
| Yin (Yin et al., 2021) | RCTs (19) | 19 (1853) | RoB 1 | Poor | Yes | Yes | Yes | No | +/− | China | Moderate |
| Zhuang (Zhuang et al., 2021) | RCT (1) + CCTs (2) | 3 (245) | RoB 1 | Poor | Yes | No | No | No | + | China | Poor |
| Wang (a) (Wang et al., 2021b) | RCT (1) + non–RCTs (6) + cohorts (2) + pre–post studies (7) | 16 (11,237) | RoB 1 and NOS | Variable | Yes | No | No | Yes | +/− | China | Poor |
| Wang (b)(Wang et al., 2021a) | RCTs (25) | 25 (2222) | RoB 2 | High† | Yes | Yes | No | Yes | +/− | China | Moderate |
| Du (Du et al., 2021) | RCTs (12) | 12 (1393) | RoB 1 | Poor | Yes | Yes | Yes | Yes | +/− | China | Poor |
| Wu (Wu et al., 2021) | RCTs (7) + cohorts (8) | 15 (1623) | modified Jadad scale | Poor | Yes | Yes | Yes | Yes | + | China | Poor |
AHRQ, Agency for Healthcare Research and Quality; CCTs, case-control trials; IHE, Institute of Health Economics scale; NOS, Newcastle-Ottawa Scale; RCTs, randomized controlled trials; ROB, Cochrane risk of bias; SR, systematic review; *, inappropriate scoring using RoB domains–not meaningful; †, high risk RCTs were excluded from analysis. +, positive conclusion; +/−, conclusion neither positive nor negative.
Thirteen SRs included only RCTs as primary studies, whereas the remaining eight included nonrandomized evidence in addition to RCTs. The primary studies included in each SRs ranged from two to 25, with significant overlap across the RCTs. There were 258 primary studies identified across the 21 SRs, including 182 RCTs, 45 case–control studies, 17 cohorts, eight pre-post studies, and six case series. Following the removal of overlapping studies, a total of 84 distinct primary studies were identified, including 49 RCTs, 22 case–control studies, four cohorts, seven pre-post studies, and two case series (Fig. 2 ).
Fig. 2.
Study types of primary studies in the included systematic reviews. CCTs, case-control trials; RCT, randomized controlled trials.
For the assessment of the methodological quality of primary studies, 19 SRs used either version 1 or 2 of the Cochrane risk of bias tool, and two SRs used the Jadad scale to assess RCTs. On the other hand, five SRs used the Newcastle–Ottawa scale (NOS), one SR used the NOS and Agency for Healthcare Research and Quality (AHRQ), one SR used the Institute of Health Economics (IHE) scale, and one SR used the modified Jadad scale for the assessment of nonrandomized evidence. The methodological quality of primary studies was reported as poor in 14 SRs and unclear in 5 SRs. Only two SRs reported high-quality primary studies. However, one SR removed high-risk RCTs from their analysis to avoid overstating and distorting their results, while one SR performed the assessment by incorrectly scoring each domain of the Cochrane risk of bias tool and judged the quality of the primary studies based on the overall calculated score.
The herbal medicine interventions were herbal formulas, patented medication, and injections. Among these, the most commonly used herbal formulas were Qingfei Touxie Fuzheng prescription (n = 10), followed by Maxing Xuanfei Jiedu decoction (n = 6) and Qingfei Paidu decoction (n = 6). The most commonly used herbal patented medication was Lianhua Qingwen capsules (n = 20), followed by Jinhua Qinggan granules (n = 15), Shufeng Jiedu capsules (n = 10), Toujie Qingwen granules (n = 10), and Lianhua Qingke granules (n = 10). Table 2 shows the types of herbal medicine interventions evaluated in the included SRs.
Table 2.
Types of herbal medicine intervention included in the evaluated systematic reviews.
| Study ID | Type of herbal medicine intervention |
|||
|---|---|---|---|---|
| Herbal formulae | Herbal patent medicine | Herbal injections | Herbal fumigation | |
| Liu (a) (Liu et al., 2020) | Qingfei Touxie Fuzheng prescription | Diammonium glycyrrhizinate enteric coated capsules, Shufeng Jiedu capsule, Lianhua Qingwen granules, Reyanning mixture, Jinhua Qinggan granules | – | – |
| Ang (Ang et al., 2020) | As recommended by China's National Health Commission guideline | Lianhua Qingke granules, Shufeng Jiedu capsule, Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingwen granules | – | – |
| Sun (Sun et al., 2020) | Qingfei Touxie Fuzheng prescription, Touxie Quwen prescription, Feiyanyihao or Feiyanerhao prescription | Reyanning mixture, Shufeng Jiedu capsule, Jinhua Qinggan granules | – | – |
| Xiong (a) (Xiong et al., 2020) | Qingfei Touxie Fuzheng prescription and as recommended by China's National Health Commission guideline | Toujie Quwen granules, Jinhua Qinggan granules, Reyanning mixture, Shufeng Jiedu capsules, Lianhua Qingwen granules, Lianhua Qingke granules | – | – |
| Fan (Fan et al., 2020) | Qingfei Touxie Fuzheng prescription, Maxing Shigan decoction -Dayuanyin or Shengfu decoction | Jinhua Qinggan granules, Toujie Qingwen granules, Lianhua Qingwen capsule, Jiawei Dayuan granules | – | – |
| Pang (Pang et al., 2020) | Qingfei Touxie Fuzheng prescription, Qingfei paidu decoction, Maxing Xuanfei Jiedu decoction | Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingwen granules, Lianhua Qingke granules, Sufeng Jiedu capsule | – | – |
| Luo (Luo et al., 2021) | Jiawei Dayuan decoction, Feiyanyihao and Feiyanerhao prescription, modified Qingfei Paidu decoction, Qingfei Touxie Fuzheng prescription | Lianhua Qingwen granules, Shufeng Jiedu capsules, Toujie Quwen granules, Reyanning mixture, Jinhua Qinggan granules | Xuebijing | Not specified |
| Liu (Liu et al., 2021) | – | Lianhua Qingwen granules | – | – |
| Zeng (Zeng et al., 2020) | – | Lianhua Qingwen granules | – | – |
| Yan (Yan et al., 2021) | Feiyanyihao prescription, Fuzheng Touxie decoction, Qingfei Paidu decoction | Lianhua Qingwen granules, Shufeng Jiedu capsules | – | – |
| Zhou (Zhou et al., 2020) | Qingfei Touxie Fuzheng prescription, self-prescribed decoction | Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingke granules, FeiyanYihao Chinese Medicine granules, Jinyinhua oral liquid, Diammonium glycyrrhizinate enteric coated capsules, Lianhua Qingwen capsules | – | – |
| Li (Li et al., 2021) | – | Shufeng Jiedu capsules, Lianhua Qingwen granules, Jinhua Qinggan granules | Xuebijing | – |
| Fan (Fan et al., 2021) | – | Lianhua qingwen granules | – | – |
| Liang (Liang et al., 2021) | – | Lianhua Qingwen capsules, Jinhua Qinggan granules, Huoxiang Zhengqi dripping pills, Toujie Quwen granules, Lianhua Qingke granules | – | – |
| Jiang (Jiang et al., 2021) | Qingfei Touxie Fuzheng prescription, Qingfei Paidu decoction, Maxing Xuanfei Jiedu decoction, Xuanfei Baidu decoction, Ganlu Xiaodu decoction, Hanshiyi prescription | Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingwen capsules, Lianhua Qingke granules, Keguan-1, Huoxiang Zhengqi dropping pills, Jinyinhua oral liquid, Diammonium glycyrrhizinate enteric coated capsules, Shufeng Jiedu capsules, Reyanning mixture | Xuebijing | – |
| Yin (Yin et al., 2021) | Feiyanyihao prescription, Pneumonia recovery prescription, Qingfei Touxie Fuzheng prescription, Qingfei Paidu decoction, Maxing Xuanfei Jiedu decoction, Buzhong Yiqi decoction, Xuanfei Baidu decoction | Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingwen granules, Jinyinhua oral liquid, Lianhua Qingke granules, Compound Yin Chai granules with Qingqiao detoxification granules, Shengmai-san with Shenling Baizhu-san, Keguan-1 | – | – |
| Zhuang (Zhuang et al., 2021) | – | Lianhua Qingwen granules | – | – |
| Wang (a) (Wang et al., 2021b) | Qingfei Paidu decoction | – | – | – |
| Wang (b) (Wang et al., 2021a) | Qingfei Touxie Fuzheng prescription, Qingfei Paidu decoction, Maxing Xuanfei Jiedu decoction, Xiaochaihu Decoction and Maxing Shigan decoction, modified Sanren Decoction, Modified Maxing Shigan decoction, modified Shenfu decoction, Yidu-toxicity Blocking Lung decoction, Feiyanyihao or Feiyanerhao prescription, modified Shengmaisan prescription, Xuanfei Qingre prescription, Xuanfei Baidu decoction | Jinhua Qinggan granules, Toujie Quwen granules, Lianhua Qingke granules, Lianhua Qingwen granules, Diammonium glycyrrhizinate enteric coated capsules, Lianhua Qingwen granules, Keguan-1, modified Dayuan capsules, Honeysuckle oral liquid | Xuebijing | – |
| Du (Du et al., 2021) | Maxing Xuanfei Jiedu decoction, Jiawei Dayuan decoction | Jinhua Qinggan granules, Toujie Quwen granules, Jinyinhua oral liquid, Lianhua Qingwen granules, Lianhua Qingke granules, Reyanning mixture, Diammonium glycyrrhizinate enteric coated capsules | – | – |
| Wu (Wu et al., 2021) | Maxing Xuanfei Jiedu decoction | Jinhua Qinggan granules, Lianhua Qingke granules, Lianhua Qingwen granules, Toujie Quwen granules, Reyanning mixture, Shufeng Jiedu capsules | Xuebijing | – |
Methodology quality of included SRs
Fig. 2 depicts the overall findings of the AMSTAR 2 evaluation, and Fig. 3 depicts the assessment of each domain. The quality of the SRs varied, with 16 having serious flaws and 5 having moderate flaws. The findings revealed that the main methodological errors were often ‘extremely poor reporting’ and that the risk of bias (RoB) assessment of primary research was not considered when interpreting the data or drawing conclusions. The majority of the SRs also did not explain why studies were excluded, and many SRs included primary studies that did meet the predefined inclusion criteria. Multiple SRs carried out a meta-analysis using an inappropriate meta-analytical approach (i.e., inappropriate statistical model selection and subgroup analysis). The majority of the primary studies included in the SRs had poor methodological quality, which contributed to the poor methodological quality of the SRs.
Fig. 3.
AMSTAR 2 assessment of the included systematic reviews.
Effects of interventions
In the 21 included SRs, all the assessments of herbal medicine effectiveness on the overall effective rate and clinical symptoms were conducted using a meta-analytical technique. Seventeen SRs evaluated the overall effective rate of herbal medicine interventions in treating COVID-19 and found them to be clinically effective. Five of 17 SRs assessing this outcome were judged to be of moderate methodological quality based on the AMSTAR 2. Only six of 17 SRs employed the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework to assess evidence quality, and all reported low certainty of evidence.
All 21 SRs assessed the clinical symptoms of herbal medicine interventions in COVID-19 treatment and reported favorable results in terms of symptom disappearance rate and time. According to the AMSTAR 2, five of the 21 SRs that evaluated this outcome were found to have a moderate level of methodological quality. Only seven of the 21 SRs utilized the GRADE method to assess evidence quality and indicated a moderate to very low level of certainty in the evidence.
Safety outcomes
Sixteen SRs reported adverse effects (AEs). No serious adverse events were noted. The reported AEs included nausea and vomiting, diarrhea, poor appetite, dizziness, headache, abnormal liver and renal function, and skin rash. No significant difference was found in the occurrence of AEs between the herbal medicine intervention group and comparator group.
Discussion
This overview of SRs analyzed 21 SRs on herbal medicine intervention published after the outbreak of COVID-19, with 84 unique primary studies, including RCTs and nonrandomized controlled studies with a total of more than 36,000 patients. All SRs included a meta-analysis and mainly presented favorable results regarding the use of herbal medicine intervention. Using the AMSTAR 2, the confidence in these reviews was generally low, severely limiting the ability to draw a clear conclusion regarding the effectiveness of herbal medicine in treating COVID-19.
The findings in this study also revealed that a remarkably high number of SRs on herbal medicine intervention for COVID-19 have been published during the past one and a half years, with an average of one SR published per month. Their conclusions were positively biased. One of the reasons might be that only positive results were reported in primary research related to this type of intervention. Most SRs had poor ratings on the AMSTAR 2. The majority of the primary studies included in the SRs were poorly designed RCTs, which strongly impacted the quality of evidence. This might be due to the nature and severity of the disease, where evidence of possible alternative medicines is greatly sought. The pressing need for information in the event of a pandemic does not excuse poor methodological approaches or reporting. Furthermore, the number of main studies did not grow linearly across the published SRs chronologically, indicating that these SRs may not have been completed with adequate methodology, resulting in the low quality of these SRs.
In addition, the primary studies in the included SRs relied heavily on Chinese studies. It is probable that the findings reported in the included SRs were not generalizable since herbal medicine therapies for COVID-19 were only used (and relevant clinical research was only authorized) in China at the time.
This overview of SRs shows that while there is much evidence demonstrating the advantages of herbal medicine interventions, the quality of the evidence is insufficient to provide solid and accurate judgments about the effectiveness of herbal medicine therapies for COVID-19. The inability to draw any conclusions about the efficacy of herbal therapy is strongly based on the methodology and quality of the available research. Hence, instead of only providing inconclusive findings, there is an urgent need to enhance the quality of research to provide data that can support therapeutic choices. The Consolidated Standards of Reporting Trials (CONSORT) guidelines should be followed by researchers when planning and reporting clinical trials to develop high-quality primary studies. Similarly, SRs should follow PRISMA guidelines to develop high-quality SRs. Evidence from SRs often has considerable implications for evidence-based practice and policy; hence, high-quality SRs with high confidence of evidence are needed.
The preparation of SRs takes a lot of time and work, and updating them also takes a lot of effort. However, with so much research resources and output, the scientific communities have resorted to the rapid publication of research studies due to the pandemic crisis, and COVID-19-related SRs become outdated soon after their publication. A constantly increasing number of SRs on comparable issues published in a short period of time makes it difficult for healthcare decision-makers, researchers, and clinicians to stay up to date on evidence. Theoretically, SRs are conducted to filter out research waste from poorly designed and reported primary research. However, poorly conducted and reported SRs can also be categorized as research waste. The findings of this overview of SRs clearly demonstrate that the majority of the herbal medicine-related SRs published in COVID-19 can be seen as redundant, as the confidence in their evidence is extremely low. COVID-19-related evidence should ideally be synthesized on a regular basis in high-quality living SRs, for which evidence is incorporated as it becomes available.
This overview has some limitations. The literature searches were limited to English databases, and there may have been relevant papers that were overlooked. While it was expected that there would be concerns with methodology quality in primary studies within the included SRs, the quality of primary studies was not reviewed, and the assessment of the AMSTAR 2 purely relied on information provided in the included SRs.
Conclusion
In conclusion, in this overview of SRs, evidence from 21 SRs was analyzed. However, confidence in the findings of the majority of the included SRs was low. The included SRs either provide positive conclusions despite their poor methodological quality or are inconclusive. Clinical studies and SRs should still adhere to established methodological standards to provide high-quality reliable evidence to decision-makers, clinicians, and patients even during the pandemic. Nevertheless, this overview of SRs adhered to strict methodological standards and included a thorough and extensive search, as well as a standard tool for the methodological assessment of SRs, hoping to provide a comprehensive summary of all the evidence on the use of herbal medicine in the management of COVID-19.
Authors contributions
This work is conceptualized by JZ and MSL. The study methodology was designed by LA and ES, and validated by HWL, MSL, and JZ. Formal analysis and data curation were performed by LA, ES, and MSL. The original draft was written by LA, reviewed and edited by ES, JZ and MSL. This project was supervised and administered by MSL. All data were generated in-house, and no paper mill was used. All authors agree to be accountable for all aspects of work ensuring integrity and accuracy.
Funding
This research was supported by Korea Institute of Oriental Medicine (KSN20214115 and KSN2022210).
Ethical statement
This research did not involve any human or animal experiment.
Data availability
Data will be made upon reasonable request.
Declaration of Competing Interest
The authors declare no conflict of interest.
Acknowledgement
None.
References
- Ang L., Song E., Lee H.W., Lee M.S. Herbal medicine for the treatment of coronavirus disease 2019 (COVID-19): a systematic review and meta-analysis of randomized controlled trials. J. Clin. Med. 2020;9:1583. doi: 10.3390/jcm9051583. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dai T., Zhang L., Dai X., Zhang X., Lu B., Zheng Y., Shen D., Yan Y., Ji C., Yu J., Sun L. Multimode participation of traditional Chinese medicine in the treatment of COVID-19. Integr. Med. Res. 2021;10 doi: 10.1016/j.imr.2021.100781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Du X., Shi L., Cao W., Zuo B., Zhou A. Add-on effect of Chinese herbal medicine in the treatment of mild to moderate COVID-19: a systematic review and meta-analysis. PLoS One. 2021;16 doi: 10.1371/journal.pone.0256429. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fan A.Y., Gu S., Alemi S.F. Chinese herbal medicine for COVID-19: current evidence with systematic review and meta-analysis. J. Integr. Med. 2020;18:385–394. doi: 10.1016/j.joim.2020.07.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fan Z., Guo G., Che X., Yang Y., Liu Y., Li L., Chang X., Han L., Cai X., Tang H. Efficacy and safety of Lianhuaqingwen for mild or moderate coronavirus disease 2019: a meta-analysis of randomized controlled trials. Medicine (Baltimore) 2021;100(21) doi: 10.1097/MD.0000000000026059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fidahic M., Nujic D., Runjic R., Civljak M., Markotic F., Lovric Makaric Z., Puljak L. Research methodology and characteristics of journal articles with original data, preprint articles and registered clinical trial protocols about COVID-19. BMC Med. Res. Methodol. 2020;20:161. doi: 10.1186/s12874-020-01047-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Higgins J., Thomas J., Chandler J., Cumpston M., Li T., Page M., Welch V. 2nd Ed. John Wiley & Sons; Chichester (UK): 2019. Cochrane Handbook for Systematic Reviews of Interventions. [Google Scholar]
- Hunt H., Pollock A., Campbell P., Estcourt L., Brunton G. An introduction to overviews of reviews: planning a relevant research question and objective for an overview. Syst Rev. 2018;7:39. doi: 10.1186/s13643-018-0695-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Jiang F., Xu N., Zhou Y., Song J., Liu J., Zhu H., Jiang J., Xu Y., Li R. Contribution of traditional Chinese medicine combined with conventional western medicine treatment for the novel coronavirus disease (COVID-19), current evidence with systematic review and meta-analysis. Phytother. Res. 2021 doi: 10.1002/ptr.7209. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lee B.-.J., Lee J.A., Kim K.-.I., Choi J.-.Y., Jung .H..-J. A consensus guideline of herbal medicine for coronavirus disease 2019. Integr. Med. Res. 2020;9 doi: 10.1016/j.imr.2020.100470. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Li F., Jiang Y., Yue B., Luan L. Use of traditional Chinese medicine as an adjunctive treatment for COVID-19: a systematic review and meta-analysis. Medicine (Baltimore) 2021;100(30) doi: 10.1097/MD.0000000000026641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liang S.-.B., Fang M., Liang C.-.H., Lan H.-.D., Shen C., Yan L.-.J., Hu X.-.Y., Han M., Robinson N., Liu .J..-P. Therapeutic effects and safety of oral Chinese patent medicine for COVID-19: a rapid systematic review and meta-analysis of randomized controlled trials. Complement. Ther. Med. 2021;60 doi: 10.1016/j.ctim.2021.102744. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu M., Gao Y., Yuan Y., Yang K., Shi S., Tian J., Zhang J. Efficacy and safety of herbal medicine (Lianhuaqingwen) for treating COVID-19: a systematic review and meta-analysis. Integr. Med. Res. 2021;10 doi: 10.1016/j.imr.2020.100644. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Liu M., Gao Y., Yuan Y., Yang K., Shi S., Zhang J., Tian J. Efficacy and safety of integrated traditional chinese and western medicine for corona virus disease 2019 (covid-19): a systematic review and meta-analysis. Pharmacol. Res. 2020;158 doi: 10.1016/j.phrs.2020.104896. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Luo X., Ni X., Lin J., Zhang Y., Wu L., Huang D., Liu Y., Guo J., Wen W., Cai Y., Chen Y., Lin L. The add-on effect of Chinese herbal medicine on COVID-19: a systematic review and meta-analysis. Phytomedicine. 2021;85 doi: 10.1016/j.phymed.2020.153282. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Page M.J., McKenzie J.E., Bossuyt P.M., Boutron I., Hoffmann T.C., Mulrow C.D., Shamseer L., Tetzlaff J.M., Akl E.A., Brennan S.E., Chou R., Glanville J., Grimshaw J.M., Hróbjartsson A., Lalu M.M., Li T., Loder E.W., Mayo-Wilson E., McDonald S., McGuinness L.A., Stewart L.A., Thomas J., Tricco A.C., Welch V.A., Whiting P., Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi: 10.1136/bmj.n71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pang W., Liu Z., Li N., Li Y., Yang F., Pang B., Jin X., Zheng W., Zhang J. Chinese medical drugs for coronavirus disease 2019: a systematic review and meta-analysis. Integr. Med. Res. 2020;9 doi: 10.1016/j.imr.2020.100477. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shankar A., Dubey A., Saini D., Prasad C.P. Role of complementary and alternative medicine in prevention and treatment of covid-19: an overhyped hope. Chin. J. Integr. Med. 2020;26:565–567. doi: 10.1007/s11655-020-2851-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smith V., Devane D., Begley C.M., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med. Res. Methodol. 2011;11:15. doi: 10.1186/1471-2288-11-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sun C.-.Y., Sun Y.-.L., Li .X..-M. The role of Chinese medicine in COVID-19 pneumonia: a systematic review and meta-analysis. Am. J. Emerg. Med. 2020;38:2153–2159. doi: 10.1016/j.ajem.2020.06.069. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Teo C.S., Tan P.M., Shu C.S.I., Choo Z.X., Te K.K. Challenges and strategies for implementing Chinese medicine during COVID-19 in Malaysia. Integr. Med. Res. 2021;10 doi: 10.1016/j.imr.2021.100783. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang H., Xu B., Zhang Y., Duan Y., Gao R., He H., Li X., Li J. Efficacy and safety of traditional Chinese medicine in coronavirus disease 2019 (covid-19): a systematic review and meta-analysis. Front. Pharmacol. 2021;12 doi: 10.3389/fphar.2021.609213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wang Q., Zhu H., Li M., Liu Y., Lai H., Yang Q., Cao X., Ge L. Efficacy and safety of Qingfei Paidu decoction for treating covid-19: a systematic review and meta-analysis. Front. Pharmacol. 2021;12 doi: 10.3389/fphar.2021.688857. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Wu X., Li W., Qin Z., Xue L., Huang G., Luo Z., Chen Y. Traditional Chinese medicine as an adjunctive therapy for mild and common COVID-19: a systematic review and network meta-analysis. Medicine (Baltimore) 2021;100(40) doi: 10.1097/MD.0000000000027372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Xiong X., Wang P., Su K., Cho W.C., Xing Y. Chinese herbal medicine for coronavirus disease 2019: a systematic review and meta-analysis. Pharmacol. Res. 2020;160 doi: 10.1016/j.phrs.2020.105056. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Yan L.-.Z., Mao F.-.W., Cao Y.-.H., Xie M. Clinical effects of the combination of Traditional Chinese and Western Medicines on coronavirus disease 2019: a systematic review and Meta-analysi. J. Tradit. Chin. Med. 2021;41:499–506. doi: 10.19852/j.cnki.jtcm.2021.03.001. [DOI] [PubMed] [Google Scholar]
- Yin B., Bi Y.-.M., Sun L., Huang J.-.Z., Zhao J., Yao J., Li A.-.X., Wang X.-.Z., Fan .G..-J. Efficacy of integrated traditional chinese and western medicine for treating COVID-19: a systematic review and meta-analysis of RCTs. Front. Public Health. 2021;9 doi: 10.3389/fpubh.2021.622707. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zeng M., Li L., Wu Z. Traditional Chinese medicine Lianhua Qingwen treating corona virus disease 2019(COVID-19): meta-analysis of randomized controlled trials. PLoS One. 2020;15 doi: 10.1371/journal.pone.0238828. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhou L.-p., Wang J., Xie R.-h., Pakhale S., Krewski D., Cameron D.W., Wen S.W. The effects of traditional Chinese medicine as an auxiliary treatment for COVID-19: a systematic review and meta-analysis. J. Alternat. Complement. Med. 2020;27:225–237. doi: 10.1089/acm.2020.0310. [DOI] [PubMed] [Google Scholar]
- Zhuang J., Dai X., Wu Q., Cai H., Fu X., Zhang W., Chen B. A meta-analysis for Lianhua Qingwen on the treatment of Coronavirus disease 2019 (COVID-19) Complement. Ther. Med. 2021;60 doi: 10.1016/j.ctim.2021.102754. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
Data will be made upon reasonable request.