Choose your shortcuts wisely: COVID-19 rapid reviews of traditional, complementary and integrative medicine

Abstract

Background: The COVID-19 pandemic has led to an explosion of rapid reviews geared towards providing time sensitive answers for clinical and policy decision-makers. Rapid reviews (RRs) strike a balance between rigour and rapidity to minimise bias and optimise transparency within specified constraints. Methods: This review article appraised the methods and reporting standards of a convenience sample of RR protocols and RRs of COVID-19 clinical management questions, published in the first six-months of 2020. Inclusion criteria were all RR protocols evaluating traditional, complementary, and integrative medicine (TCIM) registered on PROSPERO, and all RRs indexed on PubMed or published on the Oxford COVID-19 Evidence Service. A purpose-specific 9-item reporting checklist reflecting recommended minimum requirements for RRs was applied. Findings were synthesised and narrated in the context of methodological considerations for conducting and reporting RRs of TCIM. Results: Included studies were five RR protocols of TCIM and 16 RRs, of which five considered TCIM. Wide variations in RR methods were proposed or applied, as were the reporting standards. All five RRs that evaluated TCIM had the lowest reporting standards that limited reproducibility and transparency. Despite accepted recommendations, most RRs did not publish a protocol. Conclusions: We propose that specific research disciplines, such as TCIM, have a uniqueness that may lead to unacceptable outputs if minimum methodological standards are not applied. The recommended minimum requirements will optimise the credibility of rapid reviews of TCIM and limit the risk of prematurely disregarding a potentially effective intervention.

1. Introduction

The purpose of this paper is to clarify the minimum standards required for rapid reviews (RRs) of traditional, complementary and integrative medicine (TCIM) in the context of the COVID-19 pandemic. The intention is to guide researchers in TCIM to ensure that new RRs are of a minimum acceptable quality to maximise their credibility and impact.

During a global health crisis, such as the COVID-19 pandemic, RRs are needed to provide up to date summaries of important evidence. The methods for RRs are used as an alternative to the ‘gold standard’ systematic review (SR) methods when there are legitimate time, resource or other logistical constraints.2., 3., 4. Restrictions may be applied to any aspect of the methods or reporting of a RR. For example, the scope of the question might be limited to a narrower population or set of outcomes. Methods may be streamlined by searching fewer databases, excluding grey literature, using single rather than dual reviewers for some or all review steps, not conducting a risk of bias appraisal and/or only conducting a qualitative analysis with a narrative summary.⁴ Whilst the use of less rigorous methods may be unavoidable, concern has also been raised about poor quality reporting in RRs that is completely avoidable.⁴

What is a rapid review?

“A form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a resource-efficient manner.”

Cochrane Rapid Reviews Methods Group¹

The need to optimise methodological rigour has prompted the much anticipated release of the ‘Interim Guidance from the Cochrane Rapid Reviews Methods Group’.¹ The guidance builds on earlier guidance from the World Health Organisation (WHO)⁵ and leading evidence reviewers.² It complements the section of the Cochrane Collaboration website: ‘Rapid Reviews in response to COVID-19’. The site also includes a set of streamlined templates, fast-track editorial services, and their ‘Special Collection’ series on COVID-19 that are brief, best-evidence summaries mostly reporting indirect evidence from published Cochrane reviews. Other initiatives to support rapid reviewers include fast-tracking of PROSPERO registration for COVID-19 systematic, umbrella/overviews and rapid reviews, and free access to Covidence software for researchers who are pursuing work on COVID-19.

To date, most of the COVID-19 RRs are evidence appraisals of interventions, which can also include meta-analyses.⁴ Like any review, however, other questions and data sources can be rapidly reviewed. For example, there is a RR of traditional herbal medicine pattern recognition for COVID-19 symptoms.⁶

2. COVID-19 rapid reviews protocols

In line with RR recommendations, numerous COVID-19 RR protocols have been published. As of 9 July 2020, 142 RR protocols, including 2 protocols for Traditional Chinese Medicine (TCM),7., 8. were listed on the Cochrane Coronavirus (COVID-19) Resources website, and 102 RR protocols, including five TCIM protocols9., 10., 11., 12., 13. were registered with PROSPERO. A further 72 SR protocols categorised by PROSPERO as TCM (one of which included other types of TCIM interventions) were also registered. Whilst none were RRs, all bar one of the TCM SR protocols¹⁴ limited the population to SARS-CoV-2 infections and at this early stage in the pandemic a rapid turn-around would be possible.

Table 1 summarises the methodological constraints proposed in the five RR TCIM protocols registered with PROSPERO.9., 10., 11., 12., 13. The table does not included the two TCM RRs protocols listed on the Cochrane website, as neither were Cochrane reviews and a citation for the registered protocol was not provided.7., 8. Author JH conducted the search and extracted the data that was then verified by SA.

Table 1.

Examples of Methodological Constraints Published in COVID-19 Rapid Review Protocols of Traditional, Complementary and Integrative Medicine Interventions

Arentz et al. Protocol for a rapid review of zinc for the prevention or treatment of COVID-19 and other coronavirus-related respiratory tract infections in humans PROSPERO CRD42020182044.9

PICO: Extended to include all viral respiratory infections. Post-hoc decisions declared to change critical and important outcomes.a

Search strategy: Limited to 1 major Chinese and 6 major English language databases. Searches of clinical trial registries limited to Covid-19. Limited bibliography searches. No updating of search prior to final analysis. Following calibration, single screening, with a second reviewer screening the full-text articles. Post-hoc decisions declared to adjust search terms.

Types of studies: Limited to RCTs and pseudo-RCTs published in peer-review journals.

Data extraction: Following calibration, limited to single reviewer data extraction, with a second reviewer checking for accuracy. Limitations also included not contacting authors of published studies for further information.

Risk of bias: Standardised tool. Following calibration, limited to single reviewer appraisal, with a second reviewer verifying judgements. Additional GRADE approach for the certainty of the evidence included downgrading non-SARS-CoV-2 due to indirectness.

Synthesis: No limits applied.a

Aucoin et al. A rapid review of Echinacea for the prevention or treatment of COVID-19 and other respiratory tract infections in humans PROSPERO 2020 CRD42020186339.10

PICO: Extended to include all viral respiratory infections and cytokine storms.

Search strategy: Limited to 4 major English language databases and no clinical trial registries. Limited bibliography searches. No updating of search prior to final analysis. NI: dual or single screening.

Types of studies: No limits applied.

Data extraction: Limited to single reviewer, with a second reviewer checking for accuracy.

Risk of bias: Standardised tool for RCTs limited to single reviewer appraisal, with a second reviewer verifying judgements. NI: for other types of included studies.

Synthesis: NI: about type of analysis, presumably only qualitative. Limited to no subgroup/subset analysis.

James et al. Could nutrition modulate COVID-19 susceptibility and severity of disease? A rapid systematic review PROSPERO CRD42020186194.11

PICO: Limited to Covid-19 yet a large range of nutritional conditions and interventions. Outcomes are also broad and non-specific, including respiratory, gastrointestinal, multi-organ and laboratory.

Search strategy: Limited to 2 major English language databases, 8 English language clinical trial registries and pre-print severs. Bibliography searches includes systematic reviews. Dual screening of peer-reviewed articles limited to approximately 10% of articles. Dual screening of pre-print servers. Single screening of clinical trial registries. Non-systematic, landscape review of other pertinent information is planned.

Types of studies: No limits, including systematic reviews. Post-hoc decisions declared to exclude low quality pre-print articles with no peer review from the narrative synthesis that will be noted in the Annexes. NI: non-human studies.

Data extraction: Limited to single reviewer data extraction. Limitations also include not contacting authors for further information.

Risk of bias: No formal appraisal.

Synthesis: Synthesis without Meta-analysis (SWiM). Review limitation will be addressed in the discussion.

Wieland et al. Elderberry for prevention and treatment of viral respiratory illnesses PROSPERO CRD42020186339.12

PICO: Extended to include all viral respiratory infections and cytokine storms.

Search strategy: Limited to 3 major English language databases and 9 other English language databases that include clinical trial registries and pre-print severs. NI bibliography search. Title/abstract screening appears to be single, followed by dual screening of full text.

Types of studies: Limited to RCTs investigating efficacy. Limited to human studies for cytokine storm.

Data extraction: Limited to single reviewer data extraction, with a second reviewer checking for accuracy.

Risk of bias: Standardised tools. Following calibration, limited to single reviewer appraisal, with a second reviewer verifying judgements.

Synthesis: No limits applied.

Xiao-Yang et al. Shufeng Jiedu capsule for Covid-19 and other respiratory viral infections: a rapid review on current evidence PROSPERO 2020 CRD42020194320.13

PICO: Extended to include all viral respiratory infections.

Search strategy: Limited to 3 major Chinese and 4 major English language databases and no clinical trial registries. NI: bibliography search, contacting authors. Dual screening.

Types of studies: No restrictions for Covid-19. Limited to preclinical studies and RCTs for other viral RTIs.

Data extraction: No restrictions for Covid-19. Limited to PICO for other viral RTIs. NI: number of reviewers.

Risk of bias: Standardised tools. NI: number of reviewers.

Synthesis: Meta-analysis restricted to Covid-19 outcomes.

^aA revised protocol has since been published providing more clarity on priority outcomes, along with the decision to limit the meta-analysis to critical outcomes and only report critical outcomes in the summary of findings table.¹⁵PICO: Population, Intervention, Comparison, Outcomes; NI: no information; RCTs: randomised controlled trials; RTIs: respiratory tract infections.

The RR protocols varied in the amount of detail provided. Most did not explicitly state when RR methodological compromises were being made. Presumably, standard SR methods would be applied unless stated otherwise.

In line with accepted RR compromises, it was common for the protocols to limit the number of databases to be searched and to use single, rather than dual reviewers for various tasks. However, the amount of detail regarding the use of single reviewers varied, with quite a few protocols not stating if single or dual reviewers would be used to appraise the risk of bias and extract data. Often information about calibration and verification of accuracy was missing. A few protocols made provisions for post-hoc amendments to parts of the protocol. One planned to supplement the systematic RR findings with a non-systematic landscape review. At least one of the RRs planned only to conduct a qualitative analysis.

Areas where methodologically accepted RR compromises were rarely proposed, included the use of informal tools for the risk of bias appraisal and few instances where the PICO (i.e. populations, interventions, comparisons, and outcomes) was restricted. Only one protocol stated the risk of bias of included studies would not be appraised. The rest planned to use standardised tools, such as RoB-2,¹⁶ for randomised controlled trials (RCTs). However, it was not always clear how, or if non-RCTs would be appraised. Regarding the PICO, one protocol published an amended protocol,¹⁵ in which the meta-analysis of outcomes would now be limited to critically important outcomes. Another protocol planned only to conduct a meta-analysis of SARS-CoV-2 outcomes and a narrative review of other respiratory viral infections. No limits on outcomes nor their analysis were made in any of the other protocols. Interestingly, rather than limiting the PICO, most protocols expanded the population to include people with any type of viral respiratory infection. In some instances, this was accompanied with an a priori statement about how non-SARS-CoV-2 evidence would be downgraded. The decision to include indirect evidence was not explained by the reviewers, that in part may reflect the PROSPERO protocol template.

3. COVID-19 rapid reviews

Whilst “velocity does not have to impact transparency and appropriate methods”,³ we are concerned that this may be occurring in the context of RR of TCIM in the current pandemic. For instance, by the 1 July 2020, the Oxford COVID-19 Evidence Service (Centre for Evidence-Based Medicine (CEBM), Oxford University) had published 193 articles on its website that were described as rapid evidence reviews, data analysis and thought-provoking writing.¹⁷ The terms “treatment”, “management” and “effective” were each entered into the website's Quick search function, yielding 26 articles of which 10 include the term “rapid review” in their title, abstract or main text (Table 2). Five (half) of these RRs included TCIM interventions in the research question and/or search terms. None of the RRs had been peer reviewed, so a second search was conducted on PubMed using the following: “rapid review”[All Fields] with the filters: English, MEDLINE, Nursing journals, Humans, from 1 January to 31 June 2020. Of the twenty articles retrieved, 15 were RRs, 10 were specific to COVID-19 of which six aimed to answer clinical management questions. As these RRs are readily available and demonstrate a range of quality issues, they will be used for this illustration.

Table 2.

A Content Analysis of the Reporting of Rapid Reviews of COVID-19 Clinical Management Questions Posted on the Centre for Evidence Based Medicine: COVID-19 EVIDENCE¹³ or Indexed on PubMed between 1 January to 31 June 2020

Aim	Protocol	Inclusion	Search	Screen	Extract	Search Results	Appraisal	Tables	Reporting Score	Synthesis	IM	Peer reviewed
Noone et al. Video calls for reducing social isolation and loneliness in older people: a rapid review.18
Yes	Yes	Adequate	Adequate	Adequate	Adequate	Adequate	Formal	Adequate	18	Meta-analysis	No	Yes
Valk et al. Convalescent plasma or hyperimmune immunoglobulin for people with COVID-19: a rapid review.19
Yes	Yes	Adequate	Adequate	Adequate	Adequate	Adequate	Formal	Adequate	18	Narrated	No	Yes
Kisely et al. Occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: rapid review and meta-analysis.20
Yes	No	Adequate	Adequate	Adequate	Adequate	Adequate	Formal	Adequate	16	Meta-analysis	No	Yes
Mullins et al. Coronavirus in pregnancy and delivery: rapid review.21
Yes	Noa	Adequate	Adequate	Adequate	Limited	Adequate	Informal	Adequate	14	Narrated	No	Yes
Brooks et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence.22
Yes	No	Limited	Adequate	Limited	Limited	Adequate	Simple	Adequate	12	Narrated	No	Yes
Dorward et al. Lopinavir/ritonavir: A rapid review of effectiveness in COVID-19.23
Yes	No	Adequate	Adequate	Adequate	Limited	Adequate	Informal	No	12	Narrated	No	No
Soliman et al. Does BCG vaccination protect against acute respiratory infections and COVID-19? A rapid review of current evidence.24
Yes	No	Adequate	Adequate	Limited	No	Adequate	Informal	Adequate	12	Narrated	No	No
Jones et al. How can healthcare workers adapt non-pharmacological treatment – whilst maintaining safety – when treating people with COVID-19 and delirium?25
Yes	No	Adequate	Adequate	No	No	No	Formal	Limited	9	Narrated	No	No
Li et al. Rapid review for the anti-coronavirus effect of remdesivir.26
Yes	No	No	Limited	No	No	No	Informal	Adequate	6	Narrated	No	Yes
Raymond et al. Mast cell stabilisers, leukotriene antagonists and antihistamines: A rapid review of effectiveness in COVID-19.27
No	No	Adequate	Adequate	Adequate	No	No	Simple	No	6	Narrated	No	No
Ahmad et al. Rapid Review: Diabetic retinopathy screening during the COVID-19 pandemic.28
Yes	No	Limited	Adequate	No	No	No	Simple	No	5	Narrated	No	No
Lee et al. Vitamin D: A rapid review of the evidence for treatment or prevention in COVID-19.29
Yes	No	Limited	Adequate	No	No	No	Simple	No	5	Narrated	Yes	No
Zimmerman et al. Practical tips for clinicians helping patients with COVID-related anxiety/distress.30
Yes	No	No	Adequate	No	No	No	Simple	Limited	5	Narrated	Yes	No
	McCall et al. Does physical exercise prevent or treat acute respiratory infection (ARI)?31
Yes	No	No	Adequate	No	No	No	Simple	No	4	Narrated	Yes	No
Van Hecke et al. N-acetylcysteine: A rapid review of the evidence for effectiveness in treating COVID-19.32
Yes	No	No	Adequate	No	No	No	Simple	No	4	Narrated	Yes	No
Lloyd-Williams et al. Activities delivered at home by family carers to maintain cognitive function in people with dementia socially isolating during COVID-19: Evidence for Non – technology based activities/interventions.33
Yes	No	No	No	No	No	No	Informal	No	3	Narrated	Yes	No

^aA rationale for not publishing a protocol was given.Aim: [Yes/No] clearly formulated research question (title is excluded). Protocol: [Yes/No] publicly available or submitted for publication Inclusion: [Adequate/Limited/No] Adequate reports the inclusion criteria for the types of studies, populations, intervention, control and outcomes. Search: [Adequate/Limited/No] Adequate search strategy reports that >1 major database, plus another source was searched and the search terms used. Screen: [Adequate/Limited/No] Adequate reports number of reviewers that screened title/abstract and full-text, and whether any calibration/verification was undertaken. Extract: [Adequate/Limited/No] Adequate reports number of reviewers that extracted data, types of data extracted, and whether any calibration/verification was undertaken. Results: [Adequate/Limited/No] refers to search flow results Adequate narrates the number of articles screened and included at title/abstract, and full text with reasons for exclusion. Appraisal: [Simple/Informal/Formal] Simple is a best evidence synthesis reporting highest level of evidence with little independent critique, Informal is a narrated, basic critique of the certainty/quality of the evidence, Formal states that standardised tools were used for appraising risk of bias/certainty of evidence. Table: [Adequate/Limited/No] Adequate provides at least one summary table reporting characteristics of included studies or summary of findings. Reporting score: [total score] out of a maximum possible score of 18: 2 points for Yes/Adequate/Formal, 1 point for Limited/Informal and no points for No/Simple. The type of analysis was not included in the score as CEBM deemed a qualitative analysis (i.e. narrated) was adequate and conducting a meta-analysis does not always correlate with quality IM: [Yes/No] traditional, complementary or integrative medicine interventions were included in the search terms and/or reported in the results.

Table 2 presents the results of a content analysis of the 16 shortlisted RRs. A purpose-specific 9-item reporting checklist was created that reflected the recommended minimum requirements for RRs published by CEBM authors (note: at the time of publication, the authors preferred to call them restricted reviews).² Author JH independently appraised and rated the RRs. The ratings were verified by SM, GY and/or JG. Discrepancies were resolved through consensus.

Notably, the CEBM recommended minimum requirements² are much more flexible than the recently released interim guidance provided by the Cochrane Collaboration. Despite applying this lower standard, we found that the reporting standards of many of the included RRs were below par. All five of the RRs that considered TCIM were rated as having the lowest reporting standards. In contrast, the two Cochrane RRs easily met all the minimum reporting standards. All bar one of the CEBM RRs included a standard disclaimer statement noting the articles had not been peer reviewed and that all sources cited should be checked. With the exception of one RR,²⁶ all the other peer reviewed articles had the highest reporting scores.

Despite CEBM's recommendation,² none of their RRs had published a basic protocol. However, this was not uncommon as the only two RRs to publish a protocol were the Cochrane reviews.18, 19 Mullins et al.,²¹ was the only RR without a protocol to note this was a limitation and provide a reason for not publishing a protocol.

In line with the CEBM's recommendations,² all the RRs provided adequate background information justifying the rationale for the review and bar one CEBM RR,²⁷ they all stated their research question/s in the body of the text. Most (14) reported information about the search strategy; this included searching the minimum requirement of one major database plus one other source. Yet, eight of the CEBM RRs and one peer-reviewed RR did not report their search flow results.

Wide variations were also observed in the other minimum reporting requirements. All of which were avoidable and unnecessarily limited the transparency and quality of the RRs. For example, whilst it is acceptable for some or all of the screening, appraising and data extraction to be conducted by one reviewer rather than the gold-standard of two reviewers independently doing each task, details about the methods applied should still be reported and the limitations noted.

Other variations reflected pragmatic constraints that were applied to the methods or reporting and for the most part, aligned with the more flexible approach recommended by CEBM.² Examples included only four RRs that formally critiqued the risk of bias/quality of the included studies and cited the appraisal tools. Another five narrated an overall summary of limitations of the included evidence, with limited or no information specific to each study. The remaining seven, appeared to defer to a simple hierarchy of evidence (i.e. levels of evidence) in which the findings of a SR supersede primary studies that in turn supersede non-human studies, possibly irrespective of their quality. Only two RRs, one of which was a Cochrane review, conducted a meta-analysis.18, 20. The remaining chose to only narrate their findings. Various types of tables were used to summarise and present the data. Jones et al.²⁵ used a novel approach in which the references at the end of the review included three lists with the references that informed the three key review findings. Only the two Cochrane RRs presented a summary of findings table and graded the certainty of the evidence.18, 19

Given the widespread global use of TCIM, it is commendable that the CEBM has included TCIM topics. This is in sharp contrast with the Cochrane COVID Rapid Reviews website and Question Bank that has only listed pharmacological interventions as priority Clinical Management questions, including for prophylaxis, and appears to also ignore the potential role of TCIM in the clinical management of pandemic-related impacts on health.³⁴ However, it is very concerning that given CEBM's highly influential position, all of the RRs that considered TCIM had consistently poor reporting standards that limited reproducibility and transparency. Already, at least one of these RRs has been used to inform national guidelines, yet the methodological limitations of the RR were not noted.²⁹ This is concerning as it increases the risk of bias.

4. Reproducibility, transparency and bias

At the outset, it is important to note that whilst RRs are similar to traditional SRs, they are not a replacement. Both should be protocol driven, however, RRs tend to have poorer reporting quality, provide less detail with fewer considerations and can miss key studies. In RRs any methodological shortcomings will be amplified.³⁵ Trade-offs in comprehensiveness and rigour increase the risk of bias with resultant type 1 (false positive) and type 2 (false negative) errors, and reduce the certainty/quality of the review findings.³⁵ This concept is critical to consider in RRs and particularly for TCIM, as the conclusions of the review are rarely based on the totality of the literature,³⁶ but instead a limited subset, which is therefore more prone to bias.

Whilst no research is methodologically flawless, at the core of modern science are the concepts of reproducibility and transparency. Reproducibility simply means, if another investigator follows a published set of methods, they will obtain similar results. The notion of transparency follows, in that, the description of the methods and reporting of results must be comprehensive enough to replicate the study. Transparency in research also relates to ensuring that the results reflect the methods and that anomalous data which does not fit the a priori hypothesis is not buried, it is explored and openly discussed. It is for these reasons that like SRs,³⁷ a published protocol for a RR is recommended along with minimum reporting standards.2., 6., 38.

Rapid reviewers must also be careful not to misrepresent nor distort their conclusions and recommendations due to the conscious or unconscious ‘spin’ of their findings. Boutron and Ravaud define spin as “specific intentional or unintentional reporting that fails to faithfully reflect the nature and range of findings and that could affect the impression the results produce in readers”.³⁹ The concept is closely linked to a type of cognitive bias called confirmation bias that increases the likelihood of favouring personal beliefs and ideologies when interpreting, reporting and recalling information.⁴⁰ Evidence is accumulating that this type of bias is highly problematic across scientific practices⁴⁰ and given the polarised opinions around TCIM,41, 42. it is an issue of particular relevance for RRs in the field. Along with impacting the rigour of a RR, confirmation bias may also result in the end-user of a RR prematurely disregarding or accepting the findings without further critique. Whilst it is impossible to compensate for the outright dismissal of TCIM by those with rigid ideologies, a high-quality RR can provide a counterbalance for highly critical readers by ensuring that the methodological scholarship is above reproach.

In the case of TCIM, the risk of type 2 errors is particularly concerning as it can amplify cognitive biases and erroneously steer interest, resources and knowledge away from a potentially effective intervention. The CEBM RR of N-acetylcysteine (NAC) (Table 2) is a good example.³² In the conclusions, the reviewers downgraded promising clinical evidence due to the in vitro findings that any observed therapeutic efficacy of NAC was likely to be strain dependent. However, the RR failed to include two key non-human studies, that were quickly identified on PubMed using the first two search terms in the addendum: N-acetylcysteine AND coronavirus. Both studies demonstrated that NAC also has anti-viral activity on the coronavirus, porcine epidemic diarrhoea virus.43., 44 Further, the methodological decision to downgrade the certainty of clinical evidence based on in vitro evidence appeared to be a post-hoc decision. This decision warranted further explanation as the potential antiviral and antioxidant actions of NAC may not be precisely demonstrated in petri dishes and may be more clearly demonstrated in studies of humans with viral illnesses, irrespective of viral species. A registered protocol articulating the methodological rationale of this RR, coupled with a more robust search strategy and reporting of the results, would have improved the reproducibility and transparency of the findings. As it stands, the impact of these type 2 errors may include prematurely dismissing further investigation of NAC as a potential adjuvant treatment for SARS-CoV-2 infections.

5. Rapid review recommendations

In light of the above, in addition to the excellent guidance from the Cochrane Collaboration and others,2., 5., 38. we suggest those conducting a RR of TCIM carefully consider the following to optimise the rapidity, rigour and credibility of their reviews.

Generally, all RRs make some methodological sacrifices to ensure timely publication. It is important to note that these methodological shortcomings will be amplified in the RR setting, and it is therefore critical that a review follows an exact method, which requires careful planning. A structured protocol, even a basic protocol as recommended by CEBM,² will ultimately save time, reduce errors, and minimise bias. It will also allow for transparency and reproducibility which are essential for scientific credibility. Accompanying the ‘Interim Guidance from the Cochrane Rapid Reviews Methods Group’ is a RR protocol template that includes prompts for commonly used methodological compromises.¹

Typically, RR are better suited to narrow research questions applied to limited populations, interventions, comparisons and outcomes (PICO).⁴ Broader questions and complex TICM interventions are more difficult, due the additional challenges with constraining search results and the need for a lengthier, detailed analysis conducted by content experts. Due to the paucity of TCIM evidence on a topic, rather than publishing an ‘empty review’, reviewers may decide to expand the inclusion criteria to include indirect evidence (e.g. all acute viral respiratory infections instead of only SARS-CoV2 infections).¹⁵ However, this may also result in too many studies to rapidly appraise. Additionally, such an approach may not be worthwhile as the final certainty/quality of the evidence will be downrated nevertheless, as it cannot be assumed that the evidence applies to SARS-CoV-2.

Prior to finalising the study design a quick scope of the literature is therefore recommended. This will help inform the search strategy, the types of studies to be included and whether a meta-analysis is possible and/or warranted. If there are a large number of potentially eligible primary studies and high-quality SRs have already been published, it might be worth considering an Overview (i.e. umbrella review of SRs),³⁷ or a best evidence synthesis (BES).¹ The BES approach begins with a search for SRs followed by a search for primary studies published after the last SR.¹ Methodological challenges to be aware of in undertaking Overviews and BES include dealing with the issue of overlap when a primary study is included in multiple SRs, and attempting to assess the methods and quality of primary studies included in a SR when limited information is reported.³⁷

Most of the methodological recommendations and minimum standards for RRs1, 2., 5. apply to TCIM, however, some may not be appropriate. Language restrictions might not be acceptable for TCIM interventions when the modality originated in and/or the research has primarily been undertaken in a country where English is not an official language.45., 46. Databases other than the minimum recommended MEDLINE, Embase and the Cochrane Central Register of Controlled Trials (CENTRAL) may also need to be searched.¹ Whilst these three databases are likely to be sufficient for non-TCIM interventions, unless the TICM therapy has recently been reviewed in a Cochrane SR that was not limited by language, key studies may still be missed.45., 46. Consider using platforms such as Ovid, EBSCO Host and the Pan American Health Organization (PAHO) Virtual Health Library (VHL) that can simultaneously search multiple databases relevant to TCIM.

Reviewers should be cautious about using rapid appraisal tools such as the Critical Appraisal Skills Program (CASP) checklist for risk of bias,⁴⁷ as these methods are less robust than standard tools. Such a shortcut may negatively impact the credibility of the review findings. Instead, consider highly regarded tools such as the recently released RoB-2 that is used to individually appraise each outcome reported in a study,¹⁶ and then apply other limitations, such as only appraising the outcomes that are listed a priori in the RR as critically important, or only appraising the primary outcome of the included study.

An experienced research team who have previously collaborated, will help expedite the review.² At least one TCIM content expert is recommended. Including experienced reviewers fluent in languages other than English enables a broader database search, and saves time and money translating articles when screening and extracting data. Engaging a research librarian with SR experience and a thorough understanding of the TCIM research databases is extremely valuable.⁴⁸ Optimising the sensitivity and specificity of the search terms will minimise the time spent screening articles, and reduce the risk of missing key studies.

In conclusion, RRs of TCIM must still strive to optimise rigour, reproducibility and transparency. Otherwise, any positive findings may be discredited nonetheless, and negative findings may be accepted without further critique. Above all, avoid ‘spinning’³⁹ the findings, objectively report the results and explicitly discuss the potential biases that arise from any RR limitations that have been employed.⁴⁹

In line with the guidelines from leading evidence review bodies that are articulated in detail elsewhere,1, 2., 5. recommended minimum requirements for RRs of TICM therefore include:

• 1.A basic protocol that is submitted for publication or made publicly available prior to commencing the review, with any post-hoc changes declared when reporting the RR.
• 2.A description of the methods, including explicit statements outlining any RR constraints
• 3.Search results, ideally including a PRISMA flow diagram.
• 4.A structured, critical appraisal of the included studies, ideally using standardised tools.
• 5.A meta-analysis, if possible, of at least the most important (critical) outcomes of interest.
• 6.The limitations the RR methods should be clearly acknowledged and accompanied by less authoritative evidence statements and remarks.
• 7.In the case of brief reports and plain language summaries only, either an appendix/addendum and/or an additional, publicly available document that reports additional key information on the methods and results.

If the above is not possible, with the exception of not publishing a protocol, it should not be called a rapid review (nor a restricted review), it is either a non-systematic literature review or an evidence informed commentary/review article.

6. Perspective

Systematic reviews of the research literature minimise bias and increase precision of reported effect estimates. This can be invaluable to policy makers and clinicians. Unfortunately, they are resource intensive and in the context of a rapidly evolving global pandemic, may not be a time sensitive way to synthesise evidence. In response to this concern, rapid reviews have been presented as a way of balancing the rigour and rapidity needed when there is an urgent need for high quality and timely evidence synthesis. While guidance for rapid reviews has been provided by authoritative organisations, unique issues exist in the field TCIM that need to be considered, such as strong confirmation biases, a higher prevalence of foreign language evidence sources, and unique TCIM databases. Ignoring these issues may lead to an unintended imbalance favouring rapidity over rigour if minimum requirements for RRs of TICM are not followed. It is for these reasons we recommend reinforcing and augmenting current RR guidance to maintain an acceptable balance in any rapid reviews of TCIM.

Author contribution

Conceptualisation: JH, SM. Methodology: JH, SA, JG, JB, GY, ML SM. Writing – Original Draft: JH Writing – Review & Editing: SA, JG, JB, GY, ML, SM.

Conflict of interest

The authors have no conflicts of interest. This review was not undertaken as part of a contractual relationship with any donor or sponsor.

Funding

This work received no grant from any funding agency.

Ethical statement

This research did not require an ethical approval as it does not involve any human or animal experiment.

Data availability

The data will be made available upon request.