Abstract
Abstract
Background
Increasing evidence has emerged for traditional, complementary and integrative medicine (TCIM) to treat COVID-19 which requires systematic summaries of the net benefits of interventions against standard care and one another. The study aims to conduct a systematic review and network meta-analysis (NMA) regarding TCIM therapies for treating mild/moderate acute COVID-19, potentially informing the WHO guideline development and clinical decision-making.
Methods and analysis
We will search Embase, MEDLINE, Virtual Health Library on Traditional Complementary and Integrative Medicine, Cochrane Central Register of Controlled Trials, WHO’s International Clinical Trials Registry Platform with additional searches of Chinese and Korean literature sources for randomised controlled trials comparing TCIM with placebo, standard care, no treatment or with an alternate type of TCIM to treat COVID-19. We will limit the search dates from 2020 to 28 March 2024. Reviewers will, in duplicate, screen eligible articles, extract data and evaluate the risk of bias. We will conduct frequentist network meta-analyses for network comparisons and each outcome. We will assess the certainty of evidence using the Grading of Recommendations Assessment, Development and Evaluation approach and classify interventions from most to least effective or harmful.
Ethics and dissemination
This NMA is based on the existing trials and data which is not subject to ethical approval. We will publish the NMA in a peer-reviewed journal. This may provide methodological reference and clinical evidence for TCIM on future epidemic-prone diseases.
PROSPERO registration number
CRD42024517321.
Keywords: Complementary Medicine, Network Meta-Analysis, Protocols & guidelines, COVID-19
STRENGTHS AND LIMITATIONS OF THIS STUDY.
Our multidisciplinary team ensures the evidence synthesis processes have the highest methodological quality and high clinical relevance.
A group of information specialists will develop a comprehensive search strategy.
We will consider both the effect estimates and the certainty of the evidence.
We will apply Grading of Recommendations Assessment, Development and Evaluation’s approach to classify interventions from the most to the least effective, making the results easier to interpret.
Background
As of March 2024, there have been over 774 million confirmed COVID-19 cases and more than seven million deaths globally.1 Patients have shown increased interest in using traditional, complementary and integrative medicine (TCIM) globally as 170 member states of the WHO report using TCIM.2 Among 10 000 registered COVID-19 randomised controlled trials (RCTs), more than 1000 are focused on TCIM interventions.3 Thus, TCIM represents a vital category of COVID-19 treatment strategies.
According to a systematic review of observational studies, 64% (with a 95% CI of 54% to 73%) of the population in over 20 countries of North America, Europe, Asia, Africa and Australia/Oceania used TCIM in an effort to prevent or treat COVID-19.4 Developing countries experience shortages and restricted access to healthcare resources and facilities and TCIM intervention might help.5,7 Alternately, use of ineffective TCIM interventions diverts resources and may delay evidence-based care. In developed countries such as the USA, dissatisfaction with conventional healthcare services can also lead to the high usage of TCIM.8 However, up-to-date systematic summaries of the net benefit of TCIM interventions against standard care and one another are lacking.
Facing a rapidly growing body of evidence and colossal demand, clinicians and the public need trustworthy guidance regarding TCIM for COVID-19. Trustworthy guidelines for COVID-19 need methodologically rigorous and timely updated evidence summaries. Such summaries will identify evidence-based TCIM interventions to inform decision-making between clinicians and patients.
We plan to collaborate with WHO and the MAGIC Evidence Ecosystem Foundation on the Clinical Practice Guideline (CPG). We will conduct a systematic review (SR) and network meta-analysis (NMA) to examine the benefits and harms of TCIM interventions for the treatment of mild/moderate acute COVID-19. The evidence summary will potentially inform WHO guidelines and subsequently be published on the MAGICapp platform (www.magicapp.orsg).
Methods
We drafted the protocol according to Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols 20159 and registered the research in PROSPERO.
Structure and organisation
The evidence synthesis team will carry out the project with input from the NMA Steering Committee:
Evidence synthesis team
The team includes clinicians, experienced methodologists and junior team members, they will perform the following processes of evidence synthesis in sequence: (1) study identification, (2) data collection and management, (3) data analysis and (4) grading and presentation.
NMA Steering Committee
This group comprises TCIM clinical experts, systematic review and NMA methodologists and statisticians. The NMA steering group will meet regularly and make critical decisions to ensure the highest methodological quality of the evidence synthesis process and clinical relevance.
Guideline panel involvement
If the WHO decides to develop a CPG informed by the NMA, the NMA team will collaborate with a guideline development group (GDG) convened by the WHO Steering Committee. The GDG will consist of clinicians with expertise in different types of TCIM and experiences in treating COVID, methodologists and three patient partners who will be globally representative and free of any conflicts of interest. They will: (1) confirm the outcome measures, (2) direct the performance of the final subgroup analyses and meta-regression and (3) inform thresholds for adjudicating imprecision.
Patient and public involvement
The panel, if formed, will include three patient partners who have lived experience with COVID-19. They will be involved in identifying and prioritising outcomes and informing their preferences and values.
Literature search
We will search Embase, MEDLINE, Virtual Health Library on Traditional Complementary and Integrative Medicine, Cochrane Central Register of Controlled Trials, WHO’s International Clinical Trials Registry Platform and ClinicalTrials.gov. We will conduct the final search on 28 March 2024 with a date restriction from 2020 onwards applied (online supplemental file 2). We will additionally search Chinese literature sources including Wanfang, Chinese Biomedical Literature, China National Knowledge Infrastructure, Weipu Database for Chinese Technical Periodicals, Chinese Clinical Trial Registry, Chinese Medical Journal Net (preprints) and ChinaXiv (preprints); and Korean literature sources including Research Information Sharing Service, Korea Medical Database, ScienceON and Korean studies Information Service System. We will also employ the snowballing method to identify other potentially eligible studies from the reference lists of already identified RCTs or SRs.
Eligibility criteria
Studies are eligible if they include patients with suspected, probable or confirmed infection with SARS-CoV-2 without age restrictions in any setting. We will include RCTs that compare any TCIM intervention with no intervention, placebo, another TCIM intervention, conventional medicine or standard care and report at least one outcome of interest. Trials can be published in press or preprints with no language restrictions. We will classify disease severity into mild/moderate COVID-19 and severe/critical COVID-19 as reported in RCTs (according to the WHO categorisation10) and this NMA will focus on mild/moderate COVID-19. Studies will be eligible if they include more than 80% of patients with mild/moderate COVID-19 or include a mixed population but reported patients with mild/moderate COVID-19 separately as a subgroup. Studies are also eligible if we could determine the severity of the included patients as mild/moderate from the description of clinical manifestations or the settings.
TCIM interventions (online supplemental file 1) include traditional medicine and practices, natural health products, mind-body therapies, physical therapies and manual therapies and further complex non-traditional medicine treatments.
We will exclude vitamins, minerals, probiotics, melatonin and active ingredients derived from herbs or botanicals (eg, curcumin, quercetin) which are included in another living NMA11 on COVID-19 treatment. We will also exclude highly purified or chemically modified botanical substances and drugs without clear descriptions of compositions (eg, a self-made traditional Chinese medicine decoction without included herbs). We will exclude trials evaluating plasma therapy, external organ support, gas therapy, ventilation strategies, vaccination, psychological and educational interventions or personal protective equipment interventions alone. Quasi-randomised trials will also be excluded.
The majority of outcomes of interest will be consistent with current WHO guidelines12 13 and SRs and NMAs.11 14 If we end up with WHO guidelines, the GDG will finalise the outcomes. Reviewers will extract the outcomes measured closest to the preset time point. Candidate outcomes for the treatment of mild/moderate COVID-19 include:
Mortality (time frame: closest to 90 days).
Need for mechanical ventilation (mechanical ventilation in patients not initially mechanically ventilated, time frame: closest to 90 days).
Duration of hospitalisation.
Admission to the hospital (time frame: closest to 28 days).
Adverse effects leading to discontinuation of the intervention (time frame: closest to 28 days).
Time to all symptom resolution (as reported by the authors).
Quality of life (time frame: at the end of the treatment).
New non-SARS-CoV-2 infection.
Time to viral clearance (time until the amount of SARS-CoV2 viral particles is below the threshold set by the RCT authors for being likely no longer infectious).
Undetectable SARS-CoV-2 by PCR (time frame: closest to 7 days and not less than 4 days or more than 10 days).
The rate of conversion to severe cases (time frame: closest to 90 days).
Time to specific symptom resolution (as reported by the authors).
Prevalence of single symptom closest to 7 days (time frame: closest to 7 days and not less than 4 days or more than 10 days).
Study screening, data abstraction and risk of bias assessments
Reviewers, trained in and experienced in systematic review methods, will work in pairs, independently and in duplicate to conduct title, abstract, full-text screening, data abstraction and risk of bias assessment. Please refer to figure 1 for the study screening process.
Figure 1. Flow diagram of the study screening process.
For eligible studies, reviewers will use a standardised and pilot-tested extraction form to collect the information listed in Box 1.
Box 1. Data Collection.
First author and publication year.
Publication status (peer-reviewed publication/preprint).
Study status (completed/completed, terminated early/ongoing, interim data/ongoing, no data).
Trial registration.
Source of funding.
Geographical location(s).
Patient type (children/adults/elderly/pregnant women).
Type of care (outpatient/inpatient/intensive care).
Number of randomised participants.
Mean age.
Male proportion.
Vaccination status.
Type of virus (Alpha, Beta, Delta, Omicron, etc)
Infection status (suspected/probable, lab-confirmed).
Number of patients randomised to each group at baseline.
Number of patients analyzed in each group.
Treatment name.
Description of treatment (components, dose, frequency, route of administration and duration).
Comorbidities.
Outcomes of interest (reviewers will extract outcome data measured closest to the preset time point and according to the intention-to-treat principle).
To evaluate the risk of bias of included RCTs, reviewers will use a new, more user-friendly instrument,15 Risk of Bias Instrument for Use in Systematic Reviews—for RCTs (under user testing). Reviewers will judge the risk of bias associated with sequence generation, allocation concealment, blinding of participants, blinding of healthcare providers, blinding of outcome assessors and rate these items as ‘definitely low’, ‘probably low’, ‘probably high’ and ‘definitely high’. We will also extract the number of participants lost to follow-up in each group. We will only consider the overall bias of the trial to be high if at least one item is of definitely a high or probably high risk of bias.
All subjective steps will include calibration exercises that warrant accuracy, agreement monitoring and discussion and external adjudication when necessary.
Data synthesis and statistical analysis
Prior to statistical analysis, the NMA Steering Committee including TCIM experts who were blinded to the study results will review and categorise all the interventions for pooled pairwise analyses and into different nodes in the NMA.
For dichotomous outcomes, we will estimate the relative effects using relative risks and their 95% CIs and calculate the interventions’ absolute effects. For continuous outcomes vary largely across populations, we will estimate the relative effects using the ratio of means and their 95% CIs. We will calculate mean differences (MDs) and their 95% CIs for the post-intervention scores of other continuous outcomes. When studies use different instruments to measure general quality of life, we will transform them into the most commonly used instrument to capture their common construct.16 17 We will use continuity correction to account for zero event.18
For direct comparisons of all patient-important outcomes, when there are at least two trials, we will conduct DerSimonian-Laird random-effects19 pairwise meta-analyses. In addition, for outcomes not feasible to run NMA, we will also conduct pairwise meta-analyses for direct comparisons. For each outcome, we will check network connectivity, also ensure the availability of more trials than a number of intervention nodes or having 10 or more trials contribute data to the network. When feasible, we will conduct frequentist random-effects network meta-analyses under the assumption of common heterogeneity.
To prevent highly sparse networks, for the primary analysis, we will include treatment nodes with at least 100 participants or 20 events. Although we might not include all available TCIM interventions in the NMA, we will systematically review and report the results. Even so, when data are sparse, NMA using a random-effects model may produce counter-intuitive results (ie, although the direct and indirect estimates are coherent, the NMA may yield misleadingly wide CIs, even wider than either of them, thus decreasing the precision of NMA). This is due to the use of common heterogeneity estimates in a contrast-based model network meta-analysis and thus our analyses will use a fixed effect model.20
When the network remains largely star-shaped (that is, few, if any interventions compared with other interventions rather than to placebo or standard care), the loop-specific incoherence between direct and indirect evidence is impossible to detect statistically. We will integrate the data into a network meta-analysis under the assumption of coherence and the best estimates may come from direct evidence or indirect evidence.21 We will make decisions regarding the analysis that produces optimal estimates for individual interventions on a case-by-case basis. We will perform relevant analyses with the ‘Network’ suite in Stata (V.17.0, StataCorp, College Station, Texas, USA).
Subgroup and sensitivity analysis
We will perform a limited number of prespecified subgroup analyses to detect potential effect modifiers. We plan to explore how trials’ characteristics (low vs high risk of bias), patients’ characteristics (vaccinated vs not vaccinated), infection status (suspected or probable vs lab-confirmed) and publication status (preprints vs peer-reviewed) can influence the magnitude of the effect. We hypothesise, a priori, that treatments will yield a larger effect in trials with a high risk of bias among unvaccinated patients, among suspect or probable cases and in preprint publications. We will perform subgroup analysis only if at least two studies contribute to the subgroup. We will assess the credibility of apparent effect modification (p for interaction<0.10) using the Instrument for assessing the Credibility of Effect Modification Analyses tool22 and applying Grading of Recommendations Assessment, Development and Evaluation (GRADE) to evaluate subgroup effect estimates when effect modifications are of high or moderate credibility while evaluating overall effect estimates when effect modifications are of low or very low credibility.
If primary studies face the high risk of bias that missing outcome data may bring, we will use available participants’ outcome data as the primary analysis and perform sensitivity analysis by imputation of missing values. For binary outcomes, we will use ‘plausible worst case’, that is, for the outcome of interest of undetectable SARS-CoV-2 by PCR, we assume the incidence of outcome events in participants with missing data relative to those followed successfully is 0.2 in the treatment group; for other binary outcomes, we assume the incidence is 5. And in the control group, we assume the same event rate in participants with missing outcome data as those with available data. For continuous outcomes, we will use the worst mean among the control arms to replace the missing value in the intervention arm and the best mean among the intervention arms to replace the missing value in the control arm.23 We will use the command ‘metamiss2’ in Stata for sensitivity analysis.24
Certainty of evidence
We will use the GRADE approach to assess the certainty of evidence (CoE) for each comparison and outcome in the NMA and rate the certainty as high, moderate, low or very low.25 We will start with high certainty for direct evidence and rate it down taking into account the risk of bias, inconsistency, indirectness, imprecision and publication bias. We will evaluate heterogeneity among trials in each comparison using the I2 statistic, with I2≥50% indicating a substantial heterogeneity and also consider the context when assessing inconsistency.26 For comparisons with at least 10 studies, we will assess publication bias using Egger’s test for continuous outcomes and Harbord’s test for dichotomous outcomes;27 28 in addition, we will report potential publication bias if trials registered, completed but have no corresponding publications or reports in 1 year of trial completion.29
When intransitivity is observed, we will rate down the CoE for indirect evidence. To evaluate the transitivity assumption of indirect comparisons, we will compare distributions of potential effect modifiers (eg, age, vaccination status, type of virus, co-interventions, risk of bias) across study groups using NMA-studio web application (https://www.nmastudioapp.com).30 We will also consult our clinical experts in the Steering Committee to make sure the participants in the network share similar demographic and clinical profiles that they can be jointly randomised to any treatment included in the network.31
When direct and indirect evidence contributes the same, we rate the certainty of network estimates as the higher of the two. We take the dominance of the two when they do not contribute equally. We can further rate down the certainty of network estimate for incoherence or imprecision with judgement supported by thresholds chosen by the Steering Committee (or potentially the guideline panel). We will assess the coherence assumption for the entire network using the ‘design-by-treatment’ model (global test).32 To assess local (loop-specific) incoherence and obtain indirect estimates, we will use the node-splitting method.33 34 When incoherence is detected, we will choose the best estimates from direct or indirect evidence based on their contribution weight and CoE and if direct and indirect evidence weigh equally, we will select a network estimate and rate down its certainty. We will not rate down CoE two levels for intransitivity and inconsistency simultaneously. We will use GRADE summary of findings tables to summarise the results.
Results interpretation
To better interpret the results and support the guideline development, we will use risk differences for binary outcomes and mean differences with their 95% CIs for continuous outcomes to see if they achieve the clinically important difference deemed by the Steering Committee and potentially the guideline panels in the future. When available, we will use the baseline risk from large and methodological rigorous systematic reviews or other representative observational data. For example, for mortality, we could use CDC’s data on hospitalised patients to estimate the baseline risks35 and for mechanical ventilation or length of hospital stay, we could use the International Severe Acute Respiratory and Emerging Infection COVID-19 database to identify appropriate baseline risks.36 If we cannot identify an appropriate study, we will use the median of all included studies contributing to the standard of care node to calculate the baseline risks.
Regarding the target of certainty ratings, we will consider both the effect estimates and the certainty of the evidence and apply GRADE’s approach to classify interventions from the most to the least effective using the standard of care as the reference intervention.37
Updating
After producing the first version of the NMA, we will evaluate the feasibility of making it living. If feasible, the systematic review group will make the review living including search updating, data extraction and risk of bias assessment for eligible studies, data analysis and CoE evaluation every 6 months. If the WHO wants to use the living NMA to inform their guideline, once the WHO Steering Committee or GDG considers the emerging information likely to change practice, we will promptly respond to their requests and update the analyses. During the updating process, if the systematic review team agrees to modify analysis methods, results, etc, to accommodate new data, we will document all changes in the methods section of the updated reviews.
All updated versions will be dated and detailed changes will be recorded and then submitted to international indexes through the mechanism of corrections. The first edition and the following updates will be labelled separately.
Ethics and dissemination
Since our review is based on available evidence, ethical approval is not required. We will publish the NMA through traditional publication platforms, peer reviewed journals and potentially, the MAGICapp (https://app.magicapp.org), a novel electronic format for evidence summaries and decision aids. In addition to scientific publications, if the NMA becomes living, we will create a landing page/portal.38
supplementary material
Acknowledgements
The authors would like to thank Dr Janet Victoria Diaz, Dr Jamie Rylance and Julie Viry from the WHO for their helpful discussions on topics related to this research and their advice on WHO processes. We also thank Dr Kyeore Bae from National Institute of Korean Medicine (NIKOM), HAN Eunkyung from WHO Western Pacific Regional Office, Abdala, Carmen Veronica Mendes from BIREME—Pan American Health Organization and Jesus Lopez from University Hospital Zurich, Institute for Complementary and Integrative Medicine for their support in building the search.
Footnotes
Funding: This study is funded by Canadian Institutes of Health Research (Application Number: 488412); China’s ‘National Ten Thousand Talents Programme (National Top-notch Young Professionals)’ (QHe); China Academy of Chinese Medical Sciences-Special Programme for Cultivating Outstanding Young Scientific and Technological Talents (No. CI2023D004); Innovation Team and Talents Cultivation Programme of National Administration of Traditional Chinese Medicine (grant number ZYYCXTD-C-202008); Shanghai Oriental Talent Programme (LK).
Prepub: Prepublication history and additional supplemental material for this paper are available online. To view these files, please visit the journal online (https://doi.org/10.1136/bmjopen-2024-088959).
Provenance and peer review: Not commissioned; externally peer reviewed.
Patient consent for publication: Not applicable.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Contributor Information
Anqi Li, Email: anqi3316@gmail.com.
Gordon Guyatt, Email: guyatt@mcmaster.ca.
Derek Chu, Email: derek.chu@medportal.ca.
Lehana Thabane, Email: thabanl@mcmaster.ca.
Jason W Busse, Email: bussejw@mcmaster.ca.
Behnam Sadeghirad, Email: sadeghb@mcmaster.ca.
Qingyong He, Email: heqingyongg@163.com.
Long Ge, Email: gelong2009@163.com.
Qiukui Hao, Email: haoqiukui@gmail.com.
Sunita Vohra, Email: svohra@ualberta.ca.
Benno Brinkhaus, Email: benno.brinkhaus@charite.de.
Claudia M Witt, Email: claudia.witt@charite.de.
Yachan Li, Email: liyac@who.int.
Geetha Krishnan, Email: gopalakrishnag@who.int.
Thomas Agoritsas, Email: thomas@magicevidence.org.
Lisa Susan Wieland, Email: lswieland@gmail.com.
Sarika Chaturvedi, Email: sarikabharat2005@gmail.com.
Kavita U Kothari, Email: kkothari@who.int.
Bhushan Patwardhan, Email: bpatwardhan@gmail.com.
Min Fang, Email: fm-tn0510@shutcm.edu.cn.
Lingjun Kong, Email: chunyong01@163.com.
Jun Ren, Email: tnrenjun@163.com.
Myeong Soo Lee, Email: drmslee@gmail.com.
Angela Cheung, Email: angela.cheung@uhn.ca.
Patrick M Okwen, Email: okwen@ebaseafrica.org.
Bernice Downey, Email: downebe@mcmaster.ca.
Lin Ang, Email: anglin2808@kiom.re.kr.
Jialing Zhu, Email: zhujialing@gzucm.edu.cn.
Yuqing Zhang, Email: madisonz1220@gmail.com.
References
- 1.World Health Organization COVID-19 epidemiological update– 15 March 2024. 2024. https://www.who.int/publications/m/item/covid-19-epidemiological-update-15-march-2024 Available.
- 2.World health organization WHO global report on traditional and complementary. 2019. https://www.who.int/publications/i/item/978924151536 Available.
- 3.World Health Organization International clinical trials registry platform (ICTRP) 2024. https://www.who.int/clinical-trials-registry-platform Available.
- 4.Kim T-H, Kang JW, Jeon S-R, et al. Use of Traditional, Complementary and Integrative Medicine During the COVID-19 Pandemic: A Systematic Review and Meta-Analysis. Front Med (Lausanne) 2022;9:884573. doi: 10.3389/fmed.2022.884573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Alyami HS, Orabi MAA, Aldhabbah FM, et al. Knowledge about COVID-19 and beliefs about and use of herbal products during the COVID-19 pandemic: A cross-sectional study in Saudi Arabia. Saudi Pharm J. 2020;28:1326–32. doi: 10.1016/j.jsps.2020.08.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Da’ar OB, Haji M, Jradi H. Coronavirus Disease 2019 (COVID-19): Potential implications for weak health systems and conflict zones in the Middle East and North Africa region. Int J Health Plann Manage. 2020;35:1240–5. doi: 10.1002/hpm.2982. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Usher AD. The global COVID-19 treatment divide. The Lancet. 2022;399:779–82. doi: 10.1016/S0140-6736(22)00372-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Paudyal V, Sun S, Hussain R, et al. Complementary and alternative medicines use in COVID-19: A global perspective on practice, policy and research. Res Soc Admin Pharm. 2022;18:2524–8. doi: 10.1016/j.sapharm.2021.05.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;350:g7647. doi: 10.1136/bmj.g7647. [DOI] [PubMed] [Google Scholar]
- 10.Geneva: World Health Organization; 2023. Clinical management of covid-19: living guideline. [PubMed] [Google Scholar]
- 11.Siemieniuk RA, Bartoszko JJ, Zeraatkar D, et al. Drug treatments for covid-19: living systematic review and network meta-analysis. BMJ. 2020;370:m2980. doi: 10.1136/bmj.m2980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Lamontagne F, Agarwal A, Rochwerg B, et al. A living WHO guideline on drugs for covid-19. BMJ. 2020;370:m3379. doi: 10.1136/bmj.m3379. [DOI] [PubMed] [Google Scholar]
- 13.Lamontagne F, Stegemann M, Agarwal A, et al. A living WHO guideline on drugs to prevent covid-19. BMJ. 2021;372:526. doi: 10.1136/bmj.n526. [DOI] [PubMed] [Google Scholar]
- 14.Bartoszko JJ, Siemieniuk RAC, Kum E, et al. Prophylaxis against covid-19: living systematic review and network meta-analysis. BMJ. 2021;373:949.:n949. doi: 10.1136/bmj.n949. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Wang Y, Ghadimi M, Wang Q, et al. Instruments assessing risk of bias of randomized trials frequently included items that are not addressing risk of bias issues. J Clin Epidemiol. 2022;152:218–25. doi: 10.1016/j.jclinepi.2022.10.018. [DOI] [PubMed] [Google Scholar]
- 16.Guyatt GH, Thorlund K, Oxman AD, et al. GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes. J Clin Epidemiol. 2013;66:173–83. doi: 10.1016/j.jclinepi.2012.08.001. [DOI] [PubMed] [Google Scholar]
- 17.Thorlund K, Walter SD, Johnston BC, et al. Pooling health-related quality of life outcomes in meta-analysis-a tutorial and review of methods for enhancing interpretability. Res Synth Methods. 2011;2:188–203. doi: 10.1002/jrsm.46. [DOI] [PubMed] [Google Scholar]
- 18.Bhaumik DK, Amatya A, Normand S-L, et al. Meta-Analysis of Rare Binary Adverse Event Data. J Am Stat Assoc. 2012;107:555–67. doi: 10.1080/01621459.2012.664484. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88. doi: 10.1016/0197-2456(86)90046-2. [DOI] [PubMed] [Google Scholar]
- 20.Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol. 2019;105:60–7. doi: 10.1016/j.jclinepi.2018.08.022. [DOI] [PubMed] [Google Scholar]
- 21.Yoon JH, Dias S, Hahn S. A method for assessing robustness of the results of a star-shaped network meta-analysis under the unidentifiable consistency assumption. BMC Med Res Methodol. 2021;21:113. doi: 10.1186/s12874-021-01290-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Schandelmaier S, Briel M, Varadhan R, et al. Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses. CMAJ. 2020;192:E901–6. doi: 10.1503/cmaj.200077. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Guyatt GH, Ebrahim S, Alonso-Coello P, et al. GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence. J Clin Epidemiol. 2017;87:14–22. doi: 10.1016/j.jclinepi.2017.05.005. [DOI] [PubMed] [Google Scholar]
- 24.Chaimani A, Mavridis D, Higgins JPT, et al. Allowing for informative missingness in aggregate data meta-analysis with continuous or binary outcomes: Extensions to metamiss. Stata J. 2018;18:716–40. [PMC free article] [PubMed] [Google Scholar]
- 25.Izcovich A, Chu DK, Mustafa RA, et al. A guide and pragmatic considerations for applying GRADE to network meta-analysis. BMJ. 2023;381:e074495. doi: 10.1136/bmj-2022-074495. [DOI] [PubMed] [Google Scholar]
- 26.Guyatt G, Zhao Y, Mayer M, et al. GRADE guidance 36: updates to GRADE’s approach to addressing inconsistency. J Clin Epidemiol. 2023;158:70–83. doi: 10.1016/j.jclinepi.2023.03.003. [DOI] [PubMed] [Google Scholar]
- 27.Egger M, Smith GD, Schneider M, et al. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34. doi: 10.1136/bmj.315.7109.629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Harbord RM, Egger M, Sterne JAC. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med. 2006;25:3443–57. doi: 10.1002/sim.2380. [DOI] [PubMed] [Google Scholar]
- 29.Guyatt GH, Oxman AD, Montori V, et al. GRADE guidelines: 5. Rating the quality of evidence—publication bias. J Clin Epidemiol. 2011;64:1277–82. doi: 10.1016/j.jclinepi.2011.01.011. [DOI] [PubMed] [Google Scholar]
- 30.Cochrane colloquium . London, UK: 2023. A fully interactive web-application for producing and visualising network meta-analyses. [Google Scholar]
- 31.Spineli LM, Kalyvas C, Yepes-Nuñez JJ, et al. Low awareness of the transitivity assumption in complex networks of interventions: a systematic survey from 721 network meta-analyses. BMC Med. 2024;22:112. doi: 10.1186/s12916-024-03322-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Higgins JPT, Jackson D, Barrett JK, et al. Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies. Res Synth Methods. 2012;3:98–110. doi: 10.1002/jrsm.1044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Krahn U, Binder H, König J. A graphical tool for locating inconsistency in network meta-analyses. BMC Med Res Methodol. 2013;13:35. doi: 10.1186/1471-2288-13-35. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Dias S, Welton NJ, Caldwell DM, et al. Checking consistency in mixed treatment comparison meta-analysis. Stat Med. 2010;29:932–44. doi: 10.1002/sim.3767. [DOI] [PubMed] [Google Scholar]
- 35.Prevention CfDCa Provisional death counts for covid-19. 2024. https://www.cdc.gov/nchs/nvss/vsrr/covid19/index.htm Available.
- 36.Consortium ISARaeI ISARIC covid-19 clinical data report. https://www.medrxiv.org/content/10.1101/2020.07.17.20155218v15.full.pdf Available.
- 37.Brignardello-Petersen R, Florez ID, Izcovich A, et al. GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework. BMJ. 2020;371:m3900. doi: 10.1136/bmj.m3900. [DOI] [PubMed] [Google Scholar]
- 38.university M COVID-19 living network meta-analysis. 2022. https://www.covid19lnma.com Available.