Consistency of recommendations and methodological quality of guidelines for the diagnosis and treatment of COVID‐19

Abstract

Objective

Since the beginning of the COVID‐19 epidemic, a large number of guidelines on diagnosis and treatment of COVID‐19 have been developed, but the quality of those guidelines and the consistency of recommendations are unclear. The objective of this study is to evaluate the quality of the diagnosis and treatment guidelines on COVID‐19 and analyze the consistency of the recommendations of these guidelines.

Methods

We searched for guidelines on diagnosis and/or treatment of COVID‐19 through PubMed, CBM, CNKI, and WanFang Data, from January 1, 2020 to August 31, 2020. In addition, we also searched official websites of the US CDC, European CDC and WHO, and some guideline collection databases. We included diagnosis and/or treatment guidelines for COVID‐19, including rapid advice guidelines and interim guidelines. Two trained researchers independently extracted data and four trained researchers evaluated the quality of the guidelines using the AGREE II instruments. We extracted information on the basic characteristics of the guidelines, guideline development process, and the recommendations. We described the consistency of the direction of recommendations for treatment and diagnosis of COVID‐19 across the included guidelines.

Results

A total of 37 guidelines were included. Most included guidelines were assessed as low quality, with only one of the six domains of AGREE II (clarity of presentation) having a mean score above 50%. The mean scores of three domains (stakeholder involvement, the rigor of development and applicability) were all below 30%. The recommendations on diagnosis and treatment were to some extent consistent between the included guidelines. Computed tomography (CT), X‐rays, lung ultrasound, RT‐PCR, and routine blood tests were the most commonly recommended methods for COVID‐19 diagnosis. Thirty guidelines were on the treatment of COVID‐19. The recommended forms of treatment included supportive care, antiviral therapy, glucocorticoid therapy, antibiotics, immunoglobulin, extracorporeal membrane oxygenation (ECMO), convalescent plasma, and psychotherapy.

Conclusions

The methodological quality of currently available diagnosis and treatment guidelines for COVID‐19 is low. The diagnosis and treatment recommendations between the included guidelines are highly consistent. The main diagnostic methods for COVID‐19 are RT‐PCR and CT, with ultrasound as a potential diagnostic tool. As there is no effective treatment against COVID‐19 yet, supportive therapy is at the moment the most important treatment option.

1. INTRODUCTION

It has been almost one year since the Coronavirus Disease 2019 (COVID‐19) outbreak started. As of October 28, 2020, the World Health Organization (WHO) has reported more than 43 million confirmed cases and about 1 163 000 deaths, ¹ which has taken an incalculable toll on several countries. Until now, there is no effective treatment for COVID‐19, and potential vaccines are also only under development. The optimal approach to diagnosis and treatment of COVID‐19 is uncertain. Some medications are so far only recommended for patients with severe COVID‐19 or in the context of clinical trials. The recommendations also differ between sources. For example, the WHO interim guidance ² did not suggest routinely giving systemic corticosteroids for treatment of COVID‐19 outside clinical trials, but another international guideline ³ suggested that corticosteroids should be given to patients with severe COVID‐19 and Acute Respiratory Distress Syndrome (ARDS). Different institutions have recommended differing diagnostic methods for COVID‐19 for different populations, which lead to confusion in their clinical use.

High‐quality clinical practice guidelines can regulate the diagnosis and treatment behavior of health providers and improve the quality of medical services. Guidelines are also equally important for public health emergencies. After the outbreak of COVID‐19, several institutions including WHO, National Health Commission (China), Chinese Center for Disease Control and Prevention (CCDC), the US Centers for Disease Control and Prevention (US CDC), European Center for Disease Control and Prevention (ECDC), as well as several scientific/professional associations, societies and hospitals, developed a large number of clinical practice guidelines including rapid advice guidelines and interim guidelines on diagnosis and treatment of COVID‐19.^4,5 However, due to the urgency of the situation, the quality of those guidelines and the consistency of recommendations are unclear. Previous studies ⁶ , ⁷ have shown that the clinical guidelines of COVID‐19 lacked detail and covered a narrow range of topics. Another study ⁸ suggested inconsistency between some recommendations of some clinical practice guidelines compared with those of the WHO. To our knowledge no study has yet systematically compared the recommendations of COVID‐19 diagnosis and treatment guidelines, for quality and consistency. Therefore, we conducted this study to evaluate the quality of COVID‐19 diagnosis and treatment guidelines developed exclusively for COVID‐19, and compare the similarities and differences in the diagnostic and treatment recommendations of these guidelines.

2. METHODS

We conducted a retrospective cross‐sectional analysis of diagnosis and treatment guidelines on COVID‐19. Before the study began, the reviewers familiarized themselves with the methodological appraisal tool for guidelines via AGREE II ⁹ (Appraisal of Guidelines, for Research, and Evaluation).

2.1. Search strategy

We searched MEDLINE (via PubMed), CBM (China Biology Medicine disc, CBMdisc), CNKI (China National Knowledge Infrastructure), and WanFang Data from January 1, 2020 to August 31, 2020 to identify the diagnosis and treatment guidelines on COVID‐19, using the terms guideline, statement, guidance, recommendation, COVID‐19, 2019‐nCoV, SARS‐CoV‐2, novel coronavirus, novel coronavirus pneumonia. We also searched the official websites of WHO (https://www.who.int/), US CDC (https://www.cdc.gov/), ECDC (https://www.ecdc.europa.eu/en/home), CCDC (http://www.chinacdc.cn), and the National Health Commission of the People's Republic of China (http://en.nhc.gov.cn), as well as Google Scholar and the reference lists of included guidelines. We did not search preprint servers because we wanted to include only final versions of guidelines that are already applied in clinical practice. The search strategy is given in the Supporting material.

2.2. Inclusion and exclusion criteria

We included diagnosis and/or treatment guidelines for COVID‐19, including rapid advice guidelines and interim guidelines. A diagnostic and/or treatment guideline was defined as a guideline that contains diagnostic and/or treatment elements in their recommendations. If the guideline was published in both English language and Chinese language, we only included the English‐language version. If there were multiple versions of the guidelines, we only included the latest one.

We excluded the following types of documents: (1) comprehensive or general guidelines that addressed COVID‐19, but were not exclusive; (2) guidelines published only as a book; (3) proposals of guidelines; (4) guidelines not for humans; (5) translations, interpretations, and abstracts of guidelines; (6) diagnosis or treatment guidelines for disability, sequelae, complications or other symptoms caused by novel coronavirus; (7) diagnosis or treatment guidelines for other diseases caused by SARS‐CoV‐2; (8) Traditional Chinese Medicine (TCM) guidelines; (9) guidelines for which we were unable to retrieve the full text; and (10) guidelines not published in English or Chinese.

2.3. Study selection

We used Endnote X9 for literature storage and screening. Two trained researchers (YL, ML) screened independently first by the titles and abstracts. Two researchers (YL, ML) independently screened records from the official websites manually. Discrepancies were solved by a third researcher (XL). Full text reviewing was done by one researcher (XL).

2.4. Data extraction

For each eligible diagnosis and treatment guideline, we extracted the following data: (1) basic characteristics of the guidelines (eg, title, publication date, country/setting, type of guideline); (2) characteristics of the developer and development process of the guideline (eg, the developer institution or journals, whether a systematic review was conducted, whether the evidence was formally graded, system of evidence grading); and (3) recommendations (diagnosis and treatment details). Two reviewers (XL, ML) independently extracted the data, which were then checked by a third person (YL). Disagreements were solved through discussion or with the help of a fourth author (YC).

2.5. Quality assessment of guidelines

We used the AGREE II tool to evaluate the quality of the included guidelines. The tool consists of 23 key items divided into six domains. Each item is assigned a score from 1 (strongly disagree) to 7 (strongly agree). The score for each domain was obtained by adding up the scores of all the items in the domain (of each assessor) and normalizing them (score obtained – minimum possible score)/(maximum possible score – minimum possible score). Each guideline was independently assessed by four reviewers. We calculated the mean scores for each domain across all guidelines, as well as for guidelines developed in each country, territory or area separately.

2.6. Data analysis

We conducted descriptive analyses of the general characteristics of the included guidelines. We used SPSS 14.0 to calculate the mean quality scores of included guidelines. Intraclass correlation coefficients (ICCs) were calculated to test interrater reliability among the four reviewers on the AGREE II scores. We compared and analyzed the direction of recommendations related to the diagnosis and treatment on COVID‐19. Finally, we summarized the current diagnosis and treatment recommendations for COVID‐19 and examined consistency among recommendations on various diagnostic and therapeutic options.

3. RESULTS

3.1. Search results

A total of 2018 articles were identified, and 37 records ⁴ , ⁵ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ were finally included after successive screening, including 32 guidelines ⁴ , ⁵ , ¹¹ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²² , ²³ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ (86.5%) published in English language and five ¹⁰ , ¹² , ¹³ , ²¹ , ²⁴ (13.5%) in Chinese language (Fig. 1).

FIGURE 1.

Flowchart of study selection process

3.2. Characteristics of the included guidelines

Of the 37 guidelines included, ten ⁴ , ¹⁴ , ²⁰ , ³⁰ , ³² , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ (27.0%) were developed by international panels (Table 1). Nine^{8,10‐13,15,21,24,34,44} of included guidelines were developed in China (24.3%), five ²¹ , ²⁶ , ³⁵ , ³⁶ , ⁴⁴ (13.5%) in the United States, two in Australia ³⁸ , ⁴² (5.4%), two in Canada^{22 43}(5.4%), two in South Korea^17,33 (5.4%), and one each in Germany, ²⁵ India, ¹⁶ Italy, ¹⁹ Mexico, ³¹ North America (USA and Canada), ²⁶ Poland, ²⁹ and Spain. ²⁷ These guidelines were published between January 30 and August 27, 2020.

TABLE 1.

The characteristics of the included diagnosis and treatment guidelines for COVID‐19

Full title	Journal or other publication platform	Publication date	Country, territory or area of development	Development institute or organization	Patients	Type of guideline	Type of evidence basis	Method for evaluating the quality of evidence and strength of recommendations
Rapid advice guideline for the diagnosis and treatment of the novel coronavirus pneumonia infections (3rd Edition) [in Chinese]	Herald of Medicine	30‐Jan‐2020	China	Tongji Hospital of Tongji Medical College, Huazhong University of Science and Technology	Adults and children	Diagnosis and treatment	Existing guidelines, consensus documents	NR
A rapid advice guideline for the diagnosis and treatment of 2019 novel coronavirus (2019‐nCoV) infected pneumonia (standard version)	Military Medical Research	6‐Feb‐2020	China	Center for Evidence‐Based and Translational Medicine, Zhongnan Hospital of Wuhan University	Adults and children	Diagnosis and treatment	Systematic retrieval of currently available evidence	GRADE
Guideline for imaging diagnosis of novel coronavirus (2019‐nCoV) infected pneumonia (1st edition 2020) [in Chinese]	New Medicine	25‐Feb‐2020	China	Chinese Society of Radiology	Adults and children	Diagnosis	Existing guidelines, consensus documents	NR
Rapid screening and practice guidelines for suspected and confirmed cases of novel coronavirus infection/pneumonia in children[in Chinese]	Chinese Journal of Evidence Based Pediatrics	25‐Feb‐2020	China	Children's Hospital of Fudan University	Children with COVID‐19	Treatment	Existing guidelines, consensus documents	GRADE
Clinical management of severe acute respiratory infection (SARI) when COVID‐19 disease is suspected	WHO website	13‐Mar‐2020	International	WHO	Adults and children	Diagnosis and treatment	Systematic retrieval of currently available evidence	Unclear
Diagnosis and clinical management of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2) infection: an operational recommendation of Peking Union Medical College Hospital (V2.0)	Emerging Microbes & Infections	14‐Mar‐2020	China	Peking Union Medical College Hospital	Adults and children	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
Guidelines on clinical management of COVID‐19	India Government website	17‐Mar‐2020	India	Government of India Ministry of Health & Family Welfare Directorate General of Health Services	Adults and children	Diagnosis and treatment	unclear	NR
Guidelines for laboratory diagnosis of coronavirus disease 2019 (COVID‐19) in Korea	Annals of Laboratory Medicine	25‐Mar‐2020	South Korea	Korean Society for Laboratory Medicine, COVID‐19 Task Force and the Center for Laboratory Control of Infectious Diseases, the Korea Centers for Disease Control and Prevention	Adults and children	Diagnosis	Systematic retrieval of currently available evidence	NR
Management of critically ill adults with COVID‐19	The Journal of the American Medical Association	26‐Mar‐2020	International	Surviving Sepsis Campaign	Critically ill adult patients with COVID‐19	Diagnosis and treatment	Systematic retrieval of currently available evidence	GRADE
The Italian coronavirus disease 2019 outbreak: recommendations from clinical practice	Anaesthesia	27‐Mar‐2020	Italy	The European Airway Management Society	Adults and children	Treatment	Systematic retrieval of currently available evidence	NR
Physiotherapy management for COVID‐19 in the acute hospital setting: clinical practice recommendations	Journal of Physiotherapy	30‐Mar‐2020	International	Multidisciplinary international panel	Adults and children	Treatment	Systematic retrieval of currently available evidence	NR
Guidelines for imaging diagnosis of novel coronavirus pneumonia (2nd Edition 2020, shortened version)[in Chinese]	Journal of Capital Medical University	31‐Mar‐2020	China	Committee of the Infectious Diseases Radiology Section of Chinese Medical Doctor Association	Adults and children	Diagnosis	Existing guidelines, consensus documents	NR
Clinical management of patients with moderate to severe COVID‐19 ‐ interim guidance	Canada Government website	2‐Apr‐2020	Canada	Public Health Agency of Canada	Patients with moderate to severe COVID‐19	Treatment	Adapted based on WHO guideline	Unclear
Corticosteroid guidance for pregnancy during COVID‐19 pandemic	American Journal of Perinatology	9‐Apr‐2020	USA	Medical College of Wisconsin	Pregnant women with COVID‐19	Treatment	Systematic retrieval of currently available evidence	NR
Radiological diagnosis of COVID‐19: expert recommendation from the Chinese Society of Radiology (First edition) [in Chinese]	Chinese Journal of Radiology	10‐Apr‐2020	China	Chinese Society of Radiology	Adults and children	Diagnosis	Existing guidelines, consensus documents	NR
German recommendations for critically ill patients with COVID‐19	Medizinische Klinik Intensivmedizin und Notfallmedizin Notfmed	14‐Apr‐2020	Germany	Experts panelist	Critically ill patients with COVID‐19	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
Multicenter initial guidance on use of antivirals for children with COVID‐19/SARS‐CoV‐2	Journal of the Pediatric Infectious Diseases Society	22‐Apr‐2020	North America (USA and Canada)	Experts panelist	Children with COVID‐19	Treatment	Systematic retrieval of currently available evidence	NR
Recommendations on the clinical management of the COVID‐19 infection by the «new coronavirus» SARS‐CoV2	Anales de Pediatría (English Edition)	25‐Apr‐2020	Spain	Spanish Paediatric Association	Children with COVID‐19	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
Infectious Diseases Society of America guidelines on the treatment and management of patients with coronavirus disease 2019 (COVID‐19)	Clinical Infectious Diseases	27‐Apr‐2020	USA	Infectious Diseases Society of America	Adults and children	Treatment	Systematic review	GRADE
Management of SARS‐CoV‐2 infection: recommendations of the Polish Association of Epidemiologists and Infectiologists as of March 31, 2020	Polish Archives of Internal Medicine	30‐Apr‐2020	Poland	Polish Association of Epidemiologists and Infectiologists	Adults and children	Diagnosis and treatment	Existing guidelines, consensus documents	NR
Rapid advice guidelines for management of children with COVID‐19	Annals of Translational Medicine	1‐May‐2020	International	Multidisciplinary international panel	Children with COVID‐19	Diagnosis and treatment	Rapid review	GRADE
Recommendations for the management of critically ill adult patients with COVID‐19	Gaceta Medica de Mexico	14‐May‐2020	Mexico	Unclear	Critically ill adult patients with COVID‐19	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
Treatment of patients with nonsevere and severe coronavirus disease 2019: an evidence‐based guideline	Canadian Medical Association Journal	19‐May‐2020	International	Multidisciplinary international panel	Patients with nonsevere and severe COVID‐19	Treatment	Systematic review	GRADE
Clinical management of COVID‐19 interim guidance	WHO website	27‐May‐2020	International	WHO	Adults and children	Diagnosis and treatment	Rapid review	Revised GRADE
Interim guidelines on antiviral therapy for COVID‐19	Infection & Chemotherapy	1‐Jun‐2020	South Korea	the Korean Society of Infectious Diseases, the Korean Society for Antimicrobial Therapy, and the Korean Society of Pediatric Infectious Diseases	Adults and children	Treatment	Systematic retrieval of currently available evidence	IDSA
Diagnosis and treatment recommendations for pediatric respiratory infection caused by the 2019 novel coronavirus	World Journal of Pediatrics	1‐Jun‐2020	China	the National Clinical Research Center for Child Health, National Children's Regional Medical Center, Children's Hospital, Zhejiang University School of Medicine	Children with COVID‐19	Diagnosis and treatment	Existing guidelines, consensus documents	NR
Recommendations for the prevention and treatment of the novel coronavirus pneumonia in the elderly in China	Aging Medicine (Milton)	9‐Jun‐2020	China	Experts panelist	Elderly patients with COVID‐19	Diagnosis and treatment	Existing guidelines, consensus documents	NR
Infectious Diseases Society of America guidelines on the diagnosis of COVID‐19	Clinical Infectious Diseases	16‐Jun‐2020	USA	Infectious Diseases Society of America	Adults and children	Diagnosis and treatment	Systematic review	GRADE
Interim guidelines for management of confirmed coronavirus disease (COVID‐19) clinical care guidance	US CDC website	30‐Jun‐2020	USA	US CDC	Adults and children	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
COVID‐19 PICU guidelines: for high‐ and limited‐resource settings	Pediatric Research	7‐Jul‐2020	International	Multidisciplinary international panel	Children with COVID‐19	Treatment	Existing guidelines, consensus documents	Unclear
Public Health Laboratory Network Guidance for serological testing in COVID‐19	Australian Government website	20‐Jul‐2020	Australia	Public Health Laboratory Network	Adults and children	Diagnosis	Available evidence	NR
Clinical management of coronavirus disease 2019 (COVID‐19) in pregnancy: recommendations of WAPM‐World Association of Perinatal Medicine	Journal of Perinatal Medicine	21‐Jul‐2020	International	World Association of Perinatal Medicine	Pregnant women with COVID‐19	Diagnosis and treatment	Systematic retrieval of currently available evidence	NR
Use of chest imaging in the diagnosis and management of COVID‐19: a WHO rapid advice guide	Radiology	30‐Jul‐2020	International	WHO	Adults and children	Diagnosis	Systematic review	GRADE
Remdesivir for severe COVID‐19: a clinical practice guideline	The British Medical Journal	30‐Jul‐2020	International	Multidisciplinary international panel	Patients with severe COVID‐19	Treatment	Systematic review	GRADE
PHLN guidance on laboratory testing for SARS‐CoV‐2 (the virus that causes COVID‐19)	Australian Government website	10‐Aug‐2020	Australia	Public Health Laboratory Network	COVID‐19 patients	Diagnosis	unclear	NR
Clinical management of patients with COVID‐19: second interim guidance	Canada Government website	17‐Aug‐2020	Canada	Government of Canada	Adults and children	Treatment	Adapted from a WHO guideline	Reported but unclear
Coronavirus disease 2019 (COVID‐19) treatment guidelines	NIH website	27‐Aug‐2020	USA	National Institutes of Health (NIH)	Adults and children	Diagnosis and treatment	Systematic review	NIH criteria

Abbreviations: COVID‐19, coronavirus disease 2019; GRADE, Grading of Recommendations Assessment, Development and Evaluation; NIH, National Institutes of Health; NR, not reported; US CDC, the United States Centers for Disease Control and Prevention; WHO, World Health Organization.

This article is being made freely available through PubMed Central as part of the COVID-19 public health emergency response. It can be used for unrestricted research re-use and analysis in any form or by any means with acknowledgement of the original source, for the duration of the public health emergency.

3.3. Quality of the included guidelines

The ICC between the four reviewers for AGREE II assessment in the study was 0.925 (95%CI, 0.907‐0.938). The quality of the most included guidelines was in general low, and the mean score over all guidelines was above 50% in only one domain (clarity of presentation). Three domains (stakeholder involvement, rigor of development and applicability) had a mean score below 30% over the guidelines. The guidelines developed by multicountry international panels had the highest scores, whereas the guidelines from Poland, Germany, and Australia had the lowest scores (Figure 2).

FIGURE 2.

AGREE II scores by domain and area development of the included guidelines. The different sectors represent guidelines developed in different countries or areas, and the colors the six domains of the AGREE‐II tool. The radial coordinates represent the scores of the AGREE‐II tool

3.4. Recommendations for diagnosis

Twenty‐five ⁴ , ⁵ , ¹⁰ , ¹¹ , ¹² , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ²¹ , ²⁴ , ²⁵ , ²⁷ , ²⁹ , ³⁰ , ³¹ , ³⁴ , ³⁵ , ³⁶ , ³⁸ , ³⁹ , ⁴⁰ , ⁴² , ⁴⁴ guidelines (67.6%) provided COVID‐19 diagnosis‐related recommendations, including seven guidelines ¹² , ¹⁷ , ²¹ , ²⁴ , ³⁸ , ⁴⁰ , ⁴² (18.9%) that contained only diagnostic recommendations. Computed tomography (CT), reverse transcription‐polymerase chain reaction (RT‐PCR), X‐rays, lung ultrasound, and routine blood tests were the most frequently recommended tests for COVID‐19 diagnosis (Table 2).

TABLE 2.

Consistency analysis of the diagnostic recommendations of the COVID‐19 guidelines

This article is being made freely available through PubMed Central as part of the COVID-19 public health emergency response. It can be used for unrestricted research re-use and analysis in any form or by any means with acknowledgement of the original source, for the duration of the public health emergency.

CT was recommended for the diagnosis of patients with COVID‐19 in all except three guidelines^5,30,39: one for children, one for pregnant women, and one for adults and children. The recommendation against CT was because of the variability in chest imaging findings, chest radiograph or CT alone was not recommended for the diagnosis of COVID‐19. Two guidelines^12,21 (5.4%) suggested using high‐resolution computed tomography (HRCT) for infants and the elderly. One guideline ²⁴ recommended volume CT scan as the preferred choice for early detection of lesions and assessment of the nature and extent of the lesion.

RT‐PCR testing was used as a preferred testing or conditional recommendation in fourteen guidelines. ⁴ , ¹⁰ , ¹¹ , ¹² , ¹⁵ , ¹⁶ , ¹⁷ , ²⁹ , ³⁴ , ³⁵ , ³⁶ , ³⁹ , ⁴⁰ , ⁴² Of course, it was not recommended in some cases, such as when WHO guidelines suggesting RT‐PCR testing was not available or available, but results were delayed.

Nine guidelines^{5,12,14,15,21,24,27,29,39} (24.3%) referred to X‐rays for the diagnosis of COVID‐19. Six guidelines^{12,14,15,21,27,29} (16.2%) conditionally suggested using X‐rays, as they were often used in primary care with symptoms indicating lung involvement. However, three guidelines^5,24,39(8.1%) did not recommend performing X‐rays since radiologic findings in COVID‐19 were not specific and overlapped with other infections or the high rate of fail to diagnosis.

Lung ultrasound and blood test were recommended in six guidelines ¹⁴ , ¹⁵ , ¹⁶ , ²⁷ , ³¹ , ³⁴ (16.2%). Three guidelines ¹⁴ , ¹⁶ , ³¹ suggested lung ultrasound used in screening and following up of COVID‐19. Three guidelines ¹⁵ , ²⁷ , ³⁴ suggested blood test as the routine test way.

3.5. Recommendations for treatment

A total of 30 guidelines ⁴ , ⁵ , ¹⁰ , ¹¹ , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁸ , ¹⁹ , ²⁰ , ²² , ²³ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁹ , ⁴¹ , ⁴³ , ⁴⁴ (81.1%) addressed the treatment of COVID‐19. Treatment options included supportive care, antiviral therapy, glucocorticoid therapy, antibiotics, immunoglobulin, extracorporeal membrane oxygenation (ECMO), convalescent plasma, and psychotherapy (Table 3).

TABLE 3.

Consistency analysis of the treatment recommendations of the COVID‐19 guidelines

This article is being made freely available through PubMed Central as part of the COVID-19 public health emergency response. It can be used for unrestricted research re-use and analysis in any form or by any means with acknowledgement of the original source, for the duration of the public health emergency.

Recommendations on supportive care were consistent. Twenty guidelines ⁴ , ¹⁰ , ¹¹ , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁸ , ¹⁹ , ²⁰ , ²⁵ , ²⁷ , ²⁹ , ³⁰ , ³¹ , ³⁴ , ³⁵ , ³⁷ , ⁴³ (54.1%) recommended using supportive care, mainly rest in bed, ensuring adequate calories, maintaining water‐electrolyte balance, oxygen therapy, symptomatic treatment, and nutritional supplements.

Recommendations on antiviral therapy varied across the included guidelines. Twenty‐two guidelines ⁴ , ⁵ , ¹⁰ , ¹¹ , ¹³ , ¹⁵ , ¹⁸ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁹ , ⁴¹ , ⁴³ , ⁴⁴ (59.5%) referred to antiviral therapy, whereas 15 guidelines ¹⁰ , ¹¹ , ¹³ , ¹⁵ , ¹⁸ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³³ , ³⁴ , ³⁵ , ³⁹ , ⁴¹ , ⁴³ (40.5%) were consistent in recommending that there were no effective antivirals currently available. Antiviral therapy may be considered for severe or critically ill patients with COVID‐19, or when conducting clinical trials. Antiviral drugs most commonly mentioned in the guidelines were alpha‐interferon, lopinavir/ritonavir, ribavirin, umifenovir, favipiravir, hydroxychloroquine, remdesivir, oseltamivir, chloroquine, and tocilizumab. One international guideline suggested not to use ribavirin, umifenovir, favipiravir, lopinavir‐ritonavir, hydroxychloroquine, interferon‐α and interferon‐β for neither nonsevere or severe patients with COVID‐19, but another Spanish guideline ²⁷ suggested that in severe cases requiring hospital admission, considering initiating treatment with lopinavir/ritonavir, or remdesivir if available, may be useful for severe cases. Meanwhile, the guideline developed by Mexico ³¹ emphasized that there was neither a specific antiviral treatment for patients with COVID‐19, nor any results of controlled trials supporting the use of any. For children, pregnant women and elderly patients, recommendations in using antiviral treatment were also inconsistent.

Twenty guidelines ⁴ , ¹⁰ , ¹¹ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁸ , ²² , ²³ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ⁴¹ (54.1%) addressed the use of glucocorticoids for patients with COVID‐19. Nine ⁴ , ¹⁴ , ¹⁶ , ²² , ²⁸ , ³⁰ , ³⁴ , ³⁶ , ⁴¹ (24.3%) of these did not recommend routinely using the glucocorticoids or except in the context of clinical trials. The remaining nine guidelines ¹⁰ , ¹¹ , ¹⁵ , ¹⁸ , ²⁷ , ²⁹ , ³² , ³³ , ³⁵ conditionally recommended the use of glucocorticoids in specific conditions, such as for critically ill patients with COVID‐19. The glucocorticoids often mentioned in these guidelines included methylprednisolone and dexamethasone.

The use of antibiotics during COVID‐19 treatment was also a common topic. We found a total of 17 guidelines ⁴ , ¹⁰ , ¹¹ , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ²² , ²⁴ , ²⁷ , ²⁹ , ³⁰ , ³¹ , ³³ , ³⁴ , ³⁵ , ⁴³ (45.9%), either recommending or specifically not recommending the use of antibiotics in patients with COVID‐19. Most of these guidelines ¹⁰ , ¹¹ , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ²² , ²⁵ , ²⁷ , ²⁹ , ³¹ , ³⁴ , ³⁵ , ⁴³ (n = 14, 37.8%) conditionally recommended starting an empiric broad‐spectrum antibiotic therapy when patients were suspected to have a bacterial superinfection or when their condition got worse. However, a prophylactic antibiotic therapy or routine treatment with antibiotics was not recommended in any guideline.

A total of six guidelines ¹⁵ , ¹⁸ , ²⁷ , ³⁰ , ³³ , ³⁵ (16.2%) referred to the use of immunoglobulins in patients with COVID‐19. Most of these guidelines did not advocate the routine use of immunoglobulins, but mentioned that they may be used for critically ill or elderly patients with COVID‐19 to boost their levels of immunoglobulin.

Five guidelines ⁴ , ¹⁸ , ²⁸ , ³² , ³⁹ (13.5%) on COVID‐19 treatment did not recommend using convalescent plasma because of insufficient evidence to support its use. However, one guideline ³⁵ (2.7%) from China suggested that plasma treatment could be used as appropriate for patients with fast progression, severe cases, or critical cases. Another guideline ³⁷ suggested that convalescent plasma treatment should be offered only within a research framework or on a compassionate basis for critically ill children with COVID‐19.

Other mentioned treatments included psychotherapy and extracorporeal membrane oxygenation (ECMO). Seven guidelines ¹¹ , ¹⁴ , ¹⁸ , ²⁵ , ³⁴ , ³⁷ , ⁴³ (18.9%) conditionally suggested using the ECMO. One guideline ³¹ specifically recommended not to use ECMO: according to the guideline, recent reports had found high mortality in patients with COVID‐19 treated with this form of organ support, suggesting ECMO should not be used in hospitals other than reference or high‐volume centers, and whose personnel have no experience with this procedure. Five guidelines ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ (13.5%) recommended psychotherapy, including psychological counseling, administration of antidepressants, and related projects.

4. DISCUSSION

We evaluated the quality of 37 diagnosis and treatment guidelines ⁴ , ⁵ , ¹⁰ , ¹¹ , ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² , ⁴³ , ⁴⁴ for COVID‐19 and analyzed the consistency of their recommendations. In general, we found these guidelines to be of low quality. Their recommendations for treatment and diagnosis were somewhat consistent, but there were differences in the recommendations considering some specific populations and conditions.

Our results suggested that the quality of diagnosis and treatment guidelines for COVID‐19 was low, especially regarding stakeholder involvement, rigor of development, and applicability. This was generally consistent with the findings of two earlier studies. ⁶ , ⁷ There are three potential reasons for this. First, in the first eight months of the COVID‐19 epidemic, most guidelines were rapid advice guidelines or interim guidance, and the low quality may be a result of the urgency and lack of time. However, the quality of the guidelines has increased over time. Second, some of the included guidelines were published on websites only, reported with insufficient details resulting in low quality. Third, some of the AGREE II items (eg, item 7) are not necessarily applicable to the evaluation of the rapid advice guidelines, which may also have contributed to the low quality of some of the guidelines. In addition, the lack of sufficient and direct evidence to support the guidelines in the early stages of the COVID‐19 epidemic also contributes to the poor quality, the quality of evidence for guidelines based need to be improved. ⁴⁵ To improve the quality of guidelines developed in such urgent situations, guideline developers should learn how to develop rapid advice guidelines in public health emergencies. Guideline users should also learn how to quickly assess the quality of public health emergencies guidelines in order to better guide public health and clinical practice.

An accurate diagnosis of COVID‐19 is essential to confirm the suspected cases. About two‐thirds of the included guidelines focused on the diagnosis of COVID‐19. Laboratory tests such as nucleic acid testing were recommended as routine tests. The turn‐around time of these tests ranges from 1 hour to a couple of days; this depends on the version of the PCR and the capacity of the laboratories. Furthermore, due to the low concentration of virus, RT‐PCR is less sensitive for diagnosis of early SARS‐CoV‐2 and presymptomatic patients, resulting in false‐negative results. For imaging diagnosis, for different populations, some guidelines recommended X‐rays, and some guidelines recommended CT scanning, depending on the population. X‐rays have a high rate of missed diagnosis for mild disease, but CT is associated with a high dose of radiation and there is a risk of low specificity. Point‐of‐care ultrasound (POCUS) has been considered as a safe and effective bedside option in the initial evaluation, management, and monitoring of disease progression in patients with confirmed or suspected COVID‐19 infection. ⁴⁶ A review from Buda et al ⁴⁷ showed that POCUS could be considered for patients with suspected COVID‐19 whose PCR test result is negative. Besides, ultrasound examination may be worth considering because of its convenience, low cost, and safety (compares with CT scanning).

We found the guidelines to have become more consistent in recommending diagnostic methods, as COVID‐19 has become better understood over time and diagnostic test studies have been conducted. Overall, RT‐PCR should be the first considered method of diagnosis. In the case of a negative of RT‐PCR but clearly ill and/or high‐risk patients, combinations of different diagnostic methods for different populations and settings may be considered, taking into account the cost, accessibility, and efficacy of the diagnostic methods.

At this time, there are no effective drugs for the treatment of COVID‐19. The most common treatment options remain symptomatic treatment and support therapy. The most frequently mentioned types of treatment were antiviral treatments (59.5%), supportive care (54.1%), and glucocorticoid therapy (54.1%). However, the use of these drugs or options is in most cases conditional, for example, recommended only for patients with severe COVID‐19 or for the elderly, or contraindicated for some specific populations. In the absence of sufficient or direct evidence, the recommendations for all drugs remain largely consistent. Our findings were consistent with those of Snawerdt et al, ⁴⁸ who summarized the results of previously published studies and concluded that there remains no treatment that had proven to be efficacious against COVID‐19. But at the same time, a large number of systematic reviews identifying different treatments for COVID‐19 have been published. ⁴⁹ , ⁵⁰ , ⁵¹ Many clinical trials for the treatment of COVID‐19 ⁵² had also been registered on ClinicalTrials.gov platform. On the one hand, these trials and reviews can provide evidence for future guidelines, but on the other hand, low‐quality or duplicate studies can lead to wasted research resources. ⁵³ In the absence of effective treatment drugs, a vaccine is likely to be the most effective measure to end the epidemic. In the meantime, preventing and controlling the spread of COVID‐19 is paramount, especially in low‐ and middle‐income countries and countries with fragile health systems.

To our knowledge, this is the first study to analyze and evaluate the diagnosis and treatment guidelines for SARS‐CoV‐2 infection. We used a systematic searching method, summarized recommendations, evaluated the quality of the guidelines, and examined the consistency of recommendations. However, this study also has several limitations. First, the searching time was up to August 31, 2020, and some guidelines that are still under development or have not yet been indexed may have been omitted. Second, we only analyzed and evaluated the COVID‐19 diagnosis and treatment guidelines. However, guidelines for COVID‐19 in combination with other diseases can also inform clinical decision‐making. Third, due to the diversity of contents in the guidelines, we were unable to extract and analyze all recommendations.

In conclusion, the current diagnosis and treatment guidelines for COVID‐19 are of low quality. The recommendations in diagnosis and treatment guideline for COVID‐19 are to be relatively consistent in their directions, particularly regarding the treatment. Up to now, there is no effective drug for the treatment of COVID‐19, and a vaccine is probably the only way to end the epidemic.

FUNDING

This study was supported by a 2020 Key R & D project of Gansu Province. Special funding for prevention and control of emergency of COVID‐19 from Key Laboratory of Evidence‐Based Medicine and Knowledge Translation of Gansu Province (Grant No. GSEBMKT‐2020YJ01).

AUTHOR CONTRIBUTIONS

All authors have read and agree to the published version of the manuscript. Conceptualization, literature retrieval, screening, and data extraction, XL, YL, and ML; quality assessment, YL, MR, and XZ; writing‐original draft preparation, XL and YL; writing‐review and editing, XL, YL, EJ, ML, QW, YS, JLM, HS.A, MSL, YC; supervision, XL and YL; project administration, YC.

CONFLICT OF INTEREST

The authors declare no conflict of interest.

Supporting information

Supporting Material

Luo X, Liu Y, Ren M, et al. Consistency of recommendations and methodological quality of guidelines for the diagnosis and treatment of COVID‐19. J Evid Based Med. 2021;14:40–55. 10.1111/jebm.12419