An evaluation of the COVID-19 recommendation map identified diverging clinical and public health guidance

Abstract

Objectives

To describe divergence between actionable statements issued by coronavirus disease 2019 (COVID-19) guideline developers cataloged on the “COVID-19 Recommendations and Gateway to Contextualization” platform.

Study Design and Setting

We defined divergence as at least two comparable actionable statements with different explicit judgments of strength, direction, or subgroup consideration of the population or intervention. We applied a content analysis to compare guideline development methods for a sample of diverging statements and to evaluate factors associated with divergence.

Results

Of the 138 guidelines evaluated, 85 (62%) contained at least one statement that diverged from another guideline. We identified 223 diverging statements in these 85 guidelines. We grouped statements into 66 clusters. Each cluster addressed the same population, intervention, and comparator group or just similar interventions. Clinical practice statements were more likely to diverge in an explicit judgment of strength or direction compared to public health statements. Statements were more likely to diverge in strength than direction. The date of publication, used evidence, interpretation of evidence, and contextualization considerations were associated with divergence.

Conclusion

More than half of the assessed guidelines issued at least one diverging statement. This study helps in understanding the types of differences between guidelines issuing comparable statements and factors associated with their divergence.

What is new?

Key findings

• •Divergence was observed in 62% of the included guidelines for the management of COVID-19.
• •Overall, we observed a similar frequency of guidelines diverging in explicit judgments and subgroup considerations but a higher frequency of statements diverging in strength than in direction of the recommended action. We identified associations between differences in methodological and contextualization factors and divergence.

What this adds to what was known?

• •We have developed a new concept, termed divergence, to describe differences in the explicit conclusions or subgroup considerations in clinical and public health guidelines issuing comparable statements and assessed factors associated with their differences.

What is the implication and what should change now?

• •Divergence associated with context-specific considerations can lead to more equitable outcomes between regions with diverse resources. Divergence associated with differences in other methodological steps, may contrarily compromise the overall quality and rigor of the guideline development process. Guideline users may choose to appraise differences in methods, use of evidence, and context-specific criteria to decide which diverging statement is appropriate for their setting.

1. Introduction

We have developed a digital platform (https://covid19.recmap.org/) classifying and presenting actionable statements published in guidance documents for the management of the coronavirus disease 2019 (COVID-19) [1]. Alongside each actionable statement issued on this recommendation map (RecMap), we extracted explicit judgments of strength and direction and considerations such as Evidence-to-Decision (EtDs) frameworks when issued by guideline authors to catalog advice about COVID-19. We noted important discrepancies of the content of the RecMap that may confuse users.

Discrepancies can exist between how those synthesizing and presenting research evidence interpret it. This can lead to confusion among evidence users. Studies have specifically evaluated discordance in the results and conclusions of systematic reviews answering equivalent research questions [2,3]. To select between discording reviews, Jadad et al. created a decision tool for stakeholders to apply when making clinical or policy decisions that incorporated a comparison of the search strategy and meta-analytical processes used for each review [2]. Moja et al. used this decision tool to examine how frequent duplicate systematic reviews give different results and factors contributing to their discordance [3]. The term discordance has similarly been used in the guideline development work to describe differences between the strength of recommendations and their certainty of evidence [4]. Although there is generally agreement among guideline developers, a different concept relates to scenarios when developers create recommendations or other actionable statements [5] that do not align with those from other guideline developers. We define this as divergence of actionable statements. This divergence can lead to confusion among users of recommendations if the divergence is unexplained. The primary objective of this study was to explore the explicit judgments or subgroup considerations of comparable actionable statements on the RecMap that are associated with divergence. Our secondary objective was to investigate factors associated with this divergence.

2. Methods

2.1. General approach

There are two classifications of divergence in our study. The first classification is any explicit difference in judgments that influence the strength or direction of two or more actionable statements (divergence based on the guideline developer judgment). When grouping this type of divergence, we used the Population Intervention Comparator Outcome (PICO) model to formulate a condition where statements in different guidelines addressed the same population (P) to judge whether a specific intervention (I) or comparator (C) intervention should be implemented. The second classification is any difference within subgroups of the recommendation's target population or in specific elements, for example, variation in frequencies, dosages, units, or time intervals of an otherwise similar intervention (divergence based on subgroup considerations). Examples of the classifications and subclassifications of divergence can be found in Table 1 .

Table 1.

Examples of the type of divergence between guidelines

Classification of divergence	Subclassification of divergence	PICO/Intervention	Judgment	Statements from guidelines
Divergence based on guideline developer judgment	Different strength recommended	Gelatin for the acute resuscitation of patients with COVID-19 experiencing shock	Conditional/weak recommendation	Surviving Sepsis Campaign: For the acute resuscitation of adults with COVID-19 and shock, we suggest against using gelatin. (Jan. 2021)
			Strong recommendation	World Health Organization: Do not use hypotonic crystalloids, starches, or gelatins for resuscitation. (Jan. 2021)
	Different direction recommended	Bamlanivimab for the treatment of COVID-19 patients	Strong recommendation against the intervention	Australian National COVID-19 Clinical Evidence Taskforce: Do not use bamlanivimab for the treatment of COVID-19 outside of randomized trials with an appropriate ethical approval. (April 2021)
			Recommended in favor of the intervention	The American College of Occupational and Environmental Medicine: Bamlanivimab is recommended for the treatment of patients with mild to moderate COVID-19. (Dec. 2020)
Divergence based on subgroup considerations	Population subgroup (different age cutoffs)	The use of facemasks in children in public settings	Not recommended for children younger than 2 years	Centers for Diseases Control and Prevention: Masks should not be put on children younger than 2 years (April 2021)
			Not recommended for children younger than 5 years	World Health Organization: Children aged 5 years and less should not be required to wear masks (Dec. 2020)
	Intervention subgroup (different units)	Distancing units between students and teachers in school settings	Recommended 1 m distance	The Technical Advisory Group: Teachers and support staff should keep at least 1 m apart from each other and from students (Sep. 2020)
			Recommended 2 m distance	Office of the Deputy Prime Minister – Ministry for Health: Keep 2 m between staff and students (Aug. 2020)

2.2. Inclusion and exclusion criteria

Figure 1 describes the step-by-step approach we followed. The inclusion criteria for identifying guidance documents to extract relevant recommendations for the RecMap are described in Appendix A [1]. Actionable statements can be divided based on their methodological rigor as formal or informal recommendations (Appendix A) [5]. We reviewed both types of statements displayed on the RecMap (COVID19.recmap.org). Our approach to evaluating divergence was not exhaustive as we included guidelines that were published by key organizations [1] or new actionable statements that could be compared to statements already uploaded to the RecMap through the GRADEpro application (Appendix A). We excluded older versions of updated guidelines, retracted, or archived guidelines at the time of review.

Fig. 1.

Step-by-step approach to evaluate the classifications and subclassifications of divergence between comparable statements.

2.3. Data extraction

2.3.1. Screening

We screened (Z.N.) and verified (A.M.) diverging statements published till April 30, 2021, using guidance documents identified for inclusion in the RecMap. For statements already displayed on the RecMap, we applied designated population and intervention filters available for all platform users to group comparable statements for assessments of divergence. For the remaining documents, we searched for diverging statements in order of the guideline topic.

2.3.2. Data extraction

We flagged and subsequently aggregated all diverging statements with the same PICO elements but with different explicit judgments of strength or direction as one type of clusters (Appendix A). We also aggregated actionable statements for the same intervention but with different subgroup considerations as separate clusters. For statements with explicit judgments, we coded recommendations on a 4-point ordinal scale using the strength of recommendations as described in the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach (Appendix A) [5,6]. When possible, we transformed other grades of recommendations to GRADE in GRADEpro (www.gradepro.org) using a framework applied for the RecMap [7].

2.4. Statistical analysis

2.4.1. Quantitative analysis

We performed descriptive statistics to assess the frequency of guidelines containing one or more diverging statements from at least one other guideline in our sample. We evaluated the mean and range of diverging statements in our clusters. We conducted a Fisher's exact test to evaluate if there was an observed difference in the type of divergence (i.e., explicit judgment or subgroup consideration) between clinical and public health statements.

2.4.2. Qualitative analysis

We used a content analysis to compare differences in guideline development methods between actionable statements grouped in the same cluster. We randomly selected six clusters and purposefully selected an additional six clusters from the World Health Organization (WHO) to achieve information saturation. We developed a categorization matrix consisting of four major themes using a deductive approach (Appendix C) where each theme represented methodological steps in the guideline development process that could be compared between diverging statements. We used Jadad's decision tool [2] and reviewed standardized approaches of the guideline development to form themes [[8], [9], [10]]. A single reviewer (Z.N.) evaluated each guideline in our sample of clusters and recorded information that corresponded with each theme in the matrix. A second reviewer (A.M.) revised, flagged, and inputted any missing information under each theme for completeness, and so all details were available for reproducibility. Any differences between reviewers were noted and resolved through consensus.

3. Results

3.1. Summary of guidelines and recommendations

We identified 138 guidelines that allowed us to explore the phenomenon of divergence and explore factors associated with it. Eighty five (62%) of these guidelines issued at least one diverging statement from another guideline for the same PICO or intervention. We found a total 223 diverging statements in these 85 guidelines that allowed us to evaluate and describe the phenomenon of divergence (of at least 1,330 actionable statements that were available on the RecMap at the time of our evaluation). Ninety nine (44%) diverging statements were related to clinical interventions and 124 (56%) to public health interventions, respectively. Twenty four (11%) of these statements used gradings that were readjusted to match the GRADE [7]. We observed 115 (52%) statements diverging in explicit judgment of strength or direction and 108 (48%) statements diverging in subgroup considerations of the population or intervention.

3.2. Summary of clusters

We aggregated the 223 statements to 66 clusters. Each cluster represented diverging statements with the same PICO or overarching intervention as described above (Appendix A). Twenty nine (44%) of the clusters contained clinical interventions and 37 (56%) contained public health interventions (Table 2 ). The mean number of diverging actionable statements in our clusters was 3.4 (standard deviation [SD]: 1.5) and the number of statements per cluster ranged from a minimum of 2 to a maximum of 8.

Table 2.

Summary of diverging recommendation clusters

Overall summary of diverging clusters	(n)	(%)
Number of total diverging clusters	66	100
Number of clinical clusters	29	43.9
Number of nonclinical clusters (i.e., public health)	37	56.1
Summary of Type of Diverging Clusters
Diverging in the explicit judgment of strength only	19	28.8
Diverging in the explicit judgment of direction only	9	13.6
Diverging in the explicit judgment of strength and directiona	6	9.1
Diverging in subgroup considerations of the population	6	9.1
Diverging in subgroup considerations of the intervention	26	39.4
Summary of Diverging Clusters Across Intervention Groups
Pharmacological interventions	21	31.8
Other clinical interventions	8	12.1
Infection prevention and control measures	21	31.8
Vaccination-related measures	8	12.1
School-related measures	8	12.1

^aAt least one recommendation in the cluster diverges in strength and at least one different recommendation in the same cluster diverges in direction.

3.3. Type of divergence in clusters

Thirty four clusters diverged in explicit judgment of strength or direction and 32 clusters diverged in subgroup considerations of the population or intervention (Table 3 ). Of those diverging in judgment, 19 (56%) diverged in strength only and 9 (26%) diverged in direction only. Six (18%) diverged in both, meaning at least one statement diverged in strength and at least one different statement diverged in direction within the same cluster. For clusters diverging in subgroup considerations, 6 (19%) had diverging subgroups for the population and 26 (81%) had diverging subgroups for the intervention.

Table 3.

Type of divergence within clinical and public health clusters

Health scope	Types of diverging clusters
	Judgment		Subgroup consideration		Total
	n	%	n	%	n	%
Divergence across overarching health scope
Clinical interventions	27	93.1a	2	6.9a	29	43.9
Public health interventions	7	18.9b	30	81.1b	37	56.1
Divergence across intervention groups
Pharmacological interventions	19	90.1	2	9.5	21	31.8
Other clinical interventions	8	100.0	0	0.0	8	12.1
Infection prevention control measures	5	23.8	16	76.2	21	31.8
Vaccination-related measures	2	25.0	6	75.0	8	12.1
School-related measures	0	0.0	8	100.0	8	12.1
Total	34	51.5	32	48.5	66	100.0

^aPercentage of the type of diverging clusters within clinical interventions.

^bPercentage of the type of diverging clusters within public health interventions.

We found that clinical statements were more often associated with divergence in the judgment of strength or direction, whereas public health interventions were associated with divergence in subgroup considerations of the population or intervention (Cramer's V = 0.7, Fisher's exact test: 35.8, P < 0.001). In our sample, 27 of 29 (93%) clinical clusters diverged in judgment, whereas two diverged in subgroup considerations. Conversely, 81% of the public health clusters diverged in subgroup considerations. Statements for clinical interventions were also more likely to issue explicit judgments of strength or direction, whereas statements for public health measures were more likely informal about the strength and direction, which necessitated an assessment of differences in subgroup considerations, likely contributing to our observed differences.

3.4. Qualitative findings

Twenty one guidelines with diverging statements were included in our 12 clusters evaluated for a content analysis. We have organized the methodological differences into four major categories (Table 4 ).

Table 4.

Methodological differences between actionable statements in the same cluster

Categories	Clusters
Differences in the date of publication or most recent literature search	3
Differences in the body of evidence for reasons other than the date of publication	6
Differences in the interpretation of evidence and assessments of quality	10
Differences in contextualization considerations including in EtD criteria	4

3.4.1. Differences in the date of publication or most recent literature search

Differences in the date of publication or most recent literature search may explain why some statements omit evidence included in comparable statements from other guidelines. When comparing the Centers for Disease Control and Prevention (CDC), Public Health Agency of Canada (PHAC), and WHO's guidance for cleaning public spaces, we observed differences in the recommended frequency of time surfaces should be cleaned (Table 5) [[11], [12], [13]]. CDC updated their guideline in April 2021 to recommend cleaning most high-touch surfaces in nonhealthcare settings once daily based on evidence from the Quantitative Microbial Risk Assessment models and other direct studies assessing COVID-19 transmission patterns [11]. In contrast, WHO and PHAC issued their statements in May 2020 and September 2020, respectively. Both guidelines recommended enhanced surface cleaning but do not explicitly state the frequency [12,13].

Table 5.

Simplified content analysis outcomes of methodological differences within clusters

Organizations	Divergence	Methodological differences between guidelines
Convalescent plasma for the treatment of COVID-19 patients
Australian Clinical Taskforce; IDSA; NIH; SSC	Strength and direction	Theme 1: Differences in the date of publication or most recent literature search •SSC did not use results from RECOVERY trial whereas other organizations did. Theme 3: Differences in the interpretation of evidence and assessments of quality •Guidelines had varying gradings of the certainty of evidence.
Gelatin for the acute resuscitation of patients experiencing COVID-19 and shock
SSC; WHO	Strength	Theme 3: Differences in the interpretation of evidence and assessments of quality •Guidelines agree there are unclear benefits. Theme 4: Differences in contextualization considerations •Guidelines balanced unclear benefits with high costs.
Immunoglobulins for the treatment of COVID-19 patients
Australian Clinical Taskforce; NIH; SSC	Strength	Theme 1: Differences in the date of publication or most recent literature search •Guidelines applied different evidence based on what was available at the time of publication. Theme 3: Differences in the interpretation of evidence and assessments of quality
Ivermectin for the treatment of COVID-19 patients
Australian Clinical Taskforce; IDSA; NIH	Strength and direction	Theme 2: Differences in the included studies/evidence •Differences were observed in the included studies between all guidelines. Theme 3: Differences in the interpretation of evidence and assessments of quality •All guidelines acknowledged small net benefits and limitations in evidence.
Remdesivir for the treatment of moderate to severe COVID-19 patients
Australian Clinical Taskforce; ACOEM; IDSA; NIH; NICE; PHAC; SSC; WHO	Direction	Theme 3: Differences in the interpretation of evidence and assessments of quality •Differences in certainty of evidence and interpretation of uncertainty were observed between WHO and other guidelines. Theme 4: Differences in contextualization considerations •Differences in cost effectiveness and patient preferences and values were observed.
Zinc for the treatment of COVID-19 patients
Australian Clinical Taskforce; NIH	Strength	Theme 2: Differences in the included studies/evidence Theme 3: Differences in the interpretation of evidence and assessments of quality •Differences were observed in certainty of evidence and interpretation of the uncertainty on statement strength.
Delaying the interval between the first and second dose for mRNA COVID-19 vaccines
CDC; ECDC; PHAC	Subgroup considerations (intervention)	Theme 2: Differences in the included studies/evidence Theme 3: Differences in the interpretation of evidence and assessments of quality •CDC did not recommend delaying the second dose for more than 6 weeks because of the limited available direct evidence on the effectiveness of the intervention. Theme 4: Differences in contextualization considerations •PHAC considered vaccine feasibility concerns to make their statement for the Canadian setting.
Frequency of times to clean surfaces in public settings
CDC; PHAC; WHO	Subgroup considerations (intervention)	Theme 1: Differences in the date of publication or most recent literature search •CDC updated their statement based on availability of new evidence, but other guidelines did not. Theme 2: Differences in the included evidence/studies
Minimum age cut-off for wearing facemasks in children’ populations
CDC; PHAC; WHO	Subgroup considerations (population)	Theme 2: Differences in the included evidence/studies •WHO cited evidence from influenza studies. CDC cited evidence from single observational study assessing oxygenation levels in children wearing masks. PHAC did not cite explicit evidence.
Mode of birth for pregnant women with COVID-19
Australian Clinical Taskforce; WHO	Strength	Theme 2: Differences in the included evidence/studies Theme 3: Differences in the interpretation of evidence and assessments of quality •Guidelines balanced benefits and harms with certainty of evidence to make their final judgments.
Rooming mothers with COVID-19 with their newborns after birth
Australian Clinical Taskforce; WHO	Strength	Theme 3: Differences in the interpretation of evidence and assessments of quality •Guidelines balanced benefits and harms with certainty of evidence to make their final judgments.
Relaxing certain restrictions for vaccinated travellers/population
CDC; PHAC; WHO	Direction	Theme 3: Differences in the interpretation of evidence and assessments of quality •PHAC and WHO stated there is limited evidence to relax restrictions. CDC considered low rates of asymptomatic infection and possible transmission in vaccinated groups and available data on vaccine effectiveness. Theme 4: Differences in contextualization considerations •CDC considered studies on US population attitudes and behaviors toward vaccination.

3.4.2. Differences in the body of evidence for reasons other than the date of publication

Other guidelines with diverging statements applied different bodies of evidence for reasons other than the most recent literature review (Table 5). One cluster issued diverging statements for delaying the second dose of COVID-19 vaccines that require two complete doses [[14], [15], [16]]. CDC in March 2021 recognized the limited direct evidence concerning the efficacy and effectiveness of extending the interval beyond 6 weeks to recommend against any further delay [14]. Contrarily, in April 2021, PHAC recommended delaying the interval by 4 months in the context of limited vaccine supplies in Canada [15]. Consequently, the National Advisory Committee on Immunization deliberated on whether it was feasible to extend the second dose, so more members of their population are vaccinated. They applied evidence from studies evaluating vaccine effectiveness after one dose and population modeling studies [15]. None of their included studies directly assessed delaying the second dose for a period of 4 months because of insufficient evidence.

3.4.3. Differences in the interpretation of evidence and assessments of quality

Differences in the interpretation and certainty of evidence were evident for remdesivir; WHO issued a conditional recommendation against its use, whereas other guidelines issued conditional recommendations favoring its use in patients with moderate to severe COVID-19 (Table 5) [[17], [18], [19], [20], [21], [22], [23], [24]]. Most guidelines used evidence from the same randomized clinical trials including the Adaptive COVID-19 Treatment Trial (ACTT-1) [[17], [18], [19], [20], [21], [22], [23], [24]] and WHO solidarity studies [[17], [18], [19], [20], [21], [22], [23]]. Although WHO [17] graded the overall certainty of applied evidence as low, guidelines issued by the National COVID-19 Clinical Taskforce of Australia [18], Infectious Disease Society of America (IDSA) [19], and National Institute of Health Care Excellence (NICE) [20] graded the certainty of evidence as moderate.

When comparing the balance of effects, WHO concluded there was insufficient evidence to currently prove remdesivir has an effect for patient-important outcomes such as mortality, need for mechanical ventilation, and time to clinical improvement. They balanced this with the high costs, resource requirements, and barriers in low-income and middle-income countries to recommend against its use [17]. Conversely, the Australian taskforce concluded that remdesivir has small net benefits or little differences compared to alternative options. This judgment was based on remdesivir's safety profile, probable reduction of death, and probable reduction of serious adverse events [18]. When comparing study outcomes, the solidarity trial reported no differences in outcomes of mortality and length of hospital stay [20]. However, the ACTT-1 trial reported an improved time to recovery and less percentage of patients progressing to invasive ventilation [20]. Consequently, different interpretation of these studies may be associated with different judgments.

3.4.4. Differences in contextualization considerations including in EtD criteria

We observed explicit or implicit differences in contextualization considerations (Appendix C). When evaluating CDC's statement on relaxing certain measures for vaccinated travelers, the organization partly considered the American population's values and attitudes toward COVID-19 vaccines in addition to vaccine effectiveness studies [25]. To incentivize vaccination, CDC recommended relaxing certain restrictions for vaccinated populations, whereas other organizations acknowledged the limited evidence concerning COVID-19 transmission and its effectiveness to recommend against relaxing restrictions [26].

4. Discussion

4.1. Summary of findings

We evaluated the frequency of guidelines containing diverging actionable statements for the management of COVID-19 by applying an iterative search, screening, and review process using the COVID-19 Recommendations and Gateway to Contextualization RecMap. At the time of evaluation, 62% of the included guidelines contained at least one diverging statement from at least one other guideline. Within these guidelines, we identified 223 total diverging statements. A content analysis showed differences in methodological factors including the date of publication or most recent literature search, used evidence, interpretation of evidence, and their assessments of quality between guidelines containing comparable diverging statements. We also observed differences in contextualization criteria including judgments of cost effectiveness and patient values between guidelines. We have developed a meaningful technique for those who need to understand differences between guideline recommendations to categorize the reasons for differences. The results from our study can also be used to clarify misperceptions among decision makers by promoting transparency in guideline development processes.

4.2. Strengths and limitations

Strengths in our study include the application of a rigorous search strategy to organize actionable statements from guidelines on the RecMap and using two independent reviewers for verification of divergence and a content analysis. Nonetheless, we could not quantify the exact number of actionable statements assessed from guidelines not uploaded to the RecMap, but the results suggest that divergence is common and not always explained. Given the large number of such actionable statements, our approach did not allow evaluating all COVID-19 guidelines or the total number of concordant statements between guidelines, limiting the interpretation of our findings.

4.3. Discussion of quantitative and qualitative findings

4.3.1. Quantitative findings

Overall, we identified a similar frequency of divergence in explicit judgments about the strength and direction and subgroup considerations but a larger frequency of statements diverging in strength than direction. In our assessment, we did not assign direction for statements that did not have explicit judgments. The frequency of statements diverging in direction may be higher than what was suggested in our study if we formulated implicit judgments based on which alternative was selected by organizations in the same cluster. For example, in our cluster for delaying vaccine dosages, CDC and PHAC recommend different time intervals for delaying the administration of the second dose. Because CDC did not explicitly appraise the time intervals that were recommended by PHAC to formulate their judgment, we grouped this cluster as diverging subgroup considerations of the intervention instead of direction, despite both organizations recommending alternative intervals.

4.3.2. Qualitative findings

Time constraints, insufficient evidence to inform judgment, and additional resource limitations may affect the overall methodological rigor of published guidelines in health emergencies like the COVID-19 pandemic. In some regard, we can attribute divergence associated with differences in the date of the most recent literature review and applied evidence to these barriers. Likewise, organizations may have different priorities to update existing statements in their guidelines even when new evidence is available, leading to their possible omission and subsequently different gradings of certainty or judgment. Nevertheless, these challenges cannot explain divergence between organizations applying equivalent methods and evidence.

4.4. Study implications

4.4.1. Implication for policy and practice

Policy and other decision makers may choose to use our content-analysis themes or adopt existing methods [2,3] by appraising differences in the date of literature review, applied evidence, interpretation of evidence, and context-specific considerations between diverging statements to decide which statements are supported by the best available evidence and will be most appropriate to implement in their setting.

Indeed, divergent recommendations can be based on legitimate contextual reasons. This idea is the core principle behind the GRADE ‘adolopment’ approach, a process that entails evaluating circumstantial EtD criteria from existing recommendations to decide whether they are appropriate to adopt or adapt in a new context or create new recommendations for better contextualization [27]. Guideline developers can therefore review regional evidence to re-evaluate judgments of EtD criteria or ultimately change the strength or direction of existing recommendations to reflect the new setting where it will be applied. For example, when guideline developers from a low-income region are deciding to adopt or adapt an intervention that is recommended for implementation in a high-income region, they may choose to implement a more cost-effective alternative with similar clinical outcomes instead of the original intervention. This new recommendation will therefore diverge in direction from the original statement but may yield better outcomes in the new setting.

An agreement between guidelines is important in other circumstances. For example, two diverging statements may differ in the direction of the recommendation and one guideline may issue a strong recommendation. Depending on which alternative option is implemented, inequities could increase between settings if one intervention results in better outcomes than the other. Furthermore, the guideline development process aims to use evidence-informed methods to develop trustworthy statements. Divergence can ultimately affect the quality of these statements, especially when they are associated with the use of outdated evidence, incomplete evidence, or even differences in the interpretation of similar evidence when explanations for these different interpretations are not clarified by guideline authors. In these situations, it would be difficult to justify divergence. This may reduce public’s trust in the scientific community, which may lead to greater hesitancy to comply with health regulations [28].

4.4.2. Implications for research

The COVID-19 guidelines are being formulated and updated during a period when the evidence to develop actionable statements is constantly evolving. Future studies should assess whether associations of divergence observed in our study are unique to the landscape of the COVID-19 pandemic or observed in other guideline topics. Furthermore, future studies should evaluate if these associations of divergence are linked with causation. If divergence is observed and caused by methodological differences, guideline developers will need to refine their procedures for developing actionable statements. This includes adopting methods, such as living guideline processes [29], to guarantee timely updates of actionable statements to ensure they are supported by the best available evidence.

5. Conclusion

We used a comprehensive international database to identify and describe differences between comparable statements for COVID-19, a phenomenon that we call divergence. We subsequently examined legitimate and justified methodological differences associated with divergence that can be used to assess causation in future studies.