Resources supporting trustworthy, rapid and equitable evidence synthesis and guideline development: results from the COVID-19 evidence network to support decision-making (COVID-END)

Graphical abstract

What is new?

Key findings

• •COVID-19 and subsequent infodemic represents an unprecedented global challenge for evidence synthesis and guideline development.
• •We present consolidated resources to facilitate trustworthy, rapid and equitable evidence synthesis, health technology assessments and guidelines for public health and clinical practice.

What this adds to what is known?

• •The infodemic can be tackled through equitable collaboration and key partnerships with a focus on reducing research waste and duplication of efforts.
• •We provide key examples and resources for efficient evidence synthesis and guideline development.
• •What is the implication and what should change now?
• •We also call for future efforts to further strengthen global and equitable collaboration within evidence synthesis and guidance, with potential impact beyond COVID-19.
• •We encourage decision-makers to draw on existing high-quality systematic reviews and guidelines to support their decisions and to support and fund local teams to efficiently develop adoption or adaptation guidance contextualized to their settings.

1. Introduction

Robust evidence syntheses, health technology assessments (HTAs) and trustworthy guidelines are the cornerstones of informed healthcare decision making and for moving the best-available research evidence into policy and practice. The Coronavirus disease 2019 (COVID-19) pandemic and subsequent infodemic–the rapid spread and generation of accurate and inaccurate information–represents an unprecedented global challenge. The infodemic has impacted not just public trust in the healthcare system, but also the health sector's ability to rapidly respond to misinformation. More than 700,000 scientific publications for COVID-19 and immense ongoing research efforts (e.g., 5,000 randomised controlled trials) makes it close to impossible to keep up to date with best current evidence through traditional ways of managing evidence [1,2]. This holds true for worldwide governments, decision makers, health system managers, media and healthcare workers - not to mention the public and patients.

In addition, the urgency to act (necessitated by the extent and severity of the threat to public health globally) and variable methodological quality means that the traditional timeline (months to years) of producing or updating evidence synthesis and guidance is unacceptable in a pandemic [3]. COVID-19 has therefore resulted in increased expectations of delivery of accelerated or urgent ‘real-time’ development of evidence synthesis, guidelines and policies [[4], [5], [6], [7], [8]]. Furthermore, lack of collaboration and interoperability across organizations continues to fuel duplication of efforts and leave important evidence gaps, perhaps increasing rather than reducing the well-known waste in health care research [9].

An additional challenge - exacerbated by COVID-19 especially for low-to-middle income countries (LMICs) - comes from underdeveloped or weak health research systems and resources allocated to this work along with poor health services, economic inequity, unemployment and hunger, lower income and conflict-affected states including lack of access to effective interventions [10].

COVID-19 therefore requires trustworthy, rapid, and equitable evidence syntheses and guidance to inform clinical and public health decisions and vaccination rollouts, including evidence on inequity and distribution of effects across populations. One global initiative that aims to tackle these major challenges is the COVID-19 Evidence Network to support Decision-making (COVID-END, see below). Within COVID-END, two working groups were established to solve challenges with evidence synthesis and guidelines. In this commentary, we present how COVID-END works and what became a joint mission of these working groups; consolidated resources to facilitate trustworthy, rapid and equitable evidence synthesis, health technology assessments and guidelines for public health and clinical practice. We also call for future efforts to further strengthen global and equitable collaboration within evidence synthesis and guidance, with potential impact beyond COVID-19.

1.1. COVID-END and efforts to improve evidence synthesis, guidance, and equity

Toward addressing the issues plaguing the evidence community during COVID-19, COVID-END was established in April 2020 [11]. COVID-END involves more than 50 evidence synthesis or evidence support groups working together to promote collaboration and reduce inadvertent or inappropriate duplication of efforts and address important gaps in the conduct and translation of COVID-19-related evidence synthesis and guidelines for partners. COVID-END convened topic-specific working groups to quickly address short and longer-term goals.

The Synthesizing and Recommendations Working Groups aimed to support access to and use of high-quality existing evidence syntheses, guidelines and HTAs in more coordinated and efficient ways, balancing quality and timeliness. A cross-cutting Equity Task Group was formed to evaluate synthesis methods to examine the equity impact of both the COVID-19 pandemic, and the clinical and policy interventions to mitigate its spread. These groups were created in a collaborative and inclusive space where representatives of stakeholder organizations worldwide could explore important issues and discuss solutions. These included the need for accelerated and living products to cope with the urgency and changing landscape of evidence need.

The Evidence Synthesis and Recommendations COVID-END Working Groups, comprised leaders from across evidence synthesis and guideline organisations representing both high and low-to-middle income countries. Following identification of global responses for evidence synthesis (see below), the working groups collaborated to develop a consolidated resource for evidence synthesis, clinical practice guidelines (CPGs), and HTAs available online at covid-end.com [12,13]. The goal of this resource is to support more robust evidence generation, reduce unnecessary duplication and increase efficiency of processes.

1.2. Examples of global responses for evidence synthesis and guidance

The pandemic and the subsequent infodemic have risked overwhelming the traditional system of evidence synthesis and guideline production, especially as producers of evidence and recommendations follow different processes and standards leading to divergent, and sometimes opposing, recommendations without adequate reason. This has resulted in confusion in policy, practice and for the public. The pandemic has shone a light on, and at times magnified fault lines that were already evident in our evidence eco-system [14]. Urgent decisions requiring synthesis in a faster way, poor collaboration, inconsistent quality, duplication of efforts, inequitable access to resources for review and guideline work in LMICs are issues that have been in existence for decades, but the pandemic has highlighted these stark inequities and inefficiencies which require urgent re-dress.

Fortunately, because COVID-19 was declared a pandemic, we have also seen an increase in evidence synthesis and guideline communities collaborating on a global scale to pool resources and evidence, reduce research waste and strengthen methods to support both the evidence-demand and the evidence-supply side of the pandemic response. These efforts have focused on producing rapid and living systematic reviews and guidelines, recommendation mapping, and evidence repositories, all aimed at reducing research waste and minimizing duplication of effort (see Box 1 ). Indeed, the COVID-19 pandemic has resulted in a breakthrough for living evidence and guidance, especially in response to decision maker needs.

Box 1. Examples of global evidence synthesis and guideline responses.

COVID-19 Evidence Synthesis Infrastructure (supporting evidence synthesis production): COVID-19 specific primary research repositories including the WHO COVID-19 global literature database, Epistemonikos L∗VE; COVID-19 prioritised evidence synthesis registration, such as through PROSPERO.

COVID-19 Evidence Synthesis Coordination: WHO Evidence Collaborative for COVID-19 Network (ECC-19), COVID-END, Cochrane.

COVID-19 Living Evidence Reviews: COVID-Network Meta-Analysis (NMA) by French Cochrane Group, COVID-19 Living NMA from McMaster University.

COVID-19 Evidence Synthesis Repositories (supporting evidence synthesis use) - Cochrane Review Special Collections, %%COVID-END inventory of best evidence syntheses.

COVID-19 Living Guidelines: Australian National COVID-19 Clinical Evidence Taskforce, WHO living guidelines, Pan American Health Organization (PAHO) living review, Infectious Diseases Society of America COVID-19 living guidelines, German living guidelines for COVID-19 (CEOsys)

COVID-19 Guideline Mapping: eCOVID-19 RecMap.

COVID-19 Rapid and Living Clinical Guidance and Evidence Reviews: American College of Physicians.

The evidence community has raised concerns about research waste, highlighted in the 2014 Lancet series, but still applicable today where substantial duplication of effort, in both evidence synthesis and guideline development, is prevalent [15]. The pandemic has however, afforded the opportunity for organizations doing evidence synthesis and guidelines as well as academic groups to collaborate, share evidence and reduce research waste and unnecessary duplication, and increase evidence generation efficiency.

1.3. COVID-END resources for evidence synthesis and guideline development

Our aim was to create resources that assist those already supporting decision-making to find and use the best available evidence and resources (i.e., to support the evidence-demand side) and reduce duplication and avoid redundancy to better centralize and coordinate the evidence syntheses, HTAs and guidelines being produced (i.e., the support the evidence supply side) – by compiling existing resources in one place to streamline access and enhance use. These resources are intended for developers, researchers, methodologists, academics, or people interested in using, developing, updating or adapting evidence synthesis, guidelines or HTAs.

Resources were developed during regular (often weekly) collaborative meetings of working group members with iterative input and garnering of comments via the COVID-END listserv. The working groups included clinical experts, methodologists, researchers, citizen partners and users of evidence synthesis, guidelines and HTAs from high-, middle- and low-income countries worldwide. Each resource was reviewed and shared with the wider COVID-END network and secretariat for peer review. Resources are accessible via the COVID-END website.

1.4. Evidence synthesis resources for COVID-19

This resource was developed for those considering and conducting evidence synthesis in response to a need identified. The interactive flow diagram provides users with information about different types of evidence syntheses, available resources, and highlights keys steps in the process of conducting a review of the evidence (Fig. 1 ) [12]. First, readers are advised to find and appraise existing evidence synthesis before embarking on developing a new one. The resource outlines key components, namely i) the value of evidence synthesis to inform decision making, ii) determining the need for a review through finding and appraising existing reviews for currency, credibility and comprehensiveness, and then either iii) use of an up-to-date review to inform decision making or, if there are certain gaps (i.e., not current, credible or comprehensive), then either iv) update an out-of-date review or v) take steps to conduct a new high quality, timely review. Each of these algorithmic steps is supported by a brief overview of the topic area and links to key resources.

Fig. 1.

COVID-END evidence synthesis and systematic resource flow diagram.

1.5. Support for evidence-based guidelines and health technology assessments

Similar to the evidence synthesis resource in approach and overall aims, the guideline and HTA resource has three major components, i) definitions and concepts, ii) guideline development methods iii) guideline tools and resources. We start by defining evidence-based guidelines (standard, rapid and living) and HTAs. Importantly, relevant for the COVID-19 pandemic or for future emergency situations, we provide examples of emerging methods for rapid and living guidelines and explain why robust, collaboratively developed evidence-based guidance is needed to inform decision making at clinical and policy levels.

When planning to develop a guideline, as a first step, we prompt developers to avoid duplication by initially searching for existing, high-quality, and up-to-date guidelines, including living guidelines by asking a clear guideline question. This should be followed by critical assessment of the guideline or HTA for relevance and quality, followed by convening the guideline team. Critically, a judgment must be made to either adopt an existing up-to-date, relevant, and high-quality guideline or, if gaps exist, to adapt the guideline to the context or with updated evidence synthesis. In the absence of trustworthy or up-to-date guidance; guideline organisations should consider de novo development [16]. The latter may apply to a whole guideline, or individual recommendations within the guideline. Similar processes can be followed for HTAs, with added nuances of evaluating different health or political systems for contextual alignment.

We provide a flow diagram to illustrate the process (Fig. 2 ). For each step we provide key resources, methodological research papers, approaches and examples, with some focus on the needs of LMICs where resources for de novo guideline development are more often limited. Lastly, we provide an overview of tools and resources supporting the development, publication and updating of guidelines. These tools include checklists and guides for development, reporting and appraisal according to standards for trustworthy guidelines (such as AGREE II [17] and CHEERS [18]) and software and technology (such as GRADEPro GDT [19] and MAGICapp [20]).

Fig. 2.

COVID-END Guideline and HTA flow diagram.

1.6. Example of application of these resources in South Africa

We present an example from South Africa, where its ministerially appointed National Essential Medicine List Committee urgently needed trustworthy information to inform their guidelines on how healthcare workers should treat people with COVID-19. In response to the pandemic, a subcommittee was established, supported by the South African GRADE Network, to ensure a responsive and evidence-informed guideline development approach, using adapted Cochrane rapid review methodology to ensure an acceptable compromise between rigor and timely responses [21]. Drawing from local expertise and review methods from COVID-END, the review teams conducted rapid reviews drawing on existing electronic COVID databases (e.g., L∗VE Epistemonikos and Cochrane Library's COVID Study register) and creating partnerships with living systematic review producers (covid-nma.com), which had already synthesized data and GRADE certainty assessments, when available. Rapid review reports were then drafted, input from subcommittee content experts received, and recommendations developed guided by the Evidence-to-Decision Framework. To date, this has resulted in the timely production of 42 rapid reviews informing national guidelines and is ongoing [22].

1.7. Call to action: evidence synthesis and guidelines

Despite considerable efforts from the evidence synthesis and guideline community, there will likely continue to be a degree of duplication in evidence synthesis and guideline production, not least due to differing contexts, competing interests and resources. COVID-19 has demonstrated the need for decision-making at speed and provided the context for a breakthrough for living systematic reviews and guidelines [23]. Florez et al. (2021) have proposed potential solutions to minimize waste by reducing duplication, by developing clear methods for living guidance and systematic reviews, encouraging adoption/adaptation of guidelines, strengthening guideline registration in central repositories and collaboration among others [9]. For decision makers, the Evidence Commission provides 24 recommendations that call for decisive action by multiple stakeholders to ensure evidence is consistently used to address societal challenges. The Commission grew out of a global network of 55 partners and provides a wake-up call and path forward for decision-makers, evidence intermediaries, and impact-oriented evidence producers [24].

Living systematic reviews and guidelines have been instrumental in informing decisions for COVID-19 (Box 1). As their development and updating is extremely resource-demanding they also underscore the need for broad multidisciplinary collaboration from evidence synthesis and guideline teams worldwide, including from LMICs. Living reviews and guidelines have the potential to promote equitable collaboration by building capacity in diverse teams across disciplines, income settings and geography, especially in countries where evidence synthesis capacity can be strengthened. Currently living reviews and guidelines are mostly focused around COVID topics, with most initiatives funded and run by high income settings. Other clinical topics can also benefit from the living review and guidance model (and models of guideline adaptation), such as in malaria and human immunodeficiency virus (HIV), with the additional aim of building further evidence synthesis and guideline development capacity in the lower income settings mostly affected by those diseases. Furthermore, we call for sharing of real time evidence synthesis results (supported by certainty of evidence judgements) so that governments can make rapid decisions drawing on the best available evidence and guidance.

1.8. Call to action: equitable collaborations

The COVID-19 pandemic has elucidated the enormous social injustice around health, making clear the importance of keeping equity considerations at the forefront of our endeavors in evidence synthesis, guideline development, collaborations, and decisions. Equity is not currently addressed in our resources, but it is important to note that there are different types of inequities to pay attention to in evidence synthesis. These include: disproportionate disease burden, inequitable distribution of resources; inequitable availability/access to resources; differences in effectiveness of interventions and inequitable collaborations. Inequities in distribution of resources such as therapeutics and vaccines between high income and LMIC, as well as within countries, and insufficient support for lower income settings to build stronger health systems and services, has and will prolong the pandemic for all.

Inequities impact groups of individuals differently, and we point to the PROGRESS-Plus framework to understand how best to report on population subgroups when conducting evidence syntheses, whether in a pandemic or future research [25]. Furthermore, health equity issues must be prospectively considered in the composition of the research teams, study design, conduct and analysis of systematic reviews and other types of evidence synthesis. A call for equity considerations is proposed that includes PROGRESS-Plus characteristics and considerations from Oxman (2009) [26] in evidence synthesis and guidelines (see Box 2 ). The COVID-END Equity Task group developed guidance for considering equity in the context of rapid evidence synthesis [27].

Box 2. Call for equity considerations in evidence synthesis and guidelines.

Reflect on equity in the research team composition or guideline development group;

Consider formulating research questions with a focus on health equity;

Consider using the PROGRESS-Plus framework to identify characteristics which health inequities may exist and report accordingly;

Consider whether there is evidence of differences in baseline conditions across PROGRESS-Plus characteristics or geography (e.g., LMICs and HICs), or for groups within these settings or characteristics, which would result in differences in the absolute effectiveness of the intervention;

Consider whether there is evidence of differences in access to care or the quality of care across PROGRESS-Plus characteristics; and

Consider the implications of these differences for implementing the intervention to ensure that inequities are reduced if possible and they are not increased.

2. Summary

In summary, we share details of comprehensive, pragmatic and dynamic resources, for both evidence synthesis and guideline development, to support existing efforts in combating COVID-19. Our resources, spurred by the urgent needs of decisions makers during the COVID-19 pandemic, and developed through discussions at weekly meetings, provide efficient ways to find links to cutting edge methods for high quality systematic reviews and guidelines, including the new paradigm of living evidence. We call on guideline and HTA producers, professional societies and researchers to reduce wasteful duplication of efforts by active collaboration and generous sharing of evidence. This would mean that countries and health care systems can draw on existing high-quality systematic reviews and guidelines in their work and focus their often-limited capacity toward more efficient adoption or adaptation contextualized to their settings. Furthermore, we recommend thoughtful consideration of equity as well as patient and public involvement in all endeavors related to health decision-making - evidence synthesis, guideline development, diverse collaborations and fair decisions - that ensure equitable access to resources for better health for all globally.

Declarations

Taryn Young is supported by Research, Evidence and Development Initiative (READ-It) (Project number 300342-104), funded by UK Aid from the government of the United Kingdom; however, the views expressed do not necessarily reflect the United Kingdom government's official policies. David Tovey has received consultancy funding from both COVID-END and COVID-NMA. Andrea Tricco holds a Tier 2 Canada Research Chair in Knowledge Synthesis. David Tovey is Editor-in-Chief and Andrea Tricco is an Associate Editor for the Journal of Clinical Epidemiology; neither were involved with the decision to publish or the publication process. Authors hold sole responsibility for the views expressed in the manuscript, which may not necessarily reflect the opinion or policy of the Pan American Health Organization.

CRediT authorship contribution statement

Michael McCaul: Conceptualization, Writing – original draft, Visualization, Resources. David Tovey: Writing – review & editing, Resources. Taryn Young: Writing – review & editing, Resources. Vivian Welch: Writing – review & editing, Resources. Omar Dewidar: Writing – review & editing, Resources. Mireille Goetghebeur: Writing – review & editing, Resources. Tamara Kredo: Writing – review & editing, Resources. Andrea C. Tricco: Writing – review & editing, Resources. Rebecca Glover: Writing – review & editing, Resources. Janice Tufte: Writing – review & editing, Resources. Amir Qaseem: Writing – review & editing, Resources. Reveiz Ludovic: Writing – review & editing, Resources. Rebecca L. Morgan: Writing – review & editing, Resources. Per Olav Vandvik: Writing – review & editing, Resources. Ivan D. Florez: Conceptualization, Writing – review & editing, Visualization, Resources.