What evidence-based medicine researchers can do to help clinicians fighting COVID-19?

Highlights

• •The World Health Organization has declared COVID-19 pandemic as a global public health emergency.
• •Research community has responded rapidly, and thousands of articles have been published already in scholarly journals, making it difficult for clinicians to sift through the evidence and find answers.
• •Methodologist researchers and relevant international organizations have addressed the pandemic immediately. However, these efforts are hindered with avoidable problems and massive duplication of effort.
• •We believe that research methodologists can be very useful now to clinicians. The COVID-evidence project can be an example of how to establish a living dialog of methodologists and clinicians to fight COVID-19 pandemic.

1. Introduction

A new disease, COVID-19, has emerged in December of 2019 and swept the world; the World Health Organization (WHO) has declared a COVID-19 pandemic on March 11, 2019 [1]. On April 14, 2020, there were 1,926,235 people infected with SARS-CoV-2 globally, and 119,724 have died [2]. Thus, COVID-19 became a global public health emergency. Research community has responded rapidly, and thousands of articles have been published already in scholarly journals, making it difficult for clinicians to sift through the evidence and find answers. We believe that research methodologists can be very useful now.

2. How methodologists can help

The term methodologist or methodological expert is not clearly defined. In the context of evidence-based health care and in particular in knowledge synthesis, an expert is someone with excellence in synthesizing evidence. To achieve such expertise, a lot of different skills need to be acquired along with experience in multiple projects. However, the concept of knowledge synthesis should not only be limited to performing different types of reviews (e.g., systematic, scoping, and rapid); it also includes questions of how to make evidence available, and how to communicate findings to the public and decision-makers. All of this is currently extremely important. We need to know where to identify new studies as fast as possible. Based on them, reviews are conducted, whereby many of them may follow methods developed for rapid reviews, which may reduce certainty in their findings. All needs to be communicated fast and to the appropriate decision-makers. The huge challenge for methodological experts is that given the current circumstances, it will frequently not be possible to use the optimal methods for knowledge synthesis, and to balance challenging trade-off between optimal methodology and timely results.

3. New challenges for evidence-based medicine vs. COVID-19

Several key aspects define the new context of action for clinical decision-making in the ongoing pandemic. First, clinicians lack time to critically assess the quality of the evidence that is appearing about COVID-19; on April 14, 2020, there were 5,362 articles stored in the WHO database (based on searches up to April 9, 2020) [3]. Second, although it seems that the scope of research is very narrow, because it addresses evidence on a well-defined core topic (COVID-19) and on a well-defined time frame, available publications address very different aspects of the problem and are presented in multiple formats, some of which (letters, opinion, news, comments) would be excluded when performing any methodological quality assessment or conducting a systematic review. Third, much of the published evidence is based on experience gained in similar contexts in different countries, by clinicians who have made the effort to communicate their observations and to share the results obtained after they made decisions in extreme situations, not always supported by evidence-based standards.

4. COVID-19 and knowledge synthesis: what is currently very wrong

Methodologist researchers and relevant international organizations have addressed the pandemic immediately, and responded swiftly with knowledge synthesis. However, these efforts are hindered with avoidable problems and massive duplication of effort.

Up to March 24, 2020, there were 18 systematic reviews on COVID-19 published already in scholarly journals, but 13 of them were rated as “critically low” with AMSTAR 2, in terms of confidence in their results, due to methodological shortcomings [4].

Multiple online collections of articles are dedicated to COVID-19. The WHO is curating a collection of articles, mainly journal articles, called “Coronavirus Disease (COVID-19) Pandemic: An Overview of Systematic Reviews”, by searching multiple databases and hand-searching relevant journals. The web page with the collection allows filtering of articles based on type (article vs. “nonconventional”), journal, and year of publication [5].

EPPI-centre, based in London, UK, is publishing a “COVID-19: a living systematic map of the evidence” [6], based on searching of Embase and MEDLINE. The map has a visually appealing presentation of the number of articles published in 12 thematic areas, and also studies that were not included because they do not have primary data; by clicking in each part of the map, index on the right side appears with the list of those articles [7]. LOVE platform (Living OVerview of Evidence), powered by Epistemonikos, screens articles on COVID-19 and enables PICO question builder [8].

Cochrane announced on April 7, 2020, that it has set up a registry called COVID-19 Study Register, which will be continuously updated with studies on humans about COVID-19 [9]. Primary data sources for COVID-19 Study Register are PubMed and two registries of clinical trials, including ClinicalTrials.gov and the WHO's International Clinical Trials Registry Platform [10].

A network of researchers from different institutions, led by Dr. Jeremy Grimshaw and Dr. John Lavis from Ottawa Hospital Research Institute and McMaster Health Forum/RISE (Canada), are setting up an initiative called Evidence Network to support Decision-making (COVID-END) [11]. This network aims to coordinate and reduce duplication in efforts among all types of researchers, including in the evidence synthesis, technology assessment, and guideline-development communities that have long track records of supporting decision-makers locally, nationally, and internationally.

There are other COVID-19 databases; these are probably the most prominent [12]. However, these platforms are very similar. Each of these organizations is investing massive human resources to develop and maintain these evidence maps, whereas they could be all working together and prevent overlaps and duplications. Furthermore, these resources do not offer appraisal of methodological quality of evidence. When clinicians start searching evidence on a certain topic in these collections, they will get a list of articles, but they will not know whether this evidence is good, or good enough in terms of methodological quality. Articles presented to clinicians could be useless examples of research waste. Furthermore, these databases could be more transparent in terms of their methods. In some cases, it is not well described which databases and journals are searched. In the WHO database of COVID-19 publications, it is unclear what is “nonconventional” information source and the searching option by year is not optimal, as it would be much more helpful to allow filtering in line with different article types and exact publication dates. In addition, most of these resources are in English. An updated interface of the WHO collection now includes different language options, including Chinese, French, Spanish, Russian, Arabic, and Portuguese. Clinicians in countries where English is not the first language will likely appreciate platforms in their mother tongue. Finally, these collections should offer user-friendly formats that would be more accessible to busy clinicians, such as podcasts.

5. What clinicians need and how to establish a living dialog of methodologists and clinicians

At Reina Sofia University Hospital, a tertiary hospital in Spain, which cares for a population of 786,524 citizens, we have listened to the call for help from our clinicians and have started the COVID-evidence project to give them support, taking into account all aforementioned context features. Our primary source of information is the WHO database about COVID-19. After importing records from the WHO database into the Cochrane's covidence web software [13], six experienced researchers have started curating the records by deeply tagging every study using an ad hoc living and multilevel free vocabulary approach. In the first version of the web service [April 12, 2020], there were 2,577 abstracts and 14,309 tags with up to 32 tags/abstract. The complexity of such granular tagging will serve to i) deploy a new free open tag-based searchable database to be used by any researcher worldwide [14]; ii) to locally post a weekly updated executive summary about the amount of new evidence, by topics, fields, and study designs, which will include a methodological evaluation and any concerns about the confidence of their results; iii) to weekly publish a free available podcast with some contents of the executive summary to facilitate all clinicians to consume our analyses whenever and wherever they can/want; and iv) to communicate daily with users via social messaging apps. For the latter, we have created an intrahospital WhatsApp team, “the COVID-evidence team”, involving clinicians (intensive care, internal medicine, pneumology, infectious diseases, and epidemiology and health care workers) and methodologists. The team communicates via WhatsApp chat with end users. The chat enables end users to propose new questions in a PICO format for us to search scientific answers using our COVID-evidence web service. Methodology quality assessment of primary or secondary included studies will be performed on demand to answer every specific PICO question sent by clinicians. Final reports will be uploaded to the platform to share our analysis with the scientific community. Finally, a contact form is available on the web page for those methodologists interested in collaborating on this project.

6. Final reflections

The scenario for evidence-based medicine has changed. Government agencies, institutions, and research groups keep working harder to produce summaries and quality assessments documents of existing evidence. However, we believe that there is room for specialists in research methodology to provide our best knowledge and expertise, without forgetting the rigor of evidence-based medicine research standards, to support clinicians who are making decisions on the front line. In this duality of interest, we believe that new strategies for new times will be required to address it. The example of COVID-evidence project in Spain indicates that collaboration of clinicians and methodologists can produce evidence synthesis with assessment of methodological quality “on demand” and based on “a living dialog”, in a non-English language, that the end users use and appreciate. We hope that our love for the best methodological standards will not paralyze us and prevent us to offer clinicians our support just in the right time.