Keeping Up With Emerging Evidence in (Almost) Real Time

Abstract

In Annals' first rapid, living systematic review, Chou and colleagues examined the burden of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), SARS-CoV-1, and Middle East respiratory syndrome coronavirus on health care workers and risk factors for infection. The editorialists discuss the role of these review types and describe the features they will look for when Annals is considering rapid and living reviews for publication.

Although many aspects of daily life have come to a halt during the coronavirus disease 2019 (COVID-19) pandemic, research certainly has not. The pursuit of evidence to prevent infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), improve patient outcomes, and quell the pandemic is fast and furious. Clinicians and epidemiologists are rushing to report their observations, investigators are launching trials with very short timelines for completion, and journals are receiving record numbers of manuscript submissions. In just 6 weeks during March and April 2020, Annals received one quarter the number of manuscript submissions we received during all of 2019 and, like many journals, Annals is speeding peer review and publication of manuscripts found to be suitable for our readers. This frenetic pace can leave us with “evidence whiplash.” What was thought to be true or promising one day is sometimes found to be false or dangerous a short while later. Continual piecemeal release of evidence can be confusing and can contribute to information overload, especially when the new evidence is flawed or uncertain. The rapid accumulation of evidence presents particular challenges for those working to summarize available evidence to inform answers to the many pressing questions the pandemic has generated. Rapid and/or living systematic reviews offer a potential solution (1, 2).

Annals' first rapid, living systematic review appears in this issue. Chou and colleagues examine the burden of SARS-CoV-2, SARS-CoV-1, and Middle East respiratory syndrome coronavirus on health care workers (HCWs) and risk factors for infection (3). They searched multiple electronic databases, including a World Health Organization database of publications on coronavirus diseases and the medRxiv preprint server, from 2003 through 27 March 2020, with ongoing surveillance through 24 April 2020, for studies reporting incidence of or outcomes associated with coronavirus infections in HCWs and studies on the association between risk factors and infection. They found that HCWs experienced significant burden from coronavirus infections and that personal protective equipment and infection control training were associated with decreased infection risk while certain exposures were associated with increased risk. Of the 64 studies that met inclusion criteria, only 18 addressed SARS-CoV-2. Recognizing that the conclusions of this systematic review will likely become outdated quickly as more evidence on SARS-CoV-2 emerges, the authors planned for this systematic review to be both “rapid” and “living.”

A rapid review simplifies or omits some components of the systematic review process to produce information in a timely manner (1). Given the urgency of the pandemic, Chou and colleagues streamlined the systematic review process by not registering the protocol prospectively, limiting the search of the gray literature, restricting dual review to only 25% of abstracts, conducting single-reviewer data abstraction and assessment of study limitations, and using no formal instrument for critical appraisal of the evidence (3).

A living systematic review includes a prospective plan for continual surveillance of evidence with periodic critical appraisal and synthesis of new evidence (2). The methods of living systematic reviews are similar to those of traditional systematic reviews, though living reviews also include explicit, transparent, and predefined decisions about how often and for how long new evidence will be sought and screened, as well as when and how new evidence will be incorporated into the review (4). For example, Chou and colleagues plan to update their search monthly (3). Both rapid reviews and traditional systematic reviews may become living reviews.

Those who conduct living reviews commit to immediate incorporation of any newly identified important evidence. Journals that publish living reviews must commit to communicating the updated evidence to readers. At Annals, we anticipate 3 potential outcomes of periodic evidence updates: no new evidence is identified; new evidence is identified, but it changes neither the nature nor the strength of prior conclusions; and new evidence emerges that warrants a substantive change in the nature and/or strength of the prior evidence synthesis.

Annals will require authors of living reviews to post a “surveillance comment” to the initial review, at the interval prespecified in its methods, describing the results of the updated search. When authors identify no new evidence, this comment will serve to alert readers that an updated search was performed but identified no new eligible evidence. When authors find evidence that is pertinent but does not alter prior conclusions, they should cite and briefly describe, critique, and contextualize the new evidence. Annals will publish these minor updates as “new evidence alerts” in the Letters section, where they will be linked to and indexed with the initial review article. When authors believe that new evidence changes the nature or strength of the conclusions or when evidence synthesis requires major changes in methods, they should consult the editors about whether a major update is warranted. If the editors believe that it is, the authors should submit a new systematic review for peer review.

Annals is committed to publishing high-quality, timely evidence relevant to clinical medicine and public health. We look forward to receiving rapid, living systematic reviews on rapidly changing issues. Authors interested in submitting rapid or living systematic reviews to Annals should consult the “Information for Authors” page at Annals.org for details on format. We also look forward to feedback from those who conduct the evidence reviews and from our readers to help us refine their format and publication process.

Those who conduct, publish, and rely on systematic evidence reviews have long expressed frustration when new information renders an evidence synthesis out-of-date, sometimes soon after completion (5). The substantial amount of pandemic-related evidence posted on preprint servers before peer review highlights another important feature of rapid and living reviews: the need to search the gray literature and to critically appraise and synthesize such evidence. The concept of rapid and living reviews garnered attention well before the COVID-19 pandemic, but the tsunami of new information related to this novel coronavirus creates an environment that is primed to demonstrate the promise (and also possibly the pitfalls) of rapid, living reviews.

Biography

Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M20-2627.

Corresponding Author: Christine Laine, MD, MPH, American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106; e-mail, claine@acponline.org.

Current Author Addresses: Drs. Laine and Taichman: American College of Physicians, 190 N. Independence Mall West, Philadelphia, PA 19106.

Dr. Guallar: Departments of Epidemiology and Medicine, Johns Hopkins Bloomberg School of Public Health, 2024 East Monument Street, Room 2-645, Baltimore, MD 21205.

Dr. Mulrow: University of Texas Health Science Center at San Antonio, 7703 Floyd Curl Drive, San Antonio, TX 78229.