The Fast Literature Assessment and Review (FLARE) Initiative. A Collaborative Effort for Timely Literature Appraisal during the Coronavirus Disease Pandemic

Abstract

The emergence and worldwide spread of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has caused major disruptions to the healthcare system and medical education. In response, the scientific community has been acquiring, releasing, and publishing data at a remarkable pace. At the same time, medical practitioners are taxed with greater professional duties than ever before, making it challenging to stay current with the influx of medical literature.

To address the above mismatch between data release and provider capacity and to support our colleagues, physicians at the Massachusetts General Hospital have engaged in an electronic collaborative effort focused on rapid literature appraisal and dissemination regarding SARS-CoV-2 with a focus on critical care.

Members of the Division of Pulmonary and Critical Care, the Division of Cardiology, and the Department of Medicine at Massachusetts General Hospital established the Fast Literature Assessment and Review (FLARE) team. This group rapidly compiles, appraises, and synthesizes literature regarding SARS-CoV-2 as it pertains to critical care, relevant clinical questions, and anecdotal reports. Daily, FLARE produces and disseminates highly curated scientific reviews and opinion pieces, which are distributed to readers using an online newsletter platform.

Interest in our work has escalated rapidly. FLARE was quickly shared with colleagues outside our division, and, in a short time, our audience has grown to include more than 4,000 readers across the globe.

Creating a collaborative group with a variety of expertise represents a feasible and acceptable way of rapidly appraising, synthesizing, and communicating scientific evidence directly to frontline clinicians in this time of great need.

The emergence of the novel coronavirus, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), has caused major challenges for healthcare systems and corresponding disruptions of medical education. At the same time, the scientific community has been acquiring, releasing, and publishing data at a remarkable pace. According to LitCovid, an online repository of peer-reviewed, PubMed-indexed articles published about coronavirus disease (COVID-19), there have been over 8,400 articles published since January 17, 2020 (1). Furthermore, in the Stephen B. Thacker Library of the Centers for Disease Control and Prevention, there are over 10,000 research articles pertaining to COVID-19 (2).

The rapid proliferation of papers has been facilitated by biomedical journals amending their editorial processes and removing subscription requirements to access COVID-19–related content (3–5). Furthermore, use of preprint servers to circulate prepublication, non–peer-reviewed articles has increased. As of April 30, 2020, the preprint servers bioRxiv and medRxiv contained over 2,500 articles related to COVID-19 and SARS-CoV-2 (6). Quick dissemination on social media platforms has resulted in heightened attention to these preliminary reports. This widespread availability, particularly of non–peer-reviewed literature, increases the risk that articles of questionable scientific quality may be misinterpreted, thereby misinforming clinicians and the public (7). Furthermore, many healthcare providers deployed to clinical work outside of their area of expertise may not be in a position to critically interpret the literature.

Healthcare providers therefore have an intensified need for timely, accurate, and curated data at the same time they face increased clinical and administrative duties and fewer opportunities to use traditional channels for parsing new scientific findings. To address these needs and support our colleagues, physicians at Massachusetts General Hospital (MGH) engaged in a collaborative effort focused on rapid literature appraisal and dissemination regarding SARS-CoV-2 with a focus on critical care. Our intent was to evaluate, not just summarize, the rapidly growing literature. This innovations article describes this educational initiative.

Methods

Creating the Group

In response to a call for those interested in providing education, members of the Division of Pulmonary and Critical Care, the Division of Cardiology, and the Department of Medicine at MGH established the Fast Literature Assessment and Review (FLARE) team (Figure 1). Led by a senior member of the division (C.C.H.), the group comprises five attending physicians (C.C.H., V.V., L.N.B., R.R.C., and D.M.D., with expertise in pulmonary physiology and biophysics, genetics, developmental biology and diagnostics, and cardiac critical care, respectively), two pulmonary and critical care fellows (J.H.M. and C.R.P., with expertise in healthcare delivery science and medical education, respectively), and one internal medicine resident (T.F.C.) interested in critical care.

Figure 1.

Logo for Fast Literature Assessment and Review (FLARE).

Daily Workflow

The group schedules daily video calls in the afternoon using the Zoom platform (Zoom Video Communications). Meetings consist of a discussion of the following: 1) administrative concerns, 2) topical questions or recent articles, 3) revisions and critiques of drafted pieces, and 4) planning subsequent mailings. Editing of FLARE pieces is accomplished collaboratively, in real time, via Google Docs (Alphabet Inc.) during the meeting. In this way, work can be both asynchronous and dynamic, allowing feedback and discussion. In addition, the group communicates frequently throughout the day via e-mail and on a mobile chat platform.

Identifying Topics for FLARE Issues

Potential topics for FLARE issues (referred to simply as “FLAREs”) are identified in multiple ways, including questions submitted by colleagues, dilemmas arising from clinical experience, and highly publicized articles in the scientific community or popular topics in the lay press. Databases, such as PubMed and LitCovid, and high-profile medical and critical care subspecialty journals are reviewed daily for publications of interest. There is an emphasis on reviewing subjects within our areas of expertise to ensure accuracy.

Appraising the Literature

Multiple services exist that index new literature related to COVID-19 (1, 6). However, the quality of this literature varies from journal articles that have undergone traditional review, articles that received expedited review, and even unreviewed preprints. A primary intent of FLARE is to provide a curated review that evaluates, not just summarizes, the available literature on a particular topic.

A core member selects a particular subject (e.g., what data exist about the use of tissue plasminogen activator in acute respiratory distress syndrome?) and performs a literature review. This spans databases of scientific journals and articles in the lay press, if appropriate. We prioritize peer-reviewed articles and emphasize scientific rigor in our appraisal. To that end, there is an effort to avoid prepublication reports (though often attention received by these reports generates the topic of a FLARE, and thus they are mentioned in this context). Care is taken to identify the caveats associated with preprint reports when included in a FLARE.

Pieces are written in a formal style, often including an opinion from the authors, with references throughout akin to a peer-reviewed analytic review (or editorial). The author presents the piece to the FLARE team for discussion and peer review during the meeting. Though there is a point person for each piece, the literature appraisal is accomplished in a collaborative fashion. During the daily meeting, key references are reviewed by the group, similar to a journal club format, in which the merits of the publication are discussed. This often cultivates a lively discussion, enhanced by the various expertise held by members of the FLARE team. If the particular topic is beyond the expertise of the group, outside review is sought from subject matter experts within the broader MGH community.

Producing the FLARE

After this collaborative discussion, the piece is revised in an iterative fashion by the FLARE group. All members assent to the content before production.

Dissemination

Initially, FLAREs were disseminated via our division’s internal e-mail listserv. However, as the mailings became more complex, a different platform was needed. Mailchimp (mailchimp.com) was selected for its ease of use, enhanced graphic design, and familiarity to FLARE members. Mailings are sent out nightly.

Seeking User Feedback

We track the number of subscribers to the Mailchimp campaign, their affiliated institutions, and which e-mails are opened. Furthermore, we solicit comments and feedback via e-mail and personal communication. Currently, we have collected only preliminary and qualitative data about our work.

Results

The first FLARE was distributed on March 19, 2020. Since then, FLARE has produced 40 mailings (as of April 30, 2020) covering a variety of topics (Table 1). Eight mailings have been opened by more than 70% of subscribers, with the more than 75% of mailings opened by at least half of the readership.

Table 1.

Examples of topics covered in Fast Literature Assessment and Review issues as of April 30, 2020

Pathophysiology and biology

Biology of the SARS-CoV-2 viral spike protein

Chloroquine and its potential role in COVID-19

Hyperinflammation in critical illness and its application to coronavirus infections

Exploring concerns about the renin-angiotensin system in COVID-19

Antibody-dependent enhancement in COVID-19

Respiratory failure and critical care

Is COVID-19 ARDS?

Prone position in the nonintubated patient

Use of inhaled nitric oxide in ARDS and potential role in COVID-19

Exploring the role of disordered coagulation in ARDS in COVID-19

Review of PEEP optimization and recruitment maneuvers in ARDS

Revisiting investigational therapies for ARDS in the setting of COVID-19

The role of early tracheostomy in patients with respiratory failure resulting from COVID-19

Postextubation stridor in COVID-19

Fluid management in ARDS

Clinical care

Rationale and evidence for convalescent plasma use in COVID-19

COVID-19 presenting with GI disease

Current data regarding myocarditis and COVID-19

Utility of CT scans in patients with COVID-19

Procalcitonin in SARS-CoV-2

Reviews of recent publications

Summary and analysis of recent lopinavir/ritonavir trial in COVID-19

Distillation of literature on risk factors for COVID-19

Analysis and comparison of recently published case series of patients with COVID-19

Evidence for remdesivir use

Definition of abbreviations: ARDS = acute respiratory distress syndrome; COVID-19 = coronavirus disease; CT = computed tomography; GI = gastrointestinal; PEEP = positive end-expiratory pressure; SARS-CoV-2 = severe acute respiratory syndrome coronavirus 2.

Our early work was distributed solely to colleagues at MGH in the Division of Pulmonary and Critical Care (≈162 members). Quickly, the FLAREs were shared outside of our division, and hospital leadership became interested in making FLAREs available both internally for all staff and externally for the public. With Hospital Incident Command System approval, a governing group activated to respond to the pandemic, the FLAREs are now archived and extramurally available (www.massgeneral.org/flare).

The number and diversity of subscribers to FLARE has increased rapidly (Figure 2). As of April 30, 2020, there were over 4,000 readers from over 30 different institutions and hospital systems, including subscribers in South America, Africa, Europe, Asia, and the Middle East.

Figure 2.

Line graph showing the rapid increase in subscribers outside of the Massachusetts General Hospital Division of Pulmonary and Critical Care. FLARE = Fast Literature Assessment and Review.

Qualitative feedback on our work has been positive. Comments have been received via e-mail, either unsolicited or in response to a specific FLARE. Internally, colleagues have been grateful for the concise summaries and readily available compilation of important articles and topics regarding SARS-CoV-2. Comments have included, “For those of us on service who don’t have time to parse the literature, it is really, really helpful to get these updates and thoughtful summaries as we work out the best strategies for these patients,” and, “The thought process is so clear; the review and criticism are so spot on!” Specific comments regarding the availability of a repository have included, “I’ve been sharing specific issues in response to questions, and encouraging people to read through the past issues.”

Discussion

As physicians, scientists, clinicians, and medical educators, we proactively sought to distill and analyze clinically relevant information to support our colleagues during this unprecedented emergency. Convening and formalizing a writing group composed of highly motivated physicians with a variety of expertise has facilitated the production of rigorous yet topical pieces covering a broad range of subjects relevant to the COVID-19 outbreak.

Lessons Learned

Using an online platform for dissemination via e-mail has allowed our audience to increase organically and without need for deliberate promotion. Furthermore, the support from our own institution has been key to increasing our online presence, both internally and externally. We are fortunate to work in a community of connected and supportive medical professionals, allowing FLAREs to permeate across institutions.

Partnering with physicians with a variety of expertise has strengthened the rigor of our work. We recommend that others considering similar interventions incorporate colleagues of diverse academic backgrounds into the team.

Because FLARE is endorsed by our home institution, and because we distribute to readers outside MGH, we have included a statement regarding how our recommendations should be used. This includes a recognition that our pieces represent the opinions of FLARE authors but should not be substituted for clinical judgment for an individual patient.

Challenges

Challenges include managing the rapid influx of scientific knowledge and demands on the time of the core group, particularly as the outbreak accelerates. On average, the FLARE requires 1–3 hours of work per member per day, though this varies, depending on the piece. Given growing clinical demands and the high level of engagement currently required of the core group, a question naturally arises about sustainability. In this first phase of the current COVID-19 outbreak, FLAREs are produced nightly. Going forward, the frequency will be decreased. In addition, a certain portion of the current time devoted to FLARE by the core members is spent in graphic design and other administrative tasks that we hope to offload to dedicated support personnel in the future.

Though Mailchimp provides basic information about our subscribers, we do not know how our readers use FLAREs. This limits our ability to assess the effectiveness of FLARE as an educational intervention. Important questions to answer include how and why FLAREs are accessed and what attitudes our readers have about FLARE.

Furthermore, our readership comprises a broad audience, including not only clinicians but also journalists (e.g., The New York Times and STAT), biologists, and those in the pharmaceutical industry (e.g., Merck and Pfizer). As such, the utility of each piece varies for individual reader. Though we are lucky to have attracted a diverse readership, it is our primary goal to support clinicians caring for patients with COVID-19, and thus assessing the effectiveness of the FLAREs will be judged with this in mind.

Finally, as the understanding of COVID-19 matures, prior FLAREs may become outdated. To address this, we do revisit important and evolving topics in subsequent FLAREs. In the future, curation of the online repository will be important.

Potential Solutions Encountered Thus Far

To assess the effectiveness of FLARE, we are working to glean data metrics about the use of our website. In addition, expanding this webpage will allow us to provide more sophisticated organization of the FLAREs, including a searchable database and a conduit for interaction with our readers.

Given the substantial investment required by FLARE members, we intend to apply for grant funding to support our ongoing efforts. Budgetary items may include administrative support (e.g., managing the website), graphic design (to enhance the style of the mailings), and perhaps salary support.

The COVID-19 pandemic has prompted the scientific and medical communities to develop creative solutions to balance new stresses. Creating a collaborative group comprised of critical care physicians with a variety of expertise, interacting electronically and frequently via messaging apps, has been a feasible and well-received way to rapidly appraise, synthesize, and communicate scientific evidence. Though our ability to evaluate the effectiveness of this intervention has been limited, we intend to pursue a more formal assessment of this educational endeavor in the future. Through this platform, we look forward to continuing to support our colleagues caring for patients with COVID-19.

Readers may sign up for FLARE at: eepurl.com/gXeoZf.