A systematic review that is ``rapid'' and ``living'': A specific answer to the COVID-19 pandemic

Abstract

Objective

This study aims to describe “rapid living” systematic reviews, an innovative methodological design used to systematically synthesize emerging evidence in the field of rehabilitation during the COVID-19 pandemic.

Study Design and Setting

A methodological paper, with a formative approach to rapid living systematic reviews.

Results

Based on our experience, we propose the following definition of rapid living SR: ``A dynamic method of knowledge synthesis that allows for the constant updating of new emerging evidence and refinement of its methodological quality.'' This method has the benefit of accelerating the conduct of traditional systematic reviews and allows for a synergistic adaptation of methodology based on the quality of the evidence with a flexibility to update results, methods and collaborations.

Conclusion

Our proposed methodology has been helpful to synthesize the rapidly evolving evidence in the field of rehabilitation during the pandemic. Similarly, it may be useful when a rapid answer is urgently needed to make informed decisions. The COVID-19 disease has shown that modern medical science has the ability to produce new knowledge at a rate never seen before. Therefore, our proposed rapid living systematic reviews provide the scientific community with a method to rapidly synthesize evidence when facing health emergencies.

What is new?

Key findings

• •The combination of “rapid” and “living” allowed researchers to be very efficient in retrieving and synthesizing the evidence while gradually improving the methodology of systematic reviews for a “rapid” and constant production of knowledge synthesis to inform the clinical decision-making and help to give urgent answers during COVID-19 pandemic.

What this adds to what is known?

• •The novelty of this study is the combination of “living” and “rapid” concepts that allows to be very efficient in retrieving and synthesizing the evidence while gradually improving the methodology. The idea is that, in time, the methods will meet the methodological quality of a classical SR while keeping the pace with the updates.

What is the implication, what should change now?

• •Based on our experience, we propose the following definition of rapid living SR: A type of knowledge synthesis that accelerates the process of conducting a traditional systematic review and is constantly updated by incorporating relevant new evidence as it becomes available, and gradually improving its methodological quality, if needed.

1. Introduction

From the beginning, the COVID-19 pandemic required a rapid production of evidence to inform the clinical decision-making and help to give urgent answers. The need to rapidly produce knowledge syntheses is imperative in the field of rehabilitation too, because rehabilitation, as stated by the World Health Organization (WHO), is an essential health service together with promotion, prevention, treatment, and palliative care [1]. Among Cochrane, the Rehabilitation Field [2], is responsible for knowledge translation in this specific area of health [3] by bringing the best available evidence synthesized by Cochrane to rehabilitation stakeholders. Due to the urgent needs for rehabilitation by people with COVID-19, Cochrane Rehabilitation merged the concepts or ``rapid'' and ``living'' systematic reviews adapted to the and designed an innovative and dynamic methodological approach for evidence synthesis during times of health emergencies [4]).

Cochrane defines a rapid SR ``a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a resource-efficient manner'' (https://covidreviews.cochrane.org/sites/covidreviews.cochrane.org/files/public/uploads/cochrane_rr_-_guidance-23mar2020-final.pdf). The concept was introduced to face sudden needs of the health systems while accepting necessary methodological limitations to accelerate the speed of publication. Cochrane also defines a living SR a ``systematic review which is continually updated, incorporating relevant new evidence as it becomes available'' (https://community.cochrane.org/sites/default/files/uploads/inline-files/Transform/201912_LSR_Revised_Guidance.pdf). The notion of living reviews was introduced to keep the pace with the growth of evidence, and it can be particularly useful in rapidly evolving research fields. The pandemic brings the two challenges together, i.e. the need for rapid knowledge synthesis but also that of constant update.

Therefore, this paper aims to present and define the methodological innovation of “rapid living” systematic reviews and uses our experiences to discuss its strengths and weaknesses.

2. Description of the methodological innovation

Rehabilitation is currently defined for research purposes by Cochrane Rehabilitation as “a multimodal person-centered process including functioning interventions targeting body functions, and/or activities and participation, and/or the interaction with the environment” (Intervention) aimed at “optimizing functioning” (Outcome) in “persons with health conditions (a) experiencing disability or (b) likely to experience disability, and/or persons with disability” (Population)” [5]. Rehabilitation comes partly during, but mostly after the acute phase of illness [6]. Moreover, rehabilitation research faces specific methodological challenges related to its: (1) multimodal approach which involves a combination of behavioral and physical interventions [7], [8], [9]; (2) multiprofessional team approach [10]; and (3) focus on optimizing individual functioning while accounting for multicomorbidities of people who need rehabilitation [11], [12], [13]. These needs often result in delays in completing studies when compared to other fields of medicine.

Very early in the pandemic, Cochrane Rehabilitation designed and conducted a SR to meet the demand from the stakeholders to receive timely information about the impact of COVID-19 on rehabilitation activities and patients. In April 2020, the first SR was published within 3 weeks from its initiation and included the evidence produced since the start of the pandemic in China [14]. Therefore, our first review can be defined as ``rapid'' needed to be flexible by incorporating-reported findings from all study designs, including expert opinions. Because this low-quality body of evidence was growing rapidly, we also conceived the SR as ``living'' to keep it constantly updated (each month from the first edition). In all 2020, we published a first rapid living systematic review (RLSR) (first edition) [14] with two monthly updates [15,16] keeping a time of 3–4 weeks between data collection and their publication); the same was done for the second RLSR (second edition) [17] that included five monthly updates [18], [19], [20]. The second edition introduced a methodological improvement related to data extraction and reporting, and excluded expert opinions and the updates were the “living” part of the main papers. The development and conduct of RLSRs are one of the main initiatives of REH-COVER (REHabilitation—COVID-19 Evidence-based Response) action which focuses on the timely collection, review and dissemination of evidence relating to rehabilitation during the COVID-19 pandemic (https://rehabilitation.cochrane.org/resources/cochrane-rehabilitation-versus-covid-19).

In these very dynamic COVID-19 times, the methodological improvement of our RLSR was necessary because of (1) the evolving quality of the evidence; (2) the new understandings of the pandemic evolution in rehabilitation context, and (3) the gradual involvement of experts with different clinical and methodological expertise that resulted from the work of the Cochrane Rehabilitation REH-COVER action. The most significant advancements of our methodology were the establishment of an international multiprofessional Steering Committee (https://rehabilitation.cochrane.org/resources/reh-cover-action/international-multiprofessional-steering-committee) that became engaged in the conduct of the RLSRs, and the development, in collaboration with the WHO rehabilitation program, of priority research questions on COVID-19 and rehabilitation (https://rehabilitation.cochrane.org/covid-19/priorities-research-defined-collaboration-who-rehabilitation-programme) [16]. The Steering Committee is composed of 13 participants including one infectious disease specialist, one epidemiologist and 11 rehabilitation professionals from nine countries and four continents, with one representative of Low and Middle Income Countries. The role of the Steering Committee was also to improve the dissemination of the REH-COVER action among patients and health professionals together with the Cochrane Rehabilitation Advisory Boards where patients’ organizations and all the main rehabilitation International Societies and journals are represented (https://rehabilitation.cochrane.org/about-and-contacts/advisory-board). Other gradual improvements included expanding the databases used to search the literature with the involvement of an information specialist, the accuracy of paper selection, the internal review process, the terminology used to describe the study designs, and rehabilitation setting definitions [4,15,16]. All these improvements were consolidated in the second edition of the living rapid SR [17], where methodological upgrades were introduced while adhering to Cochrane guidance for rapid review (https://covidreviews.cochrane.org/sites/covidreviews.cochrane.org/files/public/uploads/cochrane_rr_-_guidance-23mar2020-final.pdf). Specifically, we: (1) excluded expert opinions and all papers not reporting patients' data (secondary research, guidelines and consensus papers); (2) strengthened internal review processes by creating two teams (methodological and clinical) that independently checked all extracted data (with double-check by each group); (3) updated the research questions; (4) classified papers by study design. Due to heterogeneity of the papers included, we assessed the methodological quality of each study according to its design. In case of RCTs, we used the Cochrane Risk of Bias 2.0 (RoB 2) (31462531) tool; for nonrandomized studies of interventions (NRSIs), we used the Cochrane Risk Of Bias in Nonrandomized Studies—of Interventions (ROBINS-I) tool (27733354); for all the other designs, we evaluated the level of evidence using the OCEBM 2011 Levels of Evidence table (https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence) [21] The protocol for the first edition was registered in PROSPERO and updated with the second edition.

The new updates [18], [19], [20] followed the same methodology, which will be updated as needed (Fig. 1 ).

Fig. 1.

Improvements of the methodology in the rapid living systematic reviews between March 2020 edition and June 2020 edition.

Other groups have also recently proposed rapid SRs that are meant to be living [15], [16], [17]. However, they followed a different methodological evolution, because they were investigating treatments for the acute illness where strong evidence can be retrieved and synthesized more rapidly. Our initiative is not isolated; the COVID-END global network (https://www.mcmasterforum.org/networks/covid-end) is also discussing transitioning from rapid to living reviews, and shift their focus on COVID-19 management from “sprint to marathon” [22] (https://www.mcmasterforum.org/docs/default-source/covidend/presentations/covid-end_2020-09-15_cancovid.pdf?sfvrsn=16e956d5_2). To us, this is a strategic evolution, probably due to the COVID-19 pandemic, that needs better understanding.

Tricco et al. highlighted many methodological challenges when conducting a rapid review [23]. Using a living model to conduct rapid reviews allows facing some of them gradually. In our experience with rapid, living systematic reviews in rehabilitation, the pandemic has led to dynamic interactions between new partners who developed initiatives that could not be imagined when we launched our review. Keeping our methodology “living” also allows to innovate while keeping our original aims dynamic. Our rapid review ``lives'' online too, through an interactive living mapping (https://rehabilitation.cochrane.org/sites/rehabilitation.cochrane.org/files/public/uploads/covid/evidmap_table.html) and a living, dynamic table (https://rehabilitation.cochrane.org/sites/rehabilitation.cochrane.org/files/public/uploads/rapid_review/rapid_review_dynamic_table.html) available for all rehabilitation stakeholders. They are continually updated with the SR. We found a similar experience in a different field [24,25], where a living meta-analysis is published online, too [26].

3. Strengths and weaknesses

The strengths of our approach include the very high rapidity to provide an initial answer, and flexibility in updating results, methods, collaborations and even authorships. This approach has allowed us to minimize the inherent trade-offs between methodological rigor and need for rapid information, while delivering evidence syntheses that are the most trustworthy and up to date, reducing the time for new research to translate into health practice. Further, our methodology allows to build a body of evidence that grows for clinical observations to robust experimental designs.

The weaknesses of our approach include the initial reporting of papers based on weak methodology and descriptive study designs: nevertheless, they were the only available papers at the start, and stakeholders wanted to know them to start basing their work on some results. This high level of uncertainty could be remedied only by time. As soon as the primary literature became more robust, we updated to a second edition of the SR and retroactively applied our upgraded methodology to the previously included papers.

Another critical issue is the need for a firm agreement with a journal based on the understanding of the working group. The concept of a ``living'' review with a constant flow of new information is not part of the current publishing processes, that is usually more static: at the start, this created an issue in trying to link the new updates to the original paper, and it was necessary to contact the primary databases (PubMed, Web of Science) and starting a discussion with them to find the appropriate solutions.

Commitment from authors and journals is another specific concern. Being living and rapid requires significant monthly efforts in data collection, analysis and writing, especially when the volume of research grows continuously. Although we planned to complete the monthly updates in 2 weeks, it was seldom possible in less than 3 weeks. It is essential to understand that all these efforts subtract energies from other daily tasks of the researchers involved. The job is demanding for journals too, which are not used to rapid publications.

4. Conclusion

Informed by our experience, we propose the following definition of rapid living SR: ``A dynamic method of knowledge synthesis that allows for the constant updating of new emerging evidence and refinement of its methodological quality.” Combination the concepts of “rapid” and “living” allowed us to be very efficient in retrieving and synthesizing the evidence while gradually improving our methodology. We suggest that, with ongoing refinements, our methods will meet the methodological quality of a classical SR while keeping the pace with the updates.

Our experience is likely useful not only in pandemics but also in other situations where a sudden and rapid answer is urgently needed to make informed decisions. Nevertheless, we are now facing for the first time a pandemic with the strength of modern medical science. The scientific community should consider this development of the concepts of SRs useful in the future when facing new health emergencies.

Authors’ contributions

S.N.: conceptualization and draft preparation. P.C. and C.A.: methodology and writing. M.G.C.: writing and reviewing. All authors approved the final version of the manuscript.