Rapid literature review: definition and methodology

ABSTRACT

Introduction: A rapid literature review (RLR) is an alternative to systematic literature review (SLR) that can speed up the analysis of newly published data. The objective was to identify and summarize available information regarding different approaches to defining RLR and the methodology applied to the conduct of such reviews.

Methods: The Medline and EMBASE databases, as well as the grey literature, were searched using the set of keywords and their combination related to the targeted and rapid review, as well as design, approach, and methodology. Of the 3,898 records retrieved, 12 articles were included.

Results: Specific definition of RLRs has only been developed in 2021. In terms of methodology, the RLR should be completed within shorter timeframes using simplified procedures in comparison to SLRs, while maintaining a similar level of transparency and minimizing bias. Inherent components of the RLR process should be a clear research question, search protocol, simplified process of study selection, data extraction, and quality assurance.

Conclusions: There is a lack of consensus on the formal definition of the RLR and the best approaches to perform it. The evidence-based supporting methods are evolving, and more work is needed to define the most robust approaches.

Introduction

A systematic literature review (SLR) summarizes the results of all available studies on a specific topic and provides a high level of evidence. Authors of the SLR have to follow an advanced plan that covers defining a priori information regarding the research question, sources they are going to search, inclusion criteria applied to choose studies answering the research question, and information regarding how they are going to summarize findings [1].

The rigor and transparency of SLRs make them the most reliable form of literature review [2], providing a comprehensive, objective summary of the evidence for a given topic [3,4]. On the other hand, the SLR process is usually very time-consuming and requires a lot of human resources. Taking into account a high increase of newly published data and a growing need to analyze information in the fastest possible way, rapid literature reviews (RLRs) often replace standard SLRs.

There are several guidelines on the methodology of RLRs [5–11]; however, only recently, one publication from 2021 attempted to construct a unified definition [11]. Generally, by RLRs, researchers understand evidence synthesis during which some of the components of the systematic approach are being used to facilitate answering a focused research question; however, scope restrictions and a narrower search strategy help to make the project manageable in a shorter time and to get the key conclusions faster [4].

The objective of this research was to collect and summarize available information on different approaches to the definition and methodology of RLRs. An RLR has been run to capture publications providing data that fit the project objective.

Methods

To find publications reporting information on the methodology of RLRs, searches were run in the Medline and EMBASE databases in November 2022. The following keywords were searched for in titles and abstracts: ‘targeted adj2 review’ OR ‘focused adj2 review’ OR ‘rapid adj2 review’, and ‘methodology’ OR ‘design’ OR ‘scheme’ OR ‘approach’. The grey literature was identified using Google Scholar with keywords including ‘targeted review methodology’ OR ‘focused review methodology’ OR ‘rapid review methodology’. Only publications in English were included, and the date of publication was restricted to year 2016 onward in order to identify the most up-to-date literature. The reference lists of each included article were searched manually to obtain the potentially eligible articles. Titles and abstracts of the retrieved records were first screened to exclude articles that were evidently irrelevant. The full texts of potentially relevant papers were further reviewed to examine their eligibility.

A pre-defined Excel grid was developed to extract the following information related to the methodology of RLR from guidelines:

1. Definition,

2. Research question and searches,

3. Studies selection,

4. Data extraction and quality assessment,

5. Reporting,

6. Additional information.

There was no restriction on the study types to be analyzed; any study reporting on the methodology of RLRs could be included: reviews, practice guidelines, commentaries, and expert opinions on RLR relevant to healthcare policymakers or practitioners. The data extraction and evidence summary were conducted by one analyst and further examined by a senior analyst to ensure that relevant information was not omitted. Disagreements were resolved by discussion and consensus.

Results

Studies selection

A total of 3,898 records (3,864 articles from a database search and 34 grey literature from Google Scholar) were retrieved. After removing duplicates, titles and abstracts of 3,813 articles were uploaded and screened. The full texts of 43 articles were analyzed resulting in 12 articles selected for this review, including 7 guidelines [5–11] on the methodology of RLRs, together with 2 papers summarizing the results of the Delphi consensus on the topic [12,13], and 3 publications analyzing and assessing different approaches to RLRs [4,14,15].

Overall, seven guidelines were identified: from the World Health Organization (WHO) [5], National Collaborating Centre for Methods and Tools (NCCMT) [7], the UK government [8], the Oxford Centre for Evidence Based Medicine [9], the Cochrane group [6,11], and one multi-national review [10]. Among the papers that did not describe the guidelines, Gordon et al. [4] proposed 12 tips for conducting a rapid review in the right settings and discussed why these reviews may be more beneficial in some circumstances. The objective of work conducted by Tricco et al. [13] and Pandor et al. [12] was to collect and compare perceptions of rapid reviews from stakeholders, including researchers, policymakers, industry, journal editors, and healthcare providers, and to reach a consensus outlining the domains to consider when deciding on approaches for RLRs. Haby et al. [14] run a rapid review of systematic reviews and primary studies to find out the best way to conduct an RLR in health policy and practice. In Tricco et al. (2022) [15], JBI position statement for RLRs is presented.

Guidelines

From all the seven identified guidelines information regarding definitions the authors used for RLRs, approach to the PICOS criteria and search strategy development, studies selection, data extractions, quality assessment, and reporting were extracted.

Definition

Cochrane Rapid Reviews Methods Group developed methods guidance based on scoping review of the underlying evidence, primary methods studies conducted, as well as surveys sent to Cochrane representative and discussion among those with expertise [11]. They analyzed over 300 RLRs or RLR method papers and based on the methodology of those studies, constructed a broad definition RLR, one that meets a minimum set of requirements identified in the thematic analysis: ‘A rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting a variety of methods to produce evidence in a resource-efficient manner.’ This interpretation aligns with more than 50% of RLRs identified in this study. The authors additionally provided several other definitions, depending on specific situations or requirements (e.g., when RLR is produced on stakeholder’s request). It was additionally underlined that RLRs should be driven by the need of timely evidence for decision-making purposes [11].

Rapid reviews vary in their objective, format, and methods used for evidence synthesis. This is a quite new area, and still no agreement on optimal methods can be found [5]. All of the definitions are highlighting that RLRs are completed within shorter timeframes than SLRs, and also lack of time is one of the main reasons they are conducted. It has been suggested that most rapid reviews are conducted within 12 weeks; however, some of the resources suggest time between a few weeks to no more than 6 months [5,6]. Some of the definitions are highlighting that RLRs follow the SLR process, but certain phases of the process are simplified or omitted to retrieve information in a time-saving way [6,7]. Different mechanisms are used to enhance the timeliness of reviews. They can be used independently or concurrently: increasing the intensity of work by intensifying the efforts of multiple analysts by parallelization of tasks, using review shortcuts whereby one or more systematic review steps may be reduced, automatizing review steps by using new technologies [5]. The UK government report [8] referred to two different RLRs: in the form of quick scoping reviews (QSR) or rapid evidence assessments (REA). While being less resource and time-consuming compared to standard SLRs, QSRs and REAs are designed to be similarly transparent and to minimize bias. QSRs can be applied to rather open-ended questions, e.g., ‘what do we know about something’ but both, QSRs and REAs, provide an understanding of the volume and characteristics of evidence on a specific topic, allowing answering questions by maximizing the use of existing data, and providing a clear picture of the adequacy of existing evidence [8].

Research questions and searches

The guidelines suggest creating a clear research question and search protocol at the beginning of the project. Additionally, to not duplicate RLRs, the Cochrane Rapid Reviews Methods Group encourages all people working on RLRs to consider registering their search protocol with PROSPERO, the international prospective register of reviews; however, so far they are not formally registered in most cases [5,6]. They also recommend involving key stakeholders (review users) to set and refine the review question, criteria, and outcomes, as well as consulting them through the entire process [11].

Regarding research questions, it is better to structure them in a neutral way rather than focus on a specific direction for the outcome. By doing so, the researcher is in a better position to identify all the relevant evidence [7]. Authors can add a second, supportive research question when needed [8]. It is encouraged to limit the number of interventions, comparators and outcomes, to focus on the ones that are most important for decision-making [11]. Useful could be also reviewing additional materials, e.g., SLRs on the topic, as well as conducting a quick literature search to better understand the topic before starting with RLRs [7]. In SLRs researchers usually do not need to care a lot about time spent on creating PICOS, they need to make sure that the scope is broad enough, and they cannot use many restrictions. When working on RLRs, a reviewer may spend more or less time defining each of the components of the study question, and the main step is making sure that PICOS addresses the needs of those who requested the rapid review, and at the same time, it is feasible within the required time frame [7]. Search protocol should contain an outline of how the following review steps are to be carried out, including selected search keywords and a full strategy, a list of data sources, precise inclusion and exclusion criteria, a strategy for data extraction and critical appraisal, and a plan of how the information will be synthesized [8].

In terms of searches running, in most cases, an exhaustive process will not be feasible. Researchers should make sure that the search is effective and efficient to produce results in a timely manner. Cochrane Rapid Reviews Methods Group recommends involving an information specialist and conducting peer review of at least one search strategy [11]. According to the rapid review guidebook by McMaster University [7], it is important that RLRs, especially those that support policy and program decisions, are being fed by the results of a body of literature, rather than single studies, when possible. It would result in more generalizable findings applied at the level of a population and serve more realistic findings for program decisions [7]. It is important to document the search strategy, together with a record of the date and any date limits of the search, so that it can easily be run again, modified, or updated. Furthermore, the information on the individual databases included in platform services should always be reported, as this depends on organizations’ subscriptions and must be included for transparency and repeatability [7,8]. Good solution for RLRs is narrowing the scope or searching a limited number of databases and other sources [7]. Often, the authors use the PubMed/MEDLINE, Cochrane Library, and Embase databases. In most reviews, two or more databases are searched, and common limits are language (usually restricted to English), date, study design, and geographical area. Some RLRs include searching of grey literature; however, contact with authors is rather uncommon [5,8]. According to the flexible framework for restricted systematic review published by the University of Oxford, the search should be run in at least one major scientific database such as PubMed, and one other source, e.g., Google Scholar [9]. Grey literature and unpublished evidence may be particularly needed and important for intervention questions. It is related to the fact that studies that do not report the effects of interventions are less likely to be published [8]. If there is any type of evidence that will not be considered by the RLRs, e.g., reviews or theoretical and conceptual studies, it should also be stated in the protocol together with justification [8]. Additionally, authors of a practical guide published by WHO suggest using a staged search to identify existing SLRs at the beginning, and then focusing on studies with other designs [5]. If a low number of citations have been retrieved, it is acceptable to expand searches, remove some of the limits, and add additional databases and sources [7].

Searching for RLRs is an iterative process, and revising the approach is usually needed [7]. Changes should be confirmed with stakeholders and should be tracked and reflected in the final report [5].

Studies selection

The next step in the rapid review is the selection of studies consisting of two phases: screening of titles and abstracts, and analysis of full texts. Prior to screening initiation, it is recommended to conduct a pilot exercise using the same 30–50 abstracts and 5–10 full-texts for the entire screening team in order to calibrate and test the review form [11]. In contrast to SLRs, it can be done by one reviewer with or without verification by a second one. If verification is performed, usually the second reviewer checks only a subset of records and compares them. Cochrane Group, in contrast, recommends a stricter approach: at least 20% of references should be double-screened at titles and abstracts stage, and while the rest of the references may be screened by one reviewer, the excluded items need to be re-examined by second reviewer; similar approach is used in full-text screening [11]. This helps to ensure that bias was reduced and that the PICOS criteria are applied in a relevant way [5,8,9,11]. During the analysis of titles and abstracts, there is no need to report reasons for exclusion; however, they should be tracked for all excluded full texts [7].

Data extraction and quality assessment

According to the WHO guide, the most common method for data extraction in RLRs is extraction done by a single reviewer with or without partial verification. The authors point out that a reasonable approach is to use a second reviewer to check a random sample of at least 10% of the extractions for accuracy. Dual performance is more necessary for the extraction of quantitative results than for descriptive study information. In contrast, Cochrane group recommends that second reviewer should check the correctness and completeness of all data [11]. When possible, extractions should be limited to key characteristics and outcomes of the study. The same approach to data extraction is also suggested for a quality assessment process within rapid reviews [5,9,11]. Authors of the guidebook from McMaster University highlight that data extraction should be done ideally by two reviewers independently and consensus on the discrepancies should always be reached [7]. The final decision on the approach to this important step of review should depend on the available time and should also reflect the complexity of the research question [9].

For screening, analysis of full texts, extractions, and quality assessments, researchers can use information technologies to support them by making these review steps more efficient [5].

Reporting

Before data reporting, a reviewer should prepare a document with key message headings, executive summary, background related to the topic and status of the current knowledge, project question, synthesis of findings, conclusions, and recommendations. According to the McMaster University guidebook, a report should be structured in a 1:2:20 format, that is, one page for key messages, two pages for an executive summary, and a full report of up to 20 pages [7]. All the limitations of the RLRs should be analyzed, and conclusions should be drawn with caution [5]. The quality of the accumulated evidence and the strength of recommendations can be assessed using, e.g., the GRADE system [5]. When working on references quoting, researchers should remember to use a primary source, not secondary references [7]. It would be worth considering the support of some software tools to automate reporting steps. Additionally, any standardization of the process and the usage of templates can support report development and enhance the transparency of the review [5].

Ideally, all the review steps should be completed during RLRs; however, often some steps may need skipping or will not be completed as thoroughly as should because of time constraints. It is always crucial to decide which steps may be skipped, and which are the key ones, depending on the project [7]. Guidelines suggest that it may be helpful to invite researchers with experience in the operations of SLRs to participate in the rapid review development [5,9]. As some of the steps will be completed by one reviewer only, it is important to provide them with relevant training at the beginning of the process, as well as during the review, to minimize the risk of mistakes [5].

Additional information

Depending on the policy goal and available resources and deadlines, methodology of the RLRs may be modified. Wilson et al. [10] provided extensive guidelines for performing RLR within days (e.g., to inform urgent internal policy discussions and/or management decisions), weeks (e.g., to inform public debates), or months (e.g., to inform policy development cycles that have a longer timeline, but that cannot wait for a traditional full systematic review). These approaches vary in terms of data synthesis, types of considered evidence and project management considerations.

In shortest timeframes, focused questions and subquestions should be formulated, typically to conduct a policy analysis; the report should consist of tables along with a brief narrative summary. Evidence from SLRs is often considered, as well as key informant interviews may be conducted to identify additional literature and insights about the topic, while primary studies and other types of evidence are not typically feasible due to time restrictions. The review would be best conducted with 1–2 reviewers sharing the work, enabling rapid iterations of the review. As for RLRs with longer timeline (weeks), these may use a mix of policy, systems and political analysis. Structure of the review would be similar to shorter RLRs – tabular with short narrative summary, as the timeline does not allow for comprehensive synthesis of data. Besides SLRs, primary studies and other evidence may be feasible in this timeframe, if obtained using the targeted searches in the most relevant databases. The review team should be larger, and standardized procedures for reviewing of the results and data extraction should be applied. In contrast to previous timeframe, merit review process may be feasible. For both timeframes, brief consultations with small transdisciplinary team should be conducted at the beginning and in the final stage of the review to discuss important matters.

For RLRs spanning several months, more comprehensive methodology may be adapted in terms of data synthesis and types of evidence. However, authors advise that review may be best conducted with a small review team in order to allow for more in-depth interpretation and iteration.

Studies analyzing methodology

There have been two interesting publications summarizing the results of Delphi consensus on the RLR methodology identified and included in this review [12,13].

Tricco et al. [13] first conducted an international survey and scoping review to collect information on the possible approaches to the running of rapid reviews, based on which, they employed a modified Delphi method that included inputs from 113 stakeholders to explore the most optimized approach. Among the six most frequent rapid review approaches (not all detailed here) being evaluated, the approach that combines inclusion of published literature only, a search of more than one database and limitations by date and language, study selection by one analyst, data extraction, and quality assessment by one analyst and one verifier, was perceived as the most feasible approach (72%, 81/113 responses) with the potentially lowest risk of bias (12%, 12/103). The approach ranked as the first one when considering timelines assumes updating of the search from a previously published review, no additional limits on search, studies selection and data extraction done by one reviewer, and no quality assessment. Finally, based on the publication, the most comprehensive RLRs can be made by moving on with the following rules: searching more than one database and grey literature and using date restriction, and assigning one reviewer working on screening, data extraction, and risk of bias assessment (Table 1). Pandor et al. [12] introduced a decision tool for SelecTing Approaches for Rapid Reviews (STARR) that were produced through the Delphi consensus of international experts through an iterative and rigorous process. Participants were asked to assess the importance of predefined items in four domains related to the rapid review process: interaction with commissioners, understanding the evidence base, data extraction and synthesis methods, and reporting of rapid review methods. All items assigned to four domains achieved > 70% of consensus, and in that way, the first consensus-driven tool has been created that supports authors of RLRs in planning and deciding on approaches.

Table 1.

Six most frequent approaches to RLRs (adapted from Tricco et al. [13]).

Domain	Approach 1	Approach 2	Approach 3	Approach 4	Approach 5	Approach 6
Literature search	More than one database, only published data	Update of previous literature search	More than one database, grey literature included
Search limit	Date and language	None	Date and language	Date or language	Date	Date and language
Screening	One reviewer					Two reviewers
Data extraction	One reviewer and one verifier	One reviewer
Risk of bias appraisal	One reviewer and one verifier	Not performed			One reviewer	Not performed
Overall assessment	Highly feasible and timely, having the lowest potential risk of bias	Highly feasible and timely	Moderate	Moderate	Highly comprehensive, but low-ranking in terms of feasibility and timeliness	Highly comprehensive, but low-ranking in terms of feasibility and timeliness

Haby et al. [14] run searches of 11 databases and two websites and developed a comprehensive overview of the methodology of RLRs. With five SLRs and one RCT being finally included, they identified the following approaches used in RLRs to make them faster than full SLRs: limiting the number and scope of questions, searching fewer databases, limited searching of grey literature, restrictions on language and date (e.g., English only, most recent publications), updating the existing SLRs, eliminating or limiting hand searches of reference lists, noniterative search strategies, eliminating consultation with experts, limiting dual study selection, data extraction and quality assessment, minimal data synthesis with short concise conclusions or recommendations. All the SLRs included in this review were consistent in stating that no agreed definition of rapid reviews is available, and there is still no final agreement on the best methodological rules to be followed.

Gordon et al. [4] explained the advantages of performing a focused review and provided 12 tips for its conduction. They define focused reviews as ‘a form of knowledge synthesis in which the components of the systematic process are applied to facilitate the analysis of a focused research question’. The first tip presented by the authors is related to deciding if a focused review is a right solution for the considered project. RLRs will suit emerging topics, approaches, or assessments where early synthesis can support doctors, policymakers, etc., but also can direct future research. The second, third, and fourth tips highlight the importance of running preliminary searches and considering narrowing the results by using reasonable constraints taking into account the local context, problems, efficiency perspectives, and available time. Further tips include creating a team of experienced reviewers working on the RLRs, thinking about the target journal from the beginning of work on the rapid review, registering the search protocol on the PROSPERO registry, and the need for contacting authors of papers when data available in publications are missing or incongruent. The last three tips are related to the choice of evidence synthesis method, using the visual presentation of data, and considering and describing all the limitations of the focused review.

Finally, a new publication by Tricco et al. from 2022, describing JBI position statement [15] underlined that for the time being, there is no specific tool for critical appraisal of the RLR’s methodological quality. Instead, reviewers may use available tools to assess the risk of bias or quality of SLRs, like ROBIS, the JBI critical appraisal tools, or the assessment of multiple systematic reviews (AMSTAR).

Discussion

Inconsistency in the definitions and methodologies of RLR

Although RLR was broadly perceived as an approach to quicken the conduct of conventional SLR, there is a lack of consensus on the formal definition of the RLR, so as to the best approaches to perform it. Only in 2021, a study proposing unified definition was published; however, it is important to note that the most accurate definition was only matching slightly over 50% of papers analysed by the authors, which underlines the lack of homogeneity in the field [11]. The evidence-based supporting methods are evolving, and more evidence is needed to define the most robust approaches [5].

Diverse terms are used to describe the RLR, including ‘rapid review’, focused systematic review’, ‘quick scoping reviews’, and ‘rapid evidence assessments’. Although the general principles of conducting RLR are to accelerate the whole process, complexity was seen in the methodologies used for RLRs, as reflected in this study. Also, inconsistencies related to the scope of the questions, search strategies, inclusion criteria, study screening, full-text review, quality assessment, and evidence presentation were implied. All these factors may hamper decision-making about optimal methodologies for conducting rapid reviews, and as a result, the efficiency of RLR might be decreased. Additionally, researchers may tend to report the methodology of their reviews without a sufficient level of detail, making it difficult to appraise the quality and robustness of their work.

Advantages and weaknesses of RLR

Although RLR used simplified approaches for evidence synthesis compared with SLR, the methodologies for RLR should be replicable, rigorous, and transparent to the greatest extent [16]. When time and resources are limited, RLR could be a practical and efficient tool to provide the summary of evidence that is critical for making rapid clinical or policy-related decisions [5]. Focusing on specific questions that are of controversy or special interest could be powerful in reaffirming whether the existing recommendation statements are still appropriate [17].

The weakness of RLR should also be borne in mind, and the trade-off of using RLR should be carefully considered regarding the thoroughness of the search, breadth of a research question, and depth of analysis [18]. If allowed, SLR is preferred over RLR considering that some relevant studies might be omitted with narrowed search strategies and simplified screening process [14]. Additionally, omitting the quality assessment of included studies could result in an increased risk of bias, making the comprehensiveness of RLR compromised [13]. Furthermore, in situations that require high accuracy, for example, where a small relative difference in an intervention has great impacts, for the purpose of drafting clinical guidelines, or making licensing decisions, a comprehensive SLR may remain the priority [19]. Therefore, clear communications with policymakers are recommended to reach an agreement on whether an RLR is justified and whether the methodologies of RLR are acceptable to address the unanswered questions [18].

Disclosure statement

No potential conflict of interest was reported by the author(s).