Rapid reviews methods series: Guidance on literature search

Abstract

This paper is part of a series of methodological guidance from the Cochrane Rapid Reviews Methods Group. Rapid reviews (RR) use modified systematic review methods to accelerate the review process while maintaining systematic, transparent and reproducible methods. In this paper, we address considerations for RR searches. We cover the main areas relevant to the search process: preparation and planning, information sources and search methods, search strategy development, quality assurance, reporting, and record management. Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches and (2) reducing the size of the search result. Because screening search results is usually more resource-intensive than conducting the search, we suggest investing time upfront in planning and optimising the search to save time by reducing the literature screening workload. To achieve this goal, RR teams should work with an information specialist. They should select a small number of relevant information sources (eg, databases) and use search methods that are highly likely to identify relevant literature for their topic. Database search strategies should aim to optimise both precision and sensitivity, and quality assurance measures (peer review and validation of search strategies) should be applied to minimise errors.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• Compared with systematic reviews, rapid reviews (RR) often abbreviate or limit the literature search in some way to accelerate review production. However, RR guidance rarely specifies how to select topic-appropriate search approaches.

WHAT THIS STUDY ADDS

• This paper presents an overview of considerations and recommendations for RR searching, covering the complete search process from the planning stage to record management. We also provide extensive appendices with practical examples, useful sources and a glossary of terms.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• There is no one-size-fits-all solution for RR literature searching: review teams should consider what search approaches best fit their RR project.

Introduction

This paper is part of a series from the Cochrane Rapid Reviews Methods Group (RRMG) providing methodological guidance for rapid reviews (RRs).^1–3 While the RRMG’s guidance^{4 5} on Cochrane RR production includes brief advice on literature searching, we aim to provide in-depth recommendations for the entire search process.

Literature searching is the foundation for all reviews; therefore, it is important to understand the goals of a specific RR. The scope of RRs varies considerably (from focused questions to overviews of broad topics).⁶ As with conventional systematic reviews (SRs), there is not a one-size-fits-all approach for RR literature searches. We aim to support RR teams in choosing methods that best fit their project while understanding the limitations of modified search methods. Our recommendations derive from current systematic search guidance, evidence on modified search methods and practical experience conducting RRs.

This paper presents considerations and recommendations, described briefly in table 1. The table also includes a comparison to the SR search process based on common recommendations.^7–10 We provide supplemental materials, including a list of additional resources, further details of our recommendations, practical examples, and a glossary (explaining the terms written in italics) in online supplemental appendices A–C.

Table 1.

Recommendations for rapid review literature searching

Recommendation for rapid review (RR) searching		Potential differences with systematic review (SR) searching	Additional information (appendix)
Preparation and planning	Involve an information specialist (eg, librarian), ideally from the start of the project. At a minimum, an information specialist should assess the information sources, search methods, and the primary database search strategy.	None. However, information specialist involvement can speed up the further steps of the search process.	Online supplemental appendix C 1.1
	Consider using PRISMA-S11 and prepared templates for planning and conducting the search to ensure the search process is thoroughly planned.	PRISMA-S and general templates might need to be adapted to the chosen RR approach.	Online supplemental appendix B
	Conduct preliminary or scoping searches to identify a first set of potentially relevant literature, which will aid in topic refinement, selection of information sources and selection of search terms.	None. This is a crucial step for any systematic search.	Online supplemental appendix C 1.2–3
Information sources and search methods	Select a small number (at least 2) of information sources that are likely to retrieve relevant literature.	SRs generally use a larger number of information sources to ensure sensitivity.	Online supplemental appendix C 2.1–4
	For RRs based on RCTs, use, at a minimum, a combination of two of these databases: MEDLINE, CENTRAL, Embase. In some cases, combining one of these databases (in particular MEDLINE) with an appropriate supplementary search method (eg, similar articles, study register searching) may suffice.	Both bibliographic databases and grey trial registers have to be searched. E.g.,MECIR9 requires searching MEDLINE, CENTRAL, Embase, linicalTrials.gov, and the WHO International Clinical Trials Registry Platform (ICTRP) to identify RCTs.	Online supplemental appendix C 2.2
	Use the findings of preliminary searches to assess if grey literature may be relevant for a topic and what types (eg, clinical trial registrations, preprints, theses).	SRs generally include grey literature searches independent of the topic.	Online supplemental appendix B and C 2.1
Search strategies	Review the abstracts and subject headings of known relevant records for appropriate search terms.	SR search strategies generally aim to maximise sensitivity. RR searches may aim to increase precision to reduce the search result.	Online supplemental appendix B,C 3.1–2
	Identify SRs on the same or a similar topic and review the search strategies for elements that could be reused (eg, population, intervention).	Most SRs searches are developed de novo for a particular review.	Online supplemental appendix C 3.1–2
	Use limits and restrictions appropriately and with caution.	SRs should not restrict searches to languages, publication dates, etc.	Online supplemental appendix C 3.4–5
	When updating an existing review, assess the original search methods and adapt as necessary.	None. Consider utilising guidance for updating SRs.39
Quality assurance and search strategy peer review	Validate the primary search strategy by testing if known relevant records are retrieved.	None. However, SR searches generally aim to find all known relevant records, while in precision-focused RR searches, a reduced sensitivity might be acceptable.	Online supplemental appendix C 4.1
	Use the PRESS checklist42 to peer review the primary search strategy. If full peer review is not possible, check the primary search strategy for errors of spelling, operator usage, and line number combinations.	Full PRESS peer review is recommended for all SRs.	Online supplemental appendix C 4.2
	Review the appropriateness of planned information sources and search methods.	None. However, this is particularly important if few information sources/search methods are used.	Online supplemental appendix C 4.2
Reporting and record management	Decide on systems and processes for managing records early in the review planning stage.	None. However, appropriate planning can save time throughout the process.	Online supplemental appendix C 5.1–3
	Consider using PRISMA-S11 as reporting standard for RR searches.	PRISMA-S has been developed for SR searches. It might need to be adapted to the chosen RR approach .	Online supplemental appendix B and C 5.1
	Use reference management software (eg, EndNote, Zotero) and/or SR platforms (eg, Covidence, Systematic Review Data Repository Plus) to track search results throughout the review process.	None. However, the appropriate use of these tools can save time throughout the process.	Online supplemental appendix C 5.2–3

MECIR, Methodological Expectations of Cochrane Intervention Reviews; PRESS, Peer Review of Electronic Search Strategies; PRISMA-S, Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension; RCT, randomised controlled trial; RR, rapid review.

Preparation and planning

Given that the results of systematic literature searches underpin a review, planning the searches is integral to the overall RR preparation. The RR search process follows the same steps as an SR search; therefore, RR teams must be familiar with the general standards of systematic searching. Templates (see online supplemental appendix B) and reporting guidance¹¹ for SR searches can also be adapted to structure the RR search process.

Developing a plan for the literature search forms part of protocol development and should involve an information specialist (eg, librarian). Information specialists can assist in refining the research question, selecting appropriate search methods and resources, designing and executing search strategies, and reporting the search methods. At minimum, specialist input should include assessing information sources and methods and providing feedback on the primary database search strategy.

Two options exist for abbreviating the search process: (1) reducing time spent on conducting searches (eg, using automation tools, reusing existing search strategies, omitting planning or quality assurance steps) and (2) reducing the size of the search result (eg, limiting the number of information sources, increasing the precision of search strategies, using study design filters). Study selection (ie, screening search results) is usually more resource-intensive than searching,¹² particularly for topics with complex or broad concepts or diffuse terminology; thus, the second option may be more efficient for the entire RR. Investing time upfront in optimising search sensitivity (ie, completeness) and precision (ie, positive predictive value) can save time in the long run by reducing the screening and selection workload.

Preliminary or scoping searches are critical to this process. They inform the choice of search methods and identify potentially relevant literature. Texts identified through preliminary searching serve as known relevant records that can be used throughout the search development process (see sections on database selection, development and validation of search strategies).

In addition to planning the search itself, the review team should factor in time for quality assurance steps (eg, search strategy peer review) and the management of search results (eg, deduplication, full-text retrieval).

Information sources and methods

To optimise the balance of search sensitivity and precision, RR teams should prioritise the most relevant information sources for the topic and the type of evidence required. These can include bibliographic databases (eg, MEDLINE/PubMed), grey literature sources and targeted supplementary search methods. Note that this approach differs from the Methodological Expectations of Cochrane Intervention Reviews Standards⁹ where the same core set of information sources is required for every review and further supplemented by additional topic-specific and evidence-specific sources.

Choosing bibliographic databases

For many review topics, most evidence is found in peer-reviewed journal articles, making bibliographic databases the main resource of systematic searching. Limiting the number of databases searched can be a viable option in RRs, but it is important to prioritise topic-appropriate databases.

MEDLINE has been found to have high coverage for studies included in SRs^{13 14} and is an appealing database choice because access is free via PubMed. However, coverage varies depending on topics and relevant study designs.^{15 16} Additionally, even if all eligible studies for a topic were available in MEDLINE, search strategies will usually miss some eligible studies because search sensitivity is lower than database coverage.^{13 17} This means searching MEDLINE alone is not a viable option, and additional information sources or search methods are required. Known relevant records can be used to help assess the coverage of selected databases (see also online supplemental appendix C).

Further information sources and search techniques

Supplementary systematic search methods have three main goals, to identify (1) grey literature, (2) published literature not covered by the selected bibliographic databases and (3) database-covered literature that was not retrieved by the database searches.

When RRs search only a small number of databases, supplementary searches can be particularly important to pick up eligible studies not identified via database searching. While supplementary methods might increase the time spent on searching, they sometimes better optimise search sensitivity and precision, saving time in the long run.¹⁸ Depending on the topic and relevant evidence, such methods can offer an alternative to adding additional specialised database searches. To decide if and what supplementary searches are helpful, it is important to evaluate what literature might be missed by the database searches and how this might affect the specific RR.

Study registries and other grey literature

Some studies indicate that the omission of grey literature searches rarely affects review conclusions.^{17 19} However, the relevance of study registries and other grey literature sources is topic-dependent.^{16 19–21} For example, randomised controlled trials (RCTs) on newly approved drugs are typically identified in ClinicalTrials.gov.²⁰ For rapidly evolving topics such as COVID-19, preprints are an important source.²¹ For public health interventions, various types of grey literature may be important (eg, evaluations conducted by local public health agencies).²²

Further supplementary search methods

Other supplementary techniques (eg, checking reference lists, reviewing specific websites or electronic table of contents, contacting experts) may identify additional studies not retrieved by database searches.²³ One of the most common approaches involves checking reference lists of included studies and relevant reviews. This method may identify studies missed by limited database searches.¹² Another promising citation-based approach is using the ‘similar articles’ option in PubMed, although research has focused on updating existing SRs.^{24 25}

Considerations for RRs of RCTs

Databases and search methods to identify RCTs have been particularly well researched.17 20 24 26 27 For this reason, it is possible to give more precise recommendations for RRs based on RCTs than for other types of review. Table 2 provides an overview of the most important considerations; additional information can be found in online supplemental appendix C.

Table 2.

Information sources for identification of randomised controlled trials (RCTs)

Bibliographic databases

A combination of two MECIR-required databases (ie, MEDLINE, CENTRAL, Embase) is likely to be sufficient for retrieving published RCTs.17

Further information sources and search techniques

Alternatively, combining a database (eg, MEDLINE) with a supplementary search method may also suffice: Combining a simple MEDLINE search strategy with a PubMed similar articles search is viable for updating clinically focused SRs of RCTs,24 and holds potential for de novo searches for these designs.27 For RCTs on newly approved drugs, a combination of ClinicalTrials.gov20 and one MECIR-required database is promising. Combining CENTRAL or Embase with reference list checking may also suffice.17

MECIR, Methodological Expectations of Cochrane Intervention Reviews; RCT, randomised controlled trial; SRs, systematic reviews.

Search strategies

We define ‘search strategy’ as a Boolean search query in a specific database (eg, MEDLINE) using a specific interface (eg, Ovid). When several databases are searched, this query is usually developed in a primary database and interface (eg, Ovid MEDLINE) and translated to other databases.

Developing search strategies

Optimising search strategy precision while aiming for high sensitivity is critical in reducing the number of records retrieved. Preliminary searches provide crucial information to aid efficient search strategy development. Reviewing the abstracts and subject headings used in known relevant records will assist in identifying appropriate search terms. Text analysis tools can also be used to support this process,^{28 29} for example, to develop ‘objectively derived’ search strategies.³⁰

Reusing or adapting complete search strategies (eg, from SRs identified by the preliminary searches) or selecting elements of search strategies for reuse can accelerate search strategy development. Additionally, validated search filters (eg, for study design) can be used to reduce the size of the search result without compromising the sensitivity of a search strategy.³¹ However, quality assurance measures are necessary whether the search strategy is purpose-built, reused or adapted (see the ‘Quality assurance’ section.)

Database-specific and interface-specific functionalities can also be used to improve searches’ precision and reduce the search result’s size. Some options are: restricting to records where subject terms have been assigned as the major focus of an article (eg, major descriptors in MeSH), using proximity operators (ie, terms adjacent or within a set number of words), frequency operators (ie, terms have to appear a minimum number of times in an abstract) or restricting search terms to the article title.^32–34

Automated syntax translation can save time and reduce errors when translating a primary search strategy to different databases.^{35 36} However, manual adjustments will usually be necessary.

The time taken to learn how to use supporting technologies (eg, text analysis, syntax translation) proficiently should not be underestimated. The time investment is most likely to pay off for frequent searchers. A later paper in this series will address supporting software for the entire review process.

Limits and restrictions

Limits and restrictions (eg, publication dates, language) are another way to reduce the number of records retrieved but should be tailored to the topic and applied with caution. For example, if most studies about an intervention were published 10 years ago, then an arbitrary cut-off of ‘the last 5 years’ will miss many relevant studies.³⁷ Similarly, limiting to ‘English only’ is acceptable for most cases, but early in the COVID-19 pandemic, a quarter of available research articles were written in Chinese.³⁸ Depending on the RR topic, certain document types (eg, conference abstracts, dissertations) might be excluded if not considered relevant to the research question.

Note also that preset limiting functions in search interfaces (eg, limit to humans) often rely on subject headings (eg, MeSH) alone. They will miss eligible studies that lack or have incomplete subject indexing. Using (validated) search filters ³¹ is preferable.

Updating existing reviews

One approach to RR production involves updating an existing SR. In this case, preliminary searches should be used to check if new evidence is available. If the review team decide to update the review, they should assess the original search methods and adapt these as necessary.

One option is to identify the minimum set of databases required to retrieve all the original included studies.³⁹ Any reused search strategies should be validated and peer-reviewed (see below) and optimised for precision and/or sensitivity.

Additionally, it is important to assess whether the topic terminology or the relevant databases have changed since the original SR search.

In some cases, designing a new search process may be more efficient than reproducing the original search.

Quality assurance and search strategy peer review

Errors in search strategies are common and can impact the sensitivity and comprehensiveness of the search result.⁴⁰ If an RR search uses a small number of information sources, such errors could affect the identification of relevant studies.

Validation of search strategies

The primary database search strategy should be validated using known relevant records (if available). This means testing if the primary search strategy retrieves eligible studies found through preliminary searching. If some known studies are not identified, the searcher assesses the reasons and decides if revisions are necessary. Even a precision-focused systematic search should identify the majority—we suggest at least 80%–90%—of known studies. This is based on benchmarks for sensitivity-precision-maximising search filters⁴¹ and assumes that the set of known studies is representative of the whole of relevant studies.

Peer review of search strategies

Ideally, an information specialist should review the planned information sources and search methods and use the PRESS (Peer Review of Electronic Search Strategies) checklist⁴² to assess the primary search strategy. Turnaround time has to be factored into the process from the outset (eg, waiting for feedback, revising the search strategy). PRESS recommends a maximum turnaround time of five working days for feedback, but in-house peer review often takes only a few hours.

If the overall RR time plan does not allow for a full peer review of the search strategy, a review team member with search experience should check the search strategy for spelling errors and correct use of Boolean operators (AND, OR, NOT) at a minimum.

Reporting and record management

Record management requirements of RRs are largely identical to SRs and have to be factored into the time plan. Teams should develop a data management plan and review the relevant reporting standards at the project’s outset. PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses literature search extension)¹¹ is a reporting standard for SR searches that can be adapted for RRs.

Reference management software (eg, EndNote,⁴³ Zotero⁴⁴) should be used to track search results, including deduplication. Note that record management for database searches is less time-consuming than for many supplementary or grey literature searches, which often require manual entry into reference management software.¹²

Additionally, software platforms for SR production (eg, Covidence,⁴⁵ EPPI-Reviewer,⁴⁶ Systematic Review Data Repository Plus⁴⁷) can provide a unified way to keep track of records throughout the whole review process, which can improve management and save time. These platforms and other dedicated tools (eg, SRA Deduplicator)⁴⁸ also offer automated deduplication. However, the time and cost investment necessary to appropriately use these tools have to be considered.

Conclusion

Decisions about search methods for an RR need to consider where time can be most usefully invested and processes accelerated. The literature search should be considered in the context of the entire review process, for example, protocol development and literature screening: Findings of preliminary searches often affect the development and refinement of the research question and the review’s eligibility criteria. In turn, they affect the number of records retrieved by the searches and therefore the time needed for literature selection.

For this reason, focusing only on reducing time spent on designing and conducting searches can be a false economy when seeking to speed up review production. While some approaches (eg, text analysis, automated syntax translation) may save time without negatively affecting search validity, others (eg, skipping quality assurance steps, using convenient information sources without considering their topic appropriateness) may harm the entire review. Information specialists can provide crucial aid concerning the appropriate design of search strategies, choice of methods and information sources.

For this reason, we consider that investing time at the outset of the review to carefully choose a small number of highly appropriate search methods and optimise search sensitivity and precision likely leads to better and more manageable results.

Data availability statement

No data are available.

Ethics statements

Patient consent for publication

Not applicable.