Scoping reviews: establishing the role of the librarian

Health librarians have made a significant and enduring contribution to knowledge syntheses through our work on systematic reviews as expert searchers [1], methodologists [2], and information managers [3]. The scoping review is now emerging as an alternative and increasingly popular method to synthesize the literature on a given topic. Many librarians have noticed this trend and (1) may be considering getting involved or (2) may wish to learn more but may not know where to start with the growing body of literature on this type of review.

A scoping review is different than a systematic review. Librarians who participate in systematic reviews have access to a clear definition of the review, a detailed description of the various steps involved in undertaking one, peer support, and a discussion of the role of the librarian in ensuring the success of a review project. This is possible due to work conducted over four decades to formalize systematic review methodology by such organizations as the Cochrane Collaboration [4], the guidance of the widely endorsed PRISMA reporting standard for these reviews [5], and the existence of the PRESS standard for reporting searches [6, 7].

Unfortunately, the same does not apply to scoping reviews. Interested librarians quickly learn that, while there is broad agreement over the nature of scoping reviews, there are several proposed definitions of what they are, none of which has been universally accepted [8–10]. Further, there is no general agreement on what these reviews should be called or if they can be considered a form of knowledge synthesis. Similarly, while there is now a generally agreed on series of steps for conducting scoping reviews [11], debate continues over the best way to complete these steps [12].

The scoping review is thus in a state of flux and a subject of interest for a number of scholars who wish to formalize its definition and methodology. The scholarly conversation has, so far, included little input from librarians. It is now time for health librarians to join this discussion and demonstrate the unique value we can bring to the scoping review process. In this article, the authors provide an overview of the scoping review and a summary of the current status of its definition and methodology. Due to the developing nature of scoping reviews, we also suggest how librarians can and should involve themselves in this emerging area of the knowledge synthesis landscape.

BACKGROUND

What is a scoping review?

Scoping reviews are one of many forms of knowledge synthesis [13]. They appear to have been initially named in 2001 [14] and are variously referred to as “scoping studies” and “mapping studies,” this last term being potentially unclear as it could be confused with purely quantitative bibliographic studies of the same name [15]. For this reason, it has been suggested that researchers agree to call these studies using a standard term, either “scoping reviews” [12, 13, 16–19] or “scoping studies” [8–10, 12, 20–22]. Here, we chose “scoping review,” as we argue it is a clearer and more descriptive label.

While there is no single definition of the scoping review [8–10], authors generally agree that it aims to address broader, more complex, and exploratory research questions [20]. It thus contrasts with the systematic review, which is designed to answer precisely defined, narrow questions.

The first proposed definition, rationale, and methodological framework for scoping reviews were published by Arksey and O'Malley in 2005 [20] and have since been developed and clarified [12, 16, 23, 24]. Arksey and O'Malley identify four reasons for conducting a scoping review:

• 1to examine the extent, range, and nature of research activity;
• 2to determine the value of undertaking a systematic review;
• 3to summarize and disseminate research findings; and
• 4to identify research gaps in the existing literature.

Individual definitions by subsequent authors state that scoping reviews aim “to map the key concepts underpinning a research area and the main sources and types of evidence available” [20], to provide a “preliminary assessment of potential size and scope of research literature” [17], and to contextualize knowledge “in terms of identifying the current state of understanding; identifying the sorts of things we know and do not know, and then setting this within policy and practice contexts” [8].

Are scoping review searches systematic?

Inconsistencies between definitions start to emerge once a more detailed comparison is undertaken, and these differences raise a number of questions for librarians who wish to better understand these reviews. On the subject of systematicity of methodology, for example, Rumrill et al. state that “scoping reviews tend to be non-systematic in nature and tend to focus on breadth of coverage of the literature conducted on a topic rather than depth of coverage” [emphasis added] [19]. This point is supported by Anderson et al., who repeatedly describe scoping reviews as non-systematic [8]. In contradiction, the Canadian Institutes for Health Research (CIHR) advise that scoping reviews “entail the systematic selection, collection and summarization of existing knowledge in a broad thematic area” [13], and Colquhoun et al. state that scoping reviews are conducted by “systematically searching, selecting, and synthesizing existing knowledge” [16]. One survey of published scoping reviews notes that, while in many published reviews the methodology is described as “‘systematic', ‘rigorous', ‘replicable', and ‘transparent,'” others are less rigorous than a systematic review [18]. To resolve this lack of clarity, some authors have started referring to “systematic scoping reviews” [24].

What are the scoping review stages?

Arksey and O'Malley's proposed methodological framework consists of six stages (Table 1) [20]. They invited colleagues to develop this framework, and several have since taken up this invitation, most notably Levac et al., who draw on their experiences in conducting scoping reviews to propose various enhancements and clarifications [12]. These include improved clarity in the development of the initial research question, the selection of studies, data analysis, the importance of assessing the quality of included studies, and the usefulness of conducting consultations with stakeholders.

Table 1.

Methodologic framework for scoping reviews as described by Arksey and O'Malley, 2005

What is the role of librarians in the scoping review methodology?

In the basic literature in scoping reviews, we found none coauthored by a librarian [9, 10, 12, 16, 20, 22, 23]. This suggests that librarians have been little involved in areas where they are traditionally strong and have much to offer [25]. Librarians looking for further guidance on the specialized contributions that they could offer to a scoping review team will thus find little assistance.

Librarians have much to contribute at two particularly important steps: the initial formulation of the research question and the related need to balance breadth and depth, systematicity, and comprehensiveness in developing the database search strategy. Decisions taken at these stages can have dramatic implications for the number of articles retrieved, which could easily become unmanageable.

What uncertainty in methodology exists?

The uncertainty around the scoping review methodology is one of the first challenges that a librarian might encounter when working on these reviews. In our own experiences, we frequently face one of three scenarios:

• 1researchers not knowing what a scoping review is and looking for basic guidance;
• 2researchers who say that they are doing a systematic review when really their question lends itself more to a scoping review; or
• 3researchers who incorrectly believe that a scoping review is a fast systematic review.

In dealing with these scenarios, we have often taken on the role of the methodologist, as the team member who understands the scoping review process from start to finish or as the one with a better view of the bigger picture, outside of the question being posed. By taking on this role, we are able to respond to the above scenarios by (1) guiding researchers through the process; (2) helping researchers to think more deeply about the question they are asking; and (3) discussing with them the scoping review methods and emphasizing that it is not a fast process, and certainly not a fast and simple alternative to a systematic review.

How can librarian expertise improve scoping reviews?

Drawing on our experience with participating in and conducting our own scoping reviews, we present the scoping review methodology as it currently stands and suggest how librarians' specialist expertise and skills can be incorporated to improve the quality, rigor, and validity of the scoping review.

Librarians who are working with and advising a health sciences research team are likely to be most involved in steps 1 and 2 (Table 1), with the subsequent stages of interest primarily to librarians who are conducting their own scoping reviews. However, given our experiences of acting as scoping review methodologists to those research teams with whom we have collaborated, we suggest that librarians become familiar with the entire process.

STAGE ONE: IDENTIFYING THE RESEARCH QUESTION

Stage one involves clearly defining the research question, which tends to be broader in scope compared to the narrow, focused questions recommended for systematic reviews [4, 24]. For example, a scoping study question might ask, “How is walking endurance measured in people with multiple sclerosis?,” while a systematic review question might ask, “In people with multiple sclerosis, what is the extent to which a walking intervention compared to no intervention improves self-reported fatigue?” [26]. Levac et al. recommend clearly defining the concepts in the research question as early on as possible as well as carefully considering the purpose of the scoping review at this stage [12]. It is important to clearly define the question, given its implications in the later stages of the process [16].

We believe that the involvement of a librarian from the beginning of the process is vital. As in systematic reviews [3], librarians' training in reference interviews and implicit knowledge of how searches will retrieve articles can help focus and refine a study from the early stages, while changes can still be made relatively easily and before too much time is spent on later stages. Our skills in question formulation and refinement are especially useful with the broad nature of scoping review questions. Librarians can ask questions that, as Levac et al. suggest, focus the question:

Consider the purpose of the scoping study with the research question. Envision the intended outcome…to help determine the purpose of the study…Consider rationale for the scoping study to help clarify the purpose. [12]

Entering the conversation early, while the research question is still being refined, is therefore an opportunity for the librarian to shape the question and potentially save the research team a great deal of time.

Scoping questions are by nature broad, but how broad is too broad? Considerations include:

• ▪the discipline of the research; for example, the occupational therapy body of literature is smaller than that for general medicine, so occupational therapy naturally limits the breadth of the question;
• ▪practical considerations, for example, thousands of articles being screened and synthesized by two graduate students versus two faculty members, two graduate students, and a research assistant; and
• ▪the reason for doing the scoping review.

An initial search of the literature by a librarian on the research team's rough topic area can help the researcher understand just how large the question is and can facilitate the conversation about narrowing the topic. The librarian can use this information to ask questions, such as “Are you really interested in all chronic disease, or maybe you are really interested in specific chronic conditions?” A particular challenge for the librarian comes when an initial search of the literature reveals that the topic is much larger than the research team realized. If the team is not prepared to narrow their question or if they are already at the study section stage, the librarian may be left to either provide an unmanageable number of results or to narrow the search in a way that may not be as comprehensive as the research team would like. No matter when the librarian enters the conversation, managing the expectations of the research team regarding the number of articles retrieved is a common challenge.

STAGE TWO: IDENTIFYING RELEVANT STUDIES

This stage covers systematic searches of databases, the gray literature, and reference lists, as well as hand-searches of particular journals. It also covers decisions regarding appropriate search terms, time spans to cover, and language limits. Health librarians have developed substantial skills in systematic reviews [3], and these skills are directly transferrable to the scoping review.

Many authors emphasize comprehensiveness at this stage of the scoping review [8, 9, 12, 20], but researchers may not have the skills to design and execute the necessary search strategies [20] and should involve librarians [9, 12]. The Johanna Briggs Institute (JBI) manual states, “The input of a research librarian or information scientist can be invaluable in designing and refining the search” [23]. Levac et al. discuss the trade-off between comprehensiveness and feasibility and recommend the involvement of team members with methodological and content expertise, a recommendation that suggests including an information specialist [12]. Anderson et al. note that the review team should include a broad range of subject experts but do not specifically mention information specialists [8].

There is agreement in the literature on the need to balance breadth and depth when developing the search strategy [12, 18, 20], because broad research questions often produce a high number of results, as do sensitive search strategies, and the combination of both a broad research question and a sensitive strategy carries the risk of producing an unmanageable number of results. The broader nature of scoping review questions can mean that it is not possible for the search to be as comprehensive as for a systematic review [18], and finding the best balance between breadth and depth in a scoping review search, while maintaining a feasible number of search results, can be an enormous challenge. Considering that systematic reviews frequently limit to a specific study design (such as randomized controlled trials) while scoping reviews do not due to their more exploratory nature, finding this balance can be more difficult. It can be useful to explain this balance to researchers who are unaware that scoping reviews are not simply a quick and easy alternative to systematic reviews.

The creation and documentation of a systematic search strategy, which is both explicit and reproducible [27], is just as important in scoping reviews as in systematic reviews. All searches that support scoping reviews should be systematically conducted. In our own work, therefore, we record all of our search steps in detail, as for a systematic review; have our search strategies peer-reviewed [6, 7]; and use a PRISMA diagram to document the numbers of search results retrieved [5]. This ensures that our work is both systematic and reproducible. We thus disagree with Rumrill et al. [19] and Anderson et al. [8] and side with the CIHR [13] definition and with Colquhoun et al. [16] on this point.

Developing a scoping review search strategy

A list of target articles (or important articles that should be retrieved by the search) is particularly helpful in designing a scoping review search. We recommend iterative searching, which provides the librarian with flexibility in developing the search strategy. If there is uncertainty around certain terms or concepts, the librarian can always go back and find new articles.

We do not agree with the iterative search process that JBI describes, which recommends analyzing all of the initially retrieved articles for new keywords [23], as the team would be analyzing the irrelevant articles as well as the relevant ones. We have used a more targeted iterative searching technique in which we ask the research team to keep track of new keywords that they find while they screen the articles. Once the final set of included studies is isolated, the librarian can analyze the subject headings used in their indexing (a tool such as Yale's Medical Subject Headings [MeSH] analyzer [28] makes quick work of this step) in order to identify frequently appearing subject headings that were not included in the original search. A new search is then conducted combining these new subject headings along with any new keywords gathered by the research team; joining these terms with the rest of the search, as described in Table 2; and removing articles previously supplied to the research group (for example, by using the “NOT” operator). This allows the librarian to determine whether the newly identified terms retrieve any new articles of interest and should be included in the final search strategy. A scoping review by Thomas et al. employs this iterative search approach [29].

Table 2.

Workflow for incorporation of new subject headings and keywords into iterative search process

STAGE THREE: STUDY SELECTION

The third and subsequent stages are likely to interest those librarians who are considering conducting their own scoping reviews or who wish to adopt the role of methodologist in a faculty collaboration. Researchers might have a general sense of the scoping review process but might seek detailed advice on methodology or pragmatic approaches to information management, particularly if they are not using specialized software for their reviews.

Levac et al. describe this stage as iterative [12]. In the initial phase of study selection, as the research team begins to explore and more deeply engage with the literature, they may get new information to feed into the study identification phase. This can result in requests to edit and re-execute the search strategy, which is when the iterative search process described above is useful. The scoping review process is thus usually not linear.

Two considerations become important at the study selection stage: (1) development of inclusion and exclusion criteria and (2) appraisal of the quality of included studies. In contrast to systematic reviews, inclusion and exclusion criteria are developed post hoc [20] and, like the search strategy, can evolve throughout the process due to the exploratory nature of scoping reviews. Careful documentation is required to keep track of this evolution, for example, when reporting reasons for exclusion of articles in the PRISMA diagram.

The available advice on quality appraisal of included studies is inconsistent. Arksey and O'Malley [20] and others [24, 27] do not recommend it, while Daudt et al. advise that:

assessing for quality is a necessary component of scoping studies if they are to provide research that in itself can be disseminated to others in a way that is useful to practice or policymaking and for future researchers. [9]

Furthermore, Levac et al. point out that it is difficult to employ the scoping review methodology to either identify gaps in the literature or determine the value of conducting a full systematic review if quality assessment of included studies is not performed [12]. Pham et al. note that only 22% of published scoping reviews conducted a quality appraisal exercise [18]. Librarians might not be involved in this stage, but those who are conducting their own scoping reviews will decide to conduct a quality appraisal depending on the time and resources at their disposal, as well as the feasibility of appraising the quality of a potentially large number of studies.

STAGE FOUR: CHARTING THE DATA

This stage includes sifting, charting, and sorting, comparable to the data extraction phase in a systematic review. Standard information is collected from each citation, often in a spreadsheet format; for example, researchers may use a spreadsheet to keep track of citation information, study location, populations, study design and methodology, intervention type, comparator, duration of the intervention, outcomes measured, and important results. Librarians are well positioned to provide advice on information management at this stage. Some authors recommend a trial run of data charting, followed by a team meeting to ensure that there is consensus regarding the process, which also ensures the collected data are as “rich” as possible [9, 12]. Other teams may engage in a more iterative approach [12]. This stage can reveal that the data do not answer the research question and involve returning to earlier stages to adjust the question and/or inclusion and exclusion criteria. An iterative approach can also involve developing the data extraction fields throughout the entire screening process as new issues or points of interest emerge. A full description of this stage is provided by Booth et al. [27].

STAGE FIVE: COLLATING, SUMMARIZING, AND REPORTING THE RESULTS

This stage involves collating, summarizing, and reporting the data, and according to Levac et al. [12], it overlaps with stage four. It also includes analysis of the data. Methods of data reporting depend on the question, but most often involve organizing the data into themes. At the very least, we recommend a list of included studies (as an appendix if the list is lengthy), a PRISMA diagram, and at least one complete search strategy. As with systematic reviews, this stage is also an excellent opportunity for the librarian to contribute by writing the relevant sections of the methodology section in the final paper. We again refer readers to Booth et al. for a full description of this stage [27].

STAGE SIX: CONSULTATION EXERCISE

Stage six, described as optional yet recommended by Arksey and O'Malley [20] and Pham et al. [18], involves consultation with stakeholders, including practitioners, patients, information consumers, or other key informants. They found that such consultation improves their understanding of the issues that had an impact on the approach that they took in their own scoping reviews and enhanced literature retrieval when they followed up on references that their stakeholders suggested, particularly in terms of gray literature. Levac et al. [12] and Daudt et al. [9] suggested that this step be required. Taking this recommendation even further, Daudt et al. recommended consultation throughout the process, rather than just at the end [9]. In our experiences, a stakeholder has also been involved from the beginning as part of the research team. For example, in a research question about chiropractic interventions, chiropractors could be consulted at the end of the process about the information found or a practicing chiropractor could be included on the research team from the beginning of the process.

For library and information science (LIS) scoping reviews, since many librarians who conduct the reviews are also practitioners of librarianship, this step may not be necessary.

DISCUSSION AND CONCLUSION

Scoping reviews are an emerging area of scholarship offering numerous opportunities to health librarians. By using skills developed from our professional training and demonstrated during comparable projects such as systematic reviews, we have a distinctive and vital contribution to make in ensuring the rigor, validity, and success of scoping review projects, whether as expert searcher, information manager, or methodologist.

Research teams undertaking a scoping review should involve a librarian at the earliest possible stage. Like systematic reviews, scoping reviews need a rigorous, systematic search strategy, and librarians are the best-qualified people for this role. We provide a full list of recommended roles for librarians in scoping reviews in Table 3. As with systematic reviews, the librarian contribution will usually be significant enough that it fits the International Committee of Medical Journal Editors definition of authorship [30]. We recommend that a collaborating librarian request authorship status when committing to the scoping review project.

Table 3.

Recommended librarian contributions to scoping reviews

Scoping reviews are also important in librarians' own original LIS research. Our field has many questions that need to be answered, and often these questions are too broad or lacking in evidence to lend themselves to systematic reviews. Scoping reviews could be an important way for us to synthesize knowledge in LIS.

Are scoping reviews a form of knowledge synthesis?

There is some discussion in the literature regarding the appropriateness of the term “synthesis” versus “summary” in describing a scoping review [12, 22], although Arksey and O'Malley [20] and Miake-Lye et al. [31] do employ “synthesis” in describing stage five, and “synthesis” is, rather surprisingly, used to describe these reviews even in articles arguing against its applicability [12, 22]. At a 2015 meeting held to advance the scoping review methodology [22], participants' one key concern with using the term “synthesis” was that they saw it as applying to quantitative analyses [22]. However, we feel that scoping reviews fall solidly within the CIHR definition of knowledge synthesis: “the contextualization and integration of research findings of individual research studies within the larger body of knowledge on the topic” [13]. Scoping reviews are an important tool for synthesizing knowledge for its transfer into professional practice, whether in health care or LIS.

In 2005, McGowan and Sampson [3] and the Medical Library Association [1] emphasized the important role of medical librarians in conducting systematic reviews. Our subsequent activity in this field has had a significant positive impact on our colleagues' perceptions of medical librarianship [32], has reduced waste and increased value in biomedical research [33], and led to the formalization of librarians' role in the Institute of Medicine's systematic review standards [34]. A little over a decade later, the increasing popularity of scoping reviews provides librarians and our collaborators with a similar opportunity. The time has come for librarians to actively participate in the scholarly discussion on scoping reviews and to emphasize the positive role that we can play. It is also time for librarians to consider scoping reviews in addition to systematic reviews [35] as an appropriate method for advancing evidence-based practice in our own profession.

Biography

Martin Morris, MSc (corresponding author), martin.morris@mcgill.ca, Liaison Librarian; Jill T. Boruff, MLIS, jill.boruff@mcgill.ca, Liaison Librarian; Genevieve C. Gore, MLIS, genevieve.gore @mcgill.ca, Liaison Librarian; Schulich Library of Science and Engineering, McGill University, 809 rue Sherbrooke Ouest, Montréal, QC H3A 0C1, Canada