Mapping studies

When I first read about evidence-based practice, I had several thoughts. The first was a somewhat horrified question: Does this mean that medical care has not been based on research? The second question was that if we intend to build a better knowledgebase for this and other professions, how could that happen, if the base (existing research) was not itself evidence based? The image of a house built on sand would not leave my mind.

Throughout the history of librarianship, we have focused our sights on continual improvement. For at least the last twenty years, evidence-based practice has provided an evolving roadmap to increasing awareness of quality and to moving forward using the best, most appropriate, and most rigorous methods we know. To build our base and support decision making, increased awareness of research methods can help to retool a rapidly changing profession.

One such research method is the mapping study, also called a mapping review. Reviews of all kinds are done to gather information to build a base for further research or to inform decision making, and this is equally true of mapping. In the literature of medical librarianship, the best-known examples of mapping studies follow a protocol created in 1993 (updated in 2010 and currently under review) by members of the Medical Library Association (MLA) Nursing and Allied Health Resources Section (NAHRS) Subcommittee on Mapping the Literature of Nursing and Allied Health. Two broad initiatives have been supported by NAHRS to map the literature of nursing and allied health, and task force members have provided the evolving protocol [1] and other support to researchers—members of the task force and beyond—in order to expand the work.

MAPPING STUDY

Overall, mapping is a systematic approach to understanding the “map” of a profession, theory, research question, or practice. The term mapping is also used for “concept mapping,” which shows how concepts are related in a visual way. Similarly, mapping studies can show how literature is disseminated through journals, books, websites, and other channels. Examples include NAHRS-driven mapping studies on the subspecialty of emergency nursing [2] and the issue of access to healthy foods in rural communities [3]. Depending on the objectives, mapping research involves various degrees of rigor. Beyond library and information science (LIS), it has been used in medical and allied health research, software engineering, education, and public policy, but the NAHRS protocol–driven research appears to be unique to our profession.

Most readers may be familiar with literature reviews done to provide a background and rationale for research, usually found after the introduction and before the methods section of a paper. Other literature reviews may themselves be the focus of an entire publication. The process used to find the works referenced in these types of literature reviews is not usually discussed, and the reader usually has no way to know whether the search for background information has been comprehensive or whether the cited works are quality research. As a result, bias can be introduced because of the author's choice of literature reviewed, and ultimately, time for the researcher and readers might be wasted if the new work is a replication or if it is based on biased or faulty research.

If we can consider that review types occupy a spectrum in terms of comprehensiveness, transparency in methodology, and rigor, systematic reviews are at the far end of the scale that begins with the generic literature reviews described above. Both systematic reviews and meta-analyses follow carefully documented protocols, retrieving, categorizing, and then statistically synthesizing research done in a narrow topical area to draw conclusions for best practices.

Other review types measure the impact of theories or authors on the literature (critical reviews) or identify the most likely journals and database resources for specific topics (mapping studies using the NAHRS protocol). Depending on the protocol, they can also categorize literature through content analysis, showing researchers where gaps exist for future studies (systematic mapping studies). Unlike other review types (systematic reviews), mapping studies generally do not involve critical content evaluation or statistical synthesis of findings.

Mapping reviews are closest to scoping reviews and are frequently confused with one another. A scoping review is usually intended to provide an overview of “what is out there” on a particular topic but does not focus on where literature may be found. Like scoping reviews, NAHRS mapping reviews do not ordinarily include evaluation of the literature in terms of quality, and there is no statistical synthesis. The NAHRS mapping reviews are intended to identify patterns of publication, aid collection development decisions, and guide literature retrieval.

Mapping research is also performed in other disciplines and on a wide array of topics, including software engineering, science, education, terrorism research, decision sciences, social justice, health policy, and more. Outside of LIS, the method could more appropriately be termed systematic mapping review. These are done to support further research, most commonly systematic reviews, and are intended to “classif[y] the primary research papers in that specific domain” [4]. Kitchenham et al. described how mapping and systematic reviews can overlap if they also include discussion of outcomes or classify included works by research methods used, even extending to assessment of research papers in a subcategory, but similarities end where systematic reviews categorize, appraise, and synthesize the included works. As opposed to systematic reviews, mapping studies include many more citations because the research objectives are far broader.

OVERVIEW OF THE NURSING AND ALLIED HEALTH RESOURCES SECTION MAPPING REVIEW PROCESS

Beginning in 1993, members of the MLA NAHRS Task Force on Bibliographic Access for the Allied Health Literature have conducted multipart, large-scale projects titled Mapping the Literature of Allied Health (1997) [5], Mapping the Literature of Nursing Literature (2000) [6], and recently, the NAHRS Journal Project [7]. The mapping protocol, amended in 2010 and again in 2014 to include examination of database coverage [1], continues to base analysis of dispersal of publications upon a formula called Bradford's Law of Scattering [8], with the stated purpose of (1) identifying core journals from a 3-year span, (2) determining bibliographic coverage, and (3) influencing database producers to improve access. In this section, basic steps for a NAHRS protocol-driven mapping study are described, followed by a brief discussion of processes involved in mapping studies beyond the NAHRS protocol efforts. Further information can be obtained by reading the current protocol [1].

The following steps are a basic description of the NAHRS protocol. To follow other mapping study methods, you are advised to search within the discipline or subject area to learn whether there are commonly recommended practices.

• 1Determine whether the topic has already been mapped (or if your work is an update, determine whether the update is warranted). In the case of NAHRS protocol–driven mapping efforts, you are requested to contact the NAHRS Mapping Subcommittee chair, who can help to identify areas needing research and can advise you about use of the protocol.
• 2Provide background information about the specialty or subspecialty area being studied. You can approach this part by considering that your intended audience may not know much about the topic. Perhaps your reader is a medical librarian who is interested in strengthening the collection to support trauma nurses (for example), so this overview does not need to be extensive.
• 3Identify top journals in the area of your study. This is done by asking practitioners in field, as well as consulting rankings in citation or impact factor reports, MLA and other lists of key journals, and research on the topic of interest, if available. Other resources, including books and gray literature, might be included in this step of the review process, but journals are by far the most productive resources. “Core” is used here as it applies to the ranked volume of productivity for specific topics under review (e.g., journals identified as primary publication outlets for nursing administrators). Ideally, you should include three to five journals for your work, although studies have been conducted with as few as one. If you find that only one or two journals are emerging as core to the topic, you might expand your research question (e.g., emergency nursing rather than trauma nursing).
• 4In the next phase of the study, record article citations from core literature (called “source” journals in the NAHRS protocol) published in a recent time span (generally, three years) in a database; spreadsheets such as Microsoft Excel can be used. Decisions are made at this step about inclusion and exclusion of materials—for example, whether to include editorials and letters to the editor, reprints of historical articles, or non-research papers. While it may be tempting to limit to only research articles, you might base your decision on whether publication types (editorials, continuing education articles, etc.) tend to include citations. If so, they should probably be included. Publication types should be documented separately. Materials cited can, for example, be separated into journal articles, books, government documents, Internet resources, and a catch-all “miscellaneous.” At a minimum, this should a brief citation including author, title, abstract, and journal title information. Including a digital object identifier (DOI) can help if there are problems. You can use a bibliographic software program to aid this part of the process, which can occupy the majority of research time, depending upon the volume of literature.
• 5Calculate distributions and separate into 3 zones using a formula following Bradford's Law of Scattering [8], which states that level of dispersion of literature in a field of study is related to how well it is established: a small core of journals publish the bulk of the literature (Zone 1 being most likely to publish in a particular area), followed by Zone 2 (somewhat less likely) and Zone 3 (least likely, with a distribution that could be described as a “long tail,” meaning a very long list of publications with very few pertinent works). The implication is that core resources will be most fruitful when searching for the topic being mapped. As literature becomes more scattered and sparse, effort expended for retrieval increases [9]. For specific instructions on how to do this, see the NAHRS protocol [1].
• 6Finally, database coverage for the (journal) resources grouped into Zones 1 and 2 is examined and reported.

THE VALUE OF MAPPING STUDIES IN INFORMATION SCIENCE AND BEYOND

From scanning mapping reviews in LIS and other disciplines, the potential contributions of well-done reviews to our profession are evident. First, the review can efficiently acquaint readers with an area of research, benefitting students and policy makers among others. For librarians, a study focused on dispersal of the literature across journals (and other literature) and electronic resources can directly support decision making about collections and subscriptions. While it may be obvious that Zone 1 resources should be provided, budgetary constraints may mean that Zone 2 items have to be carefully selected. Simply having those items identified is a benefit to the value provided by libraries to their users. The results from such a review can also support the creation of a search string in order to expand or update knowledge.

Contributions to research include identification of trends, when a review is replicated periodically. When the review focuses upon a particular topic, gaps in research can be identified, supporting the justification for original research. The set of results can also buttress a systematic review or make evident where there is a sufficient volume of research to support a systematic review.

ENSURING MAPPING STUDY QUALITY

Mapping quality depends on objectives, so it would help a great deal to read similar published reviews before getting started. Mapping is a bibliometric method of inquiry, so data handling and transparency in processes are of utmost importance in demonstrating reliability to readers or in providing a basis for further research. Bias depends not on whether the content has been critically evaluated, but on how comprehensively and consistently the protocol has been followed and how well processes, decisions, and results have been documented.

The scope of the review should be clear, with any ambiguous terms resolved, leaving no doubt of the focus of the work. Transparent and replicable reporting of all processes is important. Searching and retrieval, filtering, data handling, and analysis should all be logically described. If the study is based upon a topic of research, incorporating inter-rater reliability procedures for categorization can lend strength to the work. Document types included are either homogenous (all one type, such as research articles) or, heterogeneous, where types are classified and analyzed separately. Categories (of subtypes), if used, should be mutually exclusive and clearly defined, should be refined to the most appropriate level in order to inform understanding, and should be made explicit to permit further mapping or other research.

When reading a mapping review, you might ask whether it accomplished its stated objectives, within the protocol being used. If appropriate to the objectives, does the conclusion clearly point out gaps, trends, or important authors? Are limitations and weaknesses of process, access, date range, or other areas clearly described? Do they make sense? Because of evolving technology, it may help those who follow if you describe stumbling blocks you encountered during the process. Talking about data handling, for example, may move the profession forward in research capability. Finally, are suggestions made for further research based on evidence gaps in topical or other coverage, or if more appropriate, is a clear connection made to the potential use of the findings?

To understand the dissemination of NAHRS mapping studies in a very non-rigorous manner, I searched for two of the studies published in the Journal of the Medical Library Association in 2006: “Mapping the Literature of Nurse Practitioners” (Study A) [10] and “Mapping the Literature of Nursing Administration” (Study B) [11]. These works were part of the early effort to map nursing literature. According to Google Scholar (which is far from perfect, as it tends to include duplicates and to index older works [12]), the articles were cited 5 and 7 times, respectively. But that is not the end of it: the articles citing Study A (subject areas: LIS and nursing) were cited a total of 39 times, and in turn, those studies were cited a total of 144 times. Study B's 7 citations were cited 40 times, which in turn were cited in 241 publications. These 2 original mapping studies made a difference far beyond the original discipline and publication.

As a research method, mapping is only as good as its transparency and rigor. Mapping helps to shape understanding of the volume and dispersion of literature within a particular topic area and period of time, and it finds its place among a wide range of review methods that are continuing to evolve. Librarians are contributing to our own knowledgebase, but we are also fully capable of supporting—even leading—mapping research studies for other disciplines. Our work guides research and advances practice.