Database searches for qualitative research

Abstract

Interest in the role of qualitative research in evidence-based health care is growing. However, the methods currently used to identify quantitative research do not translate easily to qualitative research. This paper highlights some of the difficulties during searches of electronic databases for qualitative research. These difficulties relate to the descriptive nature of the titles used in some qualitative studies, the variable information provided in abstracts, and the differences in the indexing of these studies across databases.

INTRODUCTION

Over the past few decades, there has been an increasing emphasis on ensuring that health care decisions are based on the best available evidence [1]. This evidence-based approach to health care attempts to bridge the gap between research and clinical practice [2]. An important component of this approach is the systematic review, which aims to identify, appraise, summarize, and communicate the results and implications of otherwise unmanageable quantities of research [3]. These systematic reviews provide a rigorous summary of the best available evidence.

As part of the systematic review process, a systematic and comprehensive search of the literature is undertaken. The aim of this search is to identify all studies addressing a review topic. The major focus of systematic reviews has been studies using quantitative research methods, in particular, the randomized controlled trial (RCT). As a consequence, current search strategies have focused predominantly on RCTs. However, interest in the role of qualitative research within the evidence-based framework is growing [4, 5]. This interest is witnessed by the growing number of reviews of qualitative research addressing issues such as adaptations to diabetes [6], anxiety surrounding breast disorders [7], and parents' perspectives of acute childhood illness [8].

Qualitative research has an important role in evidence-based health care, in that it represents the human dimensions and experiences of the consumers of health care. This type of research does not answer questions concerning the effectiveness of health care, rather it provides important information about such things as the appropriateness of care and the impact of illness. It also provides a means of giving consumers a voice in the decision-making process through the documentation of their experiences, preferences, and priorities. The term “qualitative” is used to represent a number of different research methods such as ethnography, phenomenology, and grounded theory. This research differs significantly from experimental research in that it focuses on narrative data rather than numbers. Despite the growing interest in this type of review, only limited attention has been given to how this qualitative information can be incorporated into systematic reviews. Additionally, the current methods used during reviews of experimental research do not translate easily to qualitative research.

One aspect of the difficulty of translating RCT review methods to other types of research is that of database searches for qualitative studies. These difficulties are encountered during searches of the titles of studies listed in databases, their abstracts, and the index terms used to describe their contents. The aim of this paper is to present a few of the difficulties encountered during electronic database searches for qualitative research.

THE SEARCH PROCESS

The task of the literature reviewer is to uncover all the articles on the review topic; however, this task is difficult because the full complement of articles is never known. Evaluation of MEDLINE searches has found that only 30% to 80% of published RCTs are identified during searches [9]. To improve the likelihood of identifying all relevant papers, there has been considerable development and discussion of optimal approaches and strategies [10–13].

To aid the search process, methodological search filters that filter out those studies using the research method of interest have been developed [14]. These methodological filters are predetermined search strategies that use terms related to research design to identify all those studies using the research method of interest to the reviewer. These methodological terms are then combined with subject terms that appear in titles, abstracts, or indexes to identify studies using the research method of interest and addressing the review topic. The purpose of these search filters is not to retrieve all publications on a topic, but rather to filter only those most relevant to the review [15].

This approach has been used with great success for the identification of RCTs. However, some of the unique characteristics of qualitative research may limit the easy translation of these search techniques. These differences relate to the titles used in the reports of qualitative studies, the content of their abstracts, and the indexing practices of electronic databases.

TITLE SEARCHES

An important component of any search for RCTs has been the use of specific words to search the titles of publications listed in databases. This approach assumes that the titles of the publications clearly describe the subjects and methods employed in the studies. For example, the title “Intravenous Heparin for the Prevention of Stroke Progression in Acute Partial Stable Stroke: A Randomized Controlled Trial” [16] provides a clear description of both the population, condition, outcome, and research method. Identification of this study during a database search would be relatively simple.

Compared to this precise terminology, qualitative research publications often use titles, which, like the studies they report, could best be termed “descriptive.” These descriptive titles clearly describe the focus of the studies and are very appropriate given the nature of qualitative research. For most situations, the use of these descriptive titles does not cause a problem, and, indeed, they provide rich descriptions of the studies. However, during database searches, they can add to the complexity of identifying qualitative research on a specific topic. Examples of the titles of qualitative studies include:

• “‘How Can I Put This?' Exaggerated Self-Disparagement as Alignment Strategy during Problematic Disclosures by Patients to Doctors” [17]

• “Unbearable Incidents: Failure to Endure the Experience of Illness” [18]

• “Setting Boundaries: A Strategy for Precarious Ordering of Women's Caring Demands” [19]

Each of these titles provides a rich description of the study's focus, while at the same time making selection of specific key search terms very difficult. The major concern during searches of study titles is that authors and searchers may differ in how they define concepts. It has also been suggested that these differences in defining concepts are important factors contributing to failed database searches [20]. It has also been suggested that database searches relying on key terms in the title of studies will miss many relevant publications. Additionally, given the nature of these qualitative study titles, database searches using broad search terms may result in the retrieval of many hundreds or thousands of irrelevant papers.

ABSTRACT SEARCHES

Electronic databases include abstracts as part of the information provided about publications and so can also be searched. This component of a search can focus on either the research method or subject. The success of these searches will be influenced by the completeness of the information provided in the abstract. The structure of abstracts for RCTs has come under some scrutiny, and there has been a call for more informative or structured abstracts [21–24]. These structured abstracts follow a standardized format and not only guide authors in summarizing the content of the manuscript precisely, but also assist in the selection of clinically relevant journal articles [25]. An evaluation of abstracts in MEDLINE conducted in the early 1990s found that the number of structured abstracts had increased significantly and that articles with structured abstracts were assigned more Medical Subject Headings (MeSH) than MEDLINE articles as a whole [26]. However, while there has been no evaluation of the abstracts from qualitative studies, inspection of these abstracts suggests that their content varies considerably, and not all abstracts state the research method. Once again, while this does not necessarily reflect on the quality of the research, it increases the difficulty of finding these studies.

INDEX TERMS

Another method used to identify research listed in electronic databases is the use of index terms. Index terms are used to describe both the subject and the method of research publications. Index terms have been used as part of RCT search strategies to improve the effectiveness of searches [27].

However, as qualitative research has not been a central component of the evidence-based movement, there has been little evaluation of the effectiveness of this approach in identifying these studies. Additionally, while it has been reported that some RCTs have been indexed incorrectly in electronic databases, thereby increasing the difficulty in locating these studies [28], there is little information on the accuracy of the indexing of qualitative research. The risk in this situation is that potentially relevant studies may be missed during the search, because inappropriate index terms have been used or the database index terms used for a study do not accurately reflect its contents.

In comparing the indexing of qualitative research publications in MEDLINE and the Cumulative Index to Nursing and Allied Health Literature (CINAHL), some differences can be noted. Qualitative research publications listed in MEDLINE appear to be indexed under what may best be described as a “quantitative framework.” In comparison, the same qualitative studies indexed in CINAHL are indexed using terms that more accurately reflect the qualitative methodology. One example of this difference is shown in Table 1. In this example, the indexing terms used by MEDLINE and CINAHL for the same publication are listed. CINAHL clearly utilizes a number of methodological indexing terms that accurately describe the qualitative study design. In contrast, MEDLINE has not used any methodological index terms in this example. The greater depth of indexing demonstrated in this example is consistent with the major focus of CINAHL being nursing and allied health literature, professions that commonly use qualitative methodologies.

Table 1 Comparison of indexing practices between MEDLINE and Cumulative Index to Nursing and Allied Health Literature (CINAHL), based on a single article titled “Infant Feeding Choices among First-Time Mothers” [29]

Searches utilizing these methodological index terms will likely identify a greater number of qualitative publications in the CINAHL database. This finding is supported by a comparison of the two databases in 1989, which finds that the descriptors used in CINAHL focus more on nursing topics than those in MEDLINE [30]. This example suggests that for the identification of qualitative research, searching the CINAHL database may identify a greater number of relevant publications. Investigation of the index terms of qualitative research cited in CINAHL suggests developing a qualitative research search filter is possible (Table 2). While the validity of such a search filter has not been established, when combined with appropriate subject terms, it may improve the yield of relevant studies.

Table 2 Methodological index terms that aid in the identification of qualitative research during CINAHL database searches

DISCUSSION

With the ever-increasing volume of health care literature, identifying relevant studies will become increasingly difficult. In response, the comprehensive search of systematic reviews will be become an important means by which this research is identified, communicated, and recorded. Without this record, many studies risk being lost in the vast amount of health care literature. Additionally, without effective methods of identifying relevant qualitative research, individual studies are unlikely to have an impact beyond the local clinical area of the researchers.

While much of the discussion in this paper is based on anecdotal information, it suggests that the identification of qualitative research in electronic databases is complex and difficult. It is not clear how significant this problem is, and it does not appear to have been previously addressed in the literature. However, based on limited information, it is likely that qualitative studies that use creative titles or provide inadequate information in their abstracts will be at greater risk of not being identified during searches. While the difficulties of identifying all RCTs on a topic of interest have long been known, the challenge of identifying qualitative studies may be greater.

This discussion also suggests that the effectiveness of electronic database searches may be improved by using a number of different search strategies and incorporating a number of different databases. However, as a result of the greater depth of indexing of qualitative research in CINAHL, this database should be a major focus of any search for this type of research.

CONCLUSION

This paper suggests that the identification of qualitative research in electronic databases is both complex and difficult. These problems relate to the lack of suitable search terms in the titles of some of this research, variable quality of abstracts, and different indexing practices utilized across databases. It appears these issues have received little attention, and, on this basis, it is suggested that this area needs further investigation.