Comparison of bibliographic databases for information on the rehabilitation of people with severe mental illness

Abstract

Objective: The research sought to examine the overlap in coverage between several health-related databases, thus enabling the identification of the most important sources for searching for information on the rehabilitation of people with severe mental illness.

Methods: The literature was searched within a systematic review. Several health-related databases were retrieved (Cumulative Index to Nursing and Allied Health Literature, The Cochrane Library, MEDLINE, PsycLIT, Sociofile, and Social Science Citation Index), noting their source and comparing results retrieved from each database.

Findings: The total number of studies retrieved from each database varied. Almost a third of the papers retrieved from each database were unique to that source. Forty-two percent of the papers were only found in one database. Restricting a search to one database alone would miss many papers and could affect the results of a systematic review. PsycLIT was the most useful database for this topic area, containing 44% of the papers. MEDLINE, the database of first choice for many health professionals, held only 29%.

Conclusions: No database was determined to be significantly more useful than any other—each warranted inclusion in the study. Reliance cannot be placed on one database alone, and other methods such as hand searching should also be used. Although this may not be new information for information professionals, it is likely to be new for health professionals and researchers who are increasingly performing their own literature searches. Information professionals have an important role to play in conveying this message to those outside their profession.

INTRODUCTION

In health care, the evidence-based practice movement has gained increased acceptance over the last decade, with systematic reviews being widely used to answer questions about the effectiveness of health care [1]. Alongside, there is increasing interest in the feasibility of extending this approach in other sectors of public policy, including education [2, 3], social work [4], and social care [5]. Central issues for debate include the need to draw on findings from qualitative research studies, given a strong tradition of qualitative as well as quantitative studies in these fields and the complexity of the interventions themselves, and ways to synthesize findings from studies with a multiplicity of designs.

The core foundation for high-quality systematic reviews is effective retrieval of research studies undertaken on the topic. Users of the resultant overviews need to be confident that the set of papers reviewed is comprehensive and complete or, more controversially, at least in a statistical sense, representative of the available papers [6]. It is essential that a thorough literature search be carried out, using a range of sources and effective search strategies.

There are however a large number of health-related databases available to search, even for English language literature. These include MEDLINE, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), and the Cochrane Library. The questions are: which database or databases to search and what is the extra gain from searching more than one database. This issue becomes much more acute when the topic area of interest is not purely biomedical (for example, the effectiveness of a drug) but also requires the explicit adoption of a sociomedical model, where multidimensionality and the joint contribution from a multiprofessional set of practitioners are central (for example, community-based interventions). It is further compounded where it is important to gather together findings from high-quality, causally focused quantitative research designs (typically, randomized controlled trials) and interpretative qualitative designs. Interest in the latter arises from the twin need to uncover the perspectives of service users and their informal caregivers and to gain insight into ways in which the intervention itself may operate.

An opportunity to explore such issues arose in the context of a U.K. Department of Health–funded research project to explore the feasibility of undertaking systematic reviews of effectiveness and outcomes in the social care field. Social care in the United Kingdom refers to care provided outside a hospital or continuing-care setting (for example, nursing homes) to support individuals with ill health or other difficulties to live their lives as independently as possible. Support may come from such groups as social workers, home care services, community-based nurses, occupational therapists, and physiotherapists. In this research project, policy makers identified two areas for systematic review: rehabilitation for people with severe and enduring mental illness (schizophrenia or affective psychoses) and preventive services for older people in the context of bereavement.

Against this background, following a short review of previous literature on locating evidence, all within the health care field, this paper explores the overlap in coverage between several bibliographic databases in social care. It draws out implications for information professionals and clinical practitioners seeking evidence to ground their practice upon or interpreting the results of systematic reviews.

INFORMATION AND SYSTEMATIC REVIEWS

In the United Kingdom, the National Health Service (NHS) Centre for Reviews and Dissemination, building on work of the Cochrane Collaboration, provides recognized guidelines on sources and searching for undertaking systematic reviews of health care interventions [7]. Their advice indicates that initial literature searches should be undertaken to assess the volume of available literature and study designs used for the topic in question. Following this assessment, an extensive search should be carried out using a wide range of sources of published and unpublished literature to ensure that as comprehensive a list as possible of primary studies is located. Looking more widely, a number of strands are evident in the published literature: effective ways to locate randomized controlled trials; searches of multiple databases; ways to overcome indexing problems, in particular, through the use of search filters; and insight into the issue of database overlap.

A study undertaken by Dickersin et al. [8] by use of comparison against a gold standard showed that searching MEDLINE alone would retrieve about half the relevant studies available on that particular topic area. Inadequate indexing related to research methodology was highlighted. A comprehensive search strategy (search filter) was presented that aimed to overcome the indexing problems in locating randomized controlled trials. Search filters have also been developed, using subjective and objective methods, and tested to overcome indexing problems in other areas such as systematic reviews, outcome measures, and qualitative research methodologies [9–11].

A number of studies have been published on the issue of database overlap and coverage, although not necessarily in relation to systematic reviews. In psychiatry [12], using journal coverage as the measure of assessment, large variations were found in overlap between PsycLIT, EMBASE/Excerpta Medica, BIOSIS, and MEDLINE, with 35% of the journals being indexed by only one of these databases. For biomedical information [13], searches of SciSearch in addition to MEDLINE produced unique references in eight out of nine cases. In pharmacy [14], using journal coverage and number of citations retrieved as a measure, little overlap was evident in coverage between International Pharmacy Abstracts and MEDLINE. There was a similar finding in four allied health areas for MEDLINE and CINAHL [15]. Finally, a study in toxicology [16] highlighted that a combination of three databases was needed to ensure that 90% of relevant literature on the subject areas was retrieved.

These papers, all within the health care field, appear to present a common picture. Locating a comprehensive set of papers is not an easy task. Using search filters improves the chance of an effective search. However, because of the difference in journal coverage of different databases, searching a range of databases is essential. This consideration is important in a context where the results of the review are likely to be taken as “gold-standard” evidence of effectiveness in the area. The present study, using search filters and searching a range of databases, aims to contribute to the body of evidence in this area.

METHODS

The topic area selected for this study was the literature searching undertaken for a systematic review of the evidence on the effectiveness and outcomes of community-based rehabilitation services provided for adults with severe and enduring mental illness. Interest lay in locating both quantitative research designs (to be at least a pre-post design, with or without a comparison group) and qualitative studies giving insight into service user and care giver experiences and their perceptions of the modes of service delivery. The review itself was organized around two specific areas:

• models of service delivery through which support is organized and provided to those with severe and enduring mental illness to enable them to remain in the community

• specific services or interventions viewed as essential for the rehabilitation of the severely mentally ill, which are also offered as specialist programs (for example, psychosocial skills training and vocational rehabilitation)

A key focal point was the rehabilitation outcome domains related to service user and care giver conceptions of an ordinary life. These covered a place to live, meaningful daytime activity, adequate income, and a varied social life.

Searching and selecting studies for inclusion in the systematic review

A multilevel iterative approach informed the literature searching [17]. In a scoping stage, experts in the topic area of severe mental illness were consulted to identify possible search terms and appropriate electronic databases for searching within the limitations of their availability at the two collaborating institutions (Universities of Salford and Leeds in the United Kingdom). Search terms (for example, “severe mental illness,” “schizophrenia and affective disorders,” and “psychotic”) were selected after test searches and discussions among the project team and were combined with published search filters for randomized controlled trials and systematic reviews [18, 19].

Seven databases were used: MEDLINE (general biomedical literature), CINAHL (nursing and allied health literature), Caredata (social work, social policy, gray literature), PsycLIT (psychological literature), Cochrane Library (health-related systematic reviews and randomized controlled trials), Sociofile (sociology literature), and Social Science Citation Index (social science literature). Relevance checks on the abstracts of the retrieved articles (967 studies, including duplicates) were then undertaken independently by two members of the research team. Focus lay inter alia on whether the intervention could be regarded as rehabilitative, what type of paper, and what type of study.

In a refinement stage, the inclusion and exclusion criteria were redrawn (Appendix A). Modifications included an exclusion of studies that involved medical treatments (drugs, surgery) and crisis interventions primarily based in secondary (hospital) care settings and an inclusion of studies from non-U.K. countries where the client group, nature of intervention, and policy implications were judged to be similar to those in the United Kingdom. The search strategies were redrafted (Appendix B) to reflect these changes. The searches were rerun, and ninety-six studies were identified as meeting the inclusion criteria. These included 5 systematic reviews, 34 randomized controlled trials, and 3 qualitative studies. In the main study, each of the individual studies was critically appraised, and a subsequent narrative synthesis of the evidence was drawn up [20].

In a confirmation stage, to check the quality of the search strategies and identify studies missed by the database searches, hand searches of a number of journals were undertaken (British Journal of Psychiatry, Journal of Mental Health, Journal of Mental Health Administration, Psycho-Social Rehabilitation, and Psychiatric Services). These journals were selected as representative of qualitative as well as quantitative material and reflective of U.S. and U.K. literature. In most cases (82%), articles were retrieved by the database searches, except for one journal, Journal of Mental Health, from which fewer articles (33%) were retrieved. Citations from the studies meeting the inclusion criteria for the review were checked to identify further studies that were not retrieved by either the hand searching or database searches. In addition, searches for gray literature were undertaken through Caredata, Local Authority Research in Action (LARIA), and the Websites of the Joseph Rowntree Foundation and the Local Government Resource Centre. This search identified one relevant document.

Overlap study

Based on the premise that the usefulness of a database can be judged by the number of relevant citations retrieved, the database source of each retrieved and included study was recorded. This enabled examination of any overlap in coverage of the databases and determination of the number of unique studies found in each database. Of the ninety-six studies included in the systematic review, eighty-six were retrieved from databases that had no restrictions on the undertaking of bibliometric analysis: MEDLINE, CINAHL, PsycLIT, Cochrane Library, Sociofile, and Social Science Citation Index. These databases formed the focus for the overlap study.

The record and database source of each retrieved study was downloaded into Idealist text retrieval software. The package was used to sort the studies into alphabetical order to enable easy identification of studies originating from more than one database. Duplicate studies were removed while retaining the information on the database source. Each study was allocated a number and loaded into the Excel software package. Excel was used to perform a frequency analysis of the number of studies retrieved from each database and the numbers of unique papers retrieved from each database (that is, a paper only retrieved by that database). Excel was also used to express the overlap between pairs of databases as a percentage of number of papers included in the review (N = 96).

RESULTS

The total number of studies included in the review was ninety-six, eighty-six of which were retrieved from the databases in question. The total number of studies retrieved from each database varied considerably (Table 1). Of all the papers included in the review, PsycLIT retrieved the highest percentage of papers (44%) and CINAHL and Sociofile the lowest (7%). In terms of unique papers retrieved per database (as a percentage of those included in the review), the Cochrane Library (17%), PsycLIT (13%), and Social Science Citation Index (13%) were the most useful, and CINAHL and Sociofile, identifying the fewest (2%), were the least useful.

Table 1 Retrieval by database

Of the total number of papers included in the review (N = 96), 42% were found in only one database (Figure 1). That is, there was no overlap in retrieval. A few papers (3%) were found by four databases, but no papers were found by five or more databases. The largest overlap in the six databases was between PsycLIT and the Social Science Citation Index (20%), followed by PsycLIT and MEDLINE (13%) and the Cochrane Library and MEDLINE (11%) (Figure 2).

Figure 1.

Number of databases from which papers were retrieved

Figure 2.

Overlapping databases

DISCUSSION

The relatively small overlap between databases in the study indicated that there was no best database to search. Each database found sufficient unique papers to warrant inclusion in the literature search. If there were restriction to one database, PsycLIT would have been an appropriate choice, given the mental health topic area. This search would have found 44% of the relevant papers. However, choosing MEDLINE would have resulted in locating only 29% of the papers. These results were similar to those obtained when identifying randomized controlled trials in that searching one database alone failed to retrieve a large number of relevant papers [21]. If the restriction was lifted to allow two databases to be searched, the best option would have been PsycLIT and the Cochrane Library to retrieve randomized controlled trials and systematic reviews (68%). A search of PsycLIT and the Social Science Citation Index or PsycLIT and MEDLINE would have retrieved 60% of studies, once overlaps were removed. A choice of MEDLINE and the Cochrane Library, not an uncommon approach in the health care field, would however have retrieved only 37% of papers.

These findings have important implications for information professionals, clinicians, and researchers as part of evidence-based practice. For a topic area such as rehabilitation services, reliance cannot be placed on a search of MEDLINE, a general biomedical source of information and the first step for many, however sophisticated the search strategy might be. For this topic, of clear interest to primary care physicians and other members of the community mental health team, 66% of relevant studies would have been missed. Even the addition of the Cochrane Library to the search would have missed more than half of the relevant papers.

At the same time, it would be important to remember that only 89% of the studies included in the systematic review were identified from these six databases together. A check of the papers not located from these databases revealed that some were not picked up because of inadequate indexing and others because of lack of journal coverage by the databases. This reinforces the need to use other methods such as citation tracking, hand searching, identifying gray literature, and consulting with experts.

It is, of course, yet a different question to examine whether or not a focus on, for example, the forty-two studies identified from PsycLIT would have led to the same conclusion in the systematic review. While further work is planned to explore the sensitivity of the conclusions of the review to the inclusion and exclusion of different types of study design, this could be taken a step further with the application of sampling techniques to the area of information searching [22].

At present, literature searches for systematic reviews aim to undertake a census of all studies meeting the inclusion criteria. This task is very difficult to achieve, even in the context of a single study design as well known as the randomized controlled trial. The exploration of the application of sampling theory to literature searching, particularly where the number of potential or actually retrieved studies is large, is an important area for further research. The current work is suggestive here. Random sampling, together with citation tracking on the papers included in the random sample, located almost a quarter (23%) of papers. A brief manual check of the conclusions highlights that these are broadly the same as the conclusions drawn in the systematic review [23].

Care must be taken in generalizing from this single study of systematic reviewing in the social care field. However, repeating the same bibliometric analysis within the second review area (preventive services for older people in the context of bereavement) points to a similar picture. The number of unique papers retrieved from each database ranged from 17% to 54%, with PsycLIT again proving to be the most useful database (retrieving 66% of studies included in the review). MEDLINE, although not the least useful, retrieved only 16% of studies included in the review.

These findings mirror those found in the health care literature field. While these issues are well recognized by information professionals, they are much less so by researchers and little or not at all by health and social care practitioners. In medicine, the database of common choice by practitioners is MEDLINE, at least in part because of its ready accessibility. Furthermore, information professionals may be aware of the need to search multiple databases, although in practice they may not have access to such a wide range of databases as those used in this study.

In this study, MEDLINE would be an inappropriate database of first choice. It would however be more likely to be a highly appropriate choice for studies on the efficacy of less complex interventions, such as drugs or forms of surgery. At the same time, it would be interesting to note that, as is common in undertaking systematic reviews though not so widely reported, the topic area was considerably refined as the search process unfolded. At the initial stages, MEDLINE appeared to be the most useful source. Once the topic area had been refined, a number of the unique papers it retrieved were excluded from the study, resulting in PsycLIT and the Social Science Citation Index becoming more useful. Other databases may also have been useful in this study, for example EMBASE/Excerpta Medica, Social Work Abstracts, or Rehabdata. These databases were not available or were unknown to the research team at the time of this study. An area for further research would be to examine the effects of searching these databases on the results of the review.

CONCLUSION

The simple message from this study is that the searched databases have an important effect on the potential conclusions of a systematic review (or any other type of research). Researchers undertaking systematic reviews and practitioners using their results need to be aware of the potential implications of a search of one, rather than another, or multiple databases. Researchers have a duty to indicate which databases have been searched and why others have not and to publish the exact search strategy employed. Careful thought must be given to the most appropriate database to search to prevent missing vital information and possibly influencing the results of the review. Further research and awareness by the research community is needed in this area. Librarians and information specialists can help by actively promoting this message.

Compiling and promoting information on these issues would be valuable to users undertaking their own searches and those using systematic reviews. This is an area where information professionals can take an active part. They can also encourage those undertaking systematic reviews to look closely at the studies retrieved from each database and publish their findings. This would provide information for a much greater range of topic areas. Searching collaboratively with practitioners is another important area for librarians and information professionals. Librarians responsible for selecting and purchasing database resources may also be able to make use of this information in making more informed choices.

The message of this paper is of little comfort to busy clinicians trying to implement evidence-based practice in areas where systematic reviews do not exist. Faced with limited time and access to sources, proposing a simple way forward is difficult. Working collaboratively with librarians and other information professionals to help with searching or searching recommended sources may be useful. It may also be helpful to obtain training in literature searching in order to ensure that any searches undertaken are efficient and make the best use of the limited time available. In the final analysis, it must be recognized that the methods of systematic reviewing are still being refined. This study has pointed to a further area of challenge to the information, research, and clinical communities.

APPENDIX A

Inclusion and exclusion criteria

The following criteria were developed as a result of a scoping exercise, undertaken as part of the systematic review of research literature into rehabilitative services for severe mental illness.

Inclusion criteria:

• Intervention: any which could be regarded as rehabilitative (loosely enabling independent living), emphasis on disabilty. Studies were included if the intervention involved a therapy or group that was nonmedical.

• User group: adults with severe mental illness (schizophrenia or affective psychoses), including those who have an active cooccurring, substance-use disorder.

• Setting: living in the community but could be attending a day center or a day hospital for rehabilitative services or multidisciplinary work undertaken by a team of one or more persons.

• Cultural: United Kingdom and studies from other countries where (1) the client group, (2) the nature of the intervention, and (3) the policy implications can be judged to be similar to those in the United Kingdom.

• Study literature: if the study was undertaken during or after 1988. Electronic databases searched were: PsycLIT, MEDLINE, Social Science Citation Index, CINAHL, Cochrane Library, Caredata, and Sociofile.

• Study types: qualitative and quantitative research designs; for the latter, at least a pre-post design, with or without a comparison group.

• Outcome measures: any.

Exclusion criteria

• Intervention: any which were medical treatments, such as drugs and obvious surgical interventions or crisis care interventions, based primarily in a secondary care setting.

• User group: where the focus of the study was on forensic clients, if clients were not diagnosed with a severe mental illness, or where the focus of the study lay on long-stay hospital patients who had been discharged into the community.

• Setting: if the intervention was undertaken in an inpatient setting.

• Cultural: if the study was undertaken in a country where cultural characteristics were dissimilar.

• Nature of paper: if the document was a commentary, book review, or abstract.

• Content: if the main emphasis of the evaluation was on the costs of the intervention or how changes in local payment systems affected outcomes.

APPENDIX B

Search strategies

Social Science Citation Index:

1. day cent*

2. day care

3. day hosp*

4. clubhous*

5. vocation*

6. meaningful living

7. sheltered work*

8. sheltered employment

9. skills train*

10. assertive

11. team

12. multidis*

13. multi-dis*

14. rehabilit*

15. or/1–14

16. schizophren*

17. severe mental illness*

18. affective psychos*

19. manic depress*

20. bipolar disord*

21. psychotic

22. community mental health

23. or 16–22

24. 15 and 23

25. child

26. adolescen*

27. youth*

28. elderly

29. lithium

30. clozapine

31. medication

32. psychotropic

33. risperidone

34. haleroperidol

35. antipsychotic*

36. or 25–35

37. 24 not 36

Cochrane Library:

1. schizophren*

2. severe mental illness

3. affective psychos*

4. manic depression

5. mania

6. bipolar disorder

7. depressive psychos*

8. community mental health services/

9. emergency services psychiatric/

10. social work psychiatric/

11. schizophrenia/

12. exp affective disorders psychotic/

13. or 1–12

14. child*

15. aged/

16. 14 or 15

17. 13 not 16

18. Rehabilitat*

19. day cent*

20. day hospital*

21. clubhous*

22. employment

23. vocation*

24. meaningful living*

25. assertive

26. team*

27. multidis*

28. multi-dis*

29. day care/

30. exp rehabilitation vocational/

31. exp patient care team/

32. exp rehabilitation/

33. or 18–32

34. 17 and 33

35. antidepressive agents/

36. antipsychotic agents/

37. clozapine

38. haloperidol

39. olanzapine

40. nadolol

41. pimozide

42. drug therapy/

43. or 35–42

44. 34 not 43

CINAHL:

1. exp interviews/

2. exp analytic studies (epidemiology) (non mesh)/

3. health care surveys/

4. questionnaires/

5. n=1.tw.

6. n-1.tw.

7. strucured interview$.tw.

8. unstructured interview$.tw.

9. qualitative.tw.

10. exp study design/

11. interview$.tw.

12. survey$.tw.

13. questionnaire$.tw.

14. (observation adj stud$).tw.

15. (descriptive adj stud$).tw.

16. (quasi adj experimental$).tw.

17. (clinical adj trial$).tw.

18. placebo$.tw.

19. or/1–18

20. schizophren$.tw.

21. (severe adj mental adj illness$).tw.

22. affective psychos$.tw.

23. (manic adj depression).tw.

24. (bipolar adj disorder$).tw.

25. exp community mental health services/

26. exp emergency services, psychiatric/

27. social work, psychiatric/

28. exp affective disorders, psychotic/

29. exp schizophrenia/

30. psychiatric nursing/

31. community mental health cent$.tw.

32. or/20–32

33. child$.tw.

34. aged/

35. 33 or 34

36. 32 not 35

37. (rehabilitation adj5 mental$).tw.

38. (psychiatric adj rehab$).tw.

39. (day adj cent$).tw.

40. (day adj hosp$).tw.

41. clubhous$.tw.

42. employment$.tw.

43. vocation$.tw.

44. (meaningful adj living).tw.

45. assertive.tw.

46. team$.tw.

47. multidis$.tw.

48. multi-dis$.tw.

49. exp day care/

50. rehabilitation/

51. rehabilitation, psychosocial/

52. rehabilitation, vocational/

53. rehabilitation, centers/

54. rehabilitation, nursing/

55. research, rehabilitation/

56. multidisciplinary care team/

57. or 37–56

58. 19 and 35 and 57

59. exp antidepressive agents/

60. exp antipsychotic agents/

61. exp drug therapy/

62. 59 or 60 or 61

63. 58 not 62

PsycLIT:

1. schizophren*

2. severe mental illness*

3. affective psych*

4. manic depression

5. bipolar disorder

6. community mental health team

7. or/1–6

8. child*

9. adolescence*

10. youth*

11. elder*

12. or/8–11

13. 7 not 12

14. day cent*

15. day care

16. day hosp*

17. clubhous*

18. vocation*

19. meaningful living

20. sheltered work

21. skills train*

22. assertive

23. team*

24. multidis*

25. multi-dis*

26. rehabil*

27. employment

28. or 14–27

29. 13 and 28

30. aged in DE

31. 29 not 30

Sociofile:

1. Schizophrenia in DE

2. Community Mental Health in DE

3. Psychosis in DE

4. Depression (Psychology) in DE

5. schizophren*

6. severe mental illness*

7. manic depress*

8. affective psychos*

9. bipolar disorder

10. or 1–10

11. child*

12. explode elderly

13. explode aged

14. explode youth

15. adolescent*

16. or 11–15

17. 10 not 16

18. explode rehabilitation

19. teams in DE

20. explode interdisciplinary

21. interdisciplinary approach in DE

22. rehabilit*

23. day cent*

24. day hospital*

25. clubhouse*

26. meaningful living

27. assertive

28. team*

29. multidis*

30. multi-dis*

31. or 18–31

32. 17 and 31

MEDLINE:

1. schizophren$.tw.

2. (severe adj mental adj illness).tw.)

3. affective psychos$.tw.

4. (manic adj depression).tw.

5. (bipolar adj disorder$).tw.

6. exp community mental health services/

7. exp emergency services, psychiatric/

8. social work, psychiatric/

9. exp affective disorders, psychotic/

10. exp schizophrenia/

11. or 1–10

12. rehabilitat$.tw.

13. (day adj hospital$).tw.

14. clubhous$.tw.

15. employment.tw.

16. vocation$.tw.

17. (meaningful adj living).tw.

18. assertive.tw.

19. team$.tw.

20. multidis$.tw.

21. multi-dis$.tw.

22. exp day care/

23. exp rehabilitation/

24. exp patient care team/

25. (severe adj mental illness).tw.

26. or 12–26

27. 11 and 26

28. child$.tw.

29. aged/

30. affective disorder.mp.

31. exp anxiety disorders/

32. exp antidepressive agents/

33. exp antipsychotic agents/

34. exp drug therapy/

35. or 28–34

36. 27 not 35

37. randomized controlled trial.pt.

38. randomized controlled trials/ 37–38 Cochrane filter for RCTs

39. 37 or 38 on MEDLINE

40. 36 and 39

41. (systematic adj review$).tw.

42. (data adj synthesis).tw.

43. (published adj studies).ab.

44. (data adj extraction).ab.

45. meta-analysis/

46. meta-analysis.ti.

47. comment.pt.

48. letter.pt.

49. editorial.pt. 41–55 NHS CRD Filter for

50. animal/ identifying systematic reviews on

51. human/ MEDLINE

52. 50 not (50 and 51)

53. 36 not (47 or 48 or 49 or 52)

54. or 41–46

55. 53 and 54

56. exp interviews/

57. health care surveys/

58. questionnaires/

59. n=1.tw.

60. n-1.tw. 56–66 stages to identify qualitative

61. structured interview$.tw. studies

62. unstructured interview$.tw.

63. qualitative$.tw.

64. nursing methodology research/

65. or 56–64

66. 36 and 65

Note: Searches were displayed and downloaded at each stage marked in bold.

Key

OVID software:

• $ = truncation symbol

• tw = searches title, abstract, and subject heading fields

• / = MESH Heading

• exp = explode MESH heading

• pt = publication type field

• ab = abstract field

SilverPlatter, BIDs, and Update software:

• * = truncation symbol

• explode = explode subject heading

• DE=Descriptor field