Evidence-based medicine at the intersection of research interests between academic health sciences librarians and medical educators: a review of the literature

Abstract

Objectives:

In 2008, the Association of Academic Health Sciences Libraries established an Education Research Task Force (ERTF) to plan research addressing research priorities outlined in key Association of American Medical Colleges reports. ERTF members conducted a literature review to describe the state of collaborative research at the intersection of medical education and health sciences librarianship. Analysis of initial results revealed instruction in evidence-based medicine (EBM) was a shared interest and is thus the focus of this review.

Methods:

Searches on EBM teaching programs were conducted, and results were posted to a shared online citation management service. Individual articles were assessed and assigned metadata describing subject matter, scope, and format.

Results:

Article analysis identified key themes. Most papers were descriptive narratives of curricular development. Evaluation studies were also prominent and often based on student satisfaction or self-reported competency. A smaller number of controlled studies provide evidence of impacts of librarian involvement in EBM instruction.

Conclusions:

Scholarship of EBM instruction is of common interest between medical educators and health sciences librarians. Coauthorship between the groups and distribution of literature points to a productive collaboration. An emerging literature of controlled studies measuring the impact of cross-disciplinary efforts signals continued progress in the arena of EBM instruction.

Highlights.

• The Association of Academic Health Sciences Libraries Education Research Task Force identified three key questions for a review of the literature that relates to library teaching programs in medical education.

• There is an ample literature that reveals that librarians are active collaborators in instruction, educational assessment, and medical education research.

• A literature is emerging that addresses successes and failures of specific evidence-based health care instructional initiatives, including reviews and controlled trials.

Implications.

• Gaps in the literature suggest a need for longitudinal follow-up and multicentered studies to validate the findings of the literature to date.

• Future studies should include effectiveness research to determine the impacts that evidence-based medicine instruction led by librarians, or in collaboration with medical educators, has on learning outcomes and practice.

INTRODUCTION

In late 2002, the Association of American Medical Colleges (AAMC) established the Institute for Improving Medical Education (IIME) to encourage and foster improvements in medical education. An ad hoc committee was established to direct the efforts of the institute, with the intention of conducting a comprehensive review of the state of medical education and making recommendations for reform. In 2004, the committee released its report and recommendations, Educating Doctors to Provide High Quality Medical Care: A Vision for Medical Education in the United States [1]. The report presented a series of strategies for reforming medical education and achieving an “Ideal Medical Education System.” The AAMC's Group on Educational Affairs (GEA) was invited to respond to the report with ideas for advancing its various recommendations. In 2006, the GEA issued its response in a report, Implementing the Vision: Group on Educational Affairs Responds to the IIME Dean's Committee Report: Educating Doctors to Provide High-Quality Medical Care: A Vision for Medical Education in the United States [2].

The GEA's response considered what was working well in medical education and what needed attention. Among the areas identified for improvement were increased integration across the continuum of medical education; recognition (academic and financial) of educators; and more “rational assessment” across the continuum. The GEA report identified areas for future research in parallel to the areas of concern expressed in the IIME report. The GEA also articulated the knowledge, skills, and attitudes essential for the successful transition of learners throughout the medical education continuum, among them skills in accessing information, understanding and using evidence-based medicine (EBM), self-directed learning, self-assessment, and reflection.

Among the suggested areas for future research and the acknowledged lifelong learning skills expressed in the report were many of potential interest to the biomedical information and library sciences community. Health information professionals are in fact actively engaged in the full complement of teaching programs on their respective campuses or sponsored by their institutions, with a focus on the acquisition and application of informatics and knowledge management skills.

The IIME and GEA reports identified four key agendas in order to achieve the proposed “Ideal Medical Education System”:

• promote a patient-centered approach to medical care,

• ensure that doctors are capable of providing high-quality medical care,

• improve the efficiency of educational processes, and

• improve the effectiveness of the education processes.

The Association of Academic Health Sciences Libraries (AAHSL) [3] Board of Directors embraced the initial IIME report and in 2008 established an Education Research Task Force (ERTF) of the association's Teaching/Learning Oversight Committee (T/LOC) to respond to both the IIME and GEA reports, with a charge to “plan and conduct collaborative research that addresses the research priorities outlined in the IIME and GEA Reports.” The ERTF was explicitly charged to define, in partnership with the GEA, a research agenda of mutual interest.

The ERTF membership, which included key leaders from five academic health sciences libraries and an AAHSL Board–assigned liaison and project manager, determined that a critical first step in addressing its charge was to conduct a comprehensive review of the literature to describe the state of collaborative research at the intersection of medical education and health sciences librarianship, focusing on the four agendas identified in the IIME and GEA reports.

METHODS

The ERTF subsequently identified three key questions in constructing its approach to reviewing the scholarly literature:

1. What is the nature of evidence regarding teaching programs in evidence-based medicine (EBM) in medical schools or other curricula?

2. What is the nature of evidence for improving the effectiveness of education processes, generally?

3. What is the nature of evidence regarding teaching programs in consumer health information in medical schools or other curricula?

Initial literature searches, conducted in 2008, focused on these three questions. Search strategies focused on the role of health sciences librarians in teaching and evaluating instructional programs, primarily in medical education. Furthermore, the focus should be on shared interests or collaborative initiatives of librarians and medical educators. Databases searched included: MEDLINE, CINAHL, and Academic Search Premier (EBSCO). Search results for each question were shared in CiteULike databases, where the citations could be ranked for relevancy and tagged with metadata tags describing subject matter and scope.

In assessing both the volume and nature of the literature regarding consumer and patient health, it was clear that while there is some shared interest between the medical educator and health information professional communities, the nature of the scholarship is neither as deep nor as rich in scope as that regarding EBM and lacked emphasis on educational programs. For this reason, the ERTF subsequently recalibrated its energies to focus on instruction and assessment of learning in evidence-based practice.

Creating a CiteULike group database revealed a degree of duplication and an overlapping research base of the EBM and education effectiveness literature. The analysis also showed that EBM was a predominant vein of common interest between authors from the medical education and health sciences librarianship fields of study. When this group database was further analyzed by publication type and research methodology, the articles on EBM emerged as evincing some of the most promising scholarship, so the decision was made to concentrate the review of the literature reported here on this subset dedicated to EBM instruction (hereafter referred to as the EBM Subset).

Evidence-Based Medicine (EBM) subset

The EBM search was updated in February 2012 using an iterative approach. A search was conducted in PubMed, CINAHL, Library Literature, and Google Scholar for all literature published since 1990. The “Related” feature of the PubMed database was used to identify articles that might have been missed by the original search strategy, and hand-searching of the references from selected articles was also conducted. The search strategy used for the update is found in Figure 1.

Figure 1 Search strategy for Evidence-Based Medicine (EBM) Subset

The inclusion criteria for the EBM subset were:

1. article was about instruction in EBM, including “critical appraisal” and

2. included librarian author(s) or

3. described research on collaborative efforts in EBM that included librarians (even if the authors were not librarians)

Articles about evidence-based practice in fields other than medical education were included if they met the other criteria (however, the search strategies used did not specifically include other disciplines). The set was restricted to papers written in English.

Exclusion criteria were:

1. editorials or opinion pieces

2. articles primarily about database searching without an EBM focus

3. articles about EBM without an explicit or implied instructional focus

4. articles about evidence-based librarianship without a focus on EBM instruction

Meta-data tags were developed to reflect key concepts and themes that emerged from the original searches. Medical Subject Headings (MeSH) were used to assign tags for publication type (PT) and research methodology. Any systematic reviews and meta-analyses were also tagged. PT and methodology tags were only assigned if they were listed in the MEDLINE record for the article. Therefore, not all of the articles were assigned tags for the PT and methodology categories. It should be noted that “Evidence-Based Medicine” did not become a MeSH term until 1997.

RESULTS

EBM subset

A total of sixty-eight articles (Appendix, online only) were selected for the EBM Subset after the inclusion criteria were applied. A majority of the articles appeared in the library literature (46/69 or 67%) (Table 1), with the greatest number of these (20) in the Journal of the Medical Library Association and its predecessor the Bulletin of the Medical Library Association. Medical Reference Services Quarterly was second, with 16 articles, followed by Health Information & Libraries Journal and its previous iterations (7) and the Journal of Hospital Librarianship (2). Twenty-three articles appeared in a variety of medical education journals, with the greatest number of those appearing in Academic Medicine (5). Twenty-five of the articles, based on institutional affiliations cited in the publications, were coauthored by librarians and medical educators.

Table 1.

Journal titles by frequency of articles in the Evidence-Based Medicine (EBM) Subset

Topical patterns that emerged pointed to librarian engagement in various aspects of instruction, including curriculum development, assessment, program evaluation, and educational models (Figure 2). Curriculum development (24/68) was identified as a tag for 35.3% of the articles. Twenty of the articles (29.4%) dealt with “librarian roles,” often with the term explicitly stated in the title or abstract. The “Searching” tag was assigned to 16 articles (23.5%) that concentrated on teaching searching techniques for finding good evidence to support decision making. Another prominent theme was “attitudes” toward EBM and EBM instruction. Eight of the studies looked at attitudes of a variety of populations including medical students, residents, physicians, and librarians. Eight studies were categorized in program evaluation and 6 in assessment, together totaling 20.6%.

Figure 2.

EBM Subset tag cloud*

* This cloud was created from the CiteULike EBM Subset database and shows the distribution of meta-data tags based on topics, student audiences, and study types.

Librarians were active in a variety of EBM venues and with a variety of audiences. The combined categories for medical students-clinical (6), medical-students-preclinical (5), and clerkship (6) totaled 17. Reports of teaching interns and residents numbered 15. Other audiences included nursing (3), allied health (3), dental (2), and veterinary (2) students. In addition to the traditional classroom, other teaching venues included the clinical environment (6), online learning (6), and morning report (3).

The articles showed variance in PT, with the most common being review (11). Three of these reviews were systematic reviews; however, very few studies in these reviews met the inclusion criteria of the systematic review authors. Walczak's review of studies looking at the effectiveness of programs to teach EBM teachers reported that only five studies met his criteria [4]. In Parkes's Cochrane review of studies reporting on critical appraisal instruction, only one study qualified according to the set criteria for the review [5]. The Cochrane review was updated in 2011, and its authors concluded that critical appraisal teaching interventions might result in modest gains, but that improvement to research examining the effectiveness of interventions was needed [6]. The review category was closely followed by 7 randomized controlled trials (RCTs) and 7 evaluation studies. Also represented were controlled studies, including comparative (6) and validation studies (1).

The studies also varied in research methodology. Questionnaires (11/25) accounted for 45% of the articles that had a MeSH research methodology term applied. Articles that did not report the use of a recognized research methodology were tagged as “descriptive.” Although many of the articles that were given the “descriptive” tag might have been classified as case studies, they were not identified as such either in the body of the article or by the assigned MeSH terms.

Significant conclusions were found in several of the areas of investigation, as summarized in Table 2. Of the studies of gaining EBM skills as a result of an educational intervention, results were mostly positive. Gardois (2011) found quantitative evidence of a significant difference in search performance in a clinical setting between residents assisted by a librarian and those searching the literature alone [7]. Likewise, a recent validation study by Rana (2011) showed improvement in information-seeking skills using a tool to measure success of an EBM intervention [8]. Schilling (2006) found that an electronic learning approach to teaching search skills for finding evidence-based articles can result in higher-quality results [9]. Gruppen (2005) reported that even a brief training session can have marked benefit on quality of subsequent EBM literature searching [9]. Dorsch (2004) found a statistically significant increase in medical students' self-assessment of EBM skills [11], and Frasca (1992) reported significantly higher scores on a critical appraisal test in a group of medical students who had instruction when compared to a control group [12]. Bradley (2002) found that residents improved and retained searching skills six months after an intervention [13].

Table 2.

Highlights of conclusions from studies with designated publication types

Although a number of the articles on librarian roles were descriptive or review papers, a few of the papers studied the impact of librarian involvement in EBM. Aitken (2011) found that clinical librarian involvement led to a positive effect on information retrieval and clinical decision making in a clinical environment [14]. Mulvaney (2008) reported that decision making was impacted by a library consult service [15], and Banks (2007) found that librarian-led searches at morning report were an effective means of introducing EBM [16]. On the other hand, Koufgiannakis (2005) felt the impact of librarian participation in problem-based learning groups did not merit their continued participation [17]. Another form of assessment, the Objective Structured Clinical Examinations (OSCE), was reported by Davidson (2004) and Burrows (1999) as an effective method of evaluating EBM and searching skills [18, 19].

DISCUSSION

The findings from this review suggest that there is a body of literature addressing EBM instructional initiatives. It is notable that these publications appear in both the library and medical education literature, thus reaching cross-disciplinary audiences concerned with improving education in the EBM process and lifelong learning. Furthermore, many of the publications are coauthored by librarians and medical educators, highlighting the collaboration of these two groups in EBM education and an overlap of research interests.

Topical patterns that emerged from this review show that librarians are actively engaged in curriculum development, deployment, and assessment. The nature of the evidence regarding teaching programs in EBM generally comprises descriptive narratives of how a particular course was developed or curricular change implemented; evaluative studies that measure success of courses or programs based on student satisfaction; and studies that rely on self-reports of competency or one-time course evaluations, with little or no follow up. However, some higher level studies show that librarian-led programs or teaching programs in which librarians are collaborators contribute to building EBM competencies. To be expected, competency in search skills is one of the dominant themes in this literature. This may be due not only to the domain of librarian expertise, but perhaps also because demonstration of search competency lends itself to skill measurement. In fact, four of the seven RCTs measure search performance. However, one of the RCTs reported here went beyond search skill measurement on to examine the effect librarian involvement had in influencing clinical decision making.

The various roles of librarians in EBM emerged as another dominant thread in this review of the literature. These studies, often case based, showed librarians successfully engaged in EBM activities using a great variety of teaching methods; participating in traditional teaching venues, in the clinical environment, and at the forefront of emerging online formats; and reaching audiences across the medical education continuum. With these studies as a foundation, librarians and their collaborators in EBM education can expand their efforts guided by the experiences and lessons learned presented in this case-based literature.

Table 2 highlights conclusions of some of the publications in this review that provide evidence that supports the premise that EBM instruction contributes to competencies in search skills, critical appraisal of the literature, clinical decision making, and ultimately lifelong learning. The studies collected in Table 2 represent RCTs, controlled trials (evaluation, comparative, and validation studies) and systematic reviews. These studies and the case-based studies in this review form a body of research that will only benefit from future longitudinal and multicenter studies that replicate and validate the evidence to date and consider the long-term impacts and outcomes that collaborative teaching is having or could have.

CONCLUSIONS

This review uncovered a scholarship of common interest between peers in medical education and health sciences librarianship in the area of EBM instruction. Coauthorship between the two groups and the distribution of the literature across library and medical education journals points to a productive collaboration. A portion of this literature is case based and describes the collaboration of librarians and medical educators in the development, delivery, and assessment of EBM instruction across the continuum of medical education. An emerging literature of controlled studies that measures the impact of cross-disciplinary EBM instruction points to continued collaboration in the classroom and the promise of a robust research agenda.