Improving instruction in medical libraries: the research agenda

As a doctoral student, I was taught that to gain an unbiased understanding of an issue, it was important to question common assumptions. My doctorate is in child psychology, so I was taught to ask questions such as “Why do we assume that it is good for children to live with their parents?” or “Why do we think that intact families are preferable to broken homes?” Examining such assumptions led to new research questions and, one hoped, to new knowledge.

Thus, as I thought about the Journal of the Medical Library Association's (JMLA's) special issue on instruction, I asked myself, “Why do librarians instruct users anyways?” As Michel Atlas points out in her comment and opinion piece in this issue, the stereotypical image of a librarian rarely includes instruction [1]. In fact, many of my friends were amazed to learn that, as a librarian, I taught courses for students and faculty. Nevertheless, instruction has a long history in librarianship. For example, I was interested to learn that the history of library instruction in the United States goes back at least to Ralph Waldo Emerson, who called for a “professor of books” in 1870, when he wrote:

Meantime, the colleges, whilst they provide us with libraries, furnish no professor of books; and, I think, no chair is so much wanted. In a library we are surrounded by many hundreds of dear friends, but they are imprisoned by an enchanter in these paper and leathern boxes;…your chance of hitting on the right one is to be computed by the arithmetical rule of Permutation and Combination,—not a choice out of three caskets, but out of half a million caskets, all alike. [2]

Sounds familiar, doesn't it? Even then, when the volume of published information was so much smaller, Emerson recognized that users needed instruction to help them find the information that they required.

In his history of library instruction, Stephen C. Weiss identifies the following changes in education in the 1880s and 1890s as critical to the growth of instruction by librarians:

1. The adoption of original research as a necessary function of academia;

2. The introduction of the seminar method of instruction featuring student presentations;

3. The birth of new curricula in the social sciences and in their German counterparts; and

4. The creation of doctoral programs designed to make graduate-level research the capstone of higher learning in American universities. [2]

We should note the emphasis on research here. Among librarians, it seems that those in the health sciences were a little “late to the party” when it came to instruction, perhaps because medical schools at the turn of the last century put more emphasis on the acquisition of clinical skills than on the need for independent research. In this environment, library research may have been seen as less important than the clinical experience. Certainly, when Estelle Brodman reviewed the roles of librarians in the first half of the 20th century, she made no mention of instruction, focusing instead on the role of librarians as “purveyors” of medical information [3]. That characterization of the role of the medical librarian seems to have changed in recent years, however. Whether because of the increase in research activities in US medical schools in the second half of the 20th century, the explosive growth in biomedical publications, the adoption of problem-based learning (similar to the “seminar method” mentioned above) in medical education in the United States, or perhaps because of all three, the need to educate library users has become a paramount concern for health sciences librarians. A search for “instruction and librar?” in PubMed yields a mere 10 articles from 1929 to 1969, 13 from 1970 to 1979, 25 from 1980 to 1989, 78 from 1990 to 1999, and an amazing 204 published between 2000 and May 2012. Instruction is now such a focus of our profession that Gerald (Jerry) Perry, AHIP, Medical Library Association (MLA) president in 2011/12, suggested that MLA establish an MLA Academy of Teaching Excellence devoted to improving our instructional skills. Brodman said that rather than ask, “How much should I do for our readers?,” librarians need to identify, “What needs to be done?” [3]. Clearly, we have taken this advice to heart and identified instruction as something that our clientele needs.

If the purpose of library instruction then is to provide users with the tools they need to get the information they require, and health sciences librarians have agreed that instruction needs to be done, just how well are we doing? This issue of the JMLA suggests that evidence of our success is decidedly mixed. Melissa L. Just, AHIP, in her review of the literature on the effectiveness of literature searching skills instruction for medical students or residents, concludes that:

Although the findings of most of the studies indicate that the current instructional methods are effective, the study designs are generally weak, there is little evidence that learning persists over time, and few validated methods of skill measurement have been developed. [4]

Similarly, Josephine L. Dorsch, AHIP, and Jerry Perry, after reviewing the literature on librarian involvement in teaching evidence-based medicine (EBM), conclude that:

The nature of the evidence regarding teaching programs in EBM generally comprises descriptive narratives of how a particular course was developed or curricular change implemented; evaluative studies that measure success of courses or programs based on student satisfaction; and studies that rely on self-reports of competency or one-time course evaluations, with little or no follow up. However, some higher level studies show that librarian-led programs or teaching programs in which librarians are collaborators contribute to building EBM competencies. [5]

Katherine Schilling and Rachel Applegate, reviewing the literature of evaluation of instruction, highlight “the significant disparities between measures of skills, knowledge, attitudes, and information usage behaviors” and point out that “While affective measures [for example, measures of attendee satisfaction] are more common, they are not likely to provide meaningful evidence in terms of students' skills, course grades, or learning outcomes” [6].

In other words, while there are reports in the literature of successful instructional programs, there is a dearth of evidence to help us understand which techniques are most effective and even less evidence indicating which techniques will have a lasting impact on our clientele's information-seeking skills.

Nevertheless, the current issue is full of interesting reports of innovative instructional programs that have received positive reviews from attendees. In some cases, these reports also provide evidence, albeit fairly weak, that these innovative classes successfully achieved their stated learning objectives. Three themes emerge from these reports:

1. The importance of partnerships between librarians and faculty may be key to successful instructional efforts. In their review of instruction related to EBM, Dorsch and Perry found numerous examples of successful collaborations between librarians and faculty (or clinicians) [5]. Similarly, Jane P. Gagliardi and colleagues report on a successful course to teach EBM at Duke University that was developed and taught collaboratively by librarians and clinical faculty [7]. Mary R. Simons and colleagues report on another collaborative effort, this time between librarians and neurosurgeons, resulting in weekly case-based presentations designed to improve searching skills. Librarians and faculty worked together to design the course and grade presentations collaboratively as well [8].

2. An interactive curriculum may help to assure positive outcomes. A 2010 study of EBM courses [9] reported that greater improvement in scores on the Berlin questionnaire was associated with “active engagement in the learning process,” and students in programs using problem-based small group learning as opposed to lecture formats received the highest scores. While that study focused on general EBM instruction, librarians seem to have had some success extending the principle to teaching skills related to seeking information. Several instructional innovations in this issue included an interactive component. Gagliardi [7] used small group sessions; Donna O'Malley and Frances A. Delwiche introduced more hands-on practice [10]; interactivity was also a feature of Marci D. Brandenburg and Jean Song's instruction in network visualization [11]; and Simons [8] used case presentations.

3. Assessment can be a tool to enhance learning and retention. While few students enjoy taking tests, research has shown that assessment can be an important tool for enhancing learning and retention [12]. Librarians struggle with assessment. Testing concepts associated with information-seeking skills are not as straightforward as, for example, testing whether a student has learned the function of the liver or the signs of hepatitis. Moreover, librarians often only have an hour or two to instruct students; time for assessment in most library-related instruction is quite limited. Several articles in this issue suggest ways to overcome these obstacles. Lana V. Ivanitskaya and colleagues report on a tool to assess undergraduates' understanding of research methodology [13]; Gagliardi [7] employs the Berlin questionnaire to measure learning of third-year students; Michele R. Tennant, AHIP, and colleagues explore ways in which a multiple choice format could be used to measure information-seeking skills [14]; and Mê-Linh Lê introduces the concept of “pop quizzes,” which can easily be incorporated even in a one-hour teaching session to increase retention [15].

The presence of these three themes in our special issue may be a welcome indication that medical librarians are using what little is known with respect to the best way to provide information literacy skills to improve their classes. The literature suggests that use of student-centered, active, and collaborative learning methods; adherence to instructional design principles during planning; relevance to particular course goals and, ultimately, the overall curriculum; formation of partnerships between library, faculty, and other campus departments; support of faculty learning and development; and scalability for large numbers of students are all important for success [16]. Still, as noted above, there is little evidence that what we are doing is actually making a difference in users' ability to find the information they need. In fact, several of the papers in this issue report that learning is less than expected, while Schilling and Applegate, in their review, note that even successful reports show little evidence that skills are retained [6]. Further complicating the issue, Just notes that while her literature review uncovered multiple reports of successful efforts, these efforts had few aspects in common, consequently providing little to guide us in developing additional successful endeavors [4].

Why, with all our efforts to improve instructional offerings, do we still have so little evidence of what makes instruction effective? The answer to this question may be that we need to “drill down” further to discover what actually makes for a successful instructional session. For example, it seems that partnerships are important, but we do not know when in the process they are critical. Is it most important to partner with faculty in designing a course? Or is the presence of a faculty member in the classroom critical? Or does that faculty member actually have to teach some of the content for partnership to be a successful element of instruction in information-seeking skills? Similar questions could be asked about interactivity. If the best sessions are interactive, then how much interactivity is necessary and what kind? Do students need practice or is a lively discussion sufficient interactivity to assure a positive learning outcome? Assessment, too, needs further exploration. Would more assessment increase retention of information-seeking skills as it seems to do in the traditional classroom? Does the use of pop quizzes increase retention?

Answering these questions requires thinking about instruction from a research as well as a practical perspective. For practicing librarians, that means not just modifying course delivery and/or assessment in order to improve it, but taking the time to measure whether changes have any impact on learning outcomes or student satisfaction. Fortunately, in the last few years, a number of new tools have become available that librarians can use to measure changes in health sciences students' information-seeking skills: the Fresno test, the Berlin questionnaire, the Research Readiness Self-Assessment (RRSA) [13], and the University of Michigan's standardized tool [17]. Regular use of these tools in library instructional settings would make it far easier to understand just what innovations result in improved learning and retention. Using these tools year after year—as we redesign our courses, start new collaborations, and adopt new interactive designs—would enable us to determine which changes actually make a difference in what students learn and what they retain.

The current issue then, provides us not only with interesting reports of exciting innovations, but also with a research agenda. It suggests that further research that would explore the most successful ways to apply collaboration, interactivity, and assessment could provide the evidence we need to deliver effective instruction to our users. Equally important, the issue shows that we have the tools, in the form of standardized tests, to pursue this agenda. Now, we need to begin.