From both sides now: librarians' experiences at the Rocky Mountain Evidence-Based Health Care Workshop

Abstract

The Colorado Health Outcomes (COHO) Department of the School of Medicine at the University of Colorado Health Sciences Center (UCHSC) coordinates the Rocky Mountain Evidence-Based Health Care (EBHC) Workshop, which has been held annually since 1999. The goals of the workshop include helping participants—physicians, pharmacists, health care policy makers, journalists and librarians—learn and apply skills for critically appraising medical research literature and for effective use of evidence-based information resources. Participants are encouraged to share ideas and to plan local services and instruction for those working in clinical settings. Each year, librarians from UCHSC Denison Memorial Library participate as faculty by teaching searching skills (PubMed, Cochrane Library, ACP Journal Club, etc.), providing support to small groups, and staffing two computer labs. In 2002, Denison Library received a National Network of Libraries of Medicine (NN/LM) MidContinental Region Impact Award to fund the attendance of three health sciences librarians from the MidContinental Region, an academic education librarian, a clinical medical librarian, and a department librarian. In this paper, the participating librarians share the lessons they learned about how health care practitioners approach evidence-based practice. The participating librarians also share how they incorporated these lessons into their support of evidence-based practice related to teaching about evidence-based resources, assisting health care practitioners with developing answerable questions, enhancing the clinician-librarian partnership, and assisting practitioners in selecting evidence-based resources for quick answers to clinical questions.

HISTORY

The Colorado Health Outcomes (COHO) Department of the School of Medicine at the University of Colorado Health Sciences Center (UCHSC) coordinates the Rocky Mountain Evidence-Based Health Care (EBHC) Workshop, which has been held annually since 1999. The workshop director, Andy Oxman, director of the Health Services Research Unit at the National Institute of Public Health in Oslo, Norway, and director of the Norwegian Branch of the Nordic Cochrane Centre, has conducted workshops like this for about twenty years primarily for clinicians, and more recently for policy makers, consumers, and journalists. Well-known and highly respected authors of evidence-based health care articles and textbooks, people such as Lisa Bero, Kay Dickersin, Martha Gerrity, Lee Greenfield, Alejandro Jadad, Scott Richardson, John Steiner, and Peter Tugwell, serve as workshop faculty.

The EBHC Workshop consists of one large group session each morning, followed by small-group sessions, individual study time, and ad hoc meetings. Workshop organizers emphasize self-directed, small-group, problem-based learning, with a minimum of lecturing. The learning is applied, focusing on actual problems that the participants identify in their individual groups. Participants learn how to effectively search evidence-based databases and Websites and to critically appraise medical research literature. They share ideas about how to apply evidence-based practice and plan local services and instruction at their home institutions. An internationally known expert, a COHO faculty member, and a medical librarian facilitate each group. In total, there are eight small groups, including one group of policy makers, one group of health care journalists, and six groups of health care professionals.

DENISON MEMORIAL LIBRARY'S INVOLVEMENT

Librarians from Denison Memorial Library participate as faculty. A librarian is assigned to each small group, providing individual and group instruction in searching for the evidence in the medical literature while acting as a facilitator in the small-group learning process. Rather than simply providing the requested information, the librarians encourage participants to learn searching skills to support lifelong learning. When not teaching searching skills (PubMed, Cochrane Library, ACP Journal Club, etc.) or providing support to the small groups, the librarians staff two computer labs. The computer labs provide an opportunity for participants to search on their own, to work one-on-one with a librarian, and to explore the many resources made available during the course of the workshop. The EBHC Workshop computer lab Web page* demonstrates the scope and quantity of resources that the librarians share with the participants.

The librarian tutors participate in the EBHC Workshop as fully acknowledged members of the faculty/tutor team. Judy Baxter, the EBHC Workshop assistant director, asked Denison Library to help with the workshop in 1999 because the workshop could not afford to recruit librarians from McMaster University or other Cochrane Centers. After the first year with two librarian tutors, the other faculty and participants encouraged COHO to enlist more librarians. The goal of one librarian tutor per group was achieved for the 2002 workshop through a generous grant from the Agency for Healthcare Research and Quality (AHRQ).† AHRQ had awarded the workshop grants in 1999 and 2000, but the 2002 grant was the first to provide funding for all eight librarians. Margaret Bandy of Exempla Saint Joseph's Hospital was invited to participate with the seven Denison faculty librarians: Martha Burroughs, Liz D'Antonio-Gan, Lynne Fox, Marla Graber, Sandi Parker, Lisa Traditi, and Stephanie Weldon. AHRQ funding continued for the 2003 workshop; the librarian team included Martha Burroughs, Lynne Fox, Sandi Parker, and Lisa Traditi from Denison Library; Margaret Bandy and Joyce Condon of Exempla Saint Joseph's Hospital; and E. Diane Johnson of J. Otto Lottes Health Sciences Library at the University of Missouri–Columbia.

The library's initial reason for participating was to support a university-sponsored workshop and to be seen as a collaborative partner of the faculty involved in planning this workshop. Many other benefits were immediately realized; these continue to grow as the librarians' participation increases. Benefits and outcomes include the following:

• strengthened relationships with UCHSC and Denver-area clinicians and researchers

• requests for clinical librarian services

• increased requests for special classes on searching for the evidence by clerkship directors, attending physicians, and residency directors

• enhanced professional reputation of librarians in the UCHSC and Denver community

One of the major outcomes is that Denison librarians are treated as professional colleagues by the clinical and research faculty. The expertise of librarians is viewed as a vital part of patient care and education.

Current discussions in the health sciences library community have centered on the librarian as “informationist” and the value this person could provide in supporting health care professionals as they care for their patients. Written and verbal evaluations submitted at the end of the EBHC Workshop demonstrate that the librarians are perceived as providing substantial value to the clinical, policy-making, and journalist participants. Many of the responsibilities taken on by the Denison librarians might be considered services that an informationist would provide. In addition to giving online-resource instruction, the librarians respond to information requests; search for and package information specific to group needs; direct participants to quality-filtered databases, full-text resources, and Websites; participate in and lead technology discussions; and provide the technical expertise for operating the computer lab. The Denison librarians lead by example, and it is hoped that when workshop participants return to their home institutions that they return determined to fully utilize the skills and services of their local librarians.

LIBRARIANS EXPERIENCE EVIDENCE-BASED HEALTH CARE (EBHC) WITH IMPACT AWARD

Descriptions of evidence-based health care abound, with many articles citing Sackett [1, 2] and other internationally recognized experts, including many of the tutors of the Rocky Mountain EBHC Workshop. The need for librarians to be part of the health care team is also well documented in the literature [3]. Scherrer and Dorsch [4] state:

EBM extends the librarians' role beyond identification of the literature to involvement in practicing and teaching quality filtering and critical appraisal of the literature. These activities require librarians to acquire new knowledge and develop new skills.

This project was an attempt to provide a forum for librarians to acquire that knowledge, while improving their ability to build constructive relationships with all members of the health care team.

Each year as the workshop concluded, the library faculty encouraged the participants and faculty to consult their own local medical librarians. The Colorado library faculty also thought it would be beneficial if more medical librarians could attend the workshop as participants in order to experience the EBM process from the health care provider's perspective and improve their skills. Applying for an Impact Award was one way to encourage librarian involvement in the evidence-based process and provide the experience to others. By including health sciences librarians from the MidContinental Region (MCR) of NN/LM in this workshop, Colorado librarians hoped to encourage the same benefits and outcomes to the health sciences librarians in the MCR region.

An Impact Award proposal was written and an award received to fund three health sciences librarians from the MidContinental Region. Invitations to apply for the Impact Award‡ were sent via email and U.S. mail to all health sciences libraries in the region. From the initial pool of nine applicants, three librarians were selected: an academic education librarian, a clinical medical librarian, and a department librarian. Their stories are below.

ACADEMIC LIBRARIAN'S PERSPECTIVE

As the education services librarian at the Spencer S. Eccles Health Sciences Library, I work with the School of Medicine and the Colleges of Health, Nursing, and Pharmacy to provide instruction to improve the evidence-based information-seeking skills of faculty and their students. Our library's Education Team works cooperatively with academic health sciences faculty of all disciplines to develop and present classes either on-request or integrated into their curriculum. Many of these classes are structured to teach attendees the skills they need to search for the best literature in their area of interest or expertise. In our information-rich world, these skills are important for teaching students how to become proficient lifelong learners.

Goals in applying for the EBHC Impact Award

My primary reason for wanting to attend the EBHC Workshop was to gain additional skills and knowledge to increase my effectiveness as an instructor. Improving my skill level would potentially improve the quality of instruction to my students and allow me to promote evidence-based competencies to health sciences faculty as well as to colleagues in academic, hospital, and public libraries. Improving my skill and knowledgebase would help me to partner effectively with health sciences faculty and to encourage them to incorporate evidence-based competencies into their curriculum.

Outcomes

The structure of the workshop itself was a model for how to teach evidence-based skills. Teaching formats varied and included didactic, small-group, and hands-on sessions, with scheduled independent study time and a few participant activated discussions (PADs) thrown in for good measure. At the initial small-group session, students were expected to set rules of behavior, establish their learning goals and objectives, schedule computer time, and select the materials that would support their learning. Students were asked to come to small-group sessions prepared to talk about the journal articles they had selected related to diagnosis, therapy, prognosis, etiology, and guidelines.

As an example, one of my sessions reviewed a clinical practice guideline for the evaluation of blunt abdominal trauma. At first, the group expressed mild support for the work, but after more meaningful discussion it was determined that the guideline was not very well written and did not provide the evidence for their recommendations. Alejandro Jadad, our group leader, provided an appraisal instrument§ that helped sort out issues with the guideline. After additional discussion, the group determined that the “who, what, when, why, how and where” were inadequately defined and referenced. It was interesting to watch the group dynamics as we worked to reevaluate the guideline based on the appraisal instrument. The process of using the power of the group to work through a problem and draw conclusions was effective and a valuable lesson appropriate to my own instructional setting.

The small-group leaders ensured that the discussion was on target, but they did not dominate the conversation. They allowed the group time to explore and learn from one another and to set their own learning pace. Every group member participated in the process of evaluating the journal articles for the evidence, bringing their own skills, knowledge, and experience to the discussion. I was impressed with how seriously the group leaders took their responsibility to ensure that workshop participants were provided with a supportive learning environment.

Participating in and observing the various workshop sessions provided meaningful techniques and tools to improve my own teaching methods. As a result of the knowledge and skills gained at the EBHC Workshop, I have become more comfortable evaluating research articles and determining whether the research methods were appropriate for the question or problem being addressed. In addition, I am now comfortable with concepts such as number to treat, confidence intervals, absolute risk, relative risk, and likelihood ratio. Acquiring this knowledge has helped me become a more informed instructor.

During the EBHC Workshop, I was introduced to Web-based critical appraisal tools such as CASP** and DISCERN.†† Jadad demonstrated the use of critically appraised topics (CATs), which are short summaries of evidence for a specific clinical question. These are tools I can introduce to health sciences students as they explore the clinical evidence for their case questions.

Having observed the workshop facilitators in action, I understand the value of encouraging students to learn from one another. Providing a collegial learning environment has improved student participation in the classroom and instills a commitment to information literacy and lifelong learning. Another benefit of attending the workshop is being able to share knowledge and skills learned with my library colleagues.

THE CLINICAL LIBRARIAN'S PERSPECTIVE

As a clinical medical librarian at the University of Missouri−Kansas City (UMKC), I am part of the health care team. I visit internal medicine patients with the faculty, students, and residents several days a week and support clinicians and future-clinicians “in the trenches.” My job is to assist the patient care team in locating the best available evidence for treatment decisions. Working with the health care team on rounds provides “teachable moments” in which I seize opportunities to demonstrate the value of information retrieval skills.

Goals in applying for the EBHC Impact Award

The librarians' role in providing evidence-based resources is clear, but I needed techniques for promoting the evidence-based approach to the health care teams on a daily basis, providing strategies for the incorporation of EBHC into busy health care practice. I needed strategies that I could employ to make finding the evidence simple and straightforward for clinicians.

Outcomes

The EBHC Workshop enhanced my support of clinicians by providing me with skills to better help them develop answerable questions. At the EBHC Workshop, we learned how to develop questions using the patient-intervention-comparison-outcome (PICO) format, which I now promote to my patient care teams. The PICO format helps clinicians clarify their questions and make sense of complex clinical scenarios to more easily search for an evidence-based answer. I have modified my method of communication with my clients, incorporating the PICO format into the reference interview to discern my client's true information need.

Librarians must go beyond MEDLINE to help clinicians practice evidence-based medicine. Reading and evaluating the primary literature requires a substantial time investment. The EBHC Workshop provided me with a new appreciation for several resources that allow clinicians to quickly and efficiently incorporate EBM into daily practice, such as the Cochrane Database of Systematic Reviews and the ACP Journal Club. The Denison librarians introduced me to several new evidence-based resources that I have passed along to my students and faculty such as SumSearch‡‡ and Bandolier Evidence-Based Health Care.§§ My EBHC Workshop small group ranked EBM resources based on ease of use and breath and depth of resource.

The teaching style of the instructors and small-group leaders at the EBHC Workshop provided an excellent model for small-group learning and influenced how I teach my small classes. One of the goals of the EBHC Workshop is to help attendees become better teachers of EBM. After observing the effectiveness of using an interactive small-group method with the teacher as facilitator, my classes are now based on student-led discussion. The classes generate more ideas, become louder more quickly, and create more enthusiasm in the learners. This method of teaching certainly requires more class time and, just like real life, the librarian does not always find the answer.

Many of the health care providers at the EBHC Workshop were enthusiastic about the impact of handheld computers on practicing EBM. I had the opportunity to co-lead a participant-activated discussion on EBM resources for personal digital assistants (PDAs). This session gave me the opportunity to learn about several useful EBM tools for PDAs, such as the EBM Calculator*** and EBM Tables††† from the Centres for Evidence-Based Medicine.

DEPARTMENTAL LIBRARIAN'S PERSPECTIVE

The A. Sherwood Baker Information Management and Resource Center is an information center for the Department of Family and Community Medicine at the University of Missouri–Columbia. The center's staff provides database search support and information services for approximately sixty-five clinical and research faculty members and thirty-six residents. The faculty's teaching, clinical, and research activities all emphasize to some degree the concepts of evidence-based medicine, critical appraisal of the literature, and putting evidence into practice. I participate in many of these activities as department librarian, and I am increasingly aware of the natural migration to resources that provide topical best evidence, the need to understand what the “best” evidence is, and the need for physicians to be able to frame a question in its most answerable form.

Goals in applying for the EBHC Impact Award

My desire to learn more about evidence-based medicine and to provide access to evidence-based resources within our department has been spurred by the growing number of requests from faculty, fellows, and residents for evidence-based information. I attend rounds with the family practice inpatient team on a weekly basis, and I find that many of the team members make requests for evidence-based literature as opposed to requests for background information or review articles.

I also participate as a member of the Family Practice Inquiries Network (FPIN), a national, not-for-profit consortium of primary care health professionals whose mission is to provide evidence-based answers to physicians' questions in sixty seconds at the point of care. FPIN member librarians are responsible for conducting searches in support of clinical inquiries, which are published in the Journal of Family Practice and on several Websites.‡‡‡ The standard search protocol, developed by a team of member librarians, covers prescribed databases and uses specific evidence-based hedges to produce search results for authors involved in writing “Clinical Inquiries.” As leader of the FPIN librarian team, I am also responsible for developing an appropriate evidence-based training module for new FPIN librarian members.

My goals in applying for the EBHC Impact Award were twofold: to identify and deliver timely, evidence-based clinical information to individuals in my department as well as to the family practice inpatient team and to enhance my understanding of basic epidemiologic concepts so that I am able to search and identify relevant evidence-based literature for the individual FPIN authors with whom I work.

Outcomes

The EBHC Workshop provided me the opportunity to interact with a diverse group of health care professionals and journalists, all of whom brought a unique perspective to the pursuit and use of evidence-based information. The small-group sessions gave me valuable experience in learning to phrase questions using the PICO model, and in getting hands-on experience in critical appraisal of articles.

As I observed group members working to build their questions, it occurred to me that physicians often struggle to formulate questions for which they are seeking evidence-based literature; more often than not, they ask a “background question,” that is, a question that a librarian is likely to interpret as a request for a review article. I concluded that librarians should take an active role in helping physicians better define their questions, thereby making it possible to conduct a more effective search for the evidence-based answer.

The skill-building presession was a particularly helpful introduction to epidemiology, and I plan to develop a training module that will introduce searchers to the basic epidemiologic concepts. Also, the FPIN librarians are required to use a designated evidence-based search hedge as part of the FPIN search protocol. While the current recommended hedge lends itself appropriately to therapeutics, it does not necessarily generate effective search results for FPIN questions focused toward diagnosis, prognosis, or harm; consequently, the FPIN librarian team has been given the charge of developing additional evidence-based search strategies for the above-mentioned question types. The critical appraisal experience I received in the small-group sessions has enhanced my understanding of such concepts as sensitivity and specificity, predictive value, confidence intervals, and so forth, and I will be able to incorporate these concepts into effective search hedges for diagnostic and prognostic “Clinical Inquiries” questions.

The EBHC Workshop also provided practical and helpful ad hoc discussion sessions, one of which focused upon clinical applications for the PDA. In the coming year I would like to develop a departmental seminar on evidence-based PDA applications. I also see the potential for a seminar based upon building end-user search skills for evidence-based literature.

CONCLUSIONS

For the three recipients of the Impact Award, the EBHC Workshop was a valuable, librarian-friendly experience. The participation and support provided by the Denison librarians was exceptional and a positive role model for the awardees. One of the most important messages was that librarians have a vital role in delivering evidence-based information and resources, which ultimately affects physician training and patient care. The three EBHC Workshop awardees and library faculty encourage health sciences librarians to proactively seek opportunities to enhance their evidence-based information skills. The knowledge gained ensures that they can confidently work with the health care provider team to deliver reliable, pertinent, and evidence-based information.