Abstract
Purpose: The authors studied the effectiveness of a train-the-trainer collaboration model between librarians and medical faculty to instruct librarians and health professionals in teaching evidence-based medicine (EBM) principles.
Methods: A telephone survey was administered to graduates of an EBM course who agreed to participate in the study. They were asked if and how they taught EBM on returning to their institutions, if they felt competent to critically appraise an article, if their skill in searching PubMed improved, and if they collaborated with others in teaching EBM.
Results: Most respondents were librarians. The class was successful in that most taught EBM on return to their home institutions. Most initiated collaboration with health professionals. The goals of improving PubMed searching and achieving statistical competency had less success.
Conclusion: This model is effective in preparing librarians to teach EBM. Modeling and encouraging collaboration between librarians and health professionals were successful techniques. Librarians would like more instruction in statistical concepts and less in searching PubMed. Conclusions cannot be made for health professionals because of the low response rate from this group. As evidence-based health care continues to extend to other disciplines, librarians can position themselves to participate fully in the EBM educational process.
INTRODUCTION
Health sciences librarians have been involved in bringing research (or evidence) to the patient's bedside since Lamb introduced clinical librarianship in 1971. More recently, leaders in the field of health sciences librarianship have become aware of the need to teach evidence-based health care (EBHC) techniques and principles in the medical school curriculum for evidence-based medicine (EBM) to become a part of general practice.
In her 1997 editorial in the Bulletin of the Medical Library Association, Giuse explicitly urged readers to study the tenets of EBM and to become proactive in the clinical setting, with an equal voice on the health care team [1]. Librarians attending the annual meeting of the Medical Library Association (MLA) in Seattle that year could, in fact, attend a continuing education course McKibbon taught, “Evidence Based Medicine for Librarians: Panning for Gold.”
One of the first reports of librarians being involved in a formal program in how to teach EBM principles was published in May 2000 in Academic Medicine. An EBM working group of internists and librarians under the auspices of the New York Chapter of the American College of Physicians began offering a course in 1996 for physicians and librarians “to learn the basic skills of EBM and to share experience about how these can best be taught, practiced, and disseminated” by forming a cadre of physician-librarian partnerships [2]. An evaluation of this model, published in 2003, showed significant changes in self-rated abilities to assess and teach article validity, with 28% training other faculty on returning to their home institutions. By the end of the class, 84% of internists felt able to teach statistical concepts such as absolute risk reduction, relative risk reduction, and number needed to treat, but only 30% to 50% of the librarians felt able to teach these concepts. The majority of the participants were “EBM novices,” contrary to the expectations of the course planners [3]. The authors concluded that participation in a single class would not result in EBM being integrated into a residency training program but was an essential first step.
The “Rocky Mountain Evidence-Based Health Care Workshop,” taught continually since 1999, was another attempt to develop a group of EBM instructors that included librarians as “fully acknowledged members of the faculty/tutor team” [4]. The class itself, with librarians included as faculty, was intended to serve as a model for how to teach EBM skills. The class developers also hoped “that when workshop participants return to their home institutions that they return determined to fully utilize the skills and services of their local librarians” [4].
Three librarians who participated in the 2002 workshop reported that the benefits they experienced included a strengthened relationship with clinicians and researchers, requests for clinical librarian services, increased requests for classes, and an enhanced professional reputation. The academic librarian reported a greater understanding of statistical concepts, which enabled her to become a more informed instructor. She was able to share her knowledge and skills with library colleagues. The clinical librarian listed in her outcomes that the model for learning influenced how she taught her classes. The departmental librarian reported that she found the introduction to epidemiology particularly helpful and that it enhanced her understanding of the statistical concepts involved [4].
In 1999, three years after the New York class was initiated and the same year the Rocky Mountain workshop was first offered, a week-long course was developed by College of Medicine (COM) faculty in the Department of Medical Education (DME) and librarians from the Library of the Health Sciences (LHS) at the University of Illinois at Chicago (UIC) with the intention of training both medical faculty and librarians in ways to teach EBM skills at their home institutions. The course used a train-the-trainer approach, presenting EBM principles and skills while also considering appropriate teaching methodologies and techniques. The course provided an overview of EBM, statistical concepts, critical analysis, EBM products, and literature searching and filtering. Exercises and discussions were patterned after EBM instructional formats such as journal club and morning report to give participants experience and increase their comfort levels in leading students through the EBM process. The course was offered for four successive years. The development of this course is described in detail elsewhere [5].
DME participants, a large number of whom were international students, included physicians, deans of medical schools, dentists, public health workers, and other professionals such as nurses and physical therapists. Health practitioners were awarded two hours of credit toward a master's degree in medical education. Librarians received twenty-seven MLA-approved continuing education credits. The course used a train-the-trainer model. The outcome goals were that the participants would:
teach EBM principles on their return to their home institutions
learn how to critically appraise a clinical article for methodological validity including an understanding of statistical concepts presented
improve their PubMed searching ability
increase collaboration between clinicians and librarians
The study reported here sought to determine if this model, which partnered COM faculty and librarians both as teachers and students, was an effective means for students to meet these goals.
METHODOLOGY
A questionnaire with twenty questions was designed to be completed in a twenty-minute telephone interview. To avoid bias and encourage uncensored answers from participants, a research assistant who was not connected with the project administered the survey. Institutional review board approval was secured at the host institution. Email invitations to take part in the study were sent to all seventy students who completed the class over the four years. Thirty-nine of these participants were librarians, and thirty-one were health care professionals enrolled in the DME's master's degree program. If a student responded positively to the email, an appointment was scheduled to complete the survey over the telephone with the independent surveyor. A pilot project was conducted with seven participants to identify and clarify any questions that might be ambiguous. The final survey contained twenty-three questions and took about fifteen minutes to complete (Appendix).
RESULTS
The researchers originally intended to survey all the clinicians and librarians who attended these sessions. However, efforts to contact the thirty-one DME participants were not successful, even though an extensive effort was made to locate contact information for them. No current email addresses could be found for six (19%) of the former students, fourteen (45%) invitations received no reply, seven (23%) resulted in a failure notice (email address was defunct), and one (3%) declined the invitation. Only three (10%) consented to participate. Because only two clinicians ultimately completed the questionnaire, their results were not included in the study.
Efforts to contact the librarians were much more successful. Email addresses could be found for all thirty-nine librarians. Four (10%) invitations received no reply, three (8%) resulted in failure notices, four (10%) librarians declined the invitation, and twenty-eight (72%) consented to participate. The results presented here include responses from these twenty-eight librarians. Not everyone answered every question. The results discussed below are not presented in the order the questions were asked in the survey, but rather in order of the class goals, and several responses may have been used to measure one goal. Results are summarized in Table 1.
Table 1 Summary of study findings
Teaching evidence-based medicine principles at home institutions
A major area the researchers were interested in was the question of whether or not the course resulted in more participants teaching EBM concepts and processes once they returned to their home institutions (question 1). In this regard, the course was a success with those who responded to the survey. Only seven (25%) of the twenty-eight librarians responding to this question taught any EBM concepts before taking the class, while twenty-five (89%) did after completing the class (question 2). This teaching took place in a variety of settings, with formal classes being the leading format (17), followed by journal club (6), the clinical setting (5), and, finally, morning report (4) (question 4). Other venues included continuing education, integrated course work, presentations, and one-on-one sessions. Most of the sessions lasted one hour (question 5).
Respondents were asked to explain what influence, if any, the course had on their participation in teaching EBM principles (question 6). Of the twenty-six respondents, only two (7%) replied that the class did not influence their participation in teaching EBM. The remaining twenty-four (93%) who answered this question said that the course had a positive impact on their teaching. The positive responses to this open-ended question tended to fall into three general areas: the knowledge gained from the course, the confidence to teach the topics covered in the course, and the credentials they acquired by having completed the course. Most respondents cited more than one influence.
The respondents who discussed the knowledge they gained from the course mentioned that it gave them good background information to build on, that it clarified concepts for them, and that it gave them a greater understanding and awareness of EBM principles. Moreover, they stated that the course gave them the confidence to teach EBM concepts and that they would not be teaching now without having taken the class. Two respondents said that the course gave them the credentials they needed to teach at their institution, and two said that they enjoyed the pairing of health care providers with librarians.
When respondents were asked specifically if they taught statistical concepts after the class (question 8), sixteen (57%) said no. When asked why they did not teach statistics, the most common comment was that the respondents did not feel they had enough knowledge of statistics to teach it. Several mentioned that the opportunity to teach statistics had not come up at their institutions, pointing out that the medical faculty usually taught statistics. One person did report that the statistical concepts learned in the class helped clarify the significance of the statistics presented in medical articles.
Reading an article for methodological validity with an understanding of statistical concepts
Participants were asked what skills the course gave them that they did not have before taking the class (question 12). Increased skill in critical appraisal was mentioned by several respondents in a number of ways: practical experience in evaluating a study, analysis of study validity, and overall skills in reading and analyzing clinical studies.
When asked specifically about how much their understanding of statistical concepts improved as a result of taking the class (question 7), twenty-five (89%) said very much or somewhat. These results echoed a common theme in the evaluations taken immediately following the courses. Students expressed increased confidence in reading an article for validity—which included looking at study design, intention-to-treat principles, follow-up, blinding, and appropriate spectrum of patients—but they still struggled with understanding the epidemiological statistics presented in research articles. A majority of the surveyed students appeared to have come into the class with little or no understanding of statistics. While the introduction to these concepts improved their knowledge considerably, few felt it improved enough to actually teach these concepts to others. When asked if there were areas of EBM they would be interested in receiving more training in, seven of the twenty-eight (25%) mentioned statistics, possibly in the form of a refresher course (question 13).
Improving PubMed searching ability
The researchers anticipated that improving PubMed searching proficiency would be a major contribution of the class, especially for the health care professionals. Because, as noted above, too few health professionals responded, this hypothesis could not be assessed. Among the librarians, the results were modest. Only eight (29%) of the students felt their PubMed searching ability improved very much or somewhat, while eleven (39%) perceived themselves as already an expert (question 9), not a surprising result for librarians. The remaining nine (32%) reported that their skills improved a little or not at all. Evaluations received immediately after completion of the course indicated that some of the librarians had been more proficient using other MEDLINE interfaces, and the class offered an opportunity to become more familiar with PubMed.
Increasing collaboration
For the librarians completing the course, increasing collaboration with others in teaching EBM was viewed very positively. After the class, thirteen of nineteen responding (68%) contacted faculty at their institution to promote EBM practice (question 19). For eleven of the thirteen (85%) who contacted other faculty, this resulted in collaborative teaching (question 20). Unfortunately, the very valuable feedback on whether the course fostered health care providers' collaboration with librarians could not be determined because so few participated in the survey.
Other findings
When students were asked what skills, in general, the course had given them (question 12), their replies fell into the following categories: overall understanding of EBM principles, search skills, EBM tools and resources, critical appraisal of the literature, statistical concepts, and ways to teach EBM concepts. Many respondents indicated that the course had given them a better understanding of the EBM process, its scope, its components, and the steps involved. An increase in search skills was also a common answer, with references to PubMed Clinical Queries, Boolean logic, searching for clinical evidence, use of clinical filters, and increased confidence in searching skills. In conjunction with searching, several respondents indicated an increased awareness and understanding of EBM resources and databases. Several respondents indicated that having a “how to teach EBM” model throughout the course was valuable, and they were able to apply what they learned in instructional interventions at their home institutions. One respondent said that the practical applications of EBM became evident throughout the course, and another respondent said that the course overcame a personal prejudice against EBM. One respondent cited increased confidence in working with physicians, and another said that the course served to integrate library and clinical skills.
In addition to statistics mentioned earlier, when asked where more instruction was needed, the respondents expressed an interest in learning more about evidence-based practice in other health professions, particularly nursing. Qualitative research, cost studies, training in using EBM resources such as the Cochrane databases, and literature evaluation were all mentioned as possible topics for further EBM training. Fifteen (54%) of the class participants have had follow-up instruction since taking the UIC class (question 14). Eight (29%) of the respondents said they would not be interested in more training at this time (question 13).
DISCUSSION
The EBM course at UIC sought to expand the role that librarians could play in EBM, from the more traditional one of searching and filtering the literature to the broader position of teaching EBM and working collaboratively with other professionals. While librarians need to learn fundamental principles of EBM, including effective searching and appraisal of the literature, they can also learn how to become effective teachers of EBM principles and increase collaboration with other professionals. If they are prepared to do so, librarians can take advantage of many opportunities to teach EBM principles and resources because of the positions they already enjoy as teachers integrated in health professions curricula or as point-of-need instructors. The results of this follow-up survey indicate that the train-the-trainer model used in this continuing education course for librarians and health professionals is an effective approach for promoting and developing the skills librarians need to be able to teach EBM principles. It is noteworthy that 89% of respondents reported teaching EBM concepts after completing the course, whereas only 25% had done so before taking the course. An unfortunate limitation of the study was that this same determination could not be made for health professionals due to the low response rate.
Based on survey responses, the collaborative model used in the course, librarians and health professionals engaged as learners, was highly successful for the librarians: 68% of respondents said that they contacted others in their institutions on returning home to promote EBM and collaborative teaching. These results concur with the more personal reports of the librarians who attended the Rocky Mountain workshop [4].
The EBM course was less successful in achieving competency in statistical analysis among respondents, although a need was clearly expressed to better understand the statistical concepts used in reporting research results. This feedback was, in fact, the same as that received from librarians on course evaluations at the completion of the course and echoed the findings of the New York State study.
Most respondents reported that the EBM course did not significantly increase their skills in searching PubMed. Librarians did not perceive much benefit from that segment of the class. However, feedback from course evaluations collected immediately after the course indicated that health professionals enrolled in the course needed additional instruction in searching competencies.
The different levels of success in meeting the course objectives suggested a need for various approaches and models to meet the requirements of diverse audiences with different skill sets. Although the collaborative model worked well in meeting the goal of increasing cooperative teaching of EBM principles for librarians, it might not be as effective in developing specific skills in literature searching and statistical analysis.
Even in the same professional groups, the need for EBM skills will differ based on institutional cultures and curricula. The academic librarian who supports active evidence-based practice curricula in health professions colleges will need teaching approaches that are appropriate for instruction of students in a classroom, while the hospital librarian may be looking for point-of-need instructional techniques that would be effective with residents and clinicians. Nevertheless, instructing librarians in how to teach EBM, while encouraging and modeling collaboration, are important and achievable goals.
When asked in question 22 what the biggest obstacle was to becoming more involved in EBM teaching, the most common response, from seven of the twenty-eight librarians, was some variation on the theme of an institutional culture where physicians did not see librarians as key components on the health care or teaching team. Unfortunately, this study was not able to determine if this obstacle could be overcome by having health care professionals learn collaboratively with librarians to teach EBM concepts.
CONCLUSION
The extent to which health sciences librarians will be involved in using and teaching EBM principles is certain to vary, but all will need some degree of competence, given expectations in curricula, changing practice patterns, increased access to and reliance on the literature, and new roles that require more than searching skills. Continuing education courses that provide the awareness, skills, and confidence that librarians need to become engaged in the EBM process will continue to be necessary. The emerging role of the “informationist” [6] or “information specialist in context” [7] will likely create a need for more intensive training in EBM concepts, particularly in critical analysis of the literature.
The American College of Graduate Medical Education endorses competencies in EBM skills to improve patient care [8]. The American Association of Medical Colleges advocates incorporating evidence-based principles throughout medical education and specifically identifies information skills [9]. Librarian–health professional teams are well suited to blend these information and clinical skills. The collaborative model used in this course, targeted at both librarians and health professionals, advances the integration of these skills and acknowledges the expertise of both professions.
As EBM continues to progress from strictly medical school to encompass nursing, dentistry, physical therapy, and other allied health sciences, librarians as well as educators in those professions need to be prepared to step up to a teaching role. For those preparing evidence-based classes in these varied disciplines, this study suggests that a train-the-trainer model is effective, that modeling collaboration encourages collaboration, and that the different skills and knowledge gaps that each group brings to the table have to be acknowledged and addressed. For those librarians who want to support or initiate EBM or EBHC at their institutions, successful partnerships can be achieved when they take the initiative in offering to teach EBM concepts and principles to both faculty and students.
For EBHC to become a reality for all students and practitioners, it needs to be consistently taught by competent teachers across the curriculum whenever a teachable moment presents itself. Some of these teachers can be librarians; moreover, the “teachers of these teachers” can also be librarians working side by side with educators in all the health sciences.
APPENDIX
Survey
Hello, ______. This is ______. I am calling as a follow-up to the email in which you consented to answer some questions regarding your activities following the evidence-based medicine (EBM) class you took at the University of Illinois at Chicago (UIC) in July of ______. We are seeking feedback on how effective the class was in preparing its participants as EBM teachers. It is estimated that this will take ______ minutes of your time.
As we mentioned in our original email, there are no risks associated with your participation. Your anonymity will be protected. Your participation is entirely voluntary and will not affect your relations with UIC in any way.
First, do you work as an administrator, full-time teacher, clinical teacher, librarian, or something else? Please check all that apply. (If respondent says not currently employed, ask about status at time took class.)
______ Administrator
______ Full-time teacher
______ Clinical teacher
______ Librarian
______ Other (Please specify.)
I. Teaching
1a. Did you teach EBM before taking the class? (We define EBM as the three steps that were taught in class: 1. Converting information needs into focused questions. 2. Efficiently tracking down the best evidence with which to answer the question. 3. Critically appraising the evidence for validity and clinical usefulness.)
______ Yes
______ No (Skip to question 2.)
1b. If yes, in what capacity? (Check all that apply.)
______ Formal class
______ Journal club
______ In the clinical setting
______ Morning report
______ Other (Specify.)
2a. Have you taught any steps of the EBM process at anytime since taking the class?
______ Yes (Skip to question 3.)
______ No
2b. Why not?
(Skip to question 6.)
(Note: you need to skip to question 6 here, if they said “no,” because questions 3–5 only apply if they say yes to question 2a.)
3. In what capacity? (Check all that apply.)
______ Formal class
______ Journal club
______ In the clinical setting
______ Morning report
______ Other (Specify.)
4a. Have you taught EBM any time in the past twelve months?
______ Yes
______ No (Skip to question 6.)
4b. In what capacity? (Check all that apply.)
______ Formal class
______ Journal club
______ In the clinical setting
______ Morning report
______ Other (Specify.)
4c. How frequently have you taught EBM concepts since taking the class?
______ Weekly
______ Monthly
______ Quarterly
______ Irregularly
5. How long have the sessions lasted? (Check all that apply.)
______ 1 hour
______ 2–4 hours
______ Half day
______ All day
______ More than a day
6. What role, if any, did the UIC class play in influencing your participation in teaching EBM?
7. As you recall, a component of the class focused on statistical concepts such as number need to treat, absolute risk reduction, relative risk reduction, confidence interval, etc. How much did your understanding of these concepts improve as a result of taking the class? Would you say…
______ Very much
______ Somewhat
______ A little
______ Not at all?
8a. Did you ever teach these statistical concepts to others?
______ Yes (Skip to question 9)
______ No
8b. Why not?
9. Another component of the class focused on searching PubMed. How much did your skill in searching this database improve as a result of taking this class? Would you say…
______ Very much
______ Somewhat
______ A little
______ Not at all
______ You were already an expert
______ Do not use PubMed
10. Did you ever teach how to search PubMed before taking the class?
______ Yes
______ No
11a. Have you taught how to search PubMed since taking the class?
______ Yes (Skip to question 12.)
______ No
11b. Why not?
12. What skills, if any, did the class give you that you did not have before taking the class?
13. Are there other areas of EBM in which you would be interested in more training?
14. Did you take any follow-up EBM instruction after the UIC class?
______ Yes
______ No
II. Interdisciplinary cooperation
For health care workers:
15. Before you took the EBM class, had you ever contacted a librarian when you need a MEDLINE search?
______ Yes
______ No
16. After you took the class, did you consult with a librarian when you needed a search done?
______ Yes
______ No
17. After you took the class, did you to teach an EBM class with a librarian?
______ Yes
______ No
(Go to question 21.)
For librarians:
18. Before you took the EBM class had you ever contacted faculty at your institution to promote EBM activities?
______ Yes
______ No
19. After you took the class, did you contact faculty at your institution to promote EBM at your institution?
______ Yes
______ No
20. If yes, have you done any collaborative teaching?
______ Yes
______ No
Both groups:
21. How much has the class influenced the way you are working today?
______ Very much
______ Somewhat
______ A little
______ Not at all
22. If you would like to be more involved in EBM practice at your institution, what is the biggest obstacle in your way?.
23. Do you have any comments about the EBM class you would like to share?
Thank you for your time in completing this questionnaire.
Contributor Information
Carol S. Scherrer, Email: cscherre@uic.edu.
Josephine L. Dorsch, Email: jod@uic.edu.
Ann C. Weller, Email: acw@uic.edu.
REFERENCES
- Giuse NB. Advancing the practice of clinical medical librarianship [editorial]. Bull Med Libr Assoc. 1997 Oct; 85(4):437–8. [PMC free article] [PubMed] [Google Scholar]
- Dunn K, Wallace EZ, and Leipzig RM. A dissemination model for teaching evidence-based medicine. Acad Med. 2000 May; 75(5):525–6. [DOI] [PubMed] [Google Scholar]
- Leipzig RM, Wallace EZ, Smith LG, Sullivant J, Dunn K, and McGinn T. Teaching evidence-based medicine: a regional dissemination model. Teach Learn Med. 2003 Summer; 15(3):204–9. [DOI] [PubMed] [Google Scholar]
- Traditi LK, Le Ber JM, Beattie M, and Meadows SE. From both sides now: librarians' experiences at the Rocky Mountain Evidence-Based Health Care Workshop. J Med Libr Assoc. 2004 Jan; 92(1):72–7. [PMC free article] [PubMed] [Google Scholar]
- Dorsch JL, Jacobson S, and Scherrer CS. Teaching EBM teachers: a team approach. Med Ref Serv Q. 2003 Summer; 22(2):107–14. [DOI] [PubMed] [Google Scholar]
- Davidoff F, Florance V. The informationist: a new health profession? Ann Intern Med. 2000 Jun 20; 132(12):996–8. [DOI] [PubMed] [Google Scholar]
- Medical Library Association. The information specialist in context. [Web document]. Chicago, IL: The Association, 2005. [rev. 2005; cited 22 Mar 2005]. <http://www.mlanet.org/research/informationist/>. [Google Scholar]
- Accreditation Council for Graduate Medical Education. Outcome project: general competencies. [Web document]. The Council, 1999. [rev. 2001; cited 22 Mar 2005]. <http://www.acgme.org/outcome/comp/compFull.asp>. [Google Scholar]
- Medical School Objective Project Writing Group. Learning objectives for medical student education—guidelines for medical schools: report I of the Medical School Objectives Project. Acad Med. 1999 Jan; 74(1):13–8. [DOI] [PubMed] [Google Scholar]