An Evidence-based Medicine Elective Course to Improve Student Performance in Advanced Pharmacy Practice Experiences

Abstract

Objective

To implement and evaluate the impact of an elective evidence-based medicine (EBM) course on student performance during advanced pharmacy practice experiences (APPEs).

Design

A 2-hour elective course was implemented using active-learning techniques including case studies and problem-based learning, journal club simulations, and student-driven wiki pages. The small class size (15 students) encouraged independent student learning, allowing students to serve as the instructors and guest faculty members from a variety of disciplines to facilitate discussions.

Assessment

Pre- and posttests found that students improved on 83% of the core evidence-based medicine concepts evaluated. Fifty-four APPE preceptors were surveyed to compare the performance of students who had completed the EBM course prior to starting their APPEs with students who had not. Of the 38 (70%) who responded, the majority (86.9%) agreed that students who had completed the course had stronger skills in applying evidence-based medicine to patient care than other students. The 14 students who completed the elective also were surveyed after completing their APPEs and the 11 who responded agreed the class had improved their skills and provided confidence in using the medical literature.

Conclusions

The skill set acquired from this EBM course improved students' performance in APPEs. Evidence-based medicine and literature search skills should receive more emphasis in the pharmacy curriculum.

INTRODUCTION

Evidence-based medicine (EBM) is a health care principle of incorporating knowledge gained from the best available research evidence with clinical expertise, and applying this to individual patient circumstances.¹ Practicing EBM requires a clinician to perform critical skills that are not used frequently in daily practice, including performing efficient literature searches and applying the formal rules of evidence and biostatistics in evaluating the medical literature.^1,2 A 2007 survey of British Medical Journal readers elected the concept of EBM as 1 of the top 15 medical milestones of the past 150 years.^2,3 Although this concept inherently may represent one of common sense, its role in changing focused patient care is undeniable. With the shift to an interdisciplinary team approach, pharmacists play a critical role in the evidence-based decision-making process, serving as a source of scientifically valid information, and experts on best practices in the appropriate use of medications.

The American College of Clinical Pharmacy (ACCP) emphasizes that clinical pharmacists serve as a source of objective, evidence-based therapeutic information and recommendations.⁴ A pharmacist can proactively ensure rational drug therapy and avert many medication misadventures due to therapeutic decisions made at the point of prescribing.⁴ Arming student pharmacists with the ability to read and critically appraise the literature to maintain an up-to-date knowledge base is essential. The ever-expanding body of literature adds yet another layer of complexity to efficient and accurate literature identification and evaluation. As an example, the National Library of Medicine's MEDLINE database indexes over 5,000 journals and nearly 20 million citations.⁵ Each day an average of 1,950 citations are uploaded to the database, stressing the need for students to develop strong literature search skills to appropriately and efficiently maintain a working knowledge of the medical literature.

Courses enhancing EBM skills are offered throughout health sciences education and training, but only a few studies have evaluated these courses.^6-8 There is a significant need for EBM skills to be evaluated in every medical school curriculum.⁶ Medical students briefly trained in conducting EBM-searches were able to obtain more relevant and complete search results compared to students who did not receive training.⁷ Likewise, active-learning strategies in a drug information and literature evaluation course resulted in improvement in pharmacy students' ability and confidence to perform the course's objectives. The skill set used in acquiring, interpreting, and applying evidence-based medicine practices is formally taught in a required course in only 42% of surveyed US colleges and schools of pharmacy, even though the representative surveyed felt these principles should be incorporated into the curriculum to a greater degree.⁹

The Center for Advancement of Pharmaceutical Education (CAPE) outcomes suggest that providing patient-centered pharmaceutical care in an interprofessional health care team should be based on sound therapeutic principles and evidence-based data. The pharmacist is called to retrieve, analyze, and interpret the lay, professional, and scientific literature to provide drug information to patients, their families, and other involved health care providers.¹⁰ Further, in concordance with the standards outlined in the Accreditation Council for Pharmacy Education (ACPE), PharmD graduates must be educated to deliver patient-centered care as members of an interprofessional team, emphasizing evidence-based principles.¹¹

Fourth-year pharmacy students completing their advanced pharmacy practice experiences (APPEs) are given numerous opportunities to practice evidence-based medicine when conducting journal club discussions, making drug information queries, discussing clinical pearls on medical rounds, and making patient care decisions. APPEs have become the focal point for student experience in the application of literature evaluation and analytical skills to patient care decisions. To our knowledge, there are no published reports on the impact of a lecture-based evidence-based medicine course on pharmacy student performance in APPEs. The primary objective of this study was to determine the perceived impact and influence of an EBM elective course, offered during the third year, on students' performance during APPEs from the preceptors' perspective. Secondary objectives were to determine the impact of the course on APPE performance of EBM-related practices from the students' perspective, and to evaluate the ability to use the targeted skill set and core concepts taught in the course.

DESIGN

The 2-hour elective course in EBM was offered during the spring semester of the third year of the curriculum. Prior to this elective, students were exposed to skills related to retrieval and interpretation of literature and formulation of answers to clinical questions during 2 semesters of drug information courses (introductory and advanced) in the core curriculum. The goal of this course was to teach students to evaluate the medical literature critically and to apply this skill to patient decision-making. The course was designed to provide students with the knowledge, understanding, and skills to evaluate the medical literature accurately and confidently. Students learned critical-evaluation skills in addition to commonly encountered statistical and analytical concepts. The specific learning objectives of the course are outlined in Table 1. The course was limited to 15 students each semester who met for a single 2-hour class each week. The small class size fostered independent student learning and an active-learning style where the students served as the instructors and the faculty members facilitated discussion. Some of the active-learning techniques used included case studies/problem-based learning, journal club simulations, and student-driven wiki pages.

Table 1.

Specific Learning Objectives for an Evidence-based Medicine Course Elective

This course was taught via group discussions and active-learning methods incorporating multiple clinical faculty members, which enabled the students to maximize application of learned skill sets and evidence-based medicine principles. Most of the course faculty members were from the clinical pharmacy and outcomes sciences department, but several clinical specialists and residents from local academic medical centers also were involved. Practicing clinicians were invited and encouraged to participate in the course based on their day-to-day patient responsibilities and real-life application of the skills focused on in the course. Each week, students were assigned topics related to therapeutic disciplines, then faculty members who were experts in those disciplines provided 2 journal articles to all students. Each course faculty member had responsibility for 1 class each semester. Responsibilities included preparing students for their evaluation and discussing assigned literature, evaluating (along with the course coordinator) the discussions during class, and responding to wiki discussions and posts during the week following the course. Initially, 2 students were designated to lead each journal club article discussion in an open forum style and were responsible for presenting a brief review of a selected analytical or statistical tool. As the semester progressed and students became more comfortable with EBM concepts, students began leading discussions individually rather than in teams. All students were required to participate actively in the discussion, and faculty members helped facilitate discussion (if needed) by raising thought-provoking questions. Students posted article reviews on the course wiki. Accessible only by course faculty members and students, the wiki served as an online community that encouraged collaboration among its members to author and edit information in real time. Access to course documents, hosting of live discussions, and formulation of answers to patient scenarios were facilitated by the wiki.

Following each journal club discussion session, all students were required to use data reviewed in class, along with additional supporting data that they researched, to make an EBM decision regarding a patient case posted on the wiki discussion board. Patient case scenarios were taken from questions based on real-life situations submitted by practicing clinical faculty members. All students were required to contribute to the decision via the wiki discussion board and to defend their answer with supporting literature. Two students were picked at random each week to serve as wiki discussion facilitators and were assigned a grade for their response. (Examples of the learning tools described above are available from the author.)

Course grades were not evaluated as part of this study. This study was determined to be exempt by the Institutional Review Board for Research with Human Subjects at the University of South Carolina.

EVALUATION AND ASSESSMENT

Pretest and Posttest

A 15-question test (available from the author) was administered to students on day 1 of the EBM course and again on the final day of class as a tool to evaluate students' baseline knowledge and acquisition and retention of 12 key concepts related to understanding and application of biostatistics. Students were not informed in advance about administration of the pretest or posttest. The key concepts evaluated were those covered during the journal club discussions and introductory lecture hours (eg, power, p values, intention-to-treat, and sensitivity/specificity). The tests did not impact the students' grade and results were not provided to the class. The questions were designed primarily as multiple-choice with a single correct answer. For the purpose of analysis, each question was given a value of 1 point. Descriptive statistics were used for data analysis.

The mean number of correct responses on the pretest was 8.6 (out of 15) compared to 13.7 (out of 15) on the posttest, an increase of 34%. Overall, students improved on 10 of the 12 concepts evaluated, and individually, all students showed improvement in at least 3 core concepts evaluated.

Preceptor and Student Surveys

The course coordinator developed 2 survey instruments, 1 designated for APPE preceptors and 1 for students who had completed the EBM course (Tables 2 and 3). Both survey instruments used a standard 5-point Likert scale for responses (1 = strongly agree, 5 = strongly disagree) ^9,12 and both were conducted through Survey Monkey (Survey Monkey, Palo Alto, California) to maintain anonymity of reporting and blinding to study investigators.

Table 2.

Advanced Pharmacy Practice Experience Preceptors' Assessment of the Influence of an Evidence-based Medicine Elective on Student Performance (N = 38)

Table 3.

Survey Results of Students in Evidence-based Medicine Elective, N = 11

Preceptor Survey.

The primary study objective was evaluated by comparing preceptors' assessment of the AAPE performance of students who completed the EBM elective with that of students who had not completed the elective. The preceptor survey instrument was sent to medical and pharmacy clinicians who: had served as APPE preceptors during the fourth year of the pharmacy curriculum, were designated as either an adjunct clinical instructor or preceptor within the college, and had completed a formal preceptor's workshop within the previous 3 years. Preceptors were excluded if their practice areas did not involve frequent use of medical literature (eg, non-direct patient care practice areas including nuclear medicine, administration) to answer patient and professional questions.

The survey instrument consisted of 6 items with Likert-scale responses (1 = strongly agree, 5 = strongly disagree) targeted at evaluating students' performance in literature retrieval and evaluation, and application of evidence-based decision making during the APPE (Table 2). Preceptors were asked to compare achievement of APPE outcomes by students who had completed the EBM course with that of students who had not. Space for preceptors' general comments also was provided. Preceptors were asked to indicate the number of formal and informal journal clubs required of their APPE students. To ensure accuracy of reporting, preceptors were provided with a list of all students they had precepted for the year and which had completed the EBM course. A description of the elective course and objectives also were provided to the preceptors. Preceptors were given 4 weeks to complete the survey instrument. After 2 weeks, the link to the survey instrument was redistributed to all preceptors who had not responded and a final reminder was e-mailed at 3 weeks.

Fifty-four APPE preceptors met the study inclusion criteria and 38 (71%) participated in the survey. No preceptor participants chose to withdraw their consent to participate or opted out of the survey. Thirty-three of 38 respondents strongly agreed or agreed that students who had completed the elective course possessed a broader and more in-depth knowledge base to evaluate the medical literature compared to students who did not take the course. Also, the majority of respondents strongly agreed or agreed that course enrollees were more efficient in critiquing and evaluating the medical literature compared to their student peers. Thirty-two of 38 respondents strongly agreed or agreed that course enrollees were more confident in their ability to interpret the medical literature accurately. Eight preceptors strongly agreed and 23 agreed that students were better able to apply the information learned from the medical literature in response to drug information queries, in patient care decisions, and in preceptor-student discussions. Students were active in applying the course skill set during the APPEs. Preceptors required students to complete an average of 1-2 formal journal club sessions and critique and evaluate 6-10 articles during each APPE (Table 2).

Student Survey.

The student survey instrument was distributed at the conclusion of their APPEs to students who had completed the EBM elective (Table 3). Using a 5-point Likert scale (1 = strongly agree, 5 = strongly disagree), students were asked to agree or disagree with 10 statements. Students also were asked to assess the value of applying the skills learned during the EBM elective to their APPEs and future career applications, and to indicate the number of formal and informal journal clubs presented during their APPEs. Space was provided for students to submit general comments. The students were given 4 weeks to complete the survey instrument. After 2 weeks, the survey instrument was redistributed to all students who had not responded and additional reminders were e-mailed at 3 weeks.

Fourteen students were enrolled in the EBM elective course and 11 completed the survey instrument (79% response rate). Survey instruments were distributed during the final month of the APPE, which was approximately 1 year following the conclusion of the elective course. No student participants chose to withdraw their consent to participate or opted out of the survey.

All but 2 students strongly agreed that they were satisfied with their level of efficiency to read, critique, and/or evaluate the medical literature. Also, all student respondents strongly agreed or agreed they were confident in their ability to interpret the medical literature accurately. When asked if able to apply the knowledge base and skill set obtained in the course to the clinical setting, all student respondents either strongly agreed or agreed. Furthermore, the majority (10/11) of students felt the fundamentals learned in the course were most valuable to their success during their APPE (Table 3).

DISCUSSION

Incorporation of evidence-based principles into the curriculum is critical for all health science education programs, including pharmacy. Supported by primary accrediting bodies and benchmark standards, these core skills should be distributed throughout required and elective courses.^4,10,11 Our curriculum includes both a basic drug information course and an advanced drug information and health outcomes course during the first and second years of the curriculum, respectively. These courses are designed to teach students basic literature evaluation skills and analytical concepts encountered in the primary literature. Despite these courses and incorporation of active-learning techniques in clinical application and pharmacotherapy courses during the second and third years of the curriculum, the pretest evaluation in this EBM elective demonstrated a deficiency in knowledge and retention of basic concepts required to interpret and apply medical literature effectively. Allowing students to lead journal club discussions and to use targeted literature from practicing clinicians who are content experts helped students develop confidence in understanding and application. Additionally, the use of Web-based tools provided an avenue for a team approach in answering clinical questions directly from patient care scenarios. Both of these concepts were strengths of the course. The students provided favorable comments on the use of a wiki, a tool that had been incorporated previously by clinical faculty members in a number of other courses and APPEs.¹⁴

One of the primary objectives of this course was to prepare students for success during APPEs, specifically in using medical literature. Preceptor feedback indicated that students enrolled in the course were consistently more confident and efficient, and possessed a stronger ability to apply medical literature to patient care. Several preceptors commented that they felt this course should no longer be an elective, but a required course prior to APPEs. One preceptor commented, “It was much easier reviewing EBM and how to apply the information to clinical practice with the students that had the EBM elective.” Only 1 preceptor submitted a negative comment about the EBM elective. Preceptors also commented on the desire to emphasize this information and skill set in the core curriculum. Students commented on added confidence and performance on APPEs based on the skill set acquired in this course. One student stated that the course “provided a great foundation for the experiential rotations” and helped him to lead discussions on articles in journal clubs. A recurring theme among students was their ability to transcend preconceived barriers to enhance both personal and peer EBM skills. The majority (75%) of students enrolled in the course chose to continue into postgraduate training and commented that this course contributed to this decision by giving them added confidence to perform during clinically-oriented APPEs.

The course was invaluable to the delivery and maximization of students' learned skill set. Although this course is taught on a single campus, much of the curriculum at SCCP is delivered via distance education between 2 campuses. Schools should consider how an EBM course requiring significant interactivity among faculty members and students could be accommodated effectively and efficiently via distance education. The online wiki is one potential means of delivery. Although the students who were enrolled in the EBM elective felt it should be required, distance education is a barrier to full integration of the course into the curriculum, so it is still offered only as an elective. Several modifications have been made based on student and course faculty feedback. A dinner and journal club meeting has been incorporated at the end of the semester to summarize key articles from the previous year. Students are paired and assigned a prominent biomedical or pharmacy journal from which they prepare an overview of the top 6-8 articles over a 4-month span. These summaries are presented at the dinner and later posted on the course Wiki. Additional changes have been made to incorporate clinical pharmacists and pharmacy residents from local institutions for case development and facilitation of discussions. In the future, interprofessional activities will be incorporated in the class, using professionals and trainees from other health care disciplines. Interprofessional interactions in the classroom have been shown to increase overall attitude and success in several other forums of health care trainees.^15,16 Future incorporation of the course in the core curriculum will depend on its adaptation to distance education.

To our knowledge this is the first study to evaluate an evidence-based medicine elective incorporating the unique discussion and Web-based tool design within a college of pharmacy curriculum. Limitations of the study included the small sample size. The timing of the survey may have limited the accuracy of responses, as it was administered to all preceptors in April, after completion of APPEs. If they had precepted some of the students earlier in the academic year, recall bias may have occurred. Additionally, as students progress through APPEs, learning experiences themselves can build confidence and provide repeated experience with the described skill set. Thus, some of the increase in confidence and skills reported by preceptors and students may have been the result of learning experiences during the APPEs rather than from the elective course. Some nonresponse bias occurred as not all preceptors and students meeting inclusion criteria responded to the survey. Sampling bias also may have occurred as preceptors' conclusions may have varied based on the number of students they precepted. Although providing a list of course enrollees to the preceptors was necessary to ensure accurate survey response, this knowledge may have biased preceptors assessment of students' performance. Finally, because many of the students who enrolled in the elective had an interest in pursuing postgraduate training, they were more likely to be high achievers and perform better in APPEs.

SUMMARY

Completion of an evidence-based medicine elective equipped pharmacy students with the skills necessary to evaluate the medical literature critically and apply the knowledge in making patient care decisions. Precptors also found the course enrollees' abilities to be superior to those not having the course, and overwhelmingly felt the in-depth study of EBM skill set be added to the core curriculum. This course has been in place as an elective for 3 years, and student feedback has been used to improve the course, including expansion to a capacity of 20 students. Future plans include offering the course via distance education and involving professionals from other health care disciplines. The development of similar courses should be considered strongly by all colleges and schools of pharmacy in an effort to better prepare our students for APPEs, postgraduate training, and the transition to independent practitioners.