An Elective Course on Landmark Trials to Improve Pharmacy Students' Literature Evaluation and Therapeutic Application Skills

Abstract

Objectives

To determine the impact of a landmark trials elective course on pharmacy students' attitudes toward evidence-based medicine, students' comfort with technical concepts used in drug literature, and students' perceptions of accessibility of PubMed from home computers.

Design

An elective course which gave third-year pharmacy students the opportunity to discuss landmark trials in primary care and reinforced skills in applying evidence from the primary literature to support therapeutic recommendations was design and implemented. The impact of the course was evaluated via a pre- and postcourse questionnaire administered during 3 consecutive course offerings.

Assessment

Overall, students had positive attitudes toward evidence-based medicine before taking the course (97.5% positive or somewhat positive) and these attitudes were unchanged postcourse (p = 0.74). Though 97.6% (n = 40) of students had Internet access at home, only 68.3% (n = 28) indicated having PubMed access at home. The course increased self-assessed comfort with technical concepts used in literature evaluation including random assignment (p < 0.01), placebo-controlled (p < 0.01), and intention-to-treat (p = 0.02).

Conclusion

An elective course on landmark trials allowed third-year pharmacy students to increase their comfort level with literature evaluation and reinforced their positive attitudes toward the use of evidence-based medicine in pharmacy practice.

INTRODUCTION

Evidence-based medicine is the use of the best current evidence from research to make decisions in practice and policy.¹ The ability to apply new research findings to practice and patient care are important skills for future clinicians.² Skills related to drug literature evaluation and necessary to the practice of evidence-based pharmacy are required components of the doctor of pharmacy curriculum. According to both the American Council for Pharmaceutical Education (ACPE) accreditation standards and the Center for Advancement of Pharmacy Education (CAPE) outcomes statements, pharmacy students should be exposed to practical applications of primary literature in both the classroom and practice.^3,4 Curriculum committees are asked to consider inclusion of foundational content in order to develop students' ability to evaluate clinical trials of pharmacotherapeutic treatments, as well as their ability to apply these skills in the clinical setting.³ Further, current accreditation standards focus on the development of skills in critical thinking and lifelong learning. As the field of pharmacy is continuously changing, these skills contribute to the development of professionals who can stay current and competent in the field over time. While students are expected to achieve these outcomes during the core curriculum, elective courses reinforce key concepts and allow students to explore their individual interests.

One area in which further reinforcement is needed later in the curriculum is applying literature to therapeutic recommendations. In the core curriculum, students are trained in drug literature evaluation and taught the importance of using the primary literature to support therapeutic recommendations; however, students have limited opportunities to increase proficiency in such skills in the core curriculum. Thus, an elective course entitled Landmark Trials in Primary Care was established at Midwestern University Chicago College of Pharmacy. This elective was designed to offer students the opportunity to discuss landmark trials in primary care and reinforce skills in applying evidence from primary literature to support therapeutic recommendations.

Pharmacy students' attitudes toward evidence-based medicine and comfort with literature evaluation are largely unexplored. A survey of randomly selected pharmacists in Illinois determined that pharmacists have positive attitudes toward evidence-based medicine.⁵ Though most pharmacists had performed literature searches within the previous year, there was a low level of awareness that these databases were accessible for free to the public. Since training in drug literature evaluation is now computer-based, pharmacy students may be more aware of databases to search the literature than practicing pharmacists who graduated 10 or more years ago. One study of medical students exposed to an evidence-based medicine curriculum found that it positively changed student attitudes toward evidence-based medicine and self-assessment of related skills.⁶ Therefore, this study sought to determine: (1) the attitudes of pharmacy students enrolled in a Landmark Trials elective course; (2) the impact of the elective on students' comfort with technical concepts used in drug literature; and (3) students' perceptions of the accessibility of PubMed from home computers.

DESIGN

Landmark Trials Course

In 2004, an elective course in Landmark Trials in Primary Care was established. The 2-credit-hour elective course was open to third-year students seeking to fulfill a curricular requirement for 16 quarter hours of elective coursework. As a prerequisite, all students had successfully completed required coursework in drug information, research methods, and drug literature evaluation. In addition, students were required to have successfully completed the portions of the pharmacotherapeutics sequence that covered the disease-state topics encountered within the elective. Primary course objectives included the ability to apply evidence from landmark trials to support therapeutic recommendations and to evaluate trial results for clinical applicability in patient care. Secondary objectives were to review evidence-based therapy recommendations for major primary care disease states and to reinforce skills needed to critically evaluate primary literature, as well as perform basic statistical calculations. The class size was 10 to 30 students and the course met for 2 hours per week for 10 weeks. The Web-based classroom support program Blackboard (Blackboard Inc, Washington, DC) was utilized to facilitate communication and provide a central repository for the course syllabus, assignment instructions, and supplemental readings. No textbook was required; however, students were required to purchase a course packet containing the landmark trials.

Each class session provided a review of trials related to a primary care disease state, such as diabetes, atrial fibrillation, dyslipidemia or heart failure. An example class schedule is provided in Table 1; however, the landmark trials chosen for the disease states varied slightly in each course offering. Individually or in pairs, all students were responsible for presenting a 10-15 minute overview of a trial in a “Trial Pearls” presentation. A recommended format for the presentation was provided to the students and included the following sections: context of the trial, study objectives, methods, results, clinical relevance, discussion items, and a patient case. The purpose of the presentation was to highlight the main elements of the trial and clinical applicability. This presentation had to contain at least 2-3 examples of how the student would interpret results from the trial and explain those results to a health-care practitioner or patient. Two to 3 trials were covered during a 2-hour period each week. Faculty members facilitated student discussion that focused on clinical applicability to patient cases and interpretation of literature evaluation concepts. For example, faculty members could facilitate calculation of the “number needed to treat” from a particular trial followed by a discussion of clinical applicability. Students were provided with a summary card containing common calculations, including absolute risk reduction, relative risk reduction, and “number needed to treat.”

Table 1.

Landmark Trials Elective Topic Schedule

Abbreviations: ALLHAT = The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial, SHEP = Systolic Hypertension in the Elderly Program, HOT = Hypertension Optimal Treatment, 4S = Scandinavian Simvastatin Survival Study, PROVE-IT = Pravastatin or Atorvastatin Evaluation and Infection Therapy, HPS = Heart Protection Study, CARDS = Collaborative Atorvastatin Diabetes Study, AFFIRM = Atrial Fibrillation Follow-up Investigation of Rhythm Management, SOLVD = Studies of Left Ventricular Dysfunction, DIG = Digitalis Investigation Group, MERIT-HF = Metoprolol CR/XL Randomized Intervention Trial in Congestive Heart Failure, RALES = Randomized Aldactone Evaluation Study Investigators, UKPDS = UK Prospective Diabetes Study, HOPE = Heart Outcomes Prevention Evaluation.

The course was taught by 2 faculty members, with 1 faculty member designated as the main facilitator each week. The course also offered an opportunity for pharmacy practice residents to attend class and lead discussion of select topics. Since our residency programs focus on teaching and learning, the elective course provided residents with opportunities to practice presentation and small-group facilitation skills.

Student assessment was based on a combination of presentation score, student participation, in-class assessments, and a final examination consisting of essay questions based on hypothetical patient cases. All of the assessments and examinations were open-note format. The presentation score was determined using a rubric designed and modified for the course that evaluated content, presentation style, and ability to answer questions. All students had access to the rubric prior to their presentation. Student participation was determined by a faculty member on a weekly basis using set guidelines which were provided to the students. Possible weekly participation scores ranged from a minimum of 1 point for “class attendance with no active participation” to a maximum of 5 points for “consistent participation which adds to the class discussion.” The weekly assessments were typically 1 or 2 case-based questions focused on the particular week's discussion topic. The assessments were designed to be completed in 5-10 minutes. Lastly, the final examination was a more extensive assessment with 5-6 essay-based patient cases that required evidence from specific trials as support for any recommendations. After the first 2 years, a mid-quarter take-home assignment that mimicked the final examination format was initiated based on student evaluation feedback. Students had a high degree of anxiety regarding the final examination since they were unfamiliar with the examination format. The mid-quarter take-home assignment was provided so that students could be better prepared for the format of the final case-based essay examination. Each component (presentation, participation, assessments, and midterm and final examination) accounted for approximately 25% of the student's final grade.

Course Evaluation

Students were encouraged to complete a supplemental course evaluation that consisted of 6 open-ended questions created by the course faculty members. The open-ended questions solicited students' opinions on course structure and topics and invited suggestions for areas of improvement and other disease state topics for inclusion.

Pre- and Post-Elective Questionnaire

A questionnaire was administered to all pharmacy students enrolled in the elective course on the first and last day of class in 3 consecutive course offerings (fall 2004, fall 2005, and fall 2006). Non-pharmacy students and students who were added to the class late and were not present at the time the precourse questionnaire was administered were excluded. The questionnaire was based on an instrument used in a study of physicians' perceptions of evidence-based medicine ²⁴ and adapted to the targeted population of pharmacists.⁵ An abbreviated questionnaire was administered to the students. The abbreviated questionnaire consisted of 9 items that assessed the student attitudes toward evidenced-based medicine and self-assessed comfort with technical concepts used in evidence-based medicine. Questions regarding access to PubMed and the Internet were asked in the precourse questionnaire. Additional questions regarding awareness of newsletter and Web-based drug information sources were asked in the postcourse questionnaire. Responses to the questionnaire were anonymous. Students were instructed to write a code number or word on the questionnaire (not their social security number or student identification number) and then save the code in a secure location. At the time of the postcourse questionnaire, students were asked to use this same code for survey pairing. The questionnaire study was reviewed and determined to be exempt by the Institutional Review Board at Midwestern University.

The main outcome measures were the attitude of the students' toward evidence-based medicine and the students' self-assessed comfort with technical concepts in literature evaluation. Ordinal scale measures were used to assess attitudes and comfort levels, while nominal measures were used to assess access to PubMed. Ordinal scale measures for attitudes ranged from positive to negative (5 = positive; 4 = somewhat positive; 3 = neutral; 2 = somewhat negative; 1 = negative). Statements that asked respondents to assess degree of agreement were as follows: 5 = strongly agree; 4 = agree; 3 = neutral; 2 = disagree; 1 = strongly disagree. Comfort levels with technical concepts were rated as very comfortable, somewhat comfortable, somewhat uncomfortable, or uncomfortable. Data were compiled and statistically analyzed via SPSS software (Version 15.0, SPSS Inc, Chicago, IL). Descriptive statistics were used to describe access to PubMed. Wilcoxon signed rank test was used to examine ordinal data. In addition to analyzing the ordinal data, the ordinal scale measure assessing comfort was recoded into nominal groups (eg, very comfortable vs. other comfort levels) since ideal course outcomes would result in students indicating a very high level of comfort. A student self-report of “very comfortable” would provide a more meaningful representation of student self-perception of mastery. Comparisons of nominal scale variables were made using the chi-square test or Fisher exact test, as appropriate. An a priori α of < 0.05 was chosen for statistical significance.

ASSESSMENT

Course Evaluation Results

On the University's standard course evaluation, all items were rated agree to strongly agree for the 3 years of course offerings. Examples of these items are: “the course improved my understanding of course subject matter,” “course objectives met,” “course content well organized,” “course included interpretation and application of information,” “evaluation methods required that I interpret/apply information I was expected to learn,” and “teaching methods helpful in better understanding course content.” Written course comments reflected that students felt more prepared in their ability to interpret trial results and apply these results to practice (eg, “I went into the course having no idea how to apply trial results to practice and now I feel very confident on my interpretation of medical literature”). On the supplemental course evaluation (N = 27), overall, students reacted positively to the course and had minimal suggestions for changes. When students were asked how the course helped them in understanding trial evaluation, a majority commented on their improved ability to determine the clinical applicability of the trial results and felt that skill would be useful in their future practice. Students were pleased with the small class size and format and felt it improved their learning. Many students commented on their increased comfort with statistical analysis, which was also demonstrated in the questionnaire results.

Questionnaire Results

Completed responses were received from 41 students over 3 years (78.8% effective response rate). Fifty-eight students were enrolled in the course over 3 years (10, 19, and 29 students in 2004, 2005, and 2006, respectively). Two non-pharmacy students and 4 students who were added to the course roster late were excluded. This left 52 students eligible for the study. Survey instruments for 11 students were unable to be paired and therefore not included in the analysis.

Overall, students had positive attitudes toward evidence-based medicine prior to the course (97.5% positive or somewhat positive) and these were unchanged postcourse (p = 0.74). After the course, students were more likely to feel that research findings were useful to the day-to-day practice of pharmacy (28.8% very useful, 43.9% useful vs. 73.2% very useful, 24.4% useful, p = 0.03). Students largely agreed with the statement that practicing evidence-based medicine improves patient care (92.7% strongly agree or agree), and this was affirmed postcourse (100% strongly agree or agree, p = 0.03). Similarly, students disagreed that evidence-based medicine is of limited value in pharmacy practice as much of pharmacy lacks a scientific base, with 70.7% of students responding strongly disagree or disagree both before and after the course (p = 0.45). At the end of the quarter, students were more likely to feel that pharmacists would rely on practice guidelines to make patient-care decisions (4.9% strongly agree, 82.9% agree vs. 29.6% strongly agree, 63.4% agree, p < 0.01). Following participation in the elective course, students were more likely to agree that pharmacists should make patient care decisions that require them to suggest changes or adjustments in drug therapy to prescribers (26.8% strongly agree, 68.3% agree vs. 61.0% strongly agree, 29.3% agree, p < 0.01).

The course increased self-assessed comfort with some technical concepts used in literature evaluation (Table 2): random assignment (p < 0.01), placebo-controlled (p < 0.01), and intention-to-treat (p = 0.02). The proportion of students responding with this high level of comfort was assessed. For all technical concepts, the percentage of students indicating that they were very comfortable with the technical term increased from baseline. This increase in the proportion of students indicating very comfortable was significant for retrospective/prospective (39.0% vs. 61.0%, p < 0.001), clinical efficacy (23.8% vs. 53.7%, p < 0.005), confidence interval (17.1% vs. 53.7%, p < 0.001), bias (34.1% vs. 56.1%, p < 0.001), intention-to treat (22.0% vs. 53.7%, p = 0.001), placebo-controlled (39.0% vs. 61.0%, p<0.001), and random assignment (31.7% vs. 61.0%, p < 0.001). There was no significant increase in the proportion of students very comfortable with relative risk (19.5% vs. 43.9%, p = 0.06), odds ratio (12.2% vs. 26.8%, p = 0.25), absolute risk (17.1% vs. 39.0%, p = 0.41), or number-needed-to-treat (26.8% vs. 58.5%, p = 0.07).

Table 2.

Student Self-Assessed Comfort Level with Knowledge of Technical Concepts (n = 41)

^aWilcoxon signed rank test.

Students were largely aware of newsletter sources that highlight current information and changes in practice (Pharmacists Letter, 90.2% aware; Medical Letter, 92.7% aware). Similarly, students in the elective course were aware of Web-based sources for practice guidelines (guidelines.gov, 85.4% aware; Centers for Disease Control and Prevention, 95.1% aware).

Although 97.6% (n = 40) of students indicated having Internet access at home, only 68.3% (n = 28) reported having PubMed access at home. Only 39% (n = 16) of students indicated having Internet access at work and only one quarter (24.4%, n = 10) reported having access to PubMed in the workplace.

DISCUSSION

The Institute of Medicine has embraced evidence-based practice as an essential core competency for all clinicians, including pharmacists.²⁵ Evidence-based practice involves the ability to find the best evidence, critically evaluate validity and relevance, and then apply the information to a clinical problem.²⁶ Therefore, to prepare for future practice, our students must not only learn facts, but also how to identify, evaluate, and apply information to patient care.²⁷ Current accreditation standards reflect an emphasis on development of the student as a lifelong learner and assessment of student achievement of program outcomes.³ While this elective is limited in scope in that it does not assess the ability to apply skills in the clinical setting as a student or later as a clinician, informal course feedback and the questionnaire results do provide some evidence of students' attitudes toward evidence-based medicine and comfort with literature evaluation skills.

The course provided students individual experiences with taking clinical trial results and applying them to different patient case scenarios. These experiences allowed students to recognize the strengths and limitations of an evidence-based practice and feel more comfortable reading clinical trials. The faculty members and students enjoyed the discussion format of the course and the small class size. While core courses were offered to classes of 200 students, this small elective course allowed for greater student participation. This class also gave students additional exposure to controversial or “gray” areas, reinforcing that in many situations there is no one “black and white” approach to clinical care.

In addition to third-year students' attendance, some fourth-year students precepted by elective instructors for their advanced practice experiences (APPEs) attended select class periods. Many of these advanced students informally commented on the applicability of such a course to their APPEs and wished they had taken the course previously.

Overall, students had positive attitudes toward evidence-based medicine, the application of research findings to daily practice, and the impact of evidence-based medicine on patient care, both before and after the course. These results are similar to the attitudes of pharmacists in Illinois, with 90% of pharmacists reporting a positive or somewhat positive attitude toward evidence-based medicine.⁵ Many barriers exist to the implementation of evidence-based practice, including time, access to resources, and attitude.^5,24,25,28 Results of the questionnaire indicate that the attitudes of pharmacy students in this elective course were positive and not likely to be a significant contributing barrier toward evidence-based practice. Instead, pharmacy students' positive attitudes toward evidence-based medicine may encourage them to bridge the gap between evidence and practice.

Interestingly, after the course, students were more likely to agree that pharmacists should take an active role in making patient care decisions that require them to provide recommendations or adjustments in drug therapy. It is unknown whether this change in attitude was a result of this course or the result of increased exposure to the depth and range of pharmacists' responsibilities through other coursework. Third-year students enrolled in the course were also taking required courses in Pharmacotherapeutics, Quality Assurance, and Pharmacy Operations and Management. Additionally, students could select from other elective coursework including nutrition, immunizations, physical assessment, end of life care, and clinical toxicology.

Not surprisingly, additional exposure to technical concepts and experience with literature evaluation increased student self-assessed comfort with knowledge of these technical concepts. A significant proportion of students were still not comfortable with many of the technical concepts, including relative risk, absolute risk, and odds ratio. While it is unknown whether student self-assessed comfort levels corresponded with actual knowledge of these concepts and the ability to interpret and apply them in practice, student self-assessed comfort with these concepts postcourse suggests that continued reinforcement of these skills is needed. While literature evaluation is a part of APPEs, the faculty is considering incorporation of more literature evaluation into required and elective courses as part of our current curricular revision process. Future studies may compare student self-assessed comfort to objective measures of abilities. In future course offerings, the precourse questionnaire may serve as a useful tool to identify concepts with which the class has a high degree of self-rated comfort. This may allow faculty members to provide a brief review of these topics and focus time on those concepts with lower comfort ratings.

Despite prior coursework in Drug Information and Drug Literature Evaluation, many students were still unaware of the availability of free PubMed access on the Internet. At our institution students only have access to full-text journal articles if they search and access them via PubMed while on a campus-based computer. However, students seemed to be unaware that they can execute PubMed searches offsite for later literature retrieval when on campus. This misconception will be addressed in future offerings of the elective course, as well as reinforced in the required drug information sequence for all pharmacy students.

Limitations

There are a number of limitations to the questionnaire worth noting. First, the attitudes of the pharmacy student class overall were not assessed. It is unknown whether attitudes of students in the elective course are representative of the pharmacy class or if students with an interest in evidence-based medicine may be more likely to take an elective in Landmark Trials. Future studies may assess overall student attitudes toward evidence-based medicine and potential changes in attitudes with increasing exposure throughout the curriculum. In addition to limitations to the generalizability of our findings, and the use of student self-assessment data, the lack of demographic information obtained is also a limitation. For confidentiality reasons, the survey instrument did not request identifying information from the students. Therefore, correlation with student demographics and academic performance was not possible. Further, a number of the questionnaires were unable to be paired, thereby limiting the number of usable responses. In addition, the survey was conducted over several years with varying class sizes. The potential impact of minor changes in course content and assessment over time were not assessed. Lastly, since the postcourse questionnaire was conducted immediately upon course completion, it is unknown whether the change in students' attitudes and comfort level will be sustained over time.

CONCLUSIONS

A Landmark Trials in Primary Care elective provided students with opportunities to review trials that have changed the way we practice and apply clinical trial results to patient care cases. Pharmacy students who completed this course had positive attitudes toward evidence-based medicine and the application of research findings to practice. The course increased students' self-assessed comfort with technical concepts used in literature related to evidence-based medicine. We identified that further education on free access to PubMed via the Internet may be needed. Faculty at colleges of pharmacy may consider offering similar electives which would allow students the opportunity to increase their comfort with literature evaluation and maintain a positive attitude towards evidence-based medicine in pharmacy practice.