Traditional Lecture versus Case-Based Learning in a Therapeutic Drug Monitoring Course within an Integrated Pharmacy Curriculum

Abstract

Background: Case-based learning has been shown to increase student perception and performance in multiple topics in pharmacy education. However, no studies have evaluated the impact of virtual patients and case-based learning on student knowledge and knowledge retention of therapeutic drug monitoring and dosing. Innovation: Due to a curriculum overhaul promoting integration and application-based learning, the traditional third-year (P3) therapeutic drug monitoring course was reduced from four (4) credit hours to two (2), in order to add time to pharmacotherapy and skills labs. In order to adapt to this change, the course was shifted to a case-based learning format utilizing virtual patients within a simulated electronic health record (EHR) where the course grade distribution shifted in favor of patient cases versus exam questions. An analysis of student knowledge and knowledge retention of antibiotic dosing and monitoring was conducted comparing students who completed the traditional course versus those who completed the case-based course. Findings: Despite the decrease in credit hours, there was no significant difference shown in the initial knowledge assessment between the traditional and case-based courses (87.0 vs 85.5%). Knowledge retention actually improved in the students who completed the case-based course (78.1% vs 82.5%). Conclusion: Utilizing case-based instruction to teach antibiotic dosing and monitoring was successful in preparing students for these skills during their experiential rotations. Even though students had half the instruction time, they were able to perform calculations and retain knowledge as well as students in the traditional curriculum.

DESCRIPTION OF THE PROBLEM

The use of case-based learning (CBL) has been shown to increase student perception and performance in multiple topics in pharmacy education including pharmacokinetics and self-care.^1-3 In a study by Dupuis and colleagues, a clinical pharmacokinetics course was redesigned to incorporate case-based learning to enhance group interaction and individual participation.¹ They found that students rated CBL highly compared to traditional lecturing and that students scored higher on exams where CBL was utilized compared to historical controls of the same exam material.¹ Ha and Lopez sought to evaluate the efficacy of case-based learning for teaching pharmacy students health literacy concepts.³ A laboratory CBL exercise was developed around a health literacy patient case that required students to evaluate and formulate a care plan for a patient with limited health literacy. They found that the exercise significantly improved health literacy knowledge in the students before and after the intervention and that all students either agreed or strongly agreed that CBL was effective in teaching the defined learning objectives.³ It is important to note that previous studies also highlight the critical importance of providing an overview and expectations of CBL to the students prior to implementing the teaching modality.^2,3

Application-based learning, including CBL, flipped classrooms, standardized patients, and other methodologies of active learning have also been shown to improve knowledge and knowledge retention in pharmacy students.^4-7 A study by Jacobson and colleagues analyzed student knowledge retention of opioid overdose response in students who did versus did not have an added OSCE with a standardized patient (SP) to their lecture.⁴ They found that the students who interacted with the SP demonstrated a better recollection of the order of steps for naloxone administration.⁴ Another study utilized SPs in a skills lab course to analyze student knowledge retention of insulin principles and injection technique.⁵ This study found that students who had SP interaction scored higher on one-month knowledge retention assessments and on a counseling competency.⁵

These teaching modalities are not new to health education or pharmacy, though educators continue to find ways to improve content delivery by CBL. One such method is utilizing CBL with virtual patients², a capability we have at our institution through the EHR Go® Platform (Archetype Innovations, Lehi, UT) This study evaluated the impact of virtual patients and CBL on student APPE performance following their didactic education.

STATEMENT OF INNOVATION

Due to a curricular change, the instructors had to develop a plan to deliver the most pertinent material from the traditional curriculum 4-credit therapeutic drug monitoring (TDM) course to the new curriculum’s 2-credit course. The goal of this course redesign was to develop a course where students were able to utilize cases and simulated patient care to prepare them for rotations despite the decrease in class time. Utilizing course evaluations and focus groups from the traditional curriculum students, key concepts were identified for the restructuring. Over 50% of the comments on the course evaluations mentioned the benefit of the case-days that were utilized before each exam.

The CBL TDM course was designed to meet twice weekly for one-hour sessions based on the credit hour allotment. It was designed to cover conceptual materials with practice cases on the first session each week followed by a case-day for the covered material. This study focused on the 3-week antimicrobial dosing and monitoring section of the course, because it had to be decreased from 11 contact hours to 6, and was the topic of most feedback from both students and preceptors. Antimicrobial dosing and monitoring included general concepts, vancomycin traditional dosing, vancomycin AUC/MIC dosing, and aminoglycosides. Additional topics included in the course were a refresher on clinical pharmacokinetic (PK) concepts and calculations, drug dosing in renal and hepatic diseases, cardiac agents, antiepileptics, and anticoagulation. Each topic had one dedicated week with the exception of antimicrobial dosing and monitoring and anticoagulation. Prior to initiation of the course, the course director met with each instructor for the course to discuss expectations and delivery. A guide with examples was created to ensure consistency of the CBL between instructors.

All antimicrobial practice and graded cases for case-days were created in the simulated EHR with virtual patients to best simulate an inpatient APPE. There were 3 practice and 3 graded cases. Students would complete review and answer practice cases during designated course time with facilitation by the instructor. They then completed the graded case within 24 hours. One graded case was a group submission to the LMS that required a pharmacokinetic consult note uploaded along with a copy of their calculations. Two graded cases were converted to multiple choice questions to be completed by each individual student. Cases were released to students at the beginning of the practice case session. The virtual patient cases differed from review cases in the traditional curriculum course in three (3) ways; they were not provided in advance, they were in the simulated EHR program, and they were graded. In the previous curriculum, practice cases were on paper and used solely for exam preparation.

In the traditional curriculum, over 85% of time was spent on discussing concepts and working practice problems in class for the antimicrobial material. Students were not accountable for submitting any work outside of exams, leaving no true way to score the application of material or ensure understanding prior to the exam. The grade breakdown for the traditional course was comprised solely of exam and quiz scores, 80% and 20% respectively. As the new course was created, the assessment weighting was shifted to reflect the emphasis on applying the material to patient cases. Four graded cases made up 40% of the final grade, with three cases containing antimicrobial dosing as mentioned previously. The remainder of the assessment weighting was split between exams (50%) and pre-class assignments/quizzes (10%).

Design

Third-year pharmacy students (P3) were invited to participate in the study during the final year of the traditional course (Spring 2019) and the first year of the redesign (Spring 2020)(Figure 1). All participation was voluntary. The primary objective of the study was to compare knowledge scores on an antimicrobial dosing and monitoring assessment immediately following course delivery among students in the traditional TDM course versus CBL course revision. Secondary objectives were to compare scores between groups on conceptual versus calculation questions on the knowledge assessments, to compare knowledge retention assessment scores among students during their fourth professional year (P4) at the beginning of their Internal Medicine rotation, and to evaluate student perceptions towards course delivery and confidence in the material upon completion of the course and knowledge retention assessment.

Figure 1.

Study Design

Data Collection

Students were asked to complete a knowledge assessment at the end of the P3 year. The questions included on this assessment contained case-based concepts and calculations. The questions utilized were exam questions from previous years that had demonstrated high item statistics, but had not been previously seen by the students in the study. These students were asked to take the same assessment prior to their required Internal Medicine Advanced Pharmacy Practice Experience (APPE) during their P4 year. The timing of the knowledge retention assessment could vary between students depending on when they had their Internal Medicine APPE, but this was chosen to avoid confounding due to the rotation, which is a required two-month learning experience at our institution. All assessments were conducted with Qualtrics (Qualtrics Labs Inc., Provo, UT) and no baseline demographics were collected to protect student anonymity. Each month, three days prior to beginning the Internal Medicine APPE the survey link was sent to students. A separate email was also sent to their preceptors to encourage study completion.

Students were also asked to complete a perception survey with the knowledge retention assessment. All assessment scores were compared using Student T-tests in JMP-14 PRO (SAS. Cary, NC). Perception surveys were reported as means from a 5-point Likert scale and compared using a Kruskal Wallis test for non-normally distributed data. Upon completion of the study, results were shared with the course instructors to adjust the course as necessary. This project received approval from the Campbell University IRB.

Findings

Nearly every student enrolled in the two course opted to participate in the study with 100 (96%) students in the traditional course and 96 (94%) students in the CBL course completing the initial knowledge assessment, which occurred immediately following the course. There was no difference in mean knowledge assessment scores between the traditional and CBL course groups (Table 1). When items were categorized as concepts and calculations, there was no difference between the traditional and CBL groups in scores on concept questions, however the CBL course group scored higher on calculations questions (90.3 vs 86.9%, p=0.032).

Table 1.

Knowledge and Knowledge Retention Assessment Scores

Assessment	Traditional Course mean + SD	CBL Course mean + SD	95% CI, p-value
Knowledge Assessment	N=100	N=96
Exam Score	87.2 + 7.1	88.4 + 9.9	[-0.59, 4.48], p=0.33
Concepts	87.9 + 10.2	86.8 + 11.4	[-0.24,5.37], p=0.14
Calculations	86.9 + 6.8	90.3 + 9.3	[-6.23,-1.06], p=0.032
Knowledge Retention	N=72	N=60
Exam Score	80.2 + 8.9	84.6 + 10.1	[-5.15,-0.86], p<0.001
Concepts	81.0 + 8.3	82.8 + 10.9	[-1.03, 3.56], p=0.078
Calculations	78.9 + 9.6	87.0 + 9.4	[-6.92,-1.86], p<0.001

A smaller percentage of students responded to the knowledge retention assessment and perception surveys, with 72 (69%) in the traditional course and 60 (63%) in the CBL course. Mean time in months between the initial knowledge assessment and knowledge retention assessment was similar between groups (4.1 vs 4.5, p=0.37). Knowledge retention assessment scores were significantly lower in the traditional course group as compared to the CBL course group. This was primarily driven by scores on the calculations questions, which can be seen in Table 1.

Likert response statements were provided (1=strongly disagree to 5=strongly agree). The traditional course group had higher agreement having enough opportunities in class to find out if they clearly understand the material (4.11 vs 3.51, p=0.009). The CBL course group reported higher agreement that they had opportunities to discuss concepts with the instructor and other students (3.88 vs. 4.22, p=0.013) and that the course offered a variety of ways to learn the material (3.53 vs. 4.12, p=0.003). There were other areas that demonstrated higher perception scores in the CBL group, although not statistically significant (e.g. confidence in clinical practice application). Neither group demonstrated high levels of confidence in their agreement with mastery of the subject (3.40 vs. 3.37).

CRITICAL ANALYSIS

The course redesign was successful in ensuring student knowledge and knowledge retention of antimicrobial dosing and monitoring, despite a significant decrease in class time. Students had similar scores on the knowledge assessment at the end of the respective courses. Interestingly, when broken down by question type, the CBL course performed better on the calculations questions than the traditional course. This is most likely due to the real-time application in class with simulated patients and the high assessment weight of the graded cases. The CBL course also performed slightly lower on the concept questions, but it was not statistically significant. However, this could indicate that if time should be redistributed in the new course, it may need to be allocated to better establishing the concepts behind the application.

The knowledge retention scores were similar to the initial, post-instruction assessment, however, the overall assessment scores were higher in the CBL course. Once again, there was no difference in the score on concept questions between groups, but the CBL course had significantly higher scores on the calculations questions compared to the traditional course group. A key limitation was that there was no control for students learning on APPE rotations prior to their Internal Medicine APPE (where the retention test was given), and no way to account for the knowledge gained on rotation. However, the mean time from knowledge assessment to the knowledge retention assessment was similar between groups.

This analysis has additional limitations. The anonymity of the assessments and surveys does not allow for tracking individual progression, and fewer students completed the knowledge retention assessment and perception survey. Also, with the complete course redesign, it is difficult to pinpoint which component or change led to the positive results.

Key Takeaways and Next Steps

This analysis supports previous literature showing the benefit of CBL in clinical pharmacokinetics.¹ It further expands on this by showing a CBL approach, with the addition of a simulated EHR, may improve knowledge retention as students complete their rotation experiences. Core components of our CBL course redesign that contributed to this success include setting clear expectations and guidance prior to utilizing CBL, allowing for group and instructor interaction while completing cases, and promoting critical thinking through assessing knowledge application with high stakes case-based assessments.

Going forward, the CBL delivery with a simulated EHR will be continued based on the students’ abilities to perform at or above the traditional course despite the decrease in instruction time. In order to ensure students understand why they are performing certain calculations, more time on the first session of each week will focus on conceptual material.