Teaching Independent Decision-Making Through a Computer-Based Drug Utilization Review Simulation

Abstract

Background: Drug utilization review (DUR) skills are important for pharmacists across all settings. Computer-based DUR simulations to teach student pharmacists are currently scarce. This article describes a computer-based DUR simulation that required limited faculty and financial resources and was implemented in collaboration among 3 institutions. Objective: To describe the innovation of a computer-based DUR simulation and its impact on pharmacy students’ knowledge and confidence of DUR skills. Methods: This pre-post educational study assessed a computer-based DUR simulation that replicated the DUR process in dispensing systems. First- and third-year pharmacy students at 3 institutions were guided through simulated patient cases with various medication-related problems. The self-paced activity provided students with immediate, formative feedback and rationale for each option after an attempt was made in lieu of faculty intervention. Students completed pre-and post-assessments to evaluate changes in knowledge and confidence. Knowledge was assessed by comparing results of multiple choice and matching questions on the pre- and post-assessments. Confidence was assessed by the change in self-reported confidence scale measurements. Results: Students at all institutions (N = 405) had nonsignificant changes in knowledge scores from the pre-assessment to the post-assessment, with the exception of 1 question. All confidence survey questions significantly improved from pre- to post-assessment. Conclusion: The DUR educational innovation had a nonsignificant overall impact on students’ knowledge but significantly improved confidence in their abilities. Skills-based instruction provides additional practice to increase student confidence.

Background

Drug utilization review (DUR) is the process of reviewing patient medication data to ensure safe, effective, and appropriate use.^1,2 As medication experts, pharmacists conduct DUR to improve the quality of patient care in various pharmacy practice settings. The process of DUR identifies medication-related problems to prevent unnecessary or inappropriate drug therapy, improve the quality of care for populations and individuals, reinforce efficient resource use, prevent adverse drug reactions, and improve medication effectiveness. ¹ There are 3 categories of DUR: prospective, concurrent, and retrospective. ¹ Pharmacists complete prospective DUR when verifying a prescription order before the patient receives their medication. Whereas, concurrent and retrospective DUR occur during and after prescription processing and verification, respectively. While the term DUR is more common in community or outpatient pharmacy practice, all pharmacists review medication profiles and order to identify potential medication-related errors. In this article, DUR will be used to also encompass prospective pharmacist review of medication profiles and prescriptions.

Prospective DUR by pharmacists can be supported by clinical decision-support software. In pharmacy practice, dispensing systems can identify potential medication-related problems, however, clinical decision-support software widely varies and has significant limitations.^3,4 One study, which strategically evaluated pharmacy dispensing software from 24 different vendors, identified the failure of these systems to detect approximately 1 in 7 clinically significant medication-related problems. ³ Alternatively, repeated clinical decision-support alerts in dispensing software can lead to alert fatigue and the failure to act. ⁴ Practicing pharmacists must be competent in critical thinking to apply patient-specific information to automatic DUR alerts. Thus, pharmacy educators must build students’ critical thinking skills through thoughtful, guided learning environments. ⁵

DUR, in its broadest definition, is included in the curriculum at many institutions. In 2009, Saverno et al ⁶ called for studies to evaluate students’ ability to perform DUR, and 12 years later Mospan and Alexander ⁷ found DUR is still a self-identified area of weakness for pharmacy students. However, the inclusion of application-based practice through the use of technology is noted as an area of need in skills laboratory instruction. Gupta et al ⁸ noted that only 15 schools use technology-based DUR simulations to train and assess students on this topic, despite the fact that conducting DUR was identified as an essential skill for the patient care provider domain. ⁹ It was noted that tool validation, more robust student feedback, and incorporation of technology into DUR training are critical areas for improvement. ⁸ To date, few relevant and recent articles have been published that discuss or outline guidance for DUR education.^6,7,10

To answer this call and fill a gap in the literature, a multi-institution teaching innovation was designed to prepare students to perform DUR. The innovation centered on essential constructs of critical thinking and immediate, formative feedback. Previously, at the originating institution, DUR skills were taught to first-year Doctor of Pharmacy (PharmD) students through worksheet-based patient cases with written feedback provided to students the following week. This delayed feedback limited students’ ability to challenge their thinking and change strategies during the activity; In addition, it lacked technology integration of DUR found in pharmacy practice. Faculty at the originating institution converted this activity, utilizing Microsoft^® PowerPoint, to include a formative feedback model that provides timely assessment after learners have attempted a solution to each question within various patient cases. The learning activity was shared with 2 other institutions to include another group of first-year and an additional group of third-year PharmD students. The objective of this manuscript is to describe the development and implementation of the computer-based DUR simulation. In addition, it will describe how first- and third-year pharmacy students’ DUR knowledge and confidence were assessed, while highlighting the challenges in evaluating student learning and limitations in the educational activity for future studies.

Methods

Development of the DUR Simulation

The computer-based DUR simulation was designed in PowerPoint (Microsoft^®, version 16.61) and shared with students in a PowerPoint Slide Show (.ppsx) file that prevented them from editing the file and seeing the upcoming slides. Students progressed through 4 cases for first-year students and 7 cases for third-year students (Table 1). Each patient case included a patient image, basic demographic information, screenshots of patient profiles in pharmacy dispensing software, and screenshots of the DUR alert from the dispensing system. Adding the screenshots from a pharmacy dispensing software, the activity simulates DUR management systems seen in pharmacy practice The DUR simulation included 5-10 questions per case, with questions varied per patient case and learner level. First-year students answered questions that utilized knowledge from prescription dispensing, drug-information resources, and pharmacy law and regulations (eg, day supply, pharmaceutical calculations, maximum doses, drug interactions, and drug monitoring). Third-year students answered the same questions as first-year students plus additional questions integrating pharmacotherapy knowledge (eg, therapeutic appropriateness and alternative therapy recommendations). Insurance adjudication and prescription costs were not included in this simulation. All options for questions were hyperlinked to a slide that stated “correct” or “incorrect” with detailed justification. If students chose the wrong response, they were directed back to the question to try again; students had unlimited attempts for each question and were not evaluated on their choices within the activity, creating a low-stakes, self-paced learning environment. If students chose the correct answer, they then progressed to the next question or patient case.

Table 1.

DUR Simulation Case Descriptions, PPCP, and Resulting Interventions.

Medication-related problem	PPCP phases addressed	DUR intervention	DUR result	Pharmacist intervention
Drug-food interaction between simvastatin and grapefruit juice	Collect, Assess, Plan	Medication review or Patient consulted	Filled prescription as is	Patient counseled regarding interaction a
Drug-drug interaction between ciprofloxacin and warfarin	Collect, Assess, Plan, Implement, Follow-up	Prescriber consulted	Prescription not filled	Contact provider to notify of interaction a Call simulated provider and make recommendation for alternative antibiotic regimen b
High dose of warfarin	Collect, Assess, Plan	Dosing evaluation/ determination or Medication review	Filled prescription as is	INR check, determine warfarin dose and follow-up monitoring appropriate a
Medication overuse/ refill too soon with lorazepam	Collect, Assess, Plan	Patient consulted	Prescription not filled	Patient counseled on overuse of controlled substance a
Drug-drug interaction between ondansetron and sertraline: serotonin syndrome warning	Collect, Assess, Plan	Patient education/instruction	Filled prescription as is	Simulated patient educated on serotonin syndrome b
Subtherapeutic daily dose for pediatric antibiotic	Collect, Assess, Plan, Implement	Prescriber consulted	Filled prescription, with different dose	Gather information from patient’s mother, contact provider to change dosing, document change on prescription b
Drug-drug interaction between clarithromycin and lovastatin	Collect, Assess, Plan, Implement	Prescriber consulted	Filled prescription w/ different medication	Recommend a new regimen for H. Pylori, noting patient allergy profile b

Abbreviations: DUR, drug utilization review; PPCP, pharmacists’ patient care process.

^aLevel 1 complexity, first- and third-year PharmD students.

^bLevel 2 complexity, third-year PharmD student. Faculty at Institution A developed the DUR simulation to introduce the concept and skills necessary for performing DUR with 3 considerations in mind: a patient-centered approach to DUR system alerts, immediate and formative feedback to students, and minimal resources required.

The structure of the activity centralized around the Pharmacists’ Patient Care Process (PPCP) to provide patient-centered focus. ¹¹ Patient cases were central to the content and skills in the DUR simulation. All patient cases incorporated the phases of “Collect,” “Assess,” and “Plan” from the PPCP, while 3 cases addressed “Implement,” and 1 case included “Follow-up” utilizing a simulated provider recommendation (Table 1).

Immediate and formative feedback to students was provided through the hyperlinks to “correct” or “incorrect” with rationale. Formative feedback focuses on improvements in the students’ process instead of end cumulative results of the task and provides elaborate rationale in manageable units to enhance learning. ¹² In addition, the most effective forms of feedback provide cues or reinforcement to learners, are in the form of computer-assisted instructional feedback, and provide information on correct rather than incorrect responses. ¹³ While the DUR simulation provided feedback on correct responses, faculty also felt that it was important to immediately challenge students’ thought processes when they chose an incorrect response since pharmacy practice is governed by laws, regulations, and clinical knowledge. Thus, students who chose incorrectly were redirected to try the question again and had to select the most correct answer before progressing in the activity.

Minimizing faculty and institutional resources was another goal of the DUR simulation and therefore it was developed to only require 1 to 2 activity facilitators and use freely accessible platforms to limit technology resources. Microsoft^® PowerPoint was used since it’s a platform accessible by most educators and students through their institution, it allows for hyperlinking of answer choices to slides within the document, it doesn’t require extensive training to use, and it allows for file sharing in a locked format (.ppsx). Using a widely available application, the activity is able to be shared with other pharmacy educators and adapted to any cohort size with a range of previous exposure to DUR management systems. Since students received immediate feedback during the simulation, a large group debrief after the simulation was not required, reducing the number of facilitators needed for a synchronous laboratory session. Furthermore, the provision of individualized feedback independent of faculty allows for the DUR simulation to be run as an asynchronous activity.

Implementation of the DUR Simulation

The DUR simulation took place individually at 3 public institutions with 4-year PharmD curricula within a 1.5 or 2 credit-hour pharmacy skills laboratory course. All institutions have longitudinal sequences of pharmacy skills laboratory courses that progress in content and complexity. Institutions A and B provided this activity to first-year students, whereas Institution C provided it to third-year students. Laboratory sessions ranged from 3 to 4 sessions per week with each session being either 110 or 170 minutes. Students were given 50 minutes for completing the simulation as an in-person activity or were able to complete the simulation on their own time as an asynchronous activity.

Prior to the laboratory session in which the DUR simulation and assessments took place, students completed published readings about DUR^1,14 and a document containing practice-based DUR intervention codes and examples (please see Authors’ Note). No didactic instruction was offered within the pharmacy skills laboratory courses for this curricular content. Before starting the DUR simulation, students completed a preassessment to determine their baseline knowledge and confidence levels in completing DUR. Next, students individually engaged in the DUR simulation through the .ppsx file. Students were required to progress through each patient case using PPCP to select the most appropriate DUR Intervention Code and Result Code. One example is the drug-drug interaction case, where the DUR rejection stated “Drug-Drug Interaction.” Students reviewed the pharmacy dispensing system patient profile to determine that the patient was prescribed ciprofloxacin for a urinary tract infection while also taking warfarin. Students used drug information resources to evaluate the clinical significance of this interaction. The expected action was to contact the provider. Third-year students at Institution C called a Google Voice phone number and left a simulated voicemail message to the prescriber with a recommendation for a more appropriate antibiotic for the given indication that did not interact with the patient’s warfarin. Whereas first-year students at Institutions A and B filled out fax provider communications based on an alternative antibiotic recommendation provided to them. Students were required to document their final answers for each question on a worksheet or quiz within their learning management system (LMS). The final answer document was either graded for accuracy (Institution A) or for participation (Institutions B and C).

Assessment of the DUR Simulation

The DUR simulation was designed, implemented, and evaluated over Fall 2018, Spring 2019, and Fall 2019 at the 3 institutions. Results and feedback from Fall 2018 and Spring 2019 led to adjustments in the evaluation of the DUR simulation during Fall 2019. Incomplete pre- or post-assessments were excluded from analysis for all institutions. Data analysis was conducted in SPSS, v23 (SPSS Inc, Chicago, IL) with p < 0.05 denoting statistical significance. This study was approved as exempt by Institution A’s Institutional Review Board (IRB).

During Fall 2018 and Spring 2019, Institutions A and B, respectively, implemented and evaluated the DUR simulation to assess if it was effective for first-year students. Evaluation of the simulation’s effectiveness in first-year students included an assessment of knowledge and confidence. Students completed a preassessment prior to the activity with 4 multiple-choice and 1 matching question on DUR knowledge and application (Table 2: Q1-5) and 5 survey questions on confidence in their ability to conduct DUR, repeating the same questions for the post-assessment. Study investigators examined the performance on the pre- and post-assessments, submitted anonymously and unlinked per student, per the initially approved study parameters of the IRB. Knowledge was assessed by comparing average scores (out of 100 points) on the 5-question knowledge assessment before and after activity completion and were analyzed by independent t-tests for each school. Confidence was assessed by the change to the survey questions based on a 4-point Likert scale (strongly agree to strongly disagree). Responses were coded into binary positive responses (strongly agree or agree = 1) or negative responses (strongly disagree or disagree = 0), the number of positive responses per question for the pre- and post-assessment were totaled, and the change in positive responses was analyzed by Wilcoxon rank sum.

Table 2.

DUR Simulation Knowledge Assessment Questions.

Question number	Question and solution
Case 1: While processing Mrs. Helen Parr’s prescription for cyclobenzaprine (a muscle relaxant), you get a DUR alert of “HD—high dose”. Her prescription is: cyclobenzaprine 10mg tablets—ii po tid #90 tablets. You consult Lexicomp, the maximum dose is 10mg tid. You decide to call Dr. Mode to explain the maximum dose recommendation. Dr. Mode asks for you to modify the prescription to “i po tid #60 tablets”. You fill the prescription with this modification.
1	What is the most appropriate DUR intervention code for this alert? a. 00—No intervention b. CC—Coordination of care c. DE—Dosing evaluation/ determination d. TH—Therapeutic product interchange
2	What is the most appropriate DUR result code? a. 1B—filled rx as is b. 1D—filled rx with different directions c. 1G—filled rx with prescriber approval d. 2A—rx not filled
Case 2: While processing Mr. Lucius Best’s prescription for Prinivil, you receive a DUR alert of “non-formulary drug”. You call Mr. Best’s prescription insurance plan and learn that the plan prefers generic lisinopril. Since Dr. Mode has selected DAW, you call her and inform her of the insurance’s preference. She states she didn’t mean to sign on the DAW line and it is okay to fill the prescription as the generic.
3	What is the most appropriate DUR intervention code for this alert? a. AS—Patient assessment b. CC—Coordination of care c. DE—Dosing evaluation/ determination d. GP—Generic product selection
4	What is the most appropriate DUR result code? a. 1B—filled rx as is b. 1D—filled rx with different directions c. 1G—filled rx with prescriber approval a   IH—brand to generic change b d. 2A—rx not filled
5	Match the following DUR type to when the review of drug therapy takes place: B Concurrent C Retrospective A Prospective a. Review done before medication is dispensed b. Review done during medication course c. Review done after medication is dispensed
6	Match the following DUR Intervention Codes to the correct description: (Phase 2 c ) C Patient consulted A Patient education B Patient referral a. The pharmacist determines that they should educate the patient about important medication information b. The pharmacist determines that the patient needs to be seen by another healthcare provider for resolution of the DUR alert c. The pharmacist determines that they need more information from the patient to properly address the DUR alert

Abbreviations: DAW, dispensed as written; DUR, drug utilization review.

^aThis was the available answer choice for Institution A and B during Phase 1 on the pre- and post-assessment for knowledge. It was also the available answer choice for Institution C for the pre-assessment.

^bThis was the available answer choice for Institution A during Phase 2 on the pre- and post-assessment for knowledge. It was also an available answer choice for Institution C on the post-assessment.

^cThis question was available for Institution A during Phase 2 on the pre- and post-assessment for knowledge, however, it was only available to Institution C on the post-assessment during Phase 2.

Based on the first-year student results, the DUR simulation and evaluation was adjusted for Fall 2019. Institution C increased complexity of the patient cases to align with third-year students’ learning level. An additional matching question was added to the knowledge assessment questions (Table 2: Q1-6) and the confidence survey was changed to a scale of zero to 100 for Institutions A and C. An amendment to the IRB application also allowed the ability to link pre- and post-assessment data by student. Institution B did not participate during Fall 2019 as the course in which the DUR simulation was implemented is only offered in the spring semester. During Fall 2019, pre-knowledge assessments were distributed through each institution’s LMS; confidence and post-knowledge assessments were distributed via Qualtrics. Pre- and post-knowledge was assessed as change in mean score (out of 100 points) for each school by paired t-tests. The average of each question from the pre- and post-confidence survey, on a scale of 0-100, was analyzed by paired Wilcoxon signed rank sum as the scores did not have a normal distribution.

Results

During Fall 2018, Institution A had 137 students enrolled in the skills laboratory course the activity took place in, of which 131 (response rate: 96%) students fully completed the pre- and post-assessments. Institution B had 81 students enrolled during spring 2019, 80 (response rate: 99%) completed the pre- and post-assessments during Spring 2019. The total mean scores for the knowledge postassessment remained similar to the pre-assessment for both schools (96.6 + 6.4 vs 96.7 + 7, P = 0.64; 91.1 + 11.9 vs 96.1 + 7.5, P = 0.06). However, the number of positive responses significantly increased for all of the confidence questions from the pre- to post-assessment (Table 3). For example, Institution B had 40 positive responses (strongly agree or agree) for the question “I am confident in my ability to interpret DUR alerts.” in the pre-assessment that increased to 74 positive responses to the same question in the post-assessment (p < 0.001).

Table 3.

Number of Positive Responses on Student Confidence Survey for DUR Ability During 2018-2019 Academic Year.

Questions a	Institution A			Institution B
	Pre n = 137 (%)	Post n = 131 (%)	P-value	Pre n = 81 (%)	Post n = 81 (%)	P-value
I am confident in my ability to interpret DUR alerts.	122 (89)	129 (98)	0.0016 b	40 (49)	74 (91)	< 0.001 b
I am confident in my ability to select the appropriate DUR intervention and code correctly.	108 (79)	126 (96)	< 0.001 b	29 (36)	63 (78)	< 0.001 b
I have a systematic process for approaching the DUR before dispensing a medication.	65 (47)	115 (88)	< 0.001 b	21 (26)	48 (59)	< 0.001 b
I feel confident in my ability to evaluate the clinical significance of drug interactions.	89 (65)	119 (91)	< 0.001 b	43 (53)	73 (90)	< 0.001 b
I feel confident in my ability to address drug interactions that I identify during the DUR process.	91 (66)	125 (95)	< 0.001 b	49 (60)	67 (83)	0.0018 b

Abbreviation: DUR, drug utilization review.

^a4-point Likert scale (strongly agree, agree, disagree, strongly disagree).

^bWilcoxon rank sum was used to determine significance, defined as P < 0.05, between the pre- and post-assessments for change in number of positive responses (“strongly agree” and “agree”).

In Fall 2019, Institution A had 103 students enrolled in the skills laboratory course the activity took place in, of which 56 (54%) of students fully completed both pre- and post-assessments. Institution C had 84 students enrolled with 66 (79%) students that fully completed both pre- and post-assessments. The change in mean paired scores for the knowledge pre- to post-assessment was nonsignificant at both institutions (86.8% + 16.3% vs 87.9% + 15.1%, P = 0.71; 89.4% + 12.8% vs 90.6% + 14.1%, P = 0.47). All of the mean responses for the confidence survey questions increased significantly from the pre- to post-assessment (Table 4).

Table 4.

Number of Positive Responses on Student Confidence Survey for DUR Ability During 2019-2020 Academic Year.

Questions a	Institution A (n = 56)		Institution C (n = 66)		Fall 2019 combined (n = 122)
	Pre, mean (SD)	Post, mean (SD)	Pre, mean (SD)	Post, mean (SD)	Pre, mean (SD)	Post, mean (SD)	P-value
To interpret DUR alerts	49 (30)	83 (16)	54 (22)	84 (11)	52 (26)	84 (14)	< 0.001 b
To appropriately document DUR alerts	41 (30)	81 (17)	47 (23)	85 (12)	44 (27)	83 (14)	< 0.001 b
To systematically approach DUR alerts before dispensing a medication	46 (28)	79 (20)	61 (21)	85 (12)	54 (26)	82 (16)	< 0.001 b
To evaluate the clinical significance of drug interactions	46 (30)	80 (19)	58 (21)	85 (11)	52 (26)	83 (15)	< 0.001 b
To address drug-related problems identified during the DUR process	42 (31)	82 (16)	51 (22)	84 (13)	47 (27)	83 (14)	< 0.001 b

Abbreviation: DUR, drug utilization review.

^a“On a scale of 0-100 (0 = low confidence, 100 = high confidence), what is your self-rated ability to do the following items BEFORE or AFTER the DUR activity?.”

^bPaired Wilcoxon signed rank sum was used to determine significance, defined as P < 0.05, between the average response on pre- and post-assessments for Institutions A and C combined.

Discussion

At all institutions, the DUR simulation appeared to improve student confidence in DUR skills while having no statistically significant impact on DUR knowledge. This could be interpreted that students understood DUR content with the pre-activity materials and prior coursework and/or experiences but were not confident in applying that knowledge until after activity completion. One explanation for the improvement in student confidence is their ability to experience immediate, formative feedback during the activity. Formative feedback has been associated with reflective writing to positively impact self-directed learning, relativity to practice, and increase in student confidence. ¹⁵ This activity applied formative feedback to allow the learner to critically think through medication-related problems, determine a DUR intervention, and receive immediate feedback on their choice. Shute et al ¹² provides key guidelines that state formative feedback on difficult tasks should be provided immediately; In addition, students prefer formative feedback and the ability for self-guided learning.^16,17 This is compelling due to variations in prior pharmacy dispensing experience among pharmacy students: by using this self-guided learning technique, students can work on the cases at their own speed versus a small group discussion which may be naturally paced by a student with more experience. Thus, the DUR simulation provided self-directed learning for a practical activity relevant to performing DUR in pharmacy practice.

An alternative interpretation is that the tools utilized to gauge students’ learning with the DUR simulation were not sensitive enough to identify knowledge changes in the student population. Removing or changing the pre-activity assignment could allow for an evaluation of how students’ previous knowledge or learning preference impact the results. While there was no instruction on DUR in the pharmacy skills laboratory courses before the DUR simulation, students could have been introduced to the concept of DUR prior to this activity in another course of their core curriculum. This could explain why there was a high level of understanding DUR on the pre-assessment and no change was demonstrated between the pre- and post-knowledge assessments. In addition, information regarding students’ prior experience with DUR in internships or jobs could be collected, as prior experience is a confounder that was not previously considered. The inability to assess the impact of the DUR simulation on student knowledge is the most impactful limitation. The high baseline knowledge score on the pre-assessments caused difficulty in demonstrating meaningful impact on knowledge.

An additional limitation in evaluating this learning activity is the exclusion of students with incomplete surveys for both assessments which lowered the number of participants, especially during Fall 2019. Using assessment features in an LMS, such as quizzes, seems to be efficient for grading and provides class-wide analytics. However, nonrequired assessments (ie, post-assessment) led to lower response rates than required assessments (ie, pre-assessments were required pre-class quizzes). Since the response rate for Institution A in Fall 2019 was below 70%, the National Center for Education Statistics recommends a nonresponse bias analysis to be conducted. This was considered, however, the results from Fall 2019 at Institution A for student confidence are consistent with the Fall 2018, Spring 2019, and Fall 2019 Institution C’s results. In Fall 2018 and Spring 2019, while the survey response rates were higher, the study design intentionally collected anonymous data to allow for an accurate representation of skills and confidence. Therefore, the data could not be linked for analysis by individual students, precluding paired statistical tests. Instead, more conservative independent samples t-test and Wilcoxon rank sum were performed.

There is 1 major limitation to the DUR simulation itself that should be considered when utilizing it as an educational tool. The DUR simulation in PowerPoint doesn’t allow for the ability to track a student’s selections and progression through the activity, therefore, faculty are unable to determine areas that the students are struggling with or how many attempts a student is requiring before progression. Therefore, the DUR simulation, and others developed in PowerPoint, can be used for effectively introducing or practicing skills but not for skill competency assessments. If the DUR simulation is a didactic portion of the education students receive on DUR, supplemented by experiential learning experiences, it is appropriate to continue the use of the simulation for low-stakes learning activities in pharmacy skills laboratory courses. If DUR skill competency is being assessed, the DUR simulation is inappropriate, as currently developed, since assessment is not built into the simulation and students have multiple opportunities to arrive at the correct answer.

Conclusion

Despite the challenges in evaluating the DUR simulation, it is easily adaptable, provides immediate and formative feedback to students, and appears to improve students’ confidence in their DUR skills. For these reasons, all 3 institutions continue the DUR simulation as a laboratory activity with some variations in pre-activity preparation (asynchronous lecture vs readings) and assessments (paper vs LMS quiz, required vs nonrequired assessments).

All activity and assessment materials are shared to encourage adaptations by other pharmacy educators and expansion to other skills (see Authors’ Note below).