Online Virtual-Patient Cases Versus Traditional Problem-Based Learning in Advanced Pharmacy Practice Experiences

Abstract

Objective. To evaluate the efficacy of faculty-led problem-based learning (PBL) vs online simulated-patient case in fourth-year (P4) pharmacy students.

Design. Fourth-year pharmacy students were randomly assigned to participate in either online branched-case learning using a virtual simulation platform or a small-group discussion. Preexperience and postexperience student assessments and a survey instrument were completed.

Evaluation. While there were no significant differences in the preexperience test scores between the groups, there was a significant increase in scores in both the virtual-patient group and the PBL group between the preexperience and postexperience tests. The PBL group had higher postexperience test scores (74.8±11.7) than did the virtual-patient group (66.5±13.6) (p=0.001).

Conclusion. The PBL method demonstrated significantly greater improvement in postexperience test scores than did the virtual-patient method. Both were successful learning methods, suggesting that a diverse approach to simulated patient cases may reach more student learning styles.

INTRODUCTION

In 2010, the American Society of Health-System Pharmacists-Accreditation Council of Pharmacy Education Task Force challenged colleges and schools of pharmacy to integrate entry-level competencies into the PharmD curricula, which are needed for pharmacy practice in hospitals. The task force suggested that teaching styles during advanced pharmacy practice experiences (APPEs) should include innovative methods that cover the following objectives: (1) given a real or simulated pharmacy-related problem, demonstrate effective problem-solving skills; and (2) given a real or simulated case, demonstrate an appropriate level of clinical knowledge related to medications and therapeutics in making decisions or recommendations.¹

Colleges and schools of pharmacy have met these challenges by devising alternative learning strategies for their students. Problem-based learning (PBL) and virtual-patient case simulation are documented educational techniques that have had success in preparing students to focus learning on core information that is relevant to real clinical scenarios and adaptive feedback.^2.3 Problem based-learning methods, which use realistic patient-based cases and facilitate self-initiative, critical-thinking, and problem-solving,⁴ are based on the premise that self-directed students experience motivated learning in small groups in which peers seek out questions and answers. Barrow’s taxonomy ranks PBL highest in accomplishing 4 educational objectives: structuring of knowledge for use in clinical contexts, development of an effective clinical reasoning process, development of effective self-directed learning skills, and development of increased motivation for learning.⁵ These objectives are usually accomplished in small groups with a facilitator, typically a faculty member, leading the activity. Activities are designed in a variety of ways. One method is the presentation of a patient case with several therapy options. Each choice leads to modifications in subsequent therapy choices for the patient. Students are prompted to choose a therapy plan and then discuss patient-specific parameters and the most current medical evidence. Problem-based learning in medical education extends beyond a specific educational technique, presenting in many forms in the literature. The variability in definition and execution of technique has led to a divergence in reports on the effectiveness of PBL in medical, nursing, and pharmacy education.^4-6

A systematic review of PBL in medical education led to the exploration of 4 outcome variables among learners: participant’s knowledge, performance, and satisfaction; and patient’s health.⁶ This meta-analysis found that PBL was not superior to other methods of medical education. Conversely, in 2011, a review of PBL in a clinical setting for nursing students concluded that PBL improved critical-thinking skills in a clinical setting.⁷ The variability in use of the PBL techniques impeded the researcher’s ability to assess the effectiveness of PBL across the studies. Adaptive forms of PBL have included the use of Internet platforms as well as traditional small-group settings. The adaptation in colleges and schools of pharmacy has ranged from implementation of PBL in disease-state management courses as well as elective and basic science courses for over 20 years.^5,8-11

One of the first universities to use an online virtual case as a longitudinal approach assessed pharmacy students’ learning styles. Two groups were compared to evaluate proficiency in basic knowledge retention, with the finding that all students in the virtual arm gave positive feedback. The authors said that the study helped them understand the disease states of their patients as well as the continuity of care between retail and hospital practice settings.¹² Since the advent of Web-based tools, researchers have devised methods using online clinical scenarios that use virtual patient simulation.

Decisionsimulation (Decisionsimulation, LLC, Chadds Ford, PA), formerly known as VpSIM, is an online simulation platform with the unique ability to mimic a real hospital scenario using looped, branch-learning. This method of learning uses different queries and responses so that students will experience alternative outcomes and/or consequences. They can then return to see if their decision impacted their patients with respect to positive or negative health outcomes. The students’ clinical ability then becomes enhanced in patient-directed care. A study in which 1 pharmacy school combined active-learning lessons using an online platform to simulate an intensive care unit demonstrated consistent improvement in student practicum scores.¹³

Colleges and schools of pharmacy do not have the criteria to implement such a system because a study comparing simulation-based learning, specifically virtual patient simulation, to PBL, has never been conducted. One study found that simulation-based learning is superior to PBL in developing critical-thinking skills among medical students.¹⁴ This study led the current investigators to compare the relative effectiveness of a virtual-patient case to PBL among P4 students during their first APPE.

DESIGN

This study received institutional review board approval prior to initiation. All participants received instruction on the study and signed an informed consent to participate in either the virtual-patient simulation group or the PBL group. This was a single-center, randomized, parallel-group design conducted at a private college of pharmacy. The PBL arm of the study was conducted in the patient-assessment laboratory, and the online virtual-patient simulation was conducted in the pharmacy computer laboratory. All students had completed all courses in the pharmacy curriculum that were required for entrance into the APPE program.

Students were recruited for participation during the first 3 weeks of their first APPE, with 21% concurrently enrolled in an internal medicine practice experience and the remaining 69% enrolled in either institutional, community, or ambulatory care practice experiences at the time of the study. One hundred nineteen students participated (Table 1). Each student participated only once. On arrival at the study location, students were randomly assigned to participate in branched-case learning to reflect negative and positive outcomes of pharmacotherapy decisions in either the virtual-patient case or in a 6- to 8-person PBL small-group discussion.

Table 1.

Demographics of Fourth-Year Pharmacy Students in a Study Comparing Practice-Based Learning and The Virtual-Patient Cases as Learning Methods in Advanced Pharmacy Practice Experiences

The patient case was based on a hospital patient admission for atrial fibrillation (Appendix 1). The case was developed by the 2 lead investigators and sent for review to 3 clinical faculty members with credentials in internal medicine or cardiology. After revisions were made based on the consulting faculty members’ recommendations, the case was sent for a second review to the initial review panel with the accompanying preexperience and postexperience test questions. The learning objectives facilitated active learning among clinical practice experience students early in their APPE. The learning objectives for the case are outlined in Table 2.

Table 2.

Learning Objectives for Virtual Case and Small-Group Sessions in a Study Comparing Practice-Based Learning and The Virtual-Patient Cases as Learning Methods in Advanced Pharmacy Practice Experiences

The patient case for the virtual arm of the study was then developed in virtual-patient software using the online branched-learning platform. The software required an additional 15 hours for creation of the patient case in the online platform. The software allowed the patient-case author to print details for students who chose incorrect pathways. All learning-objective references and guidelines needed to facilitate clinical decision-making were drawn from the American Heart Association and American College of Cardiology. These guidelines were uploaded for student use as the students navigated through the virtual-patient case. In the PBL group, students received all necessary references in a binder. For each component of the case, they were required to respond appropriately to patient-assessment prompts, interpret laboratory and radiological findings, and initiate pharmacological interventions. In both the virtual patient and PBL groups, students received feedback to their decisions prior to moving on to the next portion of the case. Feedback consisted of references to sections within guidelines that reinforced the correct choice, explanation of incorrect choices, as well as directions to learning nodes. These learning nodes prompted students to reevaluate portions of the case, which would lead to the correct answer choice consisting of allergies, renal dysfunction, and contraindications to certain therapeutic choices). The students were evaluated in small-group learning sessions, that consisted of either dependent (PBL) or independent (virtual self-directed) learning/teaching styles to assess each student’s acquisition and application of branch-narrative case methods to prepare for practice experiences.¹⁵

Students in the virtual arm were given a 15-minute virtual tutorial to locate navigational tools outlining a sample patient case provided by in the online simulation platform. The virtual case (http://decisionsimulation.com) was composed of several learning nodes for narrative and clinical data that simulated a real hospital patient case. The node was a single-screen pathway that allowed students the opportunity to reflect before making a clinical choice for patients. The nodes were simulated to correlate with case learning objectives and to give the students feedback in a qualitative and quantitative fashion. This was the first time this learning method had been used by the study institution’s college of pharmacy.

In the 6- to 8-student PBL group, 1 student received a lead binder and directions on how to incorporate discussion and decision-making in a group setting. Students had participated in similar activities throughout the pharmacy curriculum in disease-state and therapeutic case discussions. Additionally, in the PBL arm, the faculty member received university-based training in PBL in the form of a certified workshop.

To assess the students’ application of knowledge, each student took a preexperience and postexperience test with 15 multiple-choice questions dedicated to the subject matter covered in the cardiology case, ie, focusing on atrial fibrillation, anticoagulation, and congestive heart failure, developed by the lead investigators. At the end of each session, students were asked to complete a 12-question Likert survey instrument with response options ranging from 1=strongly disagree to 5=strongly agree) to reflect their experience with the virtual patient case or the small-group case discussion.

The preexperience and postexperience examination questions were pilot-tested among 80 P4 students in the year prior to the initiation of the study, ie, the previous graduating class, during the first 4 months of their APPEs. Between 2 to 4 questions were devised to address each of the educational objectives listed in Table 2. An item analysis with point biserial correlation and difficulty index was conducted on all questions to determine if any question required revision, elimination, or if a question had more than 1 possible correct answer. The revised testing instrument was then split between preexperience and postexperience tests of 15 questions each. The study included 1 to 2 questions per educational objective. A similar difficulty index was matched between the preexperience and postexperience tests. Results were analyzed using chi-square for nominal variables and paired sample t test or ANOVA for continuous variables. Data were analyzed using SPSS, version 19 for Windows (IBM Corp, Armonk, NY). A p value of 0.05 was set as significant.

RESULTS

Of the 140 students enrolled in the class of 2013, 119 participated in the study conducted over the first 6 weeks of their fourth year of the PharmD curriculum. To achieve a 95% confidence level and 5% confidence interval, a minimum of 59 students had to be registered in each arm of the study. After randomization, the baseline demographics of the students were assessed to ensure the 2 groups were similar. The only difference noted was that more students in the PBL group were concurrently enrolled in an internal medicine practice experience during the first month of their APPEs (Table 1).

On the preexperience test, there was no significant difference between the groups (57.7±12.6 for the virtual patient group vs 61.1±12.9 for the PBL group; p=0.16). After completion of the activity in their respective groups, students completed a postexperience test and survey instrument. A paired sample t test was conducted to compare the mean preexperience and postexperience test scores in each of the groups. There was a significant increase in scores in the virtual-patient group between the preexperience test score (57.7±12.6) and postexperience test score (66.5±13.6; p<0.001). There was also a significant increase in scores in the PBL group between the preexperience test (61.1±12.9) and postexperience test score (74.8±11.7; p<0.001).

When comparing the postexperience test scores of the virtual-patient group with those of the PBL group, there was a significant difference in the mean scores of the 2 groups, with the PBL group scoring 8.3% higher than the virtual-patient group (66.5±13.6 vs 74.8±11.7, respectively; p=0.001) (Table 3).

Table 3.

Mean Scores on the Preexperience and Postexperience Test in a Study Comparing Practice-Based Learning and The Virtual-Patient Cases as Learning Methods in Advanced Pharmacy Practice Experiences

An analysis was conducted of the percentage of students who had an increase, decrease, or no change in performance between their preexperience and postexperience test performance (Table 4). Each question was valued at 6.6% of the total score. A mean difference of 6.5% was determined as a “no change in test performance” because the study was powered to detect an increase of more than 7.5% from preexperience to postexperience test scores as a significant change in learning. Less than one-fifth (16.9%) of students in each group experienced a decrease greater than 6.6% on their postexperience test scores (ie, at least 1 additional question was missed). In both the virtual-patient and PBL groups, most students experienced an increase in their postexperience test score compared with their preexperience test score, with more PBL than virtual-patient students experiencing an increase (83.3% vs 69.5%, respectively; p=0.013). Finally, 13.6% of the virtual-patient group had no change in their preexperience test score compared with their postexperience test score (p=0.013).

Table 4.

Percentage of Students with a Change in Preexperience to Postexperience Test Scores in a Study Comparing Practice-Based Learning and The Virtual-Patient Cases as Learning Methods in Advanced Pharmacy Practice Experiences

The students also completed a 12-item Likert scale survey instrument. The reliability of the Likert scale to determine satisfaction with the learning technique and environment was determined (Cronbach alpha of 0.864). Response options ranged from 1=strongly disagree to 5=strongly agree. Results are presented in Table 5. After completing the computer-based simulation experience, the virtual-patient group reported an increased comfort in Web-based learning compared with the PBL group (4.2 vs 3.7, respectively; p=0.015). In both groups, students were relatively neutral on their preference for individual vs group-based learning session. Students in both groups agreed that the learning method reinforced previous knowledge (4.0 for virtual patient] vs 4.4 for PBL, p=0.034) and that the learning method added new knowledge to the subject area (4.0 for virtual patient vs 4.4 for PBL, p=0.01), with the mean Likert score in the PBL group being significantly higher in both parameters. Both groups reported that they would recommend the respective learning method to a fellow student.

Table 5.

Results of Student-Opinion Survey At the End of the Learning Session in a Study Comparing Practice-Based Learning and The Virtual-Patient Cases as Learning Methods in Advanced Pharmacy Practice Experiences

The virtual-patient platform enabled recording of the time needed to complete the task. PBL sessions were also timed. There was no significant difference in student-session time between the groups, with each method requiring nearly 2 hours for completion (118 minutes for VpSim and 114 minutes for PBL, p=0.345). The time needed to develop the case was identical for both groups, but an additional 15 hours were required by the virtual-patient faculty lead to develop the case online in the branched-learning format.

DISCUSSION

The results of this study are consistent with previous findings regarding both PBL and virtual-patient learning in medical and nursing education. As an educational technique that arose from psychology education, PBL has been successfully used in medical education since the 1960s. In a review of PBL in medical education, focusing student learning on core information relevant to real patient scenarios is one of the primary advantages of PBL.¹⁶ Another study that attempted to integrate PBL and computers was conducted in 2000. The authors concluded that students mastered learning objectives at a similar rate in both the facilitator-led format and concept-mapping, computerized format in a medicinal chemistry and pharmaceutics course.¹⁷ Since that study was completed, computer platforms for learning have expanded and become readily incorporated into medical nursing and pharmacy education. Though research has concluded the relative efficacy of these different learning styles in medical education, our study offers a direct comparison of these teaching formats.

Given the improvement in postexperience test score compared with preexperience test score, we conclude that both the PBL and virtual-patient learning methods were successful. However, the PBL method proved slightly more efficacious, with an 8.3% improvement of postexperience test scores compared with those resulting from the virtual patient method. The implementation of PBL has correlated with pharmacy students’ reports of increased confidence in patient scenarios, which they encountered in APPE courses.³ The slight additional improvement in the PBL group compared with the virtual-patient group may be attributed to 1 of many educational learning models.

One reason for the improvements is that the students were more accustomed to group-based learning scenarios, which had been previously used in conjunction with the therapeutics series and patient-case series. Though the students had not necessarily experienced the PBL model with a faculty facilitator, they had participated in group-patient case-solving beginning in their second year of the PharmD program. Additionally, the Theory of Social Constructivism may explain the gap in the postexperience test score results as well as the increased PBL mean scores. Compared with the virtual-patient learning method, PBL seems to have more effectively reinforced and introduced new knowledge to the students.

The Theory of Social Constructivism purports that how knowledge is learned is as important as what is learned.¹⁸ Further, knowledge is acquired through social negotiation. In this model, the PBL technique lends itself more naturally to the social negotiation of knowledge exchange. Conversely, the virtual-patient platform, which monitors an individual’s attempts to navigate through a complex case, negates this basic method of social learning.

Students in the PBL group received more detailed feedback with respect to the answers they chose (correct or incorrect) compared with students in the virtual-patient group. This difference is a result of students in the virtual-patient group receiving links to correct-answer choices with the supporting literature in response to choosing an incorrect answer. This method is not consistent with experience in actual patient care. Conversely, students in the PBL group discussed the pros and cons of each therapy alternative prior to recommendation, using the literature to guide their final choice, a method that more closely parallels real patient care.

The virtual-patient model of learning speaks to the more recent generation of students, who are comfortable with alternative learning platforms for learning. As our results demonstrate, both sets of students expressed being comfortable with computer-based platforms. The virtual-patient group also received positive and negative feedback as well as prompts to their responses, suggesting a level of student-facilitator interaction.

Our study had several limitations. Students were not given a preexperience test to assess different learning styles, as could have been accomplished with the use of the Pharmacists’ Inventory of Learning Styles, an instrument designed to determine different learning techniques.¹⁹ Some learning-style dimensions were defined along the following axiom: structured-unstructured, doing-reflecting, thinking-acting, observing-trying, risk averse-risk taking, and logic-intuition. Students whose learning styles are in the doing–reflecting category are more willing to try experiments involving trial and error. Thus, these individuals would prefer the virtual-simulation arm and autonomy to complete the exercise without a facilitator. Conversely, the facilitated discussions in the PBL method would have produced the most favorable outcomes for students whose learning style had mixed axes, which suggests a preference for a structured vs unstructured and an observing-trying approach to learning. These individuals thrive in an environment wherein the learning process has clear outcomes and knowledge is gained first-hand rather than secondhand. A potential future study could correlate learning style to teaching method/case simulation.

Another limitation was that significantly more students in the PBL arm were concurrently enrolled in an internal medicine practice experience for their first APPE compared with those in the virtual patient arm (38.3% vs 3.3%, respectively; p<0.001). The study was conducted during the first 3 weeks of the first APPE block. More of the PBL students may have been exposed to a similar patient case in their practice experience, which would explain their improved postexperience test scores. However, this potential limitation was refuted by the lack of a significant difference in the preexperience test scores for the 2 groups. Additionally, a more exhaustive preexperience and postexperience test with more than 15 items may have better assessed students’ knowledge.

Time limitations during the study did not allow for a longer preexperience and postexperience test. Given that completion of the case was dependent on Internet connectivity, one subgroup of the virtual-patient arm experienced a 15-minute delay on 1 occasion, which could explain its decreased performance. Students in the PBL group discussed the strength of each therapy alternative prior to choosing a final treatment plan, compared with students in the virtual-patient group, who chose a therapy from the choices and supporting literature and then received feedback. Finally, the preexperience test, the case, postexperience test, and the survey instrument were all administered at the same sitting, potentially introducing learner exhaustion in both groups. This possibility may explain the identical decrease in postexperience test scores for both groups.

These aspects of our study detract from its validity. This study was not powered to detect the superiority of 1 method over the other. An absolute increase of 13% in the mean postexperience test score, as compared with the preexperience test score, between the 2 groups would better describe the efficacy of 1 learning method over the other. Our study established that both learning methods were effective but was not able to conclude based on the difference in the mean postexperience test scores that the PBL method was superior to the virtual-patient approach as a learning method.

SUMMARY

This study evaluated the relative efficacy of virtual patient simulation vs problem-based learning as a teaching tool among pharmacy students. Branched-outcome teaching methods were presented in both a virtual patient format and in a faculty-led problem-based learning group. Both teaching formats were proven to be effective and well received by students with the problem-based learning method providing increased postexperience performance and student perceived satisfaction. There are few published articles that have evaluated PBL vs virtual simulation among pharmacy students. The results of this study provide support for a mixed approach to patient-case scenarios. This hybrid approach facilitates learning regardless of knowledge level of the learner at the beginning of the simulation exercise.

Appendix 1. Case 1: Cardiovascular Patient Admission: “I am short of breath.”

Mr. John Smith’s wife brings him to the emergency department. He is a 65 yo African American male. She reports, “My husband has been complaining of shortness of breath and palpitations for the last 12 hours.” His wife reports a medical history of Paroxysmal Afib, HF-stage B (LVEF 45%), and HTN.

Abbreviations: Afib=afibrillation; HF=Heart Failure, LVDEF = Left Ventricular Ejection Fraction, HTN = Hypertension).