Pharmacy Students' Learning and Satisfaction With High-Fidelity Simulation to Teach Drug-Induced Dyspepsia

Abstract

Objective. To assess second-year pharmacy students’ acquisition of pharmacotherapy knowledge and clinical competence from participation in a high-fidelity simulation, and to determine the impact on the simulation experience of implementing feedback from previous students.

Design. A high-fidelity simulation was used to present a patient case scenario of drug-induced dyspepsia with gastrointestinal bleeding. The simulation was revised based on feedback from a previous class of students to include a smaller group size, provision of session material to students in advance, and an improved learning environment.

Assessment. Student performance on pre- and post-simulation knowledge and clinical competence tests documented significant improvements in students' knowledge of dyspepsia and associated symptoms, with the greatest improvement on questions relating to the hemodynamic effects of gastrointestinal bleeding. Students were more satisfied with the simulation experience compared to students in the earlier study.

Conclusion. Participation in a high-fidelity simulation allowed pharmacy students to apply knowledge and skills learned in the classroom. Improved student satisfaction with the simulation suggests that implementing feedback obtained through student course evaluations can be an effective means of improving the curriculum.

INTRODUCTION

The use of simulation-based learning, such as computerized high-fidelity manikins, to improve the training of students is commonplace in some healthcare disciplines around the world, and its benefits in clinical teaching and assessment have been well documented in medicine and nursing.^1-4 High-fidelity simulation has become increasingly popular in pharmacy education in the United States after the Accreditation Council for Pharmacy Education indicated that colleges and schools could include simulation as part of introductory pharmacy practice experiences in doctor of pharmacy (PharmD) degree programs.⁵ Many pharmacy programs have included simulation training only at certain stages in their curricula, while others have successfully incorporated it throughout the curriculum.^6-8 High-fidelity simulation is still in its infancy in pharmacy education in the United Kingdom but it is gaining pace.⁹

High-fidelity simulation in pharmacy provides a safe and controlled, realistic learning environment in which students can gain “hands on” experience and integrate pharmacotherapy knowledge and skills learned in the classroom into practice. It encourages the development of communication, critical-thinking, problem-solving, and teamwork skills.^6,10,11A unique feature of high-fidelity simulation is that disease states can be replicated to allow for standardization of student learning experiences. More importantly, students can practice and perfect their patient care skills at no risk to real patients.^6,12,13

Simulation training has even been taken beyond the safe and controlled environment of the simulation laboratory to demonstrate positive outcomes in the actual patient care environment. Pharmacists used high-fidelity simulation to educate nurses and evaluated the impact of the training on medication error rates in critically ill patients. Following the intervention, the medication administration error rates of nurses in the study group significantly decreased from 30.8% to 4% (p <0.001) compared to nurses in a control group whose error rates increased from 20.8% to 36.7% (p = 0.002) over the same period of time.¹⁴

The value of using high-fidelity manikins in pharmacy has been investigated to teach critical thinking, clinical reasoning, and problem-solving skills^7,10,15,16; advanced cardiovascular support¹⁷; medical emergencies and critical care management^7,9; team-working skills and interprofessional working^16,18; end of life care¹⁹; technical skills; and performance-based pharmacotherapy principles^{15, 20} In these studies, simulation was mainly used for formative assessment. Also, a variety of assessment methods was used to qualitatively and quantitatively examine the impact of simulation-based learning. Pre-and post-simulation and delayed post-simulation tests were used to assess knowledge and retention. The tools used to assess clinical competence, performance-based skills, and attitudes of students, either individually or as a group, included written SOAP (subjective, objective, assessment and plan) notes, pharmacotherapy plan development, skills checklists, and grading rubrics.^7,15,20,21

Student satisfaction survey instruments were the most commonly used research tool to evaluate simulation in pharmacy. While these did not objectively measure the impact of simulation, they provided information about students’ attitudes toward and perceived benefits from this novel teaching technique.

The use of student satisfaction survey instruments in health education research is extensive. Soliciting and implementing students’ feedback to improve instruction represents an essential component in the quality assurance process, and its importance is well documented in the literature.²² Some disciplines have gone beyond the use of student surveys to evaluate learning and the quality of teaching to adopt a more student-focused approach whereby students are involved in most stages of curriculum development. Students’ perceptions of an assessment or curriculum development influence the acceptability of the innovation among other students.²³ Although there is evidence of implementing students’ feedback from simulation studies to improve the simulation experience, there appears to be nothing in the pharmacy literature regarding evaluation of the impact of making these changes.²⁴ This high-fidelity simulation study therefore sought to fill this void in the literature. The first part of the study was designed to assess second-year pharmacy students’ acquisition of pharmacotherapy knowledge as well as their clinical competence. The second part of the study sought to determine what impact the feedback from students in the earlier study had on the simulation experience of the student cohort who participated in the present study.

DESIGN

In 2009, simulation-based learning activities, including high-fidelity simulation, were incorporated alongside traditional teaching methods in the delivery of the pharmacology and therapeutics module in the second year of the 4-year undergraduate pharmacy curriculum at the Medway School of Pharmacy in the United Kingdom. The aim was to allow students the opportunity to apply pharmacotherapy knowledge and to develop communication, patient care, and problem-solving skills early in the curriculum.

In all the high-fidelity simulation workshops delivered within the module, the human patient simulator is operated in real time by a staff member who also provides the voice of the patient; hence, simulation case scenarios are not preprogrammed into the simulator. Verbal cues to the patient or cues written in the patient’s script are used to initiate physiological and vital signs changes in the simulator. For example, the patient on cue, either verbally from the nurse or scripted, might be required to experience an acute medical episode during a medication history taking session. Patient responses are sparingly scripted to ensure the consistency of each group’s experience while preserving the spontaneity of normal conversation. This, coupled with the “on the fly” operation helped to make the simulation experience more authentic. This study was approved by the Medway School of Pharmacy Research Ethics Committee.

Simulation Experience in Earlier Study

The author used student feedback from an earlier study (cohort 1) to make changes to the simulation experience for students in the present study (cohort 2).²⁵ The simulation experience of the earlier study consisted of a pre-session classroom lecture on diabetes, a hypoglycemia patient case simulation session, and a post-session satisfaction questionnaire survey instrument to explore students’ views and perceptions of the value of high-fidelity simulation to teach pharmacology and therapeutics. One hundred four second-year pharmacy students in groups of 10 to 20 students took part in the 15-minute simulation session. Students were expected to use a wide range of knowledge and skills to provide appropriate patient care. While the majority of students provided positive feedback about their simulation experience, they also made suggestions on how it could be enhanced. These suggestions included providing the simulation material to students to review beforehand rather than distributing it at the simulation session; extending the simulation session; reducing group size; making the simulation experience more realistic; improving the sound quality of the simulator; and developing more patient cases covering more topic areas within the pharmacology and therapeutics module.

Revised Simulation Experience

In response to this feedback, in fall 2010 second-year pharmacy students (cohort 2) attended an orientation session 1 week prior to the simulation session to introduce them to the simulation environment and to demonstrate the features and capabilities of the human patient simulator. A brief summary of the patient case was given to students to allow them sufficient time to prepare for the simulation session using lecture notes and other reference sources.

The 127 students in cohort 2 were randomly assigned to smaller groups of 8 to 9 students (reduced from 10 to 20 students per group in cohort 1) for each 20-minute simulation session. To accommodate all of the students, the simulation sessions were conducted in 3 workshops held over 2 days. Students were presented with a patient case of non-steroidal anti-inflammatory drug-induced dyspepsia with upper gastrointestinal bleeding. Roles for a nurse and a doctor (played by the session facilitator) were also incorporated into the scenario. In response to feedback from cohort 1, the staff member who provided the voice of the patient was positioned well out of sight of the students. To improve the sound quality and minimize the echo within the simulation suite, a sound mixer with microphone preamplifier was used and a second hidden radio microphone was placed on the simulator.

All other components of the simulation experience remained the same as in the earlier study in that members of each group were expected to use a range of knowledge and skills to provide relevant care to the patient. They were to obtain a medication history from the patient, identify and solve any drug therapy problems, note the hemodynamic effects of an upper gastrointestinal bleed on the cardiac monitor, provide medicines information to the patient on request, and respond to queries from either the nurse or doctor relating to the care of the patient. To give each group a sense of independence, group members were left to assign roles among themselves, choose a leader to be the main person interacting with the simulator, and work cohesively as a team to perform all the tasks expected of them.

EVALUATION AND ASSESSMENT

The 7-question pre- and post-simulation knowledge test was designed to assess students’ knowledge of dyspepsia and their ability to recognize dyspepsia with ALARM signs (Anemia, unintentional weight Loss, Anorexia, Recent onset of progressive symptoms, Melena or upper gastrointestinal bleeding, persistent vomiting and dysphagia).²⁶ The questions on the pretest and posttest remained the same to ensure that the results of the posttest were solely a reflection of participation in the simulation session and not influenced by any external factors. To further reduce the chance of bias, students were not forewarned about the test. Students wrote a unique letter and number code on the test instrument to allow for anonymity to be maintained while facilitating identification and matching of individual students’ pretest and posttest.

A debriefing checklist developed by the facilitator and reviewed by a second instructor, was used to formatively assess clinical competence. Student groups were assessed on their application of knowledge and communication, problem-solving, and clinical skills, as well as on their professionalism. A short debriefing session was held immediately post-simulation by the facilitator to provide verbal feedback to each group regarding the students’ performance. The students were later provided with more detailed feedback on their performance as well as a hard copy of the completed debriefing checklist with the overall group score.

Finally, students’ views on the simulation experience were assessed using a post-simulation satisfaction questionnaire, the results of which were compared with those from the earlier study to evaluate the impact of feedback changes on their simulation experience. To allow for direct comparison and to minimize variability of results, the validated questionnaire used in the earlier study²⁶ was administered. The questionnaire consisted of 5 domains to gauge students’ views on the simulation experience in terms of knowledge acquisition and application; communication, problem-identification, problem-solving, and clinical skills; and satisfaction with the learning technology. Students were asked to rate their responses on a 5-point Likert type scale (1= strongly agree and 5= strongly disagree), in addition to making free-text comments. An additional question was added to the questionnaire to determine students’ preference for the other types of simulation activities or active-learning interventions used to enhance students’ knowledge and learning in pharmacology and therapeutics.

Analysis of the quantitative data was performed using Microsoft Excel for Windows 7 and IBM SPSS, version 19, statistical software. A paired t test was used to analyse pretest-posttest scores and an independent sample t test was used to compare the results of the satisfaction survey from the 2 cohorts. A p value of <0.05 was considered to be significant. The calculation of the mean rank score used for comparative data analysis in the satisfaction survey was based on a 5-point Likert scale with 1= strongly agree, 2= agree, 3 = neutral, 4 = disagree, and 5= strongly disagree. Qualitative data were analysed for thematic content by the researcher and verified by a second person.

One hundred twenty-seven students participated in the simulation session. However, in the assessment of the pre- and post-tests, only data for 123 students were analyzed because 4 students did not complete all the components of the assessment. A paired t test performed on the mean score data identified significant differences between the pretest and posttest scores (Table 1). The mean test score increased from 44% in the pretest to 63% (B grade equivalent in the United States) in the posttest. The greatest improvement between pretest and posttest scores was seen with those questions relating to the haemodynamic effects of an upper gastrointestinal bleed (p <0.001). Following the simulation session, there was also a significant improvement in students’ ability to suggest investigative tests that may be required in a patient presenting with dyspepsia with an upper gastrointestinal bleed (p <0.001).

Table 1.

Knowledge Assessment of Pharmacy Students Before and After Participation in a High-Fidelity Simulation Session (n = 123)

Students were assessed as a group for clinical competence using the debriefing checklist. Fifteen groups comprising 8 to 9 students each took part in the simulation exercise. The average grade for all groups was 60% (equivalent to a B grade in the United States), with a range of 48% to 83%. Eight out of the 15 groups scored 60% or more in the competence test. As shown in Table 2, student groups scored more than 75% or gained maximum marks in 5 out the 8 assessed elements. Students performed poorest in the areas where they had to confirm the patient’s identity and allergic status and recommend appropriate management for the patient. They excelled at gathering information on the history of the presenting complaint, taking a medication history, and identifying the drug therapy problem.

Table 2.

Performance of Pharmacy Students Who Participated in a High-Fidelity Simulation Session

Views from students in the earlier study (cohort 1) and in the present study (cohort 2) were compared. One hundred four students in cohort 1 took part in the earlier simulation study compared to 127 students in cohort 2. The percentage of students in cohort 1 and cohort 2 who completed and returned post-simulation questionnaires were 100% (104/104) and 96% (122/127) respectively. The demographics of the respondents in the 2 student cohorts are presented in Table 3. An independent samples t test used to examine the variance between the cohorts in terms of demographics and learning styles found no significant differences between the 2 groups.

Table 3.

Demographics of Pharmacy Students Who Participated in a High-Fidelity Simulation to Teach Drug-Induced Dyspepsia

The mean rank scores for the 2 student cohorts are compared in Table 4. The responses of cohort 2 were significantly different from those of cohort 1. Students in cohort 2 were more in agreement than those in cohort 1 that the experience helped them to increase their knowledge of the patient’s disease condition. Students in cohort 2 believed that, following the simulation session, they were better able to advise patients and identify, solve, and apply problem-solving skills. Finally, cohort 2 students were more likely to agree that the human patient simulator allowed them “hands on” experience.

Table 4.

Students’ Views on the Impact of Participation in a High-Fidelity Simulation Session

The vast majority of respondents in cohort 2 welcomed the idea of being introduced to the simulator and its features 1 week beforehand (Table 5), as was suggested in feedback from cohort 1. Almost twice as many respondents in cohort 2 than cohort 1 enjoyed the simulator and believed or strongly believed that the interaction with it was much better than they expected. There was no significant difference (p >0.05) between the 2 cohorts in terms of finding it stressful to work in a technological environment.

Table 5.

Pharmacy Students’ Satisfaction with the Learning Technology Used in a High-Fidelity Simulation Session

The majority of students in cohort 2 provided comments similar to those of students in cohort 1 when asked what they liked most about their simulation experience. They liked the “hands on” approach that it provided, thereby allowing them to “actually practice what has been taught.” They liked the “realism of the experience” and also enjoyed working in small groups for the interactive session, noting that it engendered teamwork. Like students in cohort 1, they were also in favor of the human patient simulator being used to showcase more patient cases covering other topic areas within the pharmacology and therapeutic module.

A group size of 8 was still considered too large by a few students who would have preferred a much smaller group to ensure more “hands on” experience with the patient simulator. Students in cohort 2 were also asked to indicate which of the 4 interactive learning interventions (human simulator, role play, paper-based case studies, and knowledge maps) they preferred. High-fidelity simulation was the teaching method most preferred by students (Table 6). The most common reasons given for their choice were that it was hands on, interactive, mirrored real life situations, and provided more learning opportunities.

Table 6.

Teaching Method Preferences Among Pharmacy Students Who Participated in a High-Fidelity Simulation Session

DISCUSSION

This study demonstrated the positive students’ outcomes associated with high-fidelity simulation-based learning as evidenced in the individual pre/post knowledge acquisition and the student group clinical competence test results. The feedback changes made to the simulation experience appeared to further enhance students’ learning and satisfaction with high-fidelity simulation.

Students demonstrated improvement in their knowledge of dyspepsia and in recognizing dyspepsia with ALARM signs as there was a mean increase in scores from 44% on the pretest to 63% on the posttest. According to the higher education grading system in the United Kingdom, the highest attainable mark of 70% to 100% is equivalent to an A letter grade in the United States. Similarly a mark of 60% to 69% equates to a B grade. Given that the minimum mark for a satisfactory pass is 40% (D grade), this meant that the group mean mark showed good improvement. The impact of simulation on knowledge gains was most noticeable in those questions that examined the hemodynamic effects of upper gastrointestinal bleeding. During the simulation session, students were able to directly observe its effects on the patient’s vital signs on the cardiac monitor, thereby enhancing their learning.

The students also scored well as a group (overall mean score of 60% or a B grade) on most of the elements of the competency-based assessment. However, students performed poorly on confirmation of the patient’s identity and allergic status. This lapse in students’ history-taking skills could not be explained. During the debriefing session, which was held to encourage self-reflection and remediation, students indicated that they thought that because the patient’s name and allergic status were already entered in the treatment chart found by the patient’s bedside, there was no need for them to confirm these details with the patient.

Common and contrasting findings were reported in 3 pharmacy studies conducted to test knowledge acquisition and a variety of skills in high-fidelity simulation-based learning. Seybert and colleagues used simulation to evaluate second-year pharmacy students’ knowledge of blood pressure assessment and their ability to accurately measure blood pressure.²⁰ In another study, Seybert and colleagues assessed students’ knowledge of and clinical performance in management of dysrhythmia and myocardial infarction.¹⁵ In both studies students’ knowledge of pharmacotherapy, confidence in interpreting data, and clinical competence improved following participation in a simulation exercise. The proportion of students in the hypertension study who were able to accurately obtain a blood pressure reading rose from 21% pre-simulation to 97.6% post-simulation. Similarly, students scored higher in their problem-solving skills in the dysrhythmia myocardial infarction management study.

In contrast, Mieure and colleagues used high-fidelity simulation to introduce third-year pharmacy students to advanced cardiovascular life support, but students performed poorly on a post-activity knowledge quiz, with a median score of 25%.¹⁷ Their poor performance was thought to be multifaceted: poor wording of the examination questions, how the assessment was conducted or the students limited exposure to the simulation.

Another area where students in the present study showed reduced competence was when suggesting appropriate treatment intervention for the patient. During the debriefing session and in their free-text comments, some students indicated they were unclear about their roles within the group, which may have hampered their performance during the simulation session. There are plans to specify roles for each group member in future iterations of the high-fidelity simulation workshops. Students’ poor performance may have also been partially a reflection of their inexperience in applying knowledge, as students at this early stage of the curriculum would have just started to develop knowledge application skills. When students were asked on the knowledge acquisition test to suggest appropriate management for a patient presenting with dyspepsia and gastrointestinal bleed, their mean score increased by 2.5 times over that in the pretest. Interestingly, their perception of their ability to apply problem-solving skills was at variance with their actual performance on the competence test.

Robinson and colleagues used simulation to teach medical emergency management to second-year PharmD students and found that 89% of students managed the simulated patient appropriately. The 11% of students who failed to provide appropriate treatment to the simulated patient did so because of their inaccurate assessment and nonadherence to treatment guidelines.⁷

A key focus of this study was to investigate what impact changes made to the simulation experience as a result of feedback from cohort 1 would have on cohort 2’s learning and satisfaction with high-fidelity simulation-based learning. This was evaluated by comparing post -simulation views of the participants in the earlier study (cohort 1) with those of participants in the present study (cohort 2). A significant finding was that orienting students to the simulator and its features, providing the simulation material beforehand, improving the sound quality, extending the simulation session, making the session more realistic, and reducing the group size appeared to benefit cohort 2 students as their satisfaction levels in all 5 domains of the satisfaction questionnaire were significantly higher than those of students in cohort 1.

The orientation to the simulation suite and the human simulator, and having access to the case scenario ahead of the session meant that cohort 2 students had sufficient time to prepare for the session, which probably led to their perceived increase in knowledge of dyspepsia and gastrointestinal bleeding, enjoyment of the simulator, and overall satisfaction with the simulation experience. Providing students with this kind of orientation prior to participating in a simulation session reduces their stress and promotes a more comfortable learning environment.^6,18,24 Positioning the staff member who provided the voice of the patient out of students’ view, and improving the sound quality resulted in more positive comments from students in cohort 2 than cohort 1 about the realism of the consultation session. Cohort 2 students also were observed “getting into their role,” communicating with the patient to obtain a medication history, provide education and counselling, and respond to the patient’s questions.

By reducing the group size, students in cohort 2 enjoyed the experience more, noting that the small group size allowed them to be more hands on with the simulator and forced them to work more effectively as a team to provide care to the patient. Some participants wanted the group size to be reduced even further to maximize their interaction with the simulator.

A review of other pharmacy studies showed varying group sizes ranging from 3 to 5, to 6 to 7 students at simulation sessions, with 10 to 12 students being assigned to a group in 1 study.^{7,11,15,17,18,20,27} There appeared to be no published evidence on ideal simulation group size, but in the present study, it was important to determine the maximum group size with which students felt comfortable. Determining that a group size of 8 to 9 students was acceptable to students was a valuable finding in this study, as large class sizes make it prohibitive to have smaller simulation group sizes, if facilitator and “patient” fatigue are to be avoided. Both cohorts wanted the human patient simulator to be used to showcase more topic areas within the pharmacology and therapeutics module. There are plans to address this as well as incorporating high-fidelity simulation in other modules thereby making its use more widespread within the school.

This study had some limitations. A follow-up post-simulation knowledge test would have strengthened the knowledge-based test results as it would have provided a measure of long-term retention of disease-specific knowledge. Because the simulation exercise was delivered in 3 workshop sessions over 2 days, students who attended the earlier workshops could have informed students attending the later workshops regarding their experience, thereby introducing possible bias in the pretest and posttest and clinical competence test results. Also, the pre- and post-simulation knowledge and clinical competence tests administered to cohort 2 were not administered to students in the earlier study and therefore scores and performance could not be compared. Such results would have provided more robust data on the full impact of the changes made to the simulation experience. Finally, the increase in cohort 2 students’ level of satisfaction with the simulation experience may have been the result of them already having an interest in high-fidelity simulation and not to the implementation of students’ feedback from the earlier study.

CONCLUSION

Participation in a high-fidelity simulation session improved pharmacy students’ knowledge and understanding of dyspepsia and associated ALARM signs. Students also displayed good clinical competence. Changes made to the human patient simulation experience in response to feedback from students in cohort 1 appeared to benefit students in cohort 2, as they reported higher levels of satisfaction with the simulation experience and perceived learning. This study also showed that participation in simulation-based learning early in the pharmacy curriculum can help to prepare students for entry into the workforce as competent practitioners.