Knowledge Gain of Pharmacy Students and Pharmacists Comparing Simulation Versus Traditional Learning Methodology

Abstract

OBJECTIVES: The purpose of this study was to evaluate the difference between education via written materials alone and written materials enhanced with hands-on simulation.

METHODS: A simulation case, educational module, and assessment regarding torsades de pointes (TdP) in an adolescent patient were designed. The written educational module was given to all study participants. A total of 92 third-year pharmacy students and 26 pharmacists participated in the study.

RESULTS: When approximately half of the participants had been to simulation, an anonymous assessment was given. Responses from those who had been to simulation and those who had not, and whether they had read, skimmed or not read the educational material were compared. A non-paired Student t-test compared the percentage correct and responses of individual questions between groups. Mean participant scores of those who went to simulation (70% ± 16%) were statistically significantly higher than mean scores of those who had not attended simulation (54% ± 21%; p<0.0001). Furthermore, those who attended simulation and read the module (72% ± 3%), skimmed (68% ± 13%), or did not read the module (66% ± 16%) had higher scores than those who did not attend simulation and read the module (62% ± 26%), skimmed the module (54 ± 17%) or did not read the module (51% ± 20%).

CONCLUSIONS: Hands-on simulation significantly improved assessment scores. Overall, reading the educational module and participating in simulation yielded the best scores. Participants who attended the simulation and did not read the module had higher average scores than participants who read the educational module and did not go to simulation.

INTRODUCTION

There are many approaches to educating and training medical professionals. Simulation provides hands-on experience crucial to learning in a safe environment where students and clinicians are able to practice without risk of harming a patient. The Association of American Medical Colleges has called simulation training a keystone of health profession education and patient safety, and organizations such as the Society for Simulation in Healthcare have been formed to help study and promote its significance.¹ High-fidelity patient simulators recreate an immersive patient care setting which allows for knowledge application thereby fostering independence and confidence, pharmacotherapeutic knowledge, and skills in a controlled, non-threatening environment.^2–3 Simulation training is an integral part of many medical, nursing and pharmacy programs.^1,4,5

The Accreditation Council for Pharmacy Education recognizes simulation as an effective method for developing critical thinking and problem-solving skills that should be incorporated in Doctor of Pharmacy curricula and schools of pharmacy are encouraged to develop interprofessional simulation experiences.^4,5 Multiple studies have been done lending to the credibility of simulation in training healthcare professionals.^2–4,6–10 However, very few studies have been conducted to compare high-fidelity simulation exercises with more conventional ways of learning, such as the use of written materials, or the effect simulation exercises have when used in conjunction with these conventional methods. The purpose of this study was to evaluate whether the addition of a simulation exercise to more conventional learning methods is of any benefit compared with education methods using written materials alone. It was hypothesized that students and pharmacists who participated in simulation would have greater knowledge acquisition than participants who only had access to written materials. Additionally, it was hypothesized that participants who went to simulation and read the written materials would learn the most.

MATERIALS AND METHODS

This study was approved by the Institutional Review Boards at both the University of Alabama at Birmingham and the Samford University. A high-fidelity simulation case, educational module, and assessment regarding torsades de pointes (TdP) in an adolescent patient were designed. Third-year pharmacy students of the McWhorter School of Pharmacy and pharmacist volunteers employed by Children's of Alabama were included in the study. The entire third-year class was required by the pharmacy curriculum and randomly scheduled to attend simulation in small groups of 3 to 5 students at a time over a 6-week period, and pharmacists attended simulation as their schedules allowed. Torsades des pointes was chosen because of its infrequency in the pediatric setting, in an attempt to eliminate knowledge and experience bias pharmacist participants would have compare with pharmacy students. Additionally, TdP requires timely and appropriate management, which translates well to a high-fidelity simulation experience. Both the pharmacists and the pharmacy students received a conventional written module at least 10 days prior to attending simulation and was available at any time during the 6-week period. At the time of simulation, students had previously had a lecture on TdP approximately 14 months prior in a pharmacotherapy course during their second year.

The written educational module was a 2-page document that included a summary of the pathophysiology of TdP, the patient presentation (including a sample electrocardiogram [ECG]), the risk factors, the treatment, common classes of medications that can prolong the QT interval on the ECG, and patient counseling points (Appendix 1). The module was written with pharmacy-relevant information from current TdP management, as described in the American Heart Association Handbook of Emergency Cardiovascular Care for Healthcare Providers.¹¹ The written module presenting TdP material was developed with information from Medscape and UpToDate, and content was reviewed for accuracy and importance by expert pharmacists.^12,13 Participants were encouraged to read the module prior to attending simulation.

In each simulation, a physician (portrayed by a fourth year medical student) and nurse were present in addition to the high-fidelity simulation mannequin patient. Both the nurse and the “physician” knew the simulation case in its entirety, whereas the pharmacist and the pharmacy student participants did not. The simulation case was designed around a male, adolescent patient who presented to the emergency department (ED) with multiple risk factors and signs and symptoms of TdP. The patient presented with dehydration from a gastrointestinal illness, causing electrolyte abnormalities, including hypokalemia and hypomagnesemia. He was taking sumatriptan for migraines and azithromycin and diphenhydramine for sinusitis and had received intravenous (IV) ondansetron upon admission to the ED due to vomiting. Shortly after receiving IV ondansetron, he developed a 20-second period of TdP, which then spontaneously converted to normal sinus rhythm. Participants, along with the “physician,” were asked by the nurse to come into the room after this event. Participants then interviewed the patient to obtain the above history. If questions about family history were asked, participants also learned the patient's father had congenital long QT syndrome. The patient's ECG was available for review at his bedside and showed a long QT. After 10 minutes, the patient again developed TdP, unless the learners had already ordered and drawn up an appropriate dose of IV magnesium. Magnesium vials were available in a standard concentration, and participants were able to use a pharmacy dosage handbook in either electronic or printed version. The nurse then administered the magnesium to the patient. At the end of the simulation, a debriefing using the “Debriefing with Good Judgment” model was conducted, where learners have the opportunity to reflect on concepts and issues presented in simulation to facilitate learning.¹⁴ A general debriefing script was drafted to help facilitate and standardize the session and ensure all learning objectives were covered with each group of learners. The same lead debriefer was used for all sessions, and all cases had 1 of 3 pharmacy content experts. The simulation scenario lasted 15 minutes, with 40 minutes of debriefing time and a 5-minute orientation to simulation center and the mannequin.

Pharmacy students were scheduled for simulation dates according to predetermined, randomized small groups for other curriculum requirements. Pharmacists signed up to attend simulation as their schedules allowed. Simulation groups consisted of either pharmacy student participants or pharmacist volunteers, there were no groups that contained both pharmacy students and pharmacists. When approximately half of all participants had participated in the simulation, an anonymous, voluntary assessment (Appendix 2) was given to all third-year pharmacy students (n = 120) and pharmacists (n = 40). Scores were compared between those who had been to simulation and those who had not and, additionally, to evaluate any differences in scores between participants who had read, skimmed, or not read the module. Although the simulation was required for all students, that assessment was voluntary. The assessment contained demographic information; asked whether or not the participant had been to a simulation regarding TdP within the last 30 days and whether or not the participant had read, skimmed, or not read the educational module; and assessed concepts described in the educational module and in simulation. By study design, it was acknowledged that learners would be the best to understand and know their own study habits, therefore, learners self-defined the meaning of read, skimmed, and not read. Conceptual questions were developed in regard to the written module given to the student and pharmacist participants, which also directly reflected the concepts demonstrated during simulation. These questions assessed knowledge of medications that prolong the QT interval, risk factors for developing TdP, electrolyte abnormalities likely to contribute to the development of TdP, patient presentation, and pharmacologic treatment of TdP; participants were also asked to identify TdP on a simple ECG rhythm strip, as the classic “twisting of the points” was clearly depicted in both the written module and during the simulation exercise (Appendix 2). Prior to distribution, expert pharmacists, pharmacy academicians, and a pediatric critical care physician with extensive simulation experience reviewed the examination.

Assessment answers in both aggregate and individual questions were statistically compared using SPSS statistical software (Chicago, IL). An unpaired Student t-test and analysis of variance were used as appropriate to compare groups. These groups were classified as those who had been to simulation and those who had not and further classified by whether the participants had read, skimmed, or not read the provided educational module. All tests were 2-sided and a p value <0.05 was considered significant.

RESULTS

A total of 92 of 120 pharmacy students (77%) and 26 of 40 pharmacists (65%) employed at our institution completed the assessment, totaling 118 participants. At the time of assessment distribution, 40 pharmacists were employed at the institution, yielding a participation rate of approximately 65%. At the time of the assessment, 48 of the 92 students (52%) and 13 of the 26 pharmacists (50%) had been to simulation. The average score for all participants was 62% ± 20%, and there were no statistically significant differences between the scores of students and those of pharmacists (p = 0.245). Mean scores of participants who went to simulation (70% ± 16%) were significantly higher than the mean scores of those who had not attended simulation (54% ± 21%; p<0.0001) regardless of whether they had read, skimmed, or did not read the module (Figure).

Figure.

Mean assessment scores.

▪Simulation + Module; Module alone

When splitting the results into just students or just pharmacists, we found a significant difference between students (p = 0.001) but not pharmacists (p = 0.21). Students who did the simulation and either read (69% ± 16%) or skimmed (67% ± 22%) the module had significantly higher scores than students who did not do the simulation and either skimmed (49% ± 11%) or did not read (50% ± 19%) the module.

The learner assessment results are in the Table. Four of the 7 clinical questions were answered correctly more frequently if the participant attended simulation. The 3 assessment questions that were not statistically significantly impacted by participation in simulation requested participants to 1) identify risk factors for TdP, 2) identify which electrolyte imbalances are risk factors for TdP, and 3) identify how a patient experiencing TdP might present.

Table.

Torsades de pointes Learner Assessment Results

DISCUSSION

The use of simulation had a significantly positive impact on TdP knowledge scores compared to scores achieved after receiving a conventional written educational module for pharmacy students and pharmacists. The results of this study suggest that the use of simulation may increase the learner's understanding and knowledge of a topic more than a conventional educational model using written materials. Overall, reading the educational module and participating in simulation yielded the best scores. Participants who attended the simulation and did not read the module had higher average scores than participants who read the educational module and did not go to simulation. By design, this study focused on a single, focused disease in an attempt to show learning differences by varying teaching modalities.

This study is unique due to the fact that both the pharmacists and the pharmacy students were assessed using the same written test and simulation scenario. Most other simulation studies involving pharmacy as a discipline have examined pharmacy students alone. Tofil et al⁶ found simulation to be effective in teaching pharmacy students about supraventricular tachycardia, particularly in the application of knowledge learned during the simulation. Branch¹⁰ studied second-year pharmacy students' pharmacotherapy knowledge using a case of drug-induced dyspepsia with gastrointestinal bleeding. This study found significant improvement in students' knowledge, especially in regard to the hemodynamic effects of bleeding. Seybert et al¹⁶ also found during a pharmacotherapy course that students were able to apply their learning, resulting in significant knowledge gains. Many of their cases involved cardiac dysrhythmia similar to our TdP scenario. Vyas et al¹⁷ evaluated fourth-year pharmacy students to supplement introductory pharmacy practice experiences and found a significant improvement in postsimulation test scores and most students felt more confident “making clinical recommendations to a health care provider.” This ability to have simulation assist with interprofessional learning was also observed by Vyas et al,¹⁸ when incorporating human patient simulation to teach patient safety and teamwork skills to a large interprofessional group of students including pharmacy, nursing, medicine, health professions, and health administration.

Our study found assessment scores were highest for learners who participated in both the simulation and module reading. Most pharmacy simulation studies assessed a pre- and posttest effect of the simulation activity, but a few did assess different learning modalities similar to our design.^{2,3,6,8,10,15–19} Ray et al¹⁹ compared different learning modalities with student knowledge gains on a multiple choice test pre-, post-, and 25 days after either simulation or a written patient case. Their case involved a patient with a narcotic and acetaminophen overdose. Unlike our study, they did not find a difference in knowledge gain between teaching modalities. That study was small, enrolling only 26 students, and the authors concluded it was likely underpowered. Seybert et al² found that examination scores were significantly improved after students participated in simulation exercises in a course that used both Web-based video presentations and simulation. Pharmacy students in a simulation elective listened to 10 different lectures over the course of the semester and completed an online pre- and postsimulation test. Postsimulation test scores were improved significantly in 9 of the 10 topics.² Although our study used a written module rather than a lecture prior to attending simulation, both methods produced results showing Bloom's higher order learning objectives, such as application and analysis,²⁰ can be significantly improved when conventional teaching methods are enhanced with simulation.

In order to eliminate an experiential advantage pharmacists might have over students in this study, a topic was chosen that is seen infrequently in the pediatric setting. However, the design of this study did not allow evaluation of the effect simulation interventions might have on real-life practices. Ford et al¹⁵ found a simulation and didactic lecture both had a positive effect on assessment scores, but only simulation had a positive impact on real-life practices. Interestingly, there was no statistically significant difference between mean assessment scores of students and pharmacists.

In our study, simulation yielded a statistically significant benefit for assessment questions regarding medications that prolong the QT interval, the identification of TdP on an ECG rhythm strip, and the pharmacologic management of TdP, but not on the questions regarding risk factors, electrolyte abnormalities, or presentation of TdP. Questions that were not different between simulation and module education focused mainly on lists of items (i.e., risk factors) which are lower in Bloom's taxonomy.²⁰ Lists and other memorized facts may not be best taught in simulation . Simulations better focus on Bloom's higher levels such as application of the knowledge. These questions were significantly improved among simulation participants. As for the question regarding the presentation of a patient experiencing TdP, the most common incorrect answer selected was a heart rate of 49–69 beats per minute (bpm). The correct and most commonly selected answer was a heart rate of 150 to 250 bpm. As bradycardia is a risk factor for TdP, participants may have misunderstood the question and selected a risk factor rather than clinical sign of TdP.^12,13 It is unknown why participants who had not attended simulation performed better on this question, as the simulated patient presented with an elevated heart rate when experiencing TdP episodes.

More and more articles are being published related to the use of simulation to enhance pharmacists and pharmacy students' learning focusing on topics such as the role in teaching advanced cardiac life support, counseling patients, and enhancing learning of pharmacokinetics. However there are very few simulation studies specific to pediatric pharmacists or students specifically related to pediatric studies. Kane-gill and Smithburger²¹ published an extensive review entitled, “Transitioning Knowledge Gained From Simulation To Pharmacy Practice.”²¹ Of the 57 references, there was only 1 article specifically focusing on simulation for pharmacy students on a pediatric elective.⁶ There was an article focusing on teamwork during multidisciplinary pediatrics trauma team training; however, there were no pharmacists included.²² Another pediatric team study evaluated the impact of interprofessional pediatric code team training with residents, nurses, respiratory therapists, and pharmacists and found self-efficacy of pediatric residents and nurses improved but failed to assess the impact on the pharmacists self-efficacy.²³ Although many topics in pediatrics overlap with adult care there are some unique aspects that could be better studied.

Limitations to this study include the fact that participants may have interpreted written assessment questions differently than the investigators intended. By design, we choose a limited subject (i.e., TdP) and focused on differential learning by teaching modality. However, we do not know if these findings are unique to just knowledge of TdP. Future studies will need to investigate whether these differences persist for other areas of knowledge. Although pharmacy students were randomly assigned to the timing of their simulation, pharmacists were not. By not randomizing pharmacists, bias could have been introduced resulting in self-selection influencing results. The students were randomized which helped decrease this effect. While the same general information regarding TdP was covered after each simulation, there were potential variations in post simulation discussions. However, this was decreased through the use of a script, a single lead debriefer, and small group of content experts. Learners self-defined the meanings of skimmed, read, and not read, which might have led to various definitions. During the study design, we discussed the value of researchers versus learners defining this term but ultimately felt each learner had a different definition based on their self-study routines and decided it was best for this to be defined by the learner. The students were required to attend simulation, but the assessment was voluntary, and the completion rate was 77%. This may have affected results in which possibly weaker students chose not to complete the assessment. This research did not assess retention of knowledge. Although there were more than 100 participants, the fewer number of pediatric pharmacists limits the applicability of these data widespread to other institutions. In an effort to assess the impact of educational modality, we enrolled both students and practicing pharmacists. It might have been better to have studied just 1 group.

In conclusion, simulation education has a statistically significant positive effect on assessment scores. Overall, reading the educational module and participating in simulation yielded the best scores. Participants who attended the simulation and did not read the module had higher average scores than participants who read the educational module and did not go to simulation. Furthermore study is needed to assess the long term retention of both knowledge and application of that knowledge with regards to this important dysrhythmia.

Abbreviations

bpm

beats per minute

ECG

electrocardiogram

ED

emergency department

IV

intravenous

TdP

torsades de pointes

APPENDIX 1. Written Knowledge Material of Torsades de Pointes

Torsades de Pointes (TdP), or “twisting of the point,” is a form of ventricular tachycardia where a patient's QRS complex appears to be twisting around the iso-electric line on an EKG (see image below).¹

Oftentimes, TdP resolves on its own, but it can recur frequently and even progress into ventricular fibrillation and cardiac arrest. Torsades is associated with a prolonged QT interval, which can be congenital or acquired. Acquired prolonged QT can be caused by electrolyte disturbances or drug therapy with QT-prolonging agents, making it of specific importance to pharmacists. While acquired prolonged QT alone can progress to TdP, it seems to happen more often when in conjunction with bradycardia.¹,³

Presentation¹,³ - Patients with TdP may experience palpitations, dizziness, and syncope. However, progression to ventricular fibrillation and cardiac arrest can occur with the first episode. General physical examination likely includes:

• ▪ Tachycardia of 150–250 bpm (sometimes preceded by bradycardia or palpitations)

• ▪ Normal or low blood pressure

• ▪ Pallor and diaphoresis

Risk Factors¹,³

• ▪ Congenital prolonged QT syndrome (LQTS)

• ▪ Acquired LQTS - Often caused by QT-prolonging drugs, especially in high doses or rapid IV infusions

• ▪ Bradycardia

• ▪ Female gender

• ▪ Renal or liver failure

• ▪ Baseline electrocardiographic abnormalities

• ▪ Advanced age

• ▪ Recent conversion from atrial fibrillation

Treatment of Acute Acquired LQTS TdP³,⁴

In hemodynamically unstable patients with TdP, nonsynchronized electric defibrillation is needed. In a conscious patient however, pharmacologic therapies can be used prior to cardioversion.

First-line: IV magnesium sulfate

Adult: 50% magnesium sulfate 2g IV/IO bolus over 1–2 minutes, repeated in 15 minutes needed. Diluted in 10 mL of D5W and given as a bolus in cardiac arrest due to TdP or hypomagnesemia or diluted in 50–100 mL of D5W and given over 5–60 minutes in TdP with a pulse. Sometimes followed by a continuous infusion of 0.5–1g/hr.

Pediatric: 50% magnesium sulfate 25–50 mg/kg IV/IO (max dose: 2g). given as a bolus in pulseless VT with TdP or give over 10–20 minutes in TdP with a pulse/hypomagnesemia

Second-line: Temporary transvenous overdrive pacing (pacemaker)

Generally reserved for patients who did not respond to IV magnesium sulfate

Other:

IV isoproterenol titrated to achieve a heart rate of 100 bpm. While cardiac pacing is preferred for TdP refractory to magnesium sulfate, isoproterenol can also be used while awaiting a temporary pacemaker. Adult starting dose: 2–10 mcg/min; Pediatric starting dose: 0.05–0.1 mcg/kg/min

Drug-induced TdP

Most patients who experience drug-induced TdP have more than one risk factor. Women are particularly prone to drug-induced TdP because they have longer baseline QTc and a greater response to drugs associated with long QT syndrome.³

Major Drug Classes Known to Cause Prolongation of the QT Interval:

• ▪ Antiarrhythmic drugs

  • Class 1A: quinidine, disopyramide, procainamide
  • Class III: sotalol, amiodarone, ibutilide, dofetilide

• ▪ Certain antihistamines (terfenadine, astemizole, diphenhydramine, hydroxyzine)

• ▪ Macrolide antibiotics

• ▪ Certain psychotropic medications

  • Tricyclic antidepressants, haloperidol, thioridazine, etc.

• ▪ Certain gastric motility agents

• cisapride

The list above is by no means all inclusive, as many drugs have the potential to cause prolonged QT interval. A helpful (and free!) resource for identifying QT-prolonging drugs and the risk of TdP is www.qtdrugs.org.

Counseling point! Patients being treated with a QT-prolonging drug should be counseled to report symptoms such as palpitations, syncope, or near syncope, or any change that could cause hypokalemia, such as gastroenteritis or starting diuretic therapy.

APPENDIX 2. Torsades de Pointes Learner Assessment

Assessment

1. Please check the following demographics as they apply to you:

   • □Student at Samford's MSOP
   • □Pharmacist at Children's of Alabama
   • □Age ≥ 19 years
   • □Male
   • □Female
2. Have you been to a simulation regarding Torsades de Pointes in the Children's of Alabama Simulation Center within the last 30 days?

   • □Yes
   • □No
3. Have you read the module on Torsades de Pointes?

   • □Yes, I read it.
   • □No, I did not read it.
   • □Kind of, I skimmed it.
4. BW is a 14yo female who received a kidney transplant 2 months ago. Her current medication regimen ineludes: tacrolimus, mycophenolate, prednisone, diphenhydramine, sulfamethoxazole-trimethoprim, valgancyclovir, ondanestron, and sertraline. If added to her regimen, which of the following drugs would be most likely to contribute to QT prolongation?

   1. p.o. minoeyeline
   2. p.o. erythromycin
   3. p.o. clindamycin
   4. topical tretinoin cream
5. Which of the following drug classes does NOT generally cause QT prolongation?

   1. psychotropic drugs
   2. tetraeyeline antibiotics
   3. antiarrhythmic drugs
   4. macrolide antibiotics
6. Which of the following is a risk factor for torsades de pointes? (select all that apply)

   1. rapidly infusing a QT prolonging drug
   2. Asian decent
   3. Female gender
   4. impaired renal/hepatic function
   5. bradyeardia
7. Which of the following electrolyte imbalances is a risk factor for torsades de pointes? (select all that apply)

   1. hypokalemia
   2. hyponatremia
   3. hypomagnesemia
   4. hyperkalemia
8. Which of the following is most representative of a rhythm strip depicting torsades de pointes?

   

9. A patient experiencing torsades de pointes might present with which of the following?

   1. heart rafe of 150–250 bpm
   2. heart rafe of 49–69 bpm
   3. tinnitus
   4. fever >101 degrees Fahrenheit
10. Which of the following pharmacologic therapy is commonly used in the acute management of torsades de pointes?

    1. intravenous epinephrine
    2. intravenous calcium chloride
    3. intravenous magnesium sulfate
    4. intravenous sodium bicarbonate