Improving Toxicology Knowledge in Preclinical Medical Students Using High-Fidelity Patient Simulators

Abstract

Background

Superior patient care and optimal physician training are often mutually elusive in the Emergency Department setting. Highfidelity patient simulators (HFPSs) are being used with increasing frequency in the training of medical students (MS) because they enable students to develop and refine medical competency in a non-threatening and safe environment. However, learner outcomes using HFPSs in this setting have not been well studied.

Objectives

The objective of this pilot study was to determine the effectiveness of HFPSs in simulation (SIM) training as a learning tool for preclinical second-year MS to further increase their toxicology knowledge.

Methods

Second-year MS at a Problem Based Learning (PBL) medical school received a PBL toxicology teaching session in the middle of the semester. One week later, the students participated in a SIM exercise based on issues taken from the PBL case. The SIM exercise required that students address learning issues such as identifying abnormal findings, ordering tests, and, ultimately, initiating treatment on a full-scale HFPS mannequin. A supervised on-line test consisting of 10 multiple choice questions regarding the student's understanding of the learning issues was completed before the PBL class and directly before and after the SIM to determine the effectiveness of the HFPS use. Immediate video-assisted feedback was provided by emergency medicine attendings.

Results

Use of HFPSs during SIM exercises and in combination with PBL significantly increased toxicology knowledge in secondyear MS as determined by the improvement of on-line test scores (% correct answers) from 59% before PBL / before SIM to 69% after PBL / before SIM to 80% after PBL / after SIM.

Conclusion

This study suggests that HFPS may be a valuable tool in helping to improve toxicology knowledge in second-year MS at a key transition period prior to beginning clerkship experiences. Incorporation of HFPS into PBL curricula may also be beneficial to MS in other areas of study where interactive learning could assist in evoking emotional realism while also enhancing critical thinking and acquisition of knowledge thereby facilitating the transition from theory to practice.

Introduction

High-fidelity patient simulators (HFPSs) are computer controlled life-size mannequins externally and internally constructed to allow multisystem physiological real-time changes to occur in response to human intervention. HFPSs are appealing in the medical field because they have the ability to evoke emotional realism to a situation in a non-threatening manner.¹ For that reason, HFPSs are being increasingly employed in the training of physicians, nurses, medical students (MS), and many other healthcare professionals.²

The practice of learning medicine without placing a patient at an increased risk of complications is of utmost importance. While it would be ideal for MS to learn from their mistakes in real-life situations, in reality, MS are commonly prevented from doing so as attending physicians are ethically bound to stop such erroneous actions. However, learning from mistakes is highly effective in acquiring factual knowledge.^3–5 Therefore, learning the consequences when making an error is ethically justifiable using an HFPS as opposed to an actual patient.⁶

Advances in computer-enhanced simulation technology have led to the development of many medical management algorithms and given MS the opportunity to experiment and learn medicine in a risk free and fairly realistic environment.⁷ Using medical management algorithms, realistic clinical scenarios can be created and tailored to individuals such that specific skills can be practiced repeatedly. Through simulation (SIM) training laboratories, preclinical MS become familiar with clinical equipment and procedures. SIM laboratories also introduce MS to the use of physiological monitoring instruments (e.g. HFPSs) and help to integrate clinical decision-making in a non-threatening environment. SIM training is especially important for first and second-year MS because, at this stage, formal learning is very limited in the hospital and MS often feel insufficiently prepared for clinical practice in the clerkships.^8,9

There are few studies which directly address SIM training for preclinical MS in an emergency department setting. Most trials are observational studies or self reported satisfaction questionnaires involving a small number of participants.⁷ A few articles address objective evaluation of learning strategies of preclinical medical students.^10,11 The subjective response of MS regarding SIM exercises are clearly positive.^12–15

The objective of this pilot study was to determine the effectiveness of HFPS use during a SIM training exercise among preclinical second-year MS. More specifically, this study was aimed at determining whether toxicology knowledge and confidence in preclinical second-year MS could be improved by augmenting a PBL case with a SIM exercise using an HFPS. Learning objectives for the students were to identify and act upon abnormal vital signs in a semi-conscious overdose victim, and recognize the signs and symptoms of opioid toxicity.

Methods

Study Setting and Population

Fifty second-year MS at a Problem Based Learning (PBL) medical school participated in this study. One week before the SIM exercise, the students studied a toxicology PBL case as part of their required preclinical curriculum. The PBL case featured a semi-conscious adolescent female who overdosed on acetaminophen. Learning issues for the case included: identifying normal/abnormal vital signs, workup and management of patients with poisoning, clinical features of toxic states, and the differential diagnosis of altered mental status.

Four weeks before the SIM exercise, the students spent time with a clinical instructor familiarizing themselves with the laboratory layout, laboratory equipment, and a full-scale HFPS Laerdal SimMan mannequin (Laerdal Medical Corporation, USA). This mannequin was used in a previous teaching session involving bag-valve mask ventilation and would also be used for the current SIM exercise. Details about the setting, the available resources, and the tasks at hand for the SIM exercise were explained to the students.

The Committee on Human Subjects at the University of Hawai‘i determined that this study was an exempt educational study.

Study Design and Measurements

Prior to implementation with the second-year MS, the toxicology SIM exercise was piloted among four third-year MS, revised with input from several attending physicians, and then finalized. Faculty were trained to observe students and provide immediate feedback after each SIM exercise.

For the current SIM exercise with the second-year MS, a medical management algorithm was constructed by the Principal Investigator (BMH) and was based on core concepts identified by the course director. The algorithm was programmed into the HFPS mannequin such that adverse physiologic responses would occur if a student did not complete an expected task in a timely fashion. A wall-mounted video camera was used to record student performance during the SIM exercise and to assist in constructive feedback after the SIM exercise.

Before studying the PBL case, the students completed a supervised on-line test consisting of 10 multiple-choice questions. The purpose of the test was to determine the student's baseline understanding of the learning issues for the PBL case; eg, normal and abnormal vital signs, toxicology, and the management of a semi-conscious individual that overdosed.

After studying the PBL case and before the SIM exercise, the students completed a second online test identical to the first one they had just taken along with a 4-question survey regarding their level of self-confidence in treating acutely sick patients in a clinical setting. Upon completing the test and survey, groups of 3 to 4 students were then given 30 minutes to complete the SIM exercise using the mannequin. For the SIM exercise, a PBL case was provided on a computer screen and featured a semi-conscious adolescent female who was found next to her bed with an almost empty bottle of acetaminophen and then taken to the emergency department by paramedics. Students were instructed to collaborate in groups of 3–4 to discuss the clinical care of the victim, including history at the bedside, physical examination, monitoring of vital signs, generation of a differential diagnosis, and initiation of workup and therapy. The students considered basic interventions such as providing bag-valve mask ventilation, starting an intravenous line, ordering tests and intravenous fluids, and giving medications. The mannequin was programmed to react appropriately to the student's interventions, eg, a decrease in oxygen saturation would occur if bag-valve mask ventilation was not started within 5 minutes, or an increase in blood pressure would occur if intravenous fluids were given. All SIM exercises were video recorded for playback during post-session debriefing. The emergency medicine physician debriefing the students was present during the SIM exercise and provided additional patient history if requested.

After the SIM exercise, the students received a 20–30 minute debriefing session in which constructive feedback was provided from emergency medicine attendings with the help of the computer facilitator and video playback. The attendings were instructed not to discuss any of the online test questions during the debriefing session. The students were not aware of the algorithm programmed for the SIM exercise nor did they know they would be receiving the same online multiple choice questions repeatedly. The students were asked not to discuss the test questions or SIM exercise amongst each other during the study phase and were assured that they would not be graded. Immediately following the debriefing session, the students completed another online test identical to the previous two online tests, another self-confidence survey, and a course evaluation.

Data Analysis

Data were analyzed by paired student's t-test and single-factor (parametric) analysis of variance (ANOVA) using Excel 2004 software (Microsoft, Redmond, WA) and were presented as mean ± standard error (SE). Differences at p<0.05 were considered significant.

Results

Evaluation of the 10 identical online multiple-choice questions (Q1–10; see Figure 1) completed among the 50 students before PBL and before SIM (stage 1; S1) versus after PBL and before SIM (stage 2; S2) versus after PBL and after SIM (stage 3; S3) revealed an overall significant improvement in correct answers from S1 to S2 (59% to 69%; p < 0.05), from S2 to S3 (69% to 80%; p < 0.05), and logically also from S1 to S3 using ANOVA and individual paired t-tests.

Figure 1.

Mean Knowledge Questionnaire Results

A trend showing improvements in knowledge for the online questions between each stage (S1–S3) was also observed except for Q1 and Q10, which exhibited no changes in knowledge between all stages. These two questions tested the students on the topics of identifying normal/abnormal vital signs and the treatment of morphine overdose, respectively. The only topic with a worse score at S2 was signs of codeine overdose (Q3). However, scores at S3 showed a level of knowledge on this topic similar to that at S1 (Figure 1). Among the 10 topics tested, students scored worst in the treatment of morphine overdose (Q10) with 8–20% correct answers while all other correct answers ranged between 30% and 96% (for details, see Figure 1).

In addition, the mean confidence level for performing basic life support (BLS), evaluating a patient with drug overdose, managing a semiconscious patient and treating a patient with an acute toxic ingestion increased from 2.5 to 3.3 using a 5-point Likert scale (1-very low to 5-very high) after the SIM exercise compared to before the exercise.

Discussion

Results from this pilot study suggest that SIM exercises may be a valuable tool in helping to improve toxicology knowledge in preclinical second-year MS at a key transition period prior to beginning clerkship experiences. Although the clinical experience is simulated, this approach can help to produce a level of emotional realism and urgency in MS thereby forcing them to reason through critical decisions that they may encounter in real-life. This is logical since learning by doing and being allowed to make mistakes enhances critical thinking and acquisition of knowledge.^3–6

For some learners, simulation may allow complex information to be understood and retained more efficiently compared to PBL alone. The PBL curriculum at our institution emphasizes self-directed learning and, in this classroom setting, first and second-year MS work cooperatively in groups of 5 to 6 to solve a series of cases on paper containing specific health care problems. In this educational model, students determine their own learning agenda based on the problems identified in each paper case. Of note, MS nearing the end of their second year tend to be very focused on independently studying basic science material for their upcoming national board examination. To generate greater interest in studying more clinically relevant issues, our faculty integrated the current SIM exercise into the existing PBL curricula in order to “bring to life” existing case material from the PBL classroom thus facilitating the transition from theory to practice. Course evaluations revealed that our SIM exercise was extremely well perceived by the students, which is consistent with other reports across the disciplines.^12–17

While popularity should not define the usefulness of an educational experience, we believe that if learning medicine can be made enjoyable for MS, it can spark their interest to learn and, therefore, the experience would be valuable. In fact, some educators suggest that HFPSs may accelerate the acquisition of knowledge due to the emotional power of the simulated encounter. The findings suggest that both student knowledge and confidence increased. How this translates to actual performance in the Emergency Department setting would be of additional importance, but is beyond the scope of this study.

The learning objectives addressed in the SIM exercise were identical to those studied in the PBL case one week prior to the SIM exercise. In order to learn effectively, the environment needs to be both participatory and interactive. A SIM exercise is a teaching method that requires learners to think and react on a minute-to-minute basis. It requires learners to apply theory to practice in an integrated manner. Therefore, SIM exercises may be more effective than passively reading a textbook or listening to a lecture.^18,19 They can be easily integrated into PBL curricula to facilitate the transition from theory to practice.

Although all students participating in the SIM exercise were certified in Basic Life Support during their first year of medical school, their knowledge did not improve in identifying normal/abnormal vital signs after the PBL session or after the SIM exercise (Q1). The findings from this SIM exercise experience support our belief that second-year MS do struggle with applying knowledge learned from textbooks and classroom settings. As such, SIM exercises requiring the use and application of BLS with focus on normal/abnormal vital signs could be a valuable teaching tool for MS at this stage of learning.

The subjective response of MS regarding SIM exercises was unanimously positive. The twenty-four comments submitted from students in the course evaluation were all positive. Such comments included: “this was a great experience since it combined both theoretical knowledge of medicine with actual application,” “great job overall in terms of real-life scenario and complexity of simulation,” “the feedback at the end of the simulation was very helpful and pointed out where we had missed important steps or tests,” “Outstanding, more sim labs please.”

Students also found the feedback session by the emergency medicine physicians to be a useful teaching strategy. The authors designed the SIM exercises so that learning would occur through “hands-on” experience with the mannequin as well as through faculty feedback in the debriefing session. Although this study was not designed to quantify which portion of the SIM experience resulted in greater learning, studies suggest that students who receive a feedback debriefing session during SIM exercise display significant increases in performance compared to students who do not receive feedback.²⁰

Limitations

Perhaps the greatest limitation in the current report is the repetition of test questions between sessions. Since the questions from the second and third test were identical to the first, it is possible that, after the first test some students may have looked up or discussed the questions and/or answers amongst each other. If so, this could have resulted in the increase in correct test answers before, but less likely after the SIM exercise. Because there was no control group in this study, it is difficult to conclude that the SIM exercise, rather than the additional teaching exercise resulted in the improvement of test scores.

Nonetheless, due to the short time span between the PBL case and SIM exercise (1 week), it is reasonable to assume that, after studying the PBL case, knowledge on the learning issues were already maximized and that the additional increase in knowledge thereafter (eg, increased test scores between sessions) was the result of the SIM exercise.

In conclusion, this pilot study suggests that incorporating HFPS into SIM exercises may be valuable in increasing knowledge and self-confidence in preclinical second-year MS during their key transition period prior to beginning clerkship experiences. Faculty who are interested in using SIM should recognize that an emphasis on decoding normal/abnormal vital signs may be important at this stage of learning. More extensive research is needed to look at longterm effects of simulation interventions in the preclinical setting in regards to retention of knowledge and acquired skills.