Improving Medical Student Toxicology Knowledge and Self-Confidence using Mannequin Simulation

Abstract

Background

Learning medicine without placing patients at increased risk of complications is of utmost importance in the medical profession. High-fidelity patient simulators can potentially achieve this and are therefore increasingly used in the training of medical students. Preclinical medical students have minimal exposure to clinical rotations and commonly feel anxious and apprehensive when starting their clinical years.

Objectives

The objective of this pilot study was to determine if toxicology knowledge and confidence of preclinical second-year medical students could be augmented with simulation training.

Methods

We designed and implemented a simulation exercise for second-year medical students to enhance learning of Basic Life Support, toxidromes, and management of a semiconscious overdose victim. Groups of 5–6 students were tasked to identify abnormal findings, order tests, and initiate treatment on a mannequin. Faculty observers provided video-assisted feedback immediately afterwards. On-line pre- and posttests were completed in the simulation lab before and after the exercise.

Results

This simulation exercise, completed by 52 students, increased test scores on average from 60% to 71% compared to a pre-test. Among the topics tested, students scored worst in identifying normal/abnormal vital signs. Mean confidence increased from 2.0 to 2.6 using a 5-point Likert scale (1-very low to 5-very high).

Conclusion

This study suggests that simulation exercises for second-year medical students may be a valuable tool to increase knowledge and student self-confidence at a key transition period prior to beginning clerkship experiences. Further research is needed to prove long-term educational benefits of simulation interventions in the preclinical setting.

Introduction

High-fidelity patient simulators are increasingly used in the training of physicians, nurses, medical students (MS), and many other healthcare professionals.¹ Learning medicine without placing a patient at an increased risk of complications is of utmost importance. In reality, MS are most commonly denied learning from mistakes because attending physicians are ethically bound to stop erroneous actions of students. However, learning from mistakes is highly effective in acquiring factual knowledge.^2–4 When teaching on a patient, the attending physician very often has limited time, especially in a busy Emergency Department (ED), and time constraints will negatively affect the MS' learning. Therefore, learning the consequences when making an error is ethically justifiable using a high-fidelity patient simulator, but not a patient.⁵

Technological advances in computer-enhanced simulation have introduced many medical management algorithms for preclinical MS and offer the ability to provide experimental learning in a risk free, fairly realistic environment, with events that can be repeated and videotaped for valuable feedback.⁶ Scenarios can be created on demand and tailored to individuals, and skills can be practiced repeatedly without undesired interference, such as a noisy ED or time constraints. Through simulation laboratories, preclinical MS become familiar with the equipment and procedures used in clinical practice; they are introduced to the use of physiological monitoring instruments and clinical decision-making in a non-threatening environment. Data suggests that this is especially important for first and second-year MS, since formal learning is very limited in the hospital environment and MS at this stage of learning feel insufficiently prepared for clinical practice in the clerkships.^7,8

Limited research has been conducted with direct relevance to simulator training for preclinical MS in an ED setting. Most trials were observational studies or self-reported satisfaction questionnaires involving small numbers of participants.⁶ Proper objective evaluation of learning strategies of preclinical MS are limited.^9,10 However, the subjective response of MS regarding the benefits of simulation exercises has been clearly positive.^11–14

The objective of this pilot study was to determine if knowledge and confidence of preclinical second-year MS could be augmented with simulation training. Learning objectives were to identify and act upon abnormal vital signs in a semiconscious overdose victim, demonstrate knowledge and protocol for basic life support (BLS), accurately perform a primary survey for a victim presenting in a semi-conscious state in an ED setting, and recognize the signs and symptoms of opioid toxicity.

Methods

Study Setting and Population

A full scale, high fidelity mannequin, model Laerdal SimMan (Laerdal Medical Corporation, USA) was utilized in a simulation center at a university-based medical school. Study participants included fifty-nine second-year MS. The University of Hawai‘i Committee on Human Subjects determined that this study was an exempt educational study.

Measurements

The students completed an on-line pre-test immediately before starting the exercise. The pre-test consisted of 10 multiple-choice questions testing the students' understanding of BLS, toxidromes, and the management of a semi-conscious victim who overdosed. Students were given 4 questions about their confidence level in providing BLS and treating acutely sick patients in a clinical setting. After the simulation exercise and a constructive feedback session, the students completed a post-test which was identical to the pre-test, the same self-confidence questions, and a course evaluation survey.

Study Design

A medical management algorithm was constructed by the Principal Investigator (BMH) and was based on core concepts identified by the course director. The algorithm was programmed into the computerized simulation mannequin such that an adverse physiologic response (i.e. dropping oxygen saturation) would occur if a student did not complete an expected task in a timely fashion. A wall-mounted video camera was positioned to record student performance during the exercise and to assist in constructive feedback after the exercise.

Prior to implementation, the simulation exercise was piloted among four 3rd year MS, revised with input from several attending physicians, and then finalized. Faculty was trained to observe students, complete a performance checklist, and provide feedback immediately following each session.

One week before the simulation exercise for the current study, all second-year MS at a Problem Based Learning (PBL) medical school studied a PBL case that was part of their required preclinical curriculum. This case featured a semi-conscious adolescent female with a complex social situation who overdosed on Tylenol. Learning issues included workup and management of patients with poisoning, clinical features of toxidromes, differential diagnosis of altered mental status, and normal/abnormal vital signs.

Several days before the actual simulation exercise, all second-year MS spent some time with a clinical instructor becoming familiar with the mannequin and the laboratory layout and equipment. All participants were certified in BLS as part of medical school orientation procedures one year prior. They also had gained experience with bag-valve mask ventilation from previous teaching sessions with the mannequin. Details about the setting, the available resources, and the tasks at hand were explained.

After completing the pretest, groups of 5 to 6 students were given 30 minutes to complete a simulation exercise using the high-fidelity mannequin (Laerdal SimMan).

A brief history was provided on the computer screen featuring a semi-conscious adolescent that was found next to her bed with an almost empty bottle of pills and then taken to the ED by paramedics. Students were instructed to collaborate with each other in the clinical care of the victim, including history at the bedside, physical examination, monitoring of vital signs, generation of a differential diagnosis, and initiation of workup and therapy. The MS were prompted to consider basic intervention such as providing bag-valve mask ventilation, starting an intravenous line, ordering tests and intravenous fluids, and giving medications. The mannequin was programmed to react to student interventions. For example, a decrease in oxygen saturation would occur if bag-valve mask ventilation was not started within 5 minutes, or an increase in blood pressure would occur if the MS gave intravenous fluids. All simulation education sessions were recorded using standard videotape for playback during post-session debriefing. The attending physician who was debriefing the students was present, gave additional history if requested by the students and also gave a few prompts if the students needed assistance.

After the simulation exercise, the students were debriefed by an attending physician with the help of the computer facilitator and video playback. The 20 to 30 minute constructive feedback sessions were provided by four different attendings, one attending specializing in pediatrics and medical education, one in family practice medicine and simulation technology, one in emergency medicine and one in pediatrics and pediatric emergency medicine. The participants were asked not to communicate about the test and the content of the simulation with the other participants during the test period. Immediately after the debriefing, the students completed a posttest which was identical to the pretest.

Data Analysis

Data analysis for the knowledge questions was conducted using a paired t-test comparing pre- versus post-test. Statistical significance was set at p<0.05. The confidence questions were analyzed using a 5-point Likert scale (1-very low to 5-very high). Fifty-nine second-year MS participated in the study successfully. Data from 7 participants who failed to complete either pre- or post-test was excluded from analysis.

Results

The simulation exercise, completed by 52 students, increased correct test answers on average from 60% to 71% (p<0.0001 by paired t-test; see details in figure 1). Increases in test results were between pre- and post-simulation identical multiple choice questions. Among the topics tested, students scored worst in identifying normal/abnormal vital signs. Mean confidence in performing BLS, evaluating a patient with drug overdose, managing a semiconscious patient and treating a patient with an acute toxic ingestion increased from 2.0 to 2.6 (p<0.0001 by paired t-test) using a 5-point Likert scale (1-very low to 5-very high) (see figure 2). On the same scale students rated the quality of the debriefing session at 4.9 for attendings with emergency medicine training versus 3.6 for other attendings (p<0.0001 by unpaired t-test).

Figure 1.

Mean Knowledge Questionnaire Results

error bars indicate standard error; *p<0.02, **p<0.002; Q1: Normal/abnormal vital signs; Q2: Signs of crystal methamphetamine overdose; Q3: Signs of codeine overdose; Q4: Difference between a sympathomimetic and an anticholinergic overdose; Q5: Class of drugs causing dilated pupils; Q6: Role of primary survey; Q7: Interpretation of an ABG; Q8: Assessment of breathing in Basic Life Support; Q9: Complications of bag mask ventilation; Q10: Treatment of morphine overdose

Figure 2.

Mean Confidence Questionnaire Results

error bars indicate standard error; **p<0.001; Q1: Confidence in performing basic life support; Q2: Confidence in evaluating a drug overdose case; Q3: Confidence in managing a semi-conscious patient; Q4: Confidence in treating a patient with an acute toxic ingestion

Thirty comments were submitted from students in the course evaluation survey, 80% were positive. Such comments included: “great chance to apply material used in the PBL case,” “great job overall in terms of real-life scenario and complexity of simulation,” “enjoyed the debriefing by our facilitator,” “really fun and eye opening,” “enjoyed the hands-on experience,” and “great learning tool.” Negative or constructive comments included: “six per group seemed too many, three or four would be a better number,” and “too clinical - better for 3rd and 4th year students.”

Discussion

This pilot study suggests that simulation exercises for second-year MS may be a valuable tool to increase self-confidence at a key transition period prior to beginning clerkship experiences. Although the clinical experience is simulated, this approach may produce a level of emotional realism that allows the students to learn as if it were a patient.

Furthermore, the study suggests that knowledge of preclinical second-year MS can be increased by augmenting a PBL case with a similar simulation exercise utilizing the same learning issues. This is logical since learning by doing and being allowed to make mistakes enhances critical thinking and acquisition of knowledge.^2–5

For some learners, simulation may allow more complex information to be understood and to be retained more efficiently compared to PBL alone. The PBL curriculum at our institution emphasizes self-directed learning. In this classroom setting, first and second-year MS work cooperatively in groups of 5 to 6 students to solve a series of cases on paper containing specific health care problems. In this educational model, students determine their own learning agenda based on the problems identified in each paper case. Of note, students at the end of the second-year tend to be very focused on independently studying basic science material for their upcoming national board examination. To generate greater interest in studying clinically relevant learning issues, our faculty integrated this simulation exercise into the existing problem-based curricula to “bring to life” existing case material from the PBL classroom. In fact, course evaluations revealed that our simulation exercise was extremely well perceived by the students which is consistent with other reports across the disciplines.^11–16

Other course evaluation comments from a minority of students suggested that the simulation experience was too advanced and “would be better for 3rd and 4th year MS.” Although the authors are aware that second-year MS have not yet had clinical experiences in an ED setting, the learning objectives addressed in the simulation exercise were the same learning issues studied in a PBL case that these students were required to complete one week prior to the intervention. In order to learn most effectively the environment needs to be both participatory and interactive. Since simulation exercises are teaching methods that require the learner to think through and react to data on a minute-to-minute basis, this teaching method requires learners to apply theory to practice in an integrated manner and therefore may be more effective than reading a textbook, listening to a lecture, or PBL.^17,18 To facilitate the transition from theory to practice, simulation exercises can easily be integrated into PBL curricula.

Several MS also had constructive comments on the course evaluations and found group sizes of 5–6 too large. Although the authors agree that a group size of 6 may limit individual student involvement around a single mannequin, at the time of intervention, our simulation laboratory only had 2 Laerdal SimMan simulators available for use for this exercise. Scheduling required that simulation sessions be completed by all second-year students over two days. Although this meant assigning students to groups of 5–6, an overwhelming majority of students still enjoyed the experience and gained both knowledge and confidence as a result. Our findings support that a group size of 5–6, although not ideal, can still result in increased knowledge and confidence among second-year students.

Interestingly, although all students participating in the intervention were certified in BLS during their first year of medical school, students scored lowest in identifying normal/abnormal vital signs (Question 1) on the multiple-choice exam. Our findings from this experience support that second-year MS do struggle with applying knowledge learned from textbooks and classroom settings. As such, a simulation exercise requiring the use and application of BLS with focus on normal/abnormal vital signs could be a valuable teaching tool for learners at this stage of training.

Students also found our feedback session to be a useful teaching strategy. The authors designed the learning in this simulation to occur through “hands-on” experience with the simulator as well as through faculty feedback in the debriefing session. Although this study was not designed to quantify which portion of the simulation experience resulted in greater learning, studies suggest that the inclusion of a debriefing session results in significantly increased performance when compared to a control group of students who do not receive feedback.¹⁹

Although no difference has been reported between the use of video-assisted feedback compared to oral feedback without videotaped review,¹⁹ our study did find differences between faculty feedback facilitators. Students in our study rated the debriefing sessions with emergency medicine trained facilitators significantly higher than debriefing sessions with clinical faculty not trained in emergency medicine. Although this study is limited by the small number of student and faculty participants, our findings support that the utilization of a debriefing session after simulation is an important component of the leaning process in simulation education.

Improving knowledge soon after a teaching exercise would be expected. However, the students maximized their knowledge on the learning issues through their very recent PBL sessions suggesting that the additional increase in knowledge resulted from the new learning approach with the simulation exercise.

The outcomes were measured immediately after the simulation and feedback sessions and more extensive research is needed to prove long-term educational benefits of simulation interventions in the preclinical setting.

In conclusion, this pilot study suggests that a simulation exercise for second-year MS may be a valuable tool to increase knowledge and student self-confidence at a key transition period prior to beginning clerkship experiences. Faculty who are interested in utilizing simulation should recognize that an emphasis on decoding normal/abnormal vital signs might be important at this stage of learning. Feedback sessions provided by attendings specializing in the topic presented may also be of benefit since they are more knowledgeable in their specialty. Smaller group sizes of about 3 to 4 may be of benefit by encouraging increased involvement of all students in the group. More extensive research is needed to look at long-term effects of simulation interventions in the preclinical setting in regards to retention of knowledge and acquired skills.