A targeted simulation curriculum can improve medical student assessment and management of acute coronary syndrome in the setting of a Clinical Practice Exam

Abstract

Background

It has been noted that increased focus on learning acute care skills is needed in undergraduate medical curricula. This study investigated whether a simulation-based curriculum improved a senior medical student's ability to manage acute coronary syndrome (ACS)as measured during a Clinical Practice Exam (CPX). We hypothesized that simulation training would improve overall performance as compared to targeted didactics or historical controls.

Methods

All fourth year medical students (N=291) over 2 years at our institution were included in this study. In the third year of medical school, the “Control” group received no intervention, the “Didactic” group received a targeted didactic curriculum, and the “Simulation” group participated in small group simulation training and the didactic curriculum. For intergroup comparison on the CPX, we calculated the percentage of correct actions completed by the student. Data is presented as Mean ± SD with significance defined as p<0.05.

Results

There was a significant improvement in overall performance with Simulation (53.5 ± 8.9%) versus both Didactics (47.7 ± 9.0%) and Control (47.9 ± 9.8%) (P<0.001).Performance on the physical exam component was significantly better in Simulation (48.5 ± 16.2%) versus both Didactics (37.6 ± 13.1%) and Control (37.7 ± 15.7%), as was diagnosis, Simulation (75.7 ± 24.2%) versus both Didactics (64.6 ± 25.1%) and Control (62.1 ± 24.2%) (P<0.02 for all comparisons).

Discussion

Simulation training had a modest impact on overall CPX performance in the management of a simulated ACS. Further studies are needed to evaluate how to further improve curricula regarding unstable patients.

Introduction

Each year in the United States millions of patients are hospitalized for medical conditions requiring urgent assessment and treatment such as acute coronary syndrome (ACS).^1-4 Best-practice guidelines have been published for proper patient assessment and management for ACS and other acute illnesses.^5-10 Adherence to these guidelines dramatically improves patient outcomes.^1,4,7,11-15 However, an equally large amount of research has shown that overall adherence to guidelines by physicians is poor.^7,16-25

Some sources have recently reported that little training is included in most medical school curricula to prepare future interns for the care of unstable patients.^26,27 Several studies have demonstrated that simulation-based medical education (SBME) can improve performance in the management of unstable patients immediately after training.^28-30 Accordingly, our institution sought to improve the training of its students for the management of acute medical conditions. One part of this curricular expansion was to teach students how to assess acute chest pain and manage ACS. We undertook the current study to investigate whether this new simulation-based curriculum improved the senior medical student's ability to properly manage a SP presenting with ACS. Our hypothesis was that SBME combined with a targeted didactic curriculum would improve student adherence to published guidelines for the assessment and management of simulated ACS several months after initial training as compared both to exposure to the targeted didactic curriculum alone and to historical controls.

Methods

The MUSC Institutional Review Board reviewed this study protocol and waived the need for IRB approval.

Students who participated in the Internal Medicine 3^rd year Clerkship the first half of year 1 were considered the “Control” group. This group received no curricular intervention. We instituted a targeted didactic curriculum during the mid-point of year 1 during the Clerkship. This group of students was the “Didactic” group whose targeted didactic curriculum included two hours of lectures entitled “The Approach to the Unstable Patient” while on the Clerkship. The content of the didactic curriculum covered the initial assessment, differential diagnosis, and management of a patient presenting with acute chest pain including the proper initial management steps of ACS. In the second year of the study, the “Simulation” group participated in small group simulation training in addition to the targeted didactic curriculum.

We administer the CPX in the first quarter of the senior year of medical school at our institution. All 4^th year medical students (N=291) that took an 8-station Clinical Performance Exam (CPX) exam over the 2-year period were included (N=144 and N=147 for years 1 and 2 of testing, respectively). The standard 7-station CPX was administered along with an added eighth station that included the unstable patient (ACS, specifically acute ST-Elevation Myocardial Infarction). Student performance on the ACS station was not included in the grade received or recorded in their transcript. The students were unaware at the time of testing that the station was not included in their grade.

We performed all simulation training and testing sessions in a setting that replicates an emergency room bay at our institution, including patient bed, bedside monitors, medical gases, etc. The environment included SPs playing the role of patient and nurse, both of whom were trained to respond to the participant via a standardized script and the use of realistic actions such as medication administration. Each station was graded by the SP playing the nurse role according to checklists developed through a modified Delphi technique. For the STEMI station, this checklist was constructed via best practice in line with current American Heart Association publications along with review by a group of interventional cardiologists at our institution according to previously reported methods (see Table 1).^31-34 All students received the didactics or the didactics plus simulation a minimum of 2 months prior to taking the CPX.

Table 1. Grading Checklist for ACS Station.

Item	Assessment
1	Performed hand hygeine
2	Introduced self to patient
3	Assessed level of consciousness (alertness/orientation: person, place, time, etc)
4	Acquired History of Present Illness (asked questions about chest pain - where, how long, how bad, etc?)
5	Acquired Past Medical History (do you have medical problems for which you see a doctor?)
6	Acquired Past Surgical History (have you had surgery? When?)
7	Acquired Family Hx (does anyone in your family have heart disease; other health problems?)
8	Acquired Social Hx (alcohol, tobacco, illicit drugs)
9	Asked about Allergies (allergies to foods or medications?)
10	Asked about Current Medications
11	Asked about last dose of Sildenafil (Viagra)
	Physical Exam (including obtaining vital signs)
12	Applied bedside ECG
13	Confirmed patent IV in place or requested IV placement (is there a working IV?)
14	Placed Pulse Oximeter
15	Applied BP cuff
16	Checked temperature
17	Checked BP in both arms
18	Auscultated heart (MUST auscultate in 4 points on chest and over carotids)
19	Ausculated lungs (Credit for >3 lung fields on EACH side)
20	Auscultated abdomen (Credit for >2 areas of auscultation PLUS palpation and “does this hurt?”/any pain?)
21	Examined extremities (Credit for checking pulses in both arms and checked for edema in legs)
22	Examined patient's neck
	Differential Diagnosis
23	Listed correct DDX (Acute Coronary (Acute MI, Heart Attack, STEMI), Pulmonary Embolus (PE), Pericarditis, Esophageal Rupture (Boerhaave's), Aortic Dissection, Pneumothorax (PTX))
	Labs and Tests
24	Ordered 12-lead ECG
25	Ordered Portable Chest x-ray
26	Ordered correct labs (BMP, CBC, Cardiac Enzymes, LFT's, Coags (PT, PTT, INR), d-dimer
	Diagnosis
27	Correctly diagnosed “Anterior-Lateral STEMI” (ST elevation MI)
28	Correctly assessed CXR - no acute cardiopulmonary problem/normal aorta/no air in mediatinum/no pericardial effusion
29	Correctly assessed labs [need to mention elevated Troponin and Glc]
	Management
30	Placed supplemental oxygen (NC if SaO2>90%, FM if SaO2<90%; must titrate for SaO2>94%)
31	Ordered Immediate Cardiology Consult
32	Requested/mentioned to activate cath lab
33	Ordered Nitroglycerin (sublingual, paste, or infusion)
34	Ordered Aspirin 325mg - verbalized that patient is to chew this
35	Ordered heparin bolus and infusion (okay to NOT know dose)
36	Participant stated that it was safe to give Heparin b/c CXR normal and BP equal in both arms
37	Ordered Plavix (clopidigrel load) 300-600mg PO
38	Ordered Lipitor or Simvastatin 80 mg PO
39	Ordered Lopressor (metoprolol) 5mg IV
40	Ordered bolus 1L Normal Saline or Lactated Ringer's
41	Ordered Mucomyst 600mg PO to be given immediately
42	Requested monitor for transport to cath lab

For grading purposes, we calculated the percentage of correct actions completed by the student in the STEMI station. In order to evaluate performance on different components of the STEMI checklist by type of training curriculum and to see if certain areas accounted for the poor performance, we performed an F-Test based on a multivariate analysis of variance (MANOVA) model, followed by one-way ANOVA on performance scores by checklist component, which were: History (11 items), Physical Exam (11 items), Labs/Test (3 items), Diagnosis (3 items), and Management/Treatment Plan (13 items). Since there were 3 groups being compared, we used the Tukey-Kramer correction for multiple comparisons within each ANOVA model.³⁵ Data is presented as Mean ± SD. All statistical analyses were conducted using SAS v9.3 (Cary, NC), and significance was defined as p<0.05.

Results

The students were divided into 3 groups based upon the curricular intervention that they received: historical controls (Control, N=80),the targeted didactic curriculum (Didactics, N=64), and the targeted didactic curriculum plus small group simulation training (Simulation, N=147). The MANOVA analysis indicated that there was significant (p<0.0001) variability in domain scores across the 3 groups. Results of the individual ANOVAs indicate that overall performance was significantly better with Simulation (53.5 ± 8.9%) versus both Didactics (47.7 ± 9.0%, p<0.02) and Control (47.9 ± 9.8%, p<0.01). Performance on the physical exam component was significantly better in Simulation(48.5 ± 16.2%) versus both Didactics (37.6 ± 13.1%, p<0.02) and Control (37.7 ± 15.7%, p<0.01), as was performance on the diagnosis component [Simulation (75.7 ± 24.2%) versus both Didactics (64.6 ± 25.1%, p<0.02) and Control (62.1 ± 24.2%), p<0.01]. There was not a significant difference in overall score or any component score when comparing Didactics with Control. Figure 1 illustrates all of these comparisons.

Figure 1.

This figure illustrates the performance in the STEMI scenario by educational intervention concerning overall performance and performance in different components of the grading checklist. The Simulation group performed better than both Didactics and Control overall and in the Physical Exam and Diagnosis components, although only to a modest degree.

Finally, the Simulation group performed significantly better than the Didactic group on the treatment component of the checklist (41.3 ± 15.0% v. 35.1 ± 14.6%, p<0.02). Of note, the physical and exam and treatment plan components of the checklist were two of the lowest scoring components in all three groups. Both components were at least 10% lower than every other component by intra-group comparison, with the exception of the differential diagnosis component among the Simulation group (see Figure 1).

Discussion

Interns must be able to assess and manage, with supervision, life threatening situations, including ACS, at least through the initial stages.^26,27,36-38 However, the best pedagogical approach for preparing medical students to enter into this role is not known. As such, we compared 3 training curricula for preparing medical students to assess and manage a patient with ACS. Our data illustrate that SBME training made a modest improvement in overall performance in management of a simulated patient with this condition. This finding contributes to what is known in this domain in a number of ways.

A number of studies have shown that SBME can markedly improve medical student performance in the management of the simulated unstable patient as compared to traditional training, in the short term.^29,30 Specifically, McCoy et al. reported in a small randomized, crossover trial that SBME was superior to didactics in training medical students to assess and manage simulated cases unstable scenarios.²⁹ But, they tested students on the same day that the educational intervention was given. In addition, the students in their study were 4^th year medical students who had self-selected to take an Emergency Medicine elective that involved this training and testing. Thus, their findings that SBME can produce near expert performance (>90% correct steps) in 4^th year medical students managing an ACS may be tempered by our data.

We have shown that this level of performance is not achieved across an entire medical school class or when the time of testing is not coincident with training. It is very likely that there is a significant decline in patient management skills over time for all students, as has been shown with respect to Advanced Cardiac Life Support testing in as little as three months after training.^39-41 It is concerning that our students performed poorly with respect to history, physical exam, and treatments. Our students perform close to the national average for all portions of USMLE exams. Simulation-based evaluation of the manner in which students assess and begin management on an unstable patient may be revealing a weakness in students that other testing is not discerning. However, as we only showed a modest improvement with the use of SBME, the curriculum needed to attain a high level of skill needs to be defined, along with whether DP and/or recurrent training can maintain this skill level over time.

Concerning the maintenance of high level performance over time, McGaghie and colleagues recently reported that SBME with DP is superior to traditional medical school curricula in a variety of specialties. There are 9 elements that make up DP, per McGaghie: highly motivated learners; clear objectives; appropriate level of difficulty; focused, repetitive practice; reliable measurements; informative feedback; error correction followed by more DP; the learner is able to master the task in the time needed; and advancement to the next task.²⁸ In our study design, the students only had one practice session in the simulation center. Perhaps the reason there was only a modest improvement is multiple opportunities are needed for DP with debriefing sessions. In addition, in our curriculum there was no system of direct advancement to the next level of performance when a certain knowledge or skill set had been demonstrated. In a meta-analysis by McGaghie and colleagues, the authors argue that the growing literature suggests that traditional clinical training is inadequate to prepare students properly in order to improve clinical performance and patient safety.²⁸ While it is likely true that SBME with DP is an effective addition to a traditional curriculum, future research needs to define how to best train medical students to reach a high level of adherence to guidelines in medical simulation involving acute care and then maintain that high level of performance into their intern year for treating acute cardiac situations, along with a myriad of other actual clinical conditions.⁴²

A few weaknesses of our study design should be mentioned. First, concerning the improved rate of correct diagnosis in the Simulation group, it is most likely that this is due to having practiced the process of diagnosing and treating a patient in acute distress during the simulation training sessions. However, this also could be due to a loss of test integrity from year to year as students share experiences. Second, the students were not randomized in this study. Student schedules are assigned by a computer so there was no deliberate placement of students. However, there is the possibility that the Simulation group performed better due to some baseline difference between groups. Third, this study was done at the beginning of the fourth year. Many students took courses in the Intensive Care Unit (ICU) or Emergency Department (ED) their fourth year, so presumably those students would be more prepared for intern year.

While the Liaison Committee on Medical Education (LCME) and ACGME do not currently require students to master management of unstable patients, many are arguing that training this domain should be included and is lacking.^26,27 Most CPX tests evaluate a student in the ambulatory setting which might be not be equivalent to evaluating a student on the acute management of an unstable patient. Furthermore, based upon this study, we believe that the addition of such training is needed. Delineating an effective pedagogical approach that can have lasting results should be the aim of future research.

Conclusion

A SBME curriculum intervention had a modest impact on performance when students were assessed at least 2 months from initial training as compared to targeted didactics and historical controls. Future research needs to elucidate the best methods by which to train students to be able to assess and manage patients with unstable conditions and how to retain these skills during their intern year.