Implementation and evaluation of a simulation curriculum for paediatric residency programs including just-in-time in situ mock codes

Abstract

OBJECTIVE:

To develop, implement and evaluate a simulation-based acute care curriculum in a paediatric residency program using an integrated and longitudinal approach.

DESIGN:

Curriculum framework consisting of three modular, year-specific courses and longitudinal just-in-time, in situ mock codes.

SETTING:

Paediatric residency program at BC Children’s Hospital, Vancouver, British Columbia.

INTERVENTIONS:

The three year-specific courses focused on the critical first 5 min, complex medical management and crisis resource management, respectively. The just-in-time in situ mock codes simulated the acute deterioration of an existing ward patient, prepared the actual multidisciplinary code team, and primed the surrounding crisis support systems. Each curriculum component was evaluated with surveys using a five-point Likert scale.

RESULTS:

A total of 40 resident surveys were completed after each of the modular courses, and an additional 28 surveys were completed for the overall simulation curriculum. The highest Likert scores were for hands-on skill stations, immersive simulation environment and crisis resource management teaching. Survey results also suggested that just-in-time mock codes were realistic, reinforced learning, and prepared ward teams for patient deterioration.

CONCLUSIONS:

A simulation-based acute care curriculum was successfully integrated into a paediatric residency program. It provides a model for integrating simulation-based learning into other training programs, as well as a model for any hospital that wishes to improve paediatric resuscitation outcomes using just-in-time in situ mock codes.

Critically ill children requiring acute resuscitation are rare (1,2). When they do occur, outcomes are poor with reports of survival-to-discharge rates at 14% to 36% (3–5). In many paediatric hospitals, residents are the first physicians to respond, but studies show that their experiences in leading or participating in real resuscitations are limited and that current training methods are inadequate (6). While Paediatric Advanced Life Support courses increase resuscitation knowledge in the short term (7), retention of skills and knowledge without the opportunity for hands-on practice has been poor (8,9).

The effectiveness of simulation in teaching acute care is well established. By creating a realistic and safe learning environment that focuses on both experiential and reflective learning, simulation has been proven to teach the knowledge, procedural skills and crisis resource management required during successful resuscitations (10–13). While multiple reports introducing mock code programs into paediatric centres have been published (14–17), few describe integrating a simulation-based curriculum into paediatric training (18,19). The goal of our project was to develop, implement and evaluate a simulation-based acute care curriculum including modular, year-specific courses and longitudinal just-in-time mock codes for our paediatric residency program at British Columbia’s Children’s Hospital (BCCH) in Vancouver, British Columbia.

METHODS

Needs assessment

A committee of content experts from BCCH reviewed the Royal College of Physicians and Surgeons of Canada (RCPSC) and Accreditation Council for Graduate Medical Education (ACGME) objectives for paediatric residency training (20,21). From these, the committee identified acute care objectives that were best suited for simulation-based learning. Resident focus groups were then conducted to determine acute care topics and skills that they felt least comfortable with. With the information obtained from the focus groups and review of RCPSC and ACGME objectives, the committee allocated learning objectives for each of the simulation-based modular courses. A summary of the objectives is outlined in Table 1.

TABLE 1.

Paediatric simulation curriculum objectives

Longitudinal just-in-time mock codes Realistic, multidisciplinary response to acute deterioration of patients
Year 1 course Critical first 5 min	Year 2 course Complex medical management	Year 3 course Crisis resource management
Knowledge	Knowledge	Knowledge
Respiratory failure	Rapid sequence intubation	Leadership
Shock	Inotropic support	Communication
Seizures	Postarrest management	Resource management
		Situational awareness
Skills	Skills	Skills
Bag-valve mask ventilation IV fluid set-up & administration	RSI and intubation Intraosseous needle insertion	Code team leader role Postresuscitation debriefing
Chest compressions
Simulation cases	Simulation cases	Simulation cases
Hypovolemic shock	Status asthmaticus	Status asthmaticus
Bronchiolitis	Septic shock	Septic shock
Anaphylaxis	Status epilepticus	Status epilepticus
Status asthmaticus	Cardiogenic shock	Cardiogenic shock
Trauma
Aspiration pneumonia
Status epilepticus

IV Intravenous; RSI Rapid sequence intubation

Modular year-specific courses

Development:

Three full-day courses were designed to meet the distinct learning objectives of paediatric residents in years 1 through 3 respectively. A blended approach was chosen to address various learning styles and core knowledge was emphasized with didactic teaching, technical competence with skill stations and clinical application with simulation scenarios. All scenarios were reviewed and pilot tested as part of the existing paediatric emergency medicine simulation-based curriculum at BCCH. Although many of the cases were similar across different years, their specific objectives and subsequent learning outcomes were noticeably different.

Implementation:

Year 1 and 2 courses were scheduled early in the academic year, while the year 3 course occurred later to prepare incoming senior residents for the code team leader role. Year-specific prereading materials including selected hospital clinical guidelines, and evidence-based consensus statements were circulated one to two weeks before the start of each course. All courses were held at the Centre for Excellence in Simulation Education and Innovation at Vancouver General Hospital (British Columbia). Courses were scheduled during protected academic time to ensure residents were able to attend. Instructors consisted of four attending staff (from emergency, intensive care and general paediatrics) familiar with simulation-based learning methodology, and each was paired with one chief resident or emergency fellow at each learning station. Instructors were provided with teaching assignments and appropriate materials one to two weeks before the start of the course.

Course agendas were developed using a similar format (Table 2). Residents were divided into groups of three to four for skill and simulation stations. Simulation stations were 15 min for the scenario followed by 15 min for staff-facilitated debriefing. Residents rotated through different roles for each simulation, providing multiple opportunities to be the team leader during each course.

TABLE 2.

Typical modular year-specific course schedule

Year 1 simulation course schedule
Time	Group A	Group B	Group C	Group D
08:00 – 08:30	Introduction to course objectives & breakfast
08:30 – 09:15	Interactive lecture: Respiratory triangle & shock
09:15 – 09:30	Orientation to simulators & debriefing
09:30 – 10:00	Sim: Hypovolemic shock/bronchiolitis	Skill: IO & IV fluid resc	Sim: Anaphylaxis/status asthmaticus	Skill: Airway & breathing
10:00 – 10:30	Skill: Airway & breathing	Sim: Hypovolemic shock/bronchiolitis	Skill: IO & IV fluid resc	Sim: Anaphylaxis/status asthmaticus
15 min break
10:45 – 11:15	Sim: Anaphylaxis/status asthmaticus	Skill: Airway & breathing	Sim: Hypovolemic shock/bronchiolitis	Skill: IO & IV fluid resc
11:15 – 11:45	Skill: IO & IV fluid resc	Sim: Anaphylaxis/status asthmaticus	Skill: Airway & breathing	Sim: Hypovolemic shock/bronchiolitis
11:45 – 12:15	Interactive lecture: Seizure clock
45 min lunch
13:00 – 13:30	Sim: Trauma/shock	Sim: Aspiration pneumonia	Sim: Status epilepticus	Skill : Q-CPR
13:30 – 14:00	Skill : Q-CPR	Sim: Trauma/shock	Sim: Aspiration pneumonia	Sim: Status epilepticus
30 min break
14:30 – 15:00	Sim: Status epilepticus	Skill : Q-CPR	Sim: Trauma/shock	Sim: Aspiration pneumonia
15:00 – 15:30	Sim: Aspiration pneumonia	Sim: Status epilepticus	Skill : Q-CPR	Sim: Trauma/shock
15:30 – 16:00	Questions & wrap up

IO Intraosseous; IV Intravenous; Q-CPR A cardiopulmonary resuscitation (CPR) measurement and feedback tool, Laerdal Medical, Norway; Resc Resuscitation; Sim Simulation

To meet the distinct learning objectives of each year, course structure and design were varied for each year. The year 1 course was designed to emphasize the assessment and management of a critically ill child in the first 5 min. This was achieved by conducting shorter, 5 min simulation scenarios with 5 min debriefings in the morning. To reinforce the importance of quality cardiopulmonary resuscitation (CPR), a CPR feedback device was used to provide real-time feedback on depth, rate and recoil of chest compressions. A fluid administration skills station was designed to give residents the opportunity for hands-on practice and to compare the efficiency of various methods. The year 2 course focused on complex medical management with longer and more challenging scenarios. The final year 3 course emphasized crisis resource management (CRM) principles. These were highlighted with carefully scripted scenarios and included one scenario with a blindfolded leader to illustrate the importance of closed-loop communication and situational awareness.

Evaluation:

Each year-specific course was evaluated with resident surveys using a five-point Likert scale, with 1 indicating strong disagreement and 5 indicating strong agreement. Categories for each course differed and were based on their unique learning objectives.

Longitudinal just-in-time mock codes

Development:

To complement and reinforce the learning from year-specific modular courses, a just-in-time, in-situ mock code program was instituted and run in a longitudinal manner by the paediatric chief residents. These simulation-based mock codes were run one to two times per month on BCCH wards. To build the just-in-time component into the mock codes, paediatric chief residents selected one of the sickest patients on the clinical teaching unit (CTU) that day and created a simulation scenario where that patient acutely deteriorated. Learning objectives for each case were determined in discussion with the CTU director and ward nurse educators (See Table 3 for sample learning objectives).

TABLE 3.

Sample learning objectives for just-in-time mock codes

Presenting diagnosis	Simulation scenario	Learning objectives
Bronchiolitis	Respiratory failure	• Effectively provide increasing respiratory support including oxygen, bag-valve mask ventilation & rapid sequence intubation
	Cardiogenic shock (viral myocarditis)	• Recognize clinical signs of and manage cardiogenic shock after deterioration following initial fluid resuscitation
Prolonged QT	Ventricular tachycardia	• Timely recognition of shockable rhythm in pulseless patient and effective CPR
Febrile neutropenia	Septic shock	• Recognize poor perfusion and provide timely administration of effective fluid resuscitation and antibiotics
Meningitis	Status epilepticus	• Identify reversible causes and progressive, anticipatory management of seizures
Rule out sepsis (infant)	Supraventricular tachycardia	• Differentiate clinical signs of and manage both stable and unstable SVT

CPR Cardiopulmonary resuscitation; SVT Supraventricular tachycardia

Implementation:

A portable, high-fidelity infant simulator was set up in a standard patient ward room. Relevant patient documents were copied and modified to coincide with deterioration while protecting patient confidentiality. Nursing records, flow sheet vital signs, medication administration records and recent physician orders were made available when appropriate, but identifying information was hidden. The actual bedside nurse was involved whenever possible and often initiated the mock code with a concerned call to the junior resident without prior notification of a mock code event.

At BCCH, the multidisciplinary code team consists of CTU residents, bedside and ward charge nurses, respiratory therapists and intensive care unit code team (nurses and residents). Each mock code was carried out in real time with progressive involvement of all members of the code team as needed and generally lasted 20 min in total. The chief resident, CTU director and nurse educator directly observed and facilitated a 10 min debriefing session immediately afterward. In addition to an annual year-specific simulation course, each paediatric resident should participate in four to eight (average of six) just-in-time mock codes on CTU and intensive care unit rotations during a three-year training period.

Evaluation:

Residents completed a survey six months after the implementation of just-in-time mock codes and after all modular courses had been run at least once. It contained 10 items and used a five-point Likert scale.

RESULTS

Modular year-specific courses

The year 1 course was evaluated with 14 resident surveys using a five-point Likert scale. Because this course focused on the critical first 5 min, it was important that residents felt shorter scenarios in the morning emphasized initial assessment and management (mean score 4.7) and felt more comfortable with initial assessment and management of sick patients at the end of the course (mean score 4.8). The year 2 course was evaluated with eight resident surveys. See Figure 1 for a summary of year 1 and 2 course survey data. The year 3 course was evaluated with 18 resident surveys. See Figure 2 for a summary of the quantitative results from the year 3 course, and Table 4 for a summary of qualitative feedback from all three courses.

Figure 1).

Summary of year 1 and 2 course evaluation survey

Figure 2).

Summary of year 3 course evaluation survey

TABLE 4.

Summary of qualitative feedback from modular courses

Question	Course	Comments
What was the best part of this experience?	Year 1	Lectures complemented/reinforced the stations well Catered to us at this stage in our training Helped us become comfortable in that first 5 scary minutes
	Year 2	Scenarios were representative of cases we see, exposure Practice skills and learn from mistakes Appreciated chance to draw up medications and prime lines
	Year 3	Oriented to real problems we encounter Good focus on process rather than medical knowledge
	All years	Fun/safe/non-threatening atmosphere Building confidence and teamwork/communication skills Much more realistic and practical – not just talking about it Lots of good/great feedback during debriefing sessions
How would you improve the course?	Year 1	Stations of intravenous starts and practicing intraosseous starts Very helpful when faculty modelled a good initial assessment Discuss communication techniques early Improve flow between stations to decrease waiting time
	Year 2	Add cardiac arrest station, more scenarios Real intravenous insertion of each other
	Year 3	Two courses per year, every 6 months
	All years	More practice opportunities, more simulation

Longitudinal just-in-time mock codes

Just-in-time mock codes were evaluated with 28 resident surveys (Figure 3). Residents agreed that they reinforced learning from year-specific courses (mean score 4.3), creating scenarios using real ward patients made mock codes more realistic (mean score 4.4), including multidisciplinary team improved learning (mean score 4.3), and they prepared ward teams for potential deterioration of sicker patients (mean score 4.5).

Figure 3).

Just-in-time mock codes evaluation survey

Overall simulation curriculum

The early impact of the entire simulation curriculum was evaluated by 28 resident surveys (Figure 4). Residents found that simulation provided a safe learning environment (mean score 4.6) and was better than didactic lectures at teaching acute care, communication and teamwork issues (mean score 4.7). They also believed that the simulation curriculum would help to decrease their level of anxiety in future resuscitations (mean score 4.5) and should be implemented in all paediatric residency programs (mean score 4.6).

Figure 4).

Overall simulation curriculum evaluation survey

DISCUSSION

Previous generations of health care providers have learned acute care medicine based on the “see one, do one, teach one” mantra, while the current generation of trainees have the opportunity to practice to perfection in simulation, grounded in the principles of improving patient safety and outcomes (22). Our curriculum safely increased resident exposure to critically ill children in a realistic, yet controlled, environment, while concurrently addressing RCPSC and ACGME learning objectives required for certification of training. Furthermore, an emerging body of evidence suggests that the greatest benefit of simulation may reside in its ability to train multidisciplinary teams, help identify human errors, and modify team behaviour leading to a reduction in medical errors (23–25). In one study, Hunt et al (26) examined 34 unannounced, hospital-based paediatric mock codes and found that 100% had communication errors and represented an important opportunity to improve real patient outcomes. We addressed this need throughout our curriculum by using multidisciplinary teams whenever possible, discussing communication and teamwork issues during all debriefing sessions, and focusing our final modular course on CRM skills. Our survey data reflect this potential benefit with the strongest scores occurring in the areas of communication and teamwork.

The evidence also suggests that teaching resuscitation through Paediatric Advanced Life Support courses and didactic lectures lead to significant decay in knowledge and skills (8,9). Without the reinforcement of regular hands-on practice, subsequent resuscitations on real patients are likely to be suboptimal. A recent review of existing best practices for resuscitation education coincidently included reducing course duration and instead distributing practice sessions over time (27). It also suggested maximizing the time spent in deliberate practice on manikins, constructing scenarios that match the learners’ usual clinical practice, and integrating simulation into resuscitation curricula. In addition, a recent study has shown that as few as two ‘rolling refreshers’ per month in the critical care setting using just-in-time CPR training can improve performance in subsequent real resuscitations (28). In designing our own curriculum and considering the evidence, the concept of just-in-time mock codes was incorporated to address multiple learning needs simultaneously. First, they were distributed at monthly intervals to allow regular hands-on practice and help prevent knowledge and skill decay. Second, they were multidisciplinary and involved the actual paediatric code team to help identify and reduce team-work and communication errors. Third and most importantly, just-in-time mock codes were as realistic as possible and meant to be predictive of patient deterioration. By using real patient information and documents in combination with a portable paediatric simulator, we were able to recruit the actual bedside nurse and have the responsible CTU residents as first responders. This not only enhanced the situational and emotional realism, but often identified issues with actual equipment, access to resources, and other obstacles to a timely response that are otherwise difficult to recognize. In essence, just-in-time mock codes trained the human element and primed the surrounding crisis support systems to have the greatest potential for improving our sick children’s care.

Our curriculum was by no means a static proposition but an evolving educational process. In comparison to the simulation-based curriculum for paediatric emergency fellows at our centre (18), our residency trainees’ learning objectives varied widely. From the first-year resident with minimal clinical experience to the fourth year resident who will soon graduate to assume responsibilities as a community paediatrician, we approximated the steep learning curve with our integrated curriculum framework while allowing for flexibility in the longitudinal growth of any given cohort as they pass through the curriculum. In accordance, we noted an evolution in perceived value of various teaching methods at different stages of training. First-year resident survey scores and feedback suggested greater relative benefit from lectures and may reflect increased knowledge gaps. In comparison, second-year residents appreciated the hands-on procedural skills and complex medical simulation scenarios, while senior resident feedback emphasized the benefit of focused teaching on communication and leadership skills. Each year-specific course built upon the foundation from the previous, every just-in-time mock code reinforced the knowledge and skills that could have potentially been lost, and annual needs assessments ensured that the curriculum continued to adapt. While many paediatric centres have used simulation in some capacity, our experience suggests that an understanding of your centre’s unique resources and opportunities, an evolving assessment of your trainees’ needs, combined with a commitment to an integrated and longitudinal approach for teaching resuscitation through simulation, may have the most lasting impact on future generations of trainees and their patients.

Our study’s primary limitation was that we were not able to evaluate resident performance because the study design did not include a pre- and postintervention comparison or control group. However, studies have shown positive correlations between resident self-assessment surveys and performance in resuscitation team leadership skills and critical resuscitation procedures (29,30). The overall curriculum surveys may have been susceptible to recall bias because they were performed at least six months after the trainees’ modular course, but this increased the likelihood of both participating in just-in-time mock codes and having the opportunity to apply their learning in real situations. Future research is needed to determine the impact of educational interventions such as ours on resuscitation performance and patient outcomes.

CONCLUSION

We used an integrated and longitudinal approach to develop, implement and evaluate a successful simulation-based, acute care curriculum for our paediatric residency program. Resident satisfaction surveys highlighted the effectiveness of hands-on skill stations, immersive simulation environment and CRM teaching. The curriculum provides a model for integrating simulation-based learning into other training programs as well as a model for any hospital that wishes to improve paediatric resuscitation outcomes using just-in-time in-situ mock codes. Further research is required to assess the impact of educational interventions such as these on performance and patient outcomes.