Rapid‐cycle deliberate practice improves time to defibrillation and reduces workload: A randomized controlled trial of simulation‐based education

Abstract

Background

The optimal structure of simulation to train teams to perform pediatric advanced life support (PALS) requires further research. Most simulation is structured with an uninterrupted scenario with postsimulation debriefing (PSD). Rapid‐cycle deliberate practice (RCDP) is structured with a series of simulations with microdebriefing quickly switching within action targeting specific performance goals.

Objective

The objective was to compare team performance immediately after training, as well as learner workload, for teams trained using either PSD or RCDP.

Methods

In 2018–2019, a total of 41 interprofessional teams of 210 residents and nurses were recruited from 250 eligible participants (84%) and randomized into either arm (RCDP or PSD) teaching the same objectives of resuscitation of a patient in PEA arrest, in the same time frame. The structure of the simulation varied. Demographic surveys were collected before training, the National Aeronautics and Space Administration‐Task Load Index (NASA‐TLX) was administered immediately after training to assess workload during training and performance was assessed immediately after training using a pulseless ventricular tachycardia arrest with the primary outcome being time to defibrillation.

Results

Thirty‐nine teams participated over a 16‐month time span. Performance of teams randomized to RCDP showed significantly better time to defibrillation, 100 s (95% confidence interval [CI] = 90–111), compared to PSD groups, 163 s (95% CI = 120–201). The workload of the groups also showed a lower total NASA‐TLX score for the RCDP groups.

Conclusions

For team‐based time‐sensitive training of PALS, RCDP outperformed PSD. This may be due to a reduction in the workload faced by teams during training.

INTRODUCTION

The care of the pediatric cardiac arrest victim requires an interprofessional team to immediately take specific team‐based actions, gather information about the patient, share a mental model, and reverse underlying causes. ¹ Deviating from guidelines for pediatric advanced life support (PALS) is associated with decreased survival. ² For expert teams the immediate actions become automatic and they can focus on the steps which require greater adaptability. Team training to improve the care of cardiac arrest patients is necessary. ³

Studies confirm the benefits of clinical simulation in the development and maintenance of skills for physicians and medical staff; however, certain simulation designs may be more effective than others. ⁴ , ⁵ , ⁶ , ⁷ , ⁸ , ⁹ Traditionally, simulations are designed to allow a participant or group of participants to complete an entire clinical scenario, followed by a reflective debriefing. Postsimulation debriefing (PSD) leads to improved performance. ¹⁰ , ¹¹ , ¹² In contrast to PSD, rapid‐cycle deliberate practice (RCDP) cases are divided into small portions or steps. ¹³ , ¹⁴ This method also has been shown to improve performance in simulation before versus after training. ¹³ , ¹⁵ , ¹⁶ Facilitators stop the simulation either when an error or suboptimal action occurs or to highlight and discuss correct actions.

Studies comparing RCDP and PSD show mixed results. This may be from variations in specific methods, level of learner, and topics being taught. ¹⁷ In prior comparison studies, teams using PSD progressed through multiple cases before training or during training before assessment and had similar improvement to RCDP. ¹⁵ Others have found immediate performance improvement with RCDP compared with PSD for basic life support and neonatal resuscitation training. ¹⁸ , ¹⁹

Cognitive load theory may partially explain the difference in effectiveness between various simulation protocols. This psychoeducational framework posits that working memory is limited and that when working memory is overburdened by high complexity or multiple concurrent tasks, learning is decreased. By reducing the workload required for educational exercises without eliminating essential tasks (reducing extraneous load and optimizing intrinsic load), learning may be fostered. ²⁰

The primary objective of this study was to compare time to first defibrillation in simulation immediately after RCDP or PSD training. Secondary outcomes measured time to first compression, time to first epinephrine, and workload of teams during RCDP and PSD.

METHODS

Study setting and participants

The study was conducted in the in‐situ simulation suite of the emergency department at Texas Children's Hospital, an urban tertiary care hospital, from January 2018 to April 2019.

As part of their pediatric emergency medicine (PEM) rotation, pediatric and emergency medicine residents were required to participate in simulation‐based resuscitation training. This training session has been part of our resident rotation and ongoing nurse education and serves to augment the regular PALS certification process. Our group of PEM physicians and PEM nurse educators has developed a set of learning objectives based on American Heart Association guidelines as adapted to our specific institution. ¹ Depending on the residency program and postgraduate year, residents were scheduled to have one to three PEM rotations during their residency. On each training day for every PEM rotation, the team included three or four residents and two nurses. The team was randomized to either RCDP or PSD. Roles included team lead, first responder, airway, cardiopulmonary resuscitation (CPR) coach, bedside provider, and recorder. For teams of five, the CPR coach was removed. Teams taught with PSD participated in a single simulation scenario with a single debriefing session. The role that each individual filled was randomized but ensured that a nurse was the recorder and a physician was the lead. All other roles could be filled by either profession. For RCDP, the initial role assignment was also random, but the structure of RCDP encouraged rotation to the other roles in subsequent rounds. Testing case role assignment was determined by the most recent role performed.

Randomization of teams

Randomization.com was used to develop block randomization in blocks of six. Everyone was unaware of that day's randomization until consent had been obtained.

Simulation curriculum design: simulation with PSD

Teams randomized to RCDP or PSD were taught the same topics in the same time frame. For PSD, the training session included one uninterrupted 20‐min scenario of an unresponsive child presenting in PEA arrest. The case started with a first responder entering the room, calling for help, and the remainder of the team entering 10 s later. The case began with initiation of CPR, including backboard and monitor placement. It then proceeded through the first pulse‐rhythm check including preparation for possible defibrillation, intraosseous (IO) access, and epinephrine administration until the patient had return of spontaneous circulation and required postarrest stabilization and intubation. This was followed by a 40‐min reflective debriefing session. PSD was conducted using the Promoting Excellence and Reflective Learning in Simulation (PEARLS) framework with a scripted debriefing. ¹¹ This has been the standard method used by our simulation instructors at our institution. Details of the curriculum are in Appendix S1 (available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10702/full).

RCDP simulation

For the RCDP educational intervention, sessions included multiple rounds of progressively more difficult scenarios with predesigned “hard stops” and “soft stops” with scripted learning points. These cases were published in MedEdPORTAL and adjusted to fit our time constraint of a 1‐h session. ²¹ Teams achieved predefined goals in each round before progressing to the next level of difficulty. Instructors focused on providing direct feedback, using a pause and rewind/restart methodology. Instructors used a debriefing script based on the PEARLS framework. ¹⁵ Teams had the opportunity to “rewind” and achieve objectives before moving onto the next round. The sessions taught with RCDP lasted 1 h.

The first round presented an unresponsive child with apnea but with a pulse. The team's objectives were to quickly assess the patient and recognize the need for additional help and a crash cart, apply monitors, and reposition the airway. Once those objectives were achieved, the team moved onto the second round, which was an apneic patient requiring bag‐mask ventilation. Subsequent rounds added additional complexity, until the final round which replicated the PSD case. With each round, teams rotated roles, giving everyone a chance to lead.

While debriefing in RCDP was more directive than in PSD, instructors were encouraged to use advocacy and inquiry methods to explore persistent performance gaps. In contrast to RCDP, PSD allowed more time for reflection on individuals’ frames and correction of the underlying frame. Even in PSD, instructors would provide direct feedback focusing on choreography of pediatric resuscitation. Both groups had the opportunity to review microprocedures, such as preparing defibrillator and IO placement. In PSD, these skills were taught through explanation and demonstration. During RCDP, individuals could practice these skills in subsequent rounds after being taught. The only differences in debriefing from PSD lay in how much emphasis was placed on direct feedback versus advocacy‐inquiry technique and the timing of that feedback.

Instructors and instructor training

A team of two PEM physicians and a nurse taught each day; a research assistant obtained consent and collected data. All educators had more than 2 years of experience teaching the material and had taken simulation instructor training along with train‐the‐trainer workshops to standardize RCDP and PSD implementation. Mannequin settings were standardized by a preprogrammed simulation in Laerdal learning application (Laerdal Medical).

Study protocol and data collection

After consent, participants completed a demographic survey administered through a Research Electronic Data Capture survey. ²² , ²³ This survey covered prior training and resuscitation experience. Next, instructors conducted an orientation to simulation and the SimJunior mannequin (Laerdal Medical). Then instructors taught for 1 h using RCDP or PSD.

After the training, individuals filled out a survey to rate the workload they experienced during training. The NASA‐Task Load Index (NASA‐TLX) ²⁴ survey measures six components of workload, three related to the demands of the task faced (mental, physical, and temporal) and three related to the reactions of the individual to the demands (frustration, effort, and performance). Every individual ranked each workload component on a visual analog scale between 1 and 100. The individual ranked the six components in order of contribution to workload, generating a weighted total scale ranging from 0 and 100. While there are no predefined levels for too high or too low workloads, comparisons can be helpful. In general, above 60 is considered to be a high workload, while below 40 is considered low. ²⁵

After the hour of education and workload survey, the team was then assessed as they completed a simulated uninterrupted test case of a child in cardiac arrest with pulseless ventricular tachycardia (VT). Of note, there was no assessment before the training to avoid introducing repetitive practice into the PSD arm. In contrast with the training, this scenario required defibrillation and use of an antiarrhythmic. This 10‐min scenario was video recorded. Key performance metrics were obtained from video review, including time from the first team member's entry into room until time of first chest compression, first defibrillation, and first epinephrine administration. All time measurements were made by one investigator (DSL), who was blinded to the study assignment.

Sample size calculation and statistical analysis

Our primary outcome was predefined as time to first defibrillation. Based on pilot data from a prior study of PEM fellows ¹⁵ we calculated a sample size of 16 teams based on an estimated average time to defibrillation of 140 s with an improvement of 30 s and a standard deviation of 30 s. Using a two‐sided tail with alpha of 0.05 and beta of 0.20, this yielded an estimated sample size of 16 in each arm. ²⁶ , ²⁷

Planned secondary analysis included the time to compressions and time to first epinephrine. Finally, the workload of each group of teams as measured by NASA‐TLX (total and subscore) was measured and compared between the groups.

Demographic data were checked for normalcy using the Shapiro‐Wilk test. Categorical comparisons were calculated using Pearson chi‐square test or the Fisher exact test if any value was <5. Continuous variables were analyzed using the Mann‐Whitney or Kruskal–Wallis test. A p‐value <0.05 was defined as statistical significance. All analyses were conducted using the Statistical Package for the Social Sciences, version 25 (IBM Corp.).

Institutional review board statement

This study was approved by institutional review board at our institution. The research and education were funded internally from our institution. Prior to the start of the day, a research assistant obtained verbal consent from all participants. If any member declined consent, the educational session continued but study data were not collected.

RESULTS

Participants

In total 250 participants were eligible for training and had been scheduled for training. As outlined in Figure 1, 21 were taught on days alongside PEM fellows and these days were not randomized into this study; 19 residents returned for a second round of training. While their workload and survey data were excluded from analysis, the teams to which they were assigned were included (Figure 1). There were 20 RCDP groups and 21 PSD groups. There were no significant differences between clinical experience of the individuals or groups. There was variation in size between groups, but the variation in size was equivalent between the groups (Table 1).

FIGURE 1.

Flow of participants through study: 250 recruited, repeat participants were allowed to participate in teams, but their responses to workload questions was not included in analysis. PSD, postsimulation debriefing; RCDP, rapid‐cycle deliberate practice

TABLE 1.

Demographic differences between randomized groups (N = 210)

	PSD, n = 102 (48.6%)	RCDP, N = 108 (51.4%)	p‐value
Current role in ED
Categorical pediatric residents	34 (33.3)	34 (31.5)	0.86
EM resident	23 (22.5)	21 (19.4)
Other combined resident (Med‐Peds, Ped‐Neuro, etc.)	11 (10.8)	15 (13.9)
RN	34 (33.3)	38 (35.2)
Current year in training (for residents)
PGY‐1	6 (8.8)	4 (5.7)	0.67
PGY‐2	50 (73.5)	50 (71.4)
PGY‐3	11 (16.2)	13 (18.6)
PGY‐4	1 (1.5)	3 (4.3)
Estimate of real codes	5.0 (2.0–10.0)	3.0 (1.75–10.0)	0.28
Is PALS certificate up to date
No	3 (2.9)	7 (6.5)	0.33 a
Yes	99 (97.1)	101 (93.5)
Previous exposure to simulation training
No	5 (4.9)	4 (3.7)	0.74 a
Yes	97 (95.1)	104 (96.3)
Previous leader in simulated code
No	32 (31.4)	33 (30.6)	0.90
Yes	70 (68.6)	75 (69.4)
Teams with CPR coach	PSD teams, n = 21	RCDP teams, n = 20
	14 (66)	13 (65)	0.91

Data are reported as n (%) or median (IQR).

Abbreviations: IQR, interquartile range; PALS, Pediatric Advanced Life Support; PSD, postsimulation debriefing; RCDP, rapid‐cycle deliberate practice.

^a Fisher's exact test was utilized when any cell value was less than 5.

Clinical performance

There was a significant difference in time to first defibrillation: RCDP averaged 100 s (95% CI = 90–111) and PSD averaged 163 s (95% CI = 120–201; Figure 2). There was no significant difference in time to first compression: RCDP averaged 18 s (95% CI = 16–21 s) and PSD averaged 19 s (95% CI = 16–22 s). There was one team in each arm that never gave epinephrine within 10 min. Excluding those, average time to epinephrine showed overlap of 95% CIs: RCDP 251 s (95% CI = 218–284 s) and PSD 321 s (95% CI = 282–361 s).

FIGURE 2.

Time to actions for each arm. Lists time for RCDP and PSD arms for seconds for first compression, first defibrillation, first dose of epinephrine. Statistical significance for time to first shock. PSD, postsimulation debriefing; RCDP, rapid‐cycle deliberate practice

Workload

RCDP had a lower frustration subscore (3.7 vs. 8.9; p < 0.001) and weighted total score (63.7 vs. 69.4; p = 0.02) when compared to PSD. There were no significant differences in the other subscores (Figure 3).

FIGURE 3.

Mean raw subscores and weighted totals for NASA‐TLX for RCDP and PSD groups. Frustration subscore and weighted total showed statistical significance. Error bars show 95% CIs. NASA‐TLX, National Aeronautics and Space Administration‐Task Load Index; PSD, postsimulation debriefing; RCDP, rapid‐cycle deliberate practice

We conducted a post hoc analysis of team size and workload (Table 2). There was no significance when looking at all team members. Examining just the team lead's workload, there was a significant reduction their workload for teams of more than six members compared to teams with five members.

TABLE 2.

Average Workload score by team size

	Small teams (five)	Large teams (six or more)	t‐test p‐value
All team members (N = 210)	57.1 (±12.8)	56.9 (±11.8)	0.90
Team lead only (n = 32)	69.4 (±8.0)	61.6 (±8.5)	0.02

Data are reported as mean (±SD). There were 41 teams total, but only 32 leaders included in study since some team leaders' data were unavailable.

DISCUSSION

After training, RCDP groups defibrillated 1 min faster than PSD groups. While teams in both arms reported high workloads through the NASA‐TLX, ²⁵ RCDP had lower NASA‐TLX workloads as compared to PSD. This reduction in workload along with an improvement in performance is consistent with our hypothesis.

The difference in time to first defibrillation was clinically important. The average time of RCDP was about 1 min faster than those trained with PSD. Time to defibrillation was chosen as our primary outcome since there was a connection shown between time to defibrillation and survival for adults with in‐hospital cardiac arrest. ²⁸ For adults and infants less than 1 year of age, this remains the goal according to American Heart Association's Get with the Guidelines. ²⁹ For children, Hunt et al. ³⁰ investigated if there was a similar association between time to defibrillation and survival and did not find a link. The impact of this research remains unclear. While no association of time to defibrillation and survival was found, most cases of shockable rhythms did receive electricity in less than 2 min. An accompanying commentary argues that rapid defibrillation in children should still be encouraged. ³¹

While the training sessions did not require defibrillation, both PSD and RCDP curricula did teach choosing the correct dosage for defibrillation and charging the defibrillator before the rhythm was analyzed. The only additional steps required during the testing case were recognition of VT, the need for defibrillation, and pushing the button.

Our findings are in line with existing research on RCDP, which has shown improved performance of teams. ¹⁵ , ¹⁷ , ¹⁸ , ³² , ³³ , ³⁴ , ³⁵ This project showed that time to defibrillation was shorter when teams were trained using RCDP compared with PSD. Prior studies on training residents the skill of defibrillation show that simulation can improve this skill and suggestions have been made to integrate defibrillation training within basic life support courses as we have done here. ³⁶ What has not been shown before is that RCDP is superior to PSD to train teams in the complex choreography needed to rapidly perform CPR and prepare for and deliver defibrillation.

For our secondary outcome of a weighted NASA‐TLX score, there is a statistically significant reduction from 69.4 to 63.7. Both of these scores are in the highest percentile of reported scores compared against other studies. ²⁵ , ³⁷ , ³⁸ , ³⁹ While there are studies comparing workload of different clinical or simulated tasks under different conditions, ³⁷ , ³⁸ , ³⁹ , ⁴⁰ , ⁴¹ , ⁴² this is the first study that we know of using workload inventories to compare different curricula.

The major contributor to the difference in overall NASA‐TLX scores was the frustration subscore. This is consistent with the findings from Chancey's qualitative analysis that suggested the RCDP method provides a safe environment to learn material in small chunks. By quickly moving between practice and feedback, RCDP gives learners a chance to make a mistake, learn how to perform a skill correctly, and then practice performing the skill. Learning skills as they are needed and then being given a chance to use those new skills can be viewed as less frustrating. ⁴³ The frustration for PSD likely came from attempting to perform actions that they knew were needed but were unlikely to have performed in clinical practice. Debriefings after simulation covering all issues requires remembering the entire scenario and these little errors during debriefing and never getting a chance to practice perfectly. This dedication of attention to many events has been hypothesized to increase workload. ⁴³ In contrast with dealing with skills as they arise, this method requires time to discuss how errors early in the simulation affected perceptions and physiologic changes later in the simulation.

The workload of the team leaders was reduced in teams that included a CPR coach. While this is a post hoc analysis, this is consistent with a prior study by INSPIRE investigators. ³⁷ Likely the dedication of an extra member of the team to serve as a monitor of compression depth, rate, and recoil takes away some of the tasks that are normally carried by the team leader. This effect is not seen when examining all members of the team.

There was no significant difference in time to first compression. Time to first defibrillation requires more steps and team interaction than initiating compressions and relies on actions of only the first responder without team work. In our testing scenario, time to first compression measured time for the first responder to check for a response, call for help, check for a pulse, and start compressions did not require a complex team‐based choreography. This type of action was unlikely to be affected by the change in simulation structure. Both groups were able to perform this skill well under the recommended 1‐min time limit proposed by Get with the Guidelines. ²⁹ Since both teams did very well on this metric of initiating chest compressions quickly, detecting a significant difference between them is difficult.

For time to first epinephrine there was a suggestive difference in means between the groups, but the CIs of the two groups overlapped. It is not clear why this did not show a similar difference as defibrillation; however, the time to epinephrine depends on many factors which introduces variability into the measurement of team‐based training methods. For example, some of the teams delayed initial defibrillation to focus on drawing up and administering epinephrine. It is difficult to tease apart these variations during data collection.

Ideally, we would like to have the teams taught by different simulation methods return and see if the difference in time to defibrillation was maintained. Instead of having entire teams return, we did have individuals return and measured their performance as team leaders. Separately, data comparing delayed performance of residents as team leaders have been reported. ⁴⁴ Similar results were found on delayed performance by Swinger et al. ³³ This improved leadership at a later simulation strengthens the evidence that given a certain amount of time for simulation training, the repeated practice in RCDP may be a better structure of simulation compared with a single simulation with more time spent on PSD.

Limitations

This study was conducted at a single institution limiting ability to generalize results. Similarly, different levels of experience of residents and nurses may change outcomes. We cannot comment on the superiority of RCDP over PSD for overall resuscitation quality. Also, individuals in the PSD arm only practiced in a single role. They were tested in that same role. This should maximize the performance of that team in testing, but it is unclear whether cross‐training in multiple roles is partially responsible for improvements in team performance. In contrast to other studies, ¹⁵ , ¹⁸ , ⁴⁵ we limited PSD to a single simulation with debriefing. This limits conclusions from our study that RCDP is superior to PSD to cases where only a single scenario takes place. Further research is needed to compare RCDP with PSD when more time and repetitions are possible.

Team makeup and size varied from day to day, but the groups in each arm had a similar distribution of sizes. This variation may have impacts on the effectiveness of the curriculum, but should have similar effects in each arm. While NASA‐TLX is widely used and helpful to examine total workload during a task, it is not designed to separate out intrinsic, extrinsic, and germane loads.

CONCLUSIONS

Teams trained using rapid‐cycle deliberate practice were faster to defibrillate and demonstrated less frustration and workload. Further work may be conducted to separate out the different kinds of workload during education, how changes in frequency or duration of education and integration into a residency curriculum affects learner outcomes, and how clinical behaviors and patient outcomes are affected by training. Next, work on feasibility of integrating rapid‐cycle deliberate practice into required life‐support courses should be done.

For team‐based time‐sensitive simulations like defibrillation as part of pediatric advanced life support, teams trained with rapid‐cycle deliberate practice outperformed those trained with postsimulation debriefing. This correlated with a reduction in workload of teams during training.

CONFLICT OF INTEREST

The authors have no potential conflicts to disclose.

Supporting information

Supplementary Material

Lemke DS, Young AL, Won SK, et al. Rapid‐cycle deliberate practice improves time to defibrillation and reduces workload: A randomized controlled trial of simulation‐based education. AEM Educ Train. 2021;5:e10702. doi: 10.1002/aet2.10702