Workload of learners during simulated paediatric cardiopulmonary resuscitation

Abstract

Introduction

Learner workload during simulated team-based resuscitations is not well understood. In this descriptive study, we measured the workload of learners in different team roles during simulated paediatric cardiopulmonary resuscitation.

Methods

Paediatric emergency nurses and paediatric and emergency medicine residents formed teams of four to eight and randomised into roles to participate in simulation-based, paediatric resuscitation. Participant workload was measured using the NASA Task Load Index, which provides an average workload score (from 0 to 100) across six subscores: mental demand, physical demand, temporal demand, performance, frustration and mental effort. Workload is considered low if less than 40, moderate if between 40 and 60 and high if greater than 60.

Results

There were 210 participants representing 40 simulation teams. 138 residents (66%) and 72 nurses (34%) participated. Team lead reported the highest workload at 65.2±10.0 (p=0.001), while the airway reported the lowest at 53.9±10.8 (p=0.001); team lead had higher scores for all subscores except physical demand. Team lead reported the highest mental demand (p<0.001), while airway reported the lowest. Cardiopulmonary resuscitation coach and first responder reported the highest physical demands (p<0.001), while team lead and nurse recorder reported the lowest (p<0.001).

Conclusions

Workload for learners in paediatric simulated resuscitation teams was moderate to high and varied significantly based on team role. Composition of workload varied significantly by team role. Measuring learner workload during simulated resuscitations allows improved processes and choreography to optimise workload distribution.

Introduction

Cardiopulmonary resuscitation skills are an essential component of every medical trainee’s education. Studies have shown that trainees are deficient in paediatric resuscitation skills and experience,^1–3 which is not unexpected given the rarity of paediatric code events. Simulation-based medical education helps address this issue by allowing learners to practice resuscitation skills in a low-risk environment^{3 4} and has been shown to increase paediatric resident confidence and performance.^3–7

Resuscitation is a complex, team-based effort where individual team members must manage the responsibilities specific to their role while monitoring the emerging needs of the patient and other team members.⁸ To address this, the American Heart Association (AHA) teaches crisis resource management skills designed to improve team dynamics and distribute workload more evenly across the resuscitation team.^{8 9} The concept of workload, which describes the cost incurred to an individual to accomplish a task,¹⁰ provides an opportunity to improve choreography and performance during team-based resuscitation. Compared with more experienced individuals, novices experience higher workloads,^{11 12} and novices may experience differing workload level and distribution during simulated team resuscitations. The National Aeronautics and Space Administration Task Load Index (NASA-TLX) is the most commonly used metric for workload in medical literature.^{10 13 14} This tool was originally designed for use in aviation, but has been validated in team-based settings such as emergency medicine, critical care, surgery and anaesthesia.^13–17 While there is no measure of a redline workload above which performance deteriorates, comparing workloads between different tasks can provide insight into how workload could be balanced more effectively.¹⁴

Few studies to date have specifically measured workload for resuscitation teams, and while work has been reported comparing selected team roles in simulated arrests, no study has compared the individual workload of learners in all team roles (including nurse recorder) in a simulated resuscitation.^18–21 By characterising the workload of each team member during simulated resuscitation, workload can be more equitably distributed.²² Our objective in this study was to measure the workload of learners in all team roles during simulated paediatric cardiopulmonary resuscitations.

Methods

Participants

This descriptive study was conducted from January 2018 through April 2019 at a major urban tertiary children’s hospital. Participants included a convenience sample of paediatric emergency nurses, paediatric residents, residents in combined or specialty paediatric training programmes, and emergency medicine residents. Residents and paediatric emergency nurses who were attending a required simulation training day were given the chance to volunteer and consent to participate in this study. If eligible participants did not consent to participate in the study, they still were able to participate in the training, but their data were not collected. Residents are scheduled to have 1–3 paediatric emergency medicine (PEM) rotations during their residency. If a resident had participated in the study on a prior simulation day, their data would not be analysed on subsequent simulation days, although they still participated in the simulation. All participants on a given day formed a team consisting of one or two nurses and three to six residents, depending on participant availability. Due to scheduling constraints, team size ranged from four to eight. Two PEM faculty members and one nurse educator served as facilitators for each session. There were no confederates.

Setting

The simulated paediatric resuscitation was performed in situ in a paediatric emergency department patient room and used a high-fidelity paediatric mannequin (SimJunior by Laerdal Corporation; Wappinger Falls, New York), patient bed, manual defibrillator (R-series by ZOLL; Chelmsford, Massachusetts or Lifepak 20 by Physio-control; Redmond, Washington), patient monitors, and mock code cart with medications and resuscitation equipment. The room was prepared by a facilitator using a checklist before each session. After a research assistant obtained consent, facilitators led the participants in a scripted orientation.

Tasks and assignment of team roles

Following orientation and prior to the start of the training session, team members were randomly assigned to one of six team roles that were maintained for the duration of both training and testing. Each role’s tasks were as follows: the first responder was called into the patient’s room, assessed the patient and initiated chest compressions; the airway connected the oxygen to the bag-valve mask and delivered rescue breaths; the cardiopulmonary resuscitation (CPR) coach placed a stepstool for the first responder, monitored compressions by the first responder, performed pulse checks, and rotated in as the secondary compressor; the bedside provider(s) placed the backboard and defibrillator pads, obtained access, and administered medications; the nurse recorder provided code documentation and prompts for pulse and rhythm checks; and the team lead identified team roles, directed team actions, calculated medication and defibrillation doses, analysed the defibrillator rhythm, and provided mental modelling. When teams of five were used, the CPR coach role was not assigned and pulse checks were reassigned to the bedside provider. When teams of four were used, both the CPR coach and the bedside provider roles were removed and responsibilities were redistributed to the remaining four roles. When more than six providers were present, extra bedside providers were assigned (up to three total bedside providers). Team members were allowed and encouraged to help each other. Only nurses performed as the nurse recorder role and only physicians performed as the team lead role; all other roles were randomly assigned among all participants using a randomisation schema.

Training and testing procedure

Total session time was 1 hour and 20 min and included a training portion (60 min), followed by a break (10 min), and concluded with a testing portion (10 min). Demographic information was collected using a survey prior to the training portion. During the training, each team learnt how to conduct a paediatric resuscitation using the 2015 AHA’s Pediatric Advanced Life Support (PALS) resuscitation algorithm for cardiac arrest for a child with pulseless electrical activity (PEA).²³ The provided verbal prompt was a previously healthy 6-year-old boy with fever, cough and congestion for 2 days, brought to the emergency room for worsening respiratory distress, who was found unresponsive by the triage nurse and brought to the resuscitation room. The mannequin was pulseless and apnoeic, disconnected from monitors, and without intravenous or intraosseous access. To begin the scenario, the facilitator read the prompt, then called for the first responder to enter the room. Following a 10 s pause to simulate the time required for help to arrive, the rest of the team members were permitted to enter. The mannequin had no change in status until after the second dose of epinephrine, after which return of spontaneous circulation was achieved with sinus rhythm. Learning objectives during the training portion included: team role identification, closed loop communication, action-linked phrases,²⁴ sharing information with respect, coordinated choreography, correct algorithm for PEA and a review of the reversible causes of cardiac arrest. Debriefing and feedback during the training portion were provided using the Promoting Excellence and Reflective Learning in Simulation framework,²⁵ a blended approach that uses direct feedback, plus-delta and advocacy inquiry, depending on the learning objective. The training portion concluded after 60 min, including a brief summary of the learning objectives.

After the training portion, a 10 min break occurred, followed by the 10 min testing portion. The prompt, room and mannequin were identical to the training portion, except that the patient in the testing portion had pulseless ventricular tachycardia. No correction or feedback was provided during the testing portion. The mannequin had no change in status until after the second dose of epinephrine, the first dose of amiodarone, and appropriately delivered defibrillations, after which return of spontaneous circulation was achieved with sinus rhythm. The testing portion would conclude if all actions were correctly performed or after 10 min had elapsed, whichever occurred first. Immediately after the testing portion, the workload of each participant was measured using an electronic version of the NASA-TLX, accessed using a smartphone by scanning a quick response code provided by study personnel. On completion, the results were saved to Research Electronic Data Capture, a secure institutional database.²⁶ To conclude, facilitators reviewed a scripted debriefing on paediatric cardiac arrest.

Workload measurement

The NASA-TLX workload score is the average of six subscores and is reported on a scale of 0–100 (figure 1). The mental demand, physical demand and temporal demand subscores describe the demands that the task places on the participant, while the performance, frustration, and mental effort subscores describe the interaction between the task and the participant. It is generally accepted that workload is low if less than 40, moderate if between 40 and 60, and high if greater than 60.¹⁴ Our primary measurement was the average across the six subscores. Our secondary measurements were the individual subscores.

Figure 1.

The National Aeronautics and Space Administration Task Load Index (NASA-TLX) used to measure workload.³⁷

Statistical analysis

For the descriptive analysis, comparisons were made between team roles and participant demographics. Categorical comparisons were calculated using the Pearson χ² test, or the Fisher exact test if any cell value was less than five. Continuous variables were analysed using the Kruskal-Wallis test for skewed data. For any significant comparisons, a Bonferroni post hoc analysis was conducted to determine pairwise comparisons that were significant. Frequencies and percentages or medians and IQRs were reported. The non-normal distribution (skewed left) of the workload scores was transformed into normally distributed data using the two-step approach.²⁷ To compare workload scores across ED roles, years of training and residency programme, analysis of variance tests were used with scores analysed as dependent variables and ED roles, training level and residency programmes as fixed factors. For any significant result, post hoc analysis was conducted using Dunnett’s C (due to the small sample sizes for the groups). When comparing only two groups, an unpaired t-test provided significance. Means, SD, between-subjects effects p values and partial eta2 values were reported. A p value of <0.05 was defined as statistical significance. All analyses were conducted using the Statistical Package for the Social Sciences, V.25 (IBM Corp., Armonk, New York, USA).

Results

There were 210 individuals eligible for the study who had been scheduled for simulation training during the study time frame. Of these 210 individuals, 20 were residents who initially participated in the study, then returned for one subsequent PEM rotation during the study time frame; these 20 residents’ initial data were included and they still participated in subsequent training days, but their return data were not included. In total, 210 participants’ initial data were included in the study, representing 40 simulation teams, with one team per simulation training day. Sixty-six per cent were residents, of which the majority were categorical paediatric residents and emergency medicine residents. The majority of residents were in their second year of postgraduate training. Over 95% of all participants were PALS certified and had prior simulation experience, and more residents than nurses had prior simulation team-lead experience (table 1). There were no significant differences in workload among training years or training programmes (online supplemental tables 1–2). However, when comparing the workload of nurses to residents, nurses had a higher mental demand subscore than residents (69.5 vs 61, respectively; p value=0.01), while residents had a higher physical demand subscore than nurses (63.5 vs 36, respectively; p value=0.004; online supplemental table 3).

Table 1.

Participant demographics by team role (n=210)

	Airway n=33 (15.7%) N (%) median (IQR)	Bedside provider n=41 (19.5%) N (%) median (IQR)	CPR coach n=28 (13.3%) N (%) median (IQR)	First responder n=37 (17.6%) N (%) median (IQR)	Nurse recorder n=39 (18.6%) N (%) median (IQR)	Team lead n=32 (15.2%) N (%) median (IQR)
Current ED role
Paediatric resident	17 (51.5)	14 (34.1)	9 (32.1)	11 (29.7)	0 (0.0)	17 (53.1)
EM resident	7 (21.2)	6 (14.6)	9 (32.1)	14 (37.8)	0 (0.0)	8 (25.0)
Other resident	7 (21.2)	4 (9.8)	2 (7.1)	6 (16.2)	0 (0.0)	7 (21.9)
RN	2 (6.1)	17 (41.5)	8 (28.6)	6 (16.2)	39 (100.0)	0 (0.0)
Year of training
PGY-1	3 (9.7)	1 (4.2)	2 (9.5)	2 (6.5)	–	2 (6.3)
PGY-2	20 (64.5)	13 (54.2)	15 (71.4)	27 (87.1)	–	26 (81.3)
PGY-3	6 (19.4)	9 (37.5)	4 (19.0)	2 (6.5)	–	3 (9.4)
PGY-4	2 (6.5)	1 (4.2)	0 (0.0)	0 (0.0)	–	1 (3.1)
Estimated # of prior codes	2 (1 to 10)	4 (1.25 to 5)	6.5 (2.25 to 10)	3 (1 to 15)	–	4 (2 to 10)
PALS certified
No	3 (9.1)	0 (0.0)	3 (10.7)	2 (5.4)	0 (0.0)	2 (6.3)
Yes	30 (90.9)	41 (100.0)	25 (89.3)	35 (94.6)	39 (100.0)	30 (93.8)
Prior simulation experience
No	0 (0.0)	0 (0.0)	3 (10.7)	2 (5.4)	4 (10.3)	0 (0.0)
Yes	33 (100.0)	41 (100.0)	25 (89.3)	35 (94.6)	35 (89.7)	32 (100.0)
Prior simulation team-lead experience
No	3 (9.1)	13 (31.7)	6 (21.4)	9 (24.3)	31 (79.5)	3 (9.4)
Yes	30 (90.9)	28 (68.3)	22 (78.6)	28 (75.7)	8 (20.5)	29 (90.6)

ED, Emergency Department; EM, Emergency Medicine; PALS, Pediatric Advanced Life Support; PGY, Post-Graduate Year; RN, Registered Nurse.

The average workload across all roles was 57.2±4.3. Table 2 and figure 2 show detailed comparisons of each role’s workload. When comparing team roles to the workload subscores and average workload, all between-subjects effect p values were significant (table 2). Notably, in the post hoc analysis, team lead was significantly different compared with other ED roles (table 2). The team lead had higher scores for all categories, except for physical demand, which had a lower score (table 2). There was a strong relationship between mental and physical scores and ED roles (η²=0.31 and η²=0.33, respectively) (table 2). The remaining relationships had a medium effect (table 2).

Table 2.

Workload by team role (n=210)

	Airway n=33 (15.7%) mean (±SD)	Bedside provider n=41 (19.5%) mean (±SD)	CPR coach n=28 (13.3%) mean (±SD)	First responder n=37 (17.6%) mean (±SD)	Nurse recorder n=39 (18.6%) mean (±SD)	Team lead n=32 (15.2%) mean (±SD)	Between-subjects effects P value*	Partial Eta2
Mental†	43.1 (23.4)	54.0 (26.8)	53.3 (24.0)	54.3 (21.1)	72.3 (19.1)	88.4 (15.8)	<0.001	0.31
Physical‡	64.4 (24.8)	50.2 (28.9)	66.3 (23.5)	66.1 (22.4)	33.4 (25.4)	22.0 (19.4)	<0.001	0.33
Temporal§	62.5 (25.3)	63.6 (19.7)	60.2 (20.1)	68.5 (17.1)	70.3 (20.0)	80.0 (17.3)	0.001	0.09
Performance¶	42.2 (26.7)	42.1 (20.2)	42.9 (25.5)	39.0 (20.9)	45.1 (25.4)	58.6 (23.1)	0.02	0.07
Effort**	67.2 (17.5)	64.2 (19.7)	70.5 (20.8)	67.5 (15.8)	64.4 (20.0)	83.1 (17.7)	<0.001	0.11
Frustration††	44.1 (26.1)	52.5 (24.7)	51.3 (24.8)	47.6 (23.4)	47.4 (25.6)	63.1 (24.1)	0.04	0.06
Average‡‡	53.9 (10.8)	54.1 (13.2)	57.2 (13.5)	56.8 (10.0)	55.7 (11.3)	65.2 (10.0)	0.001	0.10

*P value calculated using analysis of variance. Post hoc analysis used Dunnett’s C for pairwise comparisons (p<0.05).

†Nurse recorder and team lead had significantly higher post workload scores compared with all other roles.

‡Team lead had significantly lower scores when compared with airway, bedside provider, CPR coach and first responder. Nurse recorder had significantly lower scores compared with airway, CPR coach and first responders.

§Team lead had significantly higher scores than airway, bedside provider and CPR coach.

¶Team lead had significantly higher scores than bedside provider and first responder.

**Team lead had significantly higher scores compared with all other roles with the exception of CPR coach.

††Team lead had significantly higher scores compared with airway.

‡‡Team lead had significantly higher scores compared with all other roles with the exception of CPR coach.

Figure 2.

Workload by team role (N=210).

Discussion

Workload for learners in simulated paediatric resuscitation teams was moderate to high and varied significantly based on team role. In addition, the composition of workload differed depending on the team role. This study addresses a gap in educational research by providing a profile of learner workload in all team roles during simulated paediatric resuscitation. There are several studies examining the workload of learners in simulated team resuscitations but none to date have measured the workload of learners performing each team role in a full resuscitation team, including the nurse recorder.^18–21 It is important to understand the workload of novices because the rapidly changing and multitasking nature of simulated resuscitation places them at high risk of experiencing task overload. This is supported by a study by Parsons that assessed workload of team members after paediatric trauma resuscitations and showed that bedside clinicians, typically junior residents, experienced much higher levels of workload than ED attending physicians,²⁰ as well as by a 2017 Tofil simulated sepsis scenario study which demonstrated that residents had significantly higher workload scores as compared with attendings/fellows.¹⁹ Generally, when workload is unacceptably high, performance degrades, prompting system modifications so as to not overburden the operator and to restore good performance.¹⁴ A simulation study by Geis aimed at preparing a new emergency department for its opening demonstrated how individual team members’ high levels of workload and frustration could result in modifying systems to reduce team member workload.²⁸ Despite these encouraging studies, there is still much work to be done in understanding novice workload in resuscitation teams.

Our findings correlate well with recent studies by Tofil and Brown that measured the workload of team leaders and CPR providers during team-based simulation.^{18 19} The 2017 Tofil study compared team leaders against team members during simulation of sepsis scenarios, while the Brown article compared team leaders against CPR providers.¹⁹ Despite differences in simulation design and team composition to our study, these studies also found that workload in simulation was moderate to high and that workload differed based on team role. A 2020 study by Tofil examines a simulated resuscitation team of four roles, including team lead, CPR provider, CPR coach and airway, as well as using a scripted actor.²¹ Our study correlates with the 2020 Tofil study and offers additional insight into this area, as we used learners in each team role, and included a nurse recorder role to more closely resemble the interdisciplinary ED resuscitation team. Our work highlights an additional opportunity for further improvement in team workload distribution across all team members. Understanding the amount and composition of each learner’s workload in their team role helps educators address the unique demands faced by each role.^{12 23 29} It may also inform educational simulation design as an increasing emphasis is being placed on crisis resource management skills,⁸ which include optimising a team’s workload distribution.

There are multiple reasons for the team lead role to experience the highest average workload and the highest mental and temporal demands: the team lead delegates and supervises the time-sensitive tasks of the other roles, performs weight-based calculations, monitors the patient’s status and emerging needs, and provides mental modelling. The nurse recorder experienced the second highest subscores for both mental and temporal demand; this is not surprising as the recorder must be cognizant of all the events occurring during a resuscitation for their record. The high levels of mental and temporal demand contributed to the higher average workload of both roles. Using cognitive aids, such as checklists and action-linked phrases,24 28 30 31 can help mitigate this increased mental demand. Instructors can teach trainees how to manage high levels of workload by applying crisis resource management principles, including communication, leadership, teamwork, resource allocation and situational awareness.^{8 9} It also may be possible to offload both mental and temporal demand from the team lead and nurse recorder by redistributing workload to team members that are experiencing less mental and temporal demands.^{18 19 32} The two roles with the lowest mental and temporal demands, CPR coach and airway, may have experienced the least mental demand due to the focused nature of their roles. The CPR coach could be used to direct compressor rotation every 2 min using a timer and coach the rate of the compressor using a metronome, allowing the team lead to focus on other aspects of the resuscitation. The airway role, with its position at the head of the bed directly opposite the team lead and nurse recorder, could be enlisted as a sounding board for mental modelling and reviewing reversible causes of arrest. The 2020 Tofil study demonstrated that when a CPR coach was present, the CPR provider experienced higher physical workload and lower mental workload, without change to the team lead’s workload.²¹ They hypothesised that although team lead workload did not change with the presence of a CPR coach, the team lead may have been able to redirect cognitive load that may have been managing CPR quality instead towards higher level decision making. By encouraging an additional team role, the airway, in addition to the CPR coach, to cognitively unburden the team lead, it may create opportunities for the team lead to focus on decision-making and mental modelling for the entire team.

The two roles performing CPR (primary responder and CPR coach) reported the two highest physical demand subscores, reinforcing the emphasis that PALS training places on frequent compressor rotation to maintain CPR quality.³³ The airway role reported the third highest physical demand subscore. These findings are consistent with the 2020 Tofil study.²¹ The physical demands of bag-mask ventilation are not often appreciated, possibly because it is not as obviously physically taxing as performing chest compressions.³⁴ Bag-valve mask ventilation is a challenging resuscitation skill, and even more so during arrest situations. Sources of physical demand include fatigue of the intrinsic muscles of the hand, manipulating the obstructed airway, and delivering appropriate tidal volumes at a consistent rate during ongoing compressions.³⁵ To alleviate this role-specific physical demand, educators should highlight these sources of fatigue and practice troubleshooting manoeuvres during airway simulation training, particularly for trainees with smaller hands who may find traditional methods of face mask application difficult.³⁵

Our study had several limitations. This was a descriptive study; thus, results should be considered hypothesis generating. It was performed at a single institution, with a single choreography, limiting the generalisability of its results. The chosen team roles and their responsibilities are a reflection of our emergency department’s consensus (such has having the CPR coach act as a secondary compressor); other institutions using different team compositions may have different results. Due to limited nursing availability, residents could be randomly assigned to the role of ‘bedside provider’, which included medication preparation, a role that is typically assigned to a nurse or pharmacist in actual resuscitations. However, the bedside provider’s workload may have been mitigated by our specific resuscitation choreography, as the team lead was responsible for calculating weight-based emergency medication dosages that were then relayed to the bedside provider to prepare. Team size could be four to eight members based on participant availability, adding variability. For teams with less than 6 members, there was no CPR coach, which occurred in 12 of the 40 teams. Additionally, due to the presence of 20 return participants, we could not meaningfully comment on the effect of team size on workload. Because team members were permitted to assist one another, responsibilities could be offloaded to other team members, possibly affecting the measured workload for each team role. Lastly, while the NASA-TLX is widely used to measure workload, there are no defined red-line limits for workload,¹⁴ or points at which performance degrades significantly. While some studies show that ‘very high’ workload can be detrimental to performance, it is unknown at what level this attrition occurs, whether ‘very low’ workload has the same effect, and whether these effects hold true for different levels of learner or for learners operating in teams.¹¹ This is not surprising given the wide variation with which these studies have been performed.^{13 15} Despite these limitations, serial measurements of workload may offer a cost effective and more standardised approach for educators to observe the effects of different simulation interventions on learner workload and adjust their teaching accordingly.

Our study found that the workload of learners was moderate to high across roles in team-based paediatric simulated arrests and that workload and its composition varied by individual team role. By measuring the learner’s workload in all team roles during simulated resuscitations, this study provides opportunities for educators to adjust their simulation teaching to address the differing demands that each role faces and to optimise workload across the resuscitation team. Future studies should examine how restructuring team roles changes the distribution of workload,^{32 36} the effect of different simulation designs on learner and debriefer workload,²² and whether there is an ideal workload for learners that maximises performance andskillsretention.¹¹

What is already known on this subject.

• Learners participating in team-based simulation are at risk of task overload.

• There is little research on the workload of learners in all team roles during simulated paediatric cardiopulmonary resuscitation.

What this study adds.

• During team-based simulation, learners experience high levels of workload and workload composition varies by team role.

• Understanding learner workload during simulation helps educators tailor their teaching to optimise workload across the resuscitation team.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics approval

This descriptive study was approved by the Institutional Review Board and conducted from January 2018 through April 2019 at a major urban tertiary children’s hospital.