More than experience: a post-task reflection intervention among team members enhances performance in student teams confronted with a simulated resuscitation task—a prospective randomised trial

Abstract

Background

Teams that regularly step back from action and deliberately reflect on their performance and strategies show higher performance. Ad hoc emergency teams with changing team composition cannot develop such habits but may engage in short postaction reflection to discuss shortcomings of past performance and potential adaptations of their strategies for future similar tasks. This study aimed to test the effect of a short postaction self-led reflective team briefing on resuscitation performance in a simulator setting in terms of three performance parameters: hands-on time, coordination between chest compression and ventilation, and defibrillation.

Methods

We performed a randomised controlled trial including 56 ad hoc formed teams of three fourth-year medical students each. All groups performed a resuscitation task, followed by a self-guided reflective briefing, based on a general instruction (n=28 teams), or an unrelated discussion session (control condition; n=29), followed by a second resuscitation task in the same team composition.

Results

Adjusted for performance in the first task, teams in the reflection condition showed higher performance gain in the second resuscitation than teams in the control condition (6.21 percentage points (95% CI 1.31 to 11.10, p<0.001)) for basic hands-on performance; 15.0 percentage points (95% CI 2 to 28, p<0.001) for coordinative performance but non-significantly lower performance for defibrillation (−9%, 95% CI −27% to –9%, p=0.312).

Conclusion

Even very short self-led postaction reflective briefings enhance basic resuscitation performance in ad hoc groups but may not influence more complex aspects of the task. We recommend including short self-led team debriefings as part of simulator training.

Introduction

Many medical emergencies are dealt with by very closely cooperating teams. Performance of such teams depends, among other aspects, on the quality of team collaboration.¹ It is thus imperative to find ways of improving team collaboration in medical emergencies. Team training is an effective means to influence teamwork; it improves outcomes such as medical performance, patient outcomes and patient safety.^{2 3}

An important way to support team learning involves team members stepping back from the action, engaging in deliberate reflection about their past performance and their task-related and collaboration-related strategies and discussing strategies for future tasks.^{4 5} Team training providing an opportunity to engage in such reflective phases has been found to be particularly effective.^{6 7}

However, team learning does not only take place during formal training. Previous studies found that teams that develop habitual routines to regularly engage in reflection show higher performance and more innovation.^{5 7 8} As developing routines presupposes membership continuity over time, these findings mostly apply to teams with a history of collaboration⁹; such temporal and compositional team stability is rare in medical settings.^{10 11} Particularly in emergency situations, teams often are formed ad hoc and collaborate for only a short period of time and then disband,¹² making it unlikely to develop routines of reflection. Nevertheless, ad hoc teams can engage in short reflection before, during or after collaboration to enhance performance. Studies in other fields,¹³ but also in medicine,^14–18 showed mostly positive effects of short reflection phases for ad hoc teams.

In everyday practice, medical teams are encouraged to engage in reflective discussions—often called briefings.¹⁹ A recent overview on reflective briefings in medicine²⁰ distinguishes briefings by their temporal position with regard to the task. In preaction briefings, teams reflect and discuss before collaboration and reflect about the upcoming tasks and the optimal preparation of the team (eg, team timeouts in surgery²¹; huddles before starting a complex task²²). In-action reflection takes place during an ongoing task, when the team halts all but life-saving activities and engages in a briefing with the goal of optimising present care or adaptation to changing task conditions. In postaction briefings, teams reflect after the task (eg, mortality and morbidity conferences²³) and analyse past performance with a focus on future, similar tasks.

For emergency situations, which are the focus of the current study, preaction briefings are possible only under very specific circumstances (eg, while waiting for a patient to arrive²⁴). However, most emergencies require immediate action, and even short delays may harm the patient.²⁵ In-action briefings may also be very difficult to carry out, particularly if the patient needs constant attention and team members cannot interrupt the action to focus their attention on a briefing. Thus, postaction briefings seem most promising for team learning of ad hoc emergency teams. Previous research shows that reflective postaction briefings have greater impact if they are led by an expert^{6 26} or by a team leader,²⁷ and they yield better results if performance feedback is provided.¹⁷ As objective feedback or an expert adviser are often not available after an emergency, the question arises if postaction briefings relying on team self-guidance can be valuable. There is little empirical evidence that self-led postaction briefings improve performance.

The main goal of the current study is thus to test the effects on performance of a short self-led postaction briefing intervention in ad hoc teams of first responders performing a cardiopulmonary resuscitation in a simulator setting.

Primary endpoint is resuscitation performance when treating a patient suffering a cardiac arrest by teams that engaged in a short postaction briefing after a previous, similar task, as compared with teams that did not. As performance is multidimensional,^28–31 it is important to distinguish several components of performance. Based on the resuscitation algorithm,³² we assessed basic performance (measured as hands-on or flow time), coordinative performance (coordination between chest compression and ventilation) and defibrillation performance.

Methods

Participants and setting

This prospective observational study was conducted in the simulator room of an intensive care unit (ICU) at a midsize European University Hospital. All participants provided signed informed consent. Participants were fourth-year medical students, which are a good representation of medical professionals with a theoretical knowledge base (they all had theoretical knowledge of symptoms of cardiac arrests and cardiopulmonary resuscitation [CPR]) but limited hands-on practice and limited collaboration practice for medical tasks. This work was supported by the Swiss National Science Foundation, grant #149734.

Participants were assigned to teams of three according to their scheduling preferences. Because all medical students were part of the same cohort, members working as a team may have known each other, but they did not have common working experience. The students were blind to the goal of the study.

The patient simulator used consists of a high-fidelity manikin that was controlled remotely by the operator (Meti, Medical Educational Technologies, Inc). It has realistic physiological features, including the ability to talk (through a connection to the control room). The simulator room was equipped as a standard ICU patient room, with one wall as a one-way mirror that allows the observation from the control room.

Study design

This is a prospective, randomised controlled trial. The teams were first confronted with a patient suffering a cardiac arrest (task 1; see figure 1). After this, they were randomly assigned to either a discussion task unrelated to resuscitation (control group) or to a self-guided reflective postaction briefing. All teams then performed a second resuscitation task in the same team composition (task 2). We excluded one team from analysis because they misdiagnosed the patient’s condition in the second task and were joined by a confederate to successfully terminate the task. Of the 56 remaining groups, 29 were in the control condition and 27 in the briefing condition.

Figure 1.

Study design.

Scenarios and self-led briefing intervention

On arrival, the students were shown the simulator room and the manikin. They were then asked to wait in the hallway as a team. A confederate nurse stayed in the room throughout both scenarios but intervened only when asked.

Task 1: cardiac arrest

The confederate nurse met the group in the hallway and instructed them to intervene with an emergency in the simulator room. A manual defibrillator was available in the room. The nurse explained that the patient did not feel well after a diagnostic procedure and that he was asked to lie down. On entry of the group, the patient was lying on the bed and was unresponsive, not breathing and with no palpable pulse. A monitor displaying the patient’s cardiac rhythm showed ventricular tachycardia. The team had to diagnose the cardiac arrest and initiate resuscitation. The patient did not regain spontaneous circulation regardless of the actions of the team. Three minutes into the resuscitation, the team was interrupted by a confederate.

Briefing intervention

After the interruption, the teams assigned to the self-led briefing condition were handed a sheet with instructions on how to reflect. The instructions were adapted from other studies16 17 20 33 and contained a suggestion to discuss and look back at the situation just experienced, analysing: (A) what went well, (B) what did not go well and why and (C) discussing strategies on how to proceed, should the team encounter a similar situation again. Teams did not have access to electronic or other sources of information during the briefing. After 3 min of discussion, the team was interrupted by a confederate and asked to proceed to the next patient.

Control group

The teams assigned to the control group where ask to huddle around an X-ray display board displaying a thorax and were given basic patient information about another patient. The team was asked to diagnose the patient’s condition (a pneumothorax) and provide treatment recommendations. After 3 min of discussion, the team was interrupted by a confederate and asked to proceed to the next patient. The task of the control group was chosen to be similar in all aspects (talking about a medical problem as a group) with the exception of the content of the discussion. Thus, it can be ruled out that effects of the intervention are actually due to the fact that intervention groups simply spent more time together, independent of the specific content of these discussions.

Task 2: cardiac arrest

After the discussion sessions, the teams were immediately introduced to the next patient. The confederate handed over patient information and told the team that the patient was not feeling well after an ambulatory stress ECG. The team was asked to interview the responsive patient with the aim of providing a recommendation on whether the request of the patient to leave the hospital could be granted. During the interview, the patient complained of pain in the legs (from cycling) and provided standard answers to medical and demographic questions. Two minutes into the interview, the patient said that he felt dizzy, then became unresponsive, closed his eyes, stopped breathing and had no palpable pulse. The monitor showed ventricular tachycardia, and the team had to recognise the situation as a cardiac arrest and start CPR.

Measures

Three performance measures were derived from the in-hospital resuscitation algorithm³² and adapted to the simulation scenario:

Percent of hands-on time (hands-on performance) measures basic CPR efficiency as percentage of time the team provided uninterrupted chest compressions, ventilation or defibrillation on the pulseless manikin. Five seconds before and after defibrillation were counted as hands-on time to account for the shift between chest compressions and defibrillation and to allow for controlling of defibrillation effects.

Coordination between chest compressions and ventilation (coordination performance) was assessed by two independent observers based on video recordings and consisted of two subscores. Synchronisation between chest compressions and ventilation was rated on a four-point scale (0=chest compressions and ventilation overlapping during the whole procedure; 1=chest compressions and ventilation overlap more often than alternate;  2=mostly alternating, some overlap;  3=chest compression and ventilations never overlap). Correct application of the 30 (chest compressions) to 2 (ventilations) rule was coded as 0 (if not observed) and 1 (observed). The two measures were combined with equal weights into one indicator and transformed so that optimal coordination performance yielded a score of 1.

Defibrillation performance was assessed by two independent observers, based on the recordings, and consisted of two subscores: Compression/ventilation cycles were coded as incorrect (0) or correct (1) depending on whether they conformed to the number of chest compressions/ventilation cycles between two defibrillations (five according to the resuscitation algorithm in 2010). Minimising the pause between CPR and defibrillation was based on the time of interruption between CPR and defibrillation and coded as 0 (if more than 10 s), 1 (between 6 s and 10 s) or 2 (between 0 s and 5 s).³² The two measures were combined into one indicator with equal weights and adjusted to measure defibrillation performance between 0 (minimal) and 1 (maximal performance). Defibrillation performance was assessed for task 2. For task 1, it was assessed whether (1) or not (0) the team attempted to defibrillate.

Objectives of the study

Primary goal was to assess the effect of the reflective briefing intervention on three aspects of resuscitation performance: hands-on time, coordination between chest compression and ventilation and defibrillation performance.

Statistics

We used analysis of covariance to assess differences in performance between conditions in the second task. To adjust for the effects of having practised this task, we included the respective performance score of task 1 as covariate for hands-on and coordination performance. Because only half of the groups attempted defibrillation in task 1, we included defibrillation attempt, coded as yes (1) or no (0), as covariate when evaluating defibrillation performance in task 2.

Performance measured as percentages (hands-on performance) was arcsine transformed; statistical values are reported based on the transformed variables but descriptive results as percentages and proportions. Analyses were performed with IBM SPSS Statistics for Windows, V.24. P<0.05 was considered statistically significant.

Results

Demographic characteristics and gender composition of the teams are displayed in table 1. Six students did not provide data on age. The unbalanced gender team composition reflects the current gender distribution of medical students.

Table 1.

Participant demographics and team gender composition

Participants (n=168 participants)
Age	Mean (SD)	24.44	2.36
Females	n (%)	117	69.64
Males	n (%)	51	30.36
Year of study	Mean (SD)	4	0

Team composition (n=56 groups)
All female	n (%)	17	30.36
Two females, one male	n (%)	28	50.00
One female, two males	n (%)	10	17.86
All male	n (%)	1	1.79

Table 2 summarises the results for all performance measures; figure 2 illustrates the results.

Table 2.

Effect of the briefing intervention on hands-on resuscitation performance, coordinative performance and defibrillator performance

	Unadjusted results								Adjusted results*							Statistical parameters
	Before intervention				After intervention				After intervention				Difference in performance (intervention – control)			F	df	P value
	Task 1 control		Task 1 briefing		Task 2 control		Task 2 briefing		Task 2 control		Task 2 briefing
	M	SD	M	SD	M	SD	M	SD	M	SE	M	SE	M	Lower 95% CI	Upper 95% CI
Hands-on performance	60.04	16.90	56.36	14.82	68.63	13.68	73.16	8.63	67.82	1.69	74.03	1.75	6.21	1.31	11.10
Experience																32.9 1†	1	<0.001
Intervention																6.09†	1	0.017
Coordination performance	0.64	0.35	0.69	0.31	0.65	0.34	0.83	0.26	0.66	0.04	0.81	0.05	0.15	0.02	0.28
Experience																35.77	1	<0.0.001
Intervention																5.51	1	0.023
Defibrillation performance	65.50%‡		74.10%‡		0.50	0.31	0.47	0.30	0.50	0.06	0.47	0.06	−0.09	−0.27	0.09
Experience																1.04	1	0.312
Intervention																0.19	1	0.663

Note: n=56 teams.

*Adjusted for performance in task 1 (experience).

†Statistics are based on arcsine transformation of performance scores.

‡Experience for defibrillation performance represents whether the team defibrillated or not within task 1.

M, Mean.

Figure 2.

Illustration of performance gains by briefing for all three performance measures. To allow comparability, hands-on performance is expressed in proportions. Data are displayed as means and 95% CIs of performance gains/losses for teams in the briefing condition as compared with the control condition. Hands-on and coordinative performance gains were significant; defibrillation performance gains were not significant.

Hands-on performance in the first task was a significant and independent predictor of hands-on performance in the second task. Teams in the briefing condition increased their performance by 16.8 percentage points between task 1 and task 2, while teams in the control condition by 8.6. After adjusting for performance (experience) in the first task, hands-on performance gain was 6.21 percentage points larger for teams in the briefing condition as compared with teams in the control condition (table 2).

Coordination performance in the first task was also a significant and independent predictor of coordination performance in the second task. Teams in the briefing condition increased their coordination performance significantly more than teams in the control condition: after adjusting for coordination performance in the first task, performance gain for teams in the briefing condition was 0.15 points (or 15%) larger than for teams in the control condition (table 2).

Teams in the briefing condition showed a slight, but non-significant, lower defibrillation performance in the second task than teams in the control condition (−0.09 points or −9%). Experience with defibrillation in the first task was not a significant predictor for defibrillation performance in the second task.

Discussion

A 3 min self-led postbriefing after a resuscitation task increased performance in a subsequent, similar task in ad hoc teams of fourth-year medical students as compared with teams in a control group discussing other issues. Results are adjusted for performance in the first task, which therefore cannot explain the results.

The briefing enhanced basic hands-on and coordination performance but not defibrillation performance. Hands-on performance represents basic aspects of resuscitation and is particularly affected if teams have problems initiating the resuscitation or if interruptions occur.³⁴ Coordination performance also represents basic coordinative aspects of the task, as it measures the correct number of chest compressions and ventilation and their appropriate alternation. We therefore conclude that the briefing positively influenced basic aspects of task performance.

Although experienced task difficulty depends to some degree on the individual, defibrillation performance can be regarded as representing performance of more complex³⁵ and more advanced aspects of the task; it was not improved by the briefing. Two reasons may explain this result. First, the briefing instructions specifically asked the teams to reflect on their previous experience. As about half of the teams did not attempt defibrillation in task 1, defibrillation may not have been discussed in these teams during the briefing if they followed the instruction to reflect on their past experience. Second, other studies did not find effects of reflective briefings on performance if briefings were not combined with additional information or feedback,^{17 36} not led by an expert,³³ or if teams were not trained in reflecting.¹⁹ Other authors also emphasised that the quality of (self-led) debriefings is often low.³⁷ Such aspects are likely to increase in importance as the complexity of tasks increases. It could thus be that the quality of the briefings was sufficient to increase basic aspects of performance but not more complex ones. Particularly, the latter aspect can be addressed in the expert-led debriefing after the simulator session.³⁸

Self-led briefings without an expert may not only have positive effects, as there is a risk that erroneous information may be discussed and negatively influence subsequent performance. (We thank one of the reviewers for emphasising this aspect.) Unfortunately, we have no information about the content of the reflective briefing and therefore cannot assess the accuracy of the information discussed. As an approximate test to identify potential negative effects of briefings, we compared the proportion of groups with decreasing performance in the second task as compared with the first task across conditions. Hands-on as well as coordinative performance was both worse in 2/27 (7.4%) of teams in the briefing condition but in 7/29 (24.1%; hands-on) and 6/29 (20.7%; coordinative) of teams in the control condition; these differences did not reach conventional statistical significance levels. These results do not rule out negative effects of briefings, but they do indicate that negative effects may be the exception and may be offset by the positive effects of the briefing as compared with no briefing. Note that for defibrillation performance, such an analysis could not be done, as many teams did not defibrillate in the first tasks.

This study has limitations. First, it is an experimental study based on a cohort of fourth-year medical students. Although they represent relatively unexperienced medical professionals quite well, real ad hoc resuscitation teams are likely to be interdisciplinary and composed of professionals with different levels of experience; this limits the generalisability of the results to medical practice. In addition, because participants are from the same cohort, we cannot control for the effects of prior training experiences they may have together. Another limitation is the relatively small sample size. Furthermore, we did not have information on the content of the briefing discussions; therefore, we could not assess the accuracy and level of expertise of the briefing discussion, nor to what degree the teams in the briefing condition may have addressed specific performance shortcomings in task 1. Further studies should investigate the content of such briefings.

Conclusions

Learning how to reflect on one’s own teamwork and task performance is an important skill³⁹; in the optimal case, this skill is itself part of a training. This study shows that in simulation training, teams are capable to engage in self-led debriefings and profit from them, although in a limited way, that is, for basic aspects mainly. Evidently, the self-led debriefing is part of the simulation experience and should be discussed in the postsimulation expert-led briefing^{33 38} to assure that all relevant aspects were discussed accurately and to discuss the process and potential difficulties of a self-led briefing.

For everyday practice, the results underscore the utility of even very short self-led post-task briefings. The mean length of post-action debriefings in trainings is 17 minutes, according to a meta-analysis.⁴⁰ Although they may be more effective than shorter ones, such long briefings constitute major interruptions of ongoing work and therefore are not very likely to be implemented into everyday practice. Our study showed that engaging in a 3 min briefing increased performance on a subsequent similar task by 6 percentage points for basic hands-on performance and by 15 percentage points for coordinative performance. Such a cost-effective and feasible intervention may thus be worthwhile to introduce. The debriefing instructions to look back at the task, analyse what went well and what did not, and to develop plans for a similar task could well be used in everyday practice. To avoid negative effects of inaccurate knowledge, teams should be encouraged to look up information or seek expert advice if they detect knowledge gaps or insecurities during the briefings.