Evolution of clinical event debriefs in a quaternary pediatric emergency department after implementation of a debriefing tool

Abstract

Background

Debriefing clinical events in the emergency department (ED) can enhance team performance and provide mutual support. However, ED debriefing remains infrequent and nonstandardized. A clinical tool (DISCERN—Debriefing In Situ Conversation after Emergent Resuscitation Now) was developed to facilitate ED debriefing. To date, there are no studies providing qualitative analysis of clinical event debriefs done using such a tool. Our goal was to explore common themes elicited by debriefing following implementation of DISCERN.

Methods

This was a retrospective mixed‐methods study analyzing DISCERN data from 2012 through 2017 in a pediatric ED. Quantitative data were analyzed using descriptive statistics. With constant comparison analysis, themes were categorized when applicable within the context of crisis resource management (CRM) principles, previously used as a framework for description of nontechnical skills. Member checking was performed to ensure trustworthiness.

Results

We reviewed 400 DISCERN forms. Overall, 170 (41.6%) of target clinical events were debriefed during the study period. The number of clinical events debriefed per year decreased significantly over the study period, from 118 debriefed events in 2013 to 20 debriefed events in 2017 (p < 0.001). Events were more likely to be debriefed if cardiopulmonary resuscitation was needed (odds ratio [OR] = 11.8, 95% confidence interval [CI] = 4.1–33.8]) or if the patient expired (OR = 8.9, 95% CI = 2.7–29.1]). CRM principles accounted for 81% of debriefing statements, focusing on teamwork, communication, and preparation, and these themes remained consistent throughout the study period.

Conclusions

Use of the DISCERN tool declined over the study period. The DISCERN tool was utilized more commonly after the highest‐acuity events. Clinical event debriefs aligned with CRM principles, with medical knowledge discussed less frequently, and the content of debriefs remained stable over time.

INTRODUCTION

There is growing agreement that debriefing high‐stakes clinical events in the emergency department (ED) setting is useful for multiple purposes, ¹ , ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ , ⁸ including enhancement of clinical performance, optimization of teamwork, and relief of team member emotional distress. ¹ , ⁹ Despite these benefits, few standardized debriefing protocols exist, which may contribute to less frequent and less effective debriefs. ² , ⁷ , ¹⁰ , ¹¹ Further, few clinicians receive formal training on how to debrief real clinical events. ¹ , ⁴ , ¹² Prior studies surveying ED providers show that many would like additional training on how to effectively debrief clinical events. ⁴ , ⁹

A previous study performed at our institution detailed the implementation of a standardized debriefing tool in 2011—the Debriefing In Situ Conversation after Emergent Resuscitation Now (DISCERN) tool. ¹³ This tool was the result of a formal quality improvement project in collaboration with ED physician and nursing leadership. The debriefing form acts as a guide for ED staff leading or participating in clinical event debriefs and gathers both quantitative and qualitative information about debriefs that have occurred. The centerpiece of the tool is a plus‐delta debriefing template asking participants what went well during the team's care for the patient and what could go better in the future. Forms are attached to the code/trauma flow sheets that are filled out by the nurse documenter of each event, prompting a debrief to occur when a code or significant trauma event transpires. Data regarding the form's first year of use were published previously, and that initial study reported on the debriefing of resuscitation events defined as provision of cardiopulmonary resuscitation (CPR), defibrillation, endotracheal intubation, or high‐acuity trauma in the ED. ¹³ Cumulative DISCERN tool data have not been reported since that time, although many subsequent clinical event debriefs have since occurred.

One aspect of emergency care that is often addressed during debriefing is crisis resource management (CRM). CRM constitutes a set of principles, originally drawn from aviation safety work and later applied to medicine, that guide critical event management (Table 1). These practices aim to optimize utilization of available resources and promote excellent teamwork to improve patient outcomes in high‐stress settings. We hypothesized that CRM principles could provide a framework for thematic analysis of nontechnical components of clinical event debriefing data. ¹⁴

TABLE 1.

Crisis resource management principles ¹³

Know the environment

Anticipate and plan

Call for help early

Exercise leadership and followership

Distribute the workload

Mobilize all available resources

Communicate effectively

Use all available information

Prevent and manage fixation errors

Cross (double) check

Use cognitive aids

Reevaluate repeatedly

Use good teamwork

Allocate attention wisely

Set priorities dynamically

In this study, we analyzed single‐institution ED debriefing data from DISCERN within the context of CRM principles to determine trends and patterns over several years after implementation. Our primary aim was to see which themes identified by our analysis would map to specific CRM principles. It is possible that successes or gaps in patient care identified via CRM principles could serve as a scaffold to further improve care in the ED setting and elsewhere.

METHODS

Design, setting, and population

This was a retrospective mixed‐methods study evaluating both quantitative and qualitative components of debriefing high‐stakes clinical events. We examined events occurring between 2012 and 2017 in a quaternary pediatric ED that sees 74,000 patients annually. We evaluated DISCERN forms from the study period to determine the type of clinical event, whether a debrief was performed, and which team roles were represented during the debriefing. For the qualitative data collection, we included only events that were debriefed. This study was approved by an institutional review board prior to data analysis.

Quantitative protocol

Quantitative and descriptive information from each event debriefed was documented on the DISCERN form, ¹³ including team leader (attending or trainee), year, type of event (CPR, intubation, etc.), time to start and duration of debrief, patient outcome, debrief leader, and team members present.

For comparisons across multiple categories, the Pearson chi‐square test was utilized. If the p‐value was <0.05, further adjustment using the Bonferroni correction was used to identify which specific values were significant. To determine what clinical factors were associated with a debriefing occurring, two prediction models were created. The first model used a backward‐step approach that initially included all significant (p < 0.05) factors found between debriefing status. Then each factor was removed one by one based on the highest p‐value. Factors that retain a p < 0.05 remained in the model. To address overfitting and to attempt to validate the first model, a second model was generated using the least absolute shrinkage and selection operator procedure. Briefly, this procedure reduces beta coefficients that are not relevant in the model to zero, while calculating a coefficient for relevant factors. The factors that each model identified as a predictor were compared to one another. Adjusted odds ratios (aORs) with 95 confidence intervals (CIs), p‐values, and beta coefficients were reported. Statistical significance was defined as p < 0.05. Analyses were conducted using the Statistical Package for the Social Sciences, version 26 (IBM Corp.) and R: A Language and Environment for Statistical Computing, 2017 (R Foundation for Statistical Computing).

Qualitative protocol

Researchers used constant comparison analysis, whereby they repeatedly reviewed DISCERN forms and formulated an initial framework of key codes and refined these codes with successive reviews until thematic saturation was reached. Codes were clustered into themes informed by CRM principles. Aspects of the clinical event that went well and aspects that could have gone better, as described by debrief participants, were coded into general thematic categories. If CRM principles applied, the specific principle was documented. Forms were independently coded by two members of the study team to allow for investigator triangulation. The two members developed mutually agreed‐upon definitions for each code and established examples of each code to ensure reliability and trustworthiness. The same two members met to discuss the qualitative findings and develop themes from the codes. A third member of the research team was available to review any disagreements in coding that were resolved by team consensus. Memos of coding decisions were kept to provide consistency in coding as analysis progressed. Member checking was performed to review themes and to check for accuracy and completeness of the findings. This thematic approach is a widely utilized process in the analysis of qualitative data ¹⁵ and was used in our study to identify a conceptual framework of themes related to CRM principles.

RESULTS

Quantitative analysis

Of the 1875 resuscitation events that occurred in the department from 2012 to 2017, 400 (21%) were debriefed using the DISCERN tool. A trend analysis showed that the frequency of DISCERN tool usage for known target events decreased steadily over time after the first years of its implementation (p < 0.001; Figure S1). Specifically, significant differences were found after the year 2013 following residual adjustment.

A full clinical event debrief occurred 170/400 times (42% of the time). Debriefing after code or significant trauma events was encouraged but not mandatory. A debrief was declined by the participating staff the remaining 230 times, most commonly because team members felt it was not needed (63%) or due to ED acuity (25%).

Clinical event debriefs were often led by an attending (74%), fellow (22%), or both an attending and a fellow (4%). Often, debriefing was prompted by either a physician or the charge nurse or the bedside nurse in collaboration with the physician, given that shared nursing and physician ownership of the debriefing process was emphasized in the training and announcements provided to ED staff during the rollout of this project. The DISCERN form allowed teams to list more than one event type for each instance of debriefing; the most common resuscitation event types cited included intubation (85%), events requiring other respiratory interventions (64%), and medical events (non–trauma related, 29%). In a majority of cases, debriefing began within 60 min of the clinical event (60%). The majority of debriefings lasted between 5 and 10 min (46%) or more than 10 min (48%). The most common team members present were the physician team lead (96%), primary nurse (91%), other nurse (87%), and respiratory therapist (77%). Residents physicians and other trainees participated in debriefs as well, if they were part of the clinical event team.

In most of the debriefed events (82%) the patient survived. Clinical events were significantly more likely to be debriefed if the patient expired (42 of 47 events), than if the patient survived (112 of 311 events, OR = 15.64, 95% CI = 6.02–40.57). An event involving CPR had over 11 times the odds of a debrief occurring relative to an event that did not involve CPR (OR = 11.8, 95% CI = 4.12–33.8). Further, high‐acuity trauma cases had a significantly higher rate of debrief occurring (20 of 29 events debriefed, OR = 3.18, 95% CI = 1.40–7.18).

Qualitative analysis

The qualitative portion of the DISCERN form contains two main segments of free text. The first asks what went well during care of the patient. Of the entries in this section, the most frequently cited theme related to CRM principles (Figure 1). Other themes included pediatric advanced life support (PALS) and/or basic life support–related statements, actionable statements, educational items, medical management, and trauma management. Member checking was accomplished using a focus group of 10 PEM attending and fellow physicians.

FIGURE 1.

Percentage of “what went well” statements pertaining to each category by year. CRM, crisis resource management; PALS, pediatric advanced life support

CRM‐related statements were further divided into the specific CRM principle(s) that applied. The five categories most frequently mentioned in “what went well” statements were “communicate effectively,” “know the environment,” “mobilize all available resources,” “anticipate and plan,” and “distribute the workload”; trends over time are shown in Figure S2. Examples from each of these CRM principles are provided in Table 2. Two non‐CRM topics were mentioned repeatedly as examples of what resuscitation teams had handled well. Debrief team members were pleased with airway management in 8% of the statements and crowd control in 7.4% of statements.

TABLE 2.

CRM‐related statements, “what went well”

CRM principle	Illustrative quotes (pulled from free text)
Communicate effectively	Had plan clearly verbalized every step of the resuscitation
	Excellent closed‐loop communication
	Excellent handoff from EMS
Mobilize all available resources	Blood, surgery arrival fast
	X‐ray at bedside
	Having CVICU nurse show how to set up art[erial] line
Distribute the workload	Good job assigning roles
	Well defined team roles
	Everyone had a role
Know the environment	Not too many people in room
	Good standing positions
	Documenter in good position, visualize monitors and patient
Anticipate and plan	Prepared prior to arrival—blood obtained
	Staff was ready and prepared for patient's arrival
	Anticipated potential decompensation

Abbreviations: CRM, crisis resource management; CVICU, cardiovascular intensive care unit; EMS, emergency medical systems.

Further, data showed that PALS‐related statements were more common in events involving expired patients (17.4% vs. 4.1%, OR = 4.91, 95% CI = 2.68–9.01) and/or those receiving CPR (14.3% vs. 3.5%, OR = 4.58, 95% CI = 2.36–8.91). Examples of PALS‐related statements (pulled from free‐text entries on debriefing forms) include “good time keeping,” “great compressions,” and “Broselow quickly.”

The second free‐text portion of the form asks what could have gone better during care of the patient. A similar analysis was run on the statements made by debrief participants in response to this question. CRM principles were again the most commonly coded theme across these statements. Figure 2 demonstrates this finding, along with the other most frequently mentioned clinical themes from this section.

FIGURE 2.

Percentage of “what could have gone better” statements pertaining to each theme by year. CRM, crisis resource management; PALS, pediatric advanced life support

Eight percent of the free‐text statements were classified as actionable items. Additionally, 3.5% of statements were education related. Examples of both are given in Table S1. The resuscitation event with the highest frequency of actionable item statements was defibrillation. In events where defibrillation occurred, 23% of statements were actionable items (OR = 3.27, 95% CI = 1.15–0.35).

The CRM principles most frequently mentioned in “what could have gone better” statements were “know the environment,” “communicate effectively,” “mobilize all available resources,” “anticipate and plan,” and “set priorities dynamically”; trends for these statements over time are shown in Figure S3. Examples from the free‐text related to each CRM principle are included in Table 3.

TABLE 3.

CRM‐related statements, “what could have gone better”

CRM principle	Illustrative quotes (extracted from free text)
Know the environment	Too many people in the room
	Keeping compressions on the right and ultrasound on the left
	Where is the NICU equipment?
Communicate effectively	Confusing orders
	Lab calling nurse with results so she could not pay attention to resuscitation
	Better prehospital communication, did not know patient had bilateral chest tubes
Mobilize all available resources	Difficulty getting emergency release blood from blood bank
	X‐ray took way too long to arrive
	Assigning someone to be with mom; no chaplain
Anticipate and plan	Have pedi[atric] Yankauer ready
	Monitor set up and ready to go prior to patient arrival
	Equipment should be ready—there was no stylet in place
Set priorities dynamically	Getting patient on monitors quicker
	Securing airway first during primary survey
	Keep parent updated

Abbreviations: CRM, crisis resource management; NICU, neonatal intensive care unit.

DISCUSSION

To our knowledge, other than a prior operating room–based study evaluating debriefing checklists, ¹⁶ the present study comprises the largest set of single‐institution clinical event debriefing data currently available. Furthermore, it is the first to provide qualitative data from team‐based reflection on high‐value resuscitation parameters such as teamwork, communication, and resource utilization. Prior studies in the setting of trauma and resuscitation have suggested that debriefing after high‐stakes clinical events is beneficial for multiple reasons ¹ , ³ , ⁶ , ⁷ , ⁹ , ¹¹ , ¹⁷ ; however, ED providers debrief inconsistently and, in the absence of debriefer training, ineffectively. ² , ⁴ , ⁷ , ⁹ , ¹⁰ , ¹¹ , ¹² , ¹⁸ In response to such needs, the DISCERN tool provides a structured format to facilitate debriefs in the ED led by members of the clinical team. ¹³ We found that after its implementation, instances of clinical event debriefing were initially common, but then decreased in subsequent years. Using a mixed‐methods approach, we ascertained which clinical themes were most widely discussed and found that nontechnical aspects of resuscitations were mentioned most frequently. Finally, we examined perceived barriers to clinical event debriefing.

In the first 2 years after implementation of DISCERN, there was a relatively high number of debriefs per year, because the creation of a systematic method for conducting debriefs and its inclusion in a quality improvement initiative likely encouraged medical teams to discuss team performance after clinical events. However, over subsequent years, the number of debriefs per year steadily declined. This decrement may have occurred, in part, because the initial physician champion for debriefing in our ED and lead author of DISCERN moved to another institution. For clinical event debriefing to consistently occur, it is likely that having both a standardized process and a consistent debriefing advocate in the department are beneficial to achieve sustainability, as described previously in the literature. ¹¹ , ¹⁹ , ²⁰ In our study, clinical events involving CPR or patient death were significantly more likely to undergo debriefing, similar to what has been reported in previous studies. ² , ³ It may be that events involving higher emotional toll for the team create a greater sense of urgency to debrief.

The most commonly reported barriers to clinical event debriefing were that the team felt it was not necessary and/or that there were too many concurrent clinical demands at the time. Previous studies have cited lack of time, space, and facilitator training as well as uninterested, defensive, or uncomfortable participants as debriefing barriers. ¹ , ³ , ⁴ , ⁶ , ¹¹ When a debrief by one of our teams occurred, it most commonly began within an hour of the clinical event, in keeping with recommendations for “hot debriefing” ¹¹ , ²¹ and typically lasted up to or longer than 10 min, similar to previous reports. ¹⁹ , ²¹ It may be that formal training could provide tools for debrief leaders on efficiently guiding debriefs to balance team member participation with limiting the discussion to a shortened time frame. ¹² , ¹⁷ If average clinical event debrief duration were shorter, debriefing might be more likely to occur, even within a busy ED.

Throughout the years of this study, statements representing CRM principles were by far the most widely mentioned themes in both the “what went well” and the “what could have gone better” portions of clinical event debriefs. Nontechnical skills were consistently the bulk of the material that clinical teams discussed. This finding may suggest that ED teams feel comfortable with medical management and PALS algorithms, because even with the highest‐acuity events such as patient death, these topics did not surface as much. Both positive and negative appraisals voiced by team members dealt primarily with topics such as communication, planning ahead, and setting priorities. Such nontechnical aspects of resuscitation are arguably as important to target as are technical details (e.g., dose and timing of defibrillation) in our efforts to optimize patient outcomes. Our findings regarding CRM principles cited in debriefing are all the more noteworthy given that prior studies have identified failures in teamwork and communication to be common sources of medical error and adverse events. ⁷ , ²² , ²³

Of note, four of the five most frequently mentioned CRM principles in the plus/delta sections are the same. This raises the possibility of the availability heuristic impacting debrief discussions, and that topics mentioned by team members were perhaps not the ones which needed to be addressed the most but rather were the ones most mentally available. Readily available concepts may not always translate to team discussions that are the most probing and ultimately the most impactful toward improved team performance and patient care. For example, the lack of discussion regarding specifics of medical management may reflect an inherent discomfort with addressing knowledge gaps of fellow team members. It may be easier for team members to stick with group‐based, “softer” discussions regarding CRM principles as opposed to gaps in each other's knowledge and performance.

Multiple initiatives relating to resuscitation have been implemented in our ED over the past several years, many of which could address the CRM principles frequently mentioned in the “what could have gone better” section of the DISCERN forms. Examples include “breakthrough communication” courses, intubation quality improvement checklists, monthly morbidity and mortality case conferences targeting Safety II elements, and in situ simulations (e.g., interprofessional resuscitation training involving physicians and nurses, simulations specifically for residents and fellows, and trauma‐specific scenarios). However, the performance areas of greatest concern to ED team members, i.e., CRM principles, appeared relatively unchanged during the study period.

Increasing the frequency of clinical event debriefs could potentially help with improving these areas of performance concern, in part by closing the gap between educational activities and the day‐to‐day actions of patient care. Discussion during debriefing promotes ongoing conversation about the CRM principles that teams consistently felt needed to be better addressed. If debriefing became a regular and routine aspect of ED functioning, the CRM principles that need the most improvement would continually be reviewed and discussed among the ED staff. This change could lead to improved staff awareness regarding the previously noted gaps in performance during clinical events. Actionable items and education‐related statements, which are prevalent data from the “what could have gone better” portion of the DISCERN form, could be specific points for improvement.

Currently, there are several initiatives within our clinical setting to increase the frequency of clinical event debriefs. The DISCERN form has been converted to an electronic survey accessible via QR code, is being expanded for broader use as part of a hospital‐wide resuscitation quality improvement program, and has also been adapted in a new tool specifically for COVID‐19 events. ²⁴ Signs have been placed in shock/trauma rooms reminding teams to debrief. There are now regularly occurring CPR quality meetings at the hospital level to coordinate improvement work in various clinical units as well as simulation initiatives at the ED and hospital system levels with the aim of maximizing the impact of educational and quality improvement efforts on resuscitation performance.

Further, it may be that the usefulness of actionable items and education‐related statements mentioned by teams during debriefs can be maximized via a follow‐up process that includes review of past DISCERN forms, with reference to ongoing resuscitation improvement and patient safety discussions. Such review may increase the likelihood that future teams will benefit from past discussions of items requiring improvement. And while the existing DISCERN tool has relied on informal review of debriefs by a clinical nurse specialist in the ED to identify action items in need of follow‐up, it may be that more systematic review of events by a multidisciplinary team could better identify findings in need of report‐out to medical, nursing, and quality/safety leaders.

In the future, formal clinical event debriefing training could also be beneficial and might enable additional team members to act as debriefing champions as well as providing these staff members with tools to effectively lead teams through the debrief process after specific clinical events. Trained expert debriefers may also contribute to deeper team discussions targeting not just the most readily available topics but also the performance areas most needing improvement which are often difficult to bring up. Normalizing the discussion of errors likely requires a higher level of expert debriefer, difficult to attain without formal training. Previous studies have identified ED charge nurses as possible staff members to fill this role, because they often do not have individual patient assignments, but are typically present during any high‐stakes clinical event and have a strong understanding of ED functioning and processes. ¹⁷ , ¹⁹ Debriefer training has been established at various institutions around the world and could potentially empower debriefing champions within individual clinical units. ¹¹ , ¹⁷ Furthermore, additional CRM‐based training for ED staff may be helpful as well and might have the added benefit of helping further prime team members for action‐oriented discussions during clinical event debriefing.

Limitations

This was a single‐institution study presenting data from a pediatric ED within a large quaternary care hospital. Some study factors, including the number of ED staff present for a resuscitation, the different levels of trainees present, and the volume of resuscitations, may not fully apply to other institutions. Furthermore, a chart review was not performed for each resuscitation event, and thus it is unknown whether the patient population represented in these clinical events is representative of our patient population as a whole. Finally, because this was a retrospective study based on handwritten forms, some DISCERN forms had missing or illegible data that we were not able to recover. Two independent coders were used in the evaluation of the qualitative data, which may have introduced bias based on their feelings and experiences regarding debriefings. However, utilizing two coders, with a third coder available to resolve discrepancies, allowed for investigator triangulation and ensured trustworthiness. Furthermore, member checking was performed to confirm consistency and dependability of the findings.

Finally, teams in our study of debriefings seemed to focus more on CRM‐related concerns than on the medical management of PALS, but the study was not designed to correlate their qualitative discussions of performance with other parameters, such as quantitative measures of CPR quality. Within the resuscitation education literature, there is a growing awareness that reported self‐efficacy for clinical skills is not an adequate indicator of performance in real‐life emergencies. ²⁵ , ²⁶ It may be that future studies of clinical event debriefing can help address possible gaps between perception and reality regarding resuscitation performance in cardiac arrests and other high‐stakes events.

CONCLUSION

We examined 6 years of clinical event debrief data after implementation of a structured debriefing tool. Our qualitative assessment found that medical management was discussed with less relative frequency during clinical event debriefs; by comparison, crisis resource management principles were recurrent topics of discussion, and the specific principles cited remained largely consistent through the years. It is possible that initiatives to increase the frequency and usefulness of clinical event debriefs will better address existing needs and perceived patient care deficits and further the goal of continued improvement in quality of care.

CONFLICT OF INTEREST

The authors have no potential conflicts to disclose.

Supporting information

Supplementary Material

Chu J, Alawa N, Sampayo EM, Doughty C, Camp E, Welch‐Horan TB. Evolution of clinical event debriefs in a quaternary pediatric emergency department after implementation of a debriefing tool. AEM Educ Train. 2021;5:e10709. doi: 10.1002/aet2.10709