Emergency Department Thoracotomy: Development of a Reliable, Validated Checklist for Procedural Training

Abstract

Objectives

Emergency department thoracotomy (EDT) is a rare and challenging procedure. Emergency medicine (EM) residents have limited opportunities to perform the procedure in clinical or educational settings. Standardized, reliable, validated checklists do not exist to evaluate procedural competency. The objectives of this project were twofold: 1) to develop a checklist containing the critical actions for performing an EDT that can be used for future procedural skills training and 2) to evaluate the reliability and validity of the checklist for performing EDT.

Methods

After a literature review, a preliminary 22‐item checklist was developed and disseminated to experts in EM and trauma surgery. A modified Delphi method was used to revise the checklist. To assess usability of the checklist, EM and trauma surgery faculty and residents were evaluated performing an EDT while inter‐rater reliability was calculated with Cohen's kappa. A Student's t‐test was used to compare the performance of participants who had or had not performed a thoracotomy in clinical practice. Item‐total correlation was calculated for each checklist item to determine discriminatory ability.

Results

A final 22‐item checklist was developed for EDT. The overall inter‐rater reliability was strong (κ = 0.84) with individual item agreement ranging from moderate to strong (κ = 0.61 to 1.00). Experts (attending physicians and senior residents) performed well on the checklist, achieving an average score of 80% on the checklist. Participants who had performed EDT in clinical practice performed significantly better than those that had not, achieving an average of 80.7% items completed versus 52.3% (p < 0.05). Seventeen of 22 items had an item‐total correlation greater than 0.2.

Conclusions

A final 22‐item consensus‐based checklist was developed for the EDT. Overall inter‐rater reliability was strong. This checklist can be used in future studies to serve as a foundation for curriculum development around this important procedure.

Trauma is a leading cause of death for persons age 1 to 44.1 Penetrating trauma continues to represent a particularly lethal problem with a higher prehospital and emergency department (ED) mortality compared to blunt trauma.2, 3 Carefully selected patients presenting to the ED with penetrating thoracic or abdominal trauma may benefit from an emergency department thoracotomy (EDT).4, 5 When performing an EDT, the physician emergently enters the thoracic cavity with the goals of identifying and temporizing direct damage from a penetrating injury. Specifically, the physician may relieve cardiac tamponade, directly control hemorrhage, or cross‐clamp the thoracic aorta to provide hemorrhage control and prioritize cardiac and cerebral perfusion. While the indications for this procedure have become more selective over the past several decades, it remains a critical and potentially lifesaving procedure within the scope of practice for trauma and emergency medicine (EM) physicians.5 EDT is an invasive, technically challenging, and resource‐intensive process.6, 7 It is often performed with little preparatory time and almost always performed by an EM physician or trauma surgeon as opposed to a thoracic surgeon. If performed incorrectly, the lifesaving potential of an EDT may be attenuated and there may be an increased risk of blood‐borne pathogen exposure to providers. EDTs are rarely performed, and studies of EM residents show a lack of opportunities to develop competency in this procedure.8, 9 In addition to a paucity of real‐world experience, opportunities for deliberate practice are by most accounts nonexistent. Cadaveric models may be considered for procedural training; however, the expense of models and the need for repetitive deliberate practice to ensure competency make cadaveric models cost‐prohibitive for education on a widespread basis.10 The other major obstacle with cadaveric models lies with the difficulty in repeatedly simulating several of the key components for which an emergent thoracotomy is performed, such as identifying and relieving cardiac tamponade, controlling hemorrhage of cardiac injuries, and repairing cardiac wounds. Even when practice opportunities for EDT arise in clinical or educational contexts, standardized, reliable, and valid checklists do not exist to ensure procedural competency. In contrast, other rare yet lifesaving procedures, such as cricothyrotomies, have valid and reliable tools for assessing performance.11 Standardized checklists have been shown to improve trainee performance for many common procedures.12, 13 To our knowledge there is no validated checklist for performing an EDT. The objectives of this project were twofold: first, to develop a checklist containing the critical actions for performing an EDT that can be used for future procedural skills training, and second, to evaluate the reliability and validity of the checklist for EDT.

Methods

Study Design

This is a prospective checklist creation and validation study. Multiple experts in EM and trauma surgery were recruited and a modified Delphi method was used for checklist creation.14, 15 The checklist was then validated by an additional group of physicians who were observed performing the procedure by two raters. This study was reviewed by the institutional review board at Northwestern University Feinberg School of Medicine and deemed to be exempt.

Study Setting and Population

The checklist was developed and validated at an urban academic medical center in 2017.

Study Protocol

To identify items to include in the checklist, the authors conducted a review of literature and textbooks for relevant content to performing the procedure. The literature was searched using PubMed for a resuscitative or EDT. Search terms included “thoracotomy AND simulation OR simulate OR simulator,” “thoracotomy AND curriculum,” “thoracotomy AND teaching OR instruction OR practice OR education,” and “thoracotomy AND residents/residency.” A total of 318 articles were identified in the initial literature search. Of these, 20 articles were selected for relevance as they contained information on the steps necessary to successfully perform an EDT. An additional resource included review of the “Resuscitative Thoracotomy” chapter of Roberts & Hedges’ Clinical Procedures in Emergency Medicine & Acute Care, 7th Edition.16 Based on this review, a preliminary 22‐item binary‐response checklist was developed by the authors HQZ, SNS, and DTM. The checklist was developed to contain steps starting with preparation of equipment through performing the procedure and stopping at open cardiac massage based on accepted intervenable injuries that are likely to yield the most favorable outcomes as described by guidelines published by major U.S. trauma associations.4, 5 A binary model was chosen given the intended purpose and progressive stepwise approach commonly taken to perform an emergent thoracotomy.17

An iterative process was used to revise the checklist.15, 17 A group of experts including six emergency physicians and two trauma surgeons were recruited. All experts were board certified in their respective fields. The group contained two academic trauma surgeons, five academic emergency physicians, and one emergency physician in community practice, all of whom had clinical experience performing EDT and training physicians in the procedure. Geographically, the experts practiced clinically across the United States, six in the Midwest, one in the South, and one in the West. The experts were initially asked to review the preliminary checklist developed from the literature review described above. Specifically, they were asked whether or not the checklist reflected the critical steps for performing EDT. Subsequently, they were asked to comment on whether or not the checklist needed additions or removal of steps to be a complete description of how to perform an EDT. Responses were compiled, and a revised checklist was again distributed. This process was repeated and a total of three rounds of revision resulted in consensus among the experts regarding the critical steps of the EDT.

Once final consensus from the expert panel was achieved, the checklist was reviewed for usability. One of the authors (HQZ) performed an EDT on the simulator while two additional authors (DHS and SNS) observed the procedure, followed along on the checklist, and made comments to further clarify the checklist and to confirm the steps were sufficiently described and able to be scored appropriately by the evaluator. This final step resulted in changes neither to the items on the checklist nor to the order of steps, but did result in clarification of descriptions of a correct performance of an item.

After the checklist was finalized, nine physicians who were not involved in the development of the checklist were recruited to perform an EDT on a simulated model. They included six surgical residents (PGY‐4 to ‐7), two EM attending physicians, and one attending trauma surgeon. None of these physicians were involved in the creation of the checklist or had prior access to the checklist. The simulated model was designed and built at Northwestern Simulation and allowed for all of the critical steps of the procedure to be performed and observed (Figure 1).

Figure 1.

Thoracotomy simulation trainer with EDT steps being performed. (A) Incision of the chest wall. (B) Incision of the pericardium. (C) Cross‐clamping of the aorta. (D) Open cardiac massage being performed. EDT = ED thoracotomy.

Upon arrival to the simulation center, the purpose of the procedural simulation was explained to the participants. They were provided a brief clinical scenario describing a patient with a single thoracic penetrating gunshot wound with loss of vital signs just prior to arrival to the ED. They were instructed that an EDT was indicated and were asked to perform the procedure. The participants were informed that the patient would be arriving in 3 minutes. During that time, each individual prepared and gathered any equipment necessary to perform an EDT. Typical personal protective equipment (PPE) and thoracotomy equipment was made available. The group was instructed to verbalize what items they needed and correctly identify them in the thoracotomy tray. After either 3 minutes had passed or the individual had completed all correct preparatory items, the evaluator indicated that the patient had arrived and the participant could proceed to perform the procedure.

Measures

Performance of the EDT was observed in real time by two raters (DHS, ALF) who independently scored each participant. These two raters were not a part of the expert panel who developed the initial checklist content. Each step was scored by the raters as being performed “correct” or “incorrect.” The raters were positioned adjacent to the simulator and participant for the duration of the procedure. Additionally, cameras were positioned above and behind the participants so raters were also able to observe from additional angles, after their real‐time evaluation, if they deemed they needed additional views to complete their evaluation.

Data Analysis

Reliability, as measured by inter‐rater reliability of the two observers, was evaluated with Cohen's kappa coefficient for each checklist item and the checklist as a whole.18 The confidence interval was set at 95%. All statistical analyses were performed using Microsoft Excel v14.7. Descriptive statistics were completed on participants’ performance on the checklist. Participants were then grouped according to experience by years of training and by self‐reported completion of an EDT in clinical practice. An independent Student's t‐test was used to compare the performance between these groups to determine concurrent validity. Finally, an item‐total correlation was calculated to determine the ability of each checklist item to discriminate between experienced and inexperienced participants as a measure of convergent validity.

Results

A final 22‐item checklist was developed for the EDT (see Data Supplement S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10387/full). Overall inter‐rater reliability was strong, with a Cohen's kappa coefficient of 0.84. Individual item results are detailed in Table 1, with individual results ranging from moderate to strong (κ = 0.61 to 1.00), with the exception of items 8 (extend incision) and 10 (insert spreader), which had minimal agreement (κ = 0.00), and item 9 (manually spread ribs), which had minimal disagreement (κ = –0.29).

Table 1.

Inter‐rater Agreement for a 22‐item ED Thoracotomy Checklist

Item	Kappa
1. Don PPE	1.00
2. Gather equipment	1.00
3. Check equipment	0.77
4. Assemble spreader	1.00
5. Position patient	1.00
6. Prepare chest	1.00
7. Incision	1.00
8. Extend incision	0.00
9. Manually spread ribs	‐0.29
10. Insert spreader	0.00
11. Open spreader	1.00
12. Identify heart	1.00
13. Identify phrenic nerve	1.00
14. Lift pericardium	1.00
15. Incise pericardium	0.61
16. Deliver heart	1.00
17. Identify cardiac injury	1.00
18. Control cardiac hemorrhage	1.00
19. Identify aorta	1.00
20. Cross‐clamp aorta	1.00
21. Cardiac massage	0.61
22. Maintain sterility	0.77
Total	0.84

PPE = personal protective equipment.

Attending physicians (n = 3) completed an average of 73.9% of checklist items correctly when their performance on the EDT was scored via the checklist. Senior surgery residents (PGY‐6 and above; n = 1) and attendings (n = 3) completed an average of 79.5% of items correctly. The cohort of participants who had completed at least one EDT in clinical practice (n = 7) completed an average of 80.7% of items correctly. Those participants who had not completed a real EDT scored an average of 52.3% of items correctly (n = 2). A Student's t‐test was used to compare these groups (Table 2). There was no significant difference between attendings and residents (n = 3, n = 6; T = 0.28, p = 0.78) or between senior residents/attendings and junior residents (n = 4, n = 5; T = 0.057, p = 0.54). However, when comparing those who had not done a thoracotomy in clinical practice to those who had, a significant difference in checklist item completion was found (n = 7, n = 2; T = 3.47, p < 0.05). Item discrimination statistics are shown in Table 3. Item‐total correlation was less than 0.2 for only five of the 22 items (Table 3).

Table 2.

Checklist Discrimination Between Participants on a 22‐item EDT Checklist

Cohort (A vs. B)	A	B	t‐test	p‐value
Attending physician (n = 3) vs. all residents (n = 6)	73.9%	71.4%	0.28	0.78
EDT experience (n = 7) vs. no EDT experience (n = 2)	80.7%	52.2%	3.47	<0.01

EDT = ED thoracotomy; PPE = personal protective equipment.

Table 3.

Discriminant Ability and Item Discrimination of a 22‐item EDT Checklist

Item	Has Performed EDT (Mean Percent Correct†)		Item‐total Correlation
	No	Yes
1. Don PPE	100.0	100.0	0.000
2. Gather equipment	50.0	50.0	0.160
3. Check equipment	0.0	58.3	0.297
4. Assemble spreader	0.0	33.3	0.377
5. Position patient	50.0	100.0	0.745
6. Prepare chest	50.0	66.7	0.240
7. Incision	0.0	83.3	0.883
8. Extend incision	75.0	100.0	0.355
9. Manually spread ribs	50.0	58.3	0.231
10. Insert spreader	75.0	100.0	0.355
11. Open spreader	50.0	100.0	0.404
12. Identify heart	100.0	100.0	0.000
13. Identify phrenic nerve	50.0	100.0	0.745
14. Lift pericardium	50.0	83.3	0.224
15. Incise pericardium	50.0	91.7	0.607
16. Deliver heart	50.0	66.7	0.543
17. Identify cardiac injury	50.0	100.0	0.745
18. Control cardiac hemorrhage	50.0	100.0	0.745
19. Identify aorta	100.0	66.7	–0.420
20. Cross‐clamp aorta	100.0	83.3	–0.278
21. Cardiac massage	50.0	91.7	0.702
22. Maintain sterility	0.0	41.7	0.590
Total	52.7	80.7

EDT = ED thoracotomy; PPE = personal protective equipment.

^†Mean percentage of items performed correctly for both raters combined.

Discussion

Through the process of literature review, expert consensus using a modified Delphi method, pilot testing, and refinement, we have developed a checklist for the performance of an EDT. When the checklist was presented to the experts, agreement was high on a majority of the items in the initial rounds. However, there were a few items which required additional discussion regarding three domains of EDT performance: preferred thoracotomy equipment, methods to temporize cardiac wounds, and approaches to identify and cross‐clamp the aorta. These differences were hypothesized to be related to practice variation. Experts had variable responses on their preferred thoracotomy equipment. For example, multiple tools were recommended to cut through bone. To achieve consensus, we included a variety of potential equipment with the emphasis on making sure the provider had some method to cut through bone if requested (item 2). There was also variation regarding the approach for the management of cardiac wounds (item 18). Some responses were consistent with established literature and expert agreement to use sutures with pledgets.16, 19 However, our experts also suggested other methods of repair, such as staples or Foley catheter insertion, with the latter being amenable to transfusion directly into the cardiac chamber (item 18). The final item requiring additional discussion involved cross‐clamping the aorta, specifically with regards to identifying the difference between the aorta and esophagus. Although experts agreed that the aorta and esophagus must be distinguished to cross‐clamp (item 19), their preferred methods varied. For these items without consensus evidence and significant clinical equipoise, we opted to include all suggested possible approaches in the checklist. As a result, multiple approaches for temporizing cardiac wounds were included as acceptable options in the checklist. Both of the suggested approaches for identification of the aorta including placing a nasogastric tube or utilizing anatomic and tactile differences were included as acceptable for the checklist item.

When using the checklist in a simulated procedural scenario, raters found the checklist easy to use, and overall inter‐rater reliability was strong (Table 1). The majority of individual items on the checklist also had a strong kappa coefficient. The items that had lower inter‐rater agreement included extending the incision (item 8), manually spreading the ribs (item 9), and inserting the rib spreader correctly (item 10). There are several factors that likely resulted in this low inter‐rater agreement. First, several of the items may have been more subjective than initially anticipated. This may have been most apparent in the scoring of item 9–manually spreading the ribs and item 8–extending the incision. Second, while both raters were adjacent to the EDT trainer and the participant, it is possible that subtle differences in viewpoints while watching the procedure may have impacted the scoring.

While these steps did show lower inter‐rater reliability, during the process of checklist development, they were identified as critical steps according to expert consensus and review of literature on performing an emergent thoracotomy and were therefore not removed. We hypothesize the lower correlation to be due to visual limitations of the model when having two raters watching the same technical procedure. Further clarification and delineation of how to score these items correctly could improve agreement. Prompting the participant to more routinely verbalize the actions during the procedure could minimize this discrepancy. These items will be highlighted in subsequent rater training for procedure evaluations to reduce discrepancy and maximize visibility of performance. We are actively reevaluating with a larger sample size to determine if inter‐rater reliability can be improved for these few items.

Despite the few items mentioned, overall strong inter‐rater agreement of the two observers over multiple tests demonstrates reliability of the checklist.18 While demonstrating validity with clinical thoracotomies in real time is impractical due to the rarity and nature of the procedure, we are able to demonstrate concurrent validity by the significant performance difference of physicians who had completed an EDT in real life versus those who had not. In other words, the checklist can differentiate between people with experience in performing an EDT versus those who have not. This may translate to assessing whether a person has acquired sufficient skills to be considered proficient at performing an EDT after training. The checklist's convergent validity is demonstrated by the item‐total correlation. A value greater than 0.2 for an item suggests that there is good discriminatory ability for that item, which was the case for 17 of the 22 items in this checklist.20, 21 Good discriminatory ability means that someone who performs well overall on the checklist would likely score “correct” on a given item.

Although there were five items with poor discriminatory ability, they were kept in the checklist due to their consensus through the modified Delphi as necessary steps of the procedure. Several of these steps, such as donning PPE or identifying the heart, are quite fundamental and remain a critical part of the procedure. Interestingly, identifying and cross‐clamping the aorta was negatively discriminatory, meaning that the novices were more likely to perform this step compared to experts. Several possible explanations could account for this finding. These include the small sample size as well as an impact of previous clinical experience. It is also possible that the sample of novices may have approached the procedure in an algorithmic fashion, whereas experts did not perform this step as they may have integrated previous clinical experience and believe cross‐clamping the aorta would be ineffective based on the visualized cardiac injury.

With the emergence of endovascular procedures such as resuscitative endovascular balloon occlusion of the aorta (REBOA), one might argue that the role of EDT may become less important. REBOA is a novel tool to help obtain hemorrhagic control of a noncompressible subdiaphragmatic injury in a trauma patient with profound shock22 and overlaps with the EDT step of cross‐clamping the aorta. However, for cardiac or thoracic injuries particularly with the suspicion for cardiac tamponade, myocardial injury, or other thoracic vascular injury resulting in cardiac arrest, the resuscitative EDT remains the standard of care. Any suspected thoracic injury is a contraindication for the use of endovascular balloon therapy.23 In addition, recent joint statements American College of Surgeons Committee on Trauma and the American College of Emergency Physicians have recommended that EM physicians without critical care training should not perform REBOA.24 At this time there does not appear to be an alternative for the patient suffering from traumatic arrest with suspected thoracic injuries amenable to an EDT. Education and training for emergency physicians in this rare but critical procedure will need to continue until a suitable alternative is demonstrated.

Limitations

There are several limitations to our study. While we recruited experts from multiple geographic areas and types of clinical environments to participate in the checklist development process, the overall sample size was small and the majority of our experts were from urban academic institutions, which may limit the generalizability of this checklist. However, we believe the impact of this to be minimal as performing an EDT is generally agreed upon to be a procedure occurring in trauma centers. Differences in resources between community, rural, and urban environments may affect the application of the checklist in other settings. The development of the checklist may have also been limited by the content expertise of our panel. All of our experts were physicians and having members trained in human factors design may have added additional rigor to the checklist.

The inter‐rater reliability of our checklist and the concurrent validity was assessed with a small sample size. A larger sample size would have provided greater power to more precisely understand the instrument's characteristics. For the few items that had lower inter‐rater agreement, this may reflect a limitation based on the vantage of the rater, the checklist itself, or the model utilized. Further study into these items in particular with a larger sample size and using nonresearcher evaluators may help to assess and delineate the source of the lower inter‐rater agreement.

The checklist development occurred in the context of asking the expert panel to identify key steps in performing this critical procedure, the validation of the checklist, and the assessment of inter‐rater reliability was performed using a simulated model. It is possible that the model used possessed the potential limitation of overlooking key critical actions or steps in the procedure that would only be uncovered or revealed in testing the checklist on model with better fidelity characteristics such as a cadaver or human patient. However, given that cadaveric models may be cost‐prohibitive for training on a widespread basis,10 the infrequent yet critically important nature of being able to correctly perform the procedure, and the potential application to combat training, we believe that this checklist fills the long‐empty niche of a standardized tool for the assessment of competency in EDT. Future studies can assess the transferability and performance of this expert consensus derived checklist to the completion of an EDT on a cadaver or other simulated models.

Finally, this checklist is based on a hypothetical scenario of a gunshot wound leading to a single anterior ventricular injury that was simulated using a thoracotomy model. While this is a valid indication for an EDT, we did not specifically seek to create a checklist for a particular injury but rather for general performance of EDT. A variety of injuries could be discovered upon gaining surgical access to the left hemithorax. Given this phase of the project focused on developing a checklist for the performance of an EDT by an emergency physician, we focused the scenario on one that combines the most common indication with a repairable injury that would be most likely to yield a favorable outcome.4, 5, 14 There may be items on the checklist that are more critical or less critical depending on the specific injury leading to the decision to perform EDT. To further validate the checklist, it must be applied to a broader range of simulated scenarios and clinical practice.

Conclusion

We describe the development of a valid, reliable checklist for performing an ED thoracotomy. Given the critical nature of this procedure with limited opportunities for practice, we believe that the creation of this validated checklist can serve as the foundation for curricular development. Future studies should be conducted to further validate this checklist across a more diverse range of practice environments, with additional types of scenarios and, ultimately, in the clinical environment. It is our goal to improve education for an ED thoracotomy to further procedural training and competence with the ultimate aim of improved patient outcomes.

Supporting information

Data Supplement S1. EDT Checklist.

AEM Education and Training 2020;4:139–146.