Chest ultrasonography versus supine chest radiography for diagnosis of pneumothorax in trauma patients in the emergency department

Abstract

Background

Chest X‐ray (CXR) is a longstanding method for the diagnosis of pneumothorax but chest ultrasonography (CUS) may be a safer, more rapid, and more accurate modality in trauma patients at the bedside that does not expose the patient to ionizing radiation. This may lead to improved and expedited management of traumatic pneumothorax and improved patient safety and clinical outcomes.

Objectives

To compare the diagnostic accuracy of chest ultrasonography (CUS) by frontline non‐radiologist physicians versus chest X‐ray (CXR) for diagnosis of pneumothorax in trauma patients in the emergency department (ED).

To investigate the effects of potential sources of heterogeneity such as type of CUS operator (frontline non‐radiologist physicians), type of trauma (blunt vs penetrating), and type of US probe on test accuracy.

Search methods

We conducted a comprehensive search of the following electronic databases from database inception to 10 April 2020: Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, MEDLINE, Embase, Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus, Database of Abstracts of Reviews of Effects, Web of Science Core Collection and Clinicaltrials.gov. We handsearched reference lists of included articles and reviews retrieved via electronic searching; and we carried out forward citation searching of relevant articles in Google Scholar and looked at the "Related articles" on PubMed.

Selection criteria

We included prospective, paired comparative accuracy studies comparing CUS performed by frontline non‐radiologist physicians to supine CXR in trauma patients in the emergency department (ED) suspected of having pneumothorax, and with computed tomography (CT) of the chest or tube thoracostomy as the reference standard.

Data collection and analysis

Two review authors independently extracted data from each included study using a data extraction form. We included studies using patients as the unit of analysis in the main analysis and we included those using lung fields in the secondary analysis. We performed meta‐analyses by using a bivariate model to estimate and compare summary sensitivities and specificities.

Main results

We included 13 studies of which nine (410 traumatic pneumothorax patients out of 1271 patients) used patients as the unit of analysis; we thus included them in the primary analysis. The remaining four studies used lung field as the unit of analysis and we included them in the secondary analysis. We judged all studies to be at high or unclear risk of bias in one or more domains, with most studies (11/13, 85%) being judged at high or unclear risk of bias in the patient selection domain. There was substantial heterogeneity in the sensitivity of supine CXR amongst the included studies.

In the primary analysis, the summary sensitivity and specificity of CUS were 0.91 (95% confidence interval (CI) 0.85 to 0.94) and 0.99 (95% CI 0.97 to 1.00); and the summary sensitivity and specificity of supine CXR were 0.47 (95% CI 0.31 to 0.63) and 1.00 (95% CI 0.97 to 1.00). There was a significant difference in the sensitivity of CUS compared to CXR with an absolute difference in sensitivity of 0.44 (95% CI 0.27 to 0.61; P < 0.001). In contrast, CUS and CXR had similar specificities: comparing CUS to CXR, the absolute difference in specificity was −0.007 (95% CI −0.018 to 0.005, P = 0.35). The findings imply that in a hypothetical cohort of 100 patients if 30 patients have traumatic pneumothorax (i.e. prevalence of 30%), CUS would miss 3 (95% CI 2 to 4) cases (false negatives) and overdiagnose 1 (95% CI 0 to 2) of those without pneumothorax (false positives); while CXR would miss 16 (95% CI 11 to 21) cases with 0 (95% CI 0 to 2) overdiagnosis of those who do not have pneumothorax.

Authors' conclusions

The diagnostic accuracy of CUS performed by frontline non‐radiologist physicians for the diagnosis of pneumothorax in ED trauma patients is superior to supine CXR, independent of the type of trauma, type of CUS operator, or type of CUS probe used. These findings suggest that CUS for the diagnosis of traumatic pneumothorax could be incorporated into trauma protocols and algorithms in future medical training programmes. In addition, CUS may beneficially change routine management of trauma

Plain language summary

How accurate is chest ultrasonography compared to supine chest radiography for diagnosis of traumatic pneumothorax in the emergency department?

Why is improving the diagnosis of traumatic pneumothorax important?

Air that collects between the lung and the chest wall is described as a pneumothorax. Pneumothorax can cause collapse of the lung, change the position of the heart and other structures in the chest, reduce the blood flow back to the heart, and cause life‐threatening shock. Physicians may perform tube thoracostomy — a procedure with risk of complications such as haemorrhage, organ injury, and infection — to evacuate the air trapped. Not recognizing a pneumothorax (false negative (FN)) can lead to heart and lung failure and death. An incorrect diagnosis of a pneumothorax (false positive (FP)) may lead to inappropriate tube thoracostomy.

What is the aim of this review?

To determine how accurate chest ultrasonography (CUS) is compared to chest X‐ray (CXR) in diagnosing pneumothorax in trauma patients in the emergency department (ED). Researchers included 13 studies to answer this question.

What was studied in the review?

We compared the diagnostic accuracy of two tests, CUS and CXR. We then compared these two tests to computed tomography (CT) or, if clinically necessary, tube thoracostomy as the reference standard.

What are the main results of the review?

The analysis included results from 1271 trauma patients, where 410 had traumatic pneumothorax.

The results of these studies indicate that, in theory, if CUS was used on a group of 100 patients where 30 (30%) have traumatic pneumothorax, then an estimated 28 would have a CUS result positive for pneumothorax (TP) and of these one (3.6%) would be incorrectly classified as having the pneumothorax (FP); of the 72 patients with a result negative for pneumothorax, three (4.2%) would actually have a pneumothorax (FN).

In theory, if CXR was used on a group of 100 patients where 30 (30%) have traumatic pneumothorax, then an estimated 14 would have a CXR result positive for pneumothorax (TP) and of these none (0%) would be incorrectly classified as having the pneumothorax (FP); of the 86 patients with a result negative for pneumothorax, 16 (18.6%) would actually have a pneumothorax (FN).

How reliable are the results of the studies in this review?

The numbers shown in the results are averages across all studies in the review. While CUS results were fairly consistent, CXR results were quite varied; thus, we cannot be sure that CXR will always produce the same results. In the included studies, the diagnosis of traumatic pneumothorax was confirmed by CT or tube thoracostomy. Although there were some problems with how some of the studies were conducted, their results did not differ from the more reliable studies.

Who do the results of this review apply to?

The results may not be representative of patients in different settings or with pneumothorax of different aetiologies. Studies included in the review were focused on diagnosing traumatic pneumothorax in the ED, conducted in three continents. The average prevalence of traumatic pneumothorax was 30% and ranged from 21% to 52%.

What are the implications of this review?

The studies in this review show that CUS is more accurate than CXR in diagnosing pneumothorax in ED trauma patients, which may lead to more timely treatment with tube thoracostomy, reducing pneumothorax‐related complications, and improving outcomes. The risk of missing the diagnosis with CUS is low (4.2% of those whose CUS suggests they do not have a pneumothorax) suggesting that only a few patients may not immediately receive tube thoracostomy. The risk of incorrectly diagnosing traumatic pneumothorax using CUS is low (3.6% of those whose CUS suggests they have a pneumothorax) and may result in receiving unnecessary tube thoracostomy.

In comparison, the risk of missing a traumatic pneumothorax with CXR is high (18.6% of those whose CXR suggests they do not have a pneumothorax) suggesting that a large number of patients may not immediately receive tube thoracostomy. The risk of wrongly diagnosing traumatic pneumothorax using CXR is low (0% of those whose CUS suggests they have a pneumothorax).

How up to date is this review?

The review authors searched for and included publications from 1900 to 10 April 2020.

Summary of findings

Summary of findings 1. Summary of findings for chest ultrasonography versus supine chest radiography for diagnosis of pneumothorax.

What is the diagnostic accuracy of chest ultrasonography by frontline non‐radiologist physicians compared to supine chest X‐ray for diagnosis of pneumothorax in trauma patients in the emergency department?
Patient population	Trauma patients with suspected traumatic pneumothorax
Prior testing	Varied, clinical examination, supine chest X‐ray (CXR), and computed tomography (CT)
Settings	Emergency department (ED)
Index tests	Chest ultrasonography (CUS), supine CXR
Reference standard	CT of the chest or clinical findings of a rush of air or bubbling in chest drain after tube thoracostomy (TT)
Target condition	Traumatic pneumothorax of any severity
Importance	Diagnostic accuracy of test modalities to identify traumatic pneumothorax rapidly and accurately can improve the efficiency and efficacy of trauma management and potentially improve patient safety and outcomes.
Included studies	We included 13 prospective, paired comparative accuracy studies that compared CUS vs CXR. 9 studies using patients as unit of analysis were included in the primary analysis. 4 studies using lung fields as unit of analysis were included in the secondary analysis.
Risk of bias and applicability concerns	Assessment of risk of bias was largely limited by unclear reporting of blinding of outcome assessors interpreting CXR and CT. Risk of bias in patient selection was high due to inappropriate exclusion criteria or was unclear due to poorly reported patient selection methods. Risk of bias assessment in the flow and timing was largely limited by unclear intervals between index tests and reference standard and missing CT data. Applicability concerns for majority of the domains were low.
Limitations	While there was little variation in sensitivity and specificity of CUS across studies, we observed considerable heterogeneity in the estimates for CXR. All studies were judged high or unclear risk of bias in one or more domains, with most studies (11/13, 85%) being judged at high or unclear risk of bias in terms of patient selection.
Test	Studies	Patients (cases)	Sensitivity (95% CI)	Specificity (95% CI)	Consequences in a cohort of 100*
					Prevalence 28%		Prevalence 30%		Prevalence 36%
					Missed cases (95% CI)	Overdiagnosis (95% CI)	Missed cases (95% CI)	Overdiagnosis (95% CI)	Missed cases (95% CI)	Overdiagnosis (95% CI)
Primary analysis: comparison of CUS and CXR using patients as unit of analysis
CUS	9	1271 (410)	0.91 (0.85 to 0.94)	0.99 (0.97 to 1.00)	3 (2 to 4)	1 (0 to 2)	3 (2 to 4)	1 (0 to 2)	3 (2 to 5)	1 (0 to 2)
CXR	9	1271 (410)	0.47 (0.31 to 0.63)	1.00 (0.97 to 1.00)	15 (10 to 19)	0 (0 to 2)	16 (11 to 21)	0 (0 to 2)	19 (13 to 24)	0 (0 to 2)
Difference			0.44 (0.27 to 0.61), P < 0.001	−0.007 (−0.018 to 0.005), P = 0.35
CAUTION: the results in this table should be interpreted taking into account the methodological quality of the studies and the limitations highlighted.
Conclusions: the sensitivity of CUS performed by frontline non‐radiologist physicians is significantly superior to that of supine CXR for diagnosing traumatic pneumothorax in trauma patients in the ED. There was substantial heterogeneity in the sensitivity of supine CXR between studies and most studies had unclear or high risk of bias in terms of patient selection. Well‐designed and properly reported prospective comparative accuracy studies that use appropriate patient selection criteria are needed.

*The prevalence values used to illustrate the review findings as absolute frequencies are the median and 25th and 75th percentiles from the included studies

Background

Thoracic trauma can cause significant morbidity and mortality, directly accounting for 20% to 25% of deaths from trauma (Rosen 2014). Injury to any of several vital intrathoracic organs can result in immediate death. Traumatic pneumothorax is a common complication of thoracic trauma, occurring in 15% to 50% of patients with significant thoracic trauma (Khandhar 2007).

Target condition being diagnosed

Pneumothorax occurs when air collects between the parietal and visceral pleurae, causing the lung parenchyma to collapse. Traumatic pneumothorax commonly occurs when a fractured rib damages the pleural lining or punctures a lung with resultant air leakage (ATLS 2012; Rosen 2014; Sharma 2008). Traumatic pneumothorax without rib fracture occurs when a traumatic force compresses the chest in a person with a closed glottis, suddenly increasing intrathoracic pressure and resulting in alveolar rupture (Rosen 2014). The size of the pneumothorax is quantified based on the proportion of the pleural cavity that is occupied by air, with less than 15% of the pleural cavity graded as small, 15% to 60% as moderate, and more than 60% as large (Rosen 2014). Occult pneumothoraces are those that are not initially detected by chest X‐ray (CXR) but are found on computed tomography (CT) (Rosen 2014). Tension pneumothorax occurs when trapped air significantly displaces mediastinal structures, reducing blood flow back to the heart and resulting in life‐threatening cardiopulmonary collapse (Rosen 2014).

Pneumothorax results in a ventilation/perfusion mismatch (Rosen 2014). Patients typically report dyspnoea and chest pain. Early detection of pneumothorax is important for determining management and disposition in trauma patients. Failure to detect and treat pneumothorax could lead to acute complications including hypoxia, tension pneumothorax, cardiopulmonary failure, or death (Rosen 2014; ATLS 2012). This is especially important in patients undergoing general anaesthesia and positive‐pressure ventilation, and among those transported by air at high altitude, as the pneumothorax can quickly progress to a life‐threatening tension pneumothorax (ATLS 2012). Long‐term complications of untreated pneumothorax include the development of pneumomediastinum, re‐expansion pulmonary oedema, empyema, or bronchopulmonary fistula (Rosen 2014). Identification and management of occult pneumothorax is currently a topic of discussion in the trauma literature owing to the risk of clinical deterioration in a patient with an unrecognized occult pneumothorax who undergoes positive‐pressure ventilation (Mowery 2011). Clinical deterioration occurs as the result of an increase in the size of the pneumothorax, ultimately producing a tension pneumothorax, which causes shock by obstructing venous return to the heart (Rosen 2014). Early detection and decompression of significant pneumothorax are therefore imperative.

Management of pneumothorax depends on the clinical status of the patient and the volume of air trapped in the pleural space. If the pneumothorax is considered clinically significant, treatment consists of a tube thoracostomy (ATLS 2012). Studies have, however, provided conflicting evidence regarding whether to treat or not treat occult pneumothorax before the patient undergoes positive‐pressure ventilation (Enderson 1993; Kirkpatrick 2013). Emergency physicians and trauma surgeons perform this procedure at the bedside by inserting a tube into the pleural space for evacuation of collected air. The tube is typically attached to suction drainage to maintain a negative pressure within the pleural cavity while facilitating lung re‐expansion (ATLS 2012). Tube thoracostomy is associated with a reported complication rate of 5% to 40%; complications include haemorrhage, organ injury, and infection (Filosso 2017; Kwaitt 2014).

The risks of clinical deterioration associated with missing a diagnosis and the potential harm and complications that can result from incorrectly treating a non‐existent pneumothorax highlight the clinical importance of a safer, more rapid, and more accurate method of diagnosing traumatic pneumothorax.

Index test(s)

Chest ultrasonography

Chest ultrasonography (CUS) may be a safer, more rapid, and more accurate modality than CXR for the diagnosis of pneumothorax in trauma patients. Studies have shown high sensitivity and specificity of CUS in non‐trauma settings, such as in the intensive care unit, or with post‐procedure iatrogenic pneumothorax (Chung 2005; Lichtenstein 2005; Shostak 2013). The Advanced Trauma Life Support (ATLS) protocol currently recommends the use of ultrasonography (US) when Focused Assessment With Sonography for Trauma (FAST) is performed for assessment of intra‐abdominal injuries (ATLS 2012). CUS can be completed in conjunction with the FAST scan at the bedside while the patient is lying supine, without moving the patient out of the resuscitation bay, and can be an effective diagnostic tool for detecting thoracic injuries. Because US utilizes high‐frequency sound waves, the patient is not exposed to ionizing radiation.

Trauma patients are typically assessed in the supine position. Air collected in the pneumothorax rises up towards non‐dependent areas within the thoracic cavity. CUS is completed in the longitudinal plane with the indicator pointing cephalad, and the probe is placed in the third or fourth intercostal space in the midclavicular line and is repeated on both sides of the patient (Chan 2003; Husain 2012; Lichtenstein 2005). Although a microconvex probe is ideal, other transducers such as the convex or linear array probe may be used (Volpicelli 2012).

Four individual sonographic findings are associated with pneumothorax on CUS (Volpicelli 2012).

• Absence of lung sliding

• Absence of B‐lines or comet‐tail artefact

• Presence of lung point

• Absence of lung pulse

Normal lungs are attached to the visceral pleura and slide along the parietal pleura in a rhythmical pattern with the respiratory cycle. Via M‐mode, a visual representation of lung sliding over time can be generated, known as the "seashore sign" (Alrajhi 2012; Husain 2012; Lichtenstein 2005). In pneumothorax, air trapped in the pleural space disrupts this rhythmical sliding, and M‐mode would demonstrate the "barcode sign" or "stratosphere sign" (Husain 2012; Lichtenstein 2005). Comet‐tail artefacts, or B‐lines, are bright hyperechoic vertical rays produced by reverberation artefacts (Alrajhi 2012; Husain 2012). These B‐lines originate from the visceral pleura and move synchronously with lung sliding (Chan 2003; Husain 2012). Absence of B‐lines suggests the presence of a pneumothorax. The lung point is the point at which the visceral pleura of the lung begins to separate from the parietal pleura of the chest wall at the margin of a pneumothorax; this is visible on CUS (Lichtenstein 2005). Finally, lung pulse comprises the subtle rhythmical movements of the pleura due to cardiac oscillations (Volpicelli 2011; Volpicelli 2012).

While most CUS studies will assess patients as the unit of analysis, some studies may utilize lung fields. This distinction is important as clinicians typically perform CUS by comparing both sides of the chest in the same patient. If a patient has suffered previous trauma, surgery, predisposing lung pathology, or even a previous pneumothorax, these changes may alter the characteristics of the lung field (Volpicelli 2012). Using a single lung field rather than comparing both sides of the chest in the same patient may change the CUS test characteristics altogether.

Chest X‐ray

Current ATLS guidelines recommend the use of CXR as an adjunct to the primary survey in the initial trauma assessment (ATLS 2012). This diagnostic tool is commonly used to identify many thoracic injuries such as haemothorax, pneumomediastinum, pulmonary contusion, or rib fracture. Previous literature has shown that it is not a sensitive test for detecting pneumothorax, however (Wilkerson 2010). For many reasons, the trauma patient is usually kept supine during acute resuscitation until a full assessment to identify injuries is completed. Performing supine CXR requires time, resources, and equipment and may further delay the diagnosis and management of pneumothorax. A film cassette or a flat panel detector must be placed underneath the supine patient, and the X‐ray tube brought in over the top of the patient. Positioning the cassette or detector may require rolling the patient, risking further injury, and prolonging resuscitation of the patient. To protect healthcare providers from radiation exposure, all personnel within the vicinity must wear lead‐shielded personal protective gear or must vacate the area, leaving the patient unattended. CXR exposes the patient to a small dose of radiation, estimated at 0.1 millisievert (mSv), or the equivalent of exposure to natural background radiation for 10 days (Chung 2014). The X‐ray must be positioned correctly and must be timed to synchronize with the patient's inspiration. The entire process of completing supine CXR can therefore be very disruptive and may delay resuscitation of the trauma patient.

Clinical pathway

Trauma patients in the emergency department are initially assessed clinically for immediate life‐threatening conditions. Resuscitative measures such as administration of intravenous fluids or blood products, airway intubation, or tube thoracostomy may be required. Many emergency physicians and trauma surgeons consider the use of US for FAST scans as 'standard of care', as it can be used to identify intra‐abdominal injuries that may require immediate operative management. Supine CXR is used as an adjunct to the primary survey to identify intrathoracic injuries but "should be used judiciously, and should not delay patient resuscitation" (ATLS 2012).

The secondary survey allows for a more thorough clinical examination as well as specialized diagnostic tests such as X‐rays of specific areas like the spine or wrist, CT, or angiography. These specialized diagnostic tests typically require transporting the patient out of the resuscitation bay and into the diagnostic imaging department, which typically is ill equipped for resuscitative interventions. Unfortunately, if no pneumothorax is suspected on clinical examination or CXR, the clinician may opt to not do a CT scan of the chest and may miss a clinically significant pneumothorax. CT scans of the cervical spine in trauma patients have detected occult pneumothoraces that were previously missed on supine CXR, or when patients did not receive a CT of their chest (Ball 2012). Depending on the clinical status of the patient, the extent of injury, and the capability of the hospital, the patient will be further treated by a trauma surgery service or will be transferred to a centre with trauma care expertise.

CUS may have a role in the primary survey for rapidly diagnosing clinically significant traumatic pneumothorax as a source of instability in a critically ill trauma patient. Traumatic pneumothorax identified with CUS may provide an accurate and rapid diagnosis, leading to immediate decompression with needle, finger, or tube thoracostomy (Figure 1). In addition to pneumothorax, CUS has been studied to help identify other chest pathologies, such as pulmonary contusions, pulmonary oedema, pneumonia, and pleural effusions (Volpicelli 2012). Until further studies can demonstrate that CUS is a superior diagnostic tool to CXR for other traumatic pathologies, CXR will continue to play an important role in the initial diagnostic evaluation of ED trauma patients.

1.

Current and proposed clinical pathway ‐ CUS may provide a faster and more accurate diagnosis of traumatic pneumothorax, leading to immediate tube thoracostomy in an unstable trauma patient. Supine CXR is a useful diagnostic tool for identification of other traumatic pathologies, such as rib fractures, mediastinal injuries, etc.

Once a pneumothorax has been identified, clinicians will determine whether tube thoracostomy is clinically warranted. Generally speaking, it is accepted practice that a tube thoracostomy is indicated when a pneumothorax is identified in a hypotensive trauma patient (ATLS 2012). In the context of this review, producing a true positive (TP) equates to finding a pneumothorax, which may lead to a clinically appropriate tube thoracostomy, and a false positive (FP) suggests that a pneumothorax has been found when there is none, potentially leading to a clinically unnecessary tube thoracostomy. A true negative (TN) would successfully rule out a pneumothorax, leading to an appropriate decision to not perform tube thoracostomy; whereas a false negative (FN) would mean that a pneumothorax that may have required a clinically necessary tube thoracostomy might be missed.

Many hospitals and healthcare systems do not have in‐hospital trauma specialists, intensivists, or radiologists to perform CUS or tube thoracostomy. In most emergency departments, the frontline physician assessing and treating trauma patients is an emergency physician or a trauma surgeon. Hence, these physicians play a key role in the initial diagnosis and management of traumatic pneumothorax. Once a patient's condition has stabilized, and the patient has been resuscitated, the frontline physician arranges for the patient to be transferred to a designated trauma centre for further assessment and management, if clinically warranted.

Alternative test(s)

Clinical examination for pneumothorax may reveal hyper‐resonance on percussion, subcutaneous emphysema on palpation, and decreased or absent breath sounds on auscultation (Rosen 2014). These findings are not reliable for a small pneumothorax (Noppen 2008). Moreover, the accuracy and utility of these physical exam manoeuvres are limited in a noisy and chaotic resuscitation bay.

Reference standards

CT is considered the reference standard for detection of thoracic injuries including pneumothorax (Alrajhi 2012; Chung 2014; Wilkerson 2010). CT technology has drastically improved over the years, allowing for greater image resolution and improved sensitivity in detecting pathology. CT reveals the diagnosis of pathology with the perspective of its relation to the rest of the thorax. CT has limitations, however. Transporting a potentially unstable patient away from the resuscitation bay to the diagnostic imaging department has its inherent risks due to lack of equipment, space, and personnel to help with resuscitation should the patient decompensate. In addition, CT exposes the patient to ionizing radiation estimated at 7 mSv or the equivalent of exposure to two years of natural background radiation (Chung 2014). Allergic reactions to CT contrast dye present additional risk.

Depending on the patient's condition, tube thoracostomy may be performed emergently at any point during trauma resuscitation. Upon insertion of the chest tube into the pleural space, a rush of air or bubbling in the chest drain confirms the diagnosis of pneumothorax. This has been accepted in the trauma literature as an alternative reference standard (Alrajhi 2012; Wilkerson 2010).

Rationale

US technology has progressively improved over the years and has become more accessible and portable in the emergency department (Husain 2012). Image generation has become easier and more reliable with new hardware and software. Recognizing the importance of bedside US, many healthcare systems, hospitals, and specialty training programmes have incorporated US training for their non‐radiologist physicians. Emergency physicians and trauma surgeons are already using bedside US for FAST scans in trauma patients. Rapid detection of traumatic pneumothorax with CUS may lead to more timely and efficient management with tube thoracostomy, reducing the incidence of pneumothorax‐related complications, and thus improving outcomes in ED trauma patients.

Systematic reviews on the diagnostic accuracy of CUS have been published, but studies included in these reviews have significant heterogeneity in patient population, aetiology of pneumothorax, operator medical background (radiologists, intensivists, respirologists, etc.), methodological quality, and poor meta‐analytic methods (Alrajhi 2012; Alrajab 2013; Ding 2011; Ebrahimi 2014). The aetiology of pneumothorax is important to consider: trauma patients lie supine for CUS and CXR, whereas if the cause of the pneumothorax is spontaneous or iatrogenic (for example, resulting from a biopsy or from a central line insertion), or if it occurs post‐operatively, the patient may not have been lying supine, significantly altering the test characteristics of both CUS and CXR.

Therefore, we aimed to assess the diagnostic accuracy of CUS compared with supine CXR for the detection of pneumothorax in emergency department trauma patients. The findings of this review may provide evidence for the incorporation of CUS into trauma (e.g. ATLS) protocols and algorithms in future medical training programmes, and may potentially influence routine management of trauma.

Objectives

To compare the diagnostic accuracy of chest ultrasonography (CUS) by frontline non‐radiologist physicians versus chest X‐ray (CXR) for diagnosis of pneumothorax in trauma patients in the emergency department (ED).

To investigate the effects of potential sources of heterogeneity such as type of CUS operator (frontline non‐radiologist physicians), type of trauma (blunt vs penetrating), and type of US probe on test accuracy.

Secondary objectives

To determine the diagnostic accuracy of individual CUS findings such as the absence of lung sliding, absence of B‐lines or comet‐tail artefact, presence of lung point, and absence of lung pulse.

Methods

Criteria for considering studies for this review

Types of studies

We included prospective, paired comparative accuracy studies in which patients were suspected of having pneumothorax. Patients must have undergone both CUS by frontline non‐radiologist physicians and CXR as index tests, as well as CT of the chest or tube thoracostomy as the reference standard. We excluded studies involving participants with already diagnosed pneumothorax (i.e. case‐control studies); studies involving participants with non‐traumatic pneumothorax; studies involving participants who had already been treated with tube thoracostomy; and studies in which a frontline non‐radiologist physician did not perform CUS.

Participants

We included trauma patients in the emergency department setting, irrespective of age and gender.

Index tests

The two main index tests were CUS completed by a frontline non‐radiologist physician and CXR, both being performed in the supine position. If data on specific CUS findings (such as the absence of lung sliding, absence of B‐lines or comet‐tail artefact, presence of lung point, and absence of lung pulse) were available, we planned to assess the diagnostic accuracy of these individual CUS findings.

Target conditions

The target condition was traumatic pneumothorax of any severity.

Reference standards

We defined a pneumothorax identified on CT scan of the chest or via clinical findings of a rush of air or bubbling in a chest drain after tube thoracostomy as the reference standard.

Search methods for identification of studies

Electronic searches

We searched PROSPERO and the Cochrane Library for related systematic reviews. We developed and carried out systematic searches in the following electronic databases: Cochrane Database of Systematic Reviews (via OVID, 2005 to 10 April 2020); Cochrane Central Register of Controlled Trials (via OVID, 1991 to 10 April 2020); MEDLINE (via OVID, 1946 to 10 April 2020); Embase (via OVID, 1974 to 10 April 2020); Cumulative Index to Nursing and Allied Health Literature (CINAHL) Plus (via EBSCO, 1937 to 10 April 2020); Database of Abstracts of Reviews of Effects (via OVID, 1991 to 10 April 2020); Web of Science core collection (which includes: Science Citation Index Expanded, 1900 to 10 April 2020; Social Sciences Citation Index, 1900 to 10 April 2020; Arts & Humanities Citation Index, 1975 to 10 April 2020; Conference Proceedings Citation Index ‐ Science, 1990 to 10 April 2020; Conference Proceedings Citation Index ‐ Social Sciences & Humanities, 1990 to 10 April 2020; and Emerging Sources Citation Index, 2015 to 10 April 2020); and Clinicaltrials.gov (to 10 April 2020). We used sensitive search strategies as recommended in Chapter 7 of the Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (De Vet 2013). Our search strategy included subject headings and free‐text terms. We applied no language restrictions in the searches. We present the search strategies for all electronic databases in Appendix 1.

Searching other resources

We handsearched reference lists of included articles and reviews, retrieved via electronic searching, for potentially eligible studies that may have been missed in the electronic database searches. We also carried out forward citation searching of relevant articles in Google Scholar and looked at the "Related articles" on PubMed.

Data collection and analysis

Selection of studies

Two review authors (KC and DJ) screened titles and abstracts and excluded irrelevant citations. We obtained the full text of articles that potentially met the inclusion criteria based on initial screening. Two review authors (KC and DJ) independently screened these articles for inclusion. We (KC and DJ) resolved any discrepancies through discussion; if disagreements arose, a third review author (AM) arbitrated.

Data extraction and management

Two review authors (KC and DJ) independently extracted data using a standardized data collection form (Appendix 2). A third review author (AM) evaluated any discrepant judgements. When necessary, we contacted study authors for clarification or additional data.

We collected the following information.

• General characteristics: title, journal, year, institution, country where the study was performed, study period, study design, sample size, units of analysis (per patient or per lung field), type of CUS operator (frontline non‐radiologist physicians).

• Population characteristics: age, gender, type of trauma, inclusion/exclusion criteria used in study, sampling used in study.

• Accuracy data for CUS, CXR, and individual US findings (absence of lung sliding, absence of B‐lines or comet‐tail artefact, presence of lung point, and absence of lung pulse): two‐by‐two tables of the numbers of true positives, false positives, false negatives, and true negatives, or summary statistics that will enable derivation of the tables.

• Time to CUS, CXR, and CT.

• Type of US probe (curvilinear, high‐frequency linear, etc.) and transducer.

• Definitions of test positivity for each index test and reference standard.

• Reference standard: characteristics of CT or tube thoracostomy.

Assessment of methodological quality

We used the QUADAS‐2 tool to assess risk of bias and the applicability of each included study. This tool assesses risk of bias in four domains: patient selection; index tests; reference standard; and flow and timing. In addition, we examined concerns about applicability in the first three domains (Whiting 2011). We tailored the tool to our review question, as shown in Appendix 3. One of the signalling questions in the patient selection domain was not applicable because we excluded case‐control studies; we therefore deleted this question from the tool. Two review authors (KC and DJ) performed the assessments independently. We (KC and DJ) discussed and resolved any disagreements that arose through consultation with a third review author (AM). We summarized our overall assessment of the risk of bias and applicability concerns using a tabular presentation.

Statistical analysis and data synthesis

The unit of analysis for the primary analyses was per patient. In the secondary analysis, we used lung field as the unit of analysis. For the preliminary analyses of CUS and CXR, we used Review Manager 5 (RevMan 5) to plot estimates of sensitivity and specificity from studies in receiver operating characteristics (ROC) space and on forest plots (Review Manager 2014).

Since the results of CXR and CUS are binary (i.e. pneumothorax present or absent), we performed meta‐analysis by using a bivariate model to estimate summary sensitivities and specificities (Chu 2006; Reitsma 2005). We performed a direct comparison of the accuracy of the two tests because we included only comparative studies of CUS and CXR in the review. Paired results were not reported by the studies, i.e. studies did not report 2‐by‐4 tables of the cross‐classified results of CUS and CXR amongst the reference standard positives and amongst the reference standard negatives. Thus, for the primary meta‐analysis comparing the accuracy of CUS and CXR, we added test type as a covariate to a bivariate model (bivariate meta‐regression). We included covariate terms for logit sensitivity and logit specificity and allowed the variance parameters of the bivariate model to differ between tests (Macaskill 2013; Takwoingi 2015a). Post estimation of the model, we used the bivariate model parameters to calculate absolute differences in sensitivity and specificity. The 95% confidence intervals for the differences were obtained using the delta method. We used Wald tests to assess the statistical significance of differences in sensitivity and specificity. We did not perform comparative meta‐analysis using lung fields because of limited data. We were, however, able to perform meta‐analysis of each test separately by simplifying the bivariate model to univariate random‐effects logistic regression models for sensitivity and specificity (Takwoingi 2015b). We fitted univariate and bivariate models using the 'meqrlogit' command in Stata version 15 (Stata 2017).

Investigations of heterogeneity

We used forest plots and SROC plots to graphically explore heterogeneity in the estimates of sensitivity and specificity for each test. Due to the number of studies available, we investigated the effect of trauma type (blunt or blunt and penetrating) on each test separately rather than on the relative accuracy of the tests. For CUS, we also investigated the effect of type of CUS operator (emergency medicine or trauma surgery) and the type of CUS probe (curvilinear, linear, or linear and curvilinear). Data were limited in certain subgroups of each covariate so we did not perform meta‐regression to formally investigate the effect of each potential source of heterogeneity on sensitivity and specificity by adding covariate terms to a bivariate model. Instead, we performed subgroup analyses by fitting a bivariate model to each subgroup that had at least four studies.

Sensitivity analyses

We performed a sensitivity analysis to examine the impact of blinding of the outcome assessor on the comparative accuracy of CUS and CXR by excluding studies in which the same frontline non‐radiologist physician who performed the CUS interpreted the CXR.

Assessment of reporting bias

We did not assess reporting bias as the relevant methods are not well developed for systematic reviews of diagnostic test accuracy studies.

Results

Results of the search

We initially identified 3473 studies for screening and removed 1180 duplicate studies. We screened 2293 studies and excluded 2268 abstracts, leaving 25 full‐text studies for assessment of eligibility. Twelve studies were excluded: five had missing data; CUS was not performed by frontline non‐radiologist physicians in four; two included the wrong patient population; one had the wrong study design. We included 13 studies in qualitative and quantitative analysis. There was an excellent agreement between authors in the selection of included studies.

Nine studies used patients as the unit of analysis and we included them in the primary analysis; four studies used lung fields as the unit of analysis and we included them in the secondary analysis. All studies utilized CT or tube thoracostomy as the reference standard. See Figure 2 for the study flow diagram and Table 2 for the summary of key study characteristics.

2.

Study flow diagram

1. Summary of key study characteristics.

Study ID	Trauma type	Type of CUS operator	Type of CUS probe	Blinded	Prevalence (%)	Age	Male Gender (%)	Sampling	Unit of analysis
Abbasi 2013	Penetrating and blunt	Emergency medicine	Linear	Yes	37/146 (25.3%)	34	128 (87.6%)	Convenience	Patient
Blaivas 2005	Blunt	Emergency medicine	Curvilinear	Yes	53/176 (30.1%)	N/A	100 (57%)	Convenience	Patient
Karimi 2013	Unclear	Emergency medicine	Linear and curvilinear	Unclear	73/140 (52.1%)	39	87 (62%)	Unclear	Patient
Nagarsheth 2011	Penetrating and blunt	Trauma surgery	Linear and curvilinear	Yes	22/79 (27.9%)	N/A	83 (66.4%)	Convenience	Patient
Ojaghi Haghighi 2014	Blunt	Emergency medicine	Linear	Yes	52/150 (34.7%)	N/A	124 (82.7%)	Consecutive	Patient
Soldati 2006	Blunt	Emergency medicine	Curvilinear	Yes	56/186 (30.1%)	52	117 (62.9%)	Consecutive	Patient
Uz 2013	Blunt	Emergency medicine	Linear and curvilinear	No	33/107 (30.8%)	37	N/A	Unclear	Patient
Vafaei 2016	Penetrating and blunt	Emergency medicine	Linear and curvilinear	Yes	55/152 (36.2%)	31	118 (77.6%)	Consecutive	Patient
Zhang 2006	Blunt	Emergency medicine	Linear and curvilinear	Yes	29/135 (21.5%)	45	114 (84.4%)	Unclear	Patient
Abdulrahman 2015*	Blunt	Trauma surgery	Linear	Yes	75/610 (12.3%)	34	299 (98%)	Consecutive	Lung field
Hyacinthe 2012*	Penetrating and blunt	Resuscitative anaesthesiology	Curvilinear	Yes	53/237 (22.4%)	39	97 (82%)	Consecutive	Lung field
Kirkpatrick 2004*	Penetrating and blunt	Trauma surgery	Linear	Unclear	43/266 (16.2%)	37	167 (74%)	Unclear	Lung field
Soldati 2008*	Blunt	Emergency medicine	Curvilinear	Unclear	25/218 (11.5%)	41	73 (62.9%)	Consecutive	Lung field

CUS = chest ultrasonography

*Included in secondary analysis as the unit of analysis was lung field.

Methodological quality of included studies

We summarized results of the methodological quality of the included studies in Figure 3. We judged all studies at high or unclear risk of bias in one or more domains, with most studies (11/13, 85%) judged at high or unclear risk of bias in the patient selection domain.

3.

Risk of bias and applicability concerns summary: review authors' judgements about each domain for each included study. Studies included in the secondary analysis: Abdulrahman 2015, Hyacinthe 2012, Kirkpatrick 2004, Soldati 2008. CUS = chest ultrasonography; CXR = chest radiography.

Studies included in the primary analysis

Of the nine studies that we included in the primary analysis, one study had a low risk of bias, two had unclear risk of bias while the remaining six had high risk of bias in the patient selection domain, mostly due to convenience sampling or inappropriate exclusion criteria, such as excluding haemodynamically unstable patients, lack of access to CUS, chest wall injuries precluding CUS, or if CT was not indicated.

The risk of bias in the interpretation of CUS results was low in five studies, unclear in two studies, and high in two studies; this was related to unclear blinding methodology of outcome assessors interpreting CUS and CXR results. The risk of bias in the interpretation of CXR results was low in two studies, unclear in six studies, and high in one study; this was largely due to unclear blinding methodology of outcome assessors interpreting CXR and CT results, as in some studies it was not clear whether radiologists had access to both imaging results or not. The risk of bias introduced in interpretation of the reference standard results was low in three studies but unclear in six studies for similar concerns regarding blinding methodology.

The risk of bias in the flow and timing domain was low in two studies, unclear in four studies, and high in three studies; this was due to the exclusion of patients based on missing CT data or unclear/inappropriate time intervals between CUS, CXR, and CT.

We judged applicability concerns regarding patient selection as low for six studies but high for three studies; this was due to the exclusion of haemodynamically unstable patients or lack of access to CUS despite the study focusing on comparing CUS. We judged one included study to have unclear concern regarding applicability of the reference standard used as there was insufficient reporting of the method of assessment. We deemed all other domains for applicability concerns as low risk for all studies.

Studies included in the secondary analysis

Studies that used lung field as their unit of analysis had several limitations including missing CUS data for some lung fields and using two CUS tests (one for each lung field) compared to one CXR (for both lungs) on the same patient. Inherently, there would be an inability to blind the CUS operator during collection of CUS data while performing the two CUS tests (one on each side of the patient), as well as during the interpretation of the CXR and CT results between the two lung fields. By analysing lung fields separately, it is difficult to ascertain whether patient characteristics, past medical history, or traumatic injury pattern could have affected one or both lungs and may have confounded the diagnostic accuracy.

Out of the four studies included in the secondary analysis, the risk of bias in the patient selection domain was low in one study, unclear in two studies, and high in one study; this was due to inappropriate exclusion criteria, such as excluding haemodynamically unstable patients or chest wall injuries precluding CUS, or due to unclear sampling technique.

The risk of bias introduced in interpreting CUS results was low in two studies and unclear in two studies due to lack of clarity about blinding of the outcome assessors interpreting CUS and CXR results. We judged the risk of bias introduced in interpreting CXR and CT results to be low in one study, unclear in two studies, and high in one study; this was again due to definite unblinded interpretation of test results or unclear blinding methodology of outcome assessors interpreting CXR and CT results.

The risk of bias in the flow and timing domain was low in two studies and high in two studies due to missing patient data.

We judged applicability concerns in the patient selection domain as low for two studies, unclear for one study, and high for one study, due to unclear patient selection methods and exclusion of haemodynamically unstable patients or chest wall injuries precluding CUS. We judged one study to have unclear concern regarding applicability of the reference standard, as blinding of the outcome assessor interpreting the results of CUS, CXR, and CT was unclear. We deemed all other domains as 'low concern' for all studies.

Findings

Primary analysis

We included nine comparative accuracy studies of 1271 trauma patients with 410 cases of traumatic pneumothoraces (Figure 4). The median sample size was 146 (range 79 to 186) with a median prevalence of traumatic pneumothorax of 30% (range 21% to 52%).

4.

Forest plot of sensitivity and specificity of chest ultrasonography and supine chest radiography for diagnosis of pneumothorax.

For each test, studies are sorted by trauma type and study identifier.

CUS = chest ultrasonography; CXR = chest radiography; FN = false negative; FP = false positive; TN = true negative; TP = true positive.

The sensitivity of CUS ranged from 0.82 to 0.98; and specificity ranged from 0.96 to 1.00. The sensitivity of CXR ranged from 0.09 to 0.75; and specificity ranged from 0.93 to 1.00.

The summary sensitivity and specificity of CUS was 0.91 (95% confidence interval (CI) 0.85 to 0.94) and 0.99 (95% CI 0.97 to 1.00) respectively (Table 3). The summary sensitivity and specificity of supine CXR was 0.47 (95% CI 0.31 to 0.63) and 1.00 (95% CI 0.97 to 1.00) respectively (Figure 5).

2. Accuracy of chest ultrasonography and supine chest radiography for diagnosis of pneumothorax (patient level analysis).

Test	Studies	Patients (cases)	Sensitivity (95% CI)*	Specificity (95% CI)*
Main analysis: comparison of CUS and CXR
CUS	9	1271 (410)	0.91 (0.85 to 0.94)	0.99 (0.97 to 1.00)
CXR	9	1271 (410)	0.47 (0.31 to 0.63)	1.00 (0.97 to 1.00)
Difference			0.44 (0.27 to 0.61), P < 0.001	−0.007 (−0.018 to 0.005), P = 0.26
Subgroup analyses
CUS: type of trauma
Blunt	5	754 (223)	0.94 (0.86 to 0.98)	0.99 (0.97 to 1.00)
Penetrating & blunt	3	377 (114)	0.84 (0.76 to 0.90)	0.99 (0.94 to 1.00)
Unclear	1	140 (73)	0.85 (0.75 to 0.92)	0.96 (0.87 to 0.99)
CXR: type of trauma
Blunt	5	754 (223)	0.38 (0.19 to 0.62)	1.00 (0.92 to 1.00)
Penetrating & blunt	3	377 (114)	0.32 to 0.67	0.93 to 1.00
Unclear	1	140 (73)	0.75 (0.64 to 0.85)	0.99 (0.92 to 1.00)
Type of CUS operator
Emergency medicine	8	1192 (388)	0.91 (0.85 to 0.95)	0.99 (0.97 to 1.00)
Trauma surgery	1	79 (22)	0.82 (0.60 to 0.95)	1.00 (0.94 to 1.00)
Type of CUS probe
Linear	2	296 (89)	0.86 to 0.96	1.00 to 1.00
Curvilinear	2	362 (109)	0.98 (0.93 to 1.00)	1.00 (0.97 to 1.00)
Linear and curvilinear	5	613 (212)	0.84 (0.78 to 0.88)	0.98 (0.96 to 0.99)
Sensitivity analysis: only studies that blinded results of CUS and CXR between interpretations of each index test modality
CUS	7	1024 (304)	0.92 (0.85 to 0.96)	0.99 (0.97 to 1.00)
CXR	7	1024 (304)	0.49 (0.36 to 0.63)	1.00 (0.95 to 1.00)
Difference			0.43 (0.28 to 0.57), P < 0.001	−0.004 (−0.015 to 0.006), P = 0.42

CUS = chest ultrasonography; CXR = chest radiography.

*The range of sensitivities and specificities are presented where meta‐analysis was not performed because there were only 2 or 3 studies and a fixed effect for sensitivity and/or specificity was not appropriate given the substantial heterogeneity observed in ROC space. When there was only a study within a subgroup, the estimates of sensitivity and specificity and their 95% CIs are presented.

5.

Summary ROC plot of chest ultrasonography and supine chest radiography for diagnosis of pneumothorax

CUS = chest ultrasonography; CXR = chest radiography.

Each study point was scaled according to the precision of sensitivity and specificity in the study. This means that the greater the height of a study point relative to other study points, the greater the precision of the estimated sensitivity in that study. Similarly, for specificity, the greater the width of a study point relative to other study points, the greater the precision of the estimated specificity in that study. The solid circles (summary points) represent the summary estimates of sensitivity and specificity for CUS (black circle) and CXR (red circle). Each summary point is surrounded by a dotted line representing the 95% confidence region and a dashed line representing the 95% prediction region (the region within which one is 95% certain the results of a new study will lie).

There was a significant difference in the sensitivity of CUS compared to CXR with an absolute difference in sensitivity of 0.44 (95% CI 0.27 to 0.61; P < 0.001). In contrast, CUS and CXR had similar specificities: comparing CUS to CXR, the absolute difference in specificity was −0.007 (95% CI −0.018 to 0.005; P = 0.35).

Subgroup analyses

The results of the subgroup analyses for type of trauma, CUS operator, and CUS probe are summarized in Table 3.

Type of trauma

In five studies with 754 blunt trauma patients, 223 had traumatic pneumothoraces. The summary sensitivity and specificity of CUS were 0.94 (95% CI 0.86 to 0.98) and 0.99 (95% CI 0.97 to 1.00) respectively. The summary sensitivity of CXR was 0.38 (95% CI 0.19 to 0.62); and the summary specificity was 1.00 (95% CI 0.92 to 1.00). We did not perform meta‐analyses for the three studies that consisted of both penetrating and blunt trauma patients due to limited data and substantial heterogeneity observed in ROC space. The sensitivity of CUS ranged from 0.82 to 0.86 while specificity ranged from 0.98 to 1.00. The sensitivity of CXR ranged from 0.32 to 0.67 and specificity ranged from 0.93 to 1.00. One study, with an unclear patient population, had a CUS sensitivity of 0.85 (95% CI 0.75 to 0.92) and specificity of 0.96 (95% CI 0.87 to 0.99); CXR had a sensitivity of 0.75 (95% CI 0.64 to 0.85) and specificity of 0.99 (95% CI 0.92 to 1.00) (Karimi 2013).

Type of CUS operator

In eight studies involving 388 patients with traumatic pneumothoraces out of 1192 patients, the CUS operators were emergency physicians. The summary sensitivity of CUS was 0.91 (95% CI 0.85 to 0.95) with a summary specificity of 0.99 (95% CI 0.97 to 1.00). Trauma surgeons were the CUS operator in one study of 79 patients (22 had traumatic pneumothorax). The sensitivity of CUS in this study was 0.82 (95% CI 0.60 to 0.95); and specificity was 1.00 (95% CI 0.94 to 1.00).

Type of probe

Two studies (89 pneumothoraces out of 296 patients) used a linear probe. The sensitivities were between 0.86 and 0.96 with specificity of 1.00. Two studies (109 pneumothoraces out of 362 patients) used a curvilinear probe and had a summary sensitivity of 0.98 (95% CI 0.93 to 1.00); and the summary specificity was 1.00 (95% CI 0.97 to 1.00). The remaining five studies (212 of 613 patients had traumatic pneumothorax) used both linear and curvilinear probes. The summary sensitivity was 0.84 (95% CI 0.78 to 0.88); and summary specificity was 0.98 (95% CI 0.96 to 0.99).

Sensitivity analysis

We performed a sensitivity analysis assessing only studies that blinded outcome assessors evaluating the results of CUS to the results of CXR and vice versa (Table 3). A total of 304 of the 1024 patients included in the seven studies had traumatic pneumothorax. The summary sensitivity of CUS was 0.92 (95% CI 0.85 to 0.96); and summary specificity was 0.99 (95% CI 0.97 to 1.00). The summary sensitivity of CXR was 0.49 (95% CI 0.36 to 0.63); and summary specificity was 1.00 (95% CI 0.95 to 1.00). The absolute difference in the sensitivity of CUS compared to CXR was 0.43 (95% CI 0.28 to 0.57; P < 0.001). Comparing CUS to CXR, the absolute difference in specificity was −0.004 (95% CI −0.015 to 0.006; P = 0.42).

Secondary analysis

We included four comparative studies that used lung fields as the unit of analysis in this analysis (Table 4). The studies included 1331 lung fields of which 196 patients were pneumothorax cases. The odd‐numbered lung fields were due to the fact that CUS data for some lung fields were missing in some of the studies. This suggests that only one side of the patient was assessed with CUS while CXR was performed on both sides of the patient. The median sample size was 252 (range 218 to 610) with a median prevalence of traumatic pneumothorax of 14% (range 11% to 22%).

3. Accuracy of chest ultrasonography and supine chest radiography for diagnosis of pneumothorax (lung field as unit of analysis).

Test	Studies	Lung fields (cases)	Sensitivity (95% CI)*	Specificity (95% CI)*
CUS	4	1331 (196)	0.60 (0.37 to 0.80)	0.98 (0.96 to 0.99)
CXR	4	1331 (196)	0.22 (0.11 to 0.39)	1.00 (0.98 to 1.00)

CUS = chest ultrasonography; CXR = chest radiography.

*The range of sensitivities and specificities are presented where meta‐analysis was not performed because there were only 2 or 3 studies and a fixed effect for sensitivity and/or specificity was not appropriate given the substantial heterogeneity observed in ROC space. When there was only a study within a subgroup, the estimates of sensitivity and specificity and their 95% CIs are presented.

The reported CUS sensitivities ranged from 0.43 to 0.92 while CUS specificities ranged from 0.95 to 0.99. The reported CXR sensitivities ranged from 0.11 to 0.52 while CXR specificities ranged from 0.99 to 1.00. Due to the small number of studies and observed heterogeneity which precluded simplifying a bivariate meta‐regression model to univariate fixed‐effect logistic meta‐regression models, we performed meta‐analysis separately for CUS and CXR and conducted no direct comparison of diagnostic accuracy. The summary sensitivity for CUS was 0.60 (95% CI 0.37 to 0.80); and summary specificity was 0.98 (95% CI 0.96 to 0.99). The summary sensitivity for CXR was 0.22 (95% CI 0.11 to 0.39); and summary specificity was 1.00 (95% CI 0.98 to 1.00).

Discussion

Summary of main results

In the primary analysis based on nine studies, the diagnostic accuracy of CUS performed by frontline non‐radiologist physicians was superior to supine CXR for diagnosing traumatic pneumothorax in trauma patients in the ED. The estimated summary sensitivity of CUS was 0.91 (95% CI 0.85 to 0.94) and the summary specificity was 0.99 (95% CI 0.97 to 1.00); while the summary sensitivity of CXR was 0.47 (95% CI 0.31 to 0.63) and the summary specificity was 1.00 (95% CI 0.97 to 1.00). The findings imply that in a hypothetical cohort of 100 patients if 30 patients have traumatic pneumothorax (i.e. prevalence of 30%), CUS would miss 3 (95% CI 2 to 4) cases (false negatives) and overdiagnose 1 (95% CI 0 to 2) of those without pneumothorax (false positives) while CXR would miss 16 (95% CI 11 to 21) cases with 0 (95% CI 0 to 2) overdiagnosis of those who do not have pneumothorax (Table 1, Figure 6). The findings from the subgroup and sensitivity analyses were consistent with the main findings. This general trend was also consistent with the findings from the secondary analysis using lung field as the unit of analysis.

6.

Hypothetical cohort of 100 ED patients assessed for traumatic pneumothorax.

tp: true positive – test is positive (indicates pneumothorax) and patient has pneumothorax;

fp: false positive – test is positive (indicates pneumothorax) but patient does not have pneumothorax

tn: true negative – test is negative (indicates pneumothorax not present) and patient does not have pneumothorax;

fn: false negative – test is negative (indicates pneumothorax not present) but patient has pneumothorax

Strengths and weaknesses of the review

Completeness of evidence

Our search strategy was comprehensive. We searched multiple electronic databases, in addition to handsearching, and including studies in all languages. It is possible, however, that we may have missed some studies as DTA studies are known to be poorly indexed (Whiting 2009).

While there are many studies on the diagnostic accuracy of individual index tests for diagnosing pneumothorax, there is a paucity of comparative diagnostic accuracy studies. We believe that the methodological rigour of directly comparing the diagnostic accuracy of index tests on the same patient population holds higher value in terms of the level of clinical evidence than comparing diagnostic accuracies of tests performed in different patient populations (Takwoingi 2013).

Limitations in the conduct and interpretation of CUS, CXR, and CT

One limitation of diagnostic accuracy studies of imaging tests is the fact that test interpretation is operator dependent. At the individual study level, results are dependent on the interpretation of CUS findings and CXR findings which can be user and experience dependent. CT interpretation typically requires advanced diagnostic imaging training. We included studies that specified well‐defined positivity criteria. Concerns of risk of bias introduced in the interpretation of results were mostly due to potential incorporation bias from unclear blinding of the outcome assessor interpreting the results of different test modalities, such as whether or not the radiologist interpreting the CT results had access to the results of the CXR. However, the overall trend of superiority of CUS diagnostic accuracy over CXR was consistent. This was supported by our sensitivity analysis using only studies with blinded interpretation of the results of different tests. We used two reference standards (CT or tube thoracostomy if clinically necessary) and this could lead to a potential risk of differential verification bias. We deemed this as an accepted risk as clinicians cannot humanely withhold life‐saving tube thoracostomy for the sake of clinical research at the risk of patient safety.

The quality of reporting of the included studies was variable as shown by the number of 'unclear' risk of bias judgements (Figure 3). This limitation was largely due to poor reporting of methods used for blinding. However, all of the included studies in our primary analyses provided clear and robust numerical diagnostic accuracy data. For studies that used lung fields rather than patients as the unit of analysis, the odd‐numbered lung fields suggest that some patients had CUS assessments on only one side of their body. We emphasize that this is not the standard of care in trauma management and CUS should be performed on both sides of the patient for comparison. Because clinicians perform CUS by producing a live, moving image with respiration and comparing both sides of the chest in the same patient, it is easier to differentiate a pneumothorax from normal lung. By analysing lung fields separately, it is difficult to ascertain whether patient characteristics, past medical history, or traumatic injury pattern could have affected one or both lungs and may have confounded the diagnostic accuracy. For those patients where only one side was assessed, trying to diagnose a pneumothorax using only one lung field without any inherent comparator becomes much harder and we believe this is a reason why diagnostic accuracy was lower and more heterogeneous in the secondary analysis than in the primary analysis. These studies also compared two CUS lung fields to one CXR on the same patient — interpretation of CXR typically consists of comparing the pleural lining between the two lungs to diagnose pneumothorax on a supine radiograph. There is an inherent inability to blind the CUS operator while they collect CUS lung field data on both sides of the patient which could introduce bias in how the data is collected and interpreted. This also applies to the interpretation of CXR and CT results, where the images present both lung fields at the same time to the outcome assessor.

Completeness and relevance of the review

This review specifically focused on the diagnostic accuracy of CUS by frontline non‐radiologist physicians and supine CXR in diagnosing traumatic pneumothorax in the ED. There are many studies that have published CUS diagnostic accuracy data from various settings such as in the intensive care unit, the operating theatre, or in the bronchoscopy suite; by various care providers such as intensivists, anaesthesiologists, or radiologists; or for different aetiologies of pneumothorax such as iatrogenic or spontaneous pneumothorax (Alrajhi 2012; Alrajab 2013; Ding 2011; Ebrahimi 2014). These studies reflect a very different and heterogeneous patient population and pathology, which leads to significantly different diagnostic test characteristics. Typically, most trauma patients are assessed in the ED by a frontline physician who is not a radiologist. Therefore we included studies that best reflect real‐world clinical practice for the ED trauma patient and provide the best evidence relevant to routine clinical practice. To the best of our knowledge, this is the most comprehensive DTA review and meta‐analysis using the highest level of clinical evidence based on comparative accuracy studies of CUS and supine CXR in ED patients with traumatic pneumothorax.

Applicability of findings to the review question

The findings of this review may not be applicable to patients with a different pneumothorax aetiology or in a different clinical setting other than the ED, as test characteristics of CUS or CXR conducted in a non‐supine fashion would be very different. The studies we included in the primary analyses had limitations related to patient selection because some studies had unclear patient selection methods or used convenience rather than sequential patient enrolment. We also had concerns regarding inappropriate exclusion criteria, such as excluding patients in whom CUS was contraindicated (for example, patients who were haemodynamically unstable or those who had chest wall injuries). This could introduce a high risk of bias, as those patients are likely to benefit the most from a rapid and accurate diagnosis of traumatic pneumothorax and should be included in diagnostic accuracy studies. However, the overall trend of diagnostic accuracy in CUS appears superior to supine CXR regardless of studies with or without these inappropriate exclusion criteria. We therefore believe that the findings of this review are robust and CUS is more sensitive than CXR for diagnosing traumatic pneumothorax in ED trauma patients.

Authors' conclusions

Implications for practice.

The diagnostic accuracy of CUS performed by frontline non‐radiologist physicians for the diagnosis of pneumothorax in ED trauma patients is superior to supine CXR. Regardless of type of trauma, type of CUS operator, or type of CUS probe used, the overall sensitivity of CUS is superior to supine CXR and their specificities are similar. While many frontline physicians already use US for FAST scans as 'standard of care' to identify intra‐abdominal injuries, this review provides evidence that CUS is an accurate diagnostic tool compared to CXR for ED patients with traumatic pneumothorax. Rapid detection of traumatic pneumothorax with CUS may lead to more timely therapeutic intervention with tube thoracostomy, reducing the incidence of pneumothorax‐related complications, and thus improving outcomes in ED trauma patients. The findings of this review provide evidence to suggest that CUS could be incorporated into trauma (e.g. ATLS) protocols and algorithms in future medical training programmes. In addition, CUS may beneficially change routine management of trauma.

Implications for research.

Despite the positive findings of this review, there are still concerns regarding the methodological quality of the included studies. Further high‐quality, well‐reported comparative accuracy studies of CUS and CXR are required. In particular, patients should be selected consecutively and with appropriate inclusion/exclusion criteria. Study methodology should include explicit blinding between outcome assessors who are interpreting CUS, CXR, and CT separately. In addition, future studies should explicitly report individual US findings (absence of lung sliding, absence of B‐lines or comet‐tail artefact, presence of lung point, and absence of lung pulse) to enable assessment of the accuracy of these individual findings or their combinations.

What's new

Date	Event	Description
20 August 2020	Amended	Minor wording changes incorporated into Author Conclusions

History

Protocol first published: Issue 5, 2018
Review first published: Issue 7, 2020

Date	Event	Description
3 January 2019	Amended	Editorial team changed to Cochrane Emergency and Critical Care
4 October 2018	Amended	Acknowledgement section amended to include Co‐ordinating Editor

Appendices

Appendix 1. Electronic databases search strategies

Database(s): Ovid MEDLINE(R) Epub Ahead of Print, In‐Process & Other Non‐Indexed Citations, Ovid MEDLINE(R) Daily and Ovid MEDLINE(R) (via OVID, 1946 to 10 April 2020)

#	Searches
1	Pneumothorax/
2	(pneumothora* or "pneumo thora*").tw,kf.
3	PTX.tw,kf.
4	1 or 2 or 3
5	Radiography/
6	radiography, thoracic/
7	radiograph*.tw,kf.
8	roentgen*.tw,kf.
9	radiogram*.tw,kf.
10	radiology.tw,kf.
11	chest film*.tw,kf.
12	CXR.tw,kf.
13	x‐ray*.tw,kf.
14	x ray*.tw,kf.
15	xray.tw,kf.
16	5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15
17	Ultrasonography/ or focused assessment with sonography for trauma/
18	(Ultrasound* or "ultra sound*").tw,kf.
19	sonogra*.tw,kf.
20	(ultrasonogra* or "ultra sonogra*").tw,kf.
21	CUS.tw,kf.
22	ultrasonic.tw,kf.
23	echotomograph*.tw,kf.
24	echograph*.tw,kf.
25	17 or 18 or 19 or 20 or 21 or 22 or 23 or 24
26	4 and 16 and 25
27	26 not (Animals/ not (Animals/ and Humans/))

Database(s): Embase (via OVID, 1974 to 10 April 2020)

#	Searches
1	exp pneumothorax/
2	(pneumothora* or "pneumo thora*").tw,kw.
3	PTX.tw,kw.
4	1 or 2 or 3
5	radiography/
6	thorax radiography/
7	radiograph*.tw,kw.
8	radiogram*.tw,kw.
9	radiology.tw,kw.
10	roentgen*.tw,kw.
11	"chest film".tw,kw.
12	CXR.tw,kw.
13	x‐ray*.tw,kw.
14	"x ray*".tw,kw.
15	xray.tw,kw.
16	5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15
17	echography/ or focused assessment with sonography for trauma/
18	exp ultrasound/
19	(Ultrasound* or "ultra sound*").tw,kw.
20	sonogra*.tw,kw.
21	(ultrasonogra* or "ultra sonogra*").tw,kw.
22	CUS.tw,kw.
23	ultrasonic.tw,kw.
24	echotomograph*.tw,kw.
25	echograph*.tw,kw.
26	17 or 18 or 19 or 20 or 21 or 22 or 23 or 24 or 25
27	4 and 16 and 26
28	27 not (Animal/ not (Animal/ and Human/))

Database(s): EBM Reviews ‐ Database of Abstracts of Reviews of Effects (via OVID, 1991 to 10 April 2020)

#	Searches
1	(pneumothora* or “pneumo thora*”).tw,kw.
2	PTX.tw,kw.
3	1 or 2
4	radiograph*.tw,kw.
5	roentgen*.tw,kw.
6	radiogram*.tw,kw.
7	radiology.tw,kw.
8	chest film*.tw,kw.
9	CXR.tw,kw.
10	x‐ray*.tw,kw.
11	x ray*.tw,kw.
12	xray*.tw,kw.
13	4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12
14	(Ultrasound* or "ultra sound*").tw,kw.
15	sonogra*.tw,kw.
16	(ultrasonogra* or "ultra sonogra*”).tw,kw.
17	CUS.tw,kw.
18	ultrasonic.tw,kw.
19	echotomograph*.tw,kw.
20	echograph*.tw,kw.
21	14 or 15 or 16 or 17 or 18 or 19 or 20
22	3 and 13 and 21

Database(s): EBM Reviews ‐ Cochrane Central Register of Controlled Trials (via OVID, 1991 to 10 April 2020)

#	Searches
1	Pneumothorax/
2	(pneumothora* or "pneumo thora*").tw,kw.
3	PTX.tw,kw.
4	1 or 2 or 3
5	Radiography/
6	radiography, thoracic/
7	radiograph*.tw,kw.
8	roentgen*.tw,kw.
9	radiogram*.tw,kw.
10	radiology.tw,kw.
11	chest film*.tw,kw.
12	CXR.tw,kw.
13	x‐ray*.tw,kw.
14	x ray*.tw,kw.
15	xray.tw,kw.
16	5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15
17	Ultrasonography/ or focused assessment with sonography for trauma/
18	(Ultrasound* or "ultra sound*").tw,kw.
19	sonogra*.tw,kw.
20	(ultrasonogra* or "ultra sonogra*").tw,kw.
21	CUS.tw,kw.
22	ultrasonic.tw,kw.
23	echotomograph*.tw,kw.
24	echograph*.tw,kw.
25	17 or 18 or 19 or 20 or 21 or 22 or 23 or 24
26	4 and 16 and 25

Database(s): EBM Reviews ‐ Cochrane Database of Systematic Reviews (via OVID, 2005 to 10 April 2020)

#	Searches
1	(pneumothora* or "pneumo thora*").tw,kw.
2	PTX.tw,kw.
3	1 or 2
4	radiograph*.tw,kw.
5	roentgen*.tw,kw.
6	radiogram*.tw,kw.
7	radiology.tw,kw.
8	chest film*.tw,kw.
9	CXR.tw,kw.
10	x‐ray*.tw,kw.
11	x ray*.tw,kw.
12	xray.tw,kw.
13	4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12
14	(Ultrasound* or "ultra sound*").tw,kw.
15	sonogra*.tw,kw.
16	(ultrasonogra* or "ultra sonogra*").tw,kw.
17	CUS.tw,kw.
18	ultrasonic.tw,kw.
19	echotomograph*.tw,kw.
20	echograph*.tw,kw.
21	14 or 15 or 16 or 17 or 18 or 19 or 20
22	3 and 13 and 21

Web of Science Core Collection (1900 to 10 April 2020)

Set	Searches
# 4	#3 AND #2 AND #1 Indexes=SCI‐EXPANDED, SSCI, A&HCI, CPCI‐S, CPCI‐SSH, ESCI Timespan=All years
# 3	TOPIC: (ultrasound* or "ultra sound*" or sonogra* or ultrasonogra* or "ultra sonogra*" or CUS or ultrasonic or echotomograph* or echograph*) OR TITLE: (ultrasound* or "ultra sound*" or sonogra* or ultrasonogra* or "ultra sonogra*" or CUS or ultrasonic or echotomograph* or echograph*)
# 2	TOPIC: (radiograph* or roentgen* or radiogram* or radiology or "chest film*" or CXR or "x‐ray*" or "x ray*" or xray*) OR TITLE: (radiograph* or roentgen* or radiogram* or radiology or "chest film*" or CXR or "x‐ray*" or "x ray*" or xray*)
# 1	TOPIC: (pneumothora* or "pneumo thora*") OR TITLE: (pneumothora* or "pneumo thora*") OR TOPIC: (PTX) OR TITLE: (PTX)

CINAHL Plus (via EBSCO, 1937 to April 10, 2020)

#	Searches
S1	TI ( pneumothora* or "pneumo thora*" or PTX ) OR AB ( pneumothora* or "pneumo thora*" or PTX ) OR SU ( pneumothora* or "pneumo thora*" or PTX )
S2	(MH "Pneumothorax")
S3	S1 OR S2
S4	(MH "Radiography, Thoracic")
S5	(MH "Radiography")
S6	TI ( radiograph* or roentgen* or radiogram* or radiology or "chest film*" or CXR or "x‐ray*" or "x ray*" or xray* ) OR AB ( radiograph* or roentgen* or radiogram* or radiology or "chest film*" or CXR or "x‐ray*" or "x ray*" or xray* ) OR SU ( radiograph* or roentgen* or radiogram* or radiology or "chest film*" or CXR or "x‐ray*" or "x ray*" or xray* )
S7	S4 OR S5 OR S6
S8	(MH "Ultrasonography")
S9	TI (ultrasound* or "ultra sound*" or sonogra* or ultrasonogra* or "ultra sonogra*" or CUS or ultrasonic or echotomograph* or echograph* ) OR AB (ultrasound* or "ultra sound*" or sonogra* or ultrasonogra* or "ultra sonogra*" or CUS or ultrasonic or echotomograph* or echograph* ) OR SU (ultrasound* or "ultra sound*" or sonogra* or ultrasonogra* or "ultra sonogra*" or CUS or ultrasonic or echotomograph* or echograph*)
S10	S8 OR S9
S11	S3 AND S7 AND S10

ClinicalTrials.gov (to April 10, 2020)

Field searched	Terms
Condition or disease:	Pneumothorax
Other terms:	ultrasound or ultrasonography

Appendix 2. Data extraction form

General characteristics
Title:
Authors:
Journal:
Institution/Country where study was performed:
Study period:
Study design:
Sample size:
Sampling used:
Units of analysis (per patient or per lung field):
Type of US probe (curvilinear, high frequency linear, etc.):
Specialty of CUS operator:
Definition of test positivity by CUS:
Definition of test positivity by CXR:
Definition of test positivity by CT or tube thoracostomy:

Population characteristics
Age (years):
Male gender (%):
Blunt trauma (%):
Inclusion criteria:
Exclusion criteria:

Diagnostic accuracy data for CUS
	CT/Thoracostomy positive	CT/Thoracostomy negative	Total
CUS positive
CUS negative
Total
Sensitivity (%):
Specificity (%):
Absence of lung sliding
Sensitivity (%):
Specificity (%):
Absence of B‐lines or comet‐tail artefact
Sensitivity (%):
Specificity (%):
Presence of lung point
Sensitivity (%):
Specificity (%):
Absence of lung pulse
Sensitivity (%):
Specificity (%):

Diagnostic accuracy data for CXR
	CT/Thoracostomy positive	CT/Thoracostomy negative	Total
CXR positive
CXR negative
Total
Sensitivity (%):
Specificity (%):

Time to diagnostic imaging
Time to CUS (min)
Time to CXR (min)
Time to CT/Thoracostomy (min)

Notes:

CUS = Chest ultrasonography

CXR = Chest X‐ray

CT = Computed tomography

Appendix 3. QUADAS‐2 tool for assessing methodological quality of included studies

Domain	Signalling question	Signalling question	Signalling question	Risk of bias	Concerns about applicability
Domain 1: Patient selection
Patient selection	Was a consecutive or random sample of patients enrolled?	Did the study avoid inappropriate exclusions?		Could the selection of patients have introduced bias?	Is there concern that the included patients do not match the review question?
	Yes: if all consecutive or random samples of trauma patients were enrolled No: if convenience or selected samples of trauma patients were enrolled Unclear: if this was not clear from the report	Yes: if the study avoided inappropriate exclusions No: if patients were excluded inappropriately (e.g. age, gender, ethnicity) Unclear: if this was not clear from the report		Low: if "Yes" for all signalling questions High: if "No" was reported for at least 1 signalling question Unclear: if "Unclear" was reported for at least 1 signalling question	Low: if the included population consists of trauma patients in the emergency department setting, irrespective of age and gender, and if inappropriate exclusions were avoided High: if study authors used inappropriate exclusions Unclear: if insufficient information was available to make a judgement
Domain 2: Index tests
Index test ‐ CUS	Were index test results interpreted without knowledge of the results of CT or tube thoracostomy?	Were index test results interpreted without knowledge of the results of the other index test?	Did the authors prespecify the criteria for a positive CUS finding?	Could the conduct or interpretation of the index test have introduced bias?	Is there concern that the index test, its conduct, or interpretation differ from the review question?
	Yes: if CUS results were interpreted without knowledge of the results of CT No: if CUS results were interpreted with knowledge of the results of CT Unclear: if this was not clear from the report	Yes: if index test results were interpreted without knowledge of the results of the other index test No: if index test results were interpreted with knowledge of the results of the other index test Unclear: if this was not clear from the report	Yes: if criteria for positive CUS findings were prespecified No: if the criteria for positive CUS findings were not prespecified Unclear: if this was not clear from the report	Low: if "Yes" for all signalling questions High: if "No" was reported for at least 1 signalling question Unclear: if "Unclear" was reported for at least 1 signalling question	Low: if CUS was performed by frontline physicians in the emergency department High: if CUS was performed by non‐frontline physicians outside of the emergency department setting Unclear: if insufficient information was available to make a judgement
Index test ‐ CXR	Were index test results interpreted without knowledge of the results of CT or tube thoracostomy?	Were index test results interpreted without knowledge of the results of the other index test?		Could the conduct or interpretation of the index test have introduced bias?	Is there concern that the index test, its conduct, or its interpretation differ from the review question?
	Yes: if CXR results were interpreted without knowledge of the results of CT No: if CXR results were interpreted with knowledge of the results of CT Unclear: if this was not clear from the report	Yes: if index test results were interpreted without knowledge of the results of the other index test No: if index test results were interpreted with knowledge of the results of the other index test Unclear: if this was not clear from the report		Low: if "Yes" for all signalling questions High: if "No" was reported for at least 1 signalling question Unclear: if "Unclear" was reported for at least 1 signalling question	Low: if CXR was performed in the supine fashion in the emergency department High: if CXR was not performed in the supine fashion or outside of the emergency department setting Unclear: if insufficient information was available to make a judgement
Domain 3: Reference standard
Reference standard ‐ CT or tube thoracostomy	Is the reference standard likely to correctly classify the target condition?	Were the reference standard results interpreted without knowledge of the results of CUS?	Were the reference standard results interpreted without knowledge of the results of CXR?	Could the reference standard, its conduct, or its interpretation have introduced bias?	Is there concern that the target condition as defined by the reference standard does not match the review question?
	Yes: if an acceptable reference standard, such as CT or tube thoracostomy findings, was used No: if trauma patients did not undergo an acceptable reference standard Unclear: if this was not clear from the report	Yes: If CT results were interpreted without knowledge of the results of CUS (Note: Tube thoracostomy after CUS and CXR but before CT suggests clinical deterioration and was required for patient safety) No: If CT or tube thoracostomy results were interpreted with knowledge of results of CUS Unclear: If this was not clear from the report	Yes: If CT results were interpreted without knowledge of the results of CXR (Note: Tube thoracostomy after CUS and CXR but before CT suggests clinical deterioration and was required for patient safety) No: If CT or tube thoracostomy results were interpreted with knowledge of results of CXR Unclear: If this was not clear from the report	Low: if "Yes" for all signalling questions High: if "No" was reported for at least 1 signalling question. Unclear: if "Unclear" was reported for at least 1 signalling question	Low: If an acceptable reference standard, such as CT or tube thoracostomy findings, was used, and if CT results were interpreted by a radiologist High: If an acceptable reference standard, such as CT or tube thoracostomy findings, was not used, or if CT results were not interpreted by a radiologist Unclear: If insufficient information was available to make a judgement
Domain 4: Flow and timing
Flow and timing	Was there an appropriate interval between CUS, CXR, and CT/tube thoracostomy?	Did all patients receive a reference standard?	Were all patients included in the analysis?	Could patient flow have introduced bias?
	Yes: if CUS, CXR, and CT/tube thoracostomy was sequentially performed within 2 hours No: if CUS, CXR, and CT/tube thoracostomy was not sequentially performed within 2 hours Unclear: if this was not clear from the report	Yes: if all patients received a CT scan or tube thoracostomy No: if some patients did not receive a CT scan or tube thoracostomy Unclear: if this was not clear from the report	Yes: if all patients were included in the final analysis No: if all patients were not included in the final analysis Unclear: if this was not clear from the report	Low: if "Yes" for all signalling questions High: if "No" was reported for at least 1 signalling question Unclear: if "Unclear" was reported for at least 1 signalling question

Notes:

CT = Computed tomography.

CUS = Chest ultrasonography.

CXR = Chest X‐ray.

Data

Presented below are all the data for all of the tests entered into the review.

Tests. Data tables by test.

Test	No. of studies	No. of participants
1 CUS	9	1271
2 CXR	9	1271
3 CUS (lung field level analysis)	4	1331
4 CXR (lung field level analysis)	4	1331

1. Test.

CUS

2. Test.

CXR

3. Test.

CUS (lung field level analysis)

4. Test.

CXR (lung field level analysis)

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Abbasi 2013.

Study characteristics
Patient Sampling	Location: Tehran, Iran Period of data collection: June to July 2009 Sampling technique: Convenience sampling
Patient characteristics and setting	Sample size: 146 patients Mean age: 37 ± 14 years old Male gender: 87.6% Blunt trauma: 89% Penetrating trauma: 11% Inclusion criteria: Adult (≥ 16 yo) ED patients with thoracic trauma Exclusion criteria: tension pneumothorax, subcutaneous emphysema, sucking chest wound, haemodynamically instability.
Index tests	CUS Operator: emergency physicians Probe: 7.5 MHz linear probe Positivity definition: absence of lung sliding or absence of comet‐tail artefacts CXR in supine position.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by 2 blinded radiologists or rush of air or bubbling in chest drain after tube thoracostomy Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: 2 ± 1 minutes Time to CXR: 12 ± 2.6 minutes Patients lost to follow‐up or excluded: 153 patients enrolled, 7 lost to follow‐up (no CT data), 146 analysed.
Comparative
Notes	Exclusion of haemodynamically unstable patients is an inappropriate exclusion.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Was there an appropriate interval between CUS, CXR, and CT/TT?	Yes
Could the patient flow have introduced bias?		High risk

Abdulrahman 2015.

Study characteristics
Patient Sampling	Location: Doha, Qatar Period of data collection: July 2011 to January 2013 Sampling technique: consecutive sampling
Patient characteristics and setting	Sample size: 610 lung fields (305 patients) Median age: 34 years old Male gender: 98% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: adult ED patients with blunt chest trauma needing resuscitation and CT chest as per ATLS guidelines Exclusion criteria: chest tube prior to CT chest, penetrating trauma, incomplete/inaccurate data
Index tests	CUS Operator: trauma surgeons Probe: 10 MHz linear probe Positivity definition: absence of pleural glistening (lung slide) or comet‐tail artefacts CXR in supine position interpreted by trauma radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by trauma radiologist or rush of air or bubbling in chest drain after tube thoracostomy Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost to follow‐up or excluded: 12 patients excluded for duplicate entries and incomplete data
Comparative
Notes	Unclear if the same trauma radiologist interpreted both the CXR and CT.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		High risk

Blaivas 2005.

Study characteristics
Patient Sampling	Location: Augusta, GA, USA Period of data collection: September 2003 to May 2004 Sampling technique: Convenience sampling based on researcher availability
Patient characteristics and setting	Sample size: 176 patients Mean age: not reported Male gender: 57% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: blunt trauma ED patients >17yo with FAST, CXR, and CT or TT Exclusion criteria: any exam unable to be completed.
Index tests	CUS Operator: emergency physicians Probe: 4 to 2 MHz microconvex curvilinear probe Positivity definition: absence of lung slide in 4 locations of each hemithorax CXR in supine position interpreted by trauma surgeon.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists or rush of air or bubbling in chest drain after tube thoracostomy Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Hyacinthe 2012.

Study characteristics
Patient Sampling	Location: Grenoble, France Period of data collection: November 2005 to April 2007 Sampling technique: consecutive sampling
Patient characteristics and setting	Sample size: 237 lung fields (119 patients) Mean age: 39 years old Male gender: 82% Blunt trauma: 95.8% Penetrating trauma: 4.2% Inclusion criteria: adult ED patients with thoracic CT within 6 hours of trauma and needed clinical exam, CXR, and CUS within 90 min before CT scan Exclusion criteria: not prespecified in Methods section; in Results section, they excluded 18 patients for CTs not reviewed by radiology, no indication for CT scan, and CUS after CT
Index tests	CUS Operator: resuscitative anaesthesiologist Probe: 5 to 2 MHz curvilinear probe Positivity definition: Absence of lung sliding or presence of lung point CXR in supine position interpreted by physician‐in‐charge of the trauma resuscitation.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists or rush of air or bubbling in chest drain after tube thoracostomy Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Mean time to CT/TT was 85 minutes (65 to 105). Patients lost‐to‐follow‐up or excluded: 137 patients screened, 18 excluded. All 119 enrolled patients were included in the analysis
Comparative
Notes	No predefined exclusion criteria in the Methods section but exclusions were noted in the screening process as reported in the Results section.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Unclear
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Unclear
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Yes
Could the patient flow have introduced bias?		Low risk

Karimi 2013.

Study characteristics
Patient Sampling	Location: Tehran, Iran Period of data collection: July 1990 to December 1991 Sampling technique: unclear sampling technique
Patient characteristics and setting	Sample size: 140 patients Mean age: 39.4 ± 15.8 years old Male gender: 62% Blunt trauma: unclear Penetrating trauma: unclear Inclusion criteria: adult trauma ED patients who underwent surgery in 2 hours after admission with suspicion of pneumothorax on ultrasound, radiography, and CT scan Exclusion criteria: known pneumothorax or already had chest tube
Index tests	CUS Operator: emergency physicians Probe: 5.5 MHz linear and a 3.5 MHz curvilinear probe Positivity definition: absence of lung sliding, absence of comet tails, or presence of lung point CXR in supine position.
Target condition and reference standard(s)	Reference standard: CT of the chest; did not define who interpreted the CT scan Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: 1.5 to 5 minutes Time to CXR: 5 to 20 minutes Time to CT: 25 to 45 minutes Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes	The study did not describe who was interpreting the CXR or CT results.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Unclear
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Yes
Could the patient flow have introduced bias?		Low risk

Kirkpatrick 2004.

Study characteristics
Patient Sampling	Location: Vancouver, BC, Canada Period of data collection: July 2000 to October 2002 Sampling technique: unclear sampling technique
Patient characteristics and setting	Sample size: 266 lung fields (134 patients) Mean age: 37 years old Male gender: 74% Blunt trauma: 92% Penetrating trauma: 8% Inclusion criteria: adult trauma ED patients who did not require immediate invasive interventions (TT or needle decompression) Exclusion criteria: chest tube already in situ, gross subcutaneous emphysema that obscured acoustic window
Index tests	CUS Operator: trauma surgeons Probe: 10 to 5 MHz linear probe Positivity definition: absence of lung sliding or absence of comet‐tail artefacts CXR in supine position; unclear who interpreted CXR.
Target condition and reference standard(s)	Reference standard: CT of the chest; unclear who interpreted the CT Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost‐to‐follow‐up or excluded: 411 lung fields from 225 patients were enrolled. Only 266 lung fields from 134 patients who received CUS, CXR, and CT were included in the secondary analysis.
Comparative
Notes	Only the data from the secondary analysis of patients that received CUS, CXR, and CT were included.
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Unclear
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Unclear
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	No
Were all patients included in the analysis?	No
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		High risk

Nagarsheth 2011.

Study characteristics
Patient Sampling	Location: Knoxville, TN, USA Period of data collection: January 2008 to December 2009 Sampling technique: convenience sampling based on researcher availability
Patient characteristics and setting	Sample size: 125 patients Mean age: 44.5 years old Male gender: 66.4% Blunt trauma: 92.8% Penetrating trauma: 7.2% Inclusion criteria: adult trauma ED patients who received CUS, CXR, and CT Exclusion criteria: already had chest tube, needle decompressed or subcutaneous emphysema of chest/neck
Index tests	CUS Operator: trauma surgeons Probe: 2.5 MHz curvilinear and a 10.5 MHz linear probe Positivity definition: absence of lung sliding or absence of comet‐tail artefacts CXR in supine position interpreted by a blinded radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost to follow‐up or excluded: 125 patients were recruited but only 79 were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	No
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	No
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		High risk

Ojaghi Haghighi 2014.

Study characteristics
Patient Sampling	Location: Tabriz, Iran Period of data collection: "the winter of 2013" Sampling technique: unclear if truly consecutive sampling as they (quote) "enrolled all patients with severe multiple trauma"
Patient characteristics and setting	Sample size: 150 patients Mean age: unclear Male gender: 82.66% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: ED patients with severe multiple trauma including car rollover, ejection from vehicle, frontal impact, compression of chest with steering wheel/dashboard, severe side impact, fall or acceleration‐deceleration injury Exclusion criteria: already have chest tube, lack of access to ultrasound at time of admission. Lack of access to ultrasound is an inappropriate exclusion, especially when this is a US diagnostic study.
Index tests	CUS Operator: emergency physicians Probe: 6.5 to 9 MHz linear probe Positivity definition: Absence of lung sliding or absence of comet‐tail artefacts CXR in supine position interpreted by a radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by radiologists Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost to follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Soldati 2006.

Study characteristics
Patient Sampling	Location: Rome, Italy Period of data collection: September 1999 to August 2004 Sampling technique: consecutive sampling
Patient characteristics and setting	Sample size: 186 Mean age: 52.4 ± 22.9 years old Male gender: 62.9% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: adult ED patients with blunt chest trauma Exclusion criteria: unable to give informed consent, needing immediate interventions, chest wall injuries precluding US evaluation (pain, rib fractures, flail chest), subcutaneous emphysema. The exclusion of "chest wall injuries precluding US evaluation (pain, rib fractures, flail chest)" is vague and inappropriate.
Index tests	CUS Operator: emergency physicians Probe: 5 MHz curvilinear probe Positivity definition: absence of lung slide, absence of comet‐tail artefacts; lung point and ultrasonographic deep sulcus sign was only for size of pneumothorax CXR in supine position interpreted by a blinded radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: < 30 minutes Time to CXR: unclear Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Soldati 2008.

Study characteristics
Patient Sampling	Location: Rome, Italy Period of data collection: June 2005 to November 2006 Sampling technique: consecutive sampling
Patient characteristics and setting	Sample size: 218 lung fields (109 patients) Mean age: 41.4 ± 20.5 years old Male gender: 62.9% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: adult ED patients with blunt chest/multiple trauma Exclusion criteria: need for immediate chest decompression for tension pneumothorax, haemodynamic instability, need for mechanical ventilation for level of consciousness or severe respiratory distress, subcutaneous emphysema, chest wall injuries precluding US evaluation, inability to give informed consent. The exclusion of "chest wall injuries precluding US evaluation" and "haemodynamically unstable patients" is inappropriate.
Index tests	CUS Operator: trauma surgeons Probe: 3.5 MHz curvilinear and a 5.2 MHz curvilinear probe Positivity definition: absence of lung slide, absence of comet‐tail artefact, or presence of lung points CXR in supine position interpreted by a radiologist based on absent lung parenchyma or other indirect signs, such as the dishomogeneous appearance of the diaphragm, the incongruency of the pleural line, or the “deep sulcus sign.”
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by either the same or a different radiologist as the CXR, categorized as minuscule, anterior, or anterolateral Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: < 60 minutes Time to CXR: < 60 minutes Time to CT: < 60 minutes Patients lost to follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	No
Were index test results interpreted without knowledge of the results of the other index test?	Unclear
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Unclear
Were the reference standard results interpreted without knowledge of the results of CXR?	No
Could the reference standard, its conduct, or its interpretation have introduced bias?		High risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Yes
Could the patient flow have introduced bias?		Low risk

Uz 2013.

Study characteristics
Patient Sampling	Location: Izmir, Turkey Period of data collection: unclear Sampling technique: unspecified sampling technique
Patient characteristics and setting	Sample size: 107 patients Mean age: 36.7 ± 19.8 years old Male gender: unclear Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: adult ED patients with multiple trauma Exclusion criteria: unable to give consent, could not receive CT (technical problems, patient instability, no indication). Exclusion of patient instability and "no indication for CT" is inappropriate.
Index tests	CUS Operator: emergency physicians Probe: 5 to 10 MHz linear probe and a 2 to 5 MHz curvilinear probe Positivity definition: absence of lung slide, absence of comet‐tail artefacts or stratosphere sign on M‐mode (extension of lung slide) CXR in supine position interpreted by an emergency physician.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists unaware of the results of CUS and CXR Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	No
Could the selection of patients have introduced bias?		High risk
Are there concerns that the included patients and setting do not match the review question?			High
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	No
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	No
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Unclear
Could the patient flow have introduced bias?		Unclear risk

Vafaei 2016.

Study characteristics
Patient Sampling	Location: Tehran, Iran Period of data collection: December 2013 to December 2014 Sampling technique: consecutive sampling
Patient characteristics and setting	Sample size: 152 patients Mean age: 31.4 ± 13.8 years old Male gender: 77.6% Blunt trauma: 85.5% Penetrating trauma: 14.5% Inclusion criteria: trauma ED patients of all ages with traumatic intrathoracic injuries Exclusion criteria: pregnancy, haemodynamic instability, lack of interest in participating in study. Despite the exclusion criteria stating haemodynamic instability, the study included 27 of those patients.
Index tests	CUS Operator: emergency physicians Probe: 3.5 to 7 MHz linear and curvilinear probe Positivity definition: unclear CXR in supine position interpreted by an independent blinded radiologist separate from the CT interpreting radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists separate from the CXR interpreting radiologist Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: unclear Time to CXR: unclear Time to CT: 60 to 120 minutes Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Yes
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Low risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	No
Could the conduct or interpretation of the index test have introduced bias?		High risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Yes
Could the reference standard, its conduct, or its interpretation have introduced bias?		Low risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	Yes
Could the patient flow have introduced bias?		Low risk

Zhang 2006.

Study characteristics
Patient Sampling	Location: Hangzhou, China Period of data collection: September 2004 to October 2005 Sampling technique: undefined sampling technique
Patient characteristics and setting	Sample size: 135 patients Mean age: 45 ± 15 years old Male gender: 84.4% Blunt trauma: 100% Penetrating trauma: 0% Inclusion criteria: adult ED patients with multiple trauma Exclusion criteria: subcutaneous emphysema, cardiac arrest from probable tension pneumothorax
Index tests	CUS Operator: emergency physicians Probe: 3.5 MHz curvilinear and a 7.5 MHz linear probe Positivity definition: absence of lung slide or absence of comet‐tail artefacts CXR in supine position interpreted by a blinded radiologist.
Target condition and reference standard(s)	Reference standard: CT of the chest interpreted by blinded radiologists or rush of air or bubbling in chest drain after tube thoracostomy Target condition: traumatic pneumothorax
Flow and timing	Time to CUS: 2.3 ± 2.9 minutes Time to CXR: 19.9 ± 10.3 minutes Time to CT/TT including transportation of the patient, performing the scan, and receiving an oral interpretation report was 16.3 ± 7.8 minutes. This did not include time from patient assessment to transportation of patient. However, the Methods section quotes the CT scans were performed within 180 minutes. This was deemed excessive. Patients lost‐to‐follow‐up or excluded: all enrolled patients were included in the analysis
Comparative
Notes
Methodological quality
Item	Authors' judgement	Risk of bias	Applicability concerns
DOMAIN 1: Patient Selection
Was a consecutive or random sample of patients enrolled?	Unclear
Did the study avoid inappropriate exclusions?	Yes
Could the selection of patients have introduced bias?		Unclear risk
Are there concerns that the included patients and setting do not match the review question?			Low concern
DOMAIN 2: Index Test (CUS)
Were index test results interpreted without knowledge of the results of CT or TT?	Yes
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?	Yes
Could the conduct or interpretation of the index test have introduced bias?		Low risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 2: Index Test (CXR)
Were index test results interpreted without knowledge of the results of CT or TT?	Unclear
Were index test results interpreted without knowledge of the results of the other index test?	Yes
Did the authors prespecify the criteria for a positive CUS finding?
Could the conduct or interpretation of the index test have introduced bias?		Unclear risk
Are there concerns that the index test, its conduct, or interpretation differ from the review question?			Low concern
DOMAIN 3: Reference Standard
Is the reference standards likely to correctly classify the target condition?	Yes
Were the reference standard results interpreted without knowledge of the results of CUS?	Yes
Were the reference standard results interpreted without knowledge of the results of CXR?	Unclear
Could the reference standard, its conduct, or its interpretation have introduced bias?		Unclear risk
Are there concerns that the target condition as defined by the reference standard does not match the question?			Low concern
DOMAIN 4: Flow and Timing
Did all patients receive the same reference standard?	Yes
Were all patients included in the analysis?	Yes
Was there an appropriate interval between CUS, CXR, and CT/TT?	No
Could the patient flow have introduced bias?		High risk

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Abdalla 2016	Wrong patient population
Agarwal 2017	CUS not performed by frontline non‐radiologist physicians
Donmez 2012	CUS not performed by frontline non‐radiologist physicians
Ezzat 2018	Wrong study design
Heydari 2014	Missing CUS/CXR/CT/TT data
Kaya 2015	Missing CUS/CXR/CT/TT data
Ku 2013	Missing CUS/CXR/CT/TT data
Mumtaz 2016	CUS not performed by frontline non‐radiologist physicians
Nandipati 2011	Missing CUS/CXR/CT/TT data
Subramaniam 2017	CUS not performed by frontline non‐radiologist physicians
Ziapour 2015	Missing CUS/CXR/CT/TT data

Differences between protocol and review

Due to the lack of robust data and significant heterogeneity in method of reporting, we were unable to synthesize and compare data for individual CUS findings and for time to CUS, CXR, and CT.

Contributions of authors

Kenneth K Chan (KC), Daniel A Joo (DJ), Andrew D McRae (AM), Yemisi Takwoingi (YT), Zahra Premji (ZP), Eddy Lang (EL), Abel Wakai (AW).

Conceiving the review: KC.

Co‐ordinating the review: KC.

Undertaking manual searches: KC, ZP.

Screening search results: KC, DJ, AM.

Organizing retrieval of papers: KC, ZP.

Screening retrieved papers against inclusion criteria: KC, DJ, AM.

Appraising quality of papers: KC, DJ.

Abstracting data from papers: KC, DJ.

Writing to authors of papers for additional information: KC.

Providing additional data about papers: KC.

Obtaining and screening data on unpublished studies: KC, DJ.

Managing data for the review: KC.

Entering data into RevMan 5: KC.

Interpreting RevMan 5 statistical data: KC, YT.

Performing other statistical analysis not using RevMan 5: YT.

Interpreting data: KC, YT, DJ, AM.

Making statistical inferences: KC, YT, DJ, AM.

Writing the review: KC, DJ, AM, YT, ZP, EL, AW.

Securing funding for the review: N/A.

Performing previous work that was the foundation of the present study: KC.

Serving as guarantor for the review (one review author): KC.

Taking responsibility for reading and checking the review before submission: KC.

Sources of support

Internal sources

• University of Calgary Department of Emergency Medicine, Canada

  No financial support was provided for the completion of this review.

External sources

• No sources of support supplied

Declarations of interest

Kenneth K Chan is an Attending Emergency Physician with the University of Calgary and Alberta Health Services. He is also an Attending Emergency Physician with the University of British Columbia and Fraser Health Authority. He has no conflicts of interest to declare.

Daniel A Joo is an Attending Emergency Physician with the University of Calgary and Alberta Health Services. He has no conflicts of interest to declare.

Andrew D McRae is an Assistant Professor of Emergency Medicine and Community Health Sciences at the University of Calgary. His institution has been awarded research grants and consulting fees on cardiac biomarkers from Roche Diagnostics Canada.

Yemisi Takwoingi is a Professor of Test Evaluation and Evidence Synthesis, University of Birmingham, UK. She has no conflicts of interest to declare.

Zahra A Premji is a Research and Learning Librarian with the University of Calgary. She has no conflicts of interest to declare.

Eddy Lang is the Department Head of Emergency Medicine and a Clinician Scientist with the University of Calgary and Alberta Health Services. He has no conflicts of interest to declare.

Abel Wakai is the Director of Emergency Care Research Unit, Division of Population Health Sciences, Royal College of Surgeons, in Ireland. He has no conflicts of interest to declare.

Edited (no change to conclusions)