Accuracy of Resident Performed Point-of-Care Lung Ultrasound versus Chest Radiography in Pneumothorax Follow-up After Tube Thoracostomy in Rwanda

Abstract

Objectives

The aim of this study was to evaluate the accuracy and timeliness of resident-performed point-of-care lung ultrasound for the follow-up of pneumothorax after tube thoracostomy.

Methods

After a brief training, Rwandan surgical residents blinded to CXR performed and interpreted lung ultrasound (LUS) for pneumothorax (PTX) in participants undergoing chest x-ray (CXR) for PTX follow-up. Treating clinicians interpreted CXRs for the presence of PTX for therapeutic decisions. LUS were later reviewed by ultrasound experts and CXRs were reviewed by a radiologist. We defined expert LUS interpretation as the gold standard. The sensitivity and specificity of resident-performed LUS for diagnosing PTX were calculated. We assessed agreement between trained resident vs. expert LUS and clinician vs. radiology CXR using Cohen’s Kappa. We compared time to results between LUS and CXR.

Results

Over an eight-month period, 51 participants were enrolled. When compared to expert LUS interpretation, the sensitivity and specificity (95% confidence intervals) of resident LUS was 100% (85-100%) and 96% (82-100%), while the sensitivity and specificity of clinician-interpreted CXR was 48% (27-69%) and 100% (88-100%). The agreement between resident and expert LUS was excellent (Kappa=0.96) while the agreement between clinician and radiologist CXR was only moderate (Kappa=0.60). Time to results was significantly longer for CXR than for LUS (mean 1335min vs. 396min, p=0.0001).

Conclusion

Resident-performed LUS was a quicker imaging modality with superior sensitivity compared to clinician-interpreted CXR for PTX follow-up after tube thoracostomy in this Rwandan study. LUS can be a valuable imaging tool for PTX follow-up, especially in resource-limited settings.

Introduction

Pneumothorax (PTX) is a diagnosis that carries significant morbidity and mortality, and accurate recognition and management to resolution of this condition is essential.¹ The World Health Organization recognizes injuries as a neglected epidemic in low and middle-income countries, where more than 90% of injury deaths occur, with health-care systems least prepared to manage the trauma burden.² In these resource-limiting settings, the follow-up of PTX after chest tube placement can be particularly time and resource intensive, as well as diagnostically challenging. Despite being the most common imaging modality used in developing countries to evaluate PTX, chest x-ray (CXR) has relatively low sensitivity for this diagnosis, with meta-analyses showing sensitivities of 40-52% and specificities of 99-100% when compared to chest CT.^3–11 In addition to its limited sensitivity for the detection of PTX, the frequent lack of portable x-ray makes this modality challenging for PTX management in resource-limited settings. Chest CT, despite being considered the gold standard diagnostic test for PTX, is often unavailable in many hospitals in low- and middle-income countries due to lack of CT or patient cost of the study. In addition, CXR and CT require patient transport from a resuscitation area to the radiology department, which is often impractical for this patient population.

Lung ultrasound (LUS), as opposed to CXR, offers 79-91% sensitivity and 98-99% specificity for diagnosing PTX. ^{3–8,10–14} LUS is portable, relatively affordable, non-invasive, and does not carry exposure risks to ionizing radiation. It can be performed at the bedside, provides dynamic information, and can also be used for procedural guidance.^5,16,18

The superior performance of LUS relative to CXR for detecting PTX is based on studies performed in Europe and the United States. However, no studies have evaluated the accuracy of resident-performed LUS for PTX in a developing setting with more constrained hospital resources. Despite the growing availability of point-of-care ultrasound worldwide, CXR remains the standard imaging tool for the diagnosis and follow-up of PTX in our study setting and many others. Therefore, we sought to assess the accuracy of resident-performed point-of-care LUS for PTX follow-up after tube thoracostomy compared to expert ultrasound interpretation, as well as whether performing LUS instead of CXR would improve the time-to-diagnosis of persistent PTX at a Rwandan referral hospital.

Materials and Methods

This was a prospective, observational study conducted from June 2016 to January 2017 at the University Teaching Hospital of Kigali (CHUK) in Rwanda. Ethical approval was obtained from the Institutional Review Board of the College of Medicine and Health Sciences of the University of Rwanda. CHUK is a national referral hospital, the largest among four tertiary level hospitals in the country. It has a capacity of 513 beds, with a busy surgical department of 170 beds, 6 operating theatres, and an emergency department that receives the majority of surgical emergencies in the country.

Patients 5 years and older with PTX managed by chest tubes at CHUK were eligible for inclusion in this study. A signed informed consent (and assent if below 21 years of age) was required before enrollment. Exclusion criteria included chest tubes inserted post-thoracotomy or for chronic diseases such as malignancy, heart disease, liver disease or renal disease (due to expected longer drainage and recurrences), chest tubes for recurrent PTXs and chest tubes inserted prior to arrival at CHUK. Patients for whom both LUS and CXR were unable to be completed, in whom the interval between LUS and CXR was more than 72 hours, or for whom there was absence of visualization of pleural line on LUS were excluded from the analysis. Patients were enrolled after chest tube insertion for PTX and clinical status was followed up until day 7 post tube removal if in hospital, and if discharged they were called at day 7 post tube removal to determine if symptoms resolved, if there were ongoing symptoms, or if there was need for readmission. Chest tube drainage systems (underwater seal or Heimlich valve) were chosen at the discretion of the treating clinician.

Fifteen junior surgical residents (postgraduate year 1-2) received 4 hours of training focused on the use of LUS for the detection of PTX. A pre- and post-test were administered and all passed the post-test successfully. Point-of-care LUS was performed using an M-Turbo® (FUJIFILM SonoSite, Bothell, WA) portable ultrasound machine. The diagnosis of PTX was established using a combination of previously described ultrasound findings including abolition of lung sliding associated with the absence of B-lines, absence of lung pulse, or the presence of lung point.^17–19

Extra-luminal air present in PTX produces a loss of visible sliding between the visceral and parietal pleura. PTX also creates the loss of pleural reverberation artifacts (B-lines or comet-tails). Lung point demonstrates the border of a PTX as one area of visceral and parietal pleural are in contact while the other is not (Figure 1). Lung pulse represents cardiac vibrations visible at the pleural line, which can happen in the setting of complete atelectasis with ongoing contact between the visceral and parietal pleura; it is absent in well-aerated lung.¹⁵ In this study, eight lung zones were assessed for these findings including the following intercostal spaces bilaterally: parasternal 2^nd, midclavicular 4^th, anterior axillary 6^th, posterior axillary 8^th interspace.

Figure 1.

Presence of lung sliding is demonstrated on m-mode imaging by the “seashore” sign (A). Absence of lung sliding indicating pneumothorax is demonstrated on m-mode by the “barcode” sign (B). The presence of comet-tails at the pleural line indicates the absence of pneumothorax (C). Lung point indicates the edge of a pneumothorax (D).

The trained residents remained blinded to CXR results and performed LUS once on each subject after treating clinicians ordered a CXR for the follow-up of a PTX. These LUS images and videos were later reviewed in a blinded fashion by emergency physicians who had completed fellowship training in ultrasonography. Copies of all corresponding CXR films were stored for later interpretation by a radiologist, who was blinded to the clinicians’ CXR interpretation and all LUS interpretations. Therapeutic decisions were based on treating clinicians’ CXRs interpretations. Clinician CXR is used for clinical decision-making in this resource-limited clinical setting due to a lack of qualified radiologists to review each x-ray film in real-time.

Considering the extensive literature describing the superior sensitivity of LUS for the detection of PTX compared to CXR and the lack of CT scan availability, ultrasound expert LUS interpretation was taken as the diagnostic gold standard in this study. The diagnostic sensitivity and specificity of resident LUS and treating clinician CXR for detecting PTX was calculated relative to ultrasound expert interpretation. The agreement between resident-interpreted LUS and expert-interpreted LUS was calculated using Cohen’s Kappa coefficient, as was the agreement between clinician-interpreted CXR and radiologist-interpreted CXR. A paired t-test was used to compare the time to obtain results for LUS vs. CXR. Data was analyzed using Stata 14.2 (Stata Corporation, College Station, TX).

Results

Over an eight-month period from June 2016 to January 2017, 62 patients with PTX and thoracostomy management were admitted at CHUK. Among these patients, 58 met the inclusion criteria and consented for enrollment. From this total, one patient was excluded from analysis because the cause of their hydro-pneumothorax was found to be due to malignancy, three patients were excluded due to failure to complete both LUS and CXR, two patients were excluded because of subcutaneous emphysema that impaired pleural line visualization on LUS, and one patient was excluded due to a long interval (>72 hrs) between LUS and CXR. The final analysis included 51 patients with PTX managed by tube thoracostomy.

Participants’ ages varied from 6 to 62 years old with a median of 31 years. The majority (63%) were between 20-40 years of age. 41 (80%) participants were male and 10 (20%) were female. The majority (39, 76%) had PTX due to trauma, 32 of these (82%) were the result of blunt chest trauma, and 7 (18%) were due to penetrating chest injuries (Table 1). The most commonly used drainage system was the Heimlich one-way valve in 42 patients (82%) while the industrial underwater seal was used in 9 patients (18%). The drainage duration ranged between 1 to 45 days with a mean of 8.9 days.

Table 1.

Summary of Patient and Pneumothorax Characteristics

Variable	n (%)
Age group (years)
<20	4 (8%)
20-40	32 (63%)
40-60	14 (27%)
>60	1 (2%)
Gender
Male	41 (80%)
Female	10 (20%)
Pneumothorax type
Traumatic	39 (76%)
Penetrating	7 (14%)
Blunt	32 (63%)
Iatrogenic	6 (12%)
Primary Spontaneous	2 (4%)
Secondary Spontaneous	4 (8%)
Type of chest tube drain
One-way valve	42 (82%)
Underwater seal	9 (18%)
Total	51 (100%)

The sensitivity and specificity with 95% confidence intervals (CIs) of resident-interpreted LUS compared to expert-interpreted LUS was 100% (85-100%) and 96% (82-100%), respectively. The treating clinicians’ interpretations of CXR for PTX when compared to expert-interpreted LUS had a sensitivity and specificity of 48% (27-69%) and 100% (88-100%), respectively (Figure 2). The majority of the clinician-interpreted CXRs were initially read by a resident (40 cases, 78%), some by consultant general surgeons (10 cases, 20%), and one (2%) by a general practitioner.

Figure 2.

Sensitivity and specificity (with 95% confidence intervals) of resident lung ultrasound (LUS) and clinician chest x-ray (CXR) interpretations compared to expert LUS interpretation.

LUS interpreted by residents had excellent agreement with LUS interpreted by ultrasound experts with a Cohen’s Kappa of 0.96 (95% CI, 0.88-1.00). There was only moderate agreement between clinician and radiologist-interpreted CXR with a Cohen’s Kappa of 0.60 (95% CI, 0.38-0.82).

The time to perform the study and obtain results ranged between 40 to 1500 minutes (mean of 396 minutes or 6.6 hours) for LUS, and between 120 to 4300 minutes (mean of 1334 minutes or 22 hours) for CXR. LUS was significantly faster to obtain than CXR (p<0.001).

Four (8%) of the discharged patients had symptom resolution at the 7-day follow-up call. Twenty (39%) patients discharged had some ongoing symptoms on follow-up, the most common being moderate chest pain at tube insertion site, which were not severe enough to cause them to return for re-evaluation. There were seven (14%) in-hospital deaths, none due to PTX, but rather from associated trauma such as head injuries. Sixteen patients (31%) were still admitted at 7 days post tube removal, again due to associated injuries. Four patients (8%) re-presented after discharge for further evaluation, all of them due to a residual/recurrent PTX. Of the four patients who re-presented with residual/recurrent PTX, three had previously had a resident LUS interpreted as positive for the presence of PTX at discharge; notably, these patients had been discharged based on the clinician interpretation of their CXRs as negative for PTX. Of the three readmitted patients with positive LUS prior to discharge, only one had a positive CXR by radiologist post-hoc interpretation.

Discussion

Point-of-care lung ultrasonography performed by surgical residents in a Rwandan medical center had a higher sensitivity (100%) and comparable specificity (96%) when compared to clinicians’ CXR interpretation (sensitivity 48%, specificity 100%). Notably, this degree of inaccuracy in clinician-interpreted CXR suggests that relying on CXR would miss over half of pneumothoraces detectable by LUS. Moreover, only a short period of training was needed to perform LUS with high degree of accuracy in this study as residents identified all cases of PTX that were interpreted as positive by the ultrasound experts, and only misidentified one ultrasound exam that the experts read negative as positive for PTX. Choosing an imaging modality with high sensitivity for detecting residual PTX is especially important in the follow-up of PTX after tube thoracostomy, when the decision on when a chest tube can be safely removed may depend on the clinician’s bedside interpretation.

In addition, we demonstrated that LUS interpretation may be more consistent across providers than CXR. Clinician CXR interpretation for the presence of PTX only had moderate agreement (kappa=0.60) with radiologist CXR interpretation. While all CXRs read as positive by the treating clinician were also interpreted as positive by the radiologist, 9 out of the 20 CXRs that the radiologist interpreted as positive for PTX were read as negative by the clinicians. LUS on the other hand had excellent agreement between residents and ultrasound experts (kappa=0.96).

On average, study participants completed LUS 16 hours faster than CXR. The significantly decreased time required for LUS as compared to CXR may allow treating clinicians to make earlier medical decisions in the treatment and follow-up of PTX. This, in turn, could decrease the length of in-hospital stays as well as associated healthcare costs. LUS did require an average of 6.6 hours to complete, but this may be shortened if LUS had been the clinically accepted imaging modality used at the bedside during clinical decision making rather than used exclusively for research. While our study was focused only on the diagnosis of PTX, prior studies have also demonstrated the utility of LUS for the diagnosis of other chest conditions such as hemothorax, pleural effusions and lung consolidations.^16,19

In the four patients who required hospital re-evaluation, the cause of readmission or re-consultation was found to be a PTX that either failed to completely resolve or which recurred. Interestingly, three out of the four cases that were readmitted were interpreted as being positive for PTX on resident LUS and expert LUS, but all were interpreted as being negative on CXR by the treating clinicians; only one of those four CXRs was interpreted as positive by the radiologist in post-hoc review. In these limited cases, lung ultrasound was more sensitive than CXR for diagnosing pneumothorax. Clinical use of the LUS findings may have prevented the premature discharge of these patients. However, as CXR was the accepted standard of care in this clinical context during the study period, LUS was not used for clinical decision making in this observational study. These results are similar to those reported by Galbois et al. (2010), in whose study all 13 residual PTXs that were detected by LUS but were missed on CXR resulted in additional therapeutic interventions.⁹ Considering the relatively poor accuracy of clinician-interpreted CXR for the diagnosis of PTX in our setting, this study supports the increased use of point-of-care ultrasound by trained clinicians and further concurrent training in chest x-ray interpretation in this context.

This study has several limitations. First, while its results are important to our study setting and similarly resourced institutions, these findings may not be broadly generalizable since the accuracy of clinician-interpreted CXR may vary across institutions. Similarly, the standard-of-care of making treatment decisions based on clinician CXR interpretations, rather than radiologist interpretations, may not be the practice pattern in other settings with more radiology capacity. The residents performing LUS received limited training prior to performing these scans with a protocol that was focused on a set number of lung zones. It is possible that PTX could have been present in a lung region (such as a basilar or posterior zone) that was not scanned by the clinician.²¹ The expert LUS interpretations that we accepted as the gold standard imaging modality in this study were based on images acquired by recently trained resident physicians, while most other studies on LUS involve the same provider acquiring and interpreting images. Another limitation was the average of a 17-hour time difference between the acquisition of LUS and CXR. The location of the PTX or potential change in the status of PTX within that time-frame may have also contributed to the differences in interpretation between LUS and CXR. Lastly, we recognize that using LUS as the standard for interpretation may not be as accurate as the more widely accepted gold standard of chest CT, which was not available in our setting.

In conclusion, Rwandan resident physicians accurately performed and interpreted lung ultrasound for the detection of PTX as compared to the interpretation of an expert ultrasound reviewer. In comparison, clinician-interpreted CXR had poor sensitivity relative to expert-interpreted LUS and only moderate agreement with radiologist-interpreted CXR. Our results suggest that LUS may be a reasonable alternative to CXR for the follow-up of PTX in our study setting, especially given that our times to obtain LUS were significantly shorter than CXR.