Diagnostic performance of lung ultrasound in the diagnosis of pneumonia: a bivariate meta-analysis

Abstract

Background and Objective: Pneumonia is a common disease with both high morbidity and mortality, the diagnosis of pneumonia remains a clinical challenge. Many studies have been conducted to identify the usefulness of lung ultrasound for the diagnosis of pneumonia, but with inconsistent and inconclusive results. The present study aimed to establish the overall diagnostic accuracy of lung ultrasound in diagnosing pneumonia. Methods: Based on a comprehensive search of the Pubmed, Embase, and the Cochrane database, we identified out-come data from all articles estimating diagnostic accuracy with lung ultrasound for pneumonia. Quality was assessed with the Quality Assessment for Diagnostic Accuracy Studies. Results from different studies were pooled using a bivariate meta-analysis. Summary receiver operating characteristic curve was used to assess the overall performance of lung ultrasound-based assays. Results: Nine studies containing 1080 subjects were included in this meta-analysis. The summary estimates for lung ultrasound in the diagnosis of pneumonia in the studies included were as follows: sensitivity, 0.97 (95% CI: 0.93-0.99); specificity, 0.94 (95% CI: 0.85-0.98); DOR, 507.99 (95% CI: 128.11-2014.34); positive likelihood ratio, 15.62 (95% CI: 6.31-38.68); negative likelihood ratio, 0.03 (95% CI: 0.01-0.08); The area under the summary receiver operating characteristic curve was 0.99 (95% CI: 0.98-1.00). Conclusion: Lung ultrasound is a capable of diagnosing pneumonia with high accuracy and is a promising attractive alternative to chest radiography and thoracic CT scan.

Introduction

Pneumonia is a common and potentially life-threatening disease, which is associated with increasing morbidity, mortality, hospitalization rate and health care costs. In the United States in 2009, more than 54,000 deaths in the United States were attributed to pneumonia [1]. Most deaths from community acquired pneumonia (CAP) occur in elderly patients with multiple comorbid conditions [2]. Pneumonia is also held responsible of the reason for death of children [3]. More than $17 billion were paid for the overall cost of care for patients with CAP annually in the United States [4]. Because of both the clinical and the financial burden of CAP, efficient and cost-effective diagnostic options for pneumonia should be considered. Clinical history and physical examination can’t provide certainty in this diagnosis. The chest X-ray (CXR) is still recommended to be the first imaging step for diagnosing pneumonia, nevertheless, we have limited data on the sensitivity and specificity of the test [5]. Although thoracic CT scan is a diagnostic tool with high sensitivity and specificity, it is not always available in all levels hospital and has limitations of high cost and high radiation dose [6]. For children, the critically ill and the pregnant, CT and CXR are not adequate.

In the last decades, ultrasound has been shown to be highly effective in evaluating a range of pathologic conditions [7]. Lung ultrasound (LUS) is increasingly being used as a valuable bedside method in diagnosing pulmonary diseases, by providing a user-friendly, inexpensive, noninvasive, and reliable examination. The availability of portable equipment also has made chest ultrasonography an interesting and alternative method in intensive care unit and emergency department [8]. LUS takes advantage in some pulmonary conditions. Compared to CXR performed on a supine patient, LUS proves to be more sensitive in detecting pneumothorax [9]. LUS is also more efficient than CXR in pleural effusion diagnosis [10]. In addition, research has shown that modified trans-thoracic LUS is useful in the evaluation of pulmonary alveoli-interstitial involvement of rheumatoid lung disease [11]. To date, several studies have investigated the application of LUS in the diagnosis of pneumonia but with inconsistent results. The aim of this study was to evaluate the overall diagnostic accuracy of chest US for pneumonia.

Method

Data source and search strategy

Two investigators independently performed a systematic electronic search of the Pubmed, Embase and Cochrane database until September 1, 2013 to identify potentially relevant articles on the usefulness of LUS in the diagnosis of pneumonia. The following search terms were used: “Ultrasonography or Ultrasound or Sonography”, “pneumonia”, and “sensitivity or specificity”. We reviewed the bibliographies of all selection articles to identify additional relevant studies.

Selection of studies

Two reviewers independently screened titles and abstracts of all studies for relevance. Disagreements were resolved by a third opinion. The strength of the individual studies was weighed for relevance, based on following items: 1. the clinical domains should include patients with suspected pneumonia; 2. the reference diagnostic standards were clearly described and all patients were diagnosed by using the reference standards; 3. completeness of data (numbers of true-positive, false-positive, true-negative, false negative) were reported, to allow reconstruction of the diagnostic 2 by 2 table; 4. the studies were written in English. A study was excluded if it didn’t meet above mentioned criteria.

Data extraction and quality assessment

The final set of articles was assessed independently by two reviewers. The retrieved data included author, publication year, the number of included specimens (true-positive, false-positive, true-negative, false negative), sensitivity and specificity. The methodological quality of included studies was evaluated using the Quality Assessment for Studies of Diagnostic Accuracy (QUADAS) tool [12]. This is an evidence-based approach to quality assessment intended for use in systematic reviews of diagnostic accuracy studies. A quality index is generated, with a maximum value of 14.

Statistical analysis

We used standard methods recommended for meta-analyses of diagnostic test evaluations [13]. By using a bivariate regression approach, we estimated the pooled sensitivity (SEN) and specificity (SPE), positive and negative likelihood ratios (PLR and NLR, respectively), and constructed summary receiver operating characteristic (SROC) curve to summarize the study results [14]. Based on random-effects models, this bivariate approach accounts for potential between-study heterogeneity and incorporates the possible correlation between the SEN and the SPE.

Heterogeneity was assessed through the test of inconsistency (I²) of the pooled diagnostic odds ratios (DORs) [15]. DOR illustrated the odds of a positive test result in patients with pneumonia compared with patients without pneumonia: (TP/FN)/(FP/TN). It indicated the accuracy of a diagnostic test, corresponding to particular pairings of SEN and SPE [16]. Since publication bias is of concern for meta-analyses of diagnostic studies, the publication bias of included studies was inspected by using Deek’s funnel plot [17]. Posttest probability (PTP) was calculated by using the nomograms of Fagan [18]. All the statistical analysis was analyzed by using software Stata 12.0 (Stata Corporation, College Station, TX, USA).

Results

We evaluated these citations and the bibliographies of the potential studies. Consequently, nine unique publications were available for analysis of the diagnostic accuracy of lung ultrasound in pneumonia [19-27]. The main reasons of excluding studies were as follows: the study was a duplicate between the Pubmed and Embase database, the study was not diagnostic, or the study can’t reconstruct the diagnostic 2 by 2 table.

Study characteristics and quality assessment

The final set of nine studies included 1080 subjects, in which 644 cases were with pneumonia and 436 controls. In all involved studies, most used CXR or CT as a reference standard to diagnose pneumonia. We could find filtration, patch or consolidation on radiography, with pleural effusion or not [28]. Some studies adopted the comprehensive result of CT, clinical course, conventional tests and follow-up outcomes as a diagnostic standard, which was considered to be clinical diagnosis. All of the included studies were prospective. The quality of the 7 studies was generally high, satisfying the majority of the criteria. The study characteristics, along with QUADAS scores, were outlined in Table 1.

Table 1.

Summary data from the nine studies included in the meta-analysis

Author (Ref)	Country	Design	Year	n	Diagnostic standard	TP	FP	TN	FN	QUADAS score
Angelika Reissig [19]	Germany	prospective	2012	356	CXR/CT	211	3	127	15	13
Americ Testa [20]	Italy	prospective	2012	67	clinic	32	5	28	2	13
Francesca Cortellaro [21]	Italy	prospective	2010	120	clinic	80	2	37	1	11
D. Iuri [22]	Italy	prospective	2008	32	CXR	20	2	8	2	10
Vaishali P. Shah [23]	US	prospective	2012	200	CXR	31	18	146	5	13
Hadeel M. Seif El Dien [24]	Egypt	prospective	2013	75	CXR	64	4	7	0	11
Stefano Parlamento [25]	Italy	prospective	2009	49	CXR/CT	31	0	17	1	10
R. Copetti [26]	Italy	prospective	2008	79	CXR/CT	60	0	0	19	10
Caiulo VA [27]	Italy	prospective	2013	88	clinic	88	0	1	13	12

CXR: There is filtration, patch or consolidation on chest radiography, with pleural effusion or not. CXR/CT: CT is considered the reference standard when CXR is uncertain for diagnosis or CXR is negative while lung ultrasound is positive. Clinic: Diagnosis of pneumonia is based on comprehensive results of CT, clinical course, conventional tests and follow-up. TP: True positive; FP: False positive; FN: False negative; TN: True negative; QUDUAS: quality assessment for studies of diagnostic accuracy.

Diagnostic accuracy

The overall pooled sensitivity was 0.97 (95% CI: 0.93-0.99), and pooled specificity was 0.94 (95% CI: 0.85-0.98). The PLR was 15.62 (95% CI: 6.31-38.68) (Figure 1), NLR was 0.03 (95% CI: 0.01-0.08) (Figure 1), and the DOR was 507.99 (95% CI: 128.11-2014.34). The SROC curve illustrates the relationship between SEN and SPE, determining the presence of a threshold effect. Based on the bivariate approach which estimates not only the strength but also the shape of the correlation between SEN and SPE, a 95% confidence ellipse and a 95% prediction ellipse were drawn (Figure 2). The area under the SROC curve (AUC) was 0.99 (95% CI: 0.98-1.00), demonstrating the high discriminatory ability of LUS.

Figure 1.

Scattergram of the positive likelihood ratio (PLR) and negative likelihood ratio (NLR).

Figure 2.

SROC curves from the bivariate model for proven and probable cases.

The I² for the pooled DOR was 59.3%, and that was 68.6% and 71.9% for SEN and SPE, respectively, showing substantial heterogeneity among studies. Deeks’ funnel plot asymmetry test was used to evaluate the final set of studies for potential publication bias. The slope coefficient was associated with a p value of 0.78, suggesting symmetry in the data and no publication bias (Figure 3). Figure 4 shows the Fagan’s nomogram for likelihood ratios, and the results indicated that LUS for detection of pneumonia increased the post-probability to 80% when the results were positive and reduced the post-probability to 1% when the results were negative.

Figure 3.

The Deek’s funnel plot for the assessment of potential publication bias.

Figure 4.

Fagan’s nomogram for calculating posttest probabilities.

Discussion

In recent years, the incidence of pneumonia experiences an upward trend, which causes significant clinical and financial burden all over the world. CXR and thoracic CT are familiar diagnosing tests, but they are irradiating and requiring transportation. LUS has long been used in pulmonary diseases. Pleural effusion, lung consolidation, interstitial syndrome, and pneumothorax are accessible to LUS. It can locate pleural effusion, assisting in thoracentesis for diagnostic or therapeutic purposes. It is also applied in airway management, diagnosis and follow-up of pneumonia, assessment of acute respiratory distress syndrome (ARDS), diagnosis and management of pneumothorax, including assessment of volume and progression [29]. Multiple studies have shown lung ultrasound imaging to be effective in diagnosing pneumonia. In 2002, Beckh S et al paid attention to the diagnostic value of lung ultrasound in pneumonia [30]. Subsequently, extensive diagnostic study had been made. The objective of the present study was to evaluate the diagnostic accuracy of lung ultrasound in pneumonia. In this study, we clarified the diagnostic accuracy of lung ultrasound for pneumonia. To our best knowledge, this is the first meta-analysis to estimate the diagnostic accuracy of lung ultrasound in pneumonia.

The summary results of these nine studies showed that lung ultrasound plays a valuable role in diagnosing pneumonia. We found a summary estimate of 97% for sensitivity and 94% for specificity, and the AUC was 0.99, indicating a high level of overall accuracy. The DOR is a single indicator of test accuracy that combines the data from sensitivity and specificity into a single number [31]. The DOR of a test is the ratio of the odds of positive test results in the patient with disease relative to the odds of positive test results in the patient without disease [32]. The value of a DOR ranges from 0 to infinity, with higher values indicating higher accuracy. In this meta-analysis, we found that the mean DOR was 507.99, indicating a significant high level of overall accuracy. Likelihood ratios are considered to be more clinically meaningful, and we also presented both PLR and NLR as our measures of diagnostic accuracy. The pooled PLR 15.62 suggests that patients with pneumonia have an approximately 16-fold higher chance of being lung ultrasound positive compared with patients without pneumonia. On the other hand, the pooled NLR 0.03 suggests that if the lung ultrasound was negative, the probability that this patient has pneumonia was 3 percent, which is low enough to rule out pneumonia. However, these data should be interpreted in parallel with clinical findings and other tests.

The recent study by Reissig et al [19] is a prospective, multicenter study with very good specificity and sensitivity by highly trained sonographers, which has suggested the use of LUS for diagnosis and close follow-up of community-acquired pneumonia. It is a simple and noninvasive method which avoids the use of ionizing radiation, making it appropriate for children, the pregnant, and young female patients. LUS has bedside availability and feasibility, which facilitates triage and immediate decision making [33]. Additionally, LUS can detect other ancillary findings that are not always obvious on the CXR [34]. It is likely that LUS may be a replacement for chest radiography for CAP [35].

Some limitations should be considered when interpreting the results. Firstly, the sample sizes of several included studies are rather small and they may not have adequate ability to assess the diagnostic accuracy. Secondly, several of the included studies only included neonates and children, who had thinner chest wall, smaller thoracic width and lung mass and smaller volume of lung parenchyma, leading to a higher positive rate [36], which may cause the heterogeneity among included studies. Thirdly, CXR is not included in the guidelines for diagnosing pneumonia, but several of the included studies only used CXR to diagnose pneumonia, which may result in misdiagnosis and cause false negative results. Finally, this meta-analysis is limited to published studies that may miss some of the gray literature. Thus, more clinical studies are needed to further investigate the diagnostic accuracy of US for pneumonia.

Conclusion

In summary, lung ultrasound plays a valuable role in the diagnosis of pneumonia, which is a promising attractive alternative to chest radiography and thoracic CT scan. The results should be interpreted in parallel with clinical findings and the results of conventional tests.