Abstract
INTRODUCTION:
Diagnosis of pleural infection (PI) may be challenging. The purpose of this paper is to develop and validate a clinical prediction model for the diagnosis of PI based on pleural fluid (PF) biomarkers.
METHODS:
A prospective study was conducted on pleural effusion. Logistic regression was used to estimate the likelihood of having PI. Two models were built using PF biomarkers. The power of discrimination (area under the curve) and calibration of the two models were evaluated.
RESULTS:
The sample was composed of 706 pleural effusion (248 malignant; 28 tuberculous; 177 infectious; 48 miscellaneous exudates; and 212 transudates). Areas under the curve for Model 1 (leukocytes, percentage of neutrophils, and C-reactive protein) and Model 2 (the same markers plus interleukin-6 [IL-6]) were 0.896 and 0.909, respectively (not significant differences). However, both models showed higher capacity of discrimination than their biomarkers when used separately (P < 0.001 for all). Rates of correct classification for Models 1 and 2 were 88.2% (623/706: 160/177 [90.4%] with infectious pleural effusion [IPE] and 463/529 [87.5%] with non-IPE) and 89.2% (630/706: 153/177 [86.4%] of IPE and 477/529 [90.2%] of non-IPE), respectively.
CONCLUSIONS:
The two predictive models developed for IPE showed a good diagnostic performance, superior to that of any of the markers when used separately. Although IL-6 contributes a slight greater capacity of discrimination to the model that includes it, its routine determination does not seem justified.
Keywords: Diagnostic, inflammatory biomarkers, pleural effusion, pleural fluid, pleural infection
Infectious pleural effusion (IPE) is a complication that affects 57% of patients with pneumonia.[1] IPE occurs in patients of all ages,[2] although it is more common in children and older adults.[2,3] The incidence of pleural infection (PI) has increased in the last decades. In USA, the rate of admission for empyema has doubled,[2] the length of hospital stay has grown,[4] and mortality rates reach 30% in immunocompromised patients.[2,4,5]
When symptoms are atypical and no evidence of pneumonia is found on chest X-ray, PI diagnosis may be challenging.[6] In addition, the diagnostic tests most commonly used in this context (blood or pleural fluid [PF] culture) occasionally yield false-negative results.[4,7] So far, no clinical, laboratory, or radiological markers have been found to be robust enough to predict the patients with pneumonia who are more likely to have PI.[8,9]
PF culture is the most reliable method to reach a final diagnosis of PI. Yet, the markers traditionally used (total count and percentage of nucleated cells, pH, and lactate dehydrogenase [LDH] and glucose levels) do not always contribute to establish a final diagnosis. We hypothesized that predictive models based on a combination of traditional parameters and inflammatory biomarkers (procalcitonin, C-reactive protein [CRP], interleukin 6 [IL-6], and tumor necrosis factor alpha [TNF-α]) – which are generally elevated in pneumonia – could be useful for the diagnosis of IPE.
Methods
Patients
The study involved all patients with PE treated in our unit between June 1, 2013 and December 31, 2016. Patients were divided into five diagnostic groups: (1) malignant PE; (2) tuberculous PE; (3) IPE; (4) miscellaneous exudative PE; and (5) transudative PE. When a patient had undergone several thoracenteses, biochemical data were extracted from the first procedure.
Exclusion criteria were age < 18 years; previous empyema; major surgery within the previous 5 days; pneumonectomy on the infected side, pregnancy or lactation; and life expectancy of < 3 months.
Definitions
Diagnosis of IPE was associated with bacterial pneumonia, lung abscess, or infected bronchiectasis.[10] In the absence of lung infiltrates, infection was evaluated on the basis of PE, fever, and elevated serum leukocytes or inflammatory markers. If the patient responded to antibiotic therapy, PI was established as uncomplicated. If antibiotic therapy alone was not effective or loculation occurred, PI was diagnosed as complicated. Finally, the presence of purulent PF or a positive PF culture were indicators of empyema.[9] Tuberculous PE: positive staining/culture for Mycobacterium tuberculosis in sputum, PF or biopsy, or the presence of caseating granulomas in the pleural biopsy. Malignant PE: the presence of malignant cells in cytological smears or pleural biopsy specimens or, in lung cancer, positive biopsy or cytology for malignancy in airway samples without other possible causes of PE. Other diagnoses were established on the basis of the previously determined criteria.[11]
Samples
PF was extracted in fasting conditions by ultrasound-guided thoracentesis before any therapy was started. PF samples were sent for microbiological (Ziehl–Neelsen staining and in anaerobic, aerobic, and Löwenstein–Jensen medium), cytological, and biochemical analysis. Samples were centrifuged at 1500 ×g at 4°C for 15 min. Supernatants were processed within 2 h after extraction and stored at −80°C. Total cell count was determined using Siemens ADVIA 2120 Hematology System (Siemens Healthcare Diagnostics Inc., Deerfield, USA). Differential count was performed by optical microscopy. CRP (mg/L), procalcitonin (ng/mL), IL-6 (pg/mL), and TNF-α (pg/mL) were determined by high-sensitivity CRP using IMMULITE® 2000 (SIEMENS Medical Solutions Diagnostics); KRYPTOR-PCT using KRYPTOR® Compact (Brahms Diagnostic, Berlin, Germany); high-sensitivity CRP using IMMULITE® 2000 (SIEMENS Medical Solutions Diagnostics); and electrochemiluminescence using IMMULITE® 1000 (SIEMENS Medical Solutions Diagnostics), respectively. Other parameters included were pH, total protein count, albumin, n-terminal pro-brain natriuretic peptide level, glucose, cholesterol, triglycerides, and adenosine deaminase.
Pleural tissue extraction
When diagnosis was not confirmed by PF analysis, an ultrasound-guided pleural biopsy was performed using Cope or Abrams needles. If results were not conclusive, a medical thoracoscopy or videothoracoscopy was performed. In general, we used the diagnostic algorithm recommended by the Spanish Society of Pulmonology.[11] Informed consent was obtained from all patients before any procedure (chest computed tomography with contrast, thoracentesis, pleural biopsy, or thoracoscopy). The study was approved by the Ethics Committee of our center (registration code 2013/013).
Statistical analysis
Continuous variables were expressed as mean values ± standard deviation when distribution was normal or as median values (25th–75th percentiles) when distribution was nonnormal. Qualitative variables were expressed as absolute frequencies and percentages. Normality of distribution was assessed by the Kolmogorov–Smirnov test. Chi-squared test was used to compare categorical variables, whereas Mann–Whitney U-test was applied to continuous variables. The following factors were analyzed to determine the diagnostic performance of parameters, namely sensitivity, specificity, positive predictive value, negative predictive value, positive and negative likelihood ratios, and diagnostic accuracy. Optimal cutoff points were set based on Youden index.[12] Areas under the receiver operating characteristics (ROC) curves (area under the curve [AUC])[13] were used to evaluate the power of discrimination of the two models. Values were interpreted as follows: 0.60–0.69: low power of discrimination; 0.70–0.79: moderate power of discrimination; 0.80–0.89: good power of discrimination; and 0.90–1.00: excellent power of discrimination.
Binary logistic regression models were designed. First, all potential predictors were used. Next, based on the results of likelihood ratio tests, factors which hardly could explain the model were excluded (P < 0.05). On the basis of the regression coefficients obtained, odds ratios and their 95% confidence intervals (CIs) were calculated (95% CI). Diagnostic performance was assessed in terms of the following factors: discrimination, calibration, and diagnostic accuracy.[14,15] Calibration was evaluated using the Brier score. Nonparametric frequency estimations were plotted against those predicted by the models. ROC curves and their corresponding AUCs were also used to verify the power of discrimination of models. The theoretical relationship between the IPE likelihood threshold cutoff value and the relative value of false-positive and negative results were used to determine the validity of the models. Patients whose predicted likelihood was equal or above the cutoff point and were negative in other tests were considered to have IPE. This means that individuals who were predicted to have IPE but did not ultimately have it would be false negatives. Those who were not predicted to have IPE but did have it were considered false positives. Bootstrap was used to correct potential overestimation in relation to discrimination and calibration. Finally, based on logistic regression coefficients, nomograms were construed for the prediction models. Data analysis was conducted using “mgcv,”[16] “MASS,”[17] “rms,”[14] “OptimalCutpoints,”[17] and “pROC,”[18] all freely available on R.[19]
Results
During the study period, 813 thoracentesis were performed. A total of 107 effusions were excluded for the following reasons: diagnosis was not confirmed (27 patients), the patient met one or several exclusion criteria (33), data were from repeated patients (21), there was a potential double diagnosis (18), or data were uncompleted (8 patients). Of the 706 patients finally included, 241 had malignant PE, 28 tuberculous PE, 177 IPE, 48 miscellaneous exudative PE, and 212 transudative PE. Of the 177 IPE, 74 were uncomplicated IPEs, 65 complicated IPEs, and 38 were empyemas. The etiologies of the PEs included in the sample are shown in e-Table 1.
e-Table 1.
Etiology of the pleural effusions analyzed
| Etiology | n (%) |
|---|---|
| Malignant PE | 241 (34.1) |
| Lung | 113 (46.9) |
| Breast | 37 (15.4) |
| Lymphoma | 21 (8.7) |
| Ovary | 19 (7.9) |
| Stomach | 9 (3.7) |
| Colon | 6 (2.5) |
| Kidney | 3 (1.2) |
| Pancreas | 3 (1.2) |
| Sarcoma | 3 (1.2) |
| Myeloma | 3 (1.2) |
| Mesothelioma | 3 (1.2) |
| Uterine | 2 (0.8) |
| Bladder | 2 (0.8) |
| Leukemia | 2 (0.8) |
| Esophagus | 1 (0.4) |
| Liver | 1 (0.4) |
| Larynx | 1 (0.4) |
| Vesicle | 1 (0.4) |
| Peritoneum | 1 (0.4) |
| Thymoma | 1 (0.4) |
| Prostate | 1 (0.4) |
| Melanoma | 1 (0.4) |
| Cervix | 1 (0.4) |
| Tumor of the yolk sac | 1 (0.4) |
| Unknown | 5 (2.1) |
| Tuberculous PE | 28 (4) |
| Infectious PE | 177 (25.1) |
| Noncomplicated | 74 (41.8) |
| Complicated | 65 (36.7) |
| Empyemas | 38 (21.5) |
| Miscellaneous exudative PE | 48 (6.8) |
| Post-surgical | 10 (20.8) |
| Pulmonary embolism | 7 (14.6) |
| Chylothorax | 7 (14.6) |
| Chest trauma | 7 (14.6) |
| Drugs | 5 (10.4) |
| Hemothorax | 5 (10.4) |
| Viral pleuropericarditis | 2 (4.2) |
| Sarcoidosis | 2 (4.2) |
| Dressler syndrome | 1 (2.1) |
| Systemic sclerosis | 1 (2.1) |
| Unknown origin | 1 (2.1) |
| Transudative PE | 212 (30) |
| Heart failure | 179 (84.4) |
| Liver hydrothorax | 19 (9) |
| Peritoneal dialysis | 5 (2.3) |
| Hypoalbuminemia | 4 (1.9) |
| Nephrotic syndrome | 3 (1.4) |
| Trapped lung | 1 (0.5) |
| Volume overload | 1 (0.5) |
PE: Pleural effusion
Table 1 displays clinical data from study patients on the basis of the five diagnostic groups established. Significant differences were observed in all parameters among the five groups. The percentage of men was significantly higher in the IPE group as compared to the other groups. Table 2 shows the diagnostic performance of each parameter for IPE. Neutrophils percentage (0.836) and CRP (0.845) were the parameters with the best AUC for PF. When combined, these parameters showed a high sensitivity (neutrophils or CRP, 86.8%) and specificity (neutrophils and CRP, 94.7%).
Table 1.
Baseline characteristics of patients by classification group (median and 25th-75th percentiles)
| Malignant PE | Tuberculous PE | Infectious PE | Miscellaneous exudative PE | Transudative PE | P | |
|---|---|---|---|---|---|---|
| n | 241 | 28 | 177 | 48 | 212 | |
| Age (years) | 72 (62-80) | 34 (28.3-58.5) | 62 (50-79) | 65 (46-72.8) | 80 (72-85) | 0.000 |
| Men (%) | 132 (54.8) | 12 (42.9) | 126 (71.2) | 30 (62.5) | 143 (67.5) | 0.001 |
| Leukocytes PF (cells/µl) | 1680 (840-3062.5) | 2555 (1562.5-5212.5) | 3815 (1470-10970) | 2600 (1360-5160) | 595 (308-1175) | 0.000 |
| Neutrophils (%) | 13 (3-30) | 11.5 (2-21.3) | 61 (33.8-80.8) | 29 (6-48) | 11 (5-29) | 0.000 |
| Lymphocytes (%) | 47 (29-67) | 74.5 (53-89.3) | 18 (8-37.8) | 45 (25-74.5) | 35 (20-58) | 0.000 |
| pH | 7.40 (7.33-7.45) | 7.33 (7.22-7.44) | 7.33 (7.02-7.42) | 7.43 (7.40-7.47) | 7.46 (7.42-7.53) | 0.000 |
| LDH PF (IU/L) | 570 (348.5-989.5) | 864 (618-1420) | 959 (406-2252) | 465 (283-798) | 176 (133-231) | 0.000 |
| CRP PF (mg/dL) | 1.3 (0.6-0.8) | 5.2 (2.7-7.9) | 6.2 (3.2-10.6) | 2.0 (0.6-5.2) | 0.8 (0.3-1.6) | 0.000 |
| PCT PF (ng/mL) | 0.08 (0.06-0.15) | 0.10 (0.08-0.14) | 0.17 (0.09-0.32) | 0.09 (0.06-0.16) | 0.11 (0.07-0.20) | 0.000 |
| IL-6 PF (pg/mL) | 7013 (2556-17151) | 55204 (33024-77700) | 52514 (6196-143445) | 7810 (1572-33400) | 1921 (895-4159) | 0.000 |
| TNF-α PF (pg/mL) | 14.2 (10.2-20,.3) | 123.5 (75-161) | 24,.9 (15,.4-55.9) | 12.4 (9.4-19.2) | 11.8 (7.8-16.7) | 0.000 |
| Glucose PF (mg/dL) | 105 (88-126) | 65 (48-83) | 90 (41-119) | 108 (94-131) | 121 (105-150) | 0.000 |
| Cholesterol PF (mg/dL) | 83 (64-103) | 85 (64-112) | 79 (56-97) | 87 (68-103) | 32 (23-44) | 0.000 |
| ADA PF (U/L) | 21 (15-27) | 70 (61-80) | 31 (22-44) | 22 (12-31) | 14 (9-17) | 0.000 |
| CEA PF (ng/mL) | 14.4 (1.0-210.5) | 0.6 (0.5-1.4) | 0.9 (0.5-1.8) | 0.5 (0.5-1.3) | 0.8 (0.5-1.6) | 0.000 |
| NT pro-BNP PF (pg/mL) | 299 (120.5-939) | 77.5 (42.3-375.8) | 301 (133-1161) | 198 (68.5-828.5) | 3500 (1287-9990) | 0.000 |
ADA=Adenosine deaminase, CEA=Carcinoembryonic antigen, CRP=C-reactive protein, IL-6=Interleukin 6, LDH=Lactate dehydrogenase, NT-ProBNP=N-terminal pro-brain natriuretic peptide, PCT=Procalcitonin, PE=Pleural effusion, TNF-α=Tumor necrosis factor alpha, PF=Pleural fluid
Table 2.
Sensitivity, specificity, positive and negative predictive values, and area under the curve for the diagnosis of infectious pleural effusion
| Parameter | Cut off | AUC (CI 95%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | LR+ | LR- |
|---|---|---|---|---|---|---|---|---|
| Leukocytes PF | 3380 cells/µl | 0.748 (0.703-0.793) | 55.4 | 84.7 | 53.8 | 85.5 | 3.6 | 0.5 |
| Neutrophils PF | 41% | 0.836 (0.797-0.875) | 71.9 | 84.6 | 63.2 | 89.1 | 4.7 | 0.3 |
| pH PF | 7.37 | 0.736 (0.689-0.784) | 78.8 | 58.9 | 84.2 | 50 | 1.9 | 0.4 |
| LDH PF | 385 IU/L | 0.739 (0.695-0.784) | 78.9 | 57.3 | 37.7 | 89.2 | 1.8 | 0.4 |
| CRP PF | 3.44 mg/dL | 0.845 (0.81-0.88) | 74.5 | 82 | 56.4 | 91.1 | 4.1 | 0.3 |
| PCT PF | 0.13 ng/mL | 0.65 (0.591-0.709) | 65.5 | 63.4 | 31.2 | 87.,9 | 1.8 | 0.5 |
| IL-6 PF | 18,091 pg/mL | 0.763 (0.713-0.812) | 67 | 80 | 50.9 | 88.4 | 3.3 | 4.2 |
| TNF-α PF | 15.4 pg/mL | 0.723 (0.679-0.768) | 75.9 | 60.4 | 37.6 | 88.9 | 1.9 | 0.4 |
| Glucose PF | 223 mg/dL | 0.34 (0.288-0.392) | 4.7 | 96.9 | 33.3 | 75.6 | 1.5 | 1 |
AUC=Area under the curve, CRP=C-reactive protein, IL-6=Interleukin 6, LDH=Lactate dehydrogenase, LR-=Negative likelihood ratio, LR+=Positive likelihood ratio, NPV=Negative predictive value, PCT=Procalcitonin, PF=Pleural fluid, PPV=Positive predictive value, TNF-α=Tumor necrosis factor alpha, CI=Confidence interval
e-Figure 1 displays the distribution of concentrations of PF variables for the five groups. Areas under the ROC curve for the diagnosis of IPE are shown in Figure 1.
e-Figure 1.
Distribution of leukocyte count concentrations, segmented (%), pH, lactate dehydrogenase, C-reactive protein, procalcitonin, interleukin 6 and tumor necrosis factor-alpha in pleural fluid. The central box represents lower and upper quartile values (25-75th percentiles The horizontal line inside the box represents the median. A line extends from the minimum value to the maximum. The extreme values are displayed as separate points. CRP = C-reactive protein; IL6 = Interleukin 6; IPE = Infectious pleural effusion; LDH = Lactate dehydrogenase; MEPE = Miscellaneous exudative pleural effusion; MPE = Malignant pleural effusion; PCT = Procalcitonin; PF = Pleural effusion; TBPE = Tuberculous pleural effusion; TNF-alfa = Tumor necrosis factor alpha; TPE = Transudative pleural effusion
Figure 1.
Areas under the ROC curve of the leukocyte count, percentage of segmented neutrophils (%), pH, LDH, CRP, PCT, IL-6, and TNF-α in pleural fluid for the diagnosis of infectious pleural effusion.IL6 = Interleukin 6, LDH = Lactate dehydrogenase, PF = Pleural fluid, CRP = C-reactive protein, PCT = Procalcitonin, TNF-alpha = Tumor necrosis factor alpha, ROC = Receiver operating characteristics
Logistic regression was used to estimate the likelihood that a PE was infectious. Diagnoses based on laboratory data were evaluated using two prognostic models. Model 1 included three variables (leukocyte count, percentage of neutrophils, and CRP levels in PF). Model 2 included four variables (the same as in Model 1 plus IL-6 levels in PF). The coefficients obtained from regression analysis for the two models are displayed in Table 3. Both models display a good power of discrimination of IPE (Model 1, AUC 0.896 and Model 2, AUC 0.909). Although Model 2 showed that higher IL-6 values were significantly associated with a higher probability of IPE (P = 0.01778), no statistically significant differences were observed in the power of discrimination of the two models (P = n. s.). AUCs for the two models were significantly better than any of the variables used separately (P < 0.001). Bootstrap-corrected AUC values for the two models (Model 1, AUC = 0.894; Model 2, AUC = 0.899) were slightly lower than predicted AUC values, which rules out overestimation. Model 1 classified correctly 88.2% of patients (623/706: 160/177 [90.4%] with IPE and 463/529 [87.5%] with non-IPE), whereas 89.2% of patients were correctly classified by Model 2 (630/706: 153/177 [86.4%] of IPE and 477/529 [90.2%] of non-IPE).
Table 3.
Logistic regression models for the diagnosis of infectious pleural effusion
| Model 1 | Model 2 | |||||
|---|---|---|---|---|---|---|
| Coefficients (SE) | P | OR (CI 95%) | Coefficients (SE) | P | OR (CI 95%) | |
| Leukocytes (cells/µl×103) | 0.0611 (0.0239) | 0.0105 | 1.06 (1.01-1.11) | 0.0732 (0.026) | 0.0000 | 1.07 (1.02-1.13) |
| Neutrophils (%) | 0.0331 (0.0051) | 0.0000 | 1.03 (1.02-1.04) | 0.0299 (0.005) | 0.0000 | 1.03 (1.02-1.04) |
| Cutoff | Cutoff | |||||
| CRP PF (mg/dL) | 0.0000 | 0.0000 | ||||
| IL-6 PF (pg/mL) | 0.003 | Reference | 0.003 | Reference | ||
| 0.5 | 1.39 (1.01-1.94) | 0.5 | 1.67 (1.10-2.52) | |||
| 1 | 1.94 (1.56-2.40) | 1 | 2.78 (2.11-3.65) | |||
| 2 | 3.62 (2.89-4.52) | 2 | 6.97 (4.99-9.73) | |||
| 3 | 6.05 (4.41-8.30) | 3 | 13.44 (8.78-20.60) | |||
| 5 | 11.26 (7.33-17.30) | 5 | 23.08 (13.57-39.23) | |||
| 10 | 20.32 (11.31-36.50) | 10 | 29.18 (13.79-61.75) | |||
| 15 | 31.72 (12.98-77.52) | 15 | 31.11 (9.37-103.18) | |||
| 20 | 44.70 (9.36-213.2) | 20 | 26.54 (3.88,181.48) | |||
| 25 | 72.16 (2.41-2159.7) | 25 | 89.77 (1.10-3248.18) | |||
| 0.0126 | ||||||
| 4000 | Reference | |||||
| 10,000 | 1.35 (0.80-2.28) | |||||
| 50,000 | 0.87 (0.351-1.50) | |||||
| 100,000 | 1.76 (0.92-3.36) | |||||
| 500,000 | 96.45 (5.92-1570) | |||||
| 1,000,000 | 662 (4.20-104267) | |||||
Model 1=Includes leukocyte count, percentage of segmented neutrophils, and reactive C-protein values in pleural fluid. Intercept=−4.4310, R2=0.547, brier=0.105, AUC=0.898, Model 2=Includes leukocyte count, percentage of segmented neutrophils and reactive C-protein, and IL-6 values in pleural fluid. Intercept=−4.4619, R2=0.582, brier=0.097, AUC=0.907. OR=Odds ratio, CI=Confidence intervals, AUC=Area under the curve, SE=Standard error, IL-6=Interleukin 6, PF=Pleural fluid, PF=Pleural fluid
Areas under the ROC curve for the two models are shown in Figure 2. Figure 3 displays calibration graphs [Model 1, Figure 3a and Model 2, Figure 3b]. Concordance between predicted likelihood and actual frequencies was excellent for the two models. Nomographs for Models 1 and 2 are shown in the e-Appendix [e-Figure 2a and b, respectively].
Figure 2.
Areas under the receiver operating characteristics curve for the two models
Figure 3.
Calibration graphs for Models 1 (a) and 2 (b). CRP = C-reactive protein; IL6 = interleukin 6
e-Figure 2.
Nomographs for Model 1 (nomograph 1) and 2 (nomograph 2). CRP = C-reactive protein; IL-6 = Interleukin 6; Neut = Neutrophils; PF = Pleural fluid
Table 4 shows the performance of different predicted likelihood cutoff points in the diagnosis of IPE as obtained using the two models.
Table 4.
Sensitivity, specificity, positive and negative predictive values, and positive and negative likelihood ratios for the diagnosis of infectious pleural effusion using the two models
| Likelihood (%) | Sensitivity (CI) (%) | Specificity (CI) (%) | PPV (CI) (%) | |||
|---|---|---|---|---|---|---|
| Model 1 | Model 2 | Model 1 | Model 2 | Model 1 | Model 2 | |
| 10 | 90.1 (84.2-94.3) | 92.1 ((86.6-95.8) | 61.5 (56-66) | 62.2 (57.4-66.8) | 45 (40-60) | 46.1 (41.2-62.8) |
| 20 | 82.9 (75.9-88.5) | 86.2 (79.6-91.2) | 76.5 (72.2-80.4) | 77.4 (73.2-81.2) | 55.3 (49.7-66.2) | 57.2 (51.6-69.1) |
| 30 | 77 (69.5-83.4) | 78.3 (70.9-84.6) | 86.4 (82.8-89.5) | 86.9 (83.3-89.9) | 66.5 (60-74.9) | 67.6 (61.2-76) |
| 40 | 69.1 (61.1-76.3) | 72.3 (64.6-79.3) | 90.3 (87.1-92.9) | 91.2 (88.2-93.7) | 71.4 (64.4-78.3) | 74.3 (67.4-80.9) |
| 50 | 63.1 (55-70.8) | 68.4 (60.4-75.7) | 92.4 (89.5-94.7) | 94 (91.3-96) | 74.4 (67-80.4) | 80 (72.9-85.2) |
| 60 | 57.2 (49-65) | 58.5 (50.3-66.5) | 96 (94-97.8) | 96.8 (94.6-98.2) | 84.4 (76.8-88.4) | 86.4 (78.9-89.9) |
| 70 | 51.3 (43-59.4) | 51.3 (43-59.5) | 97.9 (96.1-99) | 97.9 (96.1-99) | 89.6 (81.9-92.3) | 89.6 (81.9-92.4) |
| 80 | 35.5 (27.9-43.7) | 34.8 (27.3-43) | 99.3 (97.9-99.8) | 99.1 (97.6-99.7) | 94.7 (85.9-96.2) | 93 (83.7-94.9) |
| 90 | 17.1 (11.5-24) | 17.7 (12-24.8) | 99.5 (98.3-99.9) | 99.8 (98.7-99.9) | 92.8 (78.1-95.2) | 96.4 (82.8-97.6) |
| Likelihood (%) | NPV (CI) (%) | LR+ (CI) | LR- (CI) | |||
| Model 1 | Model 2 | Model 1 | Model 2 | Model 1 | Model 2 | |
| 10 | 94.7 (91.2-95.6) | 95.7 (92.6-96.5) | 2.34 (2.05-2.66) | 2.43 (2.14-2.77) | 0.16 (0.1-0.26) | 0.12 (0.07-0.21) |
| 20 | 92.7 (89.2-94.1) | 94.1 (90.9-95.3) | 3.53 (2.93-4.24) | 3.81 (3.17-4.59) | 0.22 (0.16-0.32) | 0.17 (0.12-0.26) |
| 30 | 91.5 (87.9-93.4) | 91.9 (88.5-93.9) | 5.66 (4.39-7.28) | 5.96 (4.61-7.70) | 0.26 (0.20-0.36) | 0.25 (0.18-0.34) |
| 40 | 89.3 (85.4-92.2) | 90.4 (86.7-93.1) | 7.13 (5.25-9.69) | 8.26 (6-11.3) | 0.34 (0.27-0.43) | 0.30 (0.23-0.39) |
| 50 | 87.7 (83.6-91.3) | 89.5 (85.7-92.9) | 8.30 (5.85-11.78) | 11.4 (7.7-16.8) | 0.39 (0.32-0.49) | 0.33 (0.26-0.42) |
| 60 | 86.5 (82-92) | 86.9 (82.7-92.5) | 15.5 (9.4-25.6) | 18.2 (10.7-30.9) | 0.44 (0.37-0.53) | 0.42 (0.35-0.51) |
| 70 | 85.1 (80.4-92.6) | 85.2 (80.5-92.7) | 24.74 (12.72-48.1) | 24.7 (12.7-48.1) | 0.49 (0.42-0.58) | 0.49 (0.42-0.58) |
| 80 | 81.4 (75.6-95.5) | 81.3 (75.3-94.1) | 51.39 (16.3-161.96) | 37.8 (13.9-102.8) | 0.64 (0.57-0.73) | 0.65 (0.58-0.73) |
| 90 | 77.4 (68.3-96.6) | 77.6 (68.7-99.3) | 37.11 (8.91-154.53) | 77.1 (10.6-562.5) | 0.83 (0.77-0.89) | 0.82 (0.76-0.88) |
CI=Confidence intervals, LR+=Positive likelihood ratio, LR-=Negative likelihood ratio, NPV=Negative predictive value, PPV=Positive predictive value
We generated a three-dimensional graph displaying the correlation among the three parameters included in Model 1 for identifying IPE (leukocytes, neutrophils [%], and CRP for PF) [Figure 4]. Cases of IPE (displayed in blue circles) are prevailingly concentrated in the area with the highest leukocyte count, segmented neutrophil percentage and CRP in PF.
Figure 4.
Three-dimensional graph showing correlations among the variables (leukocytes, segmented [%], and CRP in pleural fluid) included in Model 1 for the diagnosis of IPE (dark blue circles).CRP = C-reactive protein, IPE = Infectious pleural effusions, MEPE = Miscellaneous exudatives pleural effusions, MPE = Malignant pleural effusions, Neut = Neutrophils, PF = Pleural fluid, TBPE = Tuberculous pleural effusions, TPE = Transudative pleural effusions
Errors (false positives and false negatives) in likelihood cutoffs obtained after using the models are shown in e-Table 2.
e-Table 2.
False positives and negatives for each model at different likelihoods
| Likelihood (%) | Model 1 | Model 2 | ||
|---|---|---|---|---|
| False positives | False negatives | False positives | False negatives | |
| 10 | 173 | 15 | 173 | 11 |
| 20 | 105 | 26 | 98 | 20 |
| 30 | 62 | 35 | 56 | 32 |
| 40 | 42 | 51 | 37 | 45 |
| 50 | 31 | 58 | 24 | 52 |
| 60 | 17 | 66 | 11 | 67 |
| 70 | 7 | 77 | 4 | 79 |
| 80 | 3 | 99 | 4 | 91 |
| 90 | 2 | 124 | 4 | 111 |
| 99 | 0 | 145 | 0 | 129 |
Discussion
The main finding of this study is that a combination of traditionally used parameters (leukocyte count and neutrophil percentage) and inflammatory biomarkers (CRP and IL-6) improves the capacity of discrimination of IPE as compared with any of the markers when used separately.
Diagnosis of IPE may be challenging, as symptoms may be atypical and evidence of pneumonia is not always found on chest X-ray.[6] In addition, blood culture is only positive in 14% of patients[7] and PF culture is negative in more than 40% of samples.[4] Moreover, the germs causing IPE may be rare, and their identification may require microbiological and molecular analysis.[7] In addition, the characteristics of PF change along the course of the disease as a result of the physiopathological changes that occur in the pleural cavity in each stage. In the recent years, research has been conducted to identify potential biomarkers of IPE.[20,21,22,23,24,25,26,27,28,29] Yet, none has been demonstrated to have a good diagnostic performance. According to experts, it is unlikely that a single marker can predict PI accurately without complementary laboratory, radiological, or bacteriological data.[30] Although a purulent PE is easily diagnosed, these effusions were finally included in the study because it was considered that to assess the usefulness of a predictive model, it was necessary to include IPEs in all evolutionary phases (exudative [uncomplicated], fibrinopurulent [complicated], and organizing [empyema]).
The parameters with the highest power of discrimination for IPE were CRP (AUC 0.845), percentage of neutrophils (AUC 0.836), IL-6 (AUC 0.763), and leukocyte count (AUC 0.736) [Table 2]. Results for CRP are consistent with those reported in previous studies (sensitivity 49%–88%; specificity 67%–93%; AUC 0.75–0.85).[22,26,27] The percentage of neutrophils in PF is generally used to distinguish IPE from PE of other etiologies. However, a recent study revealed that only 57% of IPE present a percentage of neutrophils above 50%. The reason is that disease stage at the moment of thoracocentesis and previous antibiotic therapy may affect neutrophil percentage.[31] Therefore, although segmented neutrophil percentage is quite specific, its sensitivity is low. Otherwise said in other words, a low percentage of neutrophils does not rule out IPE. IL-6 is an immune mediator that acts as a differentiation factor of B-cells and as an activation factor of T-cells. IL-6 levels may be higher in PF than in blood, but a correlation has not been demonstrated. Blood IL-6 levels can be elevated in sepsis, autoimmune disease, lymphoma, AIDS, liver cirrhosis, and in patients with transplant infection or rejection.[32] IL-6 is rarely used for the diagnosis of IPE. A previous study of our research group yielded a sensitivity of 38% and a specificity of 97% for IL-6, with an AUC of 0.70.[23] Although the AUC obtained in this study is slightly better (0.763), the most striking finding of this study are the significant differences observed in sensitivity and specificity between the two studies (sensitivity, 38% vs. 67%; specificity 97% vs. 80%). This could be explained by the different cutoff points used in each study (77,363 pg/mL vs. 18,091 pg/mL, respectively). As suggested in previous studies,[22,27,29] the power of discrimination of procalcitonin for IPE was poor (AUC 0.65); therefore, procalcitonin is not a useful marker for IPE.
The variables with the highest power of discrimination (as measured by AUC) when used separately were included in the two predictive models for IPE.
Model 1 was designed for centers with limited access to very specific biochemical analysis. This model is based on the number of leukocytes, percentage of segmented neutrophils, and CRP values in PF, whereas Model 2 also includes IL-6. In the two models, the likelihood that PE was infectious increased as the values for all parameters were higher. When we added the covariate IL-6 (Model 2), we found a statistically significant association with the risk of IPE. However, the increase in AUC does not seem to be clinically relevant (AUC 0.896 for Model 1 and 0.909 for Model 2). In clinical practice, this finding suggests that IL-6 count is not useful for the diagnosis of IPE. In addition, Model 1 had higher sensitivity and lower specificity than Model 2 (90.4% vs. 87.5% and 87.5% vs. 90.2%, respectively) when Youden index was used to set the optimal cutoff point for a diagnosis of IPE to be determined. Bootstrap-corrected areas under the ROC curve for the two models were slightly below the AUC of the original data. This confirms that these models do not lead to overestimation or underestimation.
None of the models identified pH, glucose, and LDH as predictors of IPE. This may be explained by the fact that pH, glucose, and LDH values in uncomplicated IPE – i.e., exudative phase – are similar to those of exudates of other etiologies and are only altered in late stages of the disease.
Calibration graphs for the two models [Figure 3] show excellent concordance between predicted likelihoods and actual frequencies, which supports the robustness of the models. Nomographs are a useful tool for representing the weight of each variable and estimate the likelihood that a PE is infectious based on a total score [e-Figure 2 in e-Appendix, at the bottom]. The diagnostic performance of each predicted likelihood value was calculated. In the two models, a predicted likelihood of 50% showed the best diagnostic performance, as the probability of error was lower [e-Table 2 in e-Appendix].
This study has some limitations. The sample of patients with tuberculous PE was small. As tuberculosis is an infectious disease, the external validity of data in regions with a high prevalence of the disease is limited, especially in relation to the power of discrimination of IL-6, which can be elevated in tuberculosis.[23] Patients were recruited from a single center. Although overoptimism was corrected by bootstrap adjustment, the results of this study should be validated in other centers.
In sum, the diagnostic performance of the two predictive models built for the diagnosis of IPE is superior to that of any of the individual markers that compose it. The excellent power of discrimination of Model 1 suggests that the determination of IL-6 does not provide greater diagnostic performance and does not seem justified its routine determination for this differentiation. Further studies are needed to validate our results and to develop predictive models that can identify uncomplicated IPEs that will progress to more complicated stages of PI.
Financial support and sponsorship
This study was supported by a grant from the Spanish Society of Pulmonology and Thoracic Surgery (SEPAR 126/2012).
Conflicts of interest
There are no conflicts of interest.
References
- 1.Dean NC, Griffith PP, Sorensen JS, McCauley L, Jones BE, Lee YC, et al. Pleural effusions at first ED encounter predict worse clinical outcomes in patients With pneumonia. Chest. 2016;149:1509–15. doi: 10.1016/j.chest.2015.12.027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Grijalva CG, Zhu Y, Nuorti JP, Griffin MR. Emergence of parapneumonic empyema in the USA. Thorax. 2011;66:663–8. doi: 10.1136/thx.2010.156406. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Givan DC, Eigen H. Common pleural effusions in children. Clin Chest Med. 1998;19:363–71. doi: 10.1016/s0272-5231(05)70083-6. [DOI] [PubMed] [Google Scholar]
- 4.Maskell NA, Davies CW, Nunn AJ, Hedley EL, Gleeson FV, Miller R, et al. U.K. Controlled trial of intrapleural streptokinase for pleural infection. N Engl J Med. 2005;352:865–74. doi: 10.1056/NEJMoa042473. [DOI] [PubMed] [Google Scholar]
- 5.Rahman NM, Maskell NA, West A, Teoh R, Arnold A, Mackinlay C, et al. Intrapleural use of tissue plasminogen activator and DNase in pleural infection. N Engl J Med. 2011;365:518–26. doi: 10.1056/NEJMoa1012740. [DOI] [PubMed] [Google Scholar]
- 6.Davies HE, Davies RJ, Davies CW BTS Pleural Disease Guideline Group. Management of pleural infection in adults: British Thoracic Society pleural disease guideline 2010. Thorax. 2010;65(Suppl 2):ii41–53. doi: 10.1136/thx.2010.137000. [DOI] [PubMed] [Google Scholar]
- 7.Maskell NA, Batt S, Hedley EL, Davies CW, Gillespie SH, Davies RJ, et al. The bacteriology of pleural infection by genetic and standard methods and its mortality significance. Am J Respir Crit Care Med. 2006;174:817–23. doi: 10.1164/rccm.200601-074OC. [DOI] [PubMed] [Google Scholar]
- 8.Corcoran JP, Hallifax R, Rahman NM. New therapeutic approaches to pleural infection. Curr Opin Infect Dis. 2013;26:196–202. doi: 10.1097/QCO.0b013e32835d0b71. [DOI] [PubMed] [Google Scholar]
- 9.Chalmers JD, Singanayagam A, Murray MP, Scally C, Fawzi A, Hill AT, et al. Risk factors for complicated parapneumonic effusion and empyema on presentation to hospital with community-acquired pneumonia. Thorax. 2009;64:592–7. doi: 10.1136/thx.2008.105080. [DOI] [PubMed] [Google Scholar]
- 10.Light RW, editor. Pleural Diseases. Philadelphia, PA: Lippincott Williams & Wilkins; 2007. Parapneumonic effusions and empiema; pp. 179–210. [Google Scholar]
- 11.Villena Garrido V, Cases Viedma E, Fernández Villar A, de Pablo Gafas A, Pérez Rodríguez E, Porcel Pérez JM, et al. Recommendations of diagnosis and treatment of pleural effusion.Update. Arch Bronconeumol. 2014;50:235–49. doi: 10.1016/j.arbres.2014.01.016. [DOI] [PubMed] [Google Scholar]
- 12.López-Ratón M, Rodríguez-Álvarez MX, Cadarso-Suárez C, Gude-Sampedro F. OptimalCutpoints: An R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw. 2014;61:1–36. [Google Scholar]
- 13.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988;44:837–45. [PubMed] [Google Scholar]
- 14.Harrell FE. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer-Verlag; 2001. [Google Scholar]
- 15.Ewout S. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. New York: Springer; 2009. [Google Scholar]
- 16.Wood SN. Generalized additive models: An introduction with R. 2nd edition. Boca RatÓn: Chapman & Hall/CRC; 2017. [Google Scholar]
- 17.Venables W N, Ripley BD. Modern Applied Statistics with S. 4th ed. New York: Springer; 2002. [Google Scholar]
- 18.Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. PROC: An open-source package for R and S+to analyze and compare ROC curves. BMC Bioinformatics. 2011;12:77. doi: 10.1186/1471-2105-12-77. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.R Development Core Team. Foundation for Statistical Computing. Vienna, Austria: R Development Core Team. R; [Last accessed on 2015 Mar 04]. R: A Language and Environment for Statistical Computing. Available from: http://www.R-project.org . [Google Scholar]
- 20.Lin MC, Chen YC, Wu JT, Ko YC, Wang CC. Diagnostic and prognostic values of pleural fluid procalcitonin in parapneumonic pleural effusions. Chest. 2009;136:205–11. doi: 10.1378/chest.08-1134. [DOI] [PubMed] [Google Scholar]
- 21.Porcel JM, Vives M, Cao G, Bielsa S, Ruiz-González A, Martínez-Iribarren A, et al. Biomarkers of infection for the differential diagnosis of pleural effusions. Eur Respir J. 2009;34:1383–9. doi: 10.1183/09031936.00197208. [DOI] [PubMed] [Google Scholar]
- 22.San José ME, Valdés L, Vizcaíno LH, Mora T, Pose A, Soneira E, et al. Procalcitonin, C-reactive protein, and cell counts in the diagnosis of parapneumonic pleural effusions. J Investig Med. 2010;58:971–6. doi: 10.231/JIM.0b013e3181f88648. [DOI] [PubMed] [Google Scholar]
- 23.San José ME, Valdes L, Gonzalez-Barcala FJ, Vizcaino L, Garrido M, Sanmartin A, et al. Diagnostic value of proinflammatory interleukins in parapneumonic effusions. Am J Clin Pathol. 2010;133:884–91. doi: 10.1309/AJCPB67PYKVRVPPR. [DOI] [PubMed] [Google Scholar]
- 24.Skouras V, Boultadakis E, Nikoulis D, Polychronopoulos V, Daniil Z, Kalomenidis I, et al. Prognostic value of C-reactive protein in parapneumonic effusions. Respirology. 2012;17:308–14. doi: 10.1111/j.1440-1843.2011.02078.x. [DOI] [PubMed] [Google Scholar]
- 25.Marchi E, Vargas FS, Acencio MM, Sigrist RM, Biscaro MD, Antonangelo L, et al. Proinflammatory and antiinflammatory cytokine levels in complicated and noncomplicated parapneumonic pleural effusions. Chest. 2012;141:183–9. doi: 10.1378/chest.10-3181. [DOI] [PubMed] [Google Scholar]
- 26.Porcel JM, Bielsa S, Esquerda A, Ruiz-González A, Falguera M. Pleural fluid C-reactive protein contributes to the diagnosis and assessment of severity of parapneumonic effusions. Eur J Intern Med. 2012;23:447–50. doi: 10.1016/j.ejim.2012.03.002. [DOI] [PubMed] [Google Scholar]
- 27.Zou MX, Zhou RR, Wu WJ, Zhang NJ, Liu WE, Fan XG, et al. The use of pleural fluid procalcitonin and C-reactive protein in the diagnosis of parapneumonic pleural effusions: A systemic review and meta-analysis. Am J Emerg Med. 2012;30:1907–14. doi: 10.1016/j.ajem.2012.04.004. [DOI] [PubMed] [Google Scholar]
- 28.San José ME, Ferreiro L, Soneira ME, González-Barcala FJ, Vázquez MC, Golpe A, et al. Utility of measurement of interleukin-1ß and interleukin-8 in the diagnosis of complicated parapneumonic pleural effusions. Am J Clin Pathol. 2014;142:467–73. doi: 10.1309/AJCPDC7PS8TIPBXP. [DOI] [PubMed] [Google Scholar]
- 29.Dixon G, Lama-Lopez A, Bintcliffe OJ, Morley AJ, Hooper CE, Maskell NA, et al. The role of serum procalcitonin in establishing the diagnosis and prognosis of pleural infection. Respir Res. 2017;18:30. doi: 10.1186/s12931-017-0501-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Tobin CL, Lee YC. Pleural infection: What we need to know but don't. Curr Opin Pulm Med. 2012;18:321–5. doi: 10.1097/MCP.0b013e328352c673. [DOI] [PubMed] [Google Scholar]
- 31.Ferreiro L, Pereiro T, San José E, Toubes ME, Suárez-Antelo J, Álvarez Dobaño JM, et al. Behaviour of nucleated cells in various types of pleural effusion. Rev Clin Esp. 2017;217:136–43. doi: 10.1016/j.rce.2016.12.014. [DOI] [PubMed] [Google Scholar]
- 32.Shirakabe A, Hata N, Yokoyama S, Shinada T, Suzuki Y, Kobayashi N, et al. Cytokine levels in pleural effusions of patients under intensive care. J Nippon Med Sch. 2008;75:262–8. doi: 10.1272/jnms.75.262. [DOI] [PubMed] [Google Scholar]