Estimating the probability of bacterial infection using a novel biomarker among pediatric patients in the emergency department

Introduction

Early identification of bacterial infection can be challenging among pediatric patients in the emergency department (ED), but is the necessary first step in recognizing potential progression of illness in the continuum of inflammation, sepsis, and septic shock. While many patients present with signs of systemic inflammation, a small percentage will have culture confirmed bacterial infection and therefore be at risk for progression of illness within the sepsis spectrum. In the ED setting, the diagnostic challenge lies in determining which patients have bacterial infection and require antibiotic therapy or hospital admission.

Recent literature has described successful quality-improvement based interventions undertaken to improve clinical recognition and treatment of sepsis in the ED(Cruz et al., 2011, Larsen et al., 2011, Paul et al., 2012). These interventions have improved the timeliness of early therapy for sepsis and septic shock. In spite of these successes, using clinical criteria alone may be inadequate for diagnosing a patient with potential sepsis and may lead to overtreatment. To improve the effectiveness of this approach, there is a need for adjunct objective measures to rule in or rule out sepsis.

Among adults, lactate is frequently used to aid identification of sepsis and is part of the guidelines for early goal directed therapy in the ED treatment of septic shock(Rivers et al., 2001). In pediatric patients, lactate displays good specificity but poor sensitivity in identifying children with bacterial sepsis(Scott et al., 2012, Reed et al., 2013). The search for a useful test in the ED setting has led to significant interest in biomarkers to predict bacterial infection. Procalcitonin (PCT) is thus far the most promising biomarker in this setting. In most studies, PCT is comparable to or outperforms white blood cell count (WBC), absolute neutrophil count (ANC) and C-reactive protein (CRP) in the identification of bacterial infection(Gomez et al., 2012, Haeusler et al., 2013, Hernandez-Bou et al., 2014, Kasem et al., 2012, Laham et al., 2014, Mahajan et al., 2014), but there is variability among patient populations(Lin et al., 2012, Manzano et al., 2011).

Recently, interleukin-27 (IL-27) was identified as a novel candidate biomarker for differentiating bacterial infection from sterile inflammation in critically ill children(Wong et al., 2012). However, biomarker performance can vary depending on the population studied and the prevalence of the condition of interest. Therefore, we sought to determine whether IL-27 would reliably estimate the probability of bacterial infection in the ED setting and compared IL-27 to PCT.

Methods

This is a prospective cohort study performed from January through March 2013. The study was performed in a large tertiary care, urban pediatric Emergency Department, with an annual volume of approximately 90,000 visits per year. Subjects from birth-18 years were eligible for inclusion if the treating clinician ordered cultures of blood, urine and/or cerebrospinal fluid during the ED visit and residual blood was available for collection from the clinical laboratory after ED laboratory studies had been performed. The study received approval from the hospital’s Institutional Review Board and a waiver of consent was obtained for sample collection. In order to determine eligible patients, an automated daily report was generated via the hospital’s electronic medical recordlisting all ED patients with cultures ordered in the previous 24 hours. This comprised the eligible patient population. Convenience sampling was employed to reach the calculated sample size. The PI retrieved and enrolled samples from the clinical lab between days 1–7 after the ED visit. Samples were not collected on consecutive days during the study period, but on each day of enrollment, all available blood samples within a 24 hour period were used.. . Enrollment continued until the calculated sample size was reached.. The samples were then frozen at −80 C.

IL-27 (EMD Millipore Corporation, Billerica, MA, USA) and PCT (Bio-Rad, Hercules, CA, USA) protein concentrations were measured in blood samples using a magnetic bead multiplex platform and a Luminex 100/200 System (Luminex Corporation, Austin, TX, USA), according to the manufacturers’ specifications. These were the same assays used in the original pediatric study (Wong et al., 2012).

The ED chart of each subject was reviewed by the PI and all relevant clinical variables and culture results recorded. Criteria for positive cultures are listed in Table 1. In addition, a list of conditions was compiled which were determined a priori to be consistent with bacterial infection (Table 2). The ED diagnosis was recorded and classified as either bacterial or non-bacterial based on these criteria. For patients admitted to the hospital from the ED, the hospital discharge diagnosis was used to determine final diagnosis. The PI was blinded to the results of IL-27 and PCT levels during the chart review. All information was entered into a REDCap database.

Table 1.

Positive Culture Criteria
CSF: any bacteria cultured from CSF
Urine: > 50,000 colony forming units (CFU)/mL with a single organism AND ≥5–10 WBC on urinalysis OR fever >38°C/history of fever at home
Blood: growth of pathogenic bacteria: S pneumoniae, Staphylococcus aureus, group A Streptococcus, Enterococcus species, Neisseria meningitidis, Enterobacteriaciae, Salmonella species, Moraxella catarrhalis, Pseudomonas species, H influenzae, Campylobacter species, Escherichia coli

Table 2.

Diagnoses classified as bacterial infection
pneumonia, defined as focal infiltrate on chest radiograph empyema osteomyelitis septic arthritis lymphadenitis abscess soft tissue infection/cellulitis appendicitis pharyngitis otitis media

Sample size was calculated at 693 subjects based on an estimated 20% prevalence of bacterial infection as a final diagnosis and assuming a 95% confidence interval. This calculation assumed a sensitivity of 0.90 a priori, resulting in a margin of error of ± .05. To accomplish the primary study aim, medians and interquartile ranges were calculated and Mann Whitney rank sum test used to look for differences between groups with and without bacterial infection. In addition, receiver operating characteristic (ROC) curves were calculated for IL-27 and PCT, and classification and regression tree (CART) analysis was performed to determine whether IL-27 was more highly predictive of bacterial infection in combination with other variables. SigmaStat Software was used for descriptive statistics and Salford Predictive Modeler v6.6 (Salford Systems, San Diego, CA) was used for CART analysis. The modeling procedures considered IL-27, PCT, white blood cell (WBC) count, and age as candidate predictor variables. The modeling procedures did not use weighting or cost factors.

Results

Primary Analysis

Assays were performed on 678 samples. Clinical characteristics of the subjects are listed in Table 3. Of the 678 subjects, there were 40 positive cultures. Of these, 31 were urine cultures, 8 were blood cultures and one was CSF. The subject with positive CSF also had a positive blood culture (N. meningitidis in both cultures). In addition, charts of those subjects with negative cultures were reviewedto identify patients with clinical evidence of bacterial infection. After chart review, an additional 141 patients were classified as having bacterial infection, for a total of 181 cases (27%). Table 4 lists the frequency of diagnoses consistent with bacterial infection and associated number of positive cultures. IL-27 levels were increased in this group compared to those without bacterial infection. Table 5 displays median IL-27 levels among patients with (culture and/or clinical diagnosis) and without bacterial infection, which demonstrate a statistically significant difference between the two groups. A sub-analysis was performed comparing subjects with and without positive culture, and median IL-27 levels did not show a statistically significant difference between the two groups (p=0.67). ROC curves were constructed for both IL-27 and PCT, and there was no significant difference in the AUC between the two (0.62 vs 0.61).

Table 3.

Baseline Characteristics of Subjects

	Bacterial Infection	No Bacterial Infection
N	181	497
Median age years (interquartile range (IQR))	3.2 (1–9.3)	6.5 (1.1–14.3)
# males (%)	78 (43)	202 (41)
Any comorbidity (%)	56 (31)	156 (31)
Immunosuppressed (%)	12 (6)	33 (7)
Malignancy/bone marrow transplant (%)	6 (3)	30 (6)
Disposition
Admit ICU (%)	11 (6)	28 (6)
Admit floor (%)	135 (75)	200 (40)
Home (%)	35 (19)	269 (54)

Table 4.

Frequency of Diagnoses

	Total number (%)	Number positive cultures
Pneumonia	57 (31)
Cellulitis/soft tissue infection	36 (20)
Urinary tract infection	31 (17)	31, urine
Otitis media	18 (10)
Abscess	14 (8)
Appendicitis	11 (6)
Sepsis/septic shock	5 (3)	5, blood
Pharyngitis	5 (3)
Osteomyelitis	2 (1)	2, blood
Meningitis	1 (0.6)	2, CSF and blood
Septic arthritis	1 (0.6)
Total	181	40

Table 5.

Median IL-27 and Procalcitonin Levels

	Bacterial infection N=180 Median (IQR)	No bacterial infection N=495 Median (IQR)	p-value Bacterial infection vs. no bacterial infection	AUC for discriminating bacterial infection from no bacterial infection (95% C.I.)
IL-27 ng/mL	1.7 (1.2 – 2.6)	1.3 (0.8 – 2.1)	<0.001	0.61 (0.56 – 0.65)
PCT ng/mL	8.6 (7.3–10.1)	7.7 (5.6 – 9.2)	<0.001	0.62 (0.57 – 0.67)

Secondary Analysis

Exploratory analysis of diagnostic subgroups analysis was performed to determine whether IL-27 correlated with specific types of infection. We focused on pneumonia because this was the most common diagnosis in this cohort. There were 52 cases of bacterial pneumonia (defined as focal infiltrate on chest x-ray at some point during the hospital course) and 136 cases of viral respiratory illness. Median IL-27 levels were higher in patients with bacterial pneumonia compared to patients with viral respiratory illness (2.3 vs 1.4 ng/ml, p <0.001). Although the AUC for IL-27 (0.73) tended to be greater than that of PCT (0.63), the difference did not reach statistical significance (p = 0.1)

We then performed CART analysis using IL-27, PCT, WBC and age as possible predictor variables to construct a decision tree to estimate the probability of bacterial pneumonia. The derived tree is shown in Figure 1. The decision tree incorporates IL-27, PCT and WBC, and has an AUC of 0.80 (Figure 2), which was significantly greater than that of IL-27 alone (p = 0.02) and PCT alone (p < 0.001). The diagnostic test characteristics of the decision tree were: sensitivity 88% (95% C.I. 76 – 95), specificity 59% (50 – 67), positive predictive value 45% (35 – 55), negative predictive value 93% (85 – 97), positive likelihood ratio 2.1 (1.7 – 2.7), and negative likelihood ratio 0.2 (0.1 – 0.4).

Figure 1. CART Analysis for the Diagnosis of Bacterial Pneumonia (Secondary Analysis).

The CART-derived decision tree for estimating the probability of bacterial pneumonia, based on IL-27, PCT, and WBC count. Each node provides the total number of subjects in the node, the IL-27-, PCT-, or WBC-based decision rule, and the number of patients with and without bacterial pneumonia, with the respective probabilities. Terminal nodes (TN) 1 and 2 are low bacterial pneumonia probability nodes (probability 0.000 to 0.083). Terminal nodes 3 and 4 are intermediate bacterial pneumonia probability nodes (probability 0.294 to 0.325). Terminal node 5 is a high bacterial pneumonia probability node (probability 0.622). To calculate the diagnostic test characteristics, all subjects in the low probability terminal nodes (n = 86) were classified as predicted no bacterial pneumonia, whereas all subjects in the intermediate and high probability terminal nodes (n = 102) were classified as predicted bacterial pneumonia.

Figure 2. AUC for CART versus IL-27 or PCT Alone for Diagnosis of Bacterial Pneumonia (Secondary Analysis).

ROCs for the decision tree, IL-27 alone, and PCT alone for patients with bacterial pneumonia. The AUC for the decision tree was significantly greater than the AUCs for IL-27 alone and PCT alone (p < 0.05).

Discussion

IL-27 has previously been shown to have >90% specificity and positive predictive value for bacterial infection in critically ill children admitted to the intensive care unit. This study tested the ability of IL-27 to estimate the probability of bacterial infection among ED patients. We found that there was a statistically significant difference between median IL-27 concentrations when comparing those with (culture positive and/or clinical diagnosis) and without bacterial infection. However, the IL-27 concentrations overlap substantially between the two groups, which in this large and diverse population may be difficult to interpret clinically. However, given that this is a novel candidate biomarker with relatively unknown characteristics, it was necessary to begin with broad inclusivity so that initial test characteristics in the ED population could be identified.

Among the subpopulation of patients with focal infiltrate on chest radiograph, we found that these subjects had significantly higher levels of IL-27 when compared with study patients meeting criteria for viral respiratory illness (at least one of cough, rhinorrhea or difficulty breathing). While the results of this analysis are limited given that chest radiographs were not obtained on all patients with viral respiratory illness, the differentiation between viral and bacterial lung infection represents a common diagnostic dilemma in the ED. Therefore, the results of this exploratory analysis may warrant further study. Our findings are in contrast to the previously published study which found that, in adults, IL-27 was not predictive of sepsis among patients with a lung source of infection(Wong et al., 2013). However, other studies have shown increased IL-27 to correlate with lung infection and inflammation, so the finding is biologically plausible (Cao et al., 2014, Xu et al., 2013).

Of interest, among the variables of IL-27, PCT, WBC and age incorporated for CART analysis, IL-27 emerged as the first level decision rule. We note that the highest probability of focal infiltrate on chest x-ray was in patients with both a high IL-27 and a high PCT value. We also note that the decision tree has a high sensitivity, which could be considered ideal for the ED setting. Finally, the high negative predictive value and low negative likelihood ratio indicate that the decision tree could be an effective “rule out” test for bacterial infection, resting on the premise that focal infiltrate on chest x-ray may be more likely to be consistent with bacterial pneumonia. These assertions require prospective, independent testing.

Defining positive cases of bacterial infection when evaluating diagnostic biomarkers remains a major challenge. If one uses culture results as the gold standard, there is a high likelihood that a substantial number of true positive cases will be misclassified as false negatives. Alternatively, using clinical criteria to define positive cases that are otherwise culture negative carries the risk of false positive classifications.

This was designed as an exploratory study to measure the performance of IL-27 as a diagnostic biomarker for bacterial infection in a heterogeneous pediatric ED cohort. Because of the relatively low prevalence of positive blood, urine or cerebrospinal fluid cultures, we defined bacterial infection to include conditions generally considered bacterial in origin. Although eligible samples were identified prospectively, using retrospective chart review to classify the diagnoses may limit the accuracy of this classification. In addition, to classify bacterial pneumonia, a definition of focal infiltrate on chest radiograph was used. Focal infiltrate is generally accepted as being more likely to be bacterial in origin (Korppi et al., 1993), but because the study was conducted during the winter months with a high incidence of viral respiratory infections, there is uncertainty as to whether the focal infiltrate is viral or bacterial in origin, particularly among children younger than 1 year of age(Kiekara et al., 1996).

Because the entry criteria included any patient with cultures ordered by the treating clinician, a broad sample was collected. In practice, biomarkers are most often used to help the clinician determine the probability of infection at the point of care, prior to knowing culture results. For patients presenting with clinically apparent infection, such as UTI diagnosed on urinalysis or cellulitis apparent on physical exam, clinicians would be unlikely to require a biomarker to help in diagnosis and management.

Conclusions

IL-27 has the potential to serve as a diagnostic biomarker for bacterial infection in ED patients. In this initial heterogeneous and highly inclusive study population, the overlap in IL-27 levels makes the result less likely to be clinically useful. Because there is a statistically significant association between increased IL-27 concentrations and bacterial infection, however, this biomarker deserves further targeted study in populations in which diagnostic testing in the ED setting would be most helpful. The correlation between bacterial pneumonia and IL-27 levels, as compared with viral lung infection, also requires further study.