The Diagnostic Accuracy of Serum Procalcitonin for Bacteremia in Critically Ill Children

Abstract

Background

Bacterial sepsis is frequently encountered in children admitted to the Pediatric Intensive Care Unit (PICU) and requires early recognition and treatment. Procalcitonin (PCT) is a serum biomarker with a high sensitivity to predict bacteremia in critically-ill adults. This study sought to evaluate the diagnostic accuracy of PCT for bacteremia in febrile children in the PICU.

Methods

This retrospective observational study used data from children admitted to the PICU from October 2010 to October 2012. Patients up to 21 years of age were included if they had an abnormal temperature, serum PCT and blood culture assayed, and were not receiving empiric antibiotics at the time.

Results

There were 202 PCT values that met inclusion criteria. The prevalence of positive blood cultures was 13.2% (27 total positive blood cultures). The area under the curve (AUC) for PCT was 0.79 (95% CI, 0.70-0.89), the AUC for lactate was 0.76 (95% CI, 0.65-0.87), and the AUC for C-reactive protein was 0.68 (95% CI, 0.57-0.80). The optimal threshold of PCT for accuracy was determined to be 2 ng/mL (sensitivity = 69.2%, specificity = 74.4%, positive predictive value = 28.6%, negative predictive value = 94.2%). The combination of an abnormal lactate (> 2.0mmol/L) increased the specificity of PCT for diagnosing bacteremia.

Conclusions

PCT has a good diagnostic accuracy to rule-out bacteremia in critically-ill, febrile children. The combination of PCT and an abnormal lactate value increases the specificity and may improve the ability to diagnose bacteremia.

INTRODUCTION

Sepsis, a systemic inflammatory state caused by an infection, is a leading cause of morbidity and mortality among both adults and children. There are nearly 100,000 emergency department visits in the United States for pediatric sepsis annually.¹ Studies have estimated over 42,000 annual pediatric hospital admissions for severe sepsis in the US with a mortality rate of 10.2% and an annual cost of $11.7 billion.^2,3 The ability to diagnose sepsis early can have a significant impact on patient outcomes. However, inflammatory states, more accurately termed the systemic inflammatory response syndrome (SIRS), can be seen in a host of illnesses. One study estimated 82% of admissions to the pediatric intensive care unit developed SIRS at one point during their course. Of those children with SIRS, only one quarter of them had bacterial sepsis.⁴ A marker that differentiates sepsis from other causes of inflammation would be instrumental in the timely diagnosis and treatment of appropriate patients.

Currently, sepsis is suspected by the presence of a nonspecific group of clinical signs and symptoms including fever, tachycardia and tachypnea. Prior to the advent of newer technologies, such as PCR that have yet to be incorporated globally, the gold standard for diagnosis of bacterial sepsis is the blood culture. While accurate, it takes 48-72 hours to yield results since it depends upon in vitro bacterial growth. A plethora of blood tests (including lactate, C-reactive protein (CRP), blood glucose, platelet count, white blood cell count (WBC) and differential) that yield results within minutes to hours are currently employed in the diagnosis of sepsis. Each is sensitive but non-specific.^5,6,7

Procalcitonin (PCT) is a cytokine produced ubiquitously in response to endotoxin and has been shown to increase within 4-12 hours of infection.^8,9 A recent meta-analysis in critically ill adults concluded PCT is a helpful biomarker in early diagnosis of sepsis, but its value must be considered carefully in the clinical context.¹⁰ In a meta-analysis of 29 studies in neonates, PCT was reported as a good marker of neonatal sepsis.¹¹ Little data exists for the pediatric intensive care unit (PICU) population. One small study of 64 critically ill children showed that a positive PCT level (defined by the authors as > 2.5ng/mL), when added to bedside clinical judgment, increased the likelihood of predicting bacterial infection from 39% to 92%.¹² A recent retrospective study examined the value of PCT to predict serious bacterial infection (including bacteremia, urinary tract infections and pneumonia) in critically-ill children. The group reported that a PCT value of ≥ 1.45 ng/dL had a positive predictive value of 30% and a negative predictive value of 93%.¹³ In a prospective, observational study of 144 critically ill children with suspected bacterial infections, PCT levels were higher in those with confirmed bacterial infections as compared with those with low suspicion for bacterial infection.¹⁴ However, no large-scale studies of the diagnostic accuracy of serum PCT for bacteremia alone have been performed in febrile, critically ill, pediatric patients.

This retrospective study was performed to evaluate the performance of PCT in febrile, critically-ill children. We hypothesized that PCT has better diagnostic accuracy for bacteremia than other biomarkers such as CRP, serum lactate or WBC.

MATERIALS AND METHODS

Study Design and Subjects

The study was conducted in a single, 23-bed tertiary medical-surgical PICU. Beginning in 2010, the evaluation of febrile patients in the PICU included sampling of serum PCT, complete blood count, and peripheral blood bacterial culture. CRP and lactate were measured in a proportion of these patients at the discretion of the treating physician. A retrospective chart review was performed from October 2010 to October 2012. Approval for the study was obtained from the Institutional Review Board of Weill Cornell Medical College.

All serum PCT values obtained from patients in the PICU were identified and the corresponding patient charts were reviewed. Values were included if obtained from patients 0-21 years of age, with new onset of abnormal temperature (> 38.5C if non-neutropenic, or > 38.0C if neutropenic, or <36.0C) within 24 hours of the acquisition of the sample, and a peripheral blood culture was drawn (within 6 hours of the PCT value). Values were excluded from the analysis if the patient was receiving empiric antibiotic therapy or if the abnormal temperature was not within 24 hours of the PCT value. If a patient had serial procalcitonin assays, subsequent values were only included if they represented a NEW abnormal temperature. The following data were collected: patient age, gender, admitting diagnosis, presence of invasive vascular catheters, mechanical ventilation, presence of SIRS criteria,¹⁵ serum PCT (both at time of fever and 24 hours later if available), WBC, CRP (if available), lactate (if available), blood culture results, and antibiotics administered. Lactate values were obtained from free-flowing specimens (either arterial catheter, central venous catheter or arterial puncture). Study data were collected and managed using REDCap.¹⁶

Laboratory Assays

PCT concentration (reportable range 0.05 to 500 ng/mL) was measured according to manufacturer’s instructions using an enzyme-linked immunofluorescence assay (VIDAS®, VITEK ImmunoDiagnostic Assay System PCT Assay; bioMerieux, France). An abnormal PCT is flagged in the electronic medical record as abnormal if > 0.09 ng/mL. Blood cultures were collected in BD BACTEC ™ Peds Plus bottles (BD, Franklin Lakes, NJ) and incubated in the BACTEC instrument following manufacturer instructions. Isolates were identified by a combination of automated and manual phenotypic identification methods, according to the laboratory protocol. In a subset of patients, serum CRP and lactate values were obtained. CRP concentrations (reference range 0.05 to 200 mg/dL) were measured according to manufacturer’s instructions using immunoturbidimetry (SYNCHRON® System, Beckman Coulter, Ireland). Lactate concentrations (reference range 0.3-20 mmol/L) were measured according to manufacturer’s instructions using amperometry (Epocal Epoc™ Blood Analysis System, Ottawa, Canada).

Statistical Analysis

Categorical variables are presented as n (%) and continuous variables as mean ± SD or median (interquartile range). The area under the curve (AUC) method was used to quantify the diagnostic accuracy of PCT and CRP. Receiver-operator characteristic (ROC) curves were plotted for PCT, CRP, lactate, WBC, and rate of rise of PCT, and the associated AUC’s were reported with 95% confidence intervals. The rate of rise of PCT was calculated by the formula (PCT₁ – PCT₀)/PCT₀, where PCT₀ was the PCT value at the time of the abnormal temperature and PCT₁ was obtained approximately 24 hours after the initial abnormal temperature. A multivariable logistic regression model was constructed to evaluate the prognostic value of more than one biomarker. The area under the curve (AUC) method was used to quantify the diagnostic accuracy of each biomarker on univariable analysis and of the combined multivariable model. The DeLong test was used to compare the AUC’s between univariate models of PCT alone, CRP alone, lactate alone, as well as with a multivariable model of both PCT + lactate.¹⁷ All p-values are two-sided with statistical significance evaluated at the 0.05 alpha level. All analyses were performed in SAS Version 9.4 (SAS Institute, Cary, NC).

RESULTS

Patients

A total of 2049 serum PCTs were assayed (from 396 patients). A total of 1847 values were excluded due to concurrent antibiotic therapy or other exclusionary criteria, leaving 202 specimens (from 144 patients) included in the study (see Figure 1). Baseline characteristics of patients included in the analysis are described in Table 1. Of note, nearly half of patients were mechanically ventilated and 95% met SIRS criteria. Antibiotic therapy was initiated in nearly 60% of the children due to concern of serious bacterial infection.

Figure 1.

Flowchart of PCT assays screened and included in the analysis

Table 1.

Baseline Clinical Data

Characteristics
Age (years)	2.7 (0.9-11.7)
Gender (male)	115 (56.4)
Admitting Diagnosis
Respiratory Insufficiency/Failure	53 (26)
Post-Op Cardiac	35 (17)
Neurosurgical	29 (14)
Burn	24 (12)
Sepsis	22 (11)
Status Epilepticus	16 (8)
Post-Operative Non-Cardiac	7 (3)
Renal Transplant	3 (1.5)
Trauma (without traumatic brain injury)	3 (1.5)
Pulmonary Hypertensive Crisis	3 (1.5)
Ingestion	2 (1)
Status Asthmaticus	1 (0.5)
Acute Disseminated Encephalomyelitis	1 (0.5)
Pericarditis	1 (0.5)
Hypernatremic Dehydration	1 (0.5)
Hemolytic Uremic Syndrome	1 (0.5)
Procalcitonin, ng/dL	0.63 (0.15-3.01)
In Patients with Confirmed Bacteremia	14.65 (0.75-33.77)
In Patients with Negative Cultures	0.53 (0.13-2.43)
White blood cell count, x103/uL	13.1 (9.8-18.0)
C-reactive protein, mg/dL	6.27 (2.24-12.1)
Lactate, mmol/L	1.39 (0.81-1.98)
Received antibiotic therapy	118 (57.8)
Positive blood culture	27 (13.2)
Central line	71 (35.3)
Urinary catheter	70 (35.0)
Mechanical ventilation	99 (49.3)
Presence of SIRS	190 (94.5)

Data expressed as n (%) or median (interquartile range)

Microbiology

The prevalence of positive blood cultures from patients tested was 13.2% (27 total positive blood cultures). The microorganisms present are depicted in Table 2. Klebsiella pneumonia (6/27 or 22%) and Staphylococcus aureus (6/27 or 22%) were the most frequently isolated bacteria.

Table 2.

Microorganisms Isolated from Blood Cultures

Microorganism	Number (%) Isolated from Blood Cultures
Gram positive bacteria	12 (44.4)
Staphylococcus aureus	6 (22)
Streptococcus agalactiae (GBS)	3 (11)
Enterococcus faecalis	2 (7)
Streptococcus pneumoniae	1 (4)
Gram negative bacteria	15 (55.6)
Klebsiella pneumonia	6 (22)
Escherichia coli	3 (11)
Pseudomonas aeruginosa	3 (11)
Enterobacter cloacae	2 (7)
Serratia marcescens	1 (4)

Biochemical Assays

The median PCT value for all of the assays was 0.63 ng/dL (0.15-3.01 ng/dL). The median PCT value in the bacteremic patients was 14.65 ng/dL (0.75-33.77 ng/dL) as compared to 0.53 (0.13-2.43 ng/dL) in the non-bacteremic patients. The median WBC was 13.1×10³/uL (9.8-18.0 ×10³/uL). Of the samples analyzed, there were 119 corresponding CRP values, with a median of 6.27 mg/dL (2.24-12.10 mg/dL). There were 94 corresponding lactate values with a median of 1.39 mmol/L (0.81-1.98 mmol/L). There were 56 PCT values measured approximately 24 hours after the abnormal temperature with an average rate of rise of 9.6ng/dL over 24 hours (ranging from −0.72 to 161.0 ng/dL over 24 hours).

The AUC for PCT was 0.79 (95% CI, 0.70-0.89), for lactate it was 0.76 (95% CI, 0.65-0.87), for CRP it was 0.68 (95% CI, 0.57-0.80), for the rate of rise of PCT it was 0.65 (95% CI, 0.47-0.82) and for WBC it was 0.60 (95% CI, 0.46-0.74) (Figure 2). No statistically significant differences were identified in the accuracy of PCT for bacteremia in our population when compared to CRP, p = 0.12 or lactate, p = 0.45. However, a statistically significant difference was identified when compared to WBC, p = 0.004. The accuracy of PCT alone was not different from the rate of rise of PCT, p = 0.81. The optimal threshold of PCT was determined to be 2 ng/dL based upon the ROC curve. At 2ng/dL, the sensitivity of PCT for bacteremia was 69.2% (95% CI, 48.2-85.7%), the specificity was 74.4% (95% CI, 67.3-81.0%), the PPV was 28.6% (95% CI, 17.9-41.3%), the NPV was 94.2% (95% CI, 89.0-97.5%) and there was an overall accuracy of 73.8% (95% CI, 67.1-79.7%). A sub-group of the children 12 years of age and younger (n = 159) showed similar but slightly less accurate values (sensitivity 57.9% [95% CI, 35.7-80.1%], specificity 73.6% [95% CI, 66.3-80.9%], PPV 22.9% [95% CI, 11.0-34.8%, NPV 92.8% [95% CI, 88.0-97.6%], accuracy 71.7% [95% CI, 63.3-80.1%]. At the laboratory reference value, a PCT of < 0.09 ng/dL had a negative predictive value of 100%.

Figure 2.

Receiver Operating Characteristic (ROC) Curves of Serum Biomarkers for the Diagnosis of Bacteremia

The multivariable logistic regression model that evaluated the prognostic value of more than one biomarker showed significant changes in the diagnostic accuracy of PCT when abnormal lactate values were added into the model. The sensitivity, specificity, PPV and NPV of PCT, and the combination of PCT and an abnormal lactate at specified cut-offs are reported in Table 3.

Table 3.

Performance of PCT Alone and Combination of PCT and Lactate to Diagnose Bacteremia

Threshold	Sensitivity	Specificity	PPV	NPV
PCT > 0.09	100	19.3	15.5	100
PCT > 0.2	96.2	34.1	17.7	98.4
PCT > 0.2 + abnormal lactate	50.0	85.9	42.1	89.3
PCT > 0.5	88.5	48.7	20.4	96.6
PCT > 0.5 + abnormal lactate	43.8	87.2	41.2	88.3
PCT > 1	69.2	62.5	21.4	93.2
PCT > 1 + abnormal lactate	31.3	91.0	41.7	86.6
PCT > 2	69.2	73.9	28.1	94.2
PCT > 2 + abnormal lactate	31.3	92.3	45.5	86.8
PCT > 5	57.7	84.7	35.7	93.1
PCT > 5 + abnormal lactate	31.3	93.6	50.0	86.9
PCT > 10	53.9	91.5	48.3	93.1
PCT > 10 + abnormal lactate	31.3	94.9	55.6	87.1

DISCUSSION

This study of the diagnostic accuracy of PCT for bacteremia alone in febrile, critically-ill children is one of the largest published to date. Though the reference PCT values considered abnormal are only based on adult studies,^18,19 our results validate these cut-offs in critically-ill children. With a negative predictive value of 100%, a PCT of < 0.09 ng/dL can reassure a practitioner to withhold antibiotics in this patient population.

The performance of PCT in our sample of critically-ill children corresponds with data from larger studies in stable adults.^20,21 In one study of over 5000 adults, the AUC for PCT to predict bacteremia was 0.77 (95% CI, 0.72-0.81). With a cut-off value of 0.5 ng/mL, the sensitivity of PCT was 86% and the specificity was 46%.¹⁹ The specificity of PCT alone to diagnose bacteremia in this dataset was poor.

One recent adult study suggested that the rate of rise of PCT over a 6 to 12 hour period has a much better predictive value.²² However, the rate of rise of PCT in our population did not bear the same results. The accuracy of the rate of rise of PCT may be worse than the accuracy of initial PCT alone. It is possible that the time interval between the two measurements may have been suboptimal. A larger, prospective study is needed.

Other adult studies have examined the ability of PCT to discriminate between Gram negative and Gram positive bacteremia.^23,24 They have consistently reported higher median serum PCT levels in Gram negative bacteremia as compared to Gram positive bacteremia. The ROC curves demonstrated discriminatory ability between the two types of bacteremia. Our sample, however, did not follow this trend. The median PCT values for Gram-positive versus Gram-negative bacteremia were nearly identical; 15.58 ng/mL (IQR 0.67, 35.01) versus 13.72 (IQR 0.63, 34.13). Similarly, the receiver operating characteristic analysis showed an AUC for PCT of 0.52 in discriminating Gram-positive versus Gram-negative bacteremia. This divergence from adult data may simply be a result of small sample size or it may represent a difference in the production of PCT in critically ill children to differing types of infection as compared to an adult cohort.

The addition of an abnormal lactate considerably increased the specificity, especially within lower ranges of PCT values. This result suggests that a combination of biomarkers may be helpful in assessing the risk of bacteremia in a febrile, critically-ill child. Brent and colleagues reported the development of a score based on clinical data to estimate the risk of serious bacterial infection.²⁵ Serum biomarkers, such as PCT and lactate, may enhance accuracy if added to scoring models.

In our study, when compared with CRP, the AUC for PCT was higher but not statistically significant. Previous systematic reviews and meta-analyses reported that PCT is a better predictor of bacterial infection or sepsis than CRP in adults and non-critically ill children.^26,27 Simon and colleagues reported on the utility of PCT and CRP in a sample of 64 critically ill children and found similar results to those reported here. The AUC for PCT was 0.71 and for CRP was 0.65 (p = 0.46).¹⁶ The greater magnitude of difference in AUC between PCT and CRP reported in our study suggests that a larger sample size might yield a statistically significant result.

Nearly 60% of our patients received at least 24 hours of antibiotic therapy. With the increasing prevalence of multi-drug resistant organisms in hospitals, many have advocated for a reduction in antibiotic usage. To date, the value of PCT to guide antibiotic therapy has been unclear. Small, prospective studies of PCT versus CRP to guide antibiotic therapy in critically ill adults showed no difference in the use of antibiotics.²⁸ On the other hand, in a larger, retrospective ICU-database study of critically ill adults, Hohn and colleagues described that implementation of a protocol based on PCT was associated with decreased duration of antibiotic treatment without compromising clinical outcomes.²⁹ Further studies in children are needed to evaluate prospectively the use of PCT-based algorithms to guide antibiotic therapy.

Several limitations of this study should be considered. It is retrospective and, therefore, may be prone to selection bias. A large number of patients were excluded because they were receiving antibiotic therapy. Though this was not ideal and limited the sample size; concurrent antibiotic therapy could change the accuracy of the blood culture. Given the retrospective design of the study, no standard protocol for interpretation of PCT in our unit and little data in the medical literature, a description of how PCT was used to prescribe antibiotics in this study could not be recounted. Additionally, CRP and lactate values were not available for each febrile episode. The performance of PCT may be limited in patients with severe malnutrition,³⁰ immunodeficiency³¹ or chronic kidney disease.³² The effects of these conditions on PCT were not considered in this study. Similarly, surgical trauma and burn patients, both of whom were included in this study, may lead to false positive PCT values.³³ Patients up to 21 years of age were included in the study; the age cut-off for pediatrics differs among PICUs but older patients may not be representative of all children. However, our sub-group analysis of children 12 years of age and younger did not show significant differences. The amount of blood obtained for culture was not controlled, possibly influencing the sensitivity of blood cultures to detect bacteremia. As more advanced technology for early detection of bacteria in cultures, such as PCR, becomes standard, these results will need to be considered. Lastly, the prediction model developed in the current study must be interpreted with caution in view of the relatively small number of bacteremic patients studied and because our prediction models have not been validated using an independent group of pediatric critically-ill patients.

In conclusion, low PCT values have a high negative predictive value to exclude bacteremia in critically-ill, febrile children. The combination of PCT and an abnormal lactate value increases the specificity and may increase the ability to diagnose bacteremia.