Implementation of a Pragmatic Biomarker-Driven Algorithm to Guide Antibiotic Use in the Pediatric Intensive Care Unit: the Optimizing Antibiotic Strategies in Sepsis (OASIS) II Study

Abstract

Background

Biomarkers can facilitate safe antibiotic discontinuation in critically ill patients without bacterial infection.

Methods

We tested the ability of a biomarker-based algorithm to reduce excess antibiotic administration in patients with systemic inflammatory response syndrome (SIRS) without bacterial infections (uninfected) in our pediatric intensive care unit (PICU). The algorithm suggested that PICU clinicians stop antibiotics if (1) C-reactive protein <4 mg/dL and procalcitonin <1 ng/mL at SIRS onset and (2) no evidence of bacterial infection by exam/testing by 48 hours. We evaluated excess broad-spectrum antibiotic use, defined as administration on days 3–9 after SIRS onset in uninfected children. Incidence rate ratios (IRRs) compared unadjusted excess length of therapy (LOT) in the 34 months before (Period 1) and 12 months after (Period 2) implementation of this algorithm, stratified by biomarker values. Segmented linear regression evaluated excess LOT among all uninfected episodes over time and between the periods.

Results

We identified 457 eligible SIRS episodes without bacterial infection, 333 in Period 1 and 124 in Period 2. When both biomarkers were below the algorithm’s cut-points (n = 48 Period 1, n = 31 Period 2), unadjusted excess LOT was lower in Period 2 (IRR, 0.53; 95% confidence interval, 0.30–0.93). Among all 457 uninfected episodes, there were no significant differences in LOT (coefficient 0.9, P = .99) between the periods on segmented regression.

Conclusions

Implementation of a biomarker-based algorithm did not decrease overall antibiotic exposure among all uninfected patients in our PICU, although exposures were reduced in the subset of SIRS episodes where biomarkers were low.

Prompt initiation of antibiotics in patients with sepsis has been associated with reduced morbidity and mortality [1, 2]. Broad-spectrum agents are necessary initially to maximize the likelihood of providing coverage for potential bacterial pathogens. Yet, many critically ill children who present with systemic inflammatory response syndrome (SIRS) ultimately do not have bacterial infections. Overuse of broad-spectrum antibiotics can lead to patient harm, development of antibiotic resistance, and excess healthcare costs [3]. Given the extent of antibiotic prescribing in pediatric intensive care units (PICUs) [4, 5], practical approaches are needed to safely limit broad-spectrum antibiotic use.

Biomarkers such as procalcitonin (PCT) and C-reactive protein (CRP) can guide antibiotic discontinuation in patients with suspected sepsis and reduce overall antibiotic use in neonates and adults [6–13]. However, data to support their use in children are limited [6, 14, 15]. We previously assessed the performance of a panel of serum biomarkers to identify critically ill children with SIRS at low-risk of bacterial infection and found that the combination of CRP and PCT, when both are low at SIRS onset, can accurately identify such children (negative predictive value 85%) [16]. Due to the routine availability of CRP and PCT at our center, these findings served as the basis for development of a practical algorithm to guide safe discontinuation of broad-spectrum antibiotics in patients with SIRS at low risk for bacterial infection admitted to our PICU. We hypothesized that, when combined with standard clinical evaluations and examinations, measurement of biomarkers at SIRS onset could facilitate identification of patients at low risk of invasive bacterial infection in whom empiric antibiotics could be safely discontinued and lead to a reduction in antibiotic use. We sought to evaluate the impact of the algorithm on excess antibiotic therapy in children without proven bacterial infections.

METHODS

Study Period and Setting

We compared broad-spectrum antibiotic prescribing before and after deployment of a biomarker-based algorithm among children with new-onset SIRS without proven bacterial infections treated in the PICU at Children’s Hospital of Philadelphia (CHOP) from August 10, 2012 to June 9, 2016. The CHOP PICU is a quaternary academic PICU with 55 beds and approximately 4000 annual admissions.

Study Population

Episodes were divided into 2 groups based on the date of SIRS onset in relation to implementation of the algorithm on June 10, 2015: Period 1 (August 10, 2012 to June 9, 2015) and Period 2 (June 10, 2015 to June 9, 2016). New-onset SIRS was defined as the presence of ≥2 age-related criteria for SIRS within a 24-hour period [17]. Episodes were included if (1) the patient was ≤18 years of age, (2) blood culture was obtained at CHOP ≤6 hours of SIRS onset, and (3) a new intravenous broad-spectrum antibiotic (see definitions below) was initiated within 12 hours after blood culture collection. All new SIRS episodes >30 days after onset of a prior qualifying episode of SIRS were eligible. Episodes in patients with advanced directives limiting care, a history of hematopoietic stem cell or solid organ transplant, or with an absolute neutrophil count <500 cells/µL were excluded because such patients were not included in the biomarker evaluation study that informed the current algorithm [16]. We also excluded patients discharged from the PICU within 2 calendar days of SIRS onset because our intervention (described below) was restricted only to the PICU, and the goal of this study was to assess the impact of the biomarker algorithm on PICU prescribing practices.

Intervention

In June 2015, CRP and PCT were incorporated into an orderset to facilitate ordering cultures, laboratory tests, and antibiotics in children with suspected sepsis in the PICU. On June 10, 2015, an algorithm was enacted stating that PICU clinicians should consider stopping antibiotics if all of the following criteria were met: (1) sterile site cultures obtained at SIRS onset revealed no growth at 48 hours, (2) both biomarkers obtained at SIRS onset were low (CRP <4 mg/dL and PCT <1 ng/mL), and (3) no focal signs of infection were present on exam or by imaging. Critical care clinicians were educated about the algorithm at a division meeting in June 2015 and sent weekly reminder e-mails from July through December 2015 that contained the algorithm information. Screensavers with the algorithm were displayed on PICU workstations throughout Period 2. Supplemental Figure 1 displays the timeline of education surrounding the intervention.

Clinicians were not required to adhere to the intervention nor to utilize the orderset when placing orders for children with suspected sepsis. Measurement of biomarkers and diagnostic testing for the source of infection were performed at the discretion of the clinical team. Our Antimicrobial Stewardship Program (ASP) team, which must grant authorization for antibiotic prescriptions >48 hours, had access to the biomarker values in the electronic health record (EHR) but did not utilize the algorithm as part of the review process for antibiotic approval. No specific education on the algorithm was provided to ASP providers. The CHOP Institutional Review Board approved the study protocol with a waiver of informed consent/assent because this project was an evaluation of a practice change.

Data Collection

We recorded information from the EHR including patient demographics, hospital admission diagnosis, presence and category of complex chronic conditions [18], vital signs, and length of hospital and PICU stay before and after enrollment. Severity of illness at SIRS onset was assessed using the PEdiatric Logistic Organ Dysfunction-2 (PELOD-2) score [19]. Medications administered, procedures performed, and results of microbiologic cultures and imaging evaluations from 48 hours before and through 10 calendar days after enrollment were collected.

Definitions

The calendar day of SIRS onset was considered day 0. The presence of bacterial infection was assessed by independent review of medical records (microbiology, laboratory, imaging, and clinical data) from days 0–2 and from days 3–9 by 1 of 3 investigators (K.J.D., M.E.R., S.E.C.). Ambiguous cases were adjudicated through review by all 3 investigators. Bacterial infections were classified as present if Centers for Disease Control and Prevention (CDC) criteria for proven or probable bacterial infection were met [20], consistent with definitions used for derivation of the algorithm in our previous work [16]. A viral infection was defined as a laboratory-confirmed pathogen by polymerase chain reaction test from a clinically obtained specimen.

The primary outcome for this study was excess antibiotic use defined as administration of broad-spectrum antibiotics on days 3–9 in a patient without a proven bacterial infection (“uninfected episode”). The following antibiotics were considered broad-spectrum: aminoglycosides, third-/fourth-generation cephalosporins, carbapenems, β-lactam/β-lactamase inhibitor combinations, daptomycin, vancomycin, and fluoroquinolones. Antibiotic exposure was defined as length of therapy ([LOT] number of days on which any antibiotic was administered) as well as days of therapy ([DOT] total number of antibiotics across all days) per 1000 patient-days [21]. Because patients with suspected sepsis at our institution can receive 48 hours of antibiotics without ASP approval, we calculated LOT and DOT from days 3–9 after SIRS onset for all eligible episodes; days were included through day 9 after SIRS onset, or until hospital discharge or death, whichever occurred first. If antibiotics were stopped for >24 hours and then restarted, antibiotics administered after stoppage were not included.

Statistical Analysis

Summary statistics were described among all eligible uninfected episodes. To compare uninfected episodes in the 2 periods, χ² and Fisher’s exact tests (categorical data) or Wilcoxon 2-sample tests (continuous data) were used, as appropriate. We also compared episodes with and without bacterial infections identified on days 0–2 irrespective of period.

Incidence rate ratios (IRRs) were calculated to compare unadjusted excess LOT and DOT per 1000 patient-days among uninfected episodes between the 2 periods. Episodes were stratified based on biomarker results: (1) both biomarkers below the cut-points (“low-risk group”), (2) either biomarker above the cut-point, or (3) both biomarkers not measured.

We then performed an interrupted time series analysis using segmented linear regression to test for a change in excess antibiotic use in the PICU after implementation of the algorithm among all uninfected episodes. The main exposure was study period (Period 2 vs 1). To assess the effect of time on antibiotic prescribing, calendar time was divided into 2-month blocks. Two-month blocks were chosen because the number of evaluable SIRS episodes per month was small. The primary outcome was cumulative excess LOT per 1000 patient-days per 2-month period. Secondary analyses were performed to assess excess DOT per 1000 patient-days per 2-month period.

Our base model included the following terms: bi-monthly time (2-month periods since the start of the study), study period (Period 2 vs 1), and number of bi-monthly periods since intervention (0 for all preintervention periods). The coefficient for periods since intervention reflects the change in the rate of antibiotic prescribing postintervention compared with preintervention. Meanwhile, the coefficient for study period represents differences in mean antibiotic prescribing between the periods.

Covariates were individually included in the model to determine their association with antibiotic prescribing during the study. Because of sample size considerations (ie, 23 total 2-month periods), covariates could only be added individually to the model. First-order positive autocorrelation was detected for both the LOT (Durbin-Watson [DW] statistic = 1.35) and DOT (DW statistic = 1.25) models using the Durbin-Watson statistic, so final models were adjusted for autocorrelation. Statistical analyses were performed using SAS 9.4 (SAS Institute Inc., Cary, NC) and R version 3.3.2 (R Foundation, Vienna, Austria).

RESULTS

We identified 1378 SIRS episodes in children ≤18 years of age during Period 1 and 495 in Period 2. A total of 507 (36.7%) and 206 (41.6%) episodes were eligible after chart review in Periods 1 and 2, respectively (Figure 1). There were 454 and 179 unique children that contributed episodes during the 2 periods. Overall, 35.9% of reviewed episodes had a bacterial infection identified (n = 256 of 713). The prevalence of bacterial infections was similar between periods (34.3% [174 of 507] vs 39.8% [82 of 206]; P = .17). Supplemental Table 1 compares the characteristics of episodes based on presence/absence of bacterial infection. Episodes without identified bacterial infection were from children who were younger (P = .02), in the PICU longer before SIRS onset (P < .001), and less often required vasopressors at SIRS onset (P = .04). Pediatric intensive care unit length of stay was similar among episodes with and without bacterial infections identified (P = .67).

Figure 1.

Flow diagram of inclusion into the Optimizing Antibiotic Strategies in Sepsis (OASIS) II study. Double-barred boxes highlight episodes without bacterial infection identified (uninfected). Uninfected episodes were the target of the intervention. Episodes in these boxes are used for primary comparisons in the manuscript, including segmented regression analyses. Hashed boxes represent a subset of uninfected episodes with both biomarkers below the algorithm’s cut-points (low-risk group). Episodes in these boxes are used for comparisons of unadjusted excess length of therapy and days of therapy pre- and postintervention. CRP, C-reactive protein; DNR, do not resuscitate; PCR, polymerase chain reaction; PICU, pediatric intensive care unit; SIRS, systemic inflammatory response syndrome.

No bacterial infection was identified in 333 and 124 SIRS episodes in Periods 1 and 2, respectively. Uninfected episodes occurred amongst similar patients in the 2 periods (Table 1), although patient races were different (P = .05). In Period 1, uninfected episodes more often involved patients with an underlying cardiovascular (P < .001), respiratory (P = .01), or gastrointestinal condition (P = .05). Among uninfected episodes, measurement of both biomarkers at SIRS onset significantly increased from Period 1 to 2 (33.0% [110 of 333] vs 60.5% [75 of 124]; P < .001) (Supplemental Figure 2). The frequency of Infectious Diseases consultation was similar in the 2 periods: 24.0% (n = 80 of 333) in Period 1 compared with 19.3% (n = 24 of 124) in Period 2 (P = .29).

Table 1.

Patient and Clinical Characteristics of the Study Population Portion Without a Bacterial Infection During Days 0–2 of Follow-Up, by Intervention Period

Characteristic	Period 1	Period 2	P Value
	(n = 333)	(n = 124)
Age in years, median (IQR)	4.1 (1.2–12.3)	3.8 (1.5–12.6)	.62
Female sex, n (%)	158 (47.5)	51 (41.1)	.23
Race, n (%)			.05
Asian	16 (4.8)	7 (5.7)
Black	125 (37.5)	30 (24.2)
White	118 (35.4)	58 (46.8)
Other/Unknown	74 (22.2)	29 (23.4)
Ethnicity, n (%)			.64
Hispanic/Latino	49 (14.7)	15 (12.1)
Non-Hispanic/Latino	283 (85.0)	109 (87.9)
Unknown	1 (0.3)	0 (0)
Presence of any complex chronic condition, n (%)	280 (84.1)	101 (81.5)	.50
Malignancy	28 (8.4)	12 (9.7)	.67
Hematologic or autoimmune	73 (21.9)	23 (18.6)	.43
Respiratory	122 (36.6)	30 (24.2)	.01
Gastrointestinal	176 (52.9)	53 (42.7)	.05
Metabolic	77 (23.1)	27 (21.8)	.76
Neuromuscular	183 (55.0)	59 (47.6)	.16
Cardiovascular	154 (46.3)	35 (28.2)	<.001
Renal	46 (13.8)	12 (9.7)	.24
Other congenital	103 (30.9)	44 (35.5)	.35
Pre-enrollment PICU length of stay in days, median (IQR)	0 (0–1)	0 (0–1)	.35
Postenrollment PICU length of stay in days, median (IQR)	6 (4–13)	5.5 (4–11.5)	.15
Presence of positive viral panel days 0, 1, or 2, n (%)	134 (40.2)	58 (46.8)	.21
PELOD-2 score,a median (IQR)	6 (4–9)	6 (4–9)	.54
Mechanical ventilation,b n (%)	163 (49.1)	62 (50.0)	.86
Receipt of vasopressor infusion,b n (%)	89 (26.8)	28 (22.6)	.36
Vasopressor use in days,b median (IQR)	0 (0–1)	0 (0–1)	.29
Surgery,b n (%)	23 (6.9)	13 (10.5)	.21
Chest tube in place,b n (%)	0 (0)	1 (0.81)	.27
Other diagnostic procedure,b n (%)	14 (4.2)	7 (5.7)	.51

Abbreviations: IQR, interquartile range; PELOD-2, PEdiatric Logistic Organ Dysfunction-2; PICU, pediatric intensive care unit.

^aAt PICU admission.

^bAt any time during 10 days after enrollment.

Unadjusted Analyses

The proportion of episodes with both biomarkers below the algorithm’s cut-points (low-risk group) was similar between the periods (43.6% [48 of 110] vs 41.3% [31 of 75]; P = .76). Unadjusted excess antibiotic LOT was significantly lower in Period 2 among uninfected episodes with both biomarkers below the cut-points (IRR, 0.53; 95% confidence interval [CI], 0.30–0.93) (Table 2), as was unadjusted excess antibiotic DOT (IRR, 0.43; 95% CI, 0.26–0.71). Meanwhile, unadjusted excess antibiotic LOT was similar among those with either biomarker above the cut-points and those in whom biomarkers were not measured, as was DOT.

Table 2.

Excess Broad-Spectrum Antibiotic Prescribing in Relation to Biomarker Cut Points Among Patients Without Proven Bacterial Infections^a

Biomarkers	Length of therapy (LOT)			Days of therapy (DOT)
	Period 1 LOT per 1000 patient-days	Period 2 LOT per 1000 patient-days	IRR (95% CI)	Period 1 DOT per 1000 patient-days	Period 2 DOT per 1000 patient-days	IRR (95% CI)
Either above cut point	240.8	272.7	1.13 (0.80–1.60)	371.2	450.0	1.21 (0.92–1.59)
Both below cut points (low-risk)	196.8	103.9	0.53 (0.30–0.93)	301.2	129.9	0.43 (0.26–0.71)
Biomarkers not measured	238.6	214.6	0.90 (0.67–1.21)	385.8	323.9	0.84 (0.66–1.07)
All episodes without proven bacterial infection	232.6	207.7	0.89 (0.73–1.09)	370.1	320.5	0.87 (0.71–1.06)

Abbreviations: DOT, days of therapy; LOT, length of therapy.

^aLow-risk cut-points defined as C-reactive protein <4 mg/dL and procalcitonin <1 ng/mL.

Time Series Analyses

Figure 2 displays the excess LOT and DOT over time among all uninfected episodes. Supplemental Table 2 displays the results of additional covariate testing in multivariable models. Segmented regression analysis showed that excess LOT among all uninfected patients in Period 2 was similar to that in Period 1 (Table 3), after adjusting for performance of surgery during the 10 days after SIRS, the most significant covariate on additional testing. There was also no difference in the slope of antibiotic prescribing after the intervention. Excess DOT was also similar among all uninfected patients in the 2 periods (Table 3).

Figure 2.

Excess broad-spectrum antibiotic administration among uninfected episodes. Excess length of therapy (A) and days of therapy (B) on days 3 through 9 in uninfected episodes. LOT, length of therapy; SIRS, systemic inflammatory response syndrome.

Table 3.

Multivariable Linear Regression for Assessment of Excess Broad-Spectrum Antibiotic Administration (Days 3–9) Among Uninfected Patients

Length of Therapy
Covariate	Coefficient	P Value
Period 2 vs Period 1	0.9	.99
Bimonthly time	−6.0	.18
Time since interventiona	10.8	.55
Surgery (Y/N)	554.7	.01
Days of Therapy
Period 2 vs Period 1	-40.6	.76
Bimonthly time	−10.9	.19
Time since interventiona	29.4	.37
Surgery (Y/N)	1144.0	.003

Abbreviations: N, no; Y, yes.

^a Coefficient of time since intervention reflects the difference in the slopes of antibiotic prescribing between study periods.

Safety Analyses

Biomarkers were below both cut-points in 12 episodes where bacterial infections were present during Period 2. All had readily identifiable infections by culture, imaging, or exam: 9 lower respiratory tract infections, 1 ventilator-associated tracheitis, 1 surgical site infection, 1 post-neurosurgical meningitis. No patients with low biomarkers had antibiotics discontinued prematurely. Of the 31 uninfected episodes in Period 2 that had low biomarkers (low-risk group), there were no identifiable bacterial infections on days 3–9 after SIRS onset requiring re-initiation of antibiotics.

DISCUSSION

We implemented a biomarker-based algorithm in our PICU to identify low-risk patients with SIRS in whom antibiotics could be safely discontinued at 48 hours. Both the unadjusted LOT and DOT decreased by approximately 50% in the subset of uninfected episodes where both biomarker values were low at SIRS onset. However, only a small fraction of uninfected episodes qualified as low-risk: 14% in Period 1 and 25% in Period 2. Thus, overall excess broad-spectrum antibiotic administration in our PICU was unchanged among all uninfected episodes. These results suggest that measurement of biomarkers can facilitate safe discontinuation of antibiotics in low-risk critically ill children with SIRS for whom bacterial infection is not clinically identified. However, future quality improvement efforts may need to focus on routine measurement of biomarkers in all children with SIRS in order to affect overall antibiotic prescribing.

When caring for critically ill children with SIRS, clinicians are often confronted with a difficult decision to determine whether a patient clinically improved because of or in spite of antibiotics. Such a decision is relatively straightforward when a bacterial infection is identified, but there is considerable uncertainty in the absence of a known infection. Biomarkers, such as CRP and PCT, when measured at SIRS onset, may provide support that the cause of illness is unlikely to be bacterial and recovery occurred in spite of antibiotics. Thus, reassuring biomarkers may offer objective data to substantiate a decision to stop antibiotics. The fact that no patients classified as low-risk in Period 2 had untoward infectious complications after discontinuation of antibiotics supports this idea, although the number was small. Although both PCT and CRP outperform traditional markers of infection in differentiating bacterial from viral infections and sterile inflammation [22], no biomarkers have perfect sensitivity and specificity. Therefore, biomarkers can only supplement and not overrule clinical judgment when it comes to antibiotic decisions.

Culture-negative sepsis may occur for a variety of reasons, including incomplete infectious evaluation or pretreatment with antibiotics, and the adequacy of the infectious evaluation must be considered in the decision to stop antibiotics. Approximately half of severe sepsis hospitalizations are attributable to culture-negative sepsis, a diagnosis that is associated with more medically complex patients and higher mortality [23, 24]. However, there is evidence that clinicians can be more aggressive in de-escalating broad-spectrum antibiotics in culture-negative sepsis, and that appropriate de-escalation may not worsen outcomes [23]. It is important to consider the need for continuing antibiotic therapy on a case-by-case basis and to develop an appropriate monitoring plan if antibiotic therapy is stopped or de-escalated.

Reducing unnecessary antibiotic use is an important goal of pediatric ASPs, but defining antibiotic use as unnecessary is difficult in the critical care setting. To be conservative, we defined excess antibiotics as those administered 3 or more calendar days after SIRS onset in a child without evidence of a bacterial infection. Because our institution’s ASP requires approval for extension of broad-spectrum antibiotics beyond 48 hours, we allowed PICU clinicians to make antibiotic decisions per standard practice before categorizing use as unnecessary. Children who were discharged from the PICU before day 3 were not analyzed, and this occurred more often during Period 2. It is possible that our algorithm helped identify these lowest-risk patients more rapidly, reducing the measurable impact of our intervention.

After implementation of our algorithm, LOT was curtailed by approximately 50% in the subset of uninfected patients with low biomarkers. The lack of a significant decrease in antibiotic LOT overall among uninfected patients might have been due to the small number of patients to which our low-risk definition applied and slow uptake of routine biomarker measurement by clinicians. Even in Period 2, less than two-thirds of children had biomarkers measured and, of uninfected children that had biomarkers measured, only 31 (40%) had low CRP and PCT. Therefore, only 25% of all uninfected episodes in Period 2 were categorized as low-risk. With increased collection of biomarkers over time, we may have seen a significant reduction in overall excess antibiotic administration had we used a longer postintervention period.

There are limitations to our study. First, this was a passive intervention on antibiotic prescribing in our PICU. We intentionally selected an algorithm that could be easily implemented in other centers; however, the use of more formal quality improvement methods may have improved adherence and further reduced unnecessary antibiotic administration. Second, a large number of SIRS episodes were excluded from our analyses (62%). Because we excluded episodes in children with underlying conditions that inherently affect antibiotic decisions (immunocompromised, do not resuscitate), our findings are not generalizable to all critically ill children. Studies suggest that the kinetics of PCT and CRP in neutropenic children may be similar to nonneutropenic children [25]. However, decisions around antibiotic cessation differ from those in other children. Third, calls and recommendations are not formally tracked by our ASP, and, therefore, we cannot determine whether our algorithm indirectly influenced ASP guidance during Period 2. Finally, we used CDC surveillance definitions to characterize infections, which risks potential misclassification bias. Although these definitions provide a structured approach for infection identification and classification, they do not always align with clinicians’ real-time diagnoses. All medical records were manually reviewed for the presence or absence of infection, and cases that could not be classified clearly were adjudicated by 3 team members. Future studies need to evaluate the cost-effectiveness of universal biomarker measurement in children with SIRS to fully understand their impact, although adult studies suggest that they are cost effective [9, 26].

CONCLUSIONS

In conclusion, after implementation of a biomarker-based algorithm, overall antibiotic prescribing was unchanged among uninfected children in our PICU. However, antibiotic use was reduced in the subset of patients with absence of clinically identified bacterial infection and low CRP and PCT values at SIRS onset. Measurement of CRP and PCT may supplement standard diagnostic tests and clinical evaluations to help to reduce unnecessary antibiotic therapy in this subset of lowest-risk patients.