Abstract
IMPORTANCE
The death of a pediatric patient with sepsis motivated New York to mandate statewide sepsis treatment in 2013. The mandate included a 1-hour bundle of blood cultures, broad-spectrum antibiotics, and a 20-mL/kg intravenous fluid bolus. Whether completing the bundle elements within 1 hour improves outcomes is unclear.
OBJECTIVE
To determine the risk-adjusted association between completing the 1-hour pediatric sepsis bundle and individual bundle elements with in-hospital mortality.
DESIGN, SETTINGS, AND PARTICIPANTS
Statewide cohort study conducted from April 1, 2014, to December 31, 2016, in emergency departments, inpatient units, and intensive care units across New York State. A total of 1179 patients aged 18 years and younger with sepsis and septic shock reported to the New York State Department of Health who had a sepsis protocol initiated were included.
EXPOSURES
Completion of a 1-hour sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour.
MAIN OUTCOMES AND MEASURES
Risk-adjusted in-hospital mortality.
RESULTS
Of 1179 patients with sepsis reported at 54 hospitals (mean [SD] age, 7.2 [6.2] years; male, 54.2%; previously healthy, 44.5%; diagnosed as having shock, 68.8%), 139 (11.8%) died. The entire sepsis bundle was completed in 1 hour in 294 patients (24.9%). Antibiotics were administered to 798 patients (67.7%), blood cultures were obtained in 740 patients (62.8%), and the fluid bolus was completed in 548 patients (46.5%) within 1 hour.
Completion of the entire bundle within 1 hour was associated with lower risk-adjusted odds of in-hospital mortality (odds ratio [OR], 0.59 [95% CI, 0.38 to 0.93], P = .02; predicted risk difference [RD], 4.0% [95% CI, 0.9% to 7.0%]). However, completion of each individual bundle element within 1 hour was not significantly associated with lower risk-adjusted mortality (blood culture: OR, 0.73 [95% CI, 0.51 to 1.06], P = .10; RD, 2.6% [95% CI, −0.5% to 5.7%]; antibiotics: OR, 0.78 [95% CI, 0.55 to 1.12], P = .18; RD, 2.1% [95% CI, −1.1% to 5.2%], and fluid bolus: OR, 0.88 [95% CI, 0.56 to 1.37], P = .56; RD, 1.1% [95% CI, −2.6% to 4.8%]).
CONCLUSIONS AND RELEVANCE
In New York State following a mandate for sepsis care, completion of a sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour was associated with lower risk-adjusted in-hospital mortality among patients with pediatric sepsis and septic shock.
In a consortium of 44 pediatric hospitals, it was estimated that more than 170000 pediatric patients were hospitalized with sepsis from 2004 to 2012, among whom approximately 8% died prior to discharge.1 Contemporary clinical practice guidelines recommend prompt sepsis identification, blood culture prior to treatment, and administration of broad-spectrum antibiotics and intravenous fluids.2 In single-center studies at pediatric specialty hospitals, the combination of these elements in a bundle can reduce time to antibiotics and time to an intravenous fluid bolus.3,4 It is unknown, however, whether the timely completion of the bundle improves outcomes such as mortality or hospital length of stay.
Motivated by the death of a pediatric patient with sepsis,5 New York State mandated in 2013 that all hospitals deliver a care bundle to pediatric patients within 1 hour of sepsis recognition.6 Using the New York State Department of Health (NYSDOH) database, an analysis was performed to determine the association between completion of a sepsis bundle within 1 hour and risk-adjusted in-hospital mortality for pediatric sepsis and septic shock.
Methods
This study was approved with waiver of informed consent by the institutional review board for the NYSDOH (1156246-1).
Study Design and Setting
In 2013, the NYSDOH required hospitals to submit evidence-informed protocols and educate clinical staff about early identification and treatment of sepsis (previously termed severe sepsis) or septic shock (New York Codes, Rules, and Regulations 405.2 and 405.4). While protocols varied by hospital, all protocols were required to include a pediatric bundle consisting of the following interventions within 1 hour: blood culture collection before administering antibiotics, administration of broad-spectrum antibiotics, and completion of a 20-mL/kg fluid bolus. Data collection began on April 1, 2014. We performed a retrospective study of all pediatric sepsis cases reported from 59 acute care hospitals in the NYSDOH database from April 1, 2014, to December 31, 2016.
Population
All hospitals were required to report patient-level data for patients with sepsis and septic shock to the NYSDOH using electronic case submission that included demographics, comorbidities, characteristics of sepsis and septic shock, organ dysfunction, and outcomes. Date and time stamps for protocol initiation and completion of individual elements of the 1-hour bundle were required for patients in whom a sepsis protocol was initiated. The state performed 10% random sample audits for all patients using manual medical record review and provided feedback to hospitals on data quality and completeness. Patient-level data were linked to hospital characteristics using the NYSDOH administrative database.
Eligible patients included those younger than 18 years with sepsis or septic shock, using criteria adapted from the American College of Critical Care Medicine7 and Goldstein et al.8 We did not study patients in whom the 1-hour bundle was clinically contraindicated, neonates who were never discharged, patients for whom advanced directives limited treatment, patients and/or surrogates who declined interventions, or patients who were enrolled in a concomitant clinical trial. For patients in whom multiple sepsis episodes were present in the database, we studied data from the final encounter to preserve independence of observations. For pediatric patients transferred from a referring to a receiving hospital, only 1-hour bundle data from the receiving hospital were available in the database.
Hospitals used a variety of methods to identify pediatric sepsis (eMethods in the Supplement). These included (1) sepsis based on clinical assessment only (suspected or confirmed infection and ≥2 age-appropriate systemic inflammatory response syndrome criteria [eg, temperature, heart rate, respiratory rate, white blood cell count8], with supporting laboratories optional), (2) sepsis by both clinical criteria and abnormal laboratories, or (3) “code sepsis or rapid response” led to sepsis by clinical criteria. The regulations permitted hospitals to have flexibility in case identification to facilitate broader adoption. Most patients submitted to the NYSDOH database (>98%) were confirmed to have severe sepsis or septic shock on manual audit.9 Cases found to be erroneously entered could be removed by hospitals.
Variables
The primary outcome was all-cause in-hospital mortality. The primary exposure was completion of all 3 elements of the 1-hour bundle (ie, blood culture(s) collected prior to antibiotic administration, broad-spectrum antibiotics administered, and completion of at least one 20-mL/kg fluid bolus) within 1 hour of protocol initiation, as reported by hospital and clinical staff. Blood culture was recorded as time collected, antibiotic administration was recorded as time of starting administration, and intravenous fluid bolus was recorded as time of completion of 20-mL/kg crystalloid fluid administration. The NYSDOH defined “broad spectrum” according to the local hospital protocol due to variable antimicrobial resistance patterns. If elements of the 1-hour bundle occurred prior to protocol initiation, the bundle was considered completed within the first hour.
We measured covariates for a risk-adjustment model to account for confounding by indication,10 primarily that sicker children would receive the bundle more promptly. These included variables specified a priori based on clinical experience and prior pediatric studies, such as patient age; payer; comorbidity burden; location of protocol initiation (eg, emergency department, intensive care unit [ICU], or inpatient unit), site of infection (eg, respiratory, urinary, or skin); and measures of organ dysfunction including presence of shock, platelet count, or mechanical ventilation prior to protocol initiation.11-13 Race and ethnicity were included because they may be important confounders in the association between sepsis care and outcome.14 The determination of race and ethnic group was made by hospital staff at reporting hospitals with fixed categories from the NYSDOH Statewide Planning and Research Cooperative System data set.15 Using these covariates, a multivariable logistic regression for in-hospital mortality had adequate calibration (eFigure 1 in the Supplement; Hosmer-Lemeshow goodness-of-fit test for 10 groups, P = .50) and discrimination (area under the receiver operating characteristic curve, 0.83) in internal validation.
Hospital-level data were also collected to illustrate howthe 1-hour bundle was completed at hospitals of different capabilities. We identified hospitals with a pediatric ICU (PICU) using data available in the NYSDOH hospital database.16 We further specified that if a hospital provided cardiac surgery or cardiac catheterization services that the ICU met criteria for a level I pediatric ICU.17
Statistical Analysis
We performed bivariate analyses of characteristics of patients who did and did not complete the 1-hour bundle within 1 hour. Continuous data are expressed as mean (SD) or median (interquartile range), depending on normality. Categorical variables are shown as proportions. Differences in predicted risk (risk difference [RD]) between groups are shown with 95% CIs.
Multivariable modeling of the association between completion of the 1-hour bundle within 1 hour and in-hospital mortality was performed using logistic regression adjusted for covariates with robust standard errors and mixed effects to account for nonindependence of cases across hospitals. Binary variables were modeled as indicator covariates and continuous variables included as linear covariates, after assessing for nonlinear relationships using fractional polynomials.18 Each exposure (ie, completion of the entire 1-hour bundle and each individual bundle element) was evaluated separately. No missing data were present for model covariates.
To predict the risk of in-hospital death across a range of times to completing the 1-hour bundle, we restricted the cohort to cases in whom the entire bundle was completed within 4 hours from protocol initiation (n = 529 [44.9%]). We then used predictive margins adjusted for covariates to predict the risk by hour. Similar models, including nonlinear models when appropriate, were used to illustrate predicted risks of death by hour for completing each individual bundle element.
We performed risk and reliability adjustment using empirical Bayes methods to determine the hospital-level rate of completing the 1-hour bundle, collecting blood cultures, administering antibiotics, and completing the initial intravenous fluid bolus within 1 hour.19 We display the ranked order of adjusted rates across hospitals in caterpillar plots.
Sensitivity Analyses
We assessed the robustness of findings in multiple sensitivity analyses. First, we performed a matched propensity score analysis to adjust for covariate imbalance between groups (eTable 1, eFigure 2, and eMethods in the Supplement). Second, we performed an inverse probability weighted least squares regression to calculate the average treatment effect for completing the 1-hour bundle within 1 hour (eMethods in the Supplement). Third, we repeated our models, accounting for discharge bias by classifying patients discharged to hospice care as in-hospital deaths. Fourth, because transferred patients may be of greater illness severity and have complex timing of bundle elements,11 we repeated models after excluding transferred patients. Fifth, we included additional confounders in the risk adjustment model: (1) an indicator variable for prior admission for sepsis because readmissions may be strongly associated with mortality20 and (2) forcing all comorbidities and acute organ dysfunctions measured by the NYSDOH in the model. Sixth, we repeated models after excluding patients who never completed all elements of the 1-hour bundle. Seventh, we repeated the analysis with a 2-element bundle that excluded the intravenous fluid bolus.
In supporting analyses, we assessed for effect modification with a likelihood ratio test for interaction between treatment and characteristic in prespecified subpopulations, including patient location when the sepsis protocol was initiated, age strata,21 presence of septic shock, or admission to a hospital with pediatric intensive care. Hospital length of stay was studied as a secondary outcome, using Poisson regression models that generated the incidence rate ratios for in-hospital death among patients who did and did not complete the bundle in 1 hour.
In addition, linear regression models with logarithmic transformation that generated a ratio of geometric means were used to evaluate hospital length of stay. One model was used for all patients, and a second model was used for survivors and decedents. Further, we performed quantitative bias analysis22 to assess the magnitude of a hypothetical, unmeasured confounder that would be necessary to account for the association between completing the 1-hour bundle within 1 hour and risk-adjusted in-hospital mortality.
All analyses were performed with Stata version 14.2 (StataCorp) with a significance threshold of less than .05 in 2-sided tests.
Results
Patient Population and Completion of the 1-Hour Bundle
Of 1669 patient visits at 59 hospitals, we excluded 297 (17.8%) who did not have a sepsis protocol initiated, 55 (3.3%) in whom the sepsis bundle was clinically contraindicated, and 138 (8.3%) who were ineligible due to duplicate visits or other reasons (Figure 1). Of the remaining 1179 eligible patients at 54 hospitals, 294 patients (24.9%) completed the 1-hour bundle within 1 hour of protocol initiation. Blood cultures were obtained within 1 hour for 740 patients (62.8%) and antibiotics were administered within 1 hour for 798 patients (67.7%) while the fluid bolus was completed within 1 hour for 548 patients (46.5%). The 1-hour bundle was completed within 1 hour more often among patients in the emergency department (69.4% vs 53.2%; P < .001) and previously healthy patients (54.8% vs 41.1%; P = .01), and less often among patients who had been transferred from another facility (13.6% vs 20.9%; P = .006). Median serum lactate level (2.2 mmol/L vs 2.1 mmol/L; P = .09) and the proportion with septic shock (70.4% vs 68.3%; P = .49; Table 1) were similar between groups. The crude mortality rate was greater among patients who did not complete the 1-hour bundle within 1 hour compared with those who did (13.2% vs 7.5%; P = .008).
Figure 1.
Patient Accrual
Table 1.
Patient Characteristics
| Characteristica | No. (%) | P Valueb | ||
|---|---|---|---|---|
| All Patients | 1-h Bundle Completed in 1 h | |||
| Yes | No | |||
| Patients, No. (%) | 1179 (100.0) | 294 (24.9) | 885 (75.1) | |
| Age | ||||
| ≤1 mo | 109 (9.2) | 28 (9.5) | 81 (9.2) | .72 |
| >1 mo-<1 y | 165 (14.0) | 40 (13.6) | 125 (14.1) | |
| 1-<2 y | 95 (8.1) | 27 (9.2) | 68 (7.7) | |
| 2-5 y | 185 (15.7) | 53 (18.0) | 132 (14.9) | |
| 6-11 y | 233 (19.8) | 55 (18.7) | 178 (20.1) | |
| 12-17 y | 392 (33.2) | 91 (31.0) | 301 (34.0) | |
| Male | 639 (54.2) | 161 (54.8) | 478 (54.0) | .82 |
| Race | ||||
| White | 551 (46.7) | 157 (53.4) | 394 (44.5) | .008 |
| Black | 255 (21.6) | 47 (16.0) | 208 (23.5) | |
| Other | 373 (31.6) | 90 (30.6) | 283 (32.0) | |
| Hispanic ethnicity | 235 (19.9) | 59 (20.1) | 176 (19.9) | .95 |
| Payer | ||||
| Medicare | 21 (1.8) | 1 (0.3) | 20 (2.3) | .17c |
| Medicaid | 509 (43.2) | 130 (44.2) | 379 (42.8) | |
| Private or HMO | 552 (46.8) | 138 (46.9) | 414 (46.8) | |
| Self-pay | 20 (1.7) | 7 (2.4) | 13 (1.5) | |
| Other | 77 (6.5) | 18 (6.1) | 59 (6.7) | |
| Comorbidities | ||||
| Malignancy | 153 (13.0) | 28 (9.5) | 125 (14.1) | .04 |
| Immunosuppressed | 269 (22.8) | 65 (22.1) | 204 (23.1) | .74 |
| Cardiopulmonary dysfunction | 143 (12.1) | 26 (8.8) | 117 (13.2) | .05 |
| Chronic renal failure or liver disease | 52 (4.4) | 7 (2.4) | 45 (5.1) | .05 |
| Diabetes | 37 (3.1) | 7 (2.4) | 30 (3.4) | .39 |
| No comorbidity | 525 (44.5) | 161 (54.8) | 364 (41.1) | .01 |
| Protocol initiation site | ||||
| Emergency department | 675 (57.3) | 204 (69.4) | 471 (53.2) | <.001 |
| Ward | 176 (14.9) | 33 (11.2) | 143 (16.2) | |
| Intensive care unit | 328 (27.8) | 57 (19.4) | 271 (30.6) | |
| Organ dysfunction at protocol initiation | ||||
| Platelet count <150 000/μL | 356 (30.2) | 71 (24.2) | 285 (32.2) | .009 |
| Altered mental status | 323 (27.4) | 100 (34.0) | 223 (25.2) | .003 |
| Acute respiratory failure requiring mechanical ventilation | 203 (17.2) | 36 (12.2) | 167 (18.9) | .009 |
| Septic shock | 811 (68.8) | 207 (70.4) | 604 (68.3) | .49 |
| Serum lactate, median (IQR), mmol/L | 2.1 (1.3-3.8) | 2.2 (1.4-4.1) | 2.1 (1.3-3.8) | .09d |
| Transferred from another facility | 225 (19.1) | 40 (13.6) | 185 (20.9) | .006 |
| Site of infection | ||||
| Urinary | 125 (10.6) | 29 (9.9) | 96 (10.8) | .002 |
| Respiratory | 373 (31.6) | 106 (36.1) | 267 (30.2) | |
| Gastrointestinal | 190 (16.1) | 38 (12.9) | 152 (17.2) | |
| Skin | 59 (5.0) | 16 (5.4) | 43 (4.9) | |
| Central nervous system | 26 (2.2) | 6 (2.0) | 20 (2.3) | |
| Other | 239 (20.3) | 75 (25.5) | 164 (18.5) | |
| Unknown | 167 (14.2) | 24 (8.2) | 143 (16.2) | |
| Type of pathogen | ||||
| Gram positive | 139 (11.8) | 47 (16.0) | 92 (10.4) | <.001 |
| Gram negative | 104 (8.8) | 27 (9.2) | 77 (8.7) | |
| Othere | 87 (7.4) | 4 (1.4) | 83 (9.4) | |
| None reported | 849 (72.0) | 216 (73.5) | 633 (71.5) | |
| Hospital with pediatric intensive care | 1031 (87.4) | 258 (87.8) | 773 (87.3) | .85 |
| Hospital length of stay, median (IQR), h | 235 (118-496) | 198 (101-358) | 244 (123-554) | <.001d |
| In-hospital death | 139 (11.8) | 22 (7.5) | 117 (13.2) | .008 |
Abbreviations: HMO, health maintenance organization; IQR, interquartile range.
No data were missing among individual bundle elements, transfer status, age, payer, place of protocol initiation, septic shock, site of infection, platelet count at protocol initiation, chronic renal disease or liver failure, diabetes, mechanical ventilation prior to protocol initiation, mortality, and length of stay.
P values are based on Pearson χ2 test unless otherwise noted.
P value based on Fisher exact test.
P value based on Wilcoxon rank-sum test.
Anaerobic bacteria, yeast, fungus, mixed pathogens, and viruses.
Primary Analyses
In a multivariable model, the completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality (odds ratio [OR], 0.59 [95% CI, 0.38-0.93], P = .02; RD, 4.0% [95% CI, 0.9%-7.0%], Figure 2; eTable 2 in the Supplement). Coefficients of the model covariates are presented in eTable 3 in the Supplement. This association was not modified by age strata, location, presence of shock, or care at a hospital with pediatric intensive care (P > .05; eTable 2 in the Supplement). Patients who had the 1-hour bundle completed in up to 3 hours remained at lower odds of in-hospital death compared with those who completed the bundle after 3 hours (OR, 0.64 [95% CI, 0.42-0.96], P = .03; RD, 3.6%[95% CI, 0.6%-6.7%]). Figure 3 shows the crude and predicted risks of in-hospital mortality across a range of 4 hours for completing the 1-hour bundle. The mean predicted in-hospital mortality increased by 2 percentage points for each hour until the 1-hour bundle was completed. Results for individual bundle element completion within 4 hours are presented in eFigure 3 in the Supplement.
Figure 2.
Risk-Adjusted Odds Ratios of In-Hospital Death in the Primary Models
Embedded table reports the total number of patients, total number of absolute deaths (%), predicted risk from adjusted models and risk difference (95% CI), and odds ratios (95% CI) comparing patients who did and did not complete the 1 hour bundle within 1 hour. Variables in the risk-adjusted model include age category, payer, protocol initiation site, diagnosis of septic shock, site of infection, platelet count <150 000/μL at protocol initiation, chronic renal disease or liver failure, diabetes, acute respiratory failure requiring mechanical ventilation at protocol initiation, serum lactate, and transfer status.
Figure 3.
Crude In-Hospital Mortality and Predicted Risk of In-Hospital Death After the Time of Sepsis Protocol Initiation
Crude in-hospital mortality (bars) and the predicted risks of in-hospital death with 95%CIs (orange line with error bars). Predicted risks derive from model adjusted for age category, payer, protocol initiation site, diagnosis of septic shock, site of infection, platelet count <150 000μL at protocol initiation, chronic renal failure or renal disease, diabetes, acute respiratory failure requiring mechanical ventilation, serum lactate, and transfer status across 4 hours after protocol initiation for the completion of the 1-hour bundle of sepsis care. As an interpretive example, for a typical pediatric patient with sepsis with average age and level of acuity in New York State, the completion of the 1-hour sepsis bundle within 1 hour was associated with an 8% risk of in-hospital death. In contrast, the same patient who completes the bundle at 4 hours will have a 13% predicted risk of in-hospital death.
The association between individual bundle elements and outcome did not reach statistical significance. The administration of antibiotics within 1 hour was not significantly associated with lower in-hospital mortality (OR, 0.78 [95% CI, 0.55-1.12], P = .18; RD, 2.1% [95% CI, −1.1% to 5.2%]; Figure 2). Neither blood culture collection within 1 hour (OR, 0.73 [95% CI, 0.51-1.06], P = .10; RD, 2.6% [95% CI, −0.5% to 5.7%]) nor the completion of a 20-cc/kg fluid bolus within 1 hour were significantly associated with lower in-hospital mortality (OR, 0.88 [95% CI, 0.56-1.37], P = .56; RD, 1.1% [95% CI, −2.6% to 4.8%]).
Sensitivity and Supporting Analyses
In the propensity matched cohort (n = 572), completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality (OR, 0.53 [95% CI, 0.30-0.94]; P = .03; eTable 4 in the Supplement). The inverse probability weighted least squares regression estimated the reduction in mortality associated with bundle completion within 1 hour as 4.2% (95% CI, 0.6%-7.6%; P = .02; eTable 4 in the Supplement). When hospice discharges were reclassified as in-hospital deaths, timely bundle completion was associated with lower in-hospital mortality (OR, 0.56 [95% CI, 0.36-0.86], P = .008; RD, 4.6% [95% CI, 1.6%-7.6%]; eTable 2 in the Supplement). In the model that excluded transferred patients (n = 954), the completion of the 1-hour bundle was significantly associated with lower mortality (OR, 0.52 [95% CI, 0.27-0.99], P = .05; RD, 4.3% [95% CI, 1.0%-7.7%]).
We ran 2 models that included additional confounders: (1) prior admission for sepsis (OR, 0.56 [95% CI, 0.34-0.92], P = .02; RD, 4.4% [95% CI, 1.1%-7.6%]) and (2) all comorbid and organ dysfunction variables (OR, 0.54 [95% CI, 0.33-0.90], P = .02; RD, 4.5% [95% CI, 1.4%-7.7%]; eTable 2 in the Supplement). After exclusion of patients who had a protocol initiated but never completed the bundle (n = 892), the association was no longer statistically significant (OR, 0.73 [95% CI, 0.47-1.13], P = .16; RD, 2.2% [95% CI, −0.7% to 5.1%]). Completion of the 2-element bundle of blood cultures before antibiotics and administration of antibiotics within 1 hour was significantly associated with lower in-hospital mortality (OR, 0.65, [95% CI, 0.44-0.96], P = .03; RD, 3.5% [95% CI, 0.6%-6.4%]).
Completion of the 1-hour bundle within 1 hour was also associated with shorter hospital length of stay among all patients (adjusted incidence rate ratio [IRR], 0.76 [95% CI, 0.64-0.89], P = .001) and survivors (IRR, 0.71 [95% CI, 0.60-0.84], P < .001), but not significant among decedents (IRR, 1.09 [95% CI, 0.71-1.69], P = .68; eTable 5 in the Supplement). The association of bundle completion within 1 hour and shorter hospital length of stay among all patients and survivors was significant in the linear regression model with logarithmic transformation that produced a ratio of geometric means (eTable 5 in the Supplement). Coefficients of the model covariates are presented in eTable 6 in the Supplement.
To assess the sensitivity of models to unmeasured confounding, quantitative bias analysis was performed (eFigure 4 in the Supplement). Across a range of scenarios, results would be robust unless a new confounder was at least 4 times as prevalent among those who did not complete the 1-hour bundle within 1 hour compared with those who did, and unless the unmeasured confounder itself was strongly associated with lower in-hospital death (OR < 0.5).
Across hospitals, the risk and reliability-adjusted rates of completing the 1-hour bundle within 1 hour ranged from 7.3% to 46.1% (median, 32.8%; interquartile range, 22.4%-37.5%; Figure 4). After ranking hospitals from the lowest to greatest likelihood of completing the 1-hour bundle, the hospitals in the highest quartile were 2.5 times as likely to complete the 1-hour bundle within 1 hour as hospitals in the lowest quartile (37.5% vs 15.0%). Hospitals with a higher rate of completing the 1-hour bundle cared for a greater number of pediatric patients (n = 686; P = .006; Table 2), and contained a greater proportion of PICUs with specialty care such as cardiac surgery (20.4%; P = .24). We observed similar variation for each individual bundle element (eFigure 5 in the Supplement).
Figure 4.
Risk and Reliability-Adjusted Rate for Each Hospital for Completion of the 1-Hour Bundle in 1 Hour, According to Hospital Rank
The 54 hospitals that were included in the study were ranked from lowest to highest, with higher numbers (x-axis) indicating a greater likelihood of completing the 1-hour bundle within 1 hour. Bars represents 95% CIs. Orange circles correspond to hospitals with both pediatric intensive care and cardiac surgery services. Risk and reliability adjustment accounts for both patient-level risk factors that may influence outcome and statistical variation attributed to small sample sizes at the level of the hospital. These adjustments allow for accurate comparisons between hospitals of different patient volume and case mix.
Table 2.
Hospital Characteristics Stratified by Quartile of Reliability-Adjusted Rate of Completing the 1-Hour Bundle Within 1 Houra
| Characteristic | Quartiles of the Probability of Completing the 1-h Bundle in 1 h | All | P Valueb | |||
|---|---|---|---|---|---|---|
| First (Lowest) | Second (Below Median) | Third (Above Median) | Fourth (Highest) | |||
| No. of hospitals | 13 | 14 | 14 | 13 | 54 | |
| No. of patients | 245 | 89 | 159 | 686 | 1179 | .006 |
| Predicted rate of completing the 1-h bundle in 1 h, median (IQR) | 15.0 (15.0-17.5) | 22.5 (22.5-22.6) | 27.8 (24.0-28.5) | 37.4 (32.7-44.5) | 32.7 (22.5-37.5) | |
| First serum lactate level, median (IQR), mmol/L | 2.2 (1.3-4.0) | 2.0 (1.3-3.3) | 1.9 (1.2-3.5) | 2.1 (1.3-3.7) | 2.1 (1.3-3.7) | .72 |
| Crude in-hospital mortality, No. (%) | 29 (11.8) | 10 (11.2) | 26 (16.4) | 74 (10.8) | 139 (11.8) | .28 |
| Length of stay, median (IQR), h | 192 (110-442) | 248 (99-364) | 265 (142-529) | 244 (117-529) | 235 (118-496) | .07 |
| No. of hospital beds, median (IQR) | 528 (388-639) | 364 (299-530) | 453 (266-825) | 450 (375-603) | 453 (321-639) | .75 |
| Teaching facility, No. (%)c | ||||||
| No | 2 (15.4) | 3 (21.4) | 1 (7.1) | 0 (0) | 6(11.1) | .42 |
| Yes | 11 (84.6) | 11 (78.6) | 13 (92.9) | 13 (100.0) | 48 (88.9) | |
| No. of pediatric hospital beds, median (IQR) | 22 (15-28) | 20 (12-32) | 23 (14-46) | 38 (22-60) | 24 (15-39) | .32 |
| Pediatric ICU present, No. (%) | 7 (53.8) | 6 (42.9) | 7 (50.0) | 9 (69.2) | 29 (53.7) | .59 |
| No. of pediatric ICU beds, median (IQR) | 7 (4-18) | 6 (5-8) | 8 (4-23) | 10 (8-15) | 8 (5-16) | .57 |
| Pediatric specialty care services, No. (%) | ||||||
| Cardiac catheterization | 2 (15.4) | 1 (7.1) | 3 (21.4) | 4 (30.8) | 10 (18.5) | .43 |
| Cardiac surgery | 2 (15.4) | 1 (7.1) | 3 (21.4) | 5 (38.5) | 11 (20.4) | .24 |
Abbreviations: ICU, Intensive care unit; IQR, interquartile.
Reliability adjustment is an application of hierarchical modeling that uses empirical Bayes techniques to quantify and remove the statistical bias associated with small sample sizes.
P values compare across quartiles of reliability-adjusted rates.
Teaching hospital defined by New York State Department of Health using graduate medical education codes on Medicaid claims, verified by the Office of Primary Care and Health Systems Management and the Department of Education.
Discussion
During a statewide mandate for pediatric sepsis care, the prompt completion of a 1-hour bundle was associated with improved survival for pediatric patients with sepsis or septic shock. The completion of the 1-hour bundle within 1 hour was associated with lower in-hospital mortality and shorter hospital length of stay. However, individual bundle elements were not significantly associated with decreased mortality, and hospitals were widely variable in completing the bundle for comparable patients.
These findings corroborate single-center studies showing an association between bundled pediatric sepsis care and improved outcomes. In the emergency department, greater adherence to a pediatric sepsis protocol was associated with greater odds of less organ dysfunction after 48 hours23 and a shorter hospital length of stay.4 This analysis extends these findings to report an association between completing a sepsis bundle within 1 hour and lower mortality in a larger cohort of pediatric patients treated in multiple locations, including the emergency department, the ICU, and in-patient care areas across 54 community and pediatric specialized hospitals. The findings also support the current recommendations from the American College of Critical Care Medicine24 for bundled care in the treatment of pediatric sepsis.
There are many possible explanations for the association between completing a 1-hour pediatric bundle and improved outcomes. First, the individual bundle elements may each contribute to specific biologic or physiologic changes that, when combined, affect outcomes. These may include shock reversal with fluid administration, reduced pathogen load and inflammatory response from prompt antibiotics, and more accurate diagnostic information from early blood cultures. Second, it is also possible that the completion of the 1-hour bundle represents important unmeasured processes of care in a time critical situation. The completion of the bundle may be a surrogate for heightened awareness by the clinical team, differential ordering of laboratory tests such as serum lactate, and greater attentiveness to treatment response. Third, a delay in completing the bundle may be due to unmeasured, competing events in sicker patients, such as tracheal intubation, administration of blood products, or difficult intravenous access, particularly in the ICU. Fourth, it is possible the study may be underpowered to detect a significant difference between completing each individual bundle element within 1 hour or not.
Across quartiles of hospitals, there was up to a 2- to 3-fold variation in completing the 1-hour pediatric bundle. This amount of variation is similar, if not greater, than the implementation of other pediatric quality improvement measures for the acutely ill.25 Barriers to pediatric bundle completion within 1 hour may vary between units and include difficult intravenous access, clinician inexperience with pediatric care, or lack of appropriately sized equipment at primarily adult hospitals. The finding that greater rates of completing the 1-hour bundle at hospitals with more case volume and pediatric specialty services is hypothesis generating and not linear. Further qualitative study is warranted into variations in the systems of pediatric care such as electronic health record prompts for case recognition or pharmacy integration at hospitals of different sizes.
These pediatric data complement similar findings reported for adult patients with sepsis in the emergency department from the NYSDOH mandate.9 Both studies found that prompt completion of the sepsis bundle was associated with improved outcomes compared with completing the bundle after a delay. But there are important differences. First, only 25% of pediatric patients completed the 1-hour sepsis bundle within the 1-hour mandate compared with 82.5% of adults who completed the 3-hour bundle within the 3-hour adult mandate. Second, the individual elements of the pediatric bundle were not statistically significant, while individual steps in the adult bundle appeared significantly associated with outcome. The pediatric cohort, smaller than the adult cohort, may have been underpowered to detect a significant association between individual bundle element completion within 1 hour and in-hospital mortality. Third, the 3-hour adult bundle was more likely to be completed at smaller, rural hospitals, whereas the 1-hour pediatric bundle was more promptly completed at larger hospitals with greater pediatric case volume. The characteristics that may account for a typical, rural hospital performing differently for an adult bundle vs a pediatric bundle deserves dedicated study.
Limitations
This study has several limitations. First, this was a retrospective study, and results may be biased by measured and unmeasured confounding at the patient and hospital levels. The matched propensity score analysis and inverse probability weighted regression provide alternative methods to handle measured confounders and support the primary model. Quantitative bias analysis suggests that although plausible, unmeasured confounders that are common and strong enough to abrogate the primary model findings are unlikely. Further study with instrumental variables, if available, are indicated. Second, per the NYSDOH reporting guidelines, the performance of the 1-hour bundle was measured at the receiving hospital for patients cared for at multiple locations and may omit care administered at the referring hospital. Exclusion of these patients did not change the results.
Third, the moment the clinical team suspected sepsis and initiated the protocol may also be subject to misclassification, a limitation that could be addressed using electronic health record data in the future.26 Fourth, the study sample did not include neonates who were never discharged and cases not reported to the NYSDOH. Fifth, although the risk adjustment model performed well, a validated pediatric severity illness score or confounders, such as acute kidney injury, presence of central catheter, appropriateness of antimicrobials, or the variability across hospitals in sepsis case identification in children were not measured. Sixth, all hospitals were mandated to include the 3 primary sepsis elements in the bundle, and data on protocol modifications and additions were not available. Given these limitations, the true effect size of timely bundle completion may be smaller than measured in this study and could be further evaluated in designs that incorporate data from before and after the implementation of the mandate.
Conclusions
In New York State following a mandate for sepsis care, completion of a sepsis bundle within 1 hour compared with not completing the 1-hour sepsis bundle within 1 hour was associated with lower risk-adjusted in-hospital mortality among patients with pediatric sepsis and septic shock.
Supplementary Material
Key Points.
Question Following statewide mandated care for pediatric sepsis, was the prompt completion of a 1-hour bundle associated with lower risk-adjusted in-hospital mortality?
Findings Among 1179 pediatric patients with sepsis at 54 adult and pediatric specialty hospitals in New York State, the completion of a 1-hour sepsis bundle that included blood cultures, broad spectrum antibiotics, and a 20-mL/kg fluid bolus was significantly associated with lower risk-adjusted in-hospital mortality compared with not completing the bundle within 1 hour (odds ratio, 0.59).
Meaning Timely completion of a 1-hour bundle of care may improve outcomes in pediatric sepsis.
Acknowledgments
Funding/Support: Drs Seymour and Angus were supported in part by grants from the National Institutes of Health (R35GM119519). Dr Evans is supported in part by a grant from the National Institutes of Health (T32HL007820). Dr Weiss is supported by a grant from the National Institute of General Medical Sciences (K23GM110496).
Footnotes
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Evans reported receiving grants from the National Institutes of Health. Mr Phillips received personal fees from IPRO for conduct of statistical analyses. Dr Seymour reported receiving grants from the National Institutes of Health/National Institute of General Medical Sciences and personal fees from Beckman Coulter and Edwards Inc. No other disclosures were reported.
Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: Derek C. Angus is Associate Editor of JAMA, but he was not involved in any of the decisions regarding review of the manuscript or its acceptance.
Additional Contributions: We thank the contributions of the New York State Sepsis Advisory Group and collaborating hospitals. We acknowledge the contributions of Colin Cooke, MD, University of Michigan, and Jason Kennedy, MS, University of Pittsburgh, to the quantitative bias analysis. Neither received compensation for their contributions.
Contributor Information
Idris V. R. Evans, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; The Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, Pittsburgh, Pennsylvania.
Gary S. Phillips, IPRO, Lake Success, New York.
Elizabeth R. Alpern, Division of Emergency Medicine, Department of Pediatrics, Ann and Robert H. Lurie Children's Hospital of Chicago, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Derek C. Angus, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; The Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, Pittsburgh, Pennsylvania; Associate Editor, JAMA, Chicago, Illinois.
Marcus E. Friedrich, New York State Department of Health, Albany.
Niranjan Kissoon, Department of Pediatrics, Division of Critical Care, BC Children's Hospital and Research Institute, University of British Columbia, Vancouver, British Columbia, Canada.
Stanley Lemeshow, Division of Biostatistics, College of Public Health, The Ohio State University, Columbus.
Mitchell M. Levy, Department of Medicine, Division of Pulmonary Critical Care & Sleep, Alpert Medical School of Brown University, Providence, Rhode Island.
Margaret M. Parker, Department of Pediatrics, Pediatric Critical Care Medicine, Stony Brook Children's Hospital, Stony Brook, New York.
Kathleen M. Terry, IPRO, Lake Success, New York.
R. Scott Watson, Pediatric Critical Care Medicine, Seattle Children's Hospital and Research Institute, Seattle, Washington.
Scott L. Weiss, Division of Critical Care Medicine, Department of Anesthesiology and Critical Care, The Children's Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia.
Jerry Zimmerman, Pediatric Critical Care Medicine, Seattle Children's Hospital and Research Institute, Seattle, Washington.
Christopher W. Seymour, Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; The Clinical Research, Investigation, and Systems Modeling of Acute Illness (CRISMA) Center, Pittsburgh, Pennsylvania.
REFERENCES
- 1.Balamuth F, Weiss SL, Neuman MI, et al. Pediatric severe sepsis in US children’s hospitals. Pediatr Crit Care Med. 2014;15(9):798–805. doi: 10.1097/PCC.0000000000000225 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign Guidelines Committee including The Pediatric Subgroup. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Intensive Care Med. 2013;39(2):165–228. doi: 10.1007/s00134-012-2769-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cruz AT, Perry AM, Williams EA, Graf JM, Wuestner ER, Patel B. Implementation of goal-directed therapy for children with suspected sepsis in the emergency department. Pediatrics. 2011;127(3):e758–e766. doi: 10.1542/peds.2010-2895 [DOI] [PubMed] [Google Scholar]
- 4.Larsen GY,Mecham N, Greenberg R. An emergency department septic shock protocol and care guideline for children initiated at triage. Pediatrics. 2011;127(6):e1585–e1592. doi: 10.1542/peds.2010-3513 [DOI] [PubMed] [Google Scholar]
- 5.Dwyer J. An infection, unnoticed, turns unstoppable. The New York Times. https://www.nytimes.com/2012/07/12/nyregion/in-rory-stauntons-fight-for-his-life-signs-that-went-unheeded.html. Published July11, 2012. Accessed July20, 2017. [Google Scholar]
- 6.New York State Department of Health. New York State report on sepsis care improvement initiative: hospital quality performance. https://www.health.ny.gov/press/reports/docs/2015_sepsis_care_improvement_initiative.pdf. Published March 2017. Accessed December 22, 2017. [Google Scholar]
- 7.Brierley J,Carcillo JA,Choong K,et al. Clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock: 2007 update from the American College of Critical Care Medicine [published correction appears in Crit Care Med. 2009;37(4):1536]. Crit Care Med. 2009;37(2):666–688. doi: 10.1097/CCM.0b013e31819323c6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Goldstein B, Giroir B, Randolph A; International Consensus Conference on Pediatric Sepsis. International pediatric sepsis consensus conference: definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med. 2005;6(1):2–8. doi: 10.1097/01.PCC.0000149131.72248.E6 [DOI] [PubMed] [Google Scholar]
- 9.Seymour CW, Gesten F, Prescott HC, et al. Time to treatment and mortality during mandated emergency care for sepsis. N Engl J Med. 2017;376 (23):2235–2244. doi: 10.1056/NEJMoa1703058 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Psaty BM, Koepsell TD, Lin D, et al. Assessment and control for confounding by indication in observational studies. J Am Geriatr Soc. 1999;47(6): 749–754. doi: 10.1111/j.1532-5415.1999.tb01603.x [DOI] [PubMed] [Google Scholar]
- 11.Odetola FO, Gebremariam A, Freed GL. Patient and hospital correlates of clinical outcomes and resource utilization in severe pediatric sepsis. Pediatrics. 2007;119(3):487–494. doi: 10.1542/peds.2006-2353 [DOI] [PubMed] [Google Scholar]
- 12.Watson RS, Carcillo JA, Linde-Zwirble WT, Clermont G, Lidicker J, Angus DC. The epidemiology of severe sepsis in children in the United States. Am J Respir Crit Care Med. 2003;167(5):695–701. doi: 10.1164/rccm.200207-682OC [DOI] [PubMed] [Google Scholar]
- 13.Leclerc F, Leteurtre S, Duhamel A, et al. Cumulative influence of organ dysfunctions and septic state on mortality of critically ill children. Am J Respir Crit Care Med. 2005;171(4):348–353. doi: 10.1164/rccm.200405-630OC [DOI] [PubMed] [Google Scholar]
- 14.Dombrovskiy VY, Martin AA,Sunderram J, Paz HL. Occurrence and outcomes of sepsis: influence of race. Crit Care Med. 2007;35(3):763–768. doi: 10.1097/01.CCM.0000256726.80998.BF [DOI] [PubMed] [Google Scholar]
- 15.New York State Department of Health. Statewide Planning and Research Cooperative System (SPARCS). https://www.health.ny.gov/statistics/sparcs/. Accessed May 18, 2018.
- 16.New York State Department of Health. Health facility certification information. https://health.data.ny.gov/Health/Health-Facility-Certification-Information/2g9y-7kqm/data. Accessed January 4, 2018.
- 17.Rosenberg DI, Moss MM; American College of Critical Care Medicine of the Society of Critical Care Medicine. Guidelines and levels of care for pediatric intensive care units. Crit Care Med. 2004; 32(10):2117–2127. doi: 10.1097/01.CCM.0000142704.36378.E9 [DOI] [PubMed] [Google Scholar]
- 18.Royston P, Sauerbrei W. Building multivariable regression models with continuous covariates in clinical epidemiology: with an emphasis on fractional polynomials. Methods Inf Med. 2005;44 (4):561–571.doi: 10.1055/s-0038-1634008 [DOI] [PubMed] [Google Scholar]
- 19.Dimick JB, Staiger DO, Birkmeyer JD. Ranking hospitals on surgical mortality: the importance of reliability adjustment. Health Serv Res. 2010;45(6, pt 1):1614–1629. doi: 10.1111/j.1475-6773.2010.01158.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Czaja AS, Zimmerman JJ, Nathens AB. Readmission and late mortality after pediatric severe sepsis. Pediatrics. 2009;123(3):849–857. doi: 10.1542/peds.2008-0856 [DOI] [PubMed] [Google Scholar]
- 21.Williams K, Thomson D, Seto I, et al. ; StaR Child Health Group. Standard 6: age groups for pediatric trials. Pediatrics. 2012;129(3)(suppl 3):S153–S160. doi: 10.1542/peds.2012-0055I [DOI] [PubMed] [Google Scholar]
- 22.Lash TL, Fox MP, Fink AK. Applying Quantitative Bias Analysis to Epidemiologic Data.New York, NY: Springer-Verlag; 2009. [Google Scholar]
- 23.Balamuth F, Weiss SL, Fitzgerald JC, et al. Protocolized treatment is associated with decreased organ dysfunction in pediatric severe sepsis. Pediatr Crit Care Med. 2016;17(9):817–822. doi: 10.1097/PCC.0000000000000858 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Davis AL, Carcillo JA, Aneja RK, et al. American College of Critical Care Medicine clinical practice parameters for hemodynamic support of pediatric and neonatal septic shock. Crit Care Med. 2017;45 (6):1061–1093. doi: 10.1097/CCM.0000000000002425 [DOI] [PubMed] [Google Scholar]
- 25.Kuhlmann S, Mason B, Ahlers-Schmidt CR. A quality improvement project to improve compliance with the joint commission children’s asthma care-3 measure. Hosp Pediatr. 2013;3(1):45–51. doi: 10.1542/hpeds.2012-0015 [DOI] [PubMed] [Google Scholar]
- 26.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8): 762–774. doi: 10.1001/jama.2016.0288 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.