Abstract
Objective:
Assess patient outcomes in patients with suspected infection and the cost-effectiveness of implementing a quality improvement program.
Design, Setting, and Participants:
We conducted an observational single-center study of 13,877 adults with suspected infection between March 1, 2014 and July 31, 2017. The 18-month period before and after the effective date for mandated reporting of the sepsis bundle was examined. The sequential organ failure assessment (SOFA) score and culture and antibiotic orders were used to identify patients meeting Sepsis-3 criteria from the electronic health record.
Interventions:
The following interventions were performed: (1) multidisciplinary sepsis committee with sepsis coordinator and data abstractor; (2) education campaign; (3) electronic health record tools; (4) and a modified early warning system.
Main Outcomes and Measures:
Primary health outcomes were in-hospital death and length of stay. The incremental cost-effectiveness ratio (ICER) was calculated and the empirical 95% confidence interval (CI) for the ICER was estimated from 5,000 bootstrap samples.
Results:
In multivariable analysis, the odds ratio (OR) for in-hospital death in the post- versus pre-implementation periods was 0.70 (95% CI 0.57–0.86) in those with suspected infection, and the hazard ratio for time to discharge (HR) was 1.25 (95% CI: 1.20–1.29). Similarly, a decrease in the odds for in-hospital death and an increase in the speed to discharge was observed for the subset that met Sepsis-3 criteria. The program was cost-saving in patients with suspected infection [-$272,645.7; 95% CI: -$757,970.3 – -$79,667.7]. Cost savings were also observed in the Sepsis-3 group.
Conclusions and Relevance:
Our health system’s program designed to adhere to the sepsis bundle metrics led to decreased mortality and length of stay in a cost-effective manner in a much larger catchment than just the cohort meeting the CMS measures. Our single-center model of interventions may serve as a practice-based benchmark for hospitalized patients with suspected infection.
INTRODUCTION
Sepsis is the leading cause of in-hospital mortality and the most expensive condition treated in United States (US), consuming more than $24 billion each year.1–3 The Center for Medicare and Medicaid Services (CMS) mandated the Severe Sepsis/Septic Shock (SEP-1) performance measure in the Hospital Inpatient Quality Reporting Program beginning in July 2015 to help address the high mortality and high cost associated with sepsis.4 The 3- and 6-hour bundles include early resuscitation and timely antibiotics, which are the hallmarks for improving health outcomes in patients with this condition.5 Seymour et al. demonstrated that rapid completion of the 3-hour sepsis bundle was associated with improved in-hospital mortality.6 However, little is known about the cost-effectiveness of implementing the CMS bundle within a health system.
Before the Surviving Sepsis Campaign incorporated a sepsis bundle into their guidelines, substantial differences existed between health systems for protocols in sepsis care, leading to heterogeneity in both process and outcome measures across studies.7 In addition, prior studies evaluating sepsis protocols mainly incorporated International Classification of Disease (ICD) codes to identify cases,8 but the coding incidence for sepsis over the years has changed since awareness and reporting requirements have increased.9, 10 With the advent of an operational definition for sepsis from the electronic health record (EHR) published by the Third International Consensus Definition for Sepsis and Septic Shock (Sepsis-3) and the mandating of a standardized sepsis bundle by CMS, bias within and between studies may be reduced.11
In anticipation of the SEP-1 performance measure, our institution implemented multiple interventions, including an education campaign for the 3- and 6-hour sepsis bundles, a modified early warning system in the EHR, hiring of a sepsis coordinator, provision of real-time physician feedback, and new sepsis order sets and alerts in the EHR to improve adherence. Our study aimed to assess the impact of our health system’s interventions on clinical outcomes and their cost-effectiveness across different case-definitions for sepsis, including Sepsis-3. We hypothesized that our system’s program is associated with a decrease in the case-rate of in-hospital deaths in a cost-effective manner regardless of the case-definition for infection or sepsis.
METHODS
Patient characteristics and setting
We conducted a single-center retrospective study of all inpatient adult encounters at a 547-bed tertiary academic health system between March 1, 2014 and July 31, 2017. Eligible encounters included all patients 18 years of age or older with suspected infection. The Seymour et al. definition for suspected infection was applied to data from the EHR.12 Specifically, cases were defined as patients who either had an antibiotic administered no more than 24 hours preceding a body fluid culture, or a body fluid culture obtained first with antibiotic administration within 72 hours was defined as having suspected infection. Time of infection onset was defined as the first of these two events. Patients were excluded if they did not meet the criteria for suspicion for infection. In addition, perioperative antibiotic administration, defined as six hours before or after the time of entering the operating room, and prophylactic oral antibiotics were excluded from the definition (Figure 1).
Figure 1.
Flow Diagram of Patients for Pre- and Post-Implementation
SOFA score derivation and sepsis definition
The Sequential Organ Failure Assessment (SOFA) score at time of onset of infection was calculated as the highest score in the interval from 48 hours before onset of infection to 24 hours after the onset of infection.12 The following variables for the SOFA score were extracted from the EHR: (1) partial pressure of oxygen (PaO2) over the fraction of inspired oxygen (FiO2) or peripheral oxygen saturation (SpO2) over the FiO2 if the PaO2 is not available; (2) Platelet count (x103/μl); (3) Bilirubin (mg/dL); (4) Mean arterial pressure or presser dose; (5) Glasgow coma score; (6) Creatinine (mg/dL). If values for PaO2/FiO2 or SpO2/FiO2 were not available in the medical record, it was assumed the patient did not have hypoxic respiratory failure and a score of 0 was assigned. Similarly, bilirubin was missing in 19% of results and a SOFA score of 0 was imputed for this item as a conservative estimate. All other SOFA criteria had at least 96% of the data available.
To meet Sepsis-3 criteria, a SOFA score change of greater than or equal to 2 after the onset of suspected infection from a baseline SOFA score presumed to be 0 was applied.12 The following were additional sepsis definitions included in the analysis: (1) implicit ICD codes put forth by Angus et al13; and (2) the CMS requirements in reporting, which are explicit ICD codes for sepsis, severe sepsis, and septic shock.
Sepsis quality improvement program to meet CMS bundle requirements
The program in our health system was largely implemented during the summer and fall months of 2015, when the new CMS sepsis management bundles were released and before they went into effect in October 2015. Our health center invested in the following resources: (1) provider and personnel time; (2) a system of clinician education regarding the new 3- and 6-hour sepsis bundles, (3) a full time sepsis program coordinator and apportioned a part-time (0.35 time effort) clinical data abstractor, (4) information technology (IT) interventions including a real-time physician feedback system, three order sets (ED, inpatient, initial response to sepsis), and a modified early warning system (MEWS) for clinical deterioration.14 The individual elements of the sepsis quality improvement program are detailed in Supplemental Tables 1–3. The interval between October 1, 2015 and December 31, 2015, when the interventions were being implemented, was excluded from the outcomes and cost-effectiveness analyses.
Analysis of clinical outcomes
The time to first antibiotic, calculated by subtracting the time of antibiotic completed from the time of first vital sign recorded, was used as the process measure for meeting the metrics of the CMS bundle because it is the strongest driver for mortality reduction.15 First vital sign was either from triage in the ED or first recorded in the EHR for direct hospital admissions. Reporting on bundle compliance was provided to CMS on a quarterly basis. Per CMS requirements, our center provided randomly selected cases chosen by CMS from the qualifying cohort of patients that met CMS criteria. During our study period, 6 quarters were reported with a total of 360 randomly selected eligible cases for audit by CMS. The primary clinical outcome was in-hospital death. Twenty eight-day free hospital survival was a secondary outcome. In unadjusted analyses, differences from pre-implementation to post-implementation were assessed for statistical significance with Wilcoxon rank sum tests for continuous variables and chi-square tests for proportions. In multivariable models, the following variables were included for adjustment: age, sex, race, insurance status, Elixhauser comorbidity score16, service type, calendar month, and admission SOFA score. Calendar month was added to the model to account for the greater frequency of winter months in the post-implementation period. Adjusted odds ratios for the study period associated with in-hospital death were estimated using logistic regression. To assess 28-day hospital-free survival, Cox proportional cause-specific hazard models were used to estimated hazard ratios for time to discharge as the outcome of interest; longer inpatient stays were censored at 28 days and in-hospital death was considered a competing risk in these models. Patients discharged alive prior to 28 days were assumed to be alive for their remaining days. Incremental increases in admission SOFA score were associated with longer lengths of stay with the greatest change in those with a SOFA score >5 (p<0.001) (Supplemental Figure 1). Additional subgroup analyses were performed in those with an admission SOFA score ≤5 (n=10,557, 76.1%), the non-CMS cohort (n=10,280, 74.1%), and the suspected infection cohort not meeting the Sepsis-3, Angus, or CMS definitions for sepsis (n=3,796, 27.4%).
Cost analysis and effectiveness of establishing and implementing the program
Costs were presented as one-time fixed costs for establishing the program and monthly variable costs for implementing the program. Average compensation rates for the region were used for valuing personnel time costs (Supplemental Table 1). The Hospital Care (HC) component of the Personal Health Care (PHC) Price Index published by CMS17 was used to inflate hospital charges to 2017 U.S. dollars.18. We assumed a similar inflation rate between 2016–2017 and 2015–2016 since 2015–2016 is the most recent available year published by CMS. The costs of fully implementing the program, or each component of the program, over a certain number of months entails summing the initial fixed costs with the monthly variable costs of the program, or the component, multiplied by the number of months during the evaluation period. Six observations had no hospital costs and were removed from the cost analysis.
The cost-effectiveness analysis adopted the health systems’ perspective. Results from the cost-effectiveness analysis are presented in the form of the incremental effectiveness ratio (ICER) of costs per in-hospital deaths averted. Details on the variables and charges in the equation may be found in Supplemental Table 4. Confidence intervals for the ICER estimate were calculated using 5,000 bootstrap samples.. The following scenario-based sensitivity analyses were conducted to further evaluate the cost-effectiveness of the program: (1) addition of providers’ time spent receiving training; and (2) cost in establishing a program with no onsite clinical expertise in sepsis management or informatics (Supplemental Table 5). Analyses were performed using SAS Version 9.4 (SAS Institute, Cary, NC) and Stata Version 15.1 (StataCorp, College Station, TX). The Institutional Review Board of Loyola University Chicago approved this study.
RESULTS
Patient cohort characteristic pre- and post-implementation of sepsis program
A total of 13,877 admissions to the hospital were suspected of infection during the study period. The proportion of patients with suspected infection who met criteria for sepsis using the Sepsis-3, Angus ICD codes, and CMS ICD codes were 60.2% (n=8349), 39.5% (n=5483) and 25.9% (n=3594), respectively (Figure 1). The differences in baseline characteristics before and after implementation of the sepsis quality improvement program were a decrease in proportion of white patients (59.5% vs. 56.9%, p<0.01), an increase in proportion of patients with Medicaid insurance (17.7% vs. 19.1%, p<0.01), and a lower proportion of patients admitted to a medicine service (79.2% vs. 76.3%, p<0.01) (Table 1). Elixhauser co-morbidity mortality and readmission scores did not differ between the pre- and post-implementation periods.
Table 1.
Demographics and clinical characteristics of patients with suspected infection
| Overall n=13877 | Pre-implementation¹ n=6107 | Post-implementation¹ n=7770 | p-value | |
|---|---|---|---|---|
| Age, median (IQR) | 63 (49–75) | 63 (50–75) | 62 (48–75) | 0.03 |
| Female, n (%) | 7070 (50.9) | 3114 (51.0) | 3956 (50.9) | 0.93 |
| Race, n (%) | ||||
| White | 8054 (58.0) | 3636 (59.5) | 4418 (56.9) | <0.01 |
| Black | 3908 (28.2) | 1685 (27.6) | 2223 (28.6) | |
| Other | 1915 (13.8) | 786 (12.9) | 1129 (14.5) | |
| Hispanic ethnicity, n (%) | 2344 (16.9) | 970 (15.9) | 1374 (17.7) | 0.01 |
| Health Insurance, n (%) | ||||
| Private | 3239 (23.3) | 1424 (23.3) | 1815 (23.4) | <0.01 |
| Medicare | 7709 (55.6) | 3471 (56.8) | 4238 (54.6) | |
| Medicaid | 2568 (18.5) | 1082 (17.7) | 1486 (19.1) | |
| Other | 358 (2.6) | 130 (2.1) | 228 (2.9) | |
| SOFA at infection onset4 | ||||
| Score, median (IQR) | 1 (0–3) | 1 (0–3) | 1 (0–3) | <0.01 |
| ≥ 2 points | 5200 (37.5) | 2117 (34.7) | 3083 (39.7) | <0.01 |
| Vasopressors, n (%) | 1319 (9.5) | 595 (9.7) | 724 (9.3) | 0.40 |
| Admission SOFA, median (IQR) | 3(1–5) | 2 (1–5) | 3 (1–5) | <0.01 |
| Elixhauser Mortality Score, median (IQR) | 10 (1–18) | 10 (1–18) | 10 (1–18) | 0.13 |
| Elixhauser Readmission Score, median (IQR) | 24 (12–38) | 25 (12–38) | 24 (12–38) | 0.06 |
| Co-existing conditions, n (%) | ||||
| Congestive heart failure | 2375 (17.1) | 1008 (16.5) | 1367 (17.6) | 0.09 |
| Chronic pulmonary disease | 2994 (21.6) | 1358 (22.2) | 1636 (21.1) | 0.10 |
| Renal failure | 3222 (23.2) | 1405 (23.0) | 1817 (23.4) | 0.59 |
| Liver disease | 1658 (11.9) | 656 (10.7) | 1002 (12.9) | <0.01 |
| AIDS | 121 (0.9) | 50 (0.8) | 71 (0.9) | 0.55 |
| Metastatic cancer | 822 (5.9) | 377 (6.2) | 445 (5.7) | 0.27 |
| Diabetes | 4864 (35.0) | 2129 (34.8) | 2735( 35.2) | <0.01 |
| Alcohol or drug abuse | 1368 (9.8) | 668 (10.9) | 700 (9.0) | <0.01 |
| Hypertension | 8388 (60.5) | 3684 (60.32) | 4704 (60.6) | 0.77 |
| Primary Service, n (%) | ||||
| Medicine | 10762 (77.6) | 4837 (79.2) | 5925 (76.3) | |
| Surgery | 1168 (8.4) | 500 (8.2) | 668 (8.6) | |
| Neurology | 541 (3.9) | 257 (4.2) | 284 (3.6) | <0.01 |
| Trauma | 521 (3.8) | 137 (2.2) | 384 (4.9) | |
| Other | 885 (6.4) | 376 (6.1) | 509 (6.5) | |
| ICU Utilization, n (%) | 961 (6.9) | 424 (6.9) | 537 (5.9) | 0.94 |
| 28-day hospital-free days, median (IQR) | 24 (20–25) | 23 (19–25) | 24 (20–25) | <0.01 |
| 28-day ICU-free days, median (IQR) | 28 (27–28) | 28 (27–28) | 28 (26–28) | 0.21 |
| Time to first antibiotic, median (IQR) | 3.6 (1.6–7.6) | 4.1 (1.9–9.1) | 3.3 (1.4–6.7) | <0.01 |
| In-hospital death, n (%) | 617(4.45) | 311 (5.09) | 306 (3.94) | <0.01 |
| Sepsis-3 Cohort | 565 (6.7) | 284 (8.0) | 281 (5.9) | <0.001 |
| Angus Cohort | 500 (9.1) | 259 (10.3) | 241 (8.1) | 0.01 |
| CMS Cohort | 427 (11.9) | 217 (14.6) | 210 (10.0) | <0.001 |
Time to receipt of first antibiotic between pre- and post-implementation
In those with suspected infection, the median time to first antibiotics decreased after the sepsis program implementation from 4.1 hours (IQR 1.9–9.1) to 3.3 hours (IQR 1.4–6.7) (p<0.01). In the sepsis groups, the median time to first antibiotic was shortest in the group defined by the CMS criteria with a decrease from 2.5 hours (IQR 1.1–5.9) to 1.8 hours (IQR 0.87–3.7) (p<0.01) followed by the Angus criteria where a decrease from 3.5 hours (IQR 1.5–8.3) to 2.6 hours (IQR 1.2–5.5) (p<0.01) was observed. Lastly, patients meeting the Sepsis-3 definition of sepsis had a decrease in median time to first antibiotic from 3.6 hours (IQR 1.5–9.2) to 2.6 hours (IQR 1.1–5.9) (p<0.01). The compliance reported to CMS for the 3- and 6-hour bundle during the study period was 75.23% and 41.0%, respectively.
Clinical Outcomes
In those with suspected infection, in-hospital death decreased from 5.1% in the pre-implementation to 3.9% in post-implementation period (p<0.01). The median 28-day hospital-free days increased pre- and post-implementation from 23 days (IQR 19–25) to 24 days (IQR 20–25) (p<0.01). In multivariable analysis adjusted for age, sex, race, insurance status, Elixhauser comorbidity score, service type, calendar month, and admission SOFA score, the odds ratio for in-hospital death between post- versus pre-implementation period was 0.70 (95% CI 0.57–0.86, p<0.01) in those with suspected infection. Similarly, a decrease in the adjusted odds for in-hospital death was noted for all sepsis group definitions after implementation of the sepsis quality improvement program (Figure 2a). In multivariable competing risk analysis, the hazard ratio for being discharged alive within the first 28-day days of hospitalization was 1.25 (95% CI 1.20–1.29, p<0.01) from the pre-implementation group to the post- implementation group in patients with suspected infection. An increased rate of discharge was also noted across all sepsis group definitions (Figure 2b). In subgroup analyses, the suspected infection cohort with an admission SOFA score ≤5 had the lowest OR for death at 0.42 (95% CI 0.24–0.73). A similar improvement in mortality was also found in the non-CMS subgroup, and the subgroup without a sepsis definition only had 11 cases of in-hospital death (0.3%) and did not demonstrate a change in OR for death (Supplemental Table 6).
Figures 2 a-b.
In-hospital Death and Length of Stay between pre- and post-intervention of sepsis quality program
Regression analyses controlled for intervention period, age, gender, insurance type, race, ethnicity, service category, ICU stay, month of the year, admission SOFA score, and Elixhauser mortality score
Cost effectiveness
The health system invested $130,570 to establish the sepsis program and a $152,660 monthly cost thereafter to maintain the program. Inflation-adjusted hospital charges increased for the suspected infection cohort between the pre- and post-implementation period from $39,560 (95% CI: $23,047 – $65,982) to $39,110 (95% CI: $24,425 - $70,008) (Table 2). The role of program personnel, regional compensation rates for program personnel, total hours devoted to the sepsis program, fixed costs to establish the program, and variable costs to maintain the program are described in detail in Supplemental Tables 1–3. Results from the cost-effectiveness analysis showed that the program was cost-saving in patients with suspected infection [−272,645.7 (2017) U.S. $, 95% CI: (−757,970.3 – −79,667.7]. Similar cost savings were observed in the Sepsis-3 and Angus definition groups, however, the 95% CI in these cohorts included positive ICER in the upper bound (Table 3). The CMS sepsis group had a positive ICER of $100,087.9 dollars per percent point decrease of in-hospital death (95% CI: −403,458.5 – 790,985.6). In subgroup analyses, the greatest cost-effectiveness is shown in those admitted with a SOFA score ≤5 (Table 3). Figure 3 represents the distribution of the ICER around the 95% percentile CI in patients with suspected infection. Similar figures are presented for the other cohorts in Supplemental Figure 2. In sensitivity analyses with different fixed and variable cost scenarios accounting for naïve health systems without a mature clinical decision support system, cost-effectiveness was still demonstrated (Table 3).
Table 2.
Comparison of inflation-adjusted hospital charges by intervention period, 2017 U.S. $, median (IQR)
| Cohort | Pre-intervention | Post-intervention | P-value* |
|---|---|---|---|
| Suspected infection (n=13,877) | 36,560.35 (23,047.34 – 65,981.96) | 39,110.23 (24,425.16 – 70,008.78) | <0.01 |
| Sepsis-3 (n=8,349) | 43,689.49 (26,428.48 – 81,346.85) | 46,991.55 (28,379.94 – 84,750.98) | <0.01 |
| Angus criteria (n=5,483) | 52,213.82 (30,350.7 – 97,662.98) | 54,969.71 (31,284.65 – 103,759.4) | 0.10 |
| CMS (n=3,597) | 52,785.87 (31,297.42 – 105,429.6) | 54,005.15 (31,939.45 – 101,019.4) | 0.64 |
Chi-squared test for the insurance variable and nonparametric k-sample test on the equality of medians for charges. IQR, interquartile range. There were six hospitalizations with 0 charges that were excluded from the charges calculations.
Table 3.
The incremental effectiveness ratio (ICER) of the sepsis bundle compared to usual care over a two year period for the different sepsis definition cohorts
| Cohorts | Average admissions per month (S.D) | Marginal charges* (CI) | Marginal probability* (CI) | Program effects (in-hospital death averted) | Program costs or savings ($) | ICER ($/per percent point decrease in in-hospital death) (CI) |
|---|---|---|---|---|---|---|
| Suspected infection (n=13,877) | 344.5 (89.0) | −3,050.2 (−4,725.6 – −1,374.8) | 0.009 (0.004 – 0.015) | 78.5 | −21,410,544.8 | −272,645.7 (cost saving) (−757,970.3 – −79,667.7) |
| Sepsis-3 (n=8,349) | 207.1 (57.4) | −1,977.1 (−4,650.9 – 696.7) | 0.012 (0.003 – 0.02) | 57.6 | −6,025,112.0 | −104,555.5 (cost saving) (−623,367.1 – 153,330.4) |
| Angus criteria (n=5,483) | 135.9 (31.9) | −2,571.8 (−6,418.2 – 1,274.7) | 0.019 (0.006 – 0.031) | 60.9 | −4,588,687.2 | −75,397.1 (cost saving) (−466,234.5 – 151,113.5) |
| CMS (n=3,597) | 89.1 (26.3) | −12.9 (−4,790.4 – 4,764.7) | 0.018 (0.001 – 0.034) | 37.7 | 3,771,869.8 | 100,087.9 (−403,458.5 – 790,985.6) |
| SENSITIVITY AND SUBGROUP ANALYSES | ||||||
| Scenario based sensitivity analysis | Scenario 1 for suspected infection cohort: Providers time costs | −21,345,798.8 | −271,821.3 (cost saving) (−756,607.1 – −78,866.6) | |||
| Scenario 2 for suspected infection cohort: Naïve health system | −21,388,328.8 | −272,362.8 (cost saving) (−757,502.6 – −79,392.8) | ||||
| SOFA ≤5 (n=10,557) | 269.5 (48.5) | −3,118.4 (−4,351.4 – −1,885.3) | 0.004 (0.001–0.007) | 29.1 | −16,369,191.5 | −562,800.2 (cost saving) (−1,045,036 – −80,564.7) |
| Non-CMS (n=10,280) | 255.4 (65.6) | −3,859.0 (−5,476.7– −2,241.3) | 0.006 (0.001– 0.011) | 36.8 | −19,840,823.7 | −539,671.3 (cost saving) (−1,075,545 – −3,734.4.4) |
Incremental Cost Effectiveness Ratio (ICER) = $/per percent point decrease in in-hospital death. The upper and lower limits of the bootstrap-based 95% confidence interval were obtained as the 2.5% and 97.5% percentile of the 5,000 bootstrap estimates of the ICER.
Regression analyses controlled for intervention period, age, gender, insurance type, race, ethnicity, service category, ICU stay, month of the year, admission SOFA score, and Elixhauser mortality score; * p<0.05, ** p<0.05. SOFA = sequential organ failure assessment;
Subgroup analysis done for SOFA ≤5 to represent the cohort at lower risk for death and shorter length of stay (Supplemental Figure 1). Sensitivity analyses performed with 2 scenarios to account for naïve health systems that require additional investment into training and informatics (Supplemental Table 5).
Figure 3.
The distribution of the incremental cost-effectiveness ratio around the 95% percentile CI in patients with suspected infection.
DISCUSSION
A concerted effort including coordination of care, clinical decision support in the EHR, and a sepsis education campaign led to quicker antibiotic administration and better health outcomes with lower case-rates of in-hospital death in patients with suspected infection as well as those with sepsis defined by Sepsis-3, Angus, and CMS criterion. Furthermore, cost-effectiveness analyses showed that the program was cost-saving for patients with suspected infection, with the 95% CI around the ICERs within the cost-saving boundaries. The practice changes at our center to meet the requirements of the CMS measures affected all adult patients with suspected infection and had the greatest impact on outcomes and cost in those admitted SOFA scores ≤5.
In examining health outcomes before and after the sepsis CMS care bundles, we first acknowledge that profiling hospital performance is sensitive to case definitions for sepsis, as has previously been shown across US hospitals.19, 20 The CMS policy changes led to practice changes in our center that affected a much larger catchment in all hospitalized patients with suspected infection. We examined multiple definitions for sepsis so our analysis was less sensitive to changes in coding behavior.21 With potential delays in diagnosis using the Sepsis-3 definition22, we focused our primary analysis in all adult patients with suspected infection. Our bundle compliance was similar to other centers that reported to CMS’s Hospital Compare website.23, 24 In all regards, our quality improvement program led to better adherence for early treatment of sepsis indicated by a reduction in time-to-first-antibiotic administration, a hallmark for improving mortality in sepsis care.15
The odds ratio for in-hospital death and the hazard ratios for length of stay improved across all case definitions between the pre- and post-implementation periods. Even in the group of patients with suspected infection, we demonstrate similar improvements in health outcomes with the greatest reduction in mortality in those with lower SOFA admission scores. This highlights the benefit of our quality improvement program was greatest in those arriving with less organ dysfunction. No change in mortality was found in the cohort not meeting any of the sepsis definitions, likely underpowered to detect a difference from such few cases. Despite the progress that has been made in sepsis care over the years, we show our health system continued to improve health outcomes after implementation of the quality improvement program designed to adhere with the SEP-1 measures.
The use of sepsis care protocols has previously been shown to be cost effective in a prospective design across multiple centers in Spain25 and a handful of retrospective studies in the US26–28 and Brazil29; however, none of these studies are comparable across protocol metrics, and they used older sepsis definitions. Prior estimates in cost for the CMS sepsis measures have ranged between $114,000 and $2,000,000 per year.8 We provided an estimated fixed and variable costs during both start-up and maintenance periods of the sepsis program that are near the upper limits of what others have reported; therefore, we believe the costs incurred by our health system provide a conservative estimate. In this regard, our results showed that the sepsis program is cost-saving to the health system because the program decreased hospitalization charges and decreased in-hospital deaths in patients with suspected infection after two years of implementation.
The cost-effectiveness of the program changed slightly when we restricted the analyses to different sepsis group definitions; as such, the ICER became approximately $100,000 per in-hospital death averted in the CMS group, which is also within the acceptable boundaries of cost-effectiveness.30 Nevertheless, the program was implemented in all inpatients with suspected infection and interpreting the ICER for all inpatients with suspected infection would be appropriate. We show the greatest cost-effectiveness in those admitted with less severe organ dysfunction, as demonstrated by an admission SOFA ≤5. In sensitivity analyses, the confidence interval boundaries of the ICER remained in the cost-saving and cost-effective boundaries even in sensitivity analysis for a naïve health system that may require additional startup and maintenance investment.31
Several limitations exist in our study. The results may not represent a causal relationship between our interventions and the decreased odds for in-hospital death due to possible missing confounders. The findings are based on a single center study at an academic tertiary care center and case-mix, practice behaviors, and management systems may vary substantially from other US centers. The CMS measure does not incorporate the other definitions for sepsis, including Sepsis-3; however, the measure led to practice changes at our center that highlight its beneficial effect in those with suspected infection. Another study also showed positive changes with improvements in bundle compliance after providing a computerized decision support system.32
The cost estimates are based on inflation-adjusted charges and may not translate into actual costs or provide direct comparisons for other centers. We used hospitalization charges in our ICER calculations which do not reflect true hospitalization costs or reimbursements. However, our results controlled for insurance status which may account for part of the variation between charges and reimbursements by each insurance carrier..
CONCLUSION
In a US healthcare system where performance is increasingly linked to payment and measure development is resource intensive, quality improvement programs have received little attention in the comparative effectiveness literature. We share results from a single-center model of a sepsis quality improvement program designed around the CMS sepsis care bundles to examine patient outcomes and cost effectiveness. Our healthcare delivery model may serve as a benchmark for other institutions to improve health outcomes and provide cost-effective care in patients with suspected infection or sepsis.
Supplementary Material
Copyright form disclosure:
Dr. Churpek received funding from the National Institutes of Health (NIH)/National Institute of General Medical Sciences R01 GM 123193, NIH/National Heart, Lung, and Blood Institute K08 HL121080, ATS Foundation: Recognition Award for Early Career Investigators, and from a patent pending (ARCD. P0535US.P2). He received support for article research from the NIH. The remaining authors have disclosed that they do not have any potential conflicts of interest.
REFERENCES
- 1.Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009–2014. JAMA. 2017;318:1241–1249. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Norman BC, Cooke CR, Ely EW and Graves JA. Sepsis-Associated 30-Day Risk-Standardized Readmissions: Analysis of a Nationwide Medicare Sample. Crit Care Med. 2017;45:1130–1137. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Burchardi H and Schneider H. Economic aspects of severe sepsis: a review of intensive care unit costs, cost of illness and cost effectiveness of therapy. Pharmacoeconomics. 22:793–813. [DOI] [PubMed] [Google Scholar]
- 4.Septimus EJ, Coopersmith CM, Whittle J, Hale CP, Fishman NO and Kim TJ. Sepsis National Hospital Inpatient Quality Measure (SEP-1): Multistakeholder Work Group Recommendations for Appropriate Antibiotics for the Treatment of Sepsis. Clin Infect Dis. 2017;65:1565–1569. [DOI] [PubMed] [Google Scholar]
- 5.Lee SH, Hsu TC, Lee MG, et al. Nationwide Trend of Sepsis: A Comparison Among Octogenarians, Elderly, and Young Adults. Crit Care Med. 2018;46:926–934. [DOI] [PubMed] [Google Scholar]
- 6.Seymour CW, Gesten F, Prescott HC, et al. Time to Treatment and Mortality during Mandated Emergency Care for Sepsis. N Engl J Med. 2017;376:2235–2244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Damiani E, Donati A, Serafini G, et al. Effect of performance improvement programs on compliance with sepsis bundles and mortality: a systematic review and meta-analysis of observational studies. PloS One. 2015;10:e0125827. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wall MJ and Howell MD. Variation and Cost-effectiveness of Quality Measurement Programs. The Case of Sepsis Bundles. Ann Am Thorac Soc. 12:1597–9. [DOI] [PubMed] [Google Scholar]
- 9.Rhee C, Murphy MV, Li L, Platt R, and Klompas M. Comparison of trends in sepsis incidence and coding using administrative claims versus objective clinical data. Clin Infect Dis. 2015;60:88–95. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Rubens M, Saxena A, Ramamoorthy V, et al. Increasing Sepsis Rates in the United States: Results From National Inpatient Sample, 2005 to 2014. J Intensive Care Med. 2018:885066618794136. [DOI] [PubMed] [Google Scholar]
- 11.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:801–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for Sepsis: For the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315:762–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J and Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29:1303–10. [DOI] [PubMed] [Google Scholar]
- 14.Gardner-Thorpe J, Love N, Wrightson J, Walsh S and Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl. 2006;88:571–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Levy MM, Gesten FC, Phillips GS, et al. Mortality Changes Associated with Mandated Public Reporting for Sepsis: The Results of the New York State Initiative. Am J Respir Crit Care Med. 2018. [DOI] [PMC free article] [PubMed]
- 16.van Walraven C, Austin PC, Jennings A, Quan H and Forster AJ. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47:626–33. [DOI] [PubMed] [Google Scholar]
- 17.CMS. Personal Health Care (PHC) indices for all services, Table 23 of the National Health Expenditures Accounts, produced by the Centers for Medicare & Medicaid Services. Office of the Actuary, National Health Statistics Group. 2016; 2018. [Google Scholar]
- 18.Dunn AK, Grosse SD and Zuvekas SH. Adjusting Health Expenditures for Inflation: A Review of Measures for Health Services Research in the United States. Health Serv Res. 2018;53:175–196. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Walkey AJ, Shieh MS, Liu VX and Lindenauer PK. Mortality Measures to Profile Hospital Performance for Patients With Septic Shock. Crit Care Med. 2018;46:1247–1254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Johnson AEW, Aboab J, Raffa JD, et al. Comparative Analysis of Sepsis Identification Methods in an Electronic Database. Crit Care Med. 2018;46:494–499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Prescott HC, Cope TM, Gesten FC, et al. Reporting of Sepsis Cases for Performance Measurement Versus for Reimbursement in New York State. Crit Care Med. 2018;46:666–673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Townsend SR, Rivers E and Tefera L. Definitions for Sepsis and Septic Shock. JAMA. 2016;316:457–8. [DOI] [PubMed] [Google Scholar]
- 23.Barbash IJ, Davis B and Kahn JM. National Performance on the Medicare SEP-1 Sepsis Quality Measure. Crit Care Med. 2018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Lynn NB, Gupta C, Vaaler M, Held J and Leon L. Severe sepsis 3-hour bundle compliance and mortality. Am J Infect Control. 2018;46:1299–1300. [DOI] [PubMed] [Google Scholar]
- 25.Suarez D, Ferrer R, Artigas A, et al. Cost-effectiveness of the Surviving Sepsis Campaign protocol for severe sepsis: a prospective nation-wide study in Spain. Intensive Care Med. 37:444–52. [DOI] [PubMed] [Google Scholar]
- 26.Shorr AF, Micek ST, Jackson WL Jr. and Kollef MH. Economic implications of an evidence-based sepsis protocol: can we improve outcomes and lower costs? Crit Care Med. 35:1257–62. [DOI] [PubMed] [Google Scholar]
- 27.Jones AE, Troyer JL and Kline JA. Cost-effectiveness of an emergency department-based early sepsis resuscitation protocol. Crit Care Med. 39:1306–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Assuncao MS, Teich V, Shiramizo SC, et al. The cost-effectiveness ratio of a managed protocol for severe sepsis. J Crit Care. 29:692.e1–6. [DOI] [PubMed] [Google Scholar]
- 29.Noritomi DT, Ranzani OT, Monteiro MB, et al. Implementation of a multifaceted sepsis education program in an emerging country setting: clinical outcomes and cost-effectiveness in a long-term follow-up study. Intensive Care Med. 40:182–91. [DOI] [PubMed] [Google Scholar]
- 30.Hirth RA, Chernew ME, Miller E, Fendrick AM and Weissert WG. Willingness to pay for a quality-adjusted life year: in search of a standard. Med Decis Making. 2000;20:332–42. [DOI] [PubMed] [Google Scholar]
- 31.Ubel PA, Hirth RA, Chernew ME and Fendrick AM. What is the price of life and why doesn’t it increase at the rate of inflation? Arch Intern Med. 2003;163:1637–41. [DOI] [PubMed] [Google Scholar]
- 32.Raschke RA, Groves RH, Khurana HS, et al. A quality improvement project to improve the Medicare and Medicaid Services (CMS) sepsis bundle compliance rate in a large healthcare system. BMJ Open Qual. 2017;6:e000080. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.