Assessing Variability in Hospital-Level Mortality among U.S. Medicare Beneficiaries with Hospitalizations for Severe Sepsis and Septic Shock

Abstract

Objective

To assess the variability in short term sepsis mortality by hospital among Centers for Medicare and Medicaid Services beneficiaries in the United States during 2013–2014.

Design

A retrospective cohort design

Setting

Hospitalizations from 3,068 acute care hospitals that participated in the Centers for Medicare and Medicaid Services inpatient prospective payment system in 2013 and 2014.

Patients

Medicare fee-for-service beneficiaries greater than or equal to 65 years old who had an inpatient hospitalization coded with present at admission severe sepsis or septic shock.

Interventions

None.

Measurements, Main Results

Individual level mortality was assessed as death at or within 7 days of hospital discharge and aggregated to calculate hospital-level mortality rates. We used a logistic hierarchal linear model to calculate mortality risk-adjusted for patient characteristics. We quantified variability among hospitals using the median odds ratio (MOR), and calculated risk-standardized mortality rates (risk-standardized mortality rates) for each hospital. The overall crude mortality rate was 34.7%. We found significant variability in mortality by hospital (p<0.001). The middle 50% of hospitals had similar risk-standardized mortality rates (32.7–36.9%), whereas the decile of hospitals with the highest risk-standardized mortality rates had a median mortality rate of 40.7%, compared to a median of 29.2% for hospitals in the decile with the lowest risk-standardized mortality rates. The median odds ratio (1.29) was lower than the adjusted odds ratios for several measures of patient comorbidities and severity of illness, including present at admission organ dysfunction, no identified source of infection, and age.

Conclusions

In a large study of present at admission sepsis among Medicare beneficiaries, we showed that mortality was most strongly associated with underlying comorbidities and measures of illness on arrival. However, after adjusting for patient characteristics, mortality also modestly depended on where a patient with sepsis received care, suggesting that efforts to improve sepsis outcomes in lower performing hospitals could impact sepsis survival.

Introduction

Sepsis is a complicated disease process that results from a dysregulated host response to infection [1, 2]. This broad syndrome encompasses diverse patient populations with different underlying chronic illnesses, risk factors, and infection types [3, 4]. Sepsis is a widespread public health issue, present in one in 17 hospitalizations and 1 in 3 hospital deaths [5].

In 2015, as an effort to measure and improve sepsis recognition and treatment and to standardize care, the Centers for Medicare & Medicaid Services (CMS) initiated mandatory hospital reporting of adherence to the Early Management Bundle for Severe Sepsis/Septic Shock (SEP-1), a National Quality Forum-endorsed process measure. Although the ultimate goal of efforts such as the SEP-1 measure is to improve sepsis outcomes through adherence to evidence-based sepsis treatment bundles, the underlying variability of outcomes among hospitals in the United States is unknown [6]. Assessment of sepsis outcomes, such as mortality, could further drive innovation in sepsis early identification and management.

Our primary objectives were (1) to determine individual and hospital factors associated with mortality for community-onset severe sepsis or septic shock hospitalizations, and, (2) to assess the variability in sepsis mortality among U.S. hospitals when adjusting for these factors.

Methods

Study Population

We conducted a retrospective cohort analysis using administrative data from CMS research identifiable files, including the Medicare Provider and Analysis Review (MedPAR), beneficiary, and carrier files. This work was conducted under a data use agreement between the Centers for Disease Control and Prevention (CDC) and CMS and was determined by the CDC’s Human Research Protection Office to be exempt from the regulations governing the protection of human subjects in research under 45 CFR 46.101(b). All data cleaning and analysis was performed using SAS software Version 9.4 (SAS Institute Inc., Cary, NC).

We restricted the study population to fee-for-service Medicare beneficiaries 65 years and older with a hospitalization containing an administrative code for severe sepsis (995.92) or septic shock (785.52) during 2013 and 2014 from acute care hospitals participating in the Inpatient Prospective Payment System. In order to capture the subset of patients with sepsis that occurred prior to hospital admission (i.e., community-onset sepsis), we included only hospitalizations with sepsis codes flagged as present on admission (POA) and in which the patient was admitted from home, an emergency department, or a skilled nursing facility. If a patient had multiple sepsis hospitalizations within the time frame, we sorted the observations using a random number generator and selected a single hospitalization to ensure statistical independence of observations. Of the 466,089 sepsis hospitalizations identified, 43,769 hospitalizations (9%) were removed.

Risk Factors

We obtained information regarding patient demographics (i.e., age, gender, and race) and hospitalization characteristics from the MedPar files. Underlying chronic conditions which existed prior to sepsis hospitalization were obtained from the Medicare beneficiary summary files. We identified patient’s admission source (home or a skilled nursing facility) from the administrative claim and calculated a Gagne (modified comorbidity index) score using International Classification of Diseases, 9th Edition, Clinical Modification codes from each sepsis hospitalization [7]. Common infection types were categorized into four groups based on the occurrence of POA diagnosis codes: urinary tract, pulmonary, skin and soft tissue, or abdominal infections, modified from other studies [8, 9]. Patients could have had none of these infections, one infection type, or more than one infection type present. To further adjust for severity of illness at admission, we included 10 variables that approximate POA organ failure, using diagnosis codes included in the implicit Angus definition of severe sepsis/septic shock [3], and for the presence of a POA do not resuscitate order. All diagnosis codes used are described in Supplemental Table 1. We classified previous healthcare utilization in three ways: (1) any physician claim in the 7 days prior to sepsis admission; (2) any previous hospitalization during the 30 days prior to sepsis admission date, or (3) the number of hospitalizations (for any indication) in the year prior to admission date. In order to adjust for differences in coding practices among hospitals, we calculated a ratio of the number of Medicare claims in the MedPAR files explicitly coded for severe sepsis/septic shock compared to the number of claims meeting an implicit criteria of severe sepsis/septic shock (i.e., Angus defintion) for each hospital [3,8,10].

Our primary outcome was mortality, defined as death during the hospitalization or within 7 days of discharge from the sepsis hospitalization.

Statistical Analysis

Chi-square tests were used to compare unadjusted patient mortality across categorical variables. Wilcoxon rank-sum and two-sample t tests were used to compare length of stay and age among patients who died and those who did not die, respectively.

Mortality was modeled using a logistic hierarchal linear model that risk adjusted using patient characteristics as fixed effects. We included a random intercept for each hospital to account for hospital-level clustering [11]. The model was fitted using the SAS GLIMMIX procedure for 422,320 hospitalizations with 55 fixed effect variables (patient-level and hospital coding factors) included in the model. Standard diagnostics confirmed the model fit and ruled-out multicollinearity of the explanatory variables. Estimated fixed effects were used to calculate adjusted odds ratios for all variables. A median odds ratio (MOR) for the effect of hospital was estimated using the same method as Larsen and Merlo [12] to characterize the difference between a randomly chosen patient in a high-risk facility and a similar patient (i.e., the same fixed effects) in a low-risk facility.

Crude mortality rates were calculated as the number of sepsis deaths divided by the number of sepsis hospitalizations, both overall and hospital specific. The fitted logistic hierarchical linear model was used to calculate a risk-standardized mortality rate (RSMR) for each hospital. The predicted number of deaths for each hospital was obtained as the best linear unbiased predictor from the logistic hierarchical linear model and included both the fixed effects (patient-level and hospital coding factors) and the hospital-specific random effect. The expected number of deaths for each hospital was estimated from the same model, including only the fixed effects, assuming the average hospital effect. The RSMR for each hospital was calculated from the model as the ratio of predicted and expected deaths multiplied by the overall crude mortality rate [13–15]. We re-calculated RSMRs using the same modeling strategy in a sub-analysis that excluded hospitals with 50 or fewer eligible sepsis hospitalizations (n=1004 hospitals, 34%) to identify if low-volume hospitals were influencing our results. We compared the correlation of the full-model RSMRs with the sub-analysis RSMRs using Pearson’s correlation coefficients.

Results

Of the 422,320 hospitalizations included in our analysis, 146,729 (34.7%) resulted in death on or within 7 days of hospital discharge. The median length of stay for patients who died was shorter (4 days) than for patients who survived (7 days, Wilcoxon p <0.001). Among the 3,065 hospitals included in the analysis, the number of hospitalizations with a code for POA severe sepsis/septic shock at each hospital ranged from 1 to 1,688 during the 2-year period (Median: 102, IQR: 33–215). Crude hospital level mortality ranged from 0 to 100% (median, 35.2%; IQR, 28.6% to 42.0%).

Most sepsis hospitalizations (84%) had at least one of the common infection types identified; urinary tract infections (42%) and pulmonary infections (37%) were most frequent (Table 1). Crude sepsis mortality varied by infection type. Presence of a pulmonary infection, skin soft tissue, or intra-abdominal infection were associated with significantly higher mortality (38% vs 33% without a pulmonary infection, 35.4% to 34.6% without a skin soft tissue infection, and 37% vs 34% without tan intra-abdominal infection respective; p<0.001 for all) whereas presence of a urinary tract infection was associated with significantly lower mortality (29% vs 39% without a urinary tract infection; p<0.001). Notably, 14% of sepsis hospitalizations did not have any of the four common infection types identified; this group also had significantly higher mortality (45%) than the group of patients that had at least one of the four common infection types identified (33%, p<0.001). Nearly one-third of sepsis hospitalizations (27%) had at least two common infection types identified, and crude mortality significantly increased with each additional infection type. Unadjusted mortality also crudely varied by almost all assessed characteristics (Table 2, Supplemental Table 2).

Table 1.

Percent of all of hospitalizations with each potential source of infection, and the number of potential sources of infections identified. All infections were coded as present on admission. See supplemental table 1 for list of codes used.

Potential Sources of Infection	Number of hospitalizations N= 422,320 N (% of all sepsis hospitalizations)	Mortality (% in group)	Chi-square test for association
None of the below identified	66,445 (16%)	29,954 (45%)	<0.0001
Urinary Tract1	177,862 (42%)	50,781 (29%)	<0.0001
Pulmonary1	156,124 (37%)	59,849 (38%)	<0.0001
Skin Soft Tissue1	89,788 (21%)	31,817 (35%)	<0.0001
Intra-abdominal1	69,554 (16%)	26,041 (37%)	<0.0001
Number of potential sources of infections			<0.0001
None identified	66,445 (16%)	29,954 (45%)
1	238,868 (57%)	73,699 (31%)
2	97,791 (23%)	35,000 (36%)
3	17,986 (4%)	7,515 (42%)
4	1,230 (0.3%)	561 (46%)

¹Non-mutually exclusive categories (i.e., patient may have more multiple sources of infection).

Table 2.

Description of patient characteristics for each hospitalization in entire cohort (n=422,320), and the proportion of each grouping that died at or within seven days of discharge. Tests for mortality associated with characteristics using chi-square tests for independence or two-sample t-tests (all p<0.001 unless noted). Patient chronic conditions for each hospitalization are summarized in Supplemental Table 2.

Patient Characteristics	Overall Sepsis Population	Death at or within 7 days of discharge
	N= 422,320	N= 146,729 (35%)
	N (% of sepsis hospitalizations)	N (% of each grouping who died within 7 days)
Age- years, Mean (SD)	79.2 (8.9)	80.7 (9.0)
Gender a
Female	217,367 (51%)	75,950 (35%)
Male	204,953 (49%)	70,779 (35%)
Race
White	341,643 (81%)	117,875 (35%)
Black	49,995 (12%)	18,421 (37%)
Other	30,682 (7%)	10,433 (34%)
Gagne Score (comorbidity score), Mean (SD)	5.2 (3.4)	5.6 (3.4)
<= 2	100,932 (24%)	29,526 (29%)
3–5	139,075 (33%)	47,562 (34%)
6–8	106,954 (25%)	39,636 (37%)
>8	75,359 (18%)	30,005 (40%)
Admission Source
Home/Referral	381,996 (90%)	129,296 (34%)
Skilled Nursing Facility	40,324 (10%)	17,433 (43%)
Do-Not-Resuscitate Status (POA)	88,505 (21%)	47,870 (54%)
Shock Code (POA)	211,978 (50%)	93,436 (44%)
Continuous invasive mechanical ventilation (POA)	101,748 (24%)	57,991 (57%)
Hypotension (POA)	19,340 (5%)	6,123 (32%)
Transient mental disorders (POA)	5,571 (1%)	1,604 (29%)
Acute and subacute necrosis of liver or Hepatic infarction (POA)	13,896 (3%)	8,663 (62%)
Acute kidney failure (POA)	253,359 (60%)	90,547 (36%)
Shock without mention of trauma (POA)	211,978 (50%)	93,436 (44%)
Anoxic brain damage (POA)	6,049 (1%)	4,582 (75%)
Encephalopathy (POA) b	81,309 (19%)	28,452 (35%)
Thromboycytopenia (POA)	47,157 (11%)	17,479 (37%)
Defibrination syndrome (POA)	5,359 (1%)	3,529 (66%)
Coagulat defect (POA)	6,999 (2%)	3,986 (57%)
Physician visit in previous 7 days	269,430 (64%)	98,737 (37%)
Previous hospitalization (30 days)	111,746 (26%)	49,185 (44%)
Number of hospitalizations in previous year
0	160,857 (38%)	47,535 (30%)
1	100,871 (24%)	36,473 (36%)
2–3	103,432 (24%)	40,068 (39%)
4+	57,160 (14%)	22,653 (40%)
Hospital Explicit/Implicit Ratio Mean (SD)	23.3 (8.0)	(7.9)

^a.p=0.0055

^b.p=0.0972

Nearly all characteristics showed significant positive or negative associations with mortality in our adjusted model. We ranked all fixed effects by the absolute value of their parameter estimate (for categorical fixed effects, we compared variables for presence vs absence of each characteristic, and for continuous fixed effects we compared variables for an increase equivalent to the IQR). We found the patient characteristics with the strongest association with increased odds of mortality were severity of illness measures such as POA organ dysfunctions, a POA “do not resuscitate” order, not identifying a POA source of infection, and age (Table 3, Full results in Supplemental Table 3).

Table 3.

Hierarchical logistic regression model adjusted odds ratios for individual and hospital characteristics. Characteristics are listed in ranked order by the absolute value of their parameter estimate. For continuous fixed effects the comparison is equivalent to the distance of the interquartile range. Complete list of covariates in Supplemental Table 3.

Characteristics	Comparison	Adjusted Odds Ratio	95% Confidence
POA Continuous invasive mechanical ventilation	Any vs None	3.382	3.323	3.441
POA Do not resuscitate status	Any vs None	3.327	3.268	3.387
POA Anoxic brain damage	Any vs None	2.802	2.628	2.988
POA Defibrination syndrome	Any vs None	2.558	2.399	2.729
POA Acute and subacute necrosis of liver or hepatic infarction	Any vs None	2.097	2.015	2.182
POA Coagulate defect	Any vs None	1.940	1.837	2.048
POA Shock without mention of trauma	Any vs None	1.787	1.740	1.835
Potential source of infection identified	None vs Any	1.786	1.759	1.814
Age	14 year increase	1.752	1.040	1.042
Cancer (Lung)	Any vs None	1.712	1.657	1.770
Hospitalization in previous 30 days	Any vs None	1.406	1.380	1.432
POA Intra-abdominal infection	Any vs None	1.332	1.303	1.361
POA Urinary Tract infection	Any vs None	0.759	0.745	0.773
POA Pulmonary infection	Any vs None	1.312	1.288	1.337
POA Transient mental disorders	Any vs None	0.801	0.751	0.855
Number of hospitalizations in previous year	4+ vs 0	1.211	1.178	1.245
POA Skin-soft-tissue infection	Any vs None	1.194	1.171	1.217
Anemia	Any vs None	1.188	1.161	1.215
Admission source	SNF vs Home	1.179	1.149	1.210
Number of hospitalizations in previous year	2–3 vs 0	1.174	1.149	1.200

After adjusting for patient characteristics, chronic comorbidities, and hospital coding (all variables listed in Supplemental Table 3), the test of the random component had a p value of less than 0.001, indicating necessity in the model and that there is indeed variability in mortality between individual hospitals. We calculated a MOR (1.292), indicating the odds of mortality for an average patient in a hospital with high mortality rates are 29% higher than in a hospital with low mortality rates.

After risk adjustment, the range of the hospital-level mortality rate decreased from 0–100% to 22.8–55.7% (Figure 1); the median RSMR was 34.7% (IQR, 32.7% to 36.9%). RSMRs higher than 34.7% indicate that a specific hospital in our model was predicted to have more deaths than the average hospital after adjusting for patient distribution. The median RSMR for the decile of hospitals with the highest RSMRs was 40.7%, compared to a median RSMR of 29.2% for the decile of hospitals with the lowest RSMRs (Supplemental Figure 1). Distributions of RSMRs are shown stratified by hospital characteristics including census division (highest in the Middle Atlantic, and the lowest in New England), bed size, disproportionate share status, teaching status, urban/rural location, and case-mix index (calculated by CMS based on the average diagnosis-related groups), all identified using 2013 CMS public use provider of service data files [16]), in Figure 2. RSMRs from the full sample were highly correlated (r>0.99) with the RSMRs calculated from the subset of facilities with greater than 50 sepsis hospitalizations (Supplemental Figure 2).

Figure 1.

Hospital level crude and risk standardized mortality rates for hospitalizations with a code for severe sepsis or septic shock (n=3,065). Crude mortality ranges 0 – 100%, Risk Standardized Mortality Rate (RSMR) ranges 22.8–55.7%.

Figure 2.

Interquartile distribution (Quartile 1–3, Median indicated as a point) of Risk Standardized Mortality Ratios (RSMRs) for hospitals stratified by several characteristics. The overall hospital RSMR distribution is shown at the top.

Discussion

In a large study of community-onset sepsis outcomes, encompassing the entire Medicare fee-for-service population, we showed that mortality was most strongly associated with underlying comorbidities and measures of sickness on arrival. However, adjusting for patient characteristics, mortality also modestly depended on where a patient with sepsis received care (MOR: 1.29). The decile of hospitals with the highest RSMRs represented 11% of all hospitalizations, and 13% of all deaths (19,344). Of these, 4,234 (22%) were in excess of the risk-adjusted average mortality rate, suggesting that efforts to improve sepsis outcomes in lower performing hospitals could impact sepsis survival.

Because sepsis outcomes primarily depend upon patient characteristics, their use as a proxy of hospital care effectiveness should adjust for variable risk among patients. Risk adjustment should account for differences in patient characteristics to determine if observed differences should be credited to true differences among hospitals or to differences in the populations they treat [17]. We calculated RSMRs for over 3,000 US hospitals that are both risk-adjusted for patient mix and reliability adjusted. The reliability adjustment allows us to include hospitals with small sample sizes and provide a robust estimate, verified by our sub-analysis [11]. CMS uses similar methods to measure 30-day RSMR for acute myocardial infarctions, congestive heart failure, and pneumonia [13–15]. The range of the community-onset sepsis RSMR distribution that we calculated was wider (32.9%) compared with distributions generated for these other hospital measures (10%), indicating more variation either due to variation in hospital performance or additional patient-level variability not captured in the existing model. This may be because sepsis is a broader, heterogeneous disease encompassing multiple infection types or because of more unmeasured variability in care.

Variability in mortality has been seen in other, smaller studies that assessed hospital-level mortality among patients with sepsis and similar disease physiology in academic medical centers and affiliated hospitals [18], and the U.S. Veterans Health Administration [19]. In 2013, New York mandated hospitals across the state to develop and report adherence to protocols for identification and treatment of patients with sepsis and subsequently evaluated the association of time to completion of elements of the protocol and hospital-level risk-adjusted sepsis mortality [20, 21]. New York’s evaluation found a range of hospital level mortality from 0% to 59% after risk adjustment for demographics and comorbidities, including severity of illness [20] and showed that time to completion of some protocol elements was associated with improved outcomes of patients with sepsis in New York [21]. There may also be variation in other sepsis outcomes, as shown by Norman et al [22], who used risk standardized sepsis readmission rates to show variability between hospitals. Further evaluation of risk-adjusted outcomes will provide important opportunities for assessing quality improvement if other states or jurisdictions consider similar mandates or quality improvement efforts.

This diverse nature of community-onset sepsis allows for inclusion of a wide range of Medicare patients in this study. To counter concerns that the composition of certain hospitals’ patient populations – older, sicker, poorer health status, multiple or more acute infections, and/or with more comorbidities – influences the observed crude hospital-level mortality, we adjusted for a large number of patient characteristics, including several measures for severity of illness at presentation. Our MOR showed that the odds of mortality for a patient are almost 1.3 times higher at a low-performing hospital compared to a high -performing hospital, demonstrating that unmeasured hospital factors, which may include prompt and effective triage, diagnostics, and treatment, can have a significant effect on a patient’s chance of surviving sepsis. It is important to recognize that this facility effect is smaller than other predictors of mortality that were present (or started) upon admission, including continuous invasive mechanical ventilation (three times odds of mortality), other categories of organ dysfunction (two times odds of mortality), not having an identifiable source of infection (1.8 times odds of mortality), and other markers of health status including increased patient age, history of lung cancer, and previous hospitalizations (~1.5 times odds of mortality). These associations suggest that efforts that promote patients to seek care before the onset of organ failure may be impactful in reducing sepsis mortality in this population.

Our analysis is subject to several limitations. Our cohort was restricted to adults 65 years and older and may not be generalizable to a younger adult population with fewer comorbidities or different healthcare utilization patterns. However, a recent large epidemiologic study among adult patients with sepsis reported an average age of 66 years, with approximately 70% of their sepsis population over 60 years of age [5]. Thus, our study likely captures a large portion of the total sepsis burden. The data available in administrative claims is limited by the number of conditions and comorbidities recorded and may vary by hospital-specific coding practices. However, analyzing 100% of fee-for-service CMS claims allows us a comprehensive look at a patient’s healthcare utilization and chronic conditions. We attempted to mitigate the potential effect of hospital-level sepsis coding variability by adjusting for the ratio of explicit to implicit sepsis coding by hospital in our multivariable models. We used a short time window (2013–2014) that was before initiation of both the transition to International Classification of Diseases, 10th Edition, Clinical Modification codes and the initiation of the SEP-1 measure, which has since introduced mandatory actions for the rapid evaluation and treatment of patients in this study population. We selected only hospitalizations explicitly coded with severe sepsis or septic shock, potentially excluding cases, in order to achieve maximum specificity [5, 8, 10, 23]. We limited the analysis to this short time frame to minimize the effect of trends in coding behavior in hospitals [8, 10, 23–27] and the introduction of ICD-10-CM codes may have on our results. However, this may effect usability of this methodology in future longitudinal studies. Additional studies would be helpful to understand if outcomes vary in a similar manner when using standardized surveillance definitions not based on administrative claims data [5]. We could not adjust for pathogen type and quantitative measures of severity of illness (e.g., Acute Physiology and Chronic Health Evaluation II score) that are not captured in administrative data sources; adjustment for POA infection type(s), shock, and organ dysfunctions were used to approximate disease severity. This study also cannot measure the impact of differing approaches to sepsis diagnosis and care.

As hospitals, policymakers, patient advocates and public health officials have increased attention to the identification and treatment of sepsis in U.S. hospitals, quality improvement efforts have largely focused on process measures concentrating on standardized management protocols. Outcome measures like mortality are more difficult to implement since they require careful attention to appropriate risk adjustment to avoid penalizing hospitals that provide care for sicker patients. However, outcome measures are more meaningful from the patient and provider perspective. Recent work suggests national sepsis mortality has not significantly changed from 2009 to 2014 [5], and measuring outcomes may encourage innovation in early recognition and care. Our study illustrates that although risk-adjusted mortality for community-onset sepsis is similar among many U.S. hospitals, the source of variation in hospital performance requires further study. Additional studies are needed to understand how SEP-1 and other sepsis care initiatives are influencing mortality over time. We caution that due to the limitations of administrative data, these results should be validated in additional time periods and ideally with datasets that include clinical and laboratory information. Nonetheless, this study suggests that risk-adjusted measures of sepsis mortality could provide insights that would inform our understanding of variability in sepsis care outcomes.