Risk-standardized sepsis mortality map of the United States

Abstract

Objective

Sepsis is the leading cause of in-hospital mortality in the United States (US). Quality improvement initiatives for improving sepsis care depend on accurate estimates of sepsis mortality. While hospital 30-day risk-standardized mortality rates have been published for patients hospitalized with acute myocardial infarction, heart failure, and pneumonia, risk-standardized mortality rates for sepsis have not been well characterized. We aimed to construct a sepsis risk-standardized mortality rate map for the United States, to illustrate disparities in sepsis care across the country.

Methods

This cross-sectional study included adults from the US Nationwide Inpatient Sample who were hospitalized with sepsis between 1 January 2010 and 30 December 2011. Hospital-level risk-standardized mortality rates were calculated using hierarchical logistic modelling, and were risk-adjusted with predicted mortality derived from (1) the Sepsis Risk Prediction Score, a logistic regression model, and (2) gradient-boosted decision trees, a supervised machine learning (ML) algorithm.

Results

Among 1,739,033 adults hospitalized with sepsis, 50% were female, and the median age was 71 years (interquartile range: 58–81). The national median risk-standardized mortality rate for sepsis was 18.4% (interquartile range: 17.0, 21.0) by the boosted tree model, which had better discrimination than the Sepsis Risk Prediction Score model (C-statistic 0.87 and 0.78, respectively). The highest risk-standardized mortality rates were found in Wyoming, North Dakota, and Mississippi, while the lowest were found in Arizona, Colorado, and Michigan.

Conclusions

Wide variation exists in sepsis risk-standardized mortality rates across states, representing opportunities for improvement in sepsis care. This represents the first map of state-level variation of risk-standardized mortality rates in sepsis.

Background

Over the past two decades, the Surviving Sepsis Campaign ¹ and a multitude of quality improvement initiatives have aimed to standardize sepsis management and improve bundle compliance.^2,3 Central to the outcome evaluation of these initiatives are accurate estimates of sepsis mortality, as mortality reflects quality of care, and is the ultimate meaningful outcome to patients. Risk-standardized mortality rates (RSMRs) can reduce the confounding effect of case mix (e.g. age, comorbidity burden) and assess regional variation in sepsis mortality more precisely.

State-based differences in RSMR for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PNA) have been visualized based on public reporting from the Centers for Medicare and Medicaid Services (CMS),^4–7 but have not been performed for sepsis (eTable 1). Using similar methods, several studies demonstrated the inequality in sepsis care by reporting the mortality variation between hospitals. Hatfield et al. reported a difference of 11.5 percentage points in median sepsis mortality rate between the highest and lowest decile of hospitals. ⁸ Wang et al. calculated the observed-to-expected mortality ratio in severe sepsis using administrative data from the University Health System Consortium. ⁹ Darby et al. revealed that a hospital-specific RSMR for sepsis ranged from 12.2% to 24.5% using hospital discharge data only from the state of Pennsylvania. ¹⁰ Walkey et al. also found a wide difference in RSMR in septic shock by both implicit diagnosis and explicit ICD codes. ¹¹ However, these studies used mortality rates that were not risk-adjusted, employed traditional statistical regression techniques, and were restricted to populations at specific medical centers or age groups.

Thus, our proposed method differs from previous sepsis mortality measurement in two fundamental aspects. First, our RSMR estimates were based on a nationwide database with the integration of a supervised machine learning (ML)-based model allowing for relationships not based on conventional regression. Second, unlike previous studies which performed comparisons at a hospital level, we constructed a national sepsis mortality map of the United States to illustrate the geographic variation in sepsis quality of care.

Methods

Data sources

We extracted data from the 2010 and 2011 years of the US Nationwide Inpatient Sample (NIS), the largest all-payer, public claims database of inpatient encounters. The NIS contains a 20% stratified sample of all short-term, non-federal, non-rehabilitation hospitals, representing 5–8 million discharges per year. We applied the trend weights developed by the Healthcare Cost and Utilization Project to weight patient-level discharge data, to derive estimates for >35 million hospitalizations nationally. We could not use more recent data because the NIS did not provide state-specific identifiers after 2011.

Ethics approval

This study was approved by the Institutional Review Board at National Taiwan University Hospital, which waived the requirement for informed consent from patients because of the anonymous nature of this public database.

Case selection and definitions

In accordance with the Sepsis-3 definition, sepsis was defined as life-threatening organ dysfunction caused by a dysregulated host response to infection. ¹² Sepsis hospitalizations were identified by Martin and Moss’ validated International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes or procedure codes from the electronic health record. ¹³ We selected patients with any diagnosis of sepsis (995.92 severe sepsis, 785.52 septic shock) or systemic infection (038.* septicemia, 020.0 sepsis, 790.7 bacteremia, 117.9 disseminated fungal infection, 112.5 disseminated candidiasis, or 112.81 disseminated fungal endocarditis) and a diagnosis of acute organ dysfunction in any of seven organ systems: cardiovascular, respiratory, central nervous, hematologic, hepatic, renal and metabolic systems. We excluded patients not meeting Martin's sepsis criteria, missing mortality information, below 18 years of age, or who were transferred to a short-term hospital against medical advice. We collected demographic information, pre-existing comorbidities, and in-hospital mortality from claims records of the index hospitalization. Comorbidities, based on Gagne's validated ICD-9-CM codes, ¹⁴ were not included in the risk adjustment unless they were also present in the 12 months before admission. Since the NIS lacks a code for ICU admission, mechanical ventilation >1 day was used as a proxy for ICU stay. ¹⁵

Outcome

Thirty-day in-hospital crude mortality rate was calculated based on length of stay and in-hospital mortality data for each patient. The primary outcome was 30-day in-hospital risk-standardized all-cause mortality rate, as described below.

Descriptive statistics

Continuous variables were summarized as mean with standard deviation or median with interquartile range (IQR). Categorical variables were summarized as number and proportion. All analyses were performed in R (version 3.5, Vienna, Austria).

Risk-standardized mortality rates

Hierarchical logistic regression models were used to estimate hospital-level 30-day all-cause RSMRs for sepsis. This approach takes into account the two-level (patient and hospital) hierarchical structure of the data to account for patient clustering within hospitals. Each model includes age, sex, hospital-specific random effects (representing baseline mortality risk at that hospital), predicted mortality and an intercept. Higher RSMRs represent worse outcomes. RSMRs in our study are expressed as percentages except where otherwise noted as fractions of one.

Feature analysis and selection

For the reference model, we fit a logistic regression model using the widely-used Sepsis Risk Prediction Score model (SRS), which assigns patients a different number of points for each variable listed in Table 1. ¹⁵ In the feature analysis step of the construction of the ML model, we used the same variables in Table 1 and performed multiple logistic regression to obtain adjusted odds ratios (aOR) in our sample. For each variable, all remaining covariates were mutually adjusted.

Table 1.

Baseline characteristics of inpatients with sepsis in 2010–2011, with adjusted odd ratios for all-cause mortality by demographic and clinical characteristics.

Variables	N (%)	OR (95% CI)
Demographics
Age < 40 (reference)	96,717 (5.56%)	Reference
Age 40–49	122,646 (7.05%)	1.48 (1.39–1.57)
Age 50–59	254,725 (14.65%)	1.81 (1.72–1.92)
Age 60–69	343,597 (19.76%)	2.17 (2.05–2.29)
Age 70–79	391,463 (22.51%)	2.86 (2.71–3.02)
Age 80–89	402,816 (23.16%)	4.17 (3.95–4.40)
Age 90 +	127,069 (7.31%)	6.12 (5.76–6.49)
Female sex (vs male sex)	876,769 (50.42%)	1.07 (1.05–1.09)
White/Other (reference)	1337,749 (76.92%)
African-American	262,360 (15.09%)	1.05 (1.02–1.08)
Hispanic	138,924 (7.99%)	0.99 (0.96–1.03)
Measures of acute illness severity (presence vs absence)
Early mechanical ventilation	304,659 (17.52%)	4.63 (4.53–4.75)
Late mechanical ventilation	88,067 (5.06%)	5.72 (5.43–6.02)
Shock	778,820 (44.78%)	2.06 (2.02–2.10)
Hemodialysis	202,522 (11.65%)	1.43 (1.39–1.48)
ICU care (at least one day)	167,975 (9.66%)	1.16 (1.11–1.20)
Comorbidities (presence vs absence)
Anemia	610,061 (35.08%)	0.64 (0.63–0.65)
Depression	170,290 (9.79%)	0.80 (0.78–0.83)
Diabetes	597,404 (34.35%)	0.86 (0.84–0.87)
Drug and substance abuse	41,357 (2.38%)	0.77 (0.72–0.82)
Chronic lung disease	433,508 (24.93%)	0.93 (0.91–0.95)
Congestive heart failure	443,337 (25.49%)	1.07 (1.04–1.09)
Hypertension	992,158 (57.05%)	0.78 (0.77–0.80)
Hypothyroid disease	225,333 (12.96%)	0.90 (0.88–0.93)
Liver disease	100,119 (5.76%)	2.07 (2.00–2.15)
Lymphoma	36,659 (2.11%)	1.62 (1.53–1.71)
Metastatic carcinomas	88,114 (5.07%)	2.78 (2.68–2.88)
Neurological conditions	271,067 (15.59%)	0.94 (0.92–0.97)
Obesity	185,958 (10.69%)	0.72 (0.70–0.75)
Paraplegia	121,273 (6.97%)	0.81 (0.78–0.84)
Perivascular conditions	172,268 (9.91%)	1.29 (1.25–1.33)
Psychiatric diseases	91,413 (5.26%)	0.71 (0.68–0.74)
Pulmonary-circulatory disorders	98,785 (5.68%)	1.14 (1.10–1.19)
Renal failure, chronic	585,859 (33.69%)	0.98 (0.96–1.00)
Malignant solid tumors	68,063 (3.91%)	1.59 (1.53–1.66)
Weight loss	352,449 (20.27%)	0.91 (0.89–0.93)

^a Early mechanical ventilation = 0–2 days from admission.

^b Late mechanical ventilation > 2 days from admission.

Selection, training, and validation of the ML model

Gradient boosted decision tree (GBDT; hereafter referred to as the “boosted tree”) is an ensemble supervised learning method that builds an additive model of decision trees in a “greedy” stage-wise process estimated by gradient descent. ¹⁶ “Boosting” refers to the combination of weak prediction trees to produce a stronger prediction. We constructed a GBDT model with the xgboost package in R to predict state-level RSMR using the variables identified above to associate with mortality. ¹⁷ We used a maximum depth of 6 and a gamma of 0. We used 580 variables as estimators during training, and 100 variables during testing. The 2010 data was used for training with 5-fold primary validation. The 2011 data was used for secondary validation (eFigure 4). To determine the degree of each variable's influence in the boosted tree model, we aggregated the reduction in error resulting from use of that variable as the splitting variable over all trees.

Evaluation of the two models

The goodness of fit of the two models was examined using Harrell's C-statistic, the R-square, and the area under the receiver operator characteristic curve (AUROC).

Construction of national RSMR maps

Hospital-level RSMRs were analyzed for 45 states. Data for five states (Idaho, Alabama, Hawaii, New Hampshire, and Delaware) was unavailable or incomplete in 2010–2011. States were classified by their Bureau of Economic Analysis regions (eTable 4) as the US Census administrative divisions were not adequately descriptive in name and did not adequately describe the “sepsis belt.” Maps were plotted using our in-house dashboard. ¹⁸ For each state, inter-model RSMR differences were calculated by subtracting RSMRs of the SRS model from RSMRs of the boosted tree model (eFigure 3).

Hospital-level comparisons

The range of crude mortality rate and RSMR were visualized with box plots. Hospitals were divided into deciles by RSMR to examine the range in RSMR. Hospitals were divided into quintiles by annual number of sepsis admissions to examine the range in hospital volume, visualized by histograms. Differences in RMSR and sepsis admission counts across quintiles of sepsis admissions were tested using the Kruskal–Wallis test.

Results

Study population and encounter characteristics

Among 1,739,033 inpatients with sepsis identified in the NIS (eFigure 1), 15% were African-American, 8% were Hispanic, and 77% were White or Other; 50% were female (Table 1). Mortality increased with age, with 80–89-year-olds carrying an aOR for mortality of 4.17 (confidence interval (CI): 3.95–4.40) compared to <40-year-olds (Table 1). Mortality was slightly higher in women than men (aOR 1.07; CI: 1.05–1.09), and in African-American patients than White patients (aOR 1.05; CI: 1.02–1.08). Mortality was higher in patients requiring early mechanical ventilation (aOR 4.63; CI: 4.53–4.75) and late mechanical ventilation (aOR 5.72; CI: 5.43–6.02) than patients not requiring ventilation. Mortality was also higher in patients who had shock (aOR 2.06; CI: 2.02–2.10) or who required hemodialysis (aOR 1.43; CI: 1.39–1.48).

Two models of RSMR

Using the variables above, we constructed a reference model for predicted mortality rate using SRS logistic regression, and an alternative model for predicted mortality rate using boosted trees, for estimating 30-day RSMR of sepsis. In this boosted tree model, the variable with the highest influence on prediction was the need for intubation and ventilation, with a weight of 0.24 (eFigure 4), followed by age, respiratory failure, cardiac arrest and ventricular fibrillation, and shock (eTable 3).

After risk adjustment, the range of the hospital-level mortality narrowed from 0–100% to 4.7–58.2% (eFigure 6). The median RSMR for the top and bottom decile of hospitals by RSMR was 13.5% and 25.8%, respectively (eFigure 7). In the SRS model, median RSMR was 18.5% (IQR: 17.1–20.6%, representing a 3.4% difference between the 25th and 75th percentiles; Table 2). In the boosted tree model, median RSMR was 18.4% (IQR 17.0–21.0%, representing a 4.0% difference). RMSRs higher than 18.5% and 18.4% in the respective models indicate that the mortality from sepsis is higher than expected after risk adjustment.

Table 2.

Assessment of model fit for the sepsis risk prediction score (SRS) model and the boosted tree model in predicting median 30-day risk-standardized mortality rate (RSMR) of sepsis among inpatients hospitalized with sepsis in 2011.

	SRS	Boosted tree
Median 30-day RSMR (25th–75th %ile), %	18.46 (17.13–20.57)	18.39 (16.99–21.01)
Mean 30-day RSMR (SD), %	18.84 (3.20)	19.04 (3.86)
C-statistic	0.78	0.87
R-square	0.14	0.25
AUPRC	0.47	0.62
PPV a	0.60	0.69

Abbreviations: AURPC: area under the precision-recall curve; PPV: positive predictive value; %ile: percentile; SD: standard deviation.

^a Under a predefined cutoff of 0.5.

Goodness of fit of RSMR models

The boosted tree model demonstrated better calibration than the SRS model (eFigure 8). The boosted tree model also had better discrimination (AUROC: 0.87 vs 0.78; AUPRC: 0.62 vs 0.47; Table 2 and eFigure 2) and higher positive predictive value (0.69 vs 0.60) than the SRS model. Having established its improved calibration and discrimination compared to the reference SRS model, RSMR estimates in the Discussion will refer to those done by the boosted tree method, unless otherwise specified.

Geographic variation in risk-standardized all-cause mortality by state

Next, we calculated median 30-day RSMR at the state level using both the SRS model and the boosted tree model (Table 3). State-level RSMRs ranged from 14% to 23% in both models. The SRS model found clusters of high RSMR states in the Southeast (West Virginia, Virginia, Arkansas, Mississippi), New England (Massachusetts, Connecticut), and individual states such as Wyoming (Mountain), Ohio (Great Lakes), North Dakota (Plains), and Alaska (Far West). (Figure 1(a)). The boosted tree found clusters of high RSMR states in the northern Mountain states of Wyoming and Montana, the Southeast (Mississippi, West Virginia, Arkansas, Kentucky), and New England (Rhode Island, Maine, and Massachusetts), and individual states such as North Dakota (Plains) and Alaska (Far West) (Figure 1(b)). West Virginia, Massachusetts, Wyoming, Ohio, North Dakota, Connecticut, Arkansas, New York, Missouri, Alaska, Georgia, Rhode Island, Montana, Utah, Maryland, Nevada, Maine, Tennessee, Minnesota, Oklahoma, Kentucky, Indiana, New Jersey, and Vermont were above the median national RSMR in both models. The lowest RSMR states were identical in both models: Arizona (Southwest), Colorado (Mountain), and Michigan (Great Lakes). Differences between crude mortality and RSMR for each state are shown in Table 4.

Table 3.

State-level median 30-day risk-standardized mortality rate (RSMR) of sepsis predicted by the Sepsis Risk Prediction Score (SRS) model (left) and the boosted tree model (right) among inpatients hospitalized with sepsis in 2011, ranked in order of decreasing RSMR.

	Sepsis Risk Prediction Score model			Boosted tree model
Rank	State	Median 30-day RSMR (25th–75th %ile)	Region	State	Median 30-day RSMR (25th-75th %ile)	Region
1	WV	23.35 (21.22–26.19)	Southeast	WY	23.19 (20.33–24.87)	Mountain
2	WY	21.97 (20.33–23.00)	Mountain	ND	20.45 (18.48–22.42)	Plains
3	MA	20.94 (18.47–24.27)	New England	MS	20.43 (18.32–22.09)	Southeast
4	OH	20.69 (17.76–22.44)	Great Lakes	RI	20.13 (18.25–22.33)	New England
5	ND	20.44 (19.80–21.07)	Plains	ME	20.11 (15.36–21.55)	New England
6	VA	20.09 (18.01–22.68)	Southeast	MA	20.08 (17.95–21.47)	New England
7	CT	19.61 (15.78–21.51)	New England	MT	20.08 (17.94–21.23)	Mountain
8	AR	19.52 (18.37–21.49)	Southeast	WV	20.00 (18.71–22.76)	Southeast
9	NY	19.49 (17.56–21.09)	Mideast	AR	19.96 (18.28–23.41)	Southeast
10	MS	19.47 (18.02–21.59)	Southeast	KY	19.94 (17.04–23.18)	Southeast
11	AK	19.37 (18.36–20.38)	Far West	MD	19.89 (18.68–21.48)	Mideast
12	GA	19.28 (17.68–20.92)	Southeast	AK	19.80 (19.17–20.44)	Far West
13	RI	19.25 (17.29–21.80)	New England	NJ	19.79 (18.16–22.92)	Mideast
14	MT	19.24 (18.63–20.07)	Mountain	NV	19.71 (18.00–21.10)	Far West
15	UT	19.17 (17.51–22.55)	Mountain	GA	19.69 (17.28–21.20)	Southeast
16	MD	19.12 (16.35–20.18)	Mideast	VT	19.57 (18.93–19.92)	New England
17	NV	19.09 (17.39–20.08)	Far West	NY	19.50 (17.27–21.67)	Mideast
18	ME	19.07 (16.37–20.20)	New England	IN	19.44 (16.70–22.41)	Great Lakes
19	TN	19.04 (17.80–21.88)	Southeast	OH	18.93 (16.68–21.05)	Great Lakes
20	MN	18.95 (18.02–20.46)	Plains	KS	18.91 (17.61–22.39)	Plains
21	OK	18.94 (17.98–22.55)	Southwest	MN	18.83 (18.04–21.52)	Plains
22	KY	18.87 (17.11–21.24)	Southeast	OK	18.75 (17.92–20.50)	Southwest
23	IN	18.84 (16.99–20.09)	Plains	UT	18.72 (17.34–20.46)	Mountain
24	NC	18.77 (17.21–20.03)	Southeast	PA	18.68 (16.61–20.56)	Mideast
25	NJ	18.75 (17.14–21.72)	Mideast	CT	18.68 (15.85–21.71)	New England
26	VT	18.62 (18.51–19.22)	New England	LA	18.66 (16.87–21.08)	Southeast
27	NM	18.52 (18.36–22.24)	Southwest	TN	18.59 (17.06–22.68)	Southeast
28	KS	18.46 (17.75–20.82)	Plains	OR	18.44 (16.83–19.42)	Far West
29	CA	18.39 (16.74–20.75)	Far West	MO	18.42 (17.29–21.39)	Plains
30	OR	18.37 (17.31–18.98)	Far West	VA	18.30 (17.54–22.04)	Southeast
31	MO	18.30 (17.09–21.02)	Plains	FL	18.12 (15.32–20.71)	Southeast
32	PA	18.23 (16.00–20.85)	Mideast	SD	18.11 (15.64–21.01)	Plains
33	LA	18.23 (16.30–19.86)	Southeast	IA	18.11 (17.38–19.17)	Plains
34	NE	18.21 (18.13–18.31)	Plains	TX	18.10 (16.54–20.44)	Southwest
35	FL	18.19 (15.46–20.59)	Southeast	NE	18.10 (17.93–18.34)	Plains
36	SD	18.19 (15.90–19.51)	Plains	CA	18.04 (16.04–20.09)	Far West
37	IL	18.09 (16.68–19.84)	Great Lakes	NM	18.00 (16.96–22.02)	Southwest
38	IA	18.07 (17.82–18.97)	Plains	NC	17.89 (16.71–20.52)	Southeast
39	SC	18.02 (16.51–19.15)	Southeast	SC	17.88 (17.09–19.77)	Southeast
40	WA	18.01 (16.77–19.02)	Far West	WA	17.86 (15.89–19.17)	Far West
41	TX	17.98 (16.15–20.13)	Southwest	WI	17.86 (13.91–19.34)	Great Lakes
42	WI	17.97 (15.25–19.55)	Great Lakes	IL	17.66 (16.90–19.94)	Plains
43	MI	17.70 (15.52–18.80)	Great Lakes	MI	17.54 (15.79–18.48)	Great Lakes
44	CO	17.49 (14.26–18.44)	Mountain	CO	17.32 (13.05–18.58)	Mountain
45	AZ	14.41 (11.45–17.82)	Southwest	AZ	14.29 (10.84–17.36)	Southwest

Fig. 1.

State-level median 30-day risk-standardized mortality rate (RSMR) of sepsis predicted by the sepsis risk prediction score (SRS) model (top) and boosted tree model (bottom) among inpatients hospitalized with sepsis in 2011, color coded by sextile. Note that data for Idaho, Alabama, Hawaii, New Hampshire, and Delaware was unavailable or incomplete in 2010–2011 and are thus colored in white.

Table 4.

Difference between crude and risk-standardized 30-day all-cause mortality for each state in 2011 using the boosted tree model.

Rank	State (Abbreviation)	Median 30-day RSMR (25th–75th %ile) %	Median 30-day crude all-cause mortality (25th–75th %ile) %	Relative difference, %
1	WV	23.19 (20.33–24.87)	17.68 (14.66–22.92)	11.57
2	WY	20.45 (18.48–22.42)	30.77 (20.00–45.45)	−32.71
3	MA	20.43 (18.32–22.09)	25.81 (21.89–30.05)	−28.56
4	OH	20.13 (18.25–22.33)	15.74 (12.44–21.16)	16.84
5	ND	20.11 (15.36–21.55)	36.06 (22.12–50.00)	−76.34
6	VA	20.08 (17.95–21.47)	20.71 (16.71–22.75)	−13.16
7	CT	20.08 (17.94–21.23)	18.12 (14.68–23.79)	3.00
8	AR	20.00 (18.71–22.76)	18.69 (14.42–33.33)	6.35
9	NY	19.96 (18.28–23.41)	23.27 (18.83–29.48)	−19.34
10	MS	19.94 (17.04–23.18)	19.46 (15.69–26.23)	4.75
11	AK	19.89 (18.68–21.48)	30.00 (10.00–50.00)	−51.51
12	GA	19.80 (19.17–20.44)	18.18 (14.89–22.78)	7.68
13	RI	19.79 (18.16–22.92)	25.83 (22.51–27.63)	−28.30
14	MT	19.71 (18.00–21.10)	17.78 (14.29–50.00)	11.45
15	UT	19.69 (17.28–21.20)	16.02 (10.00–20.00)	14.43
16	MD	19.57 (18.93–19.92)	19.49 (16.12–22.27)	2.01
17	NV	19.50 (17.27–21.67)	25.00 (18.73–27.18)	−26.83
18	ME	19.44 (16.70–22.41)	21.54 (9.09–33.33)	−7.09
19	TN	18.93 (16.68–21.05)	18.38 (13.49–24.14)	1.15
20	MN	18.91 (17.61–22.39)	16.09 (8.33–25.00)	14.53
21	OK	18.83 (18.04–21.52)	14.29 (0.00–20.00)	23.79
22	KY	18.75 (17.92–20.50)	20.00 (14.29–30.32)	−0.31
23	IN	18.72 (17.34–20.46)	17.97 (12.82–23.08)	7.56
24	NC	18.68 (16.61–20.56)	18.13 (12.09–20.00)	−1.31
25	NJ	18.68 (15.85–21.71)	24.28 (21.00–27.99)	−22.71
26	VT	18.66 (16.87–21.08)	21.01 (15.48–29.34)	−7.33
27	NM	18.59 (17.06–22.68)	21.90 (15.97–24.26)	−21.71
28	KS	18.44 (16.83–19.42)	16.33 (4.69–34.17)	13.61
29	CA	18.42 (17.29–21.39)	19.10 (15.79–24.04)	−5.88
30	OR	18.30 (17.54–22.04)	16.33 (9.09–18.88)	11.41
31	MO	18.12 (15.32–20.71)	16.97 (8.70–24.84)	7.90
32	PA	18.11 (15.64–21.01)	18.03 (14.29–23.81)	3.47
33	LA	18.11 (17.38–19.17)	18.49 (15.56–24.09)	0.91
34	NE	18.10 (16.54–20.44)	0.00 (0.00–11.11)	100.00
35	FL	18.10 (17.93–18.34)	18.04 (15.27–22.99)	0.46
36	SD	18.04 (16.04–20.09)	8.87 (0.00–25.00)	51.04
37	IL	18.00 (16.96–22.02)	17.46 (12.5–22.35)	1.11
38	IA	17.89 (16.71–20.52)	8.86 (0.00–16.67)	51.10
39	SC	17.88 (17.09–19.77)	18.30 (13.39–20.00)	−2.38
40	WA	17.86 (15.89–19.17)	15.05 (0.00–16.92)	15.77
41	TX	17.86 (13.91–19.34)	17.35 (11.43–22.12)	4.13
42	WI	17.66 (16.90–19.94)	12.50 (8.70–16.67)	30.00
43	MI	17.54 (15.79–18.48)	12.50 (9.52–20.57)	28.74
44	CO	17.32 (13.05–18.58)	15.38 (7.28–20.58)	11.16
45	AZ	14.29 (10.84–17.36)	11.18 (7.25–16.58)	21.82

Hospital volume and risk-standardized all-cause mortality

To determine if RSMR was affected by hospital volume, we divided hospitals into quintiles by their number of sepsis admissions per year. Hospitals in the lowest and highest quintiles had a median of 3 (IQR: 1–4) and 547 (IQR: 547) sepsis admissions per year, respectively (eFigure 7). The median 30-day RSMR differed significantly across the 5 quintiles (p < 0.001), but this association was not linear (eTable 2, eFigure 5).

Discussion

In this study of 1,739,033 inpatients with sepsis drawn from a nationally-representative database, we constructed the first state-level estimates of risk-standardized all-cause mortality rates for sepsis patients in the United States. We employed a traditional logistic SRS model as the reference and constructed a supervised ML model to increase the accuracy of risk prediction, the latter of which demonstrated higher model discrimination and calibration. We further built an online dashboard to illustrate time-dependent RSMR changes by state. ¹⁸ We found significant inter-state variation in RSMR, suggesting that quality of sepsis care is not uniform across the country. These risk-standardized findings suggest that there is opportunity to improve sepsis-related survival in many states, and can inform policymaking.

Because sepsis outcomes are highly dependent on patient characteristics and comorbidities, it is important to develop accurate estimates of mortality risk with which to standardize observed mortality, in order to differentiate between whether differences in sepsis-related mortality are due to differences in patient distributions or differences in quality of care. Investment to improve sepsis care should be prioritized in states such as Wyoming, North Dakota, Mississippi, Rhode Island, Maine, Massachusetts, and Montana, which are in the top sextile of RSMR. At the extremes, Wyoming had an RMSR of 23.2% while Arizona had an RSMR of 14.3%, which were 4.8% above and 4.1% below the national median RSMR, respectively. Arizona, Colorado, Michigan, Illinois, Wisconsin, Washington, and South Carolina, in the bottom sextile of RSMR, should be studied to identify success factors and processes that can be replicated at hospitals in other states.

Our work geographically refines RSMR estimates by Hatfield, ⁸ who coded facilities at the four-region level (e.g. Midwest, Northeast, Southwest, and West), and consequently does not identify individual high-performing and low-performing states. Wang's ¹⁹ state-level analysis of mortality was age-standardized but not risk-standardized to comorbidities or in-hospital factors, and identified a cluster of states in the Southeast and Mid-Atlantic (Arkansas, Louisiana, Mississippi, Alabama, Georgia, Tennessee, North Carolina, Virginia, Maryland, Delaware, and New Jersey) with the highest age-standardized sepsis mortality (ASMR). These states were all found to have RSMR above the national median in our analysis (except Alabama for which we had no data). Moore's ²⁰ county-level analysis of ASMR found three hotspots with the highest ASMR: Central Appalachia (Kentucky, Tennessee, Virginia), Middle Georgia, and Mississippi Valley (Mississippi, Arkansas, Louisiana). These hotspots, termed the “sepsis mortality belt,” ²¹ were also identified in our analysis as areas of high RSMR. We additionally identified individual states in the Northwest, including Montana, Wyoming, and North Dakota, with high RSMR that were not identified in Wang's or Moore's ASMR analyses.

The variation of care quality between states and hospitals is an important issue for policymakers to address for reducing health disparities. The advent of standardized measurement, required public reporting, Joint Commission accreditation requirements, Medicare payment incentives, and the threat of Medicare payment penalties, have dramatically improved mortality in AMI, HF, and PNA. ²² Specific hospital-based initiatives have been shown to be associated with lower AMI RSMR ²³ There is increasing support for reporting HF readmission rates in the context of RSMR. ²⁴ There is reason to be optimistic that focusing RSMR as the outcome indicator of sepsis care quality will improve patient health. In New York, the implementation of mandatory hospital reporting of adherence to sepsis protocol initiation and completion of 3 hour and 6 hour sepsis bundle requirements led to increased compliance with these metrics, and reduced RSMR from 28.8% to 24.4%. ²⁵ More rapid completion of the 3 hour sepsis bundle was associated with reduced adjusted mortality. ²⁶ Besides improving the accessibility and the quality of health care, policymakers should also pay attention to other factors that will affect the regional disparity of sepsis outcome, such as the community poverty rate. ²⁷ Moore et al. showed that counties in Sepsis Cluster or Sepsis Belt regions had larger numbers of physicians per 100,000 persons, suggesting the existence of financial, social, cultural barriers for those people to access health care. ²⁷

Although speculation about reasons for poor sepsis outcomes in specific states is beyond the scope of this study, higher poverty rate, larger uninsured population, lower education, and higher unemployment have been suggested as factors explaining high age-adjusted sepsis mortality rate in poor-performing counties. ²⁰ Uninsured critically-ill patients are less likely to receive critical care procedures and post-acute care compared to insured critically ill patients, and thus have higher mortality rates. ²⁸ Genetics and environmental exposures, not included in our risk standardization, may alter sepsis outcomes.^29,30 Although protocoled resuscitation strategies have improved mortality in sepsis,^3,26,31–33 differential adherence to these protocols may account for variation in RSMR. Our finding that RMSR differed across quintiles of hospital volume should not necessarily be interpreted as higher volumes corresponding to improved RSMR, because the relationship was not monotonously decreasing. Some groups have shown that hospitals with higher case volumes had improved adjusted sepsis survival, ³⁴ but regional inter-hospital transfer of sepsis patients from low to high sepsis case volume hospitals has not improved outcomes, possibly due to delays in treatment and adverse events from care transitions. ³⁵ Meanwhile, interventions focusing on improving coordination of existing hospital responses to suspected severe sepsis cases have shown promise. ³⁶ Implementation of evidence-based sepsis processes of care was more likely at hospitals with higher sepsis case volumes, but did not strongly mediate case volume-mortality associations. ³⁷

Our results should be interpreted in light of several limitations. First, the NIS does not allow data linkage between hospitals, rendering information about subsequent mortality unavailable for transferred patients. The NIS did not provide state-specific data after 2011, limiting our ability to provide current estimates. Nevertheless, our study features the largest sample size to date in estimating sepsis RSMR. Second, because the data was administrative, variation in coding practices among hospitals may limit accuracy. Moreover, we did not employ advanced deep learning method in both the recognition of sepsis and the prediction of mortality as described in previous studies.^38–42 However, standardized clinical criteria has been shown to have good reliability for identifying sepsis in the EMR^43,44 and trade-off between data complexity and model interpretation also exists in deep learning algorithm.^45,46 Third, in-hospital mortality could be biased by hospital discharge practice and length of hospital stay, ⁴⁷ and may not necessarily reflect the quality of care.^48,49 In what has been termed the “early post-discharge mortality” phenomenon, 50% of older patients who died from PNA within 30 days of admission did not die in hospital but after discharge. ⁵⁰ This can be addressed by examining a different mortality time frame or examining the risk adjusted readmission rate, ⁴ which are areas of future investigation in our group. Fourth, although we utilized a robust risk-adjustment model, we cannot assure that differences in RSMRs between states are purely owing to quality differences, as we cannot identify, for example, patients who are admitted expressly for hospice care. Our estimates are strengthened by the use of complementary models to estimate predicted sepsis mortality. Compared to generalized logistic regression, boosted trees have more freedom because they do not assume any a priori hypothesis about the feature-prediction relationship; instead, they deduce that relationship from the data. To make our final result simpler and interpretable for policy makers, we have made our tool publicly available at https://sepsismap.shinyapps.io/index2/. ¹⁸

Conclusion

In conclusion, high RSMRs persist for sepsis. The highest RSMRs were found in Wyoming, North Dakota, and Mississippi, while the lowest RSMRs were found in Arizona, Colorado, and Michigan. To our knowledge, this is the first description of state-level variation in RSMR of sepsis patients. Most importantly, we constructed a national map of sepsis RSMR, using a dashboard which we are making freely available to any investigator or clinician. We hope that identification of these high-RMSR hotspots will facilitate investment and innovation in sepsis mortality-reduction strategies, improving sepsis mortality and healthcare expenditures.