Two Decades of Mortality Trends among Patients with Severe Sepsis: A Comparative Meta-analysis

Abstract

Objective

Trends in severe sepsis mortality derived from administrative data may be biased by changing ICD-9-CM coding practices. We sought to determine temporal trends in severe sepsis mortality using clinical trial data that does not rely on ICD-9-CM coding and compare mortality trends in trial data to those observed from administrative data.

Design

We searched MEDLINE for multicenter, randomized trials that enrolled patients with severe sepsis from 1991-2009. We calculated standardized mortality ratios (SMR) for each trial from observed 28-day mortality of usual care participants and predicted mortality from severity of illness scores. To compare mortality trends from clinical trials to administrative data, we identified adult severe sepsis hospitalizations in the Nationwide Inpatient Sample, 1993-2009, using two previously validated algorithms.

Setting and Patients

Hospitalized patients with severe sepsis or septic shock.

Measurements and Main Results

Of 3244 potentially eligible articles, we included 36 multicenter severe sepsis trials, with a total of 14,418 participants in a usual care arm. Participants with severe sepsis receiving usual care had a 28-day mortality of 33.2%. Observed mortality decreased 3.0% annually (95% CI 0.8%, 5.0%, p=0.009), decreasing from 46.9% [SMR 0.94, 95% CI (0.86, 1.03)] during years 1991-1995 to 29% [SMR 0.53, (95% CI (0.50, 0.57)] during years 2006-2009 (3.0% annual change). Trends in hospital mortality among patients with severe sepsis identified from administrative data [“Angus definition”: 4.7% annual change, (95% CI 4.1%, 5.3%), p=0.69), “Martin definition”: 3.5% annual change, (95% CI 3.0%, 4.1%, p=0.97)] were similar to trends identified from clinical trials.

Conclusion

Since 1991, patients with severe sepsis enrolled in usual care arms of multicenter randomized trials have experienced decreasing mortality. The mortality trends identified in clinical trial participants appear similar to those identified using administrative data and support the use of administrative data to monitor mortality trends in patients with severe sepsis.

Introduction

Recent epidemiological studies using administrative hospital data have reported trends of rising incidence and declining hospital mortality rates associated with severe sepsis in the Unites States (US).^1-5 The concurrent observations of increasing severe sepsis incidence and a greater number of acutely dysfunctional organ systems have raised the possibility that declining mortality rates may be artifacts of changing International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) coding⁶ and patient discharge practices.⁷ For example, increasing the number of claims for severe sepsis or acute organ failures by including patients who technically meet criteria for severe sepsis but have milder disease may enhance hospital reimbursement but result in lower illness severity among patients identified as having severe sepsis. Such a trend would potentially result in a spurious decline in mortality rates.^4,6 Similarly, increases in the number of patients discharged to long term acute care facilities prior to in-hospital death may further reduce hospital mortality rates associated with severe sepsis.⁷ Thus, without an alternative standard available with which to study mortality trends, it is unclear whether severe sepsis mortality is truly declining or whether reported improvements in severe sepsis outcomes are artifacts of changing coding and discharge practices.

Determining trends in severe sepsis mortality is of considerable public health importance. Severe sepsis affects approximately 1 out of 3 intensive care unit patients,⁸ and is one of the top 10 causes of death in the United States⁹ with annual hospital costs of $24.3 billion.⁴ If mortality rates are truly decreasing, then further investigation of the etiology of this decline with potential reallocation of funds towards effective practices in severe sepsis care would be warranted to sustain the trend in improved outcomes. Further, identifying secular trends in severe sepsis mortality has implications in the interpretation and design of ‘before’ and ‘after’ quality improvement studies.^10,11 However, if the decreasing mortality reflected by administrative data merely represents an artifact of changing coding patterns, then alternative methods to track severe sepsis outcomes must be identified.

Usual care control groups from multicenter randomized clinical trials of sepsis therapies provide an alternative method to estimate prevailing trends in severe sepsis mortality rates. Multicenter clinical trial participants represent a diverse patient group who are prospectively deemed to meet severe sepsis criteria¹² and risk-stratified by standardized severity-of-illness scoring systems (e.g., Acute Physiology and Chronic Health Evaluation (APACHE) II,¹³ Simplified Acute Physiology Score (SAPS) II¹⁴ , Logistic Organ Dysfunction System (LODS)¹⁵). In order to ascertain trends in severe sepsis mortality, we performed a meta-analysis of mortality associated with severe sepsis among patients receiving usual care in multicenter clinical trials that began enrollment between1991 to 2009. In addition, we investigated whether mortality trends identified from administrative data were similar to severe sepsis mortality trends identified in clinical trial participants.

Materials and Methods

Clinical trial selection and data abstraction

We used a sensitive strategy (Supplemental Digital Methods)^16,17 to search MEDLINE for randomized trials enrolling patients with severe sepsis. Based on a review of abstracts, two independent investigators (ARR and GTR) selected prospective studies enrolling patients with sepsis that reported a mortality outcome. The full text of these studies was then reviewed in detail by two independent investigators (AJW and EKS) to identify multicenter, randomized, controlled trials that enrolled patients using a modified 1991 American College of Chest Physicians/Society of Critical Care Medicine Consensus definition¹² of severe sepsis included patients with suspected infection and acute organ dysfunction. Single center studies were excluded out of concern that the reported mortality rates may be center-specific and not representative of more widespread trends.¹⁸ Observational studies were excluded because very few multicenter observational studies of prospectively-identified severe sepsis patients (that were not secondary analyses of trial data) met our criteria of providing study start dates, hospital or 28 day mortality, and baseline severity of illness scores. Within the eligible observational studies, 3 of the 4 eras in our analysis were represented by only one study limited to patients from one country, confounding our ability to separate differences over time from national differences in severe sepsis mortality. Characteristics of the 8 multicenter observational studies that met our criteria are shown in Supplemental Digital Content Table 1. Additionally, we excluded clinical trials that were not published in English and that did not specify study start dates. Because small trials may be more likely to have biased estimates,¹⁹ we performed a sensitivity analysis excluding trials in which the control group N was less than or equal to 20.

Using a standardized data abstraction form, two independent investigators (AJW and EKS) recorded the following data from each trial: enrollment start date and end date, average age, sex distribution, severity of illness score (APACHE II,¹³ SAPS II,¹⁴ Logistic Organ Dysfunction System¹⁵), enrollment of patients with severe sepsis vs. septic shock, number of patients enrolled in the usual care group, hospital and 28 day mortality of usual care group patients. Hospital mortality data were available from only 5 of 36 (14%) trials and thus we were unable to analyze trends in hospital mortality among trials. Because we used administrative data from the US to compare with the multicenter trials, we performed a sensitivity analysis stratifying by whether trials enrolled subjects from the US or did not include any US centers.

Administrative data analysis

We examined hospitalizations from adults (age ≥18 years) using year 1993-2009 discharge data from the Nationwide Inpatient Sample (NIS), Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality. Based on a large increase in population coverage starting in 1993, use of NIS data prior to 1993 for trend analyses is not recommended.²⁰ The NIS is an approximate 20% stratified probability sample of all US non-Federal acute care hospitals and contains de-identified clinical and resource use information from approximately 5-8 million hospital discharges yearly. NIS sampling strata are based on five hospital characteristics: ownership/control, teaching status, urban/rural location, geographic region and bed size. The 1993 NIS contained data from about 900 hospitals in 17 states and the 2009 NIS included data from approximately 1000 hospitals in 44 states. NIS elements include demographics, admission and discharge status, length of stay, up to 15 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis and procedure codes (increased to 25 diagnosis codes in 2009), and hospital characteristics.

We identified cases of severe sepsis in the NIS based on two previously published^1,21 and validated²² algorithms. The “Angus” algorithm selected severe sepsis cases based on the presence of an ICD-9-CM code for infection and acute organ dysfunction, whereas the “Martin” algorithm identified severe sepsis cases based on the presence of ICD-9-CM codes for septicemia, bacteremia or fungemia as well as an acute organ dysfunction code. Both algorithms included explicit severe sepsis (995.92) and septic shock (785.52) codes introduced in 2002 and 2003, respectively.

Statistical analyses

We used standardized mortality ratios (SMR) to adjust for baseline case-mix differences between trials. SMRs were calculated from the ratio of observed mortality to predicted mortality in the usual care arm. Predicted mortality for usual care arm trial participants was calculated from previously published regression equations based on the baseline severity of illness score available from each trial ( APACHE II, SAPS II, or LODS score).^13,14,15 In years during which multiple trials contributed data, SMRs were calculated from the pooled number of observed and predicted deaths from each trial.

We analyzed trends in severe sepsis mortality in clinical trial data using two methods. First, we pooled trials by the year of first patient enrollment. We used Joinpoint Regression Program version 4.0.0 (Statistical Research and Applications Branch, National Cancer Institute, Bethesda, Maryland) to evaluate trends in the observed, predicted and standardized 28-day severe sepsis mortality rates from trials that began enrolling patients from 1991-2009.²³ Joinpoint models were constructed using a heteroscedastic errors weighted least squares regression in which the standard error of the mortality estimate from each year was input to the model. We identified trends in mortality using the annual percent change (APC) in yearly pooled 28-day mortality rates. Second, we pooled clinical trials by the start date of patient enrollment into four time frames (1991-1995, 1996-2000, 2001-2005, 2006-2009).

We used SAS version 9.3 (Carey, NC) to identify the survey-weighted hospital mortality of patients identified from the NIS with severe sepsis and septic shock using ‘Angus’ and ‘Martin’ administrative data algorithms. We compared trends in severe sepsis 28-day mortality identified with clinical trial data to trends in hospital mortality identified using administrative data during the years 1993-2009 using the Joinpoint Regression Program test of parallelism.²⁴ Because of large differences in sample size between the clinical trials and the administrative data, resulting in different power to detect inflection points of trend changes, we did not evaluate for joinpoints in the models. Instead, we analyzed APC over the entire 1993-2009 date range to capture trends. Because individual patient level data was unavailable from the trials, we did not adjust for potential confounders of the comparison between trial and administrative data mortality trends. A Cochrane-Armitage test for interaction was used to test for differences in mortality trend based on whether trials were conducted among US centers or outside the US. A two-sided alpha level of 0.05 as selected for statistical significance. All study procedures were approved by the Boston University Medical Campus Institutional Review Board.

Results

Severe Sepsis Trial Meta-Analysis

Of 3244 articles identified with the highly sensitive search strategy, we included 36 multicenter severe sepsis trials that met our inclusion criteria (Figure 1). Included trials enrolled a total of 14,418 study participants in a usual care arm, with average age of 61 years and 40% females; trials lasted an average of 2.2 ± 1.1 years and had a 28 day mortality rate of 33.2%. (Supplemental Digital Content, Table 2). Sixty-four percent of trials were multinational. The percentage of patients with septic shock in each trial ranged from 33%-100%.

Figure 1.

Flow diagram for selection of severe sepsis trials.

Observed 28 day mortality among trial participants decreased with an APC of 3.0% (95% CI 0.8%, 5.0%, p=0.009) from 1991-2009; however, predicted mortality did not change significantly (APC 0.3%, 95% CI -1.6%, 2.2%, p=0.77). Severe sepsis 28-day mortality rates decreased from 46.9% (SMR 0.94, 95% CI) during the years 1991-1995 to 29% (SMR 0.53, 95% CI 0.50, 0.57) during years 2006-2009 (Table 1). Trials showed similar declines in severe sepsis mortality regardless of whether they included patients from the US (mortality rates 1991-1995: 40%, 2006-2009: 24%, p<0.001) or only enrolled patients from outside of the US (mortality rates 1991-1995: 47%, 2006-2009: 33%, p<0.001, p_interaction =0.26). A sensitivity analysis excluding small trials showed similar results (APC 2.9, 95% CI 0.8%, 5.0%, p=0.01).

Table 1.

Multicenter Randomized Trial Severe Sepsis Mortality Trends

	Trial Start Year
28-day Mortality Measure	1991-1995 n = 1,040a	1996-2000 n = 5,363	2001-2005 n = 4,745	2006-2009 n = 2,288	p
Observed mortality, %	46.9	35.9	26.6	29.2	0.009
Predicted mortality, %	49.8	53.9	43.0	54.6	0.77
Standardized mortality ratio (95% CI)	0.94 (0.86–1.03)	0.67 (0.64–0.70)	0.62 (0.58–0.65)	0.53 (0.50–0.57)	0.02

^aIncludes trials with available predicted mortality data. When including studies without published predicted mortality estimates, mortality during the years 1991–1995 was 45.8% (n = 2022).

Comparison of administrative data with trial data

Using the Martin administrative definition we identified 8.7 million severe sepsis hospitalizations, with an average patient age of 67 years, of whom 49% were female. Using the Angus administrative definition we identified 22.4 million hospitalizations, with an average patient age of 68 years, of whom 52% were female. Figure 2 demonstrates trends in severe sepsis mortality from 1993-2009 using the Angus definition, Martin definition, or multicenter trial data. Hospital mortality estimates from 1993-2009 were 10.4% lower when using the Angus definition [19.1%, (95% CI 19.0, 19.1%)] as compared with the Martin definition [29.5%, (95% CI 29.5, 29.6%)]. Trends in severe sepsis mortality differed when comparing the Martin definition (APC 3.5%, 95% CI 3.0%, 4.1%) to the Angus definition (APC 4.7%, 95% CI 4.1%, 5.3%), p=0.0002. However, trends in severe sepsis mortality over time were similar to trends observed in multicenter trials and both the Angus and Martin definitions (p=0.69 and p= 0.97, respectively).

Figure 2.

Severe sepsis mortality trends comparing multicenter trials by date of patient enrollment, Martin administrative definition and Angus administrative definition. ICD-9-CM codes for severe sepsis and septic shock were introduced in 2002 and 2003 respectively.

Discussion

Prior reports using administrative data^1-5 have shown a declining trend in hospital mortality rates for patients with severe sepsis. However, administrative data may be subject to changing trends in coding patterns⁶ or hospital discharge patterns⁷ that result in biased mortality estimates. Usual care groups from multicenter randomized trials selected prospectively based on modified Consensus severe sepsis definitions¹² may be less likely to differentially misclassify severe sepsis over time than administrative data and provide an alternative data source for estimation of prevailing mortality trends. We determined trends in severe sepsis mortality during the two decades since Consensus sepsis definitions were developed and compared outcomes from multicenter trials to those derived from administrative data. We observed that risk-standardized 28-day severe sepsis mortality declined over the past two decades. We have also shown that temporal trends in severe sepsis mortality derived from administrative claims data ICD-9-CM algorithms are likely accurate.

Our findings have important implications. Severe sepsis mortality rates have declined despite the fact that none of the multicenter trials included in our analysis has yet introduced an efficacious sepsis therapy. Our data do not allow us to ascertain the specific reasons for the improvement in mortality. However, in the absence of novel sepsis therapeutics, mortality declines may be due to improved processes of care. Potentially effective improvements include earlier antibiotic administration,²⁵ increased use of early goal directed therapy,²⁶ improvements in mechanical ventilation strategies²⁷ or increased intensivist staffing.^28,29 In order to most efficiently target quality improvement interventions, future studies should seek to determine which practice patterns are most strongly associated with outcome improvements in severe sepsis.

Identification of secular trends in severe sepsis mortality has further implications for future conduct of quality improvement studies. Studies that use a ‘before and after’ study design to determine the effect of quality improvement interventions in patients with severe sepsis, such as Surviving Sepsis Campaign care bundles,^10,11 may be confounded by secular trends of declining severe sepsis mortality. Future studies that seek to identify changes in outcomes after introduction of a quality improvement intervention for patients with severe sepsis should account for secular trends in severe sepsis mortality.³⁰

Administrative databases are frequently used to assess epidemiological trends in severe sepsis.^1-5 However, whether trends identified by administrative data are accurate has been a matter of controversy. Lindenauer et al. investigated trends in case-fatality rates among hospitalized patients with pneumonia and concluded that putative mortality improvements in patients with pneumonia were likely due to changing ICD-9-CM coding strategies.⁶ Hall et al. identified that hospital mortality rates may be reduced through earlier discharge of patients to long term care facilities.⁷ Using 28-day mortality data from trial participants prospectively ascertained to meet severe sepsis criteria, our results suggest that administrative data likely accurately reflects secular trends in severe sepsis mortality. While limited by potential differences between hospital and 28-day mortality, our data suggest that severe sepsis cases selected from administrative data using the Martin definition appear to have outcomes most similar to patients enrolled in clinical trials.

Few other studies have investigated recent trends in severe sepsis mortality outside of administrative data sources. Using a meta-analysis of prior retrospective and prospective studies, Friedman et al. showed that mortality rates for patients with septic shock had likely declined from years 1958 to 1997.³¹ However, conclusions from Friedman et al. are limited by lack of Consensus sepsis definitions prior to 1991, the wide variation in included study designs, use of hospital mortality endpoints, and the lack of data after 1997. Harrison et al.³² identified a decrease in hospital mortality (48.3% vs. 44.7%) among patients identified retrospectively with severe sepsis from the multicenter Intensive Care National Audit & Research Centre in the United Kingdom from 1996-2004. Additionally, in a 2007 retrospective analysis, the Australian and New Zealand Intensive Care Society found hospital mortality of patients with septic and septic shock to be declining at a rate similar to that of 28-day mortality among international multicenter trial participants.³³

Our study has several strengths. Participants in multicenter clinical trials were prospectively identified as meeting severe sepsis criteria and thus were less likely to be subject to substantial misclassification bias. Our requirement for use of multicenter trials allowed for selection of generally high-quality international studies with large sample size that reflected a variety of care settings. Further, we used severity of illness scores from each clinical trial to adjust for differences in case mix (e.g., proportion of patients with septic shock) and risk-standardize the severe sepsis mortality rates across clinical trials and found that severity of illness among trial participants has not changed substantially over time. In addition, we used previously validated administrative data algorithms to identify severe sepsis in a representative sample of patients hospitalized in the US. Although our data encompass a greater time span, we identified severe sepsis mortality rates that were similar to prior reports using the Nationwide Inpatient Sample.^2-4

Our study also has several limitations. First, we excluded clinical trials that were not published in English, which may limit external validity of our findings to studies performed in areas of English fluency. Further, exclusion criteria used in randomized controlled trials may select patients that differ from the underlying population. For example, severe sepsis cases identified from clinical trials had generally lower mortality than patients enrolled in observational studies and were younger than severe sepsis cases identified using administrative data algorithms. . Thus, mortality rates from patients enrolled in randomized trials may not provide “gold-standard” estimates of mortality in the underlying population. Second, most of the trials included were conducted multi-nationally while the administrative data only reflects outcomes for severe sepsis patients in the US. However, declines in severe sepsis mortality were similar regardless of trial location. Third, 28-day mortality rates measured in clinical trials likely differ from in-hospital mortality⁵ and although trends are likely similar, the absolute mortality rates may not be comparable between clinical trial and administrative data. Fourth, different algorithms used to identify severe sepsis cases in administrative data selected cases of varying disease severity: the Angus algorithm identified nearly a 3-fold larger cohort with substantially lower inhospital mortality rates and a larger decline in mortality than the Martin definition. Prior single center chart validation of severe sepsis ICD-9-CM code algorithms show hospital-level variation in the accuracy of severe sepsis ICD-9-CM coding algorithms.^1,21,34-36 Representative, multicenter chart validation studies are necessary to determine the overall sensitivity and specificity of each coding algorithm in the Nationwide Inpatient Sample. Given the large difference in sample size between the trials and administrative data, we cannot rule out the possibility that a greater number of trial participants might reveal statistically significant mortality trends when compared to administrative data. Fifth, while the Consensus severe sepsis definition of a suspected infection and presence of acute organ dysfunction was used as inclusion criteria for all trials, the number of acutely dysfunctional organs or systemic inflammatory response syndrome criteria may have differed among trials. The differences in trial inclusion criteria highlight the importance of our use of the SMR to adjust for differences case-mix between trials. That the variation in severity of illness between trials as measured by APACHE II, SAPS II, or LODs scores does not change significantly over time make it less likely that modification of Consensus sepsis definitions used in some trials affects our finding of a decreasing trend in severe sepsis mortality. Finally, the NIS does not include physiologic variables that would allow calculation of SMR in administrative data. Nonetheless, we have demonstrated that trends in mortality rates are likely comparable between 28-day and hospital mortality measurements.

Conclusions

Since 1991, patients with severe sepsis enrolled in usual care arms of multicenter randomized controlled trials experienced a trend of decreasing 28-day mortality. The mortality trends identified in clinical trial participants appear similar to those identified using administrative data and support the use of administrative data to monitor mortality trends in patients with severe sepsis. Our findings have implications for epidemiological monitoring of severe sepsis outcomes and future study designs evaluating quality improvement interventions. The mechanism for the mortality decline in severe sepsis is unclear and warrants further study.

Abbreviations

APACHE

Acute Physiology and Chronic Health Evaluation

APC

Annual percent change

ICD-9-CM

International Classification of Diseases, Ninth Edition, Clinical Modification

NIS

Nationwide Inpatient Sample

SAPS

Simplified Acute Physiology Score

LODS

Logistic Organ Dysfunction System