Do Not Resuscitate Status and Observational Comparative Effectiveness Research in Patients with Septic Shock

Abstract

Objectives

To assess the importance of including Do Not Resuscitate (DNR) status in critical care observational comparative effectiveness research.

Design

Retrospective analysis

Setting

All California hospitals participating in the 2007 California State Inpatient Database – which provides DNR status within the first 24 hours of admission.

Patients

Septic shock present on admission.

Interventions

None

Measurements

We investigated the association of early DNR status with in-hospital mortality among patients with septic shock. We also examined the strength of confounding of DNR status on the association between activated protein C therapy and mortality, an association with conflicting results between observational and randomized studies.

Main Results

We identified 24,408 patients with septic shock; 19.6% had a DNR order. Compared to patients without a DNR order, those with a DNR order were significantly more likely to be older (75±14 vs. 67±16 years), white race (62% vs. 53%), with more acute organ failures (1.44±1.15 vs. 1.38±1.15), but fewer in-patient interventions (1.0±1.0 vs. 1.4±1.1). Adding DNR status to a model with 46 covariates improved mortality discrimination (c-statistic 0.73 to 0.76, p<0.001). Addition of DNR status to a multivariable model assessing the association between activated protein C and mortality resulted in an 8% shift in the activated protein C effect estimate towards the null [odds ratio 0.78; (95% CI 0.61–0.99), p=0.04 to 0.85 (0.67–1.08), p=0.19].

Conclusions

Among patients with septic shock, DNR status acts as a strong confounder that may inform past discrepancies between observational and randomized studies of activated protein C. Inclusion of early DNR status into more administrative databases may improve observational comparative effectiveness methodology.

Introduction

Observational comparative effectiveness research (CER) provides a complementary methodology to randomized trials for identification of optimal treatments and care processes. Observational studies may complement randomized trials as a research methodology by allowing “real world” analyses of study questions for which randomization is difficult or impossible, at less expense, and with shorter completion time (1). Moreover, observational CER may be useful in validating in broader patient sets the results of randomized control trials which may have limited practical generalizability due to exclusion criteria (2, 3). Observational CER may be particularly useful in the area of critical care medicine because of the unique challenges involved in studying the critically ill, such as the need for surrogate consent in the face of time-sensitive treatment decisions. However, an important limitation in observational CER is the potential for unmeasured confounding variables to produce spurious effect estimates (4). For example, an unmeasured variable associated with disease severity that influences the selection of a treatment of interest may bias findings of observational studies through confounding by indication.

Do not resuscitate (DNR) status near the time of hospital admission has the potential to be a strong confounding variable in observational CER involving critically ill patients. In the strict sense, a DNR order relays the wishes of a patient or their surrogate that cardiopulmonary resuscitation not be administered if cardiac arrest occurs. However, in practice DNR orders may be extrapolated by clinicians to include limiting a variety of treatments such as antibiotics, central venous catheters, dialysis, intensive care, blood transfusions, and obtaining blood cultures (5, 6). The presence of a DNR order has also been associated with increased mortality in patients hospitalized with stroke or intracerebral hemorrhage (7, 8).

Because the presence of a “Do not resuscitate” (DNR) order may be strongly associated with treatment limitations as well as mortality outcomes, DNR status near the time of hospital admission may be a strong confounding variable in observational CER involving critically ill patients. In fact, the landmark observational CER study in critical illness by Connors et al. that led to the phasing out of routine right heart catheterization recorded early DNR status as a potential confounding variable (9, 10). However, few observational CER studies since Connors et al. have been able to account for early DNR status due to its lack of availability in administrative databases that are commonly used to perform observational CER.

We hypothesized that DNR status near the time of hospital admission may be an important covariate in mortality prediction models and a strong confounding variable in observational CER involving critically ill patients. In order to study the importance of DNR status to observational CER among the critically ill, we present a ‘case study’ of activated protein C (Drotrecogin alfa (activated), Xigris, APC), a treatment for septic shock with discrepant results between observational CER studies suggesting reduced mortality (11) and randomized trials showing no benefit (12–14). We sought to examine the potential importance of including early DNR status in observational CER among the critically ill by 1) examining the clinical characteristics associated with early DNR orders, 2) determining the contribution of early DNR status to a hospital mortality prediction model, and 3) determining the strength of DNR as a confounding variable for the association between APC treatment and hospital mortality in a population-based study of patients hospitalized with septic shock.

Materials and Methods

Patients

We investigated a cohort of adults (aged ≥18) using discharge data from the 2007 California State Inpatient Database (CA SID), Healthcare Cost and Utilization Project, Agency for Healthcare Research and Quality (15). The CA SID contains data for hospitalizations within California occurring in all nonfederal, general and other specialty hospitals, excluding hospital units of institutions (16). The CA SID defines a DNR order as “a directive from a physician in a patient’s current inpatient medical record instructing that the patient is not to be resuscitated in the event of a cardiac or pulmonary arrest. In the event of a cardiac or pulmonary arrest, resuscitative measures include, but are not limited to, the following: cardiopulmonary resuscitation, intubation, defibrillation, cardioactive drugs, or assisted ventilation” (17). The DNR element is a dichotomous variable with the value ‘Y’ meaning “Yes, a DNR order was written within the first 24 hours of the patient’s admission” and the value ‘N’ meaning “No, a DNR order not written or written after the first 24 hours of the patient’s admission” (17). The DNR field has been previously validated against chart data to be 84% accurate (sensitivity of 86%, specificity of 84%, positive predictive value of 70%, and negative predictive value of 93%) (18). CA SID data elements include demographics, admission and discharge status, length of stay, up to 25 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnoses, and up to 21 ICD-9-CM procedure codes. For each diagnosis the CA SID contains an additional data field that specifies whether or not the diagnosis was present on admission. The present on admission modifier allows for discrimination of pre-existing diagnoses from complications occurring after hospital admission (19–21). We included patients with a diagnosis of septic shock present on admission (ICD-9-CM code 785.52; sensitivity approximately 45% and specificity 99%) (22, 23).

Outcomes

Our primary outcome was in-hospital mortality abstracted from the CA SID.

Covariates

Demographic variables collected from the CA SID included age, sex, self-reported race/ethnicity (coded as white, black, Hispanic, or other), rural urban commuting area codes, homelessness, and insurance status. In addition, we collected 29 standard Elixhauser comorbidities (Comorbidity Software, Version 3.7, Healthcare Costs and Utilization Project of the Agency for Healthcare Research and Quality) and 11 ICD-9-CM diagnoses related to septic shock including acute organ failures present on admission (respiratory failure, hematological failure, neurological failure, acidemia, hepatic failure, renal failure) and site of infection (pneumonia, soft tissue infection, genitourinary tract infection, gastrointestinal infection, and bacteremia) (24). In addition, we collected 8 ICD-9-CM procedure codes including mechanical ventilation, atrial cardioversion, hemodialysis, right heart catheterization, vasopressor use, arterial line placement, central line placement, cardiopulmonary resuscitation and APC administration. APC administration was abstracted from procedure code ICD-9-CM 00.11 (see Supplemental Digital Content 1 - Table 1).

Statistical Analyses

We assessed differences between covariates (demographics, comorbidities and acute organ failures present-on-admission, and procedures during hospitalization) for patients with and without an early DNR order using chi-square or t-tests, as applicable.

Multivariable logistic regression models were used to determine the association between the presence of early DNR status and in-hospital mortality, adjusting for demographics, comorbidities and acute organ failures present on admission during the septic shock hospitalization. Improvement in mortality discrimination was assessed by calculating the change in c-statistic for in-hospital mortality after adding DNR status to the above model.

We assessed the possible confounding effect of early DNR status on mortality in patients who received APC by using a multivariable adjusted generalized estimating equation (with hospital included as random effect to account for correlated ICD-9-CM coding and clinical care practices) adjusting for demographics, comorbidities and acute organ failures present-on-admission. As an exploratory analysis we also examined a similar model that excluded patients with a DNR order. We assumed a priori that a >5% change in the adjusted effect estimate of APC after adding DNR status to the model would be substantial given the 46 additional covariates included in the model. We performed two sensitivity analyses: 1) excluding likely moribund patients who died in the hospital within one day or less of admission, and 2) excluding patients who received cardiopulmonary resuscitation during the first day of hospitalization despite a DNR order, who may have been falsely coded as being DNR. Statistical analyses were performed using SAS version 9.3 (Cary, NC). The Boston University Medical Campus Institutional Review Board deemed this study exempt from review.

Results

We identified 24,408 hospitalized patients with septic shock present on admission, of whom 4799 (19.6%) had a DNR order within 24 hours of admission. Characteristics of patients with and without a DNR order are shown in Supplemental Digital Content 2 - Table 2. Among patients admitted with septic shock, those with an early DNR order were older (75 ± 14 vs. 67 ±16 years, p < 0.001), more likely to be white (62% vs. 53%, p < 0.001), and more likely to be female (52% vs. 48%, p < 0.001) as compared to patients without an early DNR. Compared with patients without an early DNR order, those with an early DNR order had varying rates of many comorbid conditions including higher rates of congestive heart failure, valvular disease, peripheral vascular disease, neurological disease, hypothyroidism, metastatic cancer and depression, but lower rates of coagulopathy, anemia, psychosis, drug and alcohol abuse. The mean number of comorbidities between patients with an early DNR order and those without an early DNR order was not significantly different (4.4 ± 2.0 vs. 4.5 ± 2.0, p = 0.285). Patients with DNR orders had significantly more acute organ failures at admission (1.44 ± 1.15 vs. 1.38 ± 1.15, p = 0.002), including higher rates of acidosis and renal failure. Patients with septic shock and early DNR orders also received fewer interventions and procedures (1.0 ± 1.0 vs. 1.4 ± 1.1, p < 0.001), including less APC administration (0.54% vs. 1.29%, p < 0.001, Figure 1).

Figure 1.

Hospital procedures and interventions according to DNR status during the first 24 hours after admission

Early DNR status was associated with increased in-hospital mortality among patients with septic shock (65.3% early DNR died vs. 37.5% without early DNR, multivariable-adjusted OR 3.14, (95% CI 2.94–3.36, p < 0.001). Adding early DNR status to a model predicting inhospital mortality among patients with septic shock that included 46 additional covariates included in Supplemental Digital Content 2 - Table 2 demonstrated a significant increase in the ability to discriminate patients who would from those who would not survive a hospitalization with septic shock [c-statistic increase from 0.73 (95% CI 0.72–0.74) to 0.76 (95% CI 0.75–0.76), p<0.001, Figure 2]. Sensitivity analyses excluding 2745 patients who died within 1 day or less of hospital admission [c-statistic 0.73 (95% CI 0.72–0.74) to 0.75 (95% CI 0.75–0.76)] or excluding 91 patients who received cardiopulmonary resuscitation despite DNR status [c-statistic 0.73 (95% CI 0.73–0.74) to 0.75 (95% CI 0.75–0.76)] did not yield appreciably different results.

Figure 2.

Mortality Discrimination ROC curve for multivariate models with and without early DNR

Adding early DNR status to a multivariable model that assessed the association between APC and hospital mortality among patients admitted with septic shock (adjusted for demographics, comorbidities, acute organ failures and source of infection from Supplemental Digital Content 2 - Table 2) resulted in an 8% shift in the APC effect estimate towards the null [from odds ratio 0.78 (95% CI, 0.61–0.99), p=0.04 to odds ratio 0.85 (95% CI, 0.67–1.08), p = 0.19]. Excluding all patients with a DNR order from the analysis resulted in an odds ratio 0.82 (0.64–1.05), p=0.11, a 5.1% change in effect estimate from a model that included patients regardless of DNR status. Sensitivity analyses excluding patients who died within 1 day of admission (odds ratio shift from 0.76 to 0.82) or excluding patients who received cardiopulmonary resuscitation despite a DNR order (odds ratio shift from 0.78 to 0.85) did not yield appreciably different results.

Discussion

Our investigation begins to explore the impact of including early DNR status as a covariate in critical care outcomes and observational comparative effectiveness research. In a population-based cohort of patients admitted with septic shock, we identified that patients with an early DNR order are less likely to receive interventions and procedures despite a higher burden of acute organ failures. The addition of early DNR status to regression models predicting in-hospital mortality significantly improved model discrimination. Importantly, early DNR status was a strong confounder in a model exploring the association between APC and hospital mortality, shifting the effect estimate towards the null association suggested by randomized trials.

We found that early DNR status was associated with an adjusted 3-fold increase in hospital mortality among patients with septic shock. The increase in mortality associated with DNR status is similar to the results of a retrospective cohort of propensity matched stroke patients which showed that the adjusted odds ratio of in-hospital mortality for patients with a DNR order on the first day of admission was 2.4 (95% CI, 2.0–2.9)(7). An additional study of 8,233 patients admitted for intracerebral hemorrhage in 234 hospitals also showed that inhospital mortality was influenced by the rate of early DNR orders, with an odds ratio of 3.28 (95% CI, 2.07 – 5.19) (8). We are unaware of other studies investigating DNR status specifically among patients with septic shock.

To examine the potential importance of DNR status to observational CER, we included the effects of early DNR status in a multivariable model assessing the association between APC and hospital mortality in patients admitted with septic shock. A meta-analysis of controlled cohort studies of APC by Kalil et al. showed the relative risk of mortality associated use of APC to be 0.82 (95% CI, 0.78–0.87) (11). In contrast, the PROWESS-SHOCK study, a randomized controlled trial of mortality at 28 and 90 days in septic shock patients with APC use, found the relative risk of mortality at day 28 to be 1.09 (95% CI, 0.92 to 1.28) (14). Supporting our hypothesis that early DNR status represents a strong confounding variable, the addition of early DNR status to a multivariate model assessing the association between APC and hospital mortality resulted in an 8% change in the effect estimate toward the null (14). Thus, early DNR status likely acts as a strong confounding variable and fits the description by Lindenauer et al. of a “hypothetical confounder” that may render the results of prior observational CER studies of APC during septic shock non-significant (17).

Our effect estimate for the association between APC use and mortality differs from that found in PROWESS-SHOCK. Importantly, we did not intend our present study to represent the ‘true’ effect estimate of the association between APC and mortality; we acknowledge the likelihood of significant misclassification bias in the identification of medication utilization (eg., APC, vasopressors) using ICD-9-CM procedure codes from the CA SID. The frequency of the APC procedure code in CA SID is quite low when compared to reports in other studies (1% in CA SID vs. 6% in Lindenauer et al. using pharmacy charges Premier Perspective data and 7% in Martin et al. using the PROGRESS sepsis registry) (25, 26). Despite these limitations, we believe that our results illustrate the potential importance of DNR status to act as an important confounding variable in observational CER among patients with septic shock.

We specifically focus on DNR orders written within 24 hours of admission because a DNR order may have different implications depending on when in the hospital course it occurred. A DNR order written early in a hospitalization may reflect the patient’s desire to limit therapies because of prior beliefs and/or health status, while a DNR order later in the course of the hospitalization may reflect a failure of the patient to respond to therapy and a transition away from aggressive, but potentially futile, care (27). In the former case, an early DNR order may act as a confounding variable, whereas in the latter case the late DNR order is a consequence of the patient’s lack of response to initial aggressive therapy and may be on the etiological pathways from disease to outcome. One potential limitation to our findings would be if a large proportion of patients were falsely coded as having an early DNR order when they actually had a DNR order placed later in the hospital course. However, Goldman et al. previously found that only 5% of patients who were coded as having a DNR order placed within the first 24 hours of hospitalization actually had the DNR order placed later in the hospitalization. Thus, misclassification of patients in whom a change in DNR status during the hospitalization is likely in the etiological pathway to hospital mortality is likely rare. Moreover, we performed a sensitivity analysis excluding any patients who died within 24 hours of admission, thereby removing any patients in whom a DNR order was placed in response to rapid deterioration despite full aggressive care (i.e., cases in which an early DNR may be in the etiologic pathway from disease to outcome), and found our results to be very similar to the primary analysis.

Any variable that improves mortality discrimination, such as we have shown with early DNR, may also affect risk-adjusted mortality on an institutional level (28). Tabak et al. explored the effect of adding DNR status to models used to determine hospital performance. Although DNR status early during hospitalization significantly improved model discrimination, Tabak et al. concluded that addition of early DNR status to hospital performance reporting was limited by the wide hospital-level variation in defining what treatments would be limited in a patient with a DNR order. For example, in survey studies many physicians interpreted DNR orders variably to mean withholding treatments beyond cardiopulmonary resuscitation such as antibiotics, (5) central venous catheters, dialysis, intensive care, and blood transfusions (6). We similarly advocate further evaluation of methods to better standardize DNR reporting and to account for hospital-level variation in DNR-associated mortality before using DNR status to compare hospital outcomes.

There are several additional limitations in our study. First, our study was based on retrospective administrative claims data and thus may suffer from misclassification bias. Although we have used ICD-9-CM coding algorithms that have been previously validated and found to be fairly accurate, severity of illness and temporality of diagnoses are difficult to determine using claims data. In addition DNR is represented as a dichotomous variable without specification as to the actual limitations of care resulting from each DNR order. In “real world” clinical scenarios a DNR order may represent a range of limitations on care (such as limiting cardiopulmonary resuscitation, intubation, hemodialysis, antibiotics, or many other interventions) decided by the patient, the physician, or both. As with all observational research, we must also recognize the possibility of additional residual unmeasured confounding variables in our study.

Conclusion

In conclusion, we have shown that knowledge and inclusion of early DNR status in analyses allows for improved mortality discrimination and likely less biased effect estimates in a cohort of patients hospitalized with septic shock. Our results suggest that early DNR status is an important variable to consider in future observational CER involving the critically ill. We propose that early DNR status be better operationally defined and included into administrative databases, a process that may enhance both observational CER methodology and institutional benchmarking efforts.

Supplementary Material

eTable 1

Table 1. ICD-9 Code Listing for Diagnoses and Procedures

eTable 2

Table 2. Demographics, Comorbidities Present On Admission and Acute Organ Failure Present On Admission in Patients Without and With an Early DNR Order