Identification of Emergency Department Patients With Acute Heart Failure at Low Risk for 30-Day Adverse Events

Abstract

OBJECTIVES

No prospectively derived or validated decision tools identify emergency department (ED) patients with acute heart failure (AHF) at low risk for 30-day adverse events who are thus potential candidates for safe ED discharge. This study sought to accomplish that goal.

BACKGROUND

The nearly 1 million annual ED visits for AHF are associated with high proportions of admissions and consume significant resources.

METHODS

We prospectively enrolled 1,033 patients diagnosed with AHF in the ED from 4 hospitals between July 20, 2007, and February 4, 2011. We used an ordinal outcome hierarchy, defined as the incidence of the most severe adverse event within 30 days of ED evaluation (acute coronary syndrome, coronary revascularization, emergent dialysis, intubation, mechanical cardiac support, cardiopulmonary resuscitation, and death).

RESULTS

Of 1,033 patients enrolled, 126 (12%) experienced at least one 30-day adverse event. The decision tool had a C statistic of 0.68 (95% confidence interval: 0.63 to 0.74). Elevated troponin (p < 0.001) and renal function (p = 0.01) were significant predictors of adverse events in our multivariable model, whereas B-type natriuretic peptide (p = 0.09), tachypnea (p = 0.09), and patients undergoing dialysis (p = 0.07) trended toward significance. At risk thresholds of 1%, 3%, and 5%, we found0%, 1.4%, and 13.0%patients were at low risk, with negative predictive values of 100%, 96%, and 93%, respectively.

CONCLUSIONS

The STRATIFY decision tool identifies ED patients with AHF who are at low risk for 30-day adverse events and may be candidates for safe ED discharge. After external testing, and perhaps when used as part of a shared decision-making strategy, it may significantly affect disposition strategies. (Improving Heart Failure Risk Stratification in the ED [STRATIFY]; NCT00508638)

Nearly 1 million U.S. emergency department (ED) visits for acute heart failure (AHF) occur annually. More than 80% result in hospital admission (1) and account for the largest proportion of the projected $70 billion to be spent on heart failure (HF) care by 2030 (2,3). This high admission proportion remained unchanged from 2006 to 2010 (1). ED visits for AHF are expected to rise because of our aging population and increased survival in both chronic HF and acute coronary syndromes (ACS) (3,4). Importantly, up to 20% of hospitalized AHF patients will be readmitted within 30 days (5). Recent health policy modifications place significant pressure on hospitals and medical systems to break this cycle of admission-readmission or face financial consequences (6,7).

The identification of AHF patients who may be discharged safely from the ED is crucial to reduce costly inpatient admissions (8). Patients discharged from an ED are reportedly at increased risk of readmission and death compared with those who are hospitalized (9–12). Furthermore, post-discharge events are often perceived as unpredictable and undesirable (13). Thus, ED discharge of AHF patients becomes a challenging proposition (14,15).

Studies of risk factors in patients with AHF have identified variables associated with adverse outcomes such as death, inpatient complications, and readmission (9,16–18). They are limited in clinical applicability and have thus far not led to the development of an acute care setting decision tool. Data from inpatient sources have been combined with outpatient sources (16,18), retrospective chart review methodology has been used (9,16,18–20), and large databases designed for other purposes have been analyzed in an attempt to identify risk factors for poor outcomes (18–20). These models may be useful to identify patients who require admission for intensive monitoring and therapy; however, when more than 80% of ED patients are already being admitted, a tool to identify patients who are safe for discharge would be of greater value. Two of the above retrospective ED-based risk models have identified a cohort of 18% to 25% of AHF patients who would be considered low risk (19,20). Their external validation and impact on clinical care, however, have not been analyzed prospectively.

Shared decision making (SDM), a structured interaction between provider and patient to determine a management plan, has been successful in other ED disease processes (21). Patients, clinicians, and guideline experts believe HF patients would benefit from SDM initiatives (6,22). Objective decision support in the form of a useful decision tool is a first step toward a SDM approach for patients with AHF, perhaps facilitating early, safe ED discharge.

We designed our prospective cohort study of ED patients diagnosed with and treated for AHF to address these past limitations. Our aim was to develop an AHF decision tool to identify ED patients at low risk of death or serious complications who could therefore be considered for ED discharge and subsequent outpatient management.

METHODS

We conducted a prospective, observational cohort study, STRATIFY (Improving Heart Failure Risk Stratification in the ED), from July 20, 2007, to February 4, 2011, at 2 university-affiliated tertiary care EDs and 2 community EDs. The rationale and design have been reported previously (23). Briefly, the study team, which consisted of the principal physician investigator, trained research assistants, and study coordinator, screened ED patients and approached those whom the treating physician diagnosed and treated for AHF (Figure 1). Study associates were available 16 h daily during the week and 12 h daily on the weekends to screen and enroll patients. Study personnel obtained informed consent and enrolled a convenience sample 7 days per week. Treatment and disposition decisions were determined by the treating physicians and were not influenced by this investigation. Our medical centers’ institutional review boards reviewed and approved this study. Study data were collected and managed with REDCap (Research Electronic Data Capture), hosted at Vanderbilt University (24).

FIGURE 1. STRATIFY Study Flow.

AHF = acute heart failure; ED = emergency department.

SETTING AND SUBJECTS

Our study recruited patients at 1 ED in Nashville, Tennessee, and 3 EDs in Cincinnati, Ohio. These EDs represent demographically and socioeconomically diverse patient populations.

INCLUSION AND EXCLUSION CRITERIA

ED patients were screened for possible AHF by use of modified Framingham criteria (25). Our modification reflects contemporary practice for making a preliminary diagnosis of AHF. Three Framingham criteria were not used: 1) circulation time; 2) vital capacity; and 3) weight loss in response to treatment. Vital capacity and circulation time are not typically available in the acute setting, and weight loss in response to treatment would only assist with a retrospective diagnosis. Inclusion of patients required the presence of at least 2 major, or 1 major and 2 minor, modified Framingham criteria (major: paroxysmal nocturnal dyspnea, neck vein distention, rales, cardiomegaly on chest radiograph, acute pulmonary edema, S₃ gallop, or hepatojugular reflux; minor: ankle edema, night cough, dyspnea on exertion, hepatomegaly, pleural effusion, or tachycardia ≥120 beats/min). Patients had to be willing and able to give informed consent and be at least 18 years of age. Although use of B-type natriuretic peptide (BNP) <100 pg/ml to support a non-AHF diagnosis is standard at our institutions, it was not used for exclusion in the proposed study, because lack of BNP elevation was expected to exclude some low-risk patients. Although use of BNP to enroll patients may have increased the probability of AHF, the investigators did not want to exclude subjects who were clinically believed to have AHF because of a false negative BNP result. Furthermore, a cardiology oversight group reviewed a subset of the ED chart reviewer’s charts to determine the proportion of patients who were believed not to have AHF after in-hospital testing.

ASSESSMENTS AND ED DIAGNOSIS OF AHF

Research assistants collected data by direct questioning of the patient and treating physician, as well as by a review of the electronic medical record during the first 3 h of ED management. The principal investigator or study coordinator reviewed and confirmed the accuracy of the data recorded by the research assistants.

We used the broad, highly sensitive, modified Framingham criteria to ensure prospective capture of all potential ED patients with AHF. ED medical record review was performed independently by 2 investigators (A.B.S., S.P.C.) to confirm that patients included ultimately had an ED diagnosis of AHF. The assessors were blinded to the inpatient medical record. If the assessors agreed the ED visit was definitely or definitively not AHF-related, no further assessment was performed. For all others, a third assessor (G.J.F.) adjudicated. A subset of medical records were reviewed in duplicate to determine abstractor agreement.

OUTCOME MEASURES

The primary outcome was ordinal and represented the most severe adverse event experienced within 30 days of ED evaluation (Table 1). The ordinal scale was determined a priori on the basis of severity and created by the investigators, comprising both emergency physicians and cardiologists. The scale focused on events thought to be of most interest to physicians when considering ED discharge, including risk of death, ACS, cardiopulmonary resuscitation, mechanical cardiac support, mechanical ventilation, emergent dialysis, and emergency revascularization. An ordinal rather than binary outcome was used because the proportional odds model allows for parsimonious modeling of an ordinal outcome with increased power and precision compared with a binary logistic model. This hierarchy does not weight a more serious event over a less serious event in the regression model; it merely assigns the most serious outcome as the outcome of interest. If a subject experiences both an ACS and death, death is assigned as the outcome of interest.

TABLE 1.

Hierarchical Listing of Adverse Events and Their Weighting, Considered for the Outcomes in the Risk Model

	Outcomes
	Clinical Conditions	Inpatient Procedures
Most Severe Complication	Death, all cause [5]	Cardiopulmonary resuscitation [5]
		Mechanical cardiac support [4] Intubation/mechanical ventilation [3] Emergent Dialysis [2]
Least Severe Complication	Acute coronary syndrome [1]	Percutaneous Coronary Intervention/Coronary Artery Bypass Grafting [1]

ADVERSE EVENTS AND FOLLOW-UP

Adverse events were collected in a 2-step process. First, a chart review was performed to ascertain events that occurred during the index admission or return visits in the 30 days after enrollment. Second, phone follow-up of all patients was performed at 5 days and 30 days after enrollment. If either method indicated a 30-day event, it was counted as an event. When discrepancies occurred between the phone follow-up and chart review, those events documented by chart review were considered final. Subjects admitted to the hospital were followed up daily until hospital discharge. If hospitalized for more than 5 days, in lieu of phone follow-up, the investigators reviewed the admission medical records and documented any adverse events.

For follow-up, investigators used a standard communication process that consisted of: 1) 3 attempts to contact the patient by telephone; 2) 2 attempts to telephone the “alternate contact” provided by the patient at enrollment; 3) 1 repeat attempt to contact the patient; 4) electronic medical record review to capture ED and hospital visits within the region; and 5) a Social Security Death Index search at 30 and 90 days post-enrollment if no contact had been made.

The investigators assessing outcomes were masked to the predictor variables and vice versa. This was done to adhere to the standards for development of decision aids (26–29).

ANALYSIS

In accordance with accepted principles, a large number of ED candidate predictor variables were considered based on established risk factors for AHF and availability within 3 h of the index ED visit (26,28). Safe discharge home or admission to the hospital is dependent on the patient’s hemodynamic status, impact of associated conditions, comorbidities, and ability to provide self-care.

The STRATIFY decision tool was developed according to established strategies (26,28,30,31). First, we selected a large pool of candidate predictors based on clinical relevance and availability within the index ED visit. Then we evaluated descriptive statistics of candidate predictors for degree of missingness and level of information provided. A priori, predictors with missingness >90% were removed from consideration because their inclusion would minimize the usefulness of the tool. Categorical predictors with >95% prevalence for 1 level were also removed because of lack of variation and sufficient information to the tool. Descriptive statistics on the remaining potential predictors were calculated with median (interquartile range) or percentage (n), as appropriate. Missing data on remaining candidate predictors were imputed with single imputation that allowed for nonlinear transformations on the data to make optimal use of partial information recorded for each subject (26,28). The challenge in the development of the STRATIFY decision tool lay in the presence of a large pool of 57 candidate predictor variables that produced “noise” during traditional stepwise model selection methods. To mitigate the effects of noisy features in such a high-dimensional setting, a modeling approximation method (26) called pre-conditioning was used for model selection (32) (Online Appendix). Pre-conditioning was developed to handle high dimensional data problems, in which the number of predictors was large relative to the number of events. The method has 2 steps: 1) a continuous “pre-conditioned” outcome that characterized the underlying distribution of the 6-level ordinal outcome was derived as a linear combination of all 57 candidate predictors by fitting the proportional odds model on the ordinal outcome; and 2) the best set of predictors was selected from the pool on the basis of maximizing the Akaike information criterion with backward model selection using the pre-conditioned outcome (26). The STRATIFY decision tool was derived by fitting a proportional odds model to the 6-level ordinal outcome using 13 variables selected in the second step of the preconditioning method. Regression splines were used in the risk prediction model. Relaxing the linearity assumption with splines allows the coefficient to vary based on the covariate, which allows us to capture nonlinearities in the relationship between the coefficient and the outcomes. Proportional odds assumptions of the final model were verified using plots of the mean of each candidate predictor across the levels of the ordinal outcome.

We quantified the predictive accuracy of STRATIFY by calculating the discrimination (28) using concordance (C statistic). We calculated the calibration by constructing a smooth nonparametric calibration curve of predicted versus observed outcome, which represents the bias in predicted values. We optimized the calibration curve at the low-risk end of the spectrum to minimize false negative results and define a threshold to accurately identify low-risk ED patients.

We internally validated the calibration and discrimination for STRATIFY using bootstrap resampling to estimate the likely performance of the decision tool on a new patient sample from the same patient stream (26). All analyses were performed with R programming language (33).

RESULTS

OVERALL PATIENT CHARACTERISTICS AND RESULTS OF ED EVALUATION

A total of 2,074 subjects with signs and symptoms of AHF were recruited. Sixty-three withdrew, and 18 were lost to follow-up. Of the remaining cohort, 1,033 were identified by the study investigator’s review as having AHF in the ED. There were 94 charts reviewed in duplicate by the ED reviewers, which resulted in a kappa of 0.90 (95% confidence interval [CI]: 0.80 to 1.00). Of the 1,033 patients included in our cohort, our cardiology oversight group adjudicated 820 patient charts and determined 762 (93%) had a diagnosis of AHF. With the exception of ejection fraction (9.2%), none of the variables had more than 4% missingness.

Table 2 reports baseline characteristics. The median age was 64 years, 57% were male, and 44% were African American. Overall, patient comorbidities were consistent with a chronic HF population, with 74% having a prior history of HF, 35% with prior myocardial infarction, and 22%with a history of renal disease. As a group, renal function was impaired (median estimated glomerular filtration rate 52.8 ml/min/1.73 m²), systolic blood pressure was elevated (median 144 mm Hg), and BNP levels were elevated (median 994 pg/ml). Patients who were enrolled but did not have AHF were less likely to be African American, to have a history of HF, and to be taking beta-blockers, angiotensin-converting enzyme inhibitors, or diuretic agents as outpatients, and they had less congestion on chest radiograph.

TABLE 2.

Descriptive Statistics for Patients Enrolled in STRATIFY

	AHF	No AHF
Male	57 (590)	53 (265)
Age, yrs	64 (53–75)	62 (52–74)
African-American race	44 (455)	32 (158)
HF history	74 (769)	59 (293)
MI history	35 (365)	31 (154)
Hypertension history	82 (847)	79 (393)
Renal disease	22 (222)	21 (108)
Diabetes mellitus	33 (336)	31 (153)
Habitation status
Lives alone/homeless	29 (295)	27 (135)
Lives with others	67 (689)	69 (344)
Nursing home/assisted living	4 (49)	3 (17)
Outpatient medications
Beta-blocker	66 (675)	57 (283)
ACE inhibitor	44 (454)	36 (179)
Diuretic agent	69 (711)	61 (302)
ED testing results
Chest radiograph with congestion	40 (406)	15 (72)
SBP (mm Hg)	144 (127–171)	138 (121–154)
Pulse (beats/min)	88 (75–102)	86 (73–98)
Respiratory rate	20 (18–24)	18 (18–22)
BNP (pg/ml)	994 (475–1,932)	168 (48–540)
Sodium (mmol/l)	139 (137–141)	138 (136–140)
BUN (mg/dl)	21 (14–34)	18 (12–31)
eGFR (ml/min/1.73 m2)	52.8 (34.0–73.7)	54.1 (35.4–75.6)
Hemoglobin (g/dl)	12.1 (10.6–13.8)	12.3 (10.8–13.6)

Values are proportions (counts) or median (lower and upper quartiles).

ACE = angiotensin-converting enzyme; AHF = acute heart failure; BNP = B-type natriuretic peptide; BUN = blood urea nitrogen; ED = emergency department; eGFR = estimated glomerular filtration rate; HF = heart failure; MI = myocardial infarction; SBP = systolic blood pressure; STRATIFY = Improving Heart Failure Risk Stratification in the ED.

5-DAY AND 30-DAY OUTCOMES IN ED PATIENTS WITH AHF

Overall, 7% of patients experienced a 5-day event and 12% of patients experienced a 30-day event (Table 3). The majority of events (104 of 126, 83%) were confirmed by chart review. The early events were related to ACS (within 5 days) and the late events to death (after 5 days). The primary chief complaint was recorded as “chest pain” in 12 of 35 patients with ACS (34%) during the first 5 days. The remainder of the patients had a chief complaint of shortness of breath or edema. A small subset of events were related to emergent dialysis, intubation, or mechanical cardiac support. The breakdown between 5-day and 30-day events is illustrated in Figure 2.

TABLE 3.

Distribution of Outcomes at 30 Days for STRATIFY Patients

No event	88 (907)
ACS/PCI/CABG	4 (46)
Emergency dialysis	1 (14)
Intubation	2 (17)
Mechanical cardiac support	1 (6)
Death	4 (43)

Values are proportion (counts).

ACS = acute coronary syndrome(s); CABG = coronary artery bypass grafting; PCI = percutaneous coronary intervention; other abbreviations as in Table 2.

FIGURE 2. Frequency of Individual Components of the Decision Tool at 5 Days and 30 Days.

ACS = acute coronary syndrome(s); CABG = coronary artery bypass grafting; PCI = percutaneous coronary intervention.

PREDICTION OF LOW RISK FOR MORTALITY AND SERIOUS COMPLICATIONS

We used our methodology to filter 57 predictors through 8 principal components, which resulted in a decision tool with 13 variables readily obtained in the ED (Table 4). Using the variable coefficients, we also developed a nomogram to assist the physician in estimating the risk of 30-day events (Figure 3). The C statistic of the STRATIFY decision tool was 0.68 (95% CI: 0.63 to 0.74). The C statistic of the tool in men was 0.64 (95% CI: 0.57 to 0.72), and for women it was 0.74 (95% CI: 0.67 to 0.81). When evaluating those patients with preserved ejection fraction (≥45%), we found a C statistic of 0.72 (95% CI: 0.64 to 0.79). An elevated blood urea nitrogen and troponin level were found to be significant predictors of adverse events, whereas an elevated BNP, tachypnea, and use of dialysis trended toward a significant association with adverse events (Table 4). Other variables included in the model were age, body mass index, diastolic blood pressure, sodium, oxygen saturation, QRS duration, and the use of supplemental oxygen or an angiotensin-converting enzyme inhibitor as an outpatient. These individual variables did not have a statistically significant association with the outcome in the final model.

TABLE 4.

Model Results for 30-Day AHF Events in the STRATIFY Decision Tool

Covariate	OR	95% CI	p Value
Age	1.25	0.90–1.72	0.18
BMI	0.87	0.66–1.15	0.33
BNP	1.21	0.97–1.51	0.09
DBP	0.87	0.67–1.13	0.30
BUN
15 (ref)	1.00		0.01
20	1.13	0.97–1.32
30	1.37	0.98–1.93
Sodium	0.89	0.76–1.04	0.15
RR	1.21	0.97–1.51	0.09
saO2	0.90	0.77–1.05	0.17
Troponin I (cubic root)
0.10 (ref)	1.00		<0.001
0.25	0.91	0.72–1.15
0.50	2.87	1.49–5.53
0.75	3.73	1.96–7.09
Dialysis
No (ref)	1.00		0.07
Yes	1.89	0.95–3.76
On supplemental O2	1.31	0.90–1.93	0.16
On outpatient ACEI	0.80	0.56–1.16	0.24
QRS duration
≤120 ms (ref)	1.00		0.22
>120 ms	1.28	0.86–1.90

ACEI = angiotensin-converting enzyme inhibitor; BMI = body mass index; CI = confidence interval; DBP = diastolic blood pressure; OR = odds ratio; ref = reference; RR = respiratory rate; saO₂ = arterial oxygen saturation; other abbreviations as in Table 2.

FIGURE 3. Nomogram for Use of the STRATIFY Decision Tool.

Points for each variable are calculated on the upper “Points” bar and summed across the variables to give the total points. The total points are then found on the “Total Points” bar at the bottom, and risk is determined by drawing a perpendicular line to the “Risk of 30-Day Adverse Event” line. ACEI = angiotensin-converting enzyme inhibitor; BMI = body mass index; BNP = B-type natriuretic peptide; BUN = blood urea nitrogen; DBP = diastolic blood pressure; RR = respiratory rate; saO2 = arterial oxygen saturation.

Figure 4 presents the calibration curve and shows that STRATIFY performed well in identifying patients whose risk of a 30-day adverse event was 10% or less. Looking at risk thresholds of 1%, 3%, 5%, and 10%, we found 0%, 1.4%, 13.0%, and 49.5% of patients, respectively, were considered to be low risk. Our test characteristics (Table 5) suggest the STRATIFY rule is highly sensitive for identifying patients at low risk of subsequent adverse events. Its negative predictive values are 100%, 96%, and 93% for identifying true low-risk patients at 3%, 5%, and 10% risk of death or subsequent adverse events. There were few deaths in those identified as being at low risk. There were no deaths in the cohort with ≤3% risk of events, 1 death in the cohort with a 3% to 5% risk of events, and 6 deaths in the subjects with a 5% to 10% risk of events.

FIGURE 4. Calibration Curve for the Decision Tool.

TABLE 5.

Test Characteristics in Low-Risk Patients in the STRATIFY Decision Tool

Cutpoint (%)	TN (n)	FP (n)	FN (n)	TP (n)	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)
3	14	893	0	126	1.00 (0.97–1.00)	0.02 (0.01–0.03)	0.12 (0.10–0.15)	1.00 (0.78–1.00)
5	128	779	6	120	0.95 (0.90–0.98)	0.14 (0.12–0.17)	0.13 (0.11–0.16)	0.96 (0.91–0.98)
10	475	432	36	90	0.71 (0.63–0.79)	0.52 (0.49–0.56)	0.17 (0.14–0.21)	0.93 (0.90–0.95)

CI = confidence interval; FN = false negative; FP = false positive; NPV = negative predictive value; PPV = positive predictive value; TN = true negative; TP = true positive.

DISCUSSION

In this study, the largest prospective ED AHF investigation to date, we developed a decision tool that was able to accurately identify a low-risk patient group. Our study has 3 main findings. First, we were able to characterize 13% of patients with less than a 5% risk of 30-day adverse events with a negative predictive value of 98%. Importantly, in this cohort of 134 patients at <5% risk for events, there was only 1 death within 30 days of ED evaluation. This death occurred more than 5 days after the ED evaluation. The use of our tool could have potentially redirected an additional 105 patients (Table 6) (10%; ≤5% risk of events) from hospital admission to ED discharge. Second, our tool suggests that variables readily available during an ED workup, such as renal function, BNP, respiratory rate, and a history of dialysis, can rapidly identify ED patients with AHF at low risk of subsequent adverse events. Third, only a small proportion of AHF patients are at very low risk (<3%) of 30-day adverse events. Importantly, model calibration suggests our decision tool has high sensitivity and negative predictive value at this end of the risk continuum. If our decision tool shows similar test characteristics in a separate population, it may be incorporated into clinical practice and could have a significant impact on ED disposition decisions.

TABLE 6.

Range of Nomogram Scores and Corresponding Predicted Probabilities

Total Points	Risk	Frequency
≤137	0.00–0.03	14
138–171	0.04–0.05	120
172–218	0.06–0.10	377
219–247	0.11–0.15	247
248–269	0.16–0.20	129
270–287	0.21–0.25	63
288–303	0.26–0.30	29
304–317	0.31–0.35	25
318–330	0.36–0.40	16
331–343	0.41–0.45	4
344–356	0.46–0.50	3
≥357	≥0.51	6

Emergency physicians serve as the major decision makers for approximately one-half of all U.S. inpatient admissions for AHF (34). The lack of disposition recommendations from national guidelines, the absence of a validated decision tool to identify patients at low risk for post-discharge events (6,35), the high rates of significant adverse events after an ED discharge (10,36), and high risk intolerance lead to admission for more than 80% of ED patients with AHF (1). Although consensus guidelines have addressed AHF risk stratification, they provide little objective instruction for ED disposition decision making (37), or they base recommendations on disparate studies of isolated predictors (38,39).

We report that 0%, 1.4%, and 13.0% of patients with AHF had <1%, <3%, and <5% risk of 30-day death or serious complications, respectively. Importantly, early events were largely attributable to ACS, which often can be detected with ED-based troponin testing. There were 128 patients (12%) with a predicted probability of <5% risk who were admitted and had no 30-day events. Consistent with prior studies, we found troponin and renal function to be key components of risk evaluation (40,41). Our proportion of patients with ACS was higher than in prior risk-stratification studies. However, we enrolled a broad, heterogeneous population of ED patients early in their course, which distinguishes us from many of these studies. It is likely that minor ED troponin elevations were suggestive of evolving ACS, because a large proportion of our ACS events occurred in the first 5 days. Patients with initial troponin elevations are unlikely to be eligible for immediate ED discharge, and our decision tool suggests this marker continues to identify a higher-risk cohort (41). Preliminary data suggest higher-sensitivity cardiac troponins may also have useful prognostic utility in ED patients with AHF; it will be important to further evaluate their role as it becomes incorporated into standard practice (42). Although BNP was an important predictor in our model, it was not found to be a significant predictor of death and serious adverse events compared with troponin and renal function. This is consistent with other ED-based studies that demonstrated an association between elevated BNP and AHF readmissions but less so with early mortality (43–46). Because our model did not include ED revisits and hospital readmissions, it is not surprising that the relationship for BNP was not as significant as other variables.

Patients hospitalized with AHF have been found to have mortality rates as high as 15% at 60 days (47). AHF patients have a worse 6-month prognosis than most cancer patients (48). Whether AHF simply heralds a sicker chronic HF cohort or reflects a distinct pathophysiological entity is unclear. Although hospitalization confers benefits in rapid decongestion, symptom improvement, and monitoring, compared with outpatients with a comparable degree of cardiac dysfunction, hospitalized patients have significantly worse outcomes (49,50). It is not clear whether this is attributable to any potential adverse effect of inpatient therapy or the result of worsening underlying disease precipitating hospital admission. Although hospitalization may confer benefit in some patients more than others, thresholds for ED discharge of patients with AHF remain notoriously conservative (13) and are more aligned with those obtained after provocative testing for chest pain patients. However, the alignment of risk thresholds between ED patients with chest pain and AHF is problematic. Patients discharged from the ED with chest pain are typically younger and have few comorbidities (51,52). Moreover, fewer than 10% of ED patients admitted with AHF undergo invasive procedures or therapy, which suggests hospitalization could be prevented in a subset of patients (53–56). However, the only way we will know whether there is a protective effect related to hospitalization is after the tool is tested alongside clinical judgment and low-risk patients are managed as outpatients.

A previous rule derived and validated in 2 separate AHF datasets identified a subset of 19.2% of patients who would be considered low risk for 30-day morbidity and mortality (19,20); however, it did not consider modern biomarkers (cardiac troponins and natriuretic peptides) and was obtained from administrative data. In a Canadian study of more than 12,000 patients, a model for 7-day mortality was derived and validated. Their report suggests several ED variables can be used to categorize patients at low risk of 7-day mortality; however, limitations included retrospective patient identification by use of administrative data, a lack of consideration of natriuretic peptide testing, and a practice environment not reflective of the United States (38).

Patients with HF spend considerable time in the hospital and may be willing to trade a slightly increased risk of adverse events for ED discharge and admission avoidance. Using STRATIFY to accurately discuss a patient’s risk of death or serious complications via an SDM strategy could result in a greater number of ED discharges. However, once discharged, social, behavioral, and environmental factors strongly influence one’s ability to optimally manage a chronic illness (57,58). Furthermore, patient self-care and strategies to overcome barriers to successful self-care are associated with optimal outpatient management and reduced readmissions (59–63). Combining the events identified by decision tools such as STRATIFY with clinicians’ judgment and patient preferences via SDM, as well as obtaining and addressing patients’ self-care barriers, may provide the inertia necessary to change physicians’ practice patterns (6).

If the STRATIFY decision tool were incorporated in SDM and facilitated an increased number of ED discharges, early follow-up would be critical because it has been associated with a decreased risk of readmission (64). Collaborative post-discharge follow-up care between cardiology and primary care has also been associated with better guideline adherence and lower mortality (65). Furthermore, disease management programs that account for non–HF-related diseases have lowered readmission rates by providing close monitoring and follow-up. Connecting patients discharged from the ED to these processes needs to be systematically evaluated. However, assurance of follow-up post-ED care can be challenging. Even the best decision tool may not be able to influence physician behavior if timely follow-up is not possible, thus limiting the discharge of ED patients identified to be at low risk for subsequent adverse events (9,13).

STUDY LIMITATIONS

Ideally, a prognostic model should be validated externally before use as a decision tool across a wide range of settings. The strongest external validations require evaluation by different research groups using new data not available at the time of analysis (66); however, an external validation of this type is only relevant once internal validity of the developed model has been shown. Therefore, in this study, we focused on showing internal validity using all the data available at the time of model development and the bootstrap resampling technique to maximize precision and power. The bootstrap not only allows one to estimate the likely future performance of a decision tool in a similar population, but it also can quantify the optimism in model estimates, providing unbiased estimates of future tool performance in similar patients without using new data to perform external validation. When the resampling allows models and coefficients to disagree with themselves over hundreds of resamples, the proper price is paid for data mining. Thus, clinical utility (useful predictive discrimination) is not claimed for what is in fact overfitting. Properly penalized bootstrapping can be called rigorous or strong internal validation. Furthermore, selecting diverse settings to conduct this project allowed us to partially overcome limitations of risk assessment strategies developed in distinct patient populations. Overall, the STRATIFY decision tool could be applied to any population with similar characteristics as patients in this study. External validation should be conducted when its usage is extended to a more general population. The STRATIFY model contains 13 variables. Although this may be too complex to remember, clinical calculators have been widely used in the ED for decision tools dealing with thromboembolic disease, stroke, and atrial fibrillation (67–69). We anticipate this having similar utility. Finally, the impact of the STRATIFY tool on clinical practice needs to be determined by studying how its use in conjunction with clinicians’ judgment affects clinical care and outcomes.

CONCLUSIONS

STRATIFY is the first prospectively derived ED-based decision tool for identifying ED patients with AHF who are at low risk for 30-day adverse events using readily available variables. We found an elevated troponin and abnormal renal function to be significantly associated with adverse events. Our tool was highly sensitive and able to identify patients at low risk for 30-day adverse events. After external testing, when used as part of an SDM strategy, it may significantly affect disposition strategies

PERSPECTIVES.

COMPETENCY IN MEDICAL KNOWLEDGE

The vast majority of ED patients with AHF are admitted to the hospital. Prior studies have identified patients at high risk for adverse events, but very few have identified patients safe for ED discharge. As a result, the admission rate has remained largely unchanged.

TRANSLATIONAL OUTLOOK

Our results suggest information readily available during the first few hours of ED evaluation may be useful to identify patients at low risk for subsequent adverse events. Once tested externally, this decision tool may be used in a SDM strategy to facilitate safe ED discharge.

ABBREVIAT IONS AND ACRONYMS

ACS

acute coronary syndrome(s)

AHF

acute heart failure

BNP

B-type natriuretic peptide

CI

confidence interval

ED

emergency department

HF

heart failure

SDM

shared decision making