Validating the Acute Heart Failure Index for Patients Presenting to the Emergency Department with Decompensated Heart Failure

Abstract

Background

The Acute Heart Failure Index (AHFI) is a previously derived prediction rule to identify patients presenting to emergency departments (EDs) with decompensated heart failure (DHF) at low risk for early life-threatening events.

Study Objectives

We sought to prospectively validate the AHFI.

Methods

Using a prospective cohort study, we included adult patients presenting to an urban university hospital ED with DHF. We gathered data on 21 variables to calculate the AHFI. Primary endpoints included inpatient death and non-fatal serious outcomes (myocardial infarction, ventricular fibrillation, cardiogenic shock, cardiac arrest, intubation, or cardiac reperfusion). Secondary endpoints included death from any cause or readmission for heart failure within 30 days. We calculated primary and secondary endpoint rates with 95% confidence intervals (CI) for the low- and higher-risk subgroups.

Results

We enrolled 259 patients. 245/259 (95%) were admitted. 60/259 (23%) met low risk criteria, of whom 1/60 (1.7%, CI 0.04–8.9) was discharged after sustaining pulseless electrical activity arrest. The comparable primary outcome rate in the derivation study was 1.4% (CI 1.1–1.7). 17/199 (8.5%, CI 5.1–13.3) higher-risk patients experienced an endpoint, compared with 13.3% (CI 12.9–13.7) in the derivation cohort. 1 low-risk patient (1.7%, CI 0.04–8.9) died within 30 days, and 5 (8.3%, CI 2.8–18.4) were readmitted. Corresponding rates in the derivation study were 2% and 5% respectively.

Conclusion

Our results are consistent with those previously reported for the low-risk subgroup of the AHFI. Further research is needed to determine impact, safety and the full range of generalizability of the AHFI as an adjunct to decision making.

INTRODUCTION

Heart failure affects 5.7 million people in the United States, and 670,000 new cases are diagnosed each year.[1] Heart failure leads to one million hospital admissions each year.[2] Care of heart failure patients in the United States costs approximately $33.2 billion a year, with hospitalization accounting for $17.8 billion of that expense.[3] More than 75% of patients admitted to the hospital with heart failure arrive through the emergency department (ED).[4] There is current wide variability in practice regarding hospital admission for patients with decompensated heart failure (DHF).[5, 6] Hospital admission rates vary widely across geographic regions, with evidence suggesting that emergency physicians overestimate the probability of short-term death or severe complications for patients with heart failure.[7]

Several clinical prediction rules have been developed to help assess disease severity in heart failure patients, and may ultimately help physicians make appropriate decisions regarding hospital admission.[8–12] None of these rules have to date been prospectively validated. Prospective validation of a prediction rule in at least one small trial is considered to be a minimum criterion justifying clinical use.[13] The Acute Heart Failure Index (AHFI) is a clinical prediction rule derived by Auble et al at the University of Pittsburgh.[8] The AHFI uses 21 prognostic factors to determine if patients presenting to the ED with DHF are at low risk for short-term fatal and inpatient non-fatal outcomes. We sought to prospectively validate the performance of AHFI in predicting a low incidence of short-term clinical outcomes in patients presenting with DHF to a high volume urban emergency department.

METHODS

We used a prospective cohort design. Our study site was the emergency department of an urban university hospital, with an approximate annual adult ED volume of 73,000. The hospital serves a predominately Dominican, non-English speaking community, with many of the patients being uninsured, and without regular medical care. Approximately 60% of patients are Hispanic, with the remaining portion comprising Caucasians, African-Americans, Asians, and other nationalities. Survey data of our patient population has shown that approximately 1% of ED patients present with decompensated heart failure.

Research assistants (RAs) were available between the hours of 8am and midnight and enrolled study participants in cooperation with emergency medicine (EM) attending physicians. To be included in the study, patients were required to have a primary diagnosis of heart failure. Diagnosis of heart failure was made by the emergency medicine attending physician responsible for patients’ care and was based on several factors, including reported symptoms, clinical findings, chest x-ray findings, BNP level, and data from prior admissions, including discharge summaries and echocardiogram results, which are available 24/7 on all patients in this emergency department. A fixed set of criteria was not used in making the diagnosis of heart failure, nor were laboratory studies such as BNP level used to rule in or rule out the possibility of heart failure; instead, attending diagnostic impressions were used to make the diagnosis. Since a diagnosis of heart failure was required for enrollment, patients were enrolled after evaluation and diagnostic assessment by the attending physician.

Patients with a prior history of heart failure and those presenting a first episode of heart failure were included. We included patients who were admitted to the hospital and those discharged from the ED. Decisions regarding hospital admission versus outpatient management were made by the EM attending, based on factors such as severity of illness, likelihood of compliance, and reliability of outpatient follow-up. Decisions to admit or discharge a patient were not influenced by the AHFI or by a patient’s enrollment in the study. Patient management and site of care decisions were made at the ED attending and in-house physicians’ discretion, without knowledge, use or application of the AHFI or any specifications stemming from the study protocol.

Enrolled patients were at least 18 years of age. There were no exclusions based on gender or ethnicity. RAs identified patients who were given a primary diagnosis of heart failure, and obtained written consent from patients for study participation. The study was approved by the Institutional Review Board of Columbia University Medical Center.

For enrolled patients, we collected data on 21 prognostic factors, which included: gender; medical history; vital signs; laboratory values; electrocardiographic findings; and radiographic findings.[8] (Table 1) Data on the twenty-one prognostic factors were prospectively collected by RAs through chart review and patient interviews using a predesigned data sheet. Data on vital signs and laboratory values represented the first values available to physicians after patient presentation.

Table 1.

Prognostic Factors for Calculating the Acute Heart Failure Index

Demographics	Past medical history	Vital signs	Laboratory	ECG	Radiography
Gender	Coronary artery disease*	Pulse	Blood urea nitrogen	Acute myocardial infarction***	Pulmonary congestion
	Angina	Systolic blood pressure	Sodium	Myocardial ischemia***	Pleural effusion
	PTCA**	Respiratory rate	Potassium
	Diabetes	Temperature	Creatinine
	Lung disease		Glucose
			White blood cell count
			Arterial pH

^*As determined by self-report, or if medical records were available, review of the patient’s records

^**PTCA = Percutaneous transluminal coronary angioplasty

^***ECG interpretations were made by the EM attending physician

A software program is available that allows for calculation of the AHFI. The program allows users to input the 21 variables, and then determines if the patient is at low risk or higher risk for short-term life-threatening events. Figure 1 illustrates the branch points involved in calculation of the AHFI. All branches identify low-risk patients; all other patients are at higher risk. For each patient, we used the software program to determine if the patient fell into the low-risk or higher-risk subgroup. Though 21 variables are required for calculation of the AHFI, the software program allows for ease and expeditious use of the AHFI.

Figure 1.

All branches identify low-risk patients; all other patients are at higher risk. ECG, electrocardiography; MI, myocardial infarction; BUN, blood urea nitrogen; WBC, white blood cell count; PTCA, percutaneous transluminal coronary angiography; SBP, systolic blood pressure.

Inpatient medical charts were reviewed by study investigators to assess primary outcomes (inpatient death or serious medical complication during hospitalization). Following discharge, study investigators made telephone interviews within 45 days of the ED visit to determine if secondary outcomes were met (death from any cause or hospital readmission for heart failure within 30 days). For patients who had died, family members available at the contact number confirmed the date of death. Investigators assessing outcomes were blinded to the AHFI.

We defined our primary outcomes as inpatient death from any cause or serious medical complications. A patient was considered to have a serious medical complication if he or she experienced a life-threatening condition attributable to their underlying cardiac disease or received a lifesaving inpatient treatment, as listed in Table 2. We defined secondary outcomes as death from any cause within 30 days of admission and hospital readmission within 30 days with a primary diagnosis of heart failure. (Table 2)

Table 2.

Primary and Secondary Endpoints in the Acute Heart Failure Index

Primary endpoints	Secondary endpoints
Inpatient death	Death from any cause within 30 days of hospital admission
Serious medical complication	Readmission for heart failure within 30 days of hospital admission
Myocardial infarction
Ventricular fibrillation
Cardiogenic shock
Cardiac arrest
Intubation or mechanical ventilation
Cardiac compression
Resuscitation
Defibrillation
Cardiac reperfusion

We used the AHFI to determine the percentage of patients who were at low risk for serious outcomes. We calculated the rates of primary and secondary outcomes and their 95% confidence intervals for patients in the low-risk and higher-risk subgroups. Investigators determining event rates were blinded to the AHFI.

RESULTS

We enrolled 259 patients from January 2007 to July 2009. All enrolled patients had a primary diagnosis of heart failure, as made by the EM attending physician. A cross sectional survey performed in our ED estimated that approximately 1% of ED patients present with decompensated heart failure (730 per year). Patients were recruited into the study over a two-and-a-half year enrollment period. We discontinued enrollment when, due to logistical failures at a proposed second site, it became clear that our original target was not attainable. Over the two-and-a-half-year enrollment period, we enrolled approximately 14% of the estimated volume of DHF patients at our primary study site.

Of the 259 enrolled patients, 245 (95%) were admitted. Mean age was 68. 131 (51%) were male, and 128 (49%) were female. 144 (56%) had coronary artery disease. 60/259 (23%) were classified as low risk. Table 3 compares characteristics between low-risk and higher-risk risk subgroups.

Table 3.

Comparison of Low-risk and Higher-risk Subgroups

Demographics	Low risk (n=60)	Higher risk (n=199)	All (n=259)	p-value
Mean age	67	68	68	N/A
CAD	27 (45%)	117 (59)%	144 (56%)	0.08
Angina	15 (25%)	83 (42%)	98 (38%)	0.02
PTCA*	5 (8%)	62 (31%)	67 (26%)	0.0
Diabetes	24 (40%)	93 (47%)	117 (45%)	0.038
Lung disease	14 (23%)	74 (37%)	88 (34%)	0.06

^*PTCA = Percutaneous transluminal coronary angioplasty

No patients in the low-risk subgroup died during hospitalization. 1/60 (1.7%, CI 0.04–8.9) sustained pulseless electrical activity arrest requiring intubation and cardiopulmonary resuscitation. The combined primary outcome rate in the low-risk subgroup was 1.7% (CI 0.04–8.9).

1/60 1.7% (CI 0.04–8.9) patients in the low-risk subgroup died within 30 days of hospital admission, and 5/60 (8.3%, CI 2.8–18.4) were readmitted to the hospital within 30 days. The combined secondary outcome rate was 10.0% (CI 3.8–20.5).

Of the higher-risk patients, 6/199 (3.0%, CI 1.1–6.5) died during their hospitalization, and 11/199 (5.5%, CI 2.8–9.7) sustained a serious medical complication. The combined primary outcome rate was 8.5% (CI 5.1–13.3). 5/199 (2.5%, CI 0.8–5.8) of higher-risk patients died within 30 days of hospitalization, and 55/199 (27.6%, CI 21.6–34.4) were readmitted within 30 days of hospitalization. The combined secondary outcome rate was 30.2% (CI 23.9–37.0). Table 4 summaries primary and secondary outcome rates in the low-risk and higher-risk subgroups.

Table 4.

Primary Outcome (Inpatient Death and Serious Medical Complication) and Secondary Outcomes (Death or Readmission for heart failure within 30 days) Rates, 95% Confidence Intervals, and Relative Risk in Low-risk and Higher-risk Subgroups

Outcome	Low Risk (n=60)	Higher Risk (n=199)	Relative Risk
Inpatient death	0 (0%) (CI 0–6.0)	6 (3.0%) (CI 1.1–6.5)	N/A
Serious medical complication	1 (1.7%) (CI 0.04–8.9)*	11 (5.5%) (CI 2.8–9.7)	3.3 (CI 0.58–19.9)
Acute myocardial infarction	0 (0%) (CI 0–6.0)	1 (0.5%) (CI 0.01–2.8)	N/A
Intubation	1 (1.7%) (CI 0.04–8.9)	2 (1.0%) (CI 0.12–3.6)	0.60 (CI 0.08–4.6)
Resuscitation†	1 (1.7%) (CI 0.04–8.9)	1 (0.5%) (CI 0.01–2.8)	0.30 (CI 0.03–2.9)
Ventricular fibrillation	0 (0%) (CI 0–6.0)	1 (0.5%) (CI 0.01–2.8)	N/A
Reperfusion therapy	0 (0%) (CI 0–6.0)	7 (3.5%) (CI 1.4–7.1)	N/A
Inpatient death or complication	1 (1.7%) (CI 0.04–8.9)	17 (8.5%) (CI 5.1–13.3)	5.1 (CI 0.92–30.1)
Death within 30 days	1 (1.7%) (CI 0.04–8.9)	5 (2.5%) (CI 0.8–5.8)	1.5 (CI 0.2–9.7)
Readmission within 30 days	5 (8.3%) (CI 2.8–18.4)	55 (27.6%) (CI 21.6–34.4)	3.3 (CI 1.5–7.9)
Death or readmission	6 (10.0%) (CI 3.8–20.5)	60 (30.2%) (CI 23.9–37.0)	3.0 (CI 1.5–6.7)

^*The patient in the low-risk subgroup who had a serious medical complication sustained a PEA arrest, which required both intubation and cardiopulmonary resuscitation.

^†Includes defibrillation, cardiac arrest, and cardiopulmonary resuscitation

DISCUSSION

A retrospective cohort study by Auble et al at the University of Pittsburgh was used to derive a clinical prediction rule to identify ED patients presenting with decompensated heart failure who are at low risk for inpatient death or serious medical complications.[8]

To derive the rule, the authors analyzed data for 33,533 patients with a primary hospital discharge diagnosis of heart failure in 1999 who were admitted from EDs in Pennsylvania. The authors selected candidate predictors, including demographic and medical history variables and diagnostic test values measured in the ED. The authors constructed classification trees to identify a subgroup of patients with an observed rate of death or serious medical complications before discharge of <2%. The tree that identified the subgroup with the lowest rate of this outcome and an inpatient mortality rate of <1% was chosen. Patients in this subgroup were considered low risk; all others were considered higher risk. Of the 33,533 patients were enrolled, 5,758 (17.2%) were classified as low risk.[8]

A retrospective cohort study was performed by Hsieh et al at the University of Pittsburgh to validate the AHFI.[12] Of the 8,384 adult patients enrolled, 1,609 (19.2%) were classified as low risk.[12]

Table 5 compares primary and secondary outcome rates and 95% confidence intervals for low-risk and higher-risk subgroups among the derivation and validation studies. Event rates in the low-risk subgroup were found to be comparable among derivation and validation studies

Table 5.

Comparison of Primary and Secondary Outcome Rates and 95% Confidence Intervals for Low-risk and Higher-risk Subgroups among Derivation and Validation Studies

Subgroup	Study	Combined primary endpoint	Inpatient death	Serious medical complication	Combined secondary endpoint	Death within 30 days	Readmission within 30 days
Low-risk	Auble	1.4% (CI 1.1–1.7)	0.3% (CI 0.2–0.5)	1.0% (CI 0.8–1.3)	7.0% (CI 6.4–7.7)	2.0% (CI 1.6–2.4)	5.0% (CI 4.5–5.6)
	Hsieh	2.5% (CI 1.8–3.4)	0.7% (CI 0.4–1.3)	1.7% (CI 1.2–2.5)	Not reported	2.9% (CI 2.2–3.9)	Not reported
	Hsiao	1.7% (CI 0.04–8.9)	0% (CI 0–6.0)	1.7% (CI 0.04–8.9)	10.0% (CI 3.8–20.5)	1.7% (CI 0.04–8.9)	8.3% (CI 2.8–18.4)
Higher-risk	Auble	13.3% (CI 12.9–13.7)	5.3% (CI 5.1–5.6)	8.0% (CI 7.6–8.3)	13.3% (CI 12.9–13.7)	Not reported	Not reported
	Hsieh	11.3 (CI 10.6–12.1)	4.5 (CI 4.0–5.0)	6.8 (CI 6.2–7.5)	Not reported	10.2% (CI 9.4–10.9)	Not reported
	Hsiao	8.5% (CI 5.1–13.3)	3.0% (CI 1.1–6.5)	5.5% (CI 2.8–9.7)	30.2% (CI 23.9–37.0)	2.5% (CI 0.8–5.8)	27.6% (CI 21.6–34.4)

The purpose of the derivation study was to identify a subgroup of DHF patients (based on candidate predictor values) with a low rate of short-term life-threatening events. Our prospective study sought to clinically validate the AFHI [13] by applying it to DHF patients in our clinical setting to determine whether it could identify a subgroup of patients with a similarly low rate of short-term life-threatening events.

Given the small numbers of patients in our validation study, we avoided any attempt to simplify or otherwise modify the rule, which generates a dichotomous rather than a numerical risk projection. Rather, our objective was to contribute to validation of the rule as originally derived through a prospective application to a different population. The complexity of the AHFI potentially mandates an electronic adjunct, such as a calculator built into an electronic medical record, for the instrument to be used routinely in practice. We do not see this as ultimately limiting its adoption. Simplicity of prediction rules has not been shown to guarantee translation into practice, as experience with the straightforward Ottawa ankle rules has demonstrated.[14] Rather, routine adoption and use of almost all prediction rules is likely to depend upon use of information technology, and particularly incorporation into the workflow via the electronic health record.

Ours is the first prospective validation study on the previously derived AHFI. According to a published hierarchy of evidence for clinical decision rules, in most instances, a prospective validation should be regarded as the minimum requirement for consideration for clinical use.[13] Previous studies of the AHFI have all been done retrospectively. Our small study was done on a limited population in a single center. Our prospective design was calculated to control for uncertainty in the primary diagnosis of decompensated heart failure among the emergency patients enrolled. Previous studies have relied on retrospective diagnoses based on insurance coding. Our design also served to maximize accuracy of data collection relative to exclusive reliance on retrospectively collected data from insurance and medical records. On the other hand, the AHFI was not actually calculated or made available to the treating physicians during the care of the patients. Our study therefore, at best, moves the AHFI as a level III rule using a published hierarchy, rendering it potentially suitable for limited use in that or in very similar populations.[13] Several steps are involved in the development and testing of a clinical prediction rule. Before a rule can be fully recommended for implementation in clinical practice, it must be validated, and its impact on clinical behavior assessed through prospective use by clinicians. In the case of the AHFI, wider prospective validation studies are needed, followed by impact studies demonstrating clinical effectiveness.

The AHFI, no matter how well validated, does not constitute a direct decision rule in the same way as guidelines such as the Ottawa ankle rule. Rather it is akin to the well known and highly validated PORT score for pneumonia (Pneumonia Severity Index), which was derived using very similar methods to those later used to derive the AHFI. The PORT score does not directly determine whether a patient should be admitted.[15] Rather, a severity score such as PORT or the AHFI contributes objective information to a practitioner’s clinical assessment and evaluation of factors such as reliability of follow-up in making a recommendation regarding hospital admission. Following prospective validation of performance of a severity score in predicting clinical outcomes across different patient populations and settings, cluster randomized trials or other comparative studies may be performed to determine whether systematic availability of severity scores modifies practitioner behavior in such a fashion as to increase the efficient use of health care resources with no adverse effect on patient outcomes.[16]

The calculation of the AHFI in our study, although based on prospective data collection, was not based on assessments made exclusively by the practitioners. However, the prospective design increased the directness of both patient identification and of data collection relative to previous studies, which relied on ICD 9 coding to classify and enroll patients.[12]

LIMITATIONS

Our own study, like the previous derivation and validation studies of the AHFI, was largely limited to admitted patients. Had all low risk patients been discharged from the emergency department, event rates might have been higher. The hospital admission rate of patients with DHF in our center may be explained in part by a variety of factors including the high prevalence of patients lacking adequate primary care sources and support systems. This underscores the need to regard information from a severity score as a contributing factor, but not a sole determinant of, site of care decision making.

We did not attempt to confirm the diagnosis of CHF beyond the criteria used by the clinicians who treated them. However, we did not become aware of diagnostic misclassifications in the course of following up any of the included patients. We furthermore believe that the interests of clinical validation of the AHFI are best served if patients in a validation study are selected using the same criteria as would be used by practitioners in the course of using the instrument for decision making.

Our study enrolled only 14% of our projected eligible population over the study period. Though RAs were available most hours of the day, the limited enrollment rate may be explained by factors such as lack of coordination or communication between attending physicians and RAs. Future studies can employ strategies to maximize patient enrollment, such as by having RAs review the admitting/discharge diagnosis of all ED patients to identify potential study subjects. If enrolled patients were on average lower risk than those in our target population, the event rate in the low risk group may have been underestimated. Furthermore, our study was small and cannot in itself serve as a definitive prospective performance validation of the AHRI.

Availability of research assistants, who were responsible for patient recruitment, largely determined the composition of our convenience sample. The RAs were available from 8am to midnight during their on call months which included both summer and winter time periods. Hence, the limitations in capture largely expose the study to seasonal rather than diurnal variation. We consider a systematic error in sampling to be unlikely under the circumstances.

Our population is largely made up of Hispanic patients of Dominican origin and may not be representative of other heart failure populations. Future studies should incorporate larger study populations and multiple sites for the AHFI to attain a high level of external validity.

While DHF patients discharged home after ED stabilization were eligible for our study, we were only able to enroll 14 such patients (14/259, or 5%). Certain socioeconomic and access to care issues may be responsible for the high admission rate at our study site. Practice patterns may also be responsible, and can be positively impacted by use of a rule such as the AHFI. Future studies should try to enroll a larger number of discharged patients.

92% (55/60) of low risk patients in our study were admitted. The effect of the hospital admissions themselves on potential target clinical outcomes in these patients cannot be determined. Hence, our results do not directly predict the safety of an out patient management alternative for these patients. Future validation studies need to explore the impact of clinical use of the AHFI to guide site of care decisions.

In our study, the AHFI was not used to influence or guide clinical decision-making. Result of application of the AHFI were tabulated only after patients left the ED. Hence, decisions on whether to admit a DHF patient were made at the physician’s discretion, without use or application of the AHFI. This study does not address the direct potential impact of using the rule on site of care decisions.

CONCLUSION

The results from our prospective cohort study suggests that the clinical prediction rule derived by Auble et al may be able to identify patients presenting with heart failure to the emergency department who are at low-risk for serious outcomes and who might be considered for outpatient management. Event rates in the low-risk subgroup were found to be comparable within our prospective study and the retrospective validation study by Hsieh et al [12] and the derivation study by Auble et al.[8] Our study utilized prospective data collection, which has certain advantages given the limitations in accuracy and reliability that retrospective studies may be subjected to. Despite the small sample size and limited population of our study, the strong agreement of our results with those of the retrospective studies contributes strength to the validity of the rule and suggests that it merits additional, larger prospective validation studies. Further prospective validation studies utilizing larger and more diverse populations may demonstrate the utility of the rule in guiding hospital admission decision-making and its potential of reducing the number of hospitalizations for heart failure.