Use of Electronic Health Data to Estimate Heart Failure Events in a Population-Based Cohort with CKD

Abstract

Background and objectives

Studies that use electronic health data typically identify heart failure (HF) events from hospitalizations with a principal diagnosis of HF. This approach may underestimate the total burden of HF among persons with CKD. We assessed the accuracy of algorithms for identifying validated HF events from hospitalizations and outpatient encounters, and we used this validation information to estimate the rate of HF events in a large CKD population.

Design, setting, participants, & measurements

We identified a cohort of 15,141 adults age 18–89 years with an eGFR<60 ml/min per 1.73 m² from 2008 to 2011. Potential HF events during follow-up were randomly sampled for validation with medical record review. Positive predictive values from the validation study were used to estimate the rate of validated HF events in the full cohort.

Results

A total of 1864 participants had at least one health care encounter that qualified as a potential HF event during 2.7 years of mean follow-up. Among 313 potential events that were randomly sampled for validation, positive predictive values were 92% for hospitalizations with a principal diagnosis of HF, 32% for hospitalizations with a secondary diagnosis of HF, and 70% for qualifying outpatient HF encounters. Through use of this validation information in the full cohort, the rate of validated HF events estimated from the most comprehensive algorithm that included principal and secondary diagnosis hospitalizations and outpatient encounters was 35.2 events/1000 person-years (95% confidence interval, 33.1 to 37.4), compared with 9.5 events/1000 person-years (95% confidence interval, 8.7 to 10.5) from the algorithm that included only principal diagnosis hospitalizations. Outpatient encounters accounted for 20% of the total number of validated HF events.

Conclusions

In studies that rely on electronic health data, algorithms that include hospitalizations with a secondary diagnosis of HF and outpatient HF encounters more fully capture the burden of HF, although validation of HF events may be necessary with this approach.

Introduction

More than 20 million adults in the United States have CKD, defined as an eGFR of <60 ml/min per 1.73 m² (1). Cardiovascular disease is the leading cause of death in this population (2), and as eGFR decreases, the risks of cardiovascular mortality, cardiovascular hospitalizations, and heart failure (HF) increase in a graded fashion (3–6). Despite a growing recognition that HF is a common cardiovascular complication of CKD, previous studies have used a variety of methods to identify HF events (5–9). This has resulted in wide-ranging estimates of disease incidence and uncertainty about the true burden of HF among persons with CKD.

In studies that use electronic health data, the emerging convention has been to identify acute HF events from hospitalizations with an HF diagnosis code in the principal position, which indicates that HF was the primary reason for the hospitalization (10–13). In contrast, with acute myocardial infarction, which has a highly specific biomarker test and is usually the primary reason for a hospitalization (14,15), HF is a clinical diagnosis based on a constellation of signs and symptoms, which can develop in the setting of other acute conditions that occasion a hospitalization, and HF is often treated in the outpatient setting (16). As a result, studies that identify HF only from hospitalizations with a principal diagnosis of HF may underestimate the true burden of HF (17). Moreover, HF symptoms are common and volume overload can be multifactorial among person with CKD (18). Whether algorithms for identifying HF from electronic health data work well in persons with CKD is unknown. The accurate identification of HF events is important for both observational studies of disease surveillance and interventional studies that rely on electronic health data for outcomes ascertainment (19–21).

We used electronic health data from a large integrated health care system to study HF events in a population-based cohort of persons with CKD. Our aims were to (1) evaluate the accuracy of algorithms for identifying HF events from hospitalization and outpatient encounter diagnosis codes, and (2) estimate and compare the absolute risk of HF among different strata of sex, age, and eGFR using the positive predictive value (PPV) estimates from the validation study and information on hospitalizations and outpatient encounters from the full CKD cohort. We hypothesized that including all hospitalized and outpatient events would substantially improve the yield for identifying validated HF events compared with an evaluation of only principal diagnosis hospitalizations. We also hypothesized that there would be a strong, graded relationship between the degree of CKD and the absolute risk of HF.

Materials and Methods

Study Population

This study was conducted at Group Health Cooperative (GHC), a nonprofit integrated health care delivery system based in Washington State, as part of a pilot study for a planned randomized controlled trial (RCT) of calcitriol treatment and cardiovascular outcomes among patients with CKD. The Group Health Human Subjects Review Committee approved this study.

Data Sources

Information on demographic variables, eligibility criteria, health conditions at baseline, and potential HF events during follow-up was obtained from electronic databases that include enrollment history, laboratory results, and diagnosis codes from inpatient and outpatient encounters. Prevalent health conditions at the time of cohort entry were assessed from inpatient and outpatient diagnosis codes up to 3 years before cohort entry.

Eligibility Criteria

We assembled a cohort of GHC enrollees age 18–89 years whose last eGFR between July 1, 2008, and June 30, 2011, was between 20 and <60 ml/min per 1.73 m². The eGFR was estimated from serum creatinine values using the Modification of Diet in Renal Disease study equations, which were in clinical use at the time this study was conducted (22). The date of cohort entry was June 30, 2011. To emulate the eligibility criteria for the planned RCT, we excluded persons whose last serum calcium was >10.3 mg/dl or whose last serum phosphorus level was >5.0 mg/dl; persons with a previous diagnosis of Paget disease, ESRD, or metastatic cancer; and persons hospitalized with acute myocardial infarction, stroke, or heart failure within 6 months before cohort entry. The International Classification of Disease, Ninth Revision (ICD-9), diagnosis codes used to identify these conditions are listed in Supplemental Table 1.

Identification and Validation of Potential HF Events

Eligible participants were followed for potential HF events until the end of follow-up on June 30, 2014. Potential HF events were identified from hospitalizations and outpatient encounters with at least one of the following ICD-9 codes: 402. × 1, 404. × 1, 425.4, and 428. For potential hospitalized events, HF diagnosis codes were classified as principal, indicating that the HF diagnosis was the primary reason for the hospital admission, or secondary (23). Outpatient encounters with an HF diagnosis required a second outpatient encounter with an HF diagnosis within 90 days to be a potential event because a single instance of an outpatient diagnosis code suggests a provisional diagnosis that was unlikely to yield a validated event. All participants were eligible to have hospitalized events; however, only persons with no history of HF at the time of cohort entry were eligible to have outpatient events because in these persons a validated outpatient HF event represents a new diagnosis of HF, whereas persons with compensated chronic HF often have an HF diagnosis coded at outpatient visits in the absence of acute symptoms.

For all persons in the full cohort with at least one potential HF event during follow-up, we identified the first potential event and randomly sampled 313 for validation. Each of the three types of HF encounters were sampled separately: persons whose first potential event was a principal diagnosis hospitalization, persons whose first potential event was a secondary diagnosis hospitalization, and persons whose first potential event was an outpatient encounter. This sample size would allow us to estimate PPVs for each type of potential HF event with 95% confidence intervals (95% CIs) of roughly ±10%. For each potential event sampled for validation, a trained abstractor reviewed information from the electronic health record within 30 days before and after the qualifying health care encounter. The first 15 potential events and all uncertain cases thereafter were reviewed with a physician adjudicator. Validated HF events met all of the following criteria, which were adapted from the Cardiovascular Health Study (24,25): (1) a physician diagnosis of HF, (2) the presence of HF symptoms, and (3) the initiation of a new HF treatment or increase in dose of an HF medication. This case definition allowed for the identification of two types of validated events: acute HF that resulted in a hospitalization and new diagnoses of HF identified in the outpatient setting.

Statistical Analyses

We estimated the PPV for validated HF events for each of the three types of potential events: principal diagnosis hospitalizations, secondary diagnosis hospitalizations, and outpatient encounters. Then, in a bootstrap procedure, we used these PPVs to estimate the number of validated HF events that would be identified by three different algorithms in the full cohort if all potential events underwent validation. The three algorithms were (1) principal diagnosis HF hospitalizations only, (2) principal or secondary diagnosis HF hospitalizations, and (3) principal or secondary diagnosis HF hospitalizations or qualifying outpatient HF encounter.

For each algorithm, we sampled individuals with replacement from the full cohort of eligible individuals and retained individual-level information about potential HF events during follow-up. For each potential HF event within a bootstrap sample, a PPV was sampled from a normal distribution with mean and SEM set to the event-type PPV estimate from the validation study. Each simulated PPV was used as the probability parameter in a draw from a single Bernoulli trial, which determined whether the potential event was validated and retained, or classified as a nonevent and discarded. For a given participant, if the random draw for the first potential HF event did not yield a validated HF event, the next potential event was assessed in this manner, and so on, until a validated HF event was observed, the patient disenrolled from GHC or died, or the study ended. The number of validated HF events, person-time of observation, and HF event rate were estimated in 1000 bootstrap samples to generate sampling distributions. For each of these values, the mean and 95% CIs (2.5th and 97.5th percentile) were estimated from the simulated sampling distribution.

Compared with the most comprehensive algorithm that included all potential hospitalized and outpatient events, we calculated the percentage of the estimated total number of validated HF events identified by the other algorithms; this is similar to an estimate of the sensitivity of a given algorithm, if all potential events underwent validation. Using the most comprehensive algorithm, we estimated and compared incidence rates for validated HF events among various strata of sex, age, and eGFR, and we conducted tests for trend with Poisson regression. Analyses were conducted with SAS software, version 9.3 (SAS Institute, Inc., Cary, NC) and Stata software, version 11.0 (Stata Corp., College Station, TX).

Results

A total of 16,140 individuals age 18–89 years had an eGFR between 20 and <60 ml/min per 1.73 m², and 15,141 (94%) met additional eligibility criteria for this study (Figure 1). The mean eGFR was 49 ml/min per 1.73 m², and three quarters had stage 3a CKD (eGFR, 45–59 ml/min per 1.73 m²) at baseline (Table 1). Of 12,857 (85%) eligible participants with an additional eGFR measurement within 1 year of the baseline measurement, 10,417 (81%) had an eGFR<60 ml/min per 1.73 m².

Figure 1.

Eligibility criteria and study design. GHC, Group Health Cooperative; PPV, positive predictive value.

Table 1.

Baseline characteristics of a population-based CKD cohort and persons with potential heart failure events sampled for validation

Characteristic	Full Cohort	Participants with Potential HF Events	Sampled for Validationa
Patients, n	15,141	1864	313
Mean age (SD), yr	69.4 (11.7)	76 (9.0)	76.1 (9.1)
Women, n (%)	9484 (63)	1043 (56)	179 (57)
Race/ethnicity, n (%)
White	13,597 (90)	1715 (92)	287 (92)
Black	443 (3)	58 (3)	13 (4)
Asian	716 (5)	52 (3)	8 (3)
Other	385 (3)	39 (2)	5 (2)
Hispanic	484 (3)	55 (3)	13 (4)
Prior CHF, n (%)	1433 (10)	682 (37)	126 (40)
Prior MI, n (%)	377 (3)	113 (6)	16 (5)
Prior stroke, n (%)	292 (2)	53 (3)	5 (2)
Mean eGFR (SD), ml/min per 1.73 m2	49.0 (8.5)	45.0 (9.7)	45.0 (9.9)
CKD stage, n (%)	11,128 (74)	1015 (54)	177 (57)
3a (eGFR 45–59 ml/min per 1.73 m2)
3b (eGFR 30–44 m/min per 1.73 m2)	3511 (23)	691 (37)	105 (34)
4 (eGFR 20–29 ml/min per 1.73 m2)	502 (3)	158 (9)	31 (10)
Mean duration GHC enrollment (SD), yr	5.7 (2.4)	5.8 (23)	4.0 (1.8)

HF, heart failure; CHF, congestive heart failure; MI, myocardial infarction; GHC, Group Health Cooperative.

^aOf 1864 participants in the full cohort with a potential HF event during follow-up, 313 were randomly sampled for validation.

Accuracy of Algorithms for Identifying Validated HF Events

A total of 1864 persons (12%) had at least one hospitalization or outpatient encounter that qualified as a potential HF event during a mean follow-up of 2.7 years. Of these patients, 313 were randomly sampled for validation; they were older, were more likely to have HF at baseline, and had a lower eGFR than persons in the full cohort but were similar to all persons who had a potential HF event during follow-up (Table 1). Principal diagnosis hospitalizations had a PPV of 92% (95% CI, 82% to 98%), secondary diagnosis hospitalizations had a much lower PPV of 32% (95% CI, 26% to 39%), and the presence of two outpatient encounters had a PPV of 70% (95% CI, 57% to 81%) (Table 2). Thirty-seven of the 250 hospitalizations (15%) sampled for validation were missing discharge summaries, but in all cases events criteria could be assessed from the electronic health record.

Table 2.

Positive predictive value of heart failure diagnosis codes for validated events according to encounter type

Event	Patients, n	Sampled, n	Validated events, n	PPV (95% CI), %
Principal diagnosis hospitalization	225	53	49	92 (82 to 98)
Secondary diagnosis hospitalization	1236	197	63	32 (26 to 39)
Two outpatient encounters	403	63	44	70 (57 to 81)

Persons with at least one potential heart failure event during follow were sampled according to the encounter type of the first potential event. Participants with prevalent heart failure at baseline were ineligible to have outpatient heart failure events during follow up. International Classification of Disease, Ninth Revision, codes were used to identify potential heart failure events: 402. × 1, 404. × 1, 425.4, 428. PPV, positive predictive value; 95% CI, 95% confidence interval.

Completeness of HF Event Ascertainment in the Full Cohort

The incidence rates for HF events estimated by different algorithms in the full cohort are displayed in Table 3. With principal diagnosis hospitalizations without validation, there were 403 HF events and the rate was 9.8 events/1000 person-years (95% CI, 8.9 to 10.8). When validated events were estimated from principal diagnosis hospitalizations using validation information, the number of events and the incidence rate decreased to 392 and 9.5 events/1000 person-years (95% CI, 8.7 to 10.5), respectively. After addition of secondary diagnosis hospitalizations to the algorithm, the estimated number of validated HF events increased to 1119 and the incidence rate increased to 27.8 events/1000 person-years (95% CI, 25.9 to 29.9). With inclusion of both hospitalizations and outpatient-identified potential events, the estimated number of validated HF events increased to 1399 and the incidence rate was 35.2 events/1000 person-years (95% CI, 33.1 to 37.4). Compared with this comprehensive algorithm, the algorithm that included only principal diagnosis hospitalizations estimated 28% of the total number of validated HF events. Outpatient-identified HF events accounted for 20% of the total number of validated HF events estimated by the comprehensive algorithm.

Table 3.

Incidence rates for heart failure events in a CKD cohort according to case-identification algorithm

Variable	Estimated HF Events	Person-Years	Rate (95% CI), per 1000 Person-Years	Events Identified, %a
Diagnosis-code only events
Principal diagnosis hospitalizations	403	41,075	9.8 (8.9 to 10.8)	33
Events estimated from validation results
Principal diagnosis hospitalizations	392	41,085	9.5 (8.7 to 10.5)	28
Principal and secondary diagnosis hospitalizations	1119	40,202	27.8 (25.9 to 29.9)	80
Hospitalizations and outpatient	1399	39,770	35.2 (33.1 to 37.4)	100

For events estimated from validation results, the number of events, person-time, and incidence rates were averaged across 100 datasets that estimated HF events from hospitalizations and outpatient encounters with HF diagnosis codes, based on the positive predicted values in Table 2. Follow-up was censored at the time of a first estimated HF event, death, or the end of the study. HF, heart failure; 95% CI, 95% confidence interval.

^aPercentage of HF events estimated by the most comprehensive algorithm that included all HF hospitalization and outpatient HF encounters.

Incidence Rates for Validated HF Events by Prevalent HF Status, Sex, Age, and eGFR

With use of the algorithm that included all HF hospitalizations and eligible outpatient HF encounters, the incidence rate for validated HF events was higher among persons with prevalent HF at baseline (135 events/1000 person-years; 95% CI, 120 to 151; Supplemental Table 2) than among persons with no prior HF at baseline (26.6 events/1000 person-years; 95% CI, 24.8 to 28.4; Supplemental Table 3), and the incidence rate was higher for men (41.8 events/1000 person-years; 95% CI, 38.2 to 45.4) than for women (31.2 events/1000 person-years; 95% CI, 28.7 to 33.8).

Incidence rates for different strata of age and eGFR, along with the incidence rate ratios comparing different groups, are displayed in Figures 2 and 3 and in Supplemental Table 4. The risk of HF was higher among older individuals (P<0.001) and those with lower eGFR (P<0.001). The incidence rate for validated HF events among persons age 80 years and older was 67.6 events/1000 person-years (95% CI, 61.5 to 74.2); compared with persons younger than age 60 years, the incidence rate ratio was 6.2 (95% CI, 5.0 to 7.9). The incidence rate for HF events among persons with eGFR of 45–59 ml/min per 1.73 m² (stage 4 CKD) was 106 events/1000 person-years (95% CI, 86 to 128); compared with persons with eGFR of 20–29 ml/min per 1.73 m² (stage 3a CKD), the incidence rate ratio was 4.2 (95% CI, 3.4 to 5.0).

Figure 2.

Estimated incidence rates for heart failure events in the full cohort by age. Heart failure events were estimated from the algorithm that included all heart failure hospitalizations and outpatient heart failure encounters using the validation results in Table 2.

Figure 3.

Estimated incidence rates for heart failure events in the full cohort by eGFR. Heart failure events were estimated from the algorithm that included all heart failure hospitalizations and outpatient heart failure encounters using the validation results in Table 2.

Discussion

In this population-based cohort study of persons with CKD, the conventional method for identifying HF events from principal diagnosis hospitalizations had a high PPV but missed two thirds of validated HF events, substantially underestimating the burden of HF in this population. A large proportion of HF events were identified by hospitalizations with a secondary diagnosis of HF and by outpatient encounters. The risk of HF was related to age and eGFR in a strong, graded fashion, and persons with stage 4 CKD experienced HF events at a rate of 10% per year, drawing attention to the role of HF as an important and common complication of CKD.

Several recent systematic reviews have evaluated the accuracy of electronic health data for identifying HF in the general population (26–28). This heterogeneous literature includes many studies that lacked information on the position of diagnosis codes or criteria for event validation. Nonetheless, most studies found that hospitalizations with a principal diagnosis code for HF had a PPV of about 90% (27), similar to our estimate of 92%. Among the studies that conducted active surveillance for events, used well defined events criteria, and distinguished principal diagnosis hospitalizations from secondary diagnosis hospitalizations, Psaty et al. (29) found that principal and secondary diagnosis HF hospitalizations identified 27% and 54% of HF events, respectively, in the Cardiovascular Health Study (81% total), and Agarwal et al. (17) used PPV estimates from the Atherosclerosis Risk in Communities study and data from the National Inpatient Sample to estimate that nearly half of acute decompensated HF hospitalizations in the United States are accounted for by secondary diagnosis HF hospitalizations. Results from these studies concur with our finding in a CKD population that secondary diagnosis HF hospitalizations had a low PPV but accounted for a large proportion of validated HF events.

Outpatient HF diagnosis codes have been evaluated in previous studies, but either the outpatient codes were included as a component of a broader algorithm that included hospitalizations (30) or the validation criteria were for chronic HF rather than new HF events (14,31,32). In our study, outpatient encounters with an HF diagnosis code followed by at least one additional outpatient HF encounter within 90 days had a PPV of 70% for validated new cases of HF; they contributed 20% of the total number of validated HF events. Given the increasing trend and preference for managing HF in the outpatient setting (33), the proportion of HF events that come from outpatient encounters is likely to increase. Furthermore, the intensification of HF treatment in the outpatient setting appears to carry the same poor prognosis as do HF hospitalizations (34).

Diagnosis codes from claims data have been a useful source of information for research on HF hospitalizations (e.g., to assess trends in the incidence of HF hospitalizations over time) (10–13). For the total burden of HF, however, principal diagnosis hospitalizations alone will yield estimates that are substantially lower than the true value, and the secondary diagnosis hospitalizations and outpatient encounters that account for most genuine HF events have PPVs that may be too low for them to be used without validation. Because it may be impracticable to validate thousands of HF events, we used validation information on a random sample of potential events to estimate the incidence of HF events in a larger population; similar methods have been used recently to estimate the overall rate of HF hospitalizations in the United States (17). For pragmatic RCTs conducted within health care systems, the accurate and complete ascertainment of events is important to achieve adequate study power and for a full evaluation of the effect of an intervention on the outcome of interest (20,21). On the basis of findings from our study and other studies (17,29), pragmatic trials that rely on electronic health data for events identification are likely to increase the number of HF events substantially by including secondary diagnosis HF hospitalization and outpatient HF encounters, but these events may require validation.

Previous studies have established strong associations between HF risk and eGFR, cystatin C levels, or claims-based diagnoses of CKD (5,6,9,35), but the relationship between HF risk and the degree of renal impairment at the moderate to severe end of the CKD spectrum is not well defined. For example, two studies failed to identify a trend in self-reported HF events with decreasing eGFR among persons with CKD (8,36), which may have been due to use of an insensitive outcome measure and HF rates that were much lower than estimates from studies that conducted more rigorous event ascertainment. In the Atherosclerosis Risk in Communities study, moderately to severely reduced eGFR (<60 ml/min per 1.73 m²) was associated with a two-fold increased risk of HF compared with both normal eGFR (≥90 ml/min per 1.73 m²) and mildly reduced eGFR (60–89 ml/min per 1.73 m²), but only 403 persons had an eGFR<60 ml/min per 1.73 m² (7). Our study extends this previous work by demonstrating in a large CKD population that HF risk was strongly related to eGFR at the moderate to severe end of the CKD spectrum.

This study had limitations. Left ventricular ejection fraction was not assessed, so we could not separately evaluate HF with reduced systolic function and HF with preserved ejection fractions, two distinct HF phenotypes that have a similar prognosis (37) but different causes and treatments (38,39). We did not attempt to validate all potential HF events in the full cohort. In October 2015, the tenth version of ICD (ICD-10) was implemented in the United States. Although many diagnosis codes from ICD-9 map to diagnosis codes from ICD-10, additional research is needed to evaluate the accuracy of ICD-10 code–based algorithms in the United States health care systems.

The accuracy of the diagnosis code algorithms and the incidence rates for HF in our study may not generalize to other settings. In addition, because our study population was identified in preparation for a planned RCT of calcitriol in patients with CKD, we excluded some patients with an elevated risk of cardiovascular events who would have been excluded from the trial, which may have resulted in conservative estimates of HF event rates. However, only 6% of the eligible study population was excluded on the basis of these criteria, so the effect on our findings may be small. Our estimated event rates may also be conservative because patients with CKD often have HF symptoms in the absence of a physician diagnosis of HF. Even with rigorous active surveillance to identify HF events, creative approaches may be required to capture the full burden of HF among patients with CKD, such as patient-reported outcomes (18) or imaging-based measures (40).

CKD is now considered an important etiologic factor in the development of cardiovascular disease rather than simply a marker of shared cardiovascular risk factors (2,41), and the absolute risk of HF in this population is substantial. In studies that rely on electronic health data, conventional methods for identifying HF events from hospitalizations with a principal diagnosis of HF appear to substantially underestimate the risk of HF events. The evaluation of secondary diagnosis HF hospitalizations and outpatient HF encounters may permit a more thorough assessment of the burden of HF in a CKD population, but the validation of HF events may be necessary with this approach.

Disclosures

B.M.P. serves on the data safety and monitoring board of a clinical trial of a device funded by the manufacturer (Zoll LifeCor, Pittsburgh, PA) and on the steering committee of the Yale Open Data Access Project, funded by Johnson & Johnson (New Brunswick, NJ).