Performance of Electronic Health Record Diagnosis Codes for Ambulatory Heart Failure Encounters

Abstract

Background:

There is interest in leveraging the electronic medical record (EMR) to improve knowledge and understanding of patient characteristics and outcomes of ambulatory heart failure (HF) patients. However, the diagnostic performance of ICD-10 diagnosis codes from the EMR for patients with heart failure with reduced and preserved ejection fraction (HFrEF and HFpEF) in the ambulatory setting are unknown.

Methods:

We examined a cohort of patients aged ≥18 with at least 1 outpatient encounter for HF between January 2016 and June 2018, and an echocardiogram conducted within 180 days of the outpatient encounter for HF. We defined HFrEF encounters as those with ICD-10 codes of I50.2x (systolic heart failure); and defined HFpEF encounters as those with ICD-10 codes of I50.3x (diastolic heart failure). The referent definitions of HFrEF and HFpEF were based on echocardiograms conducted within 180 days of the ambulatory encounter for HF.

Results:

We examined 68,952 encounters among 14,796 unique patients with HF. The diagnostic performance parameters for HFrEF (based on ICD-10 I50.2x only) depended on LVEF cutoff, with a sensitivity ranging from 68–72%, specificity 63–68%, PPV 47–63%, and NPV 73–84%. The diagnostic performance parameters for HFpEF depended on LVEF cutoff, with a sensitivity ranging from 34–39%, specificity 92–94%, PPV 86–93%, and NPV 39%−54%.

Conclusions:

ICD-10 coding abstracted from the EMR for HFrEF vs. HFpEF in the ambulatory setting had suboptimal diagnostic performance, and thus should not be used alone to examine HFrEF and HFpEF in the ambulatory setting.

Introduction

Observational studies using International Classification of Diseases and Related Health Problems (ICD) diagnostic codes and administrative claims have been used extensively in the United States to gain insights on patient characteristics and outcomes in real-world populations. Importantly, observational studies frequently provide more generalizable information about populations living with various medical conditions compared to registries or randomized controlled trials, which are vulnerable to selection bias. One of the most common conditions examined over the past several years using ICD diagnosis codes has been heart failure (HF). (1–3) Indeed, many studies have used ICD diagnostic codes to identify HF patients in large administrative claims-based databases from Medicare and the Healthcare Cost and Utilization Project (HCUP) (4,5), providing many important insights on hospitalized patients with HF.

There is emerging interest in leveraging ICD diagnosis codes within the electronic medical record (EMR) to improve our knowledge and understanding of patient characteristics and outcomes of ambulatory HF patients, which comprise a substantial proportion of the HF population. To effectively utilize the EMR for this purpose, reliably differentiating heart failure with reduced ejection fraction (HFrEF) from heart failure with preserved ejection fraction (HFpEF) is important, given known differences in their pathophysiology and disease trajectories.(4,6,7) However, the diagnostic performance of ICD-10 diagnosis codes specifically abstracted from the EMR for HFrEF and HFpEF in the ambulatory setting are unknown. We therefore sought to examine the diagnostic performance of ICD-10 diagnosis codes for HFrEF and HFpEF abstracted from the EMR in the ambulatory setting using echocardiography data as the gold standard, among patients from the New York City Clinical Data Research Network (NYC-CDRN).(8)

Methods

Data Source and Study Population

We examined all consecutive outpatient encounters for HF among patients aged ≥18 between January 2016 and June 2018, with an echocardiogram conducted within 180 days of the outpatient encounter for HF (either before or after). The full exclusion cascade is shown in Supplemental Figure 1. We derived these data from the New York City Clinical Data Research Network (NYC-CDRN). The NYC-CDRN database is a Patient-Centered Outcomes Research Institute (PCORI)-funded data network containing EMR data from seven health systems across the NYC metropolitan area.(8) We abstracted demographic factors directly from the EMR, and determined presence of comorbid conditions based on ICD-9 and ICD-10 codes within 1 year before the index outpatient HF encounter.

HF Encounters

We defined a HF encounter based on the presence of any of the following ICD-10 codes: I50.2x, I50.3x, I50.4x, I50.9, and I11.0. We defined HFrEF encounters as those with ICD-10 codes of I50.2x (systolic heart failure) and defined HFpEF encounters as those with ICD-10 codes of I50.3x (diastolic heart failure) (Table 1). We also conducted a sensitivity analysis where we defined HFrEF encounters as those with ICD-10 codes of I50.2x or I50.4x because ICD-10 code I50.4x (combined systolic and diastolic heart failure) may alternatively be used to identify HFrEF.

Table 1:

International Classification of Diseases and Related Health Problems 10^th Revision (ICD-10) diagnostic codes for heart failure with reduced ejection fraction (HFrEF) and heart failure with preserved ejection fraction (HFpEF)

HFrEF	HFpEF	Non-specific
I50.2x (systolic HF)	I50.3x (diastolic HF)	I50.1 (left ventricular failure, unspecified)
		I50.4x* (systolic and diastolic HF)
		I50.9 (HF, unspecified)
		I11.0 (hypertensive heart disease with HF)

^*Included in more-inclusive definition for HFrEF (sensitivity analysis)

Referent definitions based on echocardiogram

The referent definitions of HFrEF and HFpEF were based on echocardiograms conducted either 180 days before or 180 days after the ambulatory encounter for HF. There has been debate about the appropriate left ventricular ejection fraction (LVEF) cutoff to define HFrEF and HFpEF, with some experts suggesting that LVEF between 40 and 50 should be considered as a separate HF subtype.(9) For this study, we examined the diagnostic performance of ICD-10 codes using 3 different LVEF cutoffs as the referent definitions for HFrEF and HFpEF. We defined HFrEF as LVEF <50%, 45, and <40%; and we defined HFpEF as LVEF ≥50%, >45, and ≥40%. The LVEF measurements were derived from echocardiogram reports using validated natural language processing (NLP) algorithms developed by our team in prior work.(10)

Because the LVEF can change over time(11), we conducted a sensitivity analysis where we examined diagnostic performance only for individuals who had an echocardiogram conducted within 30 days of the ambulatory encounter for HF. We also conducted a sensitivity analysis where we examined diagnostic performance of I50.2x to identify HFrEF and I50.3x to identify HFpEF based on only the first encounter of each unique patient.

Statistical analysis

We first examined the characteristics of the eligible encounters (Table 2). For continuous variables, we described patient characteristics using means and standard deviations; and for categorical variables, we used counts and percentages. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the ICD-based definitions for HFrEF and HFpEF separately, using the echocardiogram-based referent standards. We also examined which ICD-10 codes accounted for false positives and false negatives, and examined the distribution of LVEF within each ICD-10 code. All analyses were conducted using R studio version 1.2.5019.

Table 2:

Baseline Characteristics for HF encounters

	All encounters (n=68,952)	I50.2x encounters (n=37,387)	I50.3x encounters (n=17,369)	Any other HF ICD code encounters (n=14,196)
Age, mean (±SD)	73.7 ± 14.3	71.1 ± 14.7	79.2 ± 11.7	73.6 ± 14.2
18–44, N (%)	2,481 (3.6%)	1,796 (4.8%)	177 (1.0%)	508 (3.6%)
45–64, N (%)	16,132 (23.4%)	10,865 (29.1%)	1,996 (11.5%)	3,271 (23.0%)
65–84, N (%)	32,873 (47.6%)	17,226 (46.1%)	8,671 (49.9%)	6,976 (49.1%)
≥85, N (%)	17,466 (25.3%)	7,500 (20.1%)	6,525 (37.6%)	3,441 (24.2%)
Female, N (%)	31,110 (45.1%)	13,864 (37.0%)	10,104 (58.1%)	7,142 (50.3%)
Race, N (%)
White	31,835 (46.2%)	16,973 (44.2%)	9,686 (55.8%)	5,176 (36.5%)
Black	14,726 (21.4%)	8,156 (21.2%)	2,896 (16.7%)	3,674 (25.9%)
Asian	1,978 (2.9%)	1,076 (2.8%)	463 (2.7%)	439 (3.1%)
Other	20,413 (29.6%)	11,182 (29.1%)	4,324 (24.9%)	4,907 (34.6%)
Hispanic ethnicity, N (%)	5,222 (7.6%)	2,989 (8.0%)	1,128 (6.5%)	1,105 (7.7%)
Comorbidities, N (%)
Anemia	19,997 (11.4%)	10,285 (10.8%)	6,955 (11.3%)	8,760 (13.8%)
Atrial Fibrillation	43,905 (25.1%)	24,015 (25.2%)	17,983 (29.3%)	14,431 (22.8%)
Chronic Kidney Disease	36,594 (20.9%)	19,274 (20.2%)	13,651 (22.2%)	14,606 (23.1%)
Chronic Obstructive Pulmonary Disease or Bronchiectasis	12,776 (7.3%)	6,275 (6.6%)	5,321(8.7%)	4,686 (7.4%)
Diabetes	42,955 (24.6%)	23148 (24.3%)	15,686 (25.6%)	16,844 (26.6%)
Hypertension	72,996 (41.8%)	38,313 (40.2%)	28,287 (46.1%)	25,015 (39.5%)
Ischemic Heart Disease	44,315 (25.4%)	28,566 (30.0%)	13,720 (22.4%)	12,595 (19.9%)

Results

We examined 68,952 encounters among 14,796 unique patients with HF. The mean age was 73.7±14.3 years, 45.1% were women, and 46.2% were white (Table 2). Among these encounters, 37,387 were coded as HFrEF (ICD-10 I50.2x) and 17,369 were coded as HFpEF (ICD-10 I50.3x). Clinical characteristics for each unique patient are shown in Supplemental Table 1.

HFrEF

The diagnostic performance parameters for HFrEF (based on ICD-10 I50.2x only) depended on LVEF cutoff, with a sensitivity ranging from 68–72%, specificity 63–68%, PPV 47–63%, and NPV 73–84% (Table 3). When narrowing the required time window between the ambulatory encounter for HFrEF and the referent echocardiogram to within 30 days, (N=26,693), diagnostic performance slightly improved but not dramatically—sensitivity ranged from 70–75%, specificity 67–73%, PPV 54–70%, and NPV 73–84% (Supplemental Table 2).

Table 3.

Diagnostic performance (95% confidence intervals) for ICD-10 codes I50.2x for HFrEF (N=68,952)

	Sens %	Spec %	PPV %	NPV %
EF<50	68.0% (67.0%−68.0%)	68.0% (68.0%−69.0%)	63.0% (62.0%- 63.0%)	73.0% (73.0%−73.0%)
EF≤45	69.0% (68.0%- 69.0%)	68.0% (67.0%−68.0%)	60.0% (60.0%−61.0%)	75.0% (74.0%−75.0%)
EF<40	72.0% (72.0%−73.0%)	63.0% (63.0%−64.0%)	47.0% (46.0%- 47.0%)	84.0% (83.0%−84.0%)

A sensitivity analysis that defined HFrEF as ICD-10 I50.2x or I50.4x slightly improved sensitivity and NPV, but worsened specificity and PPV—sensitivity ranged from 75–79%, specificity 57–62%, PPV 45–61%, and NPV 76–86% (Supplemental Table 3). When narrowing the required time window between the ambulatory encounter for HFrEF and the referent echocardiogram to within 30 days (N=26,693), diagnostic performance slightly improved, but not dramatically—sensitivity ranged from 78–82%, specificity 59–66%, PPV 51–67%, and NPV 77–86% (Supplemental Table 4). A sensitivity analysis for the diagnostic performance of I50.2x to identify HFrEF based on the first encounter of each unique patient showed similar results to the main findings (Supplemental Table 5)

When examining the ICD-10 codes that accounted for false positive cases for HFrEF (ICD-10 code I50.2x; echocardiogram showed an LVEF of ≥50%), we found that the majority of false positives resulted from ICD code I50.22 (chronic systolic HF) (Figure 1A); among these 9,364 false positives, most had an LVEF≥60%. When examining the ICD-10 codes that accounted for false negative cases for HFrEF (ICD-10 code was anything except I50.2x despite echocardiogram with LVEF of <50%), we found that the majority of false negatives resulted from ICD code I50.9 (HF not otherwise specified) (Figure 1B); among these false negatives, most had an LVEF<40%.

Figure 1A:

False positive encounters for ICD-10 code for HFrEF (I50.2x)

Figure 1B:

False negative encounters for HFrEF

HFpEF

The diagnostic performance parameters for HFpEF depended on LVEF cutoff, with the sensitivity ranging from 34–39%, specificity 92–94%, PPV 86–93%, and NPV 39%−54% (Table 4). When narrowing the required time window between the ambulatory encounter for HFpEF and the referent echocardiogram to within 30 days (N=26,693), diagnostic performance slightly improved but not dramatically—sensitivity ranged from 37–43%, specificity 94–96%, PPV 88–95%, and NPV 44–60% (Supplemental Table 6).

Table 4.

Diagnostic performance (95% confidence intervals) for ICD-10 codes I50.3x for HFpEF (N=68,952)

	Sens %	Spec %	PPV %	NPV %
EF≥50	39.0% (38.0%−39.0%)	92.0% (92.0%−92.0%)	86.0% (85.0%−86.0%)	54.0% (54.0%−54.0%)
EF>45	38.0% (37.0%−38.0%)	92.0% (92.0%−93.0%)	87.0% (87.0%−88.0%)	52.0% (51.0%−52.0%)
EF≥40	34.0% (34.0%−34.0%)	94.0% (94.0%−95.0%)	93.0% (93.0%−94.0%)	39.0% (39.0%−39.0%)

When examining the ICD-10 codes that accounted for false positive cases for HFpEF (ICD-10 code I50.3x; echocardiogram showed an LVEF of <50%), we found that the majority of false positives resulted from ICD code I50.32 (chronic diastolic HF) (Figure 2A); among these false positives, about half (45.6%) had an EF<40%. When examining the ICD-10 codes that accounted for false negative cases for HFpEF (ICD-10 code was anything except I50.3x despite echocardiogram with LVEF of ≥50%), we found that the majority of false negatives resulted from ICD code I50.22 followed by ICD code I50.9 (Figure 2B). Of note, the majority (52.8%) of those coded as I50.22 had an LVEF≥60%. A sensitivity analysis for the diagnostic performance of I50.3x to identify HFpEF based on the first encounter of each unique patient showed similar results to the main findings (Supplemental Table 7).

Figure 2A:

False positive encounters for ICD-10 code for HFpEF (I50.3x)

Figure 2B:

False negative encounters for HFpEF

Discussion

There were several key findings from this study of evaluating the validity of ICD-10 codes for HFrEF and HFpEF in the ambulatory setting. First, neither PPV nor NPV were particularly high for HFrEF; PPV for HFpEF was reasonable, but NPV was low. Second, the primary contributor to HFrEF false positives was ICD-10 code I50.22 (chronic systolic HF), and the primary contributor to HFpEF false positives was ICD-10 code I50.32 (chronic diastolic HF). Finally, the primary contributor to false negatives for HFrEF was I50.9 (HF, unspecified), and to HFpEF was I50.22 (chronic systolic HF).

These findings have important implications on examining EMR databases in the ambulatory setting. There has been increased interest in leveraging the EMR to identify and characterize HF patients for the purposes of identifying high-risk patients within a health system and generating real-world evidence. Our findings suggest that relying on ICD codes alone may not be a viable strategy to differentiate between HFrEF and HFpEF and supports the need to develop novel strategies to maximize the potential of the data derived from the EMR. The PPV was reasonable for HFpEF, suggesting that cohorts developed based on outpatient ICD-10 codes from the EMR could potentially be used to study HFpEF; however, the low NPV suggests that a large number of HFpEF patients would be excluded from such cohorts, somewhat limiting the value of this approach. One alternative strategy for leveraging the EMR to study HFrEF and HFpEF would be to broadly implement natural language processing to identify ejection fractions and subsequently incorporate them into definitions of HFrEF and HFpEF(12). This would potentially permit researchers to conduct studies on ambulatory patients with HFrEF and HFpEF, that would not otherwise be possible using databases that only offer ICD-based definitions.

Our observations of suboptimal diagnostic performance did not relate to the LVEF cutoff that we used to define the gold standards for HFrEF and HFpEF. Indeed, diagnostic performance of ICD-10 codes were suboptimal regardless of whether we used a LVEF cutoff of 50%, 45%, or 40%. Some might argue that poor diagnostic performance could have related to the large time window between the ambulatory encounter and the echocardiogram used as the referent, given recent observations that changes in LVEF over time are not uncommon.(11) However, even when we restricted the required time duration between the ambulatory encounter and echocardiogram to 30 days, the diagnostic performance was still suboptimal. Taken together, our findings suggest that other factors are driving suboptimal diagnostic performance. Whether there is value to developing more stringent processes to improve clinician accuracy of diagnostic coding in real-time warrants consideration.

A closer examination of the false positives revealed that the ICD codes for chronic systolic HF and chronic diastolic HF were the primary contributors to suboptimal diagnostic performance. In particular, those coded as having chronic systolic HF often had HFpEF, and those coded as having chronic diastolic HF often had HFrEF. This observation could relate to the fact that, over time, patients can transition from HFpEF to HFrEF and vice versa.(11) However, given that diagnostic performance did not improve much even when using a shorter time window between outpatient encounter and echocardiogram (less time for changes in ejection fraction), our observation more likely reflects the fact that many clinicians may not appreciate the difference between HFpEF and HFrEF. This is consistent with a recent study examining clinician knowledge about the diagnostic and treatment recommendations for HFpEF, which showed that a significant proportion of general cardiologists and primary care physicians were not aware of the specific recommendations for managing HFpEF.(13) As the prevalence of HFpEF rises (14), improving clinician awareness and understanding of HFpEF across multiple disciplines remains an important priority.

For many years, one of the major challenges with differentiating HFrEF from HFpEF using ICD codes was that most HF encounters were coded as “HF, not otherwise specified” which did not provide any information on whether the HF was systolic (HFrEF) vs. diastolic heart failure (HFpEF)(4). With an increased focus on improving the accuracy of billing that incorporates the acuity and subtype of HF over the past several years, a substantial number of ambulatory encounters in this cohort (assessed between 2016 and 2018) provided additional detail on HF subtype. However, we still found that 17% of patients were coded as “HF, unspecified,” again suggesting an area for further improvement.

A major strength of this study was that we leveraged a multi-hospital EMR-based database to examine performance of ICD diagnosis codes. However, there are also some limitations. The NYC-CDRN only includes 5 major academic health systems in a defined geographic region of the United States. Diagnostic performance at other institutions and geographies may perform differently. The institutions included were large academic institutions with integrated billing systems—accordingly, our findings may represent a best-case scenario. We did not have data on the hospital-based strategies used at these institutions to ensure accuracy of diagnostic coding; future studies that compare diagnostic performance across different institutions, and subsequently identify successful hospital-based strategies for improving diagnostic performance are warranted. Also, we could only conduct diagnostic performance parameters on individuals with echocardiograms; thus, there was selection bias since those who did not have echocardiograms may represent a specific cohort to which these data may not generalize.

Conclusion

In conclusion, we found that ICD-10 coding for HFrEF vs. HFpEF in the ambulatory setting had suboptimal diagnostic performance when compared to the gold standard of echocardiographic-based LVEF. This suggests that ICD-10 coding abstracted from the EMR should not be used alone to examine HFrEF and HFpEF in the ambulatory setting; and supports the need to develop processes to ensure more accurate coding in the future if there is continued interest in leveraging the EMR to study subtypes of HF (HFrEF and HFpEF).