A Robust e-Epidemiology Tool in Phenotyping Heart Failure with Differentiation for Preserved and Reduced Ejection Fraction: the Electronic Medical Records and Genomics (eMERGE) Network

Abstract

Identifying populations of heart failure (HF) patients is paramount to research efforts aimed at developing strategies to effectively reduce the burden of this disease. The use of electronic medical record (EMR) data for this purpose is challenging given the syndromic nature of HF and the need to distinguish HF with preserved or reduced ejection fraction. Using a gold standard cohort of manually abstracted cases, an EMR-driven phenotype algorithm based on structured and unstructured data was developed to identify all the cases. The resulting algorithm was executed in two cohorts from the Electronic Medical Records and Genomics (eMERGE) Network with a positive predictive value of > 95%. The algorithm was expanded to include three hierarchical definitions of HF (i.e., Definite, Probable, Possible) based on the degree of confidence of the classification to capture HF cases in a whole population whereby increasing the algorithm utility for use in e-Epidemiologic research.

Introduction

Electronic medical record (EMR) systems are increasing in ubiquity, functionality, and comprehensiveness across the United States and thus capitalizing on this data is a practical and cost-effective e-Epidemiology approach. The National Heart Lung and Blood Institute working group on Epidemiology and Population Sciences identified e-Epidemiology as a strategic research priority [1]. Specifically, the recommendation included the active engagement in studies “to establish the validity, reliability, and scalability of electronic tools for data collection.” Given the increasing prevalence and high cost[2–4], an e-Epidemiology approach to study the heart failure (HF) epidemic €would facilitate cost-effective research efforts aimed at developing strategies to effectively reduce the burden and cost[5, 6].

The syndromic nature of HF presents challenges in identifying patients using EMR data given that the diagnosis is clinical [7], at least two distinct types exist [8–11], HF with preserved ejection fraction (HFpEF) and HF with reduced ejection fraction (HFrEF), and previous studies have noted bias using a single modality of EMR data (e.g., diagnoses codes from administrative databases) to identify HF patients [12–16]. However, the Electronic Medical Records and Genomics (eMERGE) Network has demonstrated the applicability and portability of EMR derived phenotype algorithms using different types of clinical data for algorithm execution including billing and diagnoses codes, natural language processing (NLP) of clinical notes and unstructured data, laboratory measurements, patient procedure encounters, and medication data [17]. eMERGE has developed and validated nearly 45 EMR phenotype algorithms, many of which are currently available publically at pheKB.org, to facilitate cost-effective research [18–20]. To date, the predominant focus of eMERGE algorithms has been to accurately identify cases and non-cases of specific medical conditions using multiple types of EMR data and excluding those not meeting strict inclusion or exclusion criteria to facilitate genome-wide association studies [21]. While case/non-case EMR algorithms are powerful tools in research, particularly genome-wide association studies, the ability to characterize real-world clinical patient populations that are a comprised of a mix of primary care patients (i.e., medical home), transient patients, and referral patients resulting in varying patterns of depth and detail in EMR data is more limited. Therefore, the purpose of the current study was to develop and validate an EMR-based algorithm to accurately identify HF patients with characterization of HFpEF and HFrEF. Furthermore, we sought to broaden the EMR algorithm into a tool for e-Epidemiologic research that goes beyond the typical case/non-case identification to characterize HF and HF type of a complete population.

Methods

Development Cohort

Heart Failure in the Community Cohort (HL72435, PI Roger)

Since 2003, the Heart Failure in the Community Cohort, henceforth referred to as the HF Cohort, has prospectively recruited HF patients from Olmsted County, Minnesota, to study the heterogeneity of HF as it relates to outcomes and thus represents a gold standard cohort of manually abstracted cases defined according to Framingham Heart Failure Criteria [22]. NLP of the unstructured EMR text is used to prospectively identify patients presenting with clinical findings compatible with HF [5, 23]. The complete records of potential cases are manually reviewed by trained nurse abstractors to collect clinical data and to verify the diagnosis of HF using the Framingham criteria [24]. The feasibility and reliability of the Framingham criteria to ascertain HF in Olmsted County, Minnesota, have been previously published [22]. Consented HF patients undergo an echocardiogram, blood draw, questionnaires, and hand grip test administered by a registered nurse. Hospitalized patients were contacted during hospitalization, and outpatients at their next clinic appointment. From the HF Cohort, 706 validated HF patients were used in the development of the HF algorithm described herein.

Validation Cohorts

Mayo Genome Consortia (MayoGC)/eMERGE Cohort

MayoGC/eMERGE is a large cohort of Mayo Clinic patients with EMR and genotype data. Eligible patients include those who gave general research (i.e., not disease specific) consent in the contributing studies to share high throughput genotyping data with other investigators. The original design of the cohort has been described previously [25]. In brief, the cohort is a collaborative effort that brings together genomic data on Mayo Clinic patients obtained from research studies and EMR data to facilitate research.

Group Health Cooperative

The Group Health Cooperative/University of Washington eMERGE cohort is a collection of adult patients receiving care at Group Health Cooperative, an integrated delivery health care system in the Pacific Northwest. This cohort includes patients enrolled in the Adult Changes in Thought study and patients recruited for a biorepository by the Northwest Institute for Genetic Medicine. All participants in the cohort provided written and informed consent for their genetic information and EMR data to be used for research purposes. Study participants are at least 50 years of age, have a median of over 23 years of continuous enrollment at Group Health, and have received care in Group Health outpatient clinics documented by a comprehensive Epic© EMR system since 2004. All research involving these participants has been approved by the Group Health Cooperative Human Subjects Review Committee.

Mayo Clinic Primary Care Practice (PCIM)

PCIM is an adult internal medicine practice caring for patients over the age of 16 living within the local area. This patient population is self-insured; thus, PCIM has developed strategies including case management of chronic illnesses and EMR clinical decision support [26] to assist primary care providers with preventative services. Mayo Clinic has standardized care process models [27] throughout Mayo Clinic for chronic diseases including both HF with preserved and reduced EF. These process models and treatment recommendations differ for preserved and reduced EF.

Mayo Clinic Biobank

This Biobank is an institutional resource for biological specimens, patient-provided risk factor data, and clinical data that has been described in detail elsewhere [28]. In brief, adult patients from the Mayo Clinic/Mayo Clinic Health System sites in Rochester, Minnesota; LaCrosse, Wisconsin; and Jacksonville, Florida are invited to participate. For this study, only participants from the Rochester, MN site (n = 30,461) were included in the analyses. These participants were actively recruited from the Department of Medicine Divisions of Community Internal Medicine (18%), Executive Health (4%), Family Medicine (23%), General Internal Medicine (27%), Obstetrics and Gynecology (3%), Orthopedics (9%), or Preventive Medicine (10%). Community volunteers (6%) interested in participating were also included. Data were available from the EMR and from patient-provided information on current health, family health history, and various important factors known to confer risk for disease. Specifically, participants self-reported whether they had a personal and/or family history of HF as well as age of HF onset for those with a positive HF history.

Algorithm Development

Case/Non-case algorithm

International Classification of Diseases, 9^th Revision (ICD-9) code 428 was used based on previous previously reported yields [22]. In addition, we searched for positive mentions of HF from structured problem lists or problem list sections in clinical notes. For structured problem lists which are typically coded with SNOMED-CT, we applied recursive traversal of the descendants of the SNOMEDCT code 84114007 (HF) to indicate if the subject had a positive mention of HF. An NLP system, MedTagger, was used to help determine HF diagnosis from problem list sections of clinical notes [29]. In MedTagger (publically distributed www.ohnlp.org), besides a rule-based concept extraction engine which extracts concept mentions defined using regular expression, it also consists of i) a sectionizer adapted from SecTag to detect sections and ii) a rule-based context annotator adapted from ConText [30] assigning each concept mention a status modifier (i.e., positive, negative, and probable). Note that clinical notes in Mayo Clinic EMR are Clinical Document Architecture 1.0 compliant where sections have been codified. For non-Clinical Document Architecture compliant documents (Group Health EMR), the sectionizer was used to detect Diagnosis and other sections (i.e., Chief Complaints or Impressions as the Secondary Problem List section). To determine the date of first documented HF, the cross product of all ICD-9 and problem list dates were considered. Echocardiography measurements of left ventricular ejection fraction (EF) were extracted from structured database (Mayo Clinic EMR) and by deploying NLP to search the radiology reports for EF measurements (Group Health EMR). Multiple EF measurements from the same examination were averaged. HFrEF was defined as an average EF <50% and HFpEF ≥50% [31].

e-Epidemiology tool

Among the participants with “unknown” (i.e., not meeting the case or non-case definition), 100 patients were randomly selected for medical record abstraction. This information was used to broaden the algorithm by creating definitions for definite, probable, and possible HF as well as refinements to the non-case definition. These three hierarchical definitions of HF (i.e., definite, probable, possible) have decreasing stringency in terms of level of evidence. This classification strategy was adopted as it is used extensively to classify disease in cardiovascular epidemiologic research [32]. Definite HF requires the presence of ICD-9 and NLP within a relatively narrow time window. Probable HF requires five or more unique dates of either ICD-9 or NLP with a more liberal time window. In contrast, possible HF has minimal evidence of HF albeit the presence of an ICD-9 code, NLP hit, or low EF measurement, thus prohibiting them from being classified as non-cases. Patients are further classified by HF type within definite and probable groups, and are considered unknown to HF type if no qualifying EF is available. Non-cases were defined as the absence of any of these elements (i.e. ICD9, NLP) and a normal EF (i.e., ≥50%) if measured.

Validation and Statistical Analysis

The validation of the case/non-case algorithm was completed in two phases in the Mayo Clinic cohorts. First, 50 cases and 50 non-cases identified by the algorithm were randomly selected in the MayoGC/eMERGE Cohort. Trained nurse abstractors, blinded to disease, reviewed medical records to determine HF using Framingham Heart Failure Criteria [22], and HF date and type (i.e., HFrEF or HFpEF) for those who were identified as having HF. External validation was performed at Group Health using a trained medical chart abstractor who reviewed the charts of random samples of patients identified by the automated phenotype algorithm at Group Health. To determine the accuracy of the algorithm, positive and negative predictive values (PPV and NPV, respectively) were calculated as well as sensitivity and specificity correcting for verification bias [33, 34]. Estimated prevalence of HF corresponding to the sex averaged rates for 60–79 year olds for MayoGC/eMERGE and the 80+ year olds for Group Health were used based on published reports [35]. The expanded algorithm was validated by randomly selecting a total of 300 patients, 100 definite, 100 probable, 50 possible, and 50 non-cases equally divided between PCIM and Biobank cohorts. Trained abstractors reviewed the records to determine case/non-case, case type, and incident date. For the latter, the incident date was considered validated if the abstracted data and algorithm date occurred within 1 year of each other.

Results

Figure 1 illustrates the cohorts used in the development and validation of the case/non-case algorithm and the expanded algorithm.

Figure 1.

Algorithm development

An ICD-9 code for HF (428.X) was present in 93% of the cases. NLP analyses of the clinical notes identified six common terms/acronyms present in 89% of cases from the HF Cohort: multi-organ failure, cardiac failure, heart failure, CHF, LVF, and ventricular failure in the primary and secondary diagnosis sections. Using the combination of ICD-9 code and positive NLP hit, 99% of the cases were identified. Abstraction of the seven cases without an ICD-9 or an NLP hit revealed that all patients were in critical condition in the ICU at the time they met the Framingham HF criteria. Thus only a symptom-based algorithm would have been able to identify these patients. Since the HF Cohort study protocol included echocardiography, all cases were able to be classified as either preserved or reduced EF HF type.

Algorithm validation

The algorithm was run in 6,922 participants in the MayoGC/eMERGE (mean age 65 ± 12 years) and 5,861 at Group Health (mean age 90 ± 12 years). The algorithm performed with a positive predictive value of 0.94, a negative predictive value of 0.98 (MayoGC/eMERGE) and a positive predictive value of 0.80 and a negative predictive value of 1.0. (Group Health, Table 1). Sensitivity was 0.71 and 1.0 for MayoGC/eMERGE and Group Health, respectively. Specificity was similar across the two sites (Table 1).

Table 1.

Validation results per prevalence of the disease in each population

	*Estimated prevalence of heart failure	Positive predictive value	Negative predictive value	Sensitivity	Specificity
Mayo Genome Consortia (MayoGC)/eMERGE	6.2%	0.94	0.98	0.71	0.99
Group Health	10.1%	0.80	1.0	1.0	0.97

^*Estimated prevalence based on Go, A.S., Mozaffarian, D., Roger, V.L., et al. Heart disease and stroke statistics--2014 update: a report from the American Heart Association. Circulation. 2014;129, e28–e292 corresponding to the sex averaged rates for 60–79 year olds for MayoGC/eMERGE and the 80+ year olds for Group Health.

e-Epidemiology tool validation

The complete algorithm is provided in Web material and available online at pheKB.org. Validation of HF cases, case type, and index date was similar for the two cohorts and did not differ substantially between definite and probable definitions (Table 2). Likewise, the non-case definition which requires the complete absence of evidence of HF had good performance in both cohorts. Proportion of those with HF based on abstraction for those classified as possible HF cases differed between PCIM (48%) and Biobank (16%) populations but was poor in both cohorts despite the high prevalence of HF ICD9 codes (62% and 45% for PCIM and Biobank respectively).

Table 2.

Proportion of agreement comparing the electronic medical record algorithm to manual chart abstraction

	Case statusa		Case typeb		Incident datec
Heart failure definitions	PCIM (%)	Biobank (%)	PCIM (%)	Biobank (%)	PCIM (%)	Biobank (%)
Definite	100	96	96	98	80	83
Probable	98	100	100	96	84	76
Possible	48	16	n/a	n/a	n/a	n/a
Non-Case	100	100	n/a	n/a	n/a	n/a

PCIM Mayo Clinic Primary Care Practice

^aValidation included 300 cases randomly selected within the following strata; 100 definite, 100 probable, 50 possible, and 50 non-cases equally divided between PCIM and Biobank cohorts.

^bHeart failure type at the time of initial diagnosis; reduced ejection fraction (<50) or preserved ejection fraction (≥50) for validated cases.

^cIncident dates within 1 year of each other were considered to be in agreement for validated cases.

Characteristics of the PCIM are summarized in Table 3 by HF case/non-case and type. Of the 79,649 patients, the algorithm identified 3,318 definite and probable HF cases. Of these, case type was identified in 79% of those classified as definite and 65% of probable cases. By definition, all definite cases had ICD-9 code and NLP evidence, however, for probable cases, 99% had ICD-9 code evidence but only about 7% had NLP evidence. In either category, those of unknown HF type were less likely to have an EF measurement in their EMR. Of the PCIM population, 4.6% were categorized as possible HF. ICD-9 HF codes and EF measurements were common but NLP hits were less frequent.

Table 3.

Characteristics of Mayo Clinic Primary Care Internal Medicine Practice

Characteristics	Definite HF cases with reduced EF	Definite HF cases with preserved EF	Definite HF cases (unknown type)	Probable HF cases with reduced EF	Probable HF cases with preserved EF	Probable HF case (unknown type)	Possible HF cases	Non-cases
n	760	754	391	422	497	494	3,668	72,663
Sex, % female	39	61	53	35	62	57	50	56
Race, % white	93	92	94	96	96	96	93	80
Diabetes, % yes	45	48	48	42	46	43	34	12
Hypertension, % yes	92	95	96	93	97	95	86	31
ICD-9 Code 428, % yes	100	100	100	99	99	99	62	0
ICD-9 Code 428 unique dates, mean ± SD (range)*	19 ± 20 (1–165)	15 ± 17 (1–159)	16 ± 19 (1–189)	20 ± 19 (1–144)	15 ± 14 (1–97)	15 ± 13 (1–91)	2.0 ± 1.2 (1–10)	n/a
NLP HF term, % yes	100	100	100	7.6	7.7	5.1	12	0
NLP HF term unique dates, mean ± SD (range)*	12 ± 17 (1–200)	6.6 ± 8.6 (1–67)	8.8 ± 15 (1–154)	2.4 ± 2.6 (1–11)	2.0 ± 2.2 (1–12)	2.4 ± 2.1 (1–8)	1.4 ± 0.9 (1–8)	n/a
EF measured, % yes	100	100	80	100	99	70	87	21
EF measurement unique dates, mean ± SD (range)*	15 ± 13 (1–132)	11 ± 9.5 (1–131)	7.3 ± 7.0 (1–63)	12 ± 11 (1–87)	9.1 ± 7.1 (1–54)	5.7 ± 5.3 (1–48)	7.1 ± 6.3 (1–52)	4.1 ± 3.7 (1–54)
EF, mean ± SD	42 ± 16	59 ± 9.6	51 ± 15	40 ± 15	59 ± 10	51 ± 15	54 ± 12	61 ± 6.2
History of myocardial infarction, % yes	88	76	80	91	82	82	65	12
Medication history
Angiotensin converting enzyme use, % yes	79	69	73	83	70	67	55	14
Angiotensin receptor blocker use, % yes	20	26	17	21	23	16	12	3
Beta blocker use, % yes	80	73	69	76	69	60	58	16
Calcium channel blocker use, % yes	39	48	39	39	53	42	31	7

EF ejection fraction, HF heart failure, ICD-9 International Classification of Diseases, 9^th Revision, NLP natural language processing, SD standard deviation

^*For those with non-missing data

Of the 30,461 Biobank participants, the algorithm identified 606 definite or probable HF cases (Table 4). The Biobank was similar to PCIM, however, ICD-9 codes were somewhat less common (92–95%) and NLP evidence was more common (7–21%) compared to PCIM. Furthermore, case type was available for a greater number of patients in the Biobank, 90% of definite and 85% of probable cases.

Table 4.

Characteristics of Mayo Clinic Biobank Population

Characteristics	Definite HF cases with reduced EF	Definite HF cases with preserved EF	Definite HF cases (unknown type)	Probable HF cases with reduced EF	Probable HF cases with preserved EF	Probable HF case (unknown type)	Possible HF cases	Non- cases
n	207	173	40	75	84	27	1429	28,426
Sex, % Female	28	49	33	41	46	48	41	59
Race, % white	98	98	98	97	99	100	98	96
Diabetes, % yes	41	45	63	32	46	44	31	14
Hypertension, % yes	89	94	95	80	93	100	77	42
ICD-9 Code 428, % yes	100	100	100	92	95	100	45	0
ICD-9 Code 428 unique dates, mean ± SD (range)*	11 ± 11 (1–65)	9.6 ± 9.6 (1–61)	11 ± 9.1 (1–36)	14 ± 12 (1–54)	10 ± 7.4 (1–44)	11 ± 7.5 (1–30)	1.8 ± 1.3 (1–25)	n/a
NLP HF term, % yes	100	100	100	21	13	7.4	15	0
NLP HF term unique dates, mean ± SD (range)*	6.2 ± 8.7 (1–69)	4.7 ± 6.6 (1–47)	4.0 ± 4.7 (1–21)	4.6 ± 4.7 (1–17)	2.8 ± 2.4 (1–7)	1.5 ± 0.7 (1–2)	1.4 ± 0.9 (1–8)	n/a
EF measured, % yes	100	100	83	100	100	93	94	32
EF measurement unique dates, mean ± SD (range)*	17 ± 13 (1–96)	13 ± 9.0 (2–42)	9.3 ± 6.4 (2–23)	20 ± 16 (1–91)	14 ± 12 (2–82)	12 ± 10 (2–43)	9.2 ± 8.1 (1–66)	4.4 ± 4.2 (1–68)
EF, mean ± SD	43 ± 14	58 ± 10	46 ± 17	43 ± 13	58 ± 8.8	56 ± 7.1	53 ± 11	61 ± 6.1
History of Myocardial Infarction, % yes	88	81	93	79	83	89	62	19
Medication History
Angiotensin converting enzyme use, % yes	88	74	78	80	77	70	52	20
Angiotensin receptor blocker use, % yes	27	23	18	23	30	33	16	5.5
Beta blocker use, % yes	73	60	75	68	61	63	44	19
Calcium channel blocker use, % yes	30	47	23	35	44	44	25	9.4
Self-reported family history of HF, % Yes	36	40	36	47	32	36	34	26
Self-reported history of HF, n (%)	47	44	50	57	36	50	11	0.8

EF ejection fraction, HF heart failure, ICD-9 International Classification of Diseases, 9th Revision, NLP natural language processing, SD standard deviation

^*For those with non-missing data

Discussion

The rapid expansion of electronic health information necessitates studies to establish valid and reliable e-Epidemiologic tools. We developed a multi-modal EMR HF algorithm that combines structured and unstructured EMR data to accurately classify HF in clinical populations. We further expanded the utility of the algorithm by incorporating hierarchical categories enabling the classification of a complete population and providing multiple methods to extract EF measurement. The latter is essential for the algorithm to accurately distinguish HF with preserved or reduced EF, a critical feature to characterize the burden of HF in populations.

We have developed a cost-effective and robust EMR HF algorithm and have demonstrated its effectiveness in characterizing HF patients across institutions with different EMR systems (i.e., Mayo Clinic and Group Health) and in two different clinic-based populations (i.e., primary care and Biobank). Further, we demonstrate that combining structured (e.g., ICD-9 codes) and unstructured (i.e., NLP) data improves the accuracy of identifying HF patients compared to administrative data alone [36]. Importantly the algorithm provides several methods for capturing EF measurements to use in the classification of HF as some institutions store this information in structured databases and other sites require NLP to either extract the EF value or free text responses (i.e., normal EF). This algorithm will enable population management strategies by identifying patients in practice to build registries to facilitate quality measures.

Furthermore, understanding the performance of the algorithm in a Biobank population is crucial as participants in Biobanks may have greater diversity in terms of EMR depth and completeness as compared to a primary care population. Biobanks are commonly comprised of community volunteers who may or may not receive regular care at the institution supporting the Biobank and thus missing or incomplete data within the EMR may hinder the ability to accurately classify case/non-case for both EMR-based algorithms as well as manual abstraction and results in subsets of patients with indeterminate disease. For example, patients may not be billed if a HF episode happened in the past or it occurred at another institution and they are currently asymptomatic. Likewise, patients may be inappropriately coded with HF when in fact it was another diagnosis.

While the distinction of definite, probable, and possible HF is not clinically meaningful for a physician treating a patient, the ability to characterize who, in the given population, falls outside the case/non-case definition is essential for an effective e-Epidemiology tool to facilitating population research and understanding the potential biases present.

In conclusion, the magnitude of the HF epidemic necessitates the development of cost-effective methods to study and identify strategies to alleviate the substantial global burden and cost of HF. Further, the differentiation of HF by type (i.e., HFrEF and HFpEF) has clinical implications that may improve quality metrics for health care institutions. Using a combination of structured and unstructured EMR data, we have developed and validated a transportable e-Epidemiologic tool to facilitate population research.

Abbreviations

EF

ejection fraction

eMERGE

Electronic Medical Records and Genomics

EMR

Electronic medical record

HF

Heart failure

HFpEF

HF with preserved ejection fraction

HFrEF

HF with reduced ejection fraction

ICD-9

International Classification of Diseases, 9^th Revision

NLP

Natural language processing

PCIM

Mayo Clinic Primary Care Practice