Assessment of Administrative Data to Identify Acute Myocardial Infarction in Electronic Health Records

To improve the efficiency of clinical research, there is interest in using electronic health record (EHR) data for outcome ascertainment (1). However, EHR data are designed for clinical care and billing and their utility for event capture is largely untested (2,3). We investigated 4 published computable phenotype definitions on the basis of administrative data for acute myocardial infarction (AMI) compared with a gold standard AMI classification created through chart review. The 4 algorithms were published by:1) the Agency for Healthcare Research and Quality;2) Centers for Medicare & Medicaid Services (CMS) Chronic Conditions Warehouse; 3) CMS Quality Net; and 4) Mini-Sentinel (Table 1).

TABLE 1.

Sensitivity and Specificity of Computable Phenotype Algorithms for AMI Compared With AMI Classification on the Basis of the Third Universal Definition

Administrative Algorithm	Algorithm Components	Sensitivity (95% CI)	Specificity (95% CI)
Agency for Healthcare Research and Quality	Inpatient claims/principal (first position) discharge or final diagnosis ICD-9 codes: 410.X, 410.XX	0.85 (0.81–0.89)	0.99 (0.99–1.00)
Centers for Medicare and Medicaid Services (CMS) Chronic Conditions Warehouse	Inpatient claims/first or second position in discharge or final diagnosis ICD-9 codes: 410.X1	0.99 (0.98–1.00)	0.99 (0.99–1.00)
CMS Quality Net	Inpatient claims/principal (first position) discharge or final diagnosis/LOS >1 day ICD-9 codes: 410.X1, 410.X0	0.83 (0.79–0.87)	0.99 (0.99–1.00)
Mini-Sentinel	Inpatient claims/principal (first position) discharge or final diagnosis, ICD-9 codes: 410.X1, 410.X0 Emergency room claim/death within 1 day/principal (first position) discharge or final diagnosis, ICD-9 codes: 410.X1, 410.X0, 411.1, 411.8, 411.8X, 413.X	0.79 (0.74–0.84)	0.99 (0.99–1.00)

AMI = acute myocardial infarction; CI = confidence interval; CMS = Centers for Medicare and Medicaid Services; ICD-9 = International Classification of Diseases, Ninth Revision; LOS = length of stay.

For the present analysis, 150 encounters at Duke between January 2010 and July 2011 were selected for review via stratified random sampling on the basis of all 4 definitions. The population included Durham County residents age ≥18 years. After implementation of the algorithms, the population was stratified according to 1 of 3 criteria: 1) those encounters for which all 4 phenotypes indicated that an AMI occurred; 2) 1 to 3 phenotypes indicated that an AMI event occurred; and 3) all 4 phenotypes indicated no AMI occurred. The sample size was calculated on the basis of assumptions about positive predictive value and 1 – negative predictive value within each strata and was chosen to minimize the variance estimate of sensitivity (4). Cardiologist reviewer pairs independently assessed AMI events on the basis of the Third Universal Definition of Myocardial Infarction (UDMI) (5) through chart review. Reviewers were blinded to algorithm output. Disagreements were resolved by a third adjudicator. Following adjudication, the true AMI status was established for each encounter against which the administrative algorithms were evaluated. Inter-rater agreement was assessed using Fleiss’s kappa statistics. The sensitivity/specificity and 95% confidence intervals of each definition compared with chart review were calculated (4). Sensitivity/specificity estimates were computed on the basis of proportions in all records during the sampling period.

Of the 150 encounters, 70 (47%) were an AMI by the UDMI and 79 (53%) did not meet the UDMI definition; the specific encounter could not be located for 1 chart. Overall, the physician chart reviewers agreed on the assessment of AMI 100% of the time (kappa = 1) in the no positive phenotype stratum. There was 72% agreement in the strata in which 1 to 3 phenotypes were positive (kappa = 0.51), and 78% agreement in the strata in which all phenotypes were positive (kappa = 0.42). When assessing which components of the UDMI were most strongly associated with disagreements, we found that cardiac biomarker changes and symptoms of ischemia were most strongly associated with disagreements compared with electrocardiogram changes, imaging evidence, or cardiac death/autopsy findings of AMI.

Of the 70 AMIs by chart review, all algorithms were positive for 19 (27%) encounters, and 1 to 3 algorithms were positive for 51 (73%). Table 1 presents the sensitivity/specificity of each algorithm for AMI identification compared with the UDMI.

In summary, the CMS Chronic Conditions Warehouse was found to have the highest sensitivity and a high specificity via assessment of inpatient claims with the International Classification of Diseases, Ninth Revision codes of 410.X1 in the first or second position for discharge or final diagnosis. The high sensitivity supports its use to rule out an AMI in those with negative administrative data, and the high specificity rules in an AMI event in those with positive administrative data. Importantly, these findings were observed in a single-center analysis of Durham County residents. We did not specifically document details related to completeness of electrocardiography data or subsequent revascularization.

Importantly, prior analyses demonstrated that classification of nonfatal events in routinely-recorded EHR-type data is imperfect (2). EHRs are heterogeneous, and data elements are inconsistently defined(3). The complexity of the healthcare system can affect the utility of the EHR for outcome ascertainment, as patients receive care from multiple providers and institutions. An improved understanding of how to make administrative algorithms more accurate would facilitate outcomes research. Future research is needed to validate these observations and investigate other endpoints including revascularization, stroke, and heart failure events.