Skip to main content
CMAJ Open logoLink to CMAJ Open
. 2015 Oct 2;3(4):E413–E418. doi: 10.9778/cmajo.20150060

Validity and utility of ICD-10 administrative health data for identifying ST- and non-ST-elevation myocardial infarction based on physician chart review

Alka B Patel 1, Hude Quan 1, Robert C Welsh 1, Jessica Deckert-Sookram 1, Wayne Tymchak 1, Sunil Sookram 1, Ian Surdhar 1, Padma Kaul 1
PMCID: PMC4987155  PMID: 27570759

Abstract

Background:

Health care administrative databases are useful for assessing the population-level burden of disease and examining issues related to access, costs and quality of care. In these databases, the diagnoses and procedures are coded with the use of the World Health Organization International Classification of Diseases (ICD). We examined the validity of 2 ICD-10 coding definitions for categorizing patients with acute myocardial infarction (MI) as having ST-elevation MI (STEMI) or non-ST-elevation MI (non-STEMI).

Methods:

Charts of patients with acute MI discharged between April and June 2007 from 3 hospitals in Edmonton, were reviewed to define the acute MI subtype (i.e., STEMI v. non-STEMI). The agreement between clinician chart review and STEMI/non-STEMI classification based on the standard (ICD-10 I21.x) and the supplementary electrocardiogram (ECG) codes (R94.3x) was determined. We assessed the effect of these alternative definitions on in-hospital mortality estimates by applying them to the data for all patients with acute MI admitted to hospital in the province from April 2007 to March 2010.

Results:

Of the 297 patients, 49.2% were identified as having STEMI based on chart review, 44.4% using the standard definition, and 44.1% using the ECG definition. Both the standard and ECG definitions provided high agreement (92% for STEMI and 100% for non-STEMI) with the chart review classification. In the larger population-level cohort (n = 15 148), use of the standard definition or the ECG definition did not affect in-hospital mortality estimates for patients with STEMI and those with non-STEMI.

Interpretation:

The standard definition appears equivalent to the definition using supplementary ECG codes to subcategorize patients with acute MI as having STEMI or non-STEMI. These findings may be relevant for the development of later versions of ICD codes.


Health care administrative databases, such as those maintained by the Centers for Medicare and Medicaid Services in the United States and by provincial and federal agencies in Canada and other countries, are extremely useful for assessing the population-level burden of disease and examining issues related to access, costs and quality of care.1-5 In these databases, the diagnoses and procedures are coded with the use of the World Health Organization International Classification of Diseases (ICD).6 As of October 2015, all developed countries, including the US, are using version 10 of the ICD. Because the World Health Organization permits its member countries to add country-specific diagnosis codes or specify conditions using subcodes in ICD-10, several countries have developed their own ICD-10 versions.

The incidence and clinical characteristics of patients presenting with ST-elevation myocardial infarction (STEMI) and those of patients with non-ST-elevation myocardial infarction (non-STEMI) vary greatly.7,8 These differences underline the importance of accurately classifying acute myocardial infarction (MI) subtypes when examining patient outcomes. Historically, owing primarily to limitations in coding algorithms, studies using administrative data to examine treatment and outcomes of acute MI have grouped STEMI and non-STEMI together.9,10 However, the improved ICD-10 codes distinguish between patients with STEMI and those with non-STEMI.11 In April 2007, the Canadian version of ICD-10 introduced secondary diagnosis codes for cardiovascular function based on the electrocardiogram (ECG) findings.12 It is not known whether these supplementary codes are more accurate than the standard ICD-10 codes in differentiating between the acute MI subtypes.

The primary objective of our study was to evaluate the agreement between chart review by clinicians and the standard definition (based on ICD-10 Q-wave codes alone) and ECG definition (based on supplementary ECG codes) for classifying STEMI and non-STEMI. We also examined the extent to which in-hospital mortality estimates were affected by the use of the alternative definitions.

Methods

Defining STEMI and non-STEMI in chart data

The charts of all patients discharged from hospital with a diagnosis of acute MI in Edmonton (population of about 1 million), undergo review by a trained chart abstractor. This review is conducted as part of a dedicated quality-assurance program linked to the Vital Heart Response clinical program. This program is a regional reperfusion program focused on delivering expeditious evidence-based care to patients with STEMI.

To validate the standardized chart abstraction process and confirm appropriate patient selection, we obtained the charts of all patients discharged between Apr. 1 and June 30, 2007, from the 3 major hospitals in the Vital Heart Response program. The charts of patients with a diagnosis of acute MI were independently abstracted to obtain an approximately equal sample of patients identified as having STEMI and non-STEMI. The ECGs were then blinded. Subsequently, 2 experienced clinicians (S.S. and W.T.) interpreted the 12-lead ECG and reviewed peak myocardial enzymes to determine the clinical diagnosis of non-STEMI versus STEMI.13 If the 2 physicians disagreed on the diagnosis, a third physician (R.C.W.) read the ECG to determine STEMI or non-STEMI. The classification of acute MI subtype based on clinician chart review was used as the reference standard.

Defining STEMI and non-STEMI in administrative data

We used 2 definitions to categorize patients with acute MI as having STEMI or non-STEMI. The standard definition was the presence of the following codes in the primary diagnosis field: ICD-10 codes I21.0-3 for STEMI and I21.4x for non-STEMI.

A patient presenting with an ECG indicative of STEMI can progress to Q-wave MI, a non-Q-wave MI or an aborted MI, whereas those with no ST-elevation on the ECG are considered to have non-STEMI and can progress only to non-Q-wave MI or unstable angina.12 This means that all patients with a code for Q-wave MI (I21.0-3) should have STEMI, but some with a code for non-Q-wave MI (I21.4x) could also have STEMI. Thus, the supplementary ECG codes that can distinguish between ECG results suggestive of STEMI (R94.30) or non-STEMI (R94.31) could be useful in accurately categorizing patients into the 2 subtypes.12 The ECG codes were introduced in April 2007 in Canada, and therefore we used the charts of patients who were discharged from the 3 hospitals between April and June 2007. In our study, the ECG definition to categorize patients with acute MI as having STEMI or non-STEMI was the presence of ICD-10 code R94.30 in any diagnosis field for STEMI and ICD-10 code R94.31 in any diagnosis field for non-STEMI.

Statistical analysis

We linked the chart data with hospital discharge abstract data from April to June 2007 using the unique identifier of personal health number. The discharge abstract data include age, sex, length of stay, up to 25 diagnoses, up to 16 procedures and discharge status. We extracted these linked Discharge Abstract Database records using the date of admission and the most responsible diagnosis of I21 (acute MI). We removed the data for patients with invalid personal health numbers to create our final sample.

We compared the agreement between the clinician chart review classification and the STEMI versus non-STEMI classification based on the standard definition and the ECG definition. The data for patients with an unspecified MI (I21.9) or an unspecified ECG result (R94.38) were excluded from the analysis. The agreement between chart review and the 2 subtype definitions was calculated overall and by sex and age.

To determine the effects of subtype definition on in-hospital mortality estimates, we applied the standard and ECG definitions to a cohort of all patients admitted to hospital with acute MI in the province of Alberta between Apr. 1, 2007, and Mar. 31, 2010. The data set for this cohort contained information on age, sex, length of stay, diagnoses, procedural interventions and in-hospital mortality. Patients who had a primary diagnosis code of I21.x were included in this assessment. A record of R94.3x (supplementary ECG code) in any of the secondary diagnosis fields was captured. The unit of analysis was the patient. To test the utility of the coding definitions, we calculated the in-hospital mortality among patients with STEMI and those with non-STEMI identified with the use of the standard and ECG definitions.

We followed the Standards for Reporting of Diagnostic Accuracy (STARD) guidelines for reporting studies of diagnostic accuracy.14 Data analysis was conducted with the use of SAS version 9.2.

Ethics approval

The ethics board of the University of Alberta felt the study was for quality-assurance purposes and did not require ethics approval.

Results

The 2 physicians reviewed 319 charts of patients with a primary diagnosis of acute MI (Figure 1). They disagreed on the classification of the acute MI as STEMI or non-STEMI in 3 cases (0.9%).

Figure 1.

Figure 1

Steps in defining a study sample for validating coding of acute myocardial infarction (MI) subtypes. ECG = electrocardiogram, PHN = personal health number, STEMI = ST-elevation myocardial infarction. *Excluded from analysis.

Agreement between administrative and chart data

Administrative data were linked with chart review data for 302 of the 319 patients (Figure 1). Once duplicate records and records with invalid personal health numbers were removed, we had linked data for 297 patients. Seven patients were excluded from the validation of the standard definition because they had a code of I21.9 ("Acute myocardial infarction, unspecified"), and 8 patients were excluded from the validation of the ECG definition because they had a code of R94.38, which does not specifically define the STEMI or non-STEMI subtype, or had missing or conflicting R94.3x coding.

The number of patients classified as having STEMI or non-STEMI by chart review, standard definition and ECG definition are presented in Table 1. For STEMI, the standard definition had a higher level of agreement with chart review overall (92.3%) than did the ECG definition (91.6%). This higher level of agreement remained for men and for both younger (age < 65 yr) and older (age ≥ 65 yr) patients. The ECG definition had a higher level of agreement with chart review for women (89.1%) than did the standard definition (87.0%). With both definitions, the level of agreement was lower for older patients than for younger patients.

Table 1: Validity of the International Classification of Diseases version 10, Canadian edition, coding for acute MI subtypes.

Group Reference standard (chart review) Standard definition*
n = 290
ECG definition†
n = 289
No. of patients Agreement, % No. of patients Agreement, %
STEMI Non-STEMI STEMI Non-STEMI STEMI Non-STEMI STEMI Non-STEMI STEMI Non-STEMI
Overall 143 147 132 158 92.3 100.0 131 158 91.6 100.0
Women 46 55 40 61 87.0 100.0 41 60 89.1 100.0
Men 97 92 92 97 94.8 100.0 90 98 92.8 100.0
Age < 65 yr 82 67 81 70 98.8 100.0 79 69 96.3 100.0
Age 65 yr 61 80 51 88 83.6 100.0 52 89 85.2 100.0

Note: ECG = electrocardiogram, MI = myocardial infarction, STEMI = ST-elevation MI.

*I21.0-3 for STEMI v. I21.4x for non-STEMI.

†R94.30 ("ECG suggestive of STEMI") v. R94.31 ("ECG suggestive of NSTEMI"). n = 289 because the chart review for the ECG definition is missing 1 male patient under 65 years of age.

For non-STEMI, the level of agreement between the chart review and each of the standard and ECG definitions was perfect; that is, all patients identified by chart review as having non-STEMI were classified as having non-STEMI with both definitions. There were, however, patients with STEMI who were categorized as having non-STEMI. The level of misclassification was as follows: 7.7% of patients with STEMI were classified as having non-STEMI with the standard definition, and 8.4% were classified as having non-STEMI with the ECG definition.

Effect of definitions on in-hospital mortality estimates

We applied the standard and ECG coding definitions for STEMI and non-STEMI to 15 148 patients admitted to hospital with a primary diagnosis of acute MI (I21.x) in Alberta between April 2007 and March 2010. Based on the standard definition, 5713 patients (37.7%) were classified as having STEMI and 9435 (62.3%), non-STEMI. There were 528 patients (3.5%) with an unspecified abnormal ECG result (R94.38) or missing supplementary R94.3x code, and 20 patients (0.1%) had both the code R94.30 and the code R94.31. When only the ECG definition was used to define the acute MI subtypes, 5530 patients (36.5%) were classified as having STEMI and 9070 (59.9%), non-STEMI. Table 2 summarizes these findings and shows that the calculation of in-hospital mortality was similar irrespective of which definition was used.

Table 2: Application of the standard and ECG coding definitions to data for all patients admitted to hospital with acute MI in Alberta between Apr. 1, 2007, and Mar. 31, 2010 (n = 15 148*).

Group; acute MI subtype Coding definition Mortality, % No.
of patients
Overall
Standard definition
STEMI I21.0-3 5.51 5713
Non-STEMI I21.4x 3.95 9435
ECG definition
STEMI R94.30 5.42 5530
Non-STEMI R94.31 3.89 9070
Female
Standard definition
STEMI I21.0-3 8.79 1525
Non-STEMI I21.4x 4.82 3174
ECG definition
STEMI R94.30 8.63 1460
Non-STEMI R94.31 4.77 3038
Male
Standard definition
STEMI I21.0-3 4.32 4188
Non-STEMI I21.4x 3.51 6261
ECG definition
STEMI R94.30 4.28 4070
Non-STEMI R94.31 3.45 6032
Age < 65 yr
Standard definition
STEMI I21.0-3 2.15 3342
Non-STEMI I21.4x 0.62 4009
ECG definition
STEMI R94.30 2.20 3266
Non-STEMI R94.31 0.57 3868
Age ≥ 65 yr
Standard definition
STEMI I21.0-3 10.25 2371
Non-STEMI I21.4x 6.41 5426
ECG definition
STEMI R94.30 10.07 2264
Non-STEMI R94.31 6.36 5202

Note: ECG = electrocardiogram; MI = myocardial infarction; STEMI = ST-elevation MI.

*For ECG definition n = 14 600 (528 patients had an unspecified abnormal ECG result [R94.38] or missing supplementary R94.3x code, and 20 patients had both the code R94.30 and the code R94.31).

Interpretation

Our study examining the validity of 2 definitions, one using standard ICD-10 codes and the second using supplementary ECG codes, for categorizing patients with acute MI as having STEMI or non-STEMI has several interesting findings. First, we found that both definitions were good at differentiating between the 2 subtypes in a sample of patients with confirmed acute MI. The agreement between both definitions and clinician chart review was high across all patient subgroups with the exception of the coding of STEMI for patients older than 65 years, where the agreement was slightly lower. Second, for STEMI, the standard definition had a slightly higher level of agreement with chart review than did the ECG definition. Third, for non-STEMI, the agreement with the chart review for both the standard definition and the ECG definition was perfect, that is, all patients identified by chart review as having non-STEMI were classified as having non-STEMI with both definitions. And finally, when we applied the 2 definitions to a database of patients with acute MI admitted to hospital, we found that the calculation of in-hospital mortality was similar irrespective of which definition was used.

The validation of STEMI and non-STEMI coding relies on the validation of acute MI first. Studies have shown that acute MI is coded well with the use of ICD. In a recent systematic review, Metcalfe and associates15 reported that there have been several studies validating the case definitions for acute MI, with most studies focused on ICD-9 and no studies validating exclusively ICD-10 codes. The reported validity of these diagnostic codes for identifying acute MI has varied from a sensitivity of 66.0% to 95.1% and a specificity of 80.2% to 100.0%, depending on the subgroup, database and area studied.15 Although that systematic review did not identify any studies validating the acute MI subtypes as defined using recently revised ICD coding definitions, ICD-9-CM16 and ICD-1011 codes for acute MI have been used to differentiate between patients with STEMI and those with non-STEMI.

Studies have been conducted evaluating the validity of ICD-9 coding for acute MI. Varas-Lorenzo and colleagues17 studied Saskatchewan data from November 1999 to December 2001 and found that ICD-9 codes for acute MI had a positive predictive value of 0.95 (95% confidence interval 0.91 to 0.98) for identifying patients with acute MI based on detailed chart review as the gold standard. Although many studies validating the case definitions of acute MI using ICD coding have shown similar high positive predictive values or sensitivity,15 they have not distinguished the validity of acute MI subtypes using ICD-coded data.

Studies have been conducted in the US validating the acute MI subtype coding definitions for ICD-9-CM. With the current ICD-9-CM coding definitions, patients with non-STEMI are those with a diagnosis code of 410.7, and patients with codes 410-410.8 are classified as having STEMI.18 One US study with a very large sample (over 1 million) showed that 10.9% of patients who were found to have STEMI based on an ECG received the code 410.71, and 38.8% of patients with ECG indicative of non-STEMI had an ICD-9 code other than 410.71.19 Our study shows that the Canadian ICD-10 codes are less likely to misclassify both STEMI and non-STEMI than has previously been reported for ICD-9-CM.

Using data from 2000-2001, Woo and associates20 found that MI documentation in medical charts was incomplete and inconsistent in its ability to distinguish between STEMI and non-STEMI and that up to 20% of medical records could not be used for retrospective MI-subtype-specific research. Inconsistencies in the medical record are one possible reason for the misclassification of coded information. If the information in the chart is not recorded completely and correctly, coders cannot transfer this information to the administrative data codes that could then be used to distinguish between acute MI subtypes. When we applied the subtype coding definitions in our study, we found that the charts of about 0.1% of patients in our sample had inconsistent coding, including having an R94.3x code representing one subtype and an R94.3x code representing a different subtype. The lower level of agreement for the coding of STEMI for patients aged 65 years or more in our study may have been due to less complete documentation by physicians for older patients with several conditions than for younger patients. It is also possible that coders spent less time on older patients' records when there were large amounts of information on conditions.

Limitations

Our study has limitations. It does not provide information on the validity of acute MI coding. Instead it examines only the validity of subcategorizing patients with acute MI as having STEMI or non-STEMI. Therefore, given that our study included only patients with acute MI, we were unable to provide the positive predictive value, negative predictive value, sensitivity or specificity associated with the acute MI diagnosis. Future studies based on a random sample of all patients admitted to several acute care hospitals in Alberta would be useful in obtaining these measures. The reference standard for this study was a sample of charts of patients discharged over a 3-month period from 3 hospitals in 1 Canadian city. To ensure the generalizability of our findings, it would be valuable to conduct this type of validation study for ICD-10 coding of acute MI subtypes in different regions of the country as well as internationally. Although our study was limited to a smaller study area, it covered a range of hospital settings in a large urban centre.

Conclusion

Among patients with acute MI, the standard ICD-10 definition to classify patients as having STEMI or non-STEMI was as accurate as the supplementary definition based on ECG codes. Our findings show that the acute MI subtype codes are valid and can be used in population-based outcome studies. These findings may be relevant for the development of later versions of ICD codes.

Supplemental information

For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/3/4/E413/suppl/DC1

Supplementary Material

Online Appendices

Footnotes

Funding: During the course of the study, Alka Patel, Hude Quan and Padma Kaul were supported by Alberta Innovates Health Solutions Doctoral Studentship, Health Scholar and Population Health Investigator awards, respectively.

References

  • 1.Colonna M, Mitton N, Schott AM, et al. Joint use of epidemiological and hospital medico-administrative data to estimate prevalence. Application to French data on breast cancer. Cancer Epidemiol. 2012;36:116–21. doi: 10.1016/j.canep.2011.12.001. [DOI] [PubMed] [Google Scholar]
  • 2.Garland A, Yogendran M, Olafson K, et al. The accuracy of administrative data for identifying the presence and timing of admission to intensive care units in a Canadian province. Med Care. 2012;50:e1–6. doi: 10.1097/MLR.0b013e318245a754. [DOI] [PubMed] [Google Scholar]
  • 3.Haley VB, Van Antwerpen C, Tserenpuntsag B, et al. Use of administrative data in efficient auditing of hospital-acquired surgical site infections, New York State 2009-2010. Infect Control Hosp Epidemiol. 2012;33:565–71. doi: 10.1086/665710. [DOI] [PubMed] [Google Scholar]
  • 4.Manuel DG, Lam K, Maaten S, et al. Using administrative data to measure the extent to which practitioners work together: "interconnected" care is common in a large cohort of family physicians. Open Med. 2011;5:e177–82. [PMC free article] [PubMed] [Google Scholar]
  • 5.Meenan RT, Goodman MJ, Fishman PA, et al. Pooling multisite administrative data for economic analysis. Expert Rev Pharmacoecon Outcomes Res. 2002;2:477–83. doi: 10.1586/14737167.2.5.477. [DOI] [PubMed] [Google Scholar]
  • 6.Jetté N, Quan H, Hemmelgarn B, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data. Med Care. 2010;48:1105–10. doi: 10.1097/MLR.0b013e3181ef9d3e. [DOI] [PubMed] [Google Scholar]
  • 7.McManus DD, Gore J, Yarzebski J, et al. Recent trends in the incidence, treatment, and outcomes of patients with STEMI and NSTEMI. Am J Med. 2011;124:40–7. doi: 10.1016/j.amjmed.2010.07.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Montalescot G, Dallongeville J, Van Belle E, et al. STEMI and NSTEMI: Are they so different? 1 year outcomes in acute myocardial infarction as defined by the ESC/ACC definition (the OPERA registry). Eur Heart J. 2007;28:1409–17. doi: 10.1093/eurheartj/ehm031. [DOI] [PubMed] [Google Scholar]
  • 9.Weiss ES, Chang DD, Joyce DL, et al. Optimal timing of coronary artery bypass after acute myocardial infarction: a review of California discharge data. J Thorac Cardiovasc Surg . 2008;135:503–11. 511.e1-3. doi: 10.1016/j.jtcvs.2007.10.042. [DOI] [PubMed] [Google Scholar]
  • 10.Tu JV, Austin PC, Chan BT. Relationship between annual volume of patients treated by admitting physician and mortality after acute myocardial infarction. JAMA. 2001;285:3116–22. doi: 10.1001/jama.285.24.3116. [DOI] [PubMed] [Google Scholar]
  • 11.Patel AB, Quan H, Faris P, et al. Temporal associations of early patient transfers and mortality with the implementation of a regional myocardial infarction care model. Can J Cardiol. 2011;27:731–8. doi: 10.1016/j.cjca.2011.08.114. [DOI] [PubMed] [Google Scholar]
  • 12.Appendix A - acute coronary syndrome (ACS) and related interventions. In: Canadian coding standards for version 2012 ICD-10-CA and CCI. Rev ed. Ottawa: Canadian Institute for Health Information. 2012. pp. 383–7. [Google Scholar]
  • 13.Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in emergency medicine research: Where are the methods? Ann Emerg Med. 1996;27:305–8. doi: 10.1016/s0196-0644(96)70264-0. [DOI] [PubMed] [Google Scholar]
  • 14.Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. [accessed 2015 Sept. 23]. Available www.stard-statement.org/ [DOI] [PubMed]
  • 15.Metcalfe A, Neudam A, Forde S, et al. Case definitions for acute myocardial infarction in administrative databases and their impact on in-hospital mortality rates. Health Serv Res. 2013;48:290–318. doi: 10.1111/j.1475-6773.2012.01440.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Zhang Z, Fang J, Gillespie C, et al. Age-specific gender differences in in-hospital mortality by type of acute myocardial infarction. Am J Cardiol. 2012;109:1097–103. doi: 10.1016/j.amjcard.2011.12.001. [DOI] [PubMed] [Google Scholar]
  • 17.Varas-Lorenzo C, Castellsague J, Stang MR, et al. Positive predictive value of ICD-9 codes 410 and 411 in the identification of cases of acute coronary syndromes in the Saskatchewan Hospital automated database. Pharmacoepidemiol Drug Saf. 2008;17:842–52. doi: 10.1002/pds.1619. [DOI] [PubMed] [Google Scholar]
  • 18.Cannon CP. Update to International Classification of Diseases, 9th Revision codes: distinguishes STEMI from NSTEMI. Crit Pathw Cardiol. 2005;4:185–6. doi: 10.1097/01.hpc.0000185739.52581.92. [DOI] [PubMed] [Google Scholar]
  • 19.Steinberg BA, French WJ, Peterson ED, et al. Missed diagnosis of the diagnosis codes: comparison of International Classification of Diseases, 9th Revision coding and ST- versus non-ST-elevation myocardial infarction diagnosis in the National Registry of Myocardial Infarction. Crit Pathw Cardiol. 2006;5:59–63. doi: 10.1097/01.hpc.0000202298.34924.b1. [DOI] [PubMed] [Google Scholar]
  • 20.Woo KS, Ghali WA, Southern DA, et al. Feasibility of determining myocardial infarction type from medical record review. Can J Cardiol. 2008;24:115–7. doi: 10.1016/s0828-282x(08)70565-8. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Online Appendices

Articles from CMAJ Open are provided here courtesy of Canadian Medical Association

RESOURCES