Chart validation of inpatient ICD-9-CM administrative diagnosis codes for acute myocardial infarction (AMI) among intravenous immune globulin (IGIV) users in the Sentinel Distributed Database

Abstract

Background:

The Sentinel Distributed Database (SDD) is a large database of patient-level administrative healthcare records, primarily derived from insurance claims and electronic health records, and is sponsored by the U.S. Food and Drug Administration for medical product safety evaluations. Acute myocardial infarction (AMI) is a common study endpoint for drug safety studies that rely on health records from the SDD and other administrative databases.

Purpose:

In this chart validation study, we report on the positive predictive value (PPV) of inpatient ICD-9-CM AMI administrative diagnosis codes (410.x1 and 410.x0) in the SDD.

Methods:

As part of an assessment of thromboembolic adverse event risk following treatment with intravenous immune globulin (IVIG), charts were obtained for 103 potential post-IVIG AMI cases. Charts were abstracted by trained nurses and physician-adjudicated based on pre-specified diagnostic criteria.

Results:

AMI status could be determined for 89 potential cases. The PPVs for the inpatient AMI diagnoses recorded in the SDD were 75% overall (95% CI: 65–84%), 93% (95% CI: 78–99%) for principal-position diagnoses, 88% (95% CI: 72–97%) for secondary diagnoses, and 38% (95% CI: 20–59%) for position-unspecified diagnoses (e.g., diagnoses originating from separate physician claims associated with an inpatient stay). Of the confirmed AMI cases, demand ischemia was the suspected etiology more often for those coded in secondary or unspecified positions (72% and 40%, respectively) than for principal-position AMI diagnoses (21%).

Conclusions:

PPVs for principal and secondary AMI diagnoses were high and similar to estimates from prior chart validation studies. Position-unspecified diagnosis codes were less likely to represent true AMI cases.

Introduction

In this paper we report on the positive predictive values (PPVs) of inpatient diagnosis codes for acute myocardial infarction (AMI) within the Sentinel Distributed Database (SDD). The SDD is a database of longitudinal, patient-level medical and prescription data obtained from a number of participating Data Partners (i.e., large insurers and integrated care delivery systems) from across the U.S. The SDD and Sentinel program are sponsored by the U.S. Food and Drug Administration (FDA) for active safety surveillance of marketed medical products. As of August 2015, the SDD had 351 million person-years of longitudinal patient-level data for 193 million health plan members from 2000–2015.¹

AMI is a common endpoint in drug safety studies based on administrative healthcare records. Prior U.S. and non-U.S. validation studies of AMI diagnoses recorded in databases of insurance claims, hospital discharge abstracts, and electronic health records (EHRs) have generally found that principal diagnoses of AMI from inpatient encounters have a high positive predictive value (PPV), typically in the range of 75–95%,^2–15 and a sensitivity of 60–90%.^3,5,15,16 The PPV associated with secondary inpatient diagnoses of AMI, which has been evaluated in fewer studies, has been found to be modestly lower.^7,15,16Since not all AMI cases are identified with algorithms based on the principal diagnosis field only, a common study design dilemma is whether to additionally use other types of AMI diagnoses (e.g., secondary inpatient diagnoses, emergency department diagnoses) to identify AMIs. While the high PPV of principal inpatient diagnoses for AMI was confirmed by a prior validation study conducted within the SDD,¹¹ to date the validity of non-principal inpatient AMI diagnoses has not been assessed in the SDD.

To provide researchers and other stakeholders with data on the validity of non-principal AMI diagnoses within the SDD, we report on the results of a chart validation study of potential AMI cases identified as part of a safety assessment of intravenous immune globulin (IGIV) products.

METHODS

Data sources and study population

The administrative healthcare records and patient medical charts used to identify and validate potential AMI cases came from 13 SDD Data Partners who participated in the protocol-based Sentinel assessment of Thromboembolic Events Following Immunoglobulin Administration.¹⁷ Potential cases from the years 2006–2012 were selected for chart review if an inpatient AMI diagnosis code was recorded in the SDD up to one month following a non-specific (i.e., polyvalent) IGIV treatment episode. A complete description of the criteria used to select potential cases can be found in the Appendix. Additional details concerning the design and objectives of the parent study have been described previously.¹⁷

IGIV is used in the treatment of primary and secondary immunoglobulin deficiencies, and a variety of inflammatory and autoimmune disorders (e.g., chronic demyelinating polyneuropathy and immune thrombocytopenic purpura).¹⁸ In Table 1 we provide descriptive information on the patients included in this chart validation study, including their possible indications for IGIV use and major cardiovascular risk factors. These health conditions were defined as previously described in the protocol for the parent study.¹⁹

Table 1.

Baseline characteristics of 103 potential acute myocardial infarction (AMI) cases identified from the Sentinel Distributed Database for whom chart retrieval was completed.

Characteristic	N (%)
Demographics
Age
• 0–19 years	3 (3%)
• 20–39 years	6 (6%)
• 40–59 years	28 (27%)
• 60–79 years	51 (50%)
• 80+ years	15 (15%)
Sex
• Female	49 (48%)
• Male	54 (52%)
Possible indication for IGIV use*
Autoimmune/inflammatory condition	68 (66%)
Immune deficiency	39 (38%)
Infection	16 (16%)
Bone marrow or hematopoietic stem cell transplant	8 (8%)
Other indication	26 (25%)
Major cardiovascular risk factors*
Myocardial infarction	23 (22%)
Angina	40 (39%)
Atrial fibrillation or flutter	15 (15%)
Ischemic stroke	6 (6%)
Peripheral vascular disease	18 (17%)
Hypertension, uncomplicated	58 (56%)
Hypertension, complicated (i.e., with end-organ damage)	20 (19%)
Diabetes mellitus	27 (26%)
Data Partner type
Insurer/claims-based	83 (81%)
Integrated healthcare delivery system	20 (19%)

^*Possible indications and cardiovascular risk factors were assessed using diagnoses and procedures recorded in administrative data during the 183 days prior to the patient’s IGIV treatment date. The indication indicator variables are not mutually exclusive, so indication percentages may sum to greater than 100%.

The data presented in this paper were collected as part of a public health surveillance activity conducted under the auspices of the FDA Sentinel Initiative. For this reason, the collection and analysis of these data did not qualify as human subjects research under the Common Rule and were not subject to institutional review board (IRB) review.^20–22

Case identification and chart retrieval

The endpoint definition used to identify potential AMI cases included any International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis code of 410.x1 or 410.x0 that originated from an inpatient hospital encounter. Within the Sentinel Common Data Model (SCDM), diagnosis codes associated with inpatient encounters are categorized as principal, secondary, or “unable to classify” (i.e., position unspecified). These classifications reflect standard coding practices and the addition of a third category to accommodate heterogeneity across Sentinel Data Partners in how encounters and coding positions are defined. Under Uniform Hospital Discharge Data Set (UHDDS) guidelines used by U.S. hospitals and insurers,²³ inpatient diagnoses are coded as follows:

• Principal diagnosis: the condition established after study to be chiefly responsible for occasioning the admission of the patient to the hospital

• Secondary diagnosis: a condition also present on admission, that developed during the hospital stay, or that influenced the care of the patient or length of stay

In the SDD, there are also position-unspecified diagnoses that cannot be classified as principal or secondary. These diagnosis codes may represent diagnoses originating from non-facility claims associated with an inpatient stay, e.g., a physician services claim submitted separately from the facility claim. Codes of this type generally come from claims-based Data Partners.

Eligible post-IVIG inpatient encounters with an AMI diagnosis code listed in any position (principal, secondary or unspecified) were selected for review. For each potential AMI case meeting study eligibility criteria, Sentinel Data Partners were asked to retrieve a medical chart corresponding to the encounter during which the AMI diagnosis was recorded. In this validation report, we restricted the denominator for our PPV calculations to the sub-sample of potential cases for which we received a chart that was sufficiently complete to determine whether an AMI occurred (Figure 1).

Figure 1.

Disposition of potential acute myocardial infarction (AMI) cases identified in the SDD.

Chart abstraction

A trained nurse abstractor (L.P., K.P., A.N., or E.R) reviewed the medical chart(s) associated with the index AMI hospital encounter. The abstractors recorded information concerning symptom onset, relevant clinician notes, results from diagnostic testing including electrocardiograms, echocardiography, cardiac biomarkers, and cardiac catheterization reports, and other factors relevant for the IGIV safety assessment.

Case adjudication

Completed abstraction forms (and the original medical charts if needed) were reviewed by single a physician adjudicator with relevant clinical expertise (J.G.R., J.O.E., R.K. or S.G.). Based on the documentation available in the charts, potential cases were adjudicated as a definite, probable, or possible AMI, no AMI, or status unknown / insufficient information. Adjudication criteria, detailed in the Appendix, were adapted from the third universal definition of myocardial infarction developed under the auspices of the European Society of Cardiology, the American College of Cardiology Foundation, the American Heart Association, and the World Health Federation.²⁴ In addition to the third universal definitions of definite and probable AMIs, we added the category of possible AMI to account for cases where parts of a patient’s medical chart were unavailable or illegible. If a case could not be counted as a definite or probable AMI or ruled out based on the information available in the chart, a case was classified as a possible AMI if there was a documented physician diagnosis of AMI; otherwise the patient was classified as AMI status unknown / insufficient information. The adjudication form, which was adapted from the Women’s Health Initiative AMI adjudication form,¹² includes a full description of the adjudication criteria (see Appendix).

For a small number of potential cases, the chart(s) received contained no recorded diagnosis of an acute AMI, no indication that an acute AMI was considered as part of a differential diagnosis, no diagnostic testing, and no symptoms suggestive of a possible AMI. These cases were flagged by the abstractors and not reviewed by the physician adjudicators due to resource constraints. For these cases, if the chart(s) received included the discharge summary for the index AMI hospital encounter, the potential case was considered to have been miscoded and classified as no AMI. Otherwise the case was classified as having an unknown status due to chart incompleteness.

Positive predictive value (PPV) calculation

We calculated the PPV of the AMI diagnosis codes identified in the administrative data by dividing the number of confirmed AMI cases (definite, probable or possible) by the total number of cases for whom a sufficiently complete chart was obtained for the index AMI hospitalization. Potential cases adjudicated as unknown AMI status due to insufficient information were removed from the denominator for the PPV calculation (Figure 1). Exact binomial 95% confidence intervals (Clopper-Pearson) were calculated for the PPV estimates to quantify their precision.

RESULTS

One hundred forty potential post-IGIV AMI cases were identified in the SDD; required charts could be obtained for 103 (74%; see Figure 1). Common reasons that charts were unavailable included an inability to map the encounter record in the SDD to patient and provider identifiers required for chart requests, an inability to locate the medical chart corresponding to the requested encounter, and refusal by the healthcare provider. (See Appendix Table A1 for a complete list of reasons that charts were unobtainable.) Of the 103 cases for which charts were available, 83 were from claims-based Data Partners, and 20 from integrated care delivery systems. The median age of the patients was 65 years; 48% were female. Based on administrative diagnoses recorded during the six months prior, these patients had a high burden of pre-existing atherosclerotic cardiovascular disease (22% with prior AMI diagnosis; 39% with angina) and other major cardiovascular risk factors (75% with hypertension; 26% with diabetes). Additional descriptive information on these patients is provided in Table 1.

Outcome status could be determined for 89 potential AMI cases, of which 67 were confirmed by physician adjudicators (43 definite, 22 probable and 2 possible AMIs; see Figure 1). The PPVs for the inpatient AMI diagnoses recorded in the administrative data were 75% overall (67/89, 95% CI: 65–84%), 93% (28/30, 95% CI: 78–99%) for principal-position diagnoses, 88% (29/33, 95% CI: 72–97%) for secondary diagnoses, and 38% (10/26, 95% CI: 20–59%) for position-unspecified diagnoses. PPVs were higher for 410.x1 diagnosis codes (which denote an “initial episode of care” for AMI) than for 410.x0 codes (“episode of care unspecified”). Detailed PPV estimates stratified by coding position, ICD-9-CM diagnosis code, Data Partner type, and prior AMI diagnosis are provided in Table 2.

Table 2.

Positive predictive values (PPVs)^* associated with inpatient administrative diagnosis codes for acute myocardial infarction (AMI) by position.

	PPVs for all potential AMI cases (N = 89)	PPVs for principal position AMI diagnoses (N = 30)	PPVs for secondary AMI diagnoses (N = 33)	PPVs for position-unspecified** AMI diagnoses (N = 26)
All AMI codes	75% (67/89, 95% CI: 65–84%)	93% (28/30, 95% CI: 78–99%)	88% (29/33, 95% CI: 72–97%)	38% (10/26, 95% CI: 20–59%)
By diagnosis code recorded in administrative data
410.x0	33% (5/15, 95% CI: 12–62%)	100% (1/1, 95% CI: 3–100%)	33% (1/3, 95% CI: 1–91%)	27% (3/11, 95% CI: 6–61%)
410.x1	84% (62/74, 95% CI: 73–91%)	93% (27/29, 95% CI: 77–99%)	93% (28/30, 95% CI: 78–99%)	47% (7/15, 95% CI: 21–73%)
Data Partner type
Insurer/claims-based	71% (49/69, 95% CI: 59–81%)	92% (22/24, 95% CI: 73–99%)	89% (17/19, 95% CI: 67–99%)	38% (10/26, 95% CI: 20–59%)
Integrated care delivery systems	90% (18/20, 95% CI: 68–99%)	100% (6/6, 95% CI: 54–100%)	86% (12/14, 95% CI: 57–98%)	--
By whether an AMI code was observed in prior 183 days
No prior AMI	78% (54/69, 95% CI: 67–87%)	92% (23/25, 95% CI: 74–99%)	96% (22/23, 95% CI: 78–100%)	43% (9/21, 95% CI: 22–66%)
Prior AMI	65% (13/20, 95% CI: 41–85%)	100% (5/5, 95% CI: 48–100%)	70% (7/10, 95% CI: 35–93%)	20% (1/5, 95% CI: 1–72%)

^*Statistics reported in this table reflect the PPV of administrative ICD-9-CM AMI diagnosis codes for a confirmed (definite, probable, or possible) acute AMI. Patients with a classification of insufficient information / acute TEE status unknown were removed from the denominator for the PPV calculations and not included in this table.

^**Within the Sentinel Distributed Database, a position-unspecified inpatient diagnosis is a non-facility professional/provider claim associated with an inpatient encounter.

While data on suspected AMI etiology were not collected systematically during the adjudication process, the adjudicators noted that a substantial number of the confirmed AMIs (46%) were likely to be type 2 AMIs (i.e., attributable to demand ischemia).²⁴ The majority of these type 2 AMIs occurred in the setting of anemia, respiratory insufficiency, and/or septic shock. Such conditions are more common in certain IVIG user subgroups than in the general population. Demand ischemia was the suspected etiology in 21%, 72%, and 40% of confirmed AMIs with principal, secondary, and unspecified coding positions, respectively.

DISCUSSION

In this chart validation study, which relied on data from a protocol-based assessment of the risk of thromboembolic events following IGIV treatment,¹⁹ we evaluated the validity of inpatient administrative diagnosis codes for AMI within the SDD. PPVs were high for principal (93%) and secondary diagnoses (88%), and lower for position-unspecified diagnoses (38%). The PPV estimates from our study are broadly consistent with those reported in prior chart validation studies conducted in the U.S. and Canada over the last two decades.^2–11

Previous validation studies have reported that principal hospital discharge diagnoses of AMI are associated with PPVs of 80–95%.^2–10 Between-study variation in these estimates may be attributable to differences in coding algorithms used to identify potential cases, chart validation criteria, patient populations, and coding practices. In an earlier SDD validation study, chart validation was conducted for a random sample of 153 patients with principal inpatient diagnoses of AMI (ICD-9-CM 410.x0 or 410.x1) drawn from four Sentinel Data Partners.¹¹ They reported a PPV of 86% (95% CI: 79–91%), slightly lower than our estimate of 93% (95% CI: 78–99%) for principal-position diagnoses. In their study, cases with insufficient chart information were counted as false positives rather than as missing data. If these cases were removed from the PPV denominator, as was done in our calculations, their PPV estimate would be 95%, closer to ours.

The main contribution of our study to the existing literature was to provide PPV estimates for secondary and position-unspecified inpatient AMI diagnosis codes recorded in the SDD. Relative to principal diagnoses, PPVs were only slightly lower for secondary diagnoses (88%) in our sample. However, it was noteworthy that confirmed AMIs coded as secondary diagnoses were more likely to be type 2 AMIs (72%) than were AMIs coded as principal diagnoses (21%). This difference can likely be attributed to the greater severity and complexity of illness among hospitalized patients, as secondary diagnoses may represent conditions that develop during a hospital stay.²³ As noted before, type 2 AMIs may be more common in our sample of IGIV users due to the higher prevalence of hematopoietic dysfunction, infection and sepsis among some IGIV patient subgroups.

PPVs were significantly lower for position-unspecified diagnoses (38%). In our study, restricting to principal and secondary AMI diagnoses would have improved the overall PPV of the endpoint definition from 75% to 90%, at the cost of missing 10 of 67 confirmed AMIs (15%). In future studies of AMI where chart confirmation of outcome is not possible, investigators may consider excluding position-unspecified diagnosis codes from their endpoint definitions after weighing the tradeoff between sensitivity and a higher PPV.

A limitation of our study was that medical charts were unobtainable for 26% of potential post-IGIV cases identified in the SDD. However, the typical reasons that charts were unavailable (e.g., unable to link SDD records to patient or provider identifiers) did not give us reason to suspect that our analyzable sample was systematically different than the total set of potential cases identified. Another limitation—referred to above in the discussion of the high prevalence of type 2 AMIs—is that our sample was limited to patients receiving IGIV, and thus may not be representative of the total population of health plan beneficiaries included in the SDD. An additional limitation of our study was that each case was reviewed by a single physician due to cost constraints; a number of prior validation studies have had each case reviewed independently by two physician-adjudicators. This may have reduced the accuracy associated with the clinical diagnoses that served as the gold standard in our study.

Combined with findings from earlier research, our results indicate that inpatient diagnosis codes can be used for the identification of AMI within the SDD. While PPVs for both principal and secondary diagnosis codes are high, investigators should be aware that the latter may often represent AMIs arising in hospitalized, complex patients due to causes other than an intracoronary thrombus.

Key points:

1. The PPV for principal AMI diagnoses was 93% (95% CI: 78–99%)

2. The PPV for secondary AMI diagnoses was 88% (95% CI: 72–97%)

3. The PPV for position-unknown diagnoses was 38% (95% CI: 20–59%)

4. Secondary and position-unknown diagnoses were more often due to demand ischemia

5. AMI PPVs were similar to estimates from other studies