Identification of Adults with Congenital Heart Disease of Moderate or Great Complexity from Administrative Data

Abstract

INTRODUCTION

There is relatively sparse literature on the use of administrative datasets for research in patients with adult congenital heart disease (ACHD). The goal of this analysis is to examine the accuracy of administrative data for identifying patients with ACHD who died.

METHODS

We created a list of the International Classification of Diseases codes representing ACHD of moderate- or great-complexity. We performed a search for these codes in the electronic health record of adults who received care in 2010–2016, and used state death records to identify patients who died during this period. Manual record review was completed to evaluate performance of this search strategy. Identified patients were also compared to a list of patients with moderate- or great-complexity ACHD known to have died.

RESULTS

We identified 134 patients, of which 72 had moderate- or great-complexity ACHD confirmed by manual review, yielding a positive predictive value of 0.54 (95% CI 0.45, 0.62). Twenty six patients had a mild ACHD diagnosis. Thirty six patients had no identified ACHD on record review. Misidentifications were attributed to coding error for 19 patients (53%), and to acquired ventricular septal defects for 11 patients (31%). Diagnostic codes incorrect more than 50% of the time were those for congenitally corrected transposition, endocardial cushion defect, and hypoplastic left heart syndrome. Only 1 of 21 patients known to have died was not identified by the search, yielding a sensitivity of 0.95 (0.76, 0.99).

CONCLUSION

Use of administrative data to identify patients with ACHD of moderate or great complexity who have died had good sensitivity but suboptimal positive predictive value. Strategies to improve accuracy are needed. Administrative data is not ideal for identification of patients in this group, and manual record review is necessary to confirm these diagnoses.

INTRODUCTION

With increasingly successful surgical techniques and medical management, children with congenital heart disease are surviving longer, and there are now more adults than children with congenital heart disease (1). Despite its rapid growth, the subspecialty of adult congenital heart disease (ACHD) lacks the broad research base common in other areas of cardiology (2–4). Challenges to research include the heterogeneous mix of congenital lesions seen in ACHD, the young nature of the field, and a relatively small population, as compared to those with other types of cardiac disease. Although many patients with ACHD survive into adulthood, long-term survival with moderate or severe ACHD remains limited, and there are few data on risk factors for and circumstances surrounding death in this population. Research on death in ACHD will allow for improvement in care practices to maximize both quantity and quality of life and death. A systematic and reliable way to identify patients with moderate or severe ACHD from administrative data or from electronic health records (EHR) would greatly facilitate this research.

Because of availability and ease of access, administrative datasets are heavily utilized in the emerging body of research on ACHD (3). Administrative data are collected through contact with a medical provider, for example via hospital admissions or clinic visits. Administrative data are designed to be utilized for billing purposes. Their standard language, commonly the International Classification of Disease (ICD) system (5, 6), enables some degree of uniformity and structure. However, subjectivity and variation in coding practices are commonly recognized problems with this type of data (7–10). Any provider is able and often required to enter billing codes without rigorous training in coding. On the other hand, expert coders often lack the clinical knowledge necessary to distinguish between codes with marginal but important differences in meaning. The result is a body of data that is imperfect, and while useful for its original purpose, can lead to significant errors when used for research.

The accuracy of administrative data for use in ACHD research was addressed by Broberg and colleagues, who tested a hierarchical algorithm to aid in identification of patients with ACHD of any severity in a single institution’s EHR. They reported a sensitivity of 99% and specificity of 88% of that algorithm for identifying patients by ICD code (3).

The goal of this study is to examine the accuracy of administrative data specifically for identifying patients with ACHD of moderate or great complexity who have died, in order to understand care at the end of life for these patients. Particular effort was made to identify patients with Eisenmenger Syndrome (ES), as this is considered one of the most advanced forms of ACHD and carries high morbidity and mortality. We evaluated the sensitivity and positive predictive value of ICD-9 and ICD-10 codes assigned in the inpatient and outpatient settings compared with medical record review.

METHODS

We created a list of targeted ACHD diagnoses representing ACHD of moderate or severe complexity. Simple lesions were excluded because they are less likely to lead to death related to cardiac disease. Choice and classification of these diagnoses regarding complexity was guided by the ACC/AHA guidelines (2). We followed these guidelines in referring to the diagnosis classification as “moderate complexity” and “great complexity.” We then matched these diagnoses to representative ICD-9 and ICD-10 codes (Table 1).

Table 1.

List of ICD-9 and ICD-10 codes and their representative lesions evaluated in this study.

Complex Lesions Evaluated, by ICD Code
ICD-9	Code Description	ICD-10	Code Description
745.00	Common truncus, Persistent truncus arteriosus	Q200	Common arterial trunk
745.10	Complete/classical transposition of great vessels	Q203	Discordant ventriculoarterial connection
745.11	Double outlet right ventricle	Q201	Double outlet right ventricle
745.12	Corrected transposition of great vessels	Q205	Discordant atrioventricular connection
745.30	Common/single ventricle	Q204	Double inlet left ventricle
		Q202	Double outlet left ventricle
745.70	Cor biloculare	NA	NA
746.01	Congenital absence/atresia of pulmonary valve	Q220	Pulmonary valve atresia
746.10	Tricuspid atresia and stenosis, congenital	Q229	Congenital malformation of tricuspid valve, unspecified
746.70	Hypoplastic left heart syndrome	Q234	Hypoplastic left heart syndrome
		Q226	Hypoplastic right heart syndrome
747.30	Anomalies of pulmonary artery (includes pulmonary atresia)	NA	NA
Moderate Lesions Evaluated, by ICD Code
ICD-9	Code Description	ICD-10	Code Description
745.20	Tetralogy of Fallot	Q213	Tetralogy of Fallot
745.60	Endocardial cushion defects	Q212	Atrioventricular septal defect
745.69	Common atrioventricular canal defect; atrioventricular canal type ventricular septal defect, common atrium	Q212	Atrioventricular septal defect
746.20	Ebstein’s Anomaly	Q225	Ebstein’s anomaly
747.1	Coarctation of aorta	Q251	Coarctation of the aorta
747.41	Total anomalous pulmonary venous return	Q262	Total anomalous pulmonary venous connection
747.42	Partial anomalous pulmonary venous return	Q263	Partial anomalous pulmonary venous connection
Other Diagnoses Evaluated, by ICD Code(s)
ICD-9	Code Description	ICD-10	Code Description
745.4	Ventricular septal defect	Q210	Ventricular septal defect
745.4 + other code 745-747	“Complex VSD” - a potential proxy for Eisenmenger Syndrome	Q210 + other code Q200-264	“Complex VSD” - a potential proxy for Eisenmenger Syndrome
782.5 + other code 745-747	“Cyanosis+Other” - a potential proxy for Eisenmenger Syndrome	R230 + other code Q200-264	“Cyanosis+Other”- a potential proxy for Eisenmenger Syndrome

ACHD diagnoses were classified hierarchically by the most severe lesion, following the method of Broberg et al (3). We aimed to identify ES, which does not have a specific ICD code, by searching for patients with cyanosis plus another congenital heart disease code. We defined “complex VSD” (ventricular septal defect) using a combination of the code for VSD plus another congenital heart disease code. “Complex VSD” and “isolated VSD” were explored as additional proxies for ES.

We performed an electronic search for the list of codes described above in any position in a problem list, clinical encounter, or non-clinical encounter (such as order placement) in the EHR (ORCA Powerchart, Cerner Corporation; Epic, Epic Systems Corporation) of adults (defined as age 18 or older) who received care during 2010–2016 at four locations within a single health system. UW Medicine is a large medical network in the Pacific Northwest with four hospitals, a comprehensive cancer center, and a large clinic network that serves patients in 5 states and has more than 64,000 inpatient admissions and 1.3 million outpatient and emergency department visits per year. The ACHD program, based at the University of Washington Medical Center, includes 3 full-time ACHD cardiologists and more than 10 affiliated providers, and accommodates an estimated 3,000 outpatient visits per year.

We used Washington state death records to identify individuals who died between January 2010 and June 2016, with death not attributed to an “external event” (defined as injury or poisoning emanating from an accident, suicide, homicide, or an undetermined source). These individuals were then matched by social security number to those identified by the ICD-9 and ICD-10 ACHD code list. The resulting group of patients with ACHD of moderate or great complexity who died was restricted to patients who received care in the UW Medicine system in the two years prior to death, which we defined based on Dartmouth Atlas criteria to be one non-surgical inpatient hospitalization or two outpatient visits within the last 36 months of life, with one visit within the last two years (11).

One of the investigators (JMS) manually reviewed the EHR for all identified patients. ACHD was classified as present or absent based on information recorded in the EHR, and its complexity was determined. A standardized data abstraction form was used for data collection. A separate investigator (AH) re-reviewed a randomly selected sample comprising 20% of the total to assess inter-rater reliability.

We considered identification of ACHD diagnosis using administrative data as the test method, and manual review of the EHR as the reference method. We calculated the positive predictive value (proportion of positives reported by the test method that were verified by the reference method) and exact confidence interval (RStudio, Version 0.99.903. RStudio Statistical Software (2016): Integrated Development for R. RStudio, Inc., Boston, MA URL http://www.rstudio.com/).

The University of Washington Medical Center ACHD clinic maintains a list of patients seen at the clinic that includes information on date of death. We determined whether the listed patients were also identified by the administrative data search to assess sensitivity (proportion of positives reported by a reference method that were also reported as positives by the test method) and calculated the exact confidence interval. The university institutional review board assessed this project as not involving human subjects because all patients were deceased; a waiver of HIPAA consent was approved as required by Washington State law.

RESULTS

Using the criteria described above, 134 patients with ACHD of moderate or great complexity who were seen at UW Medicine and died between January 2010 and June 2016 (Table 2) were identified using administrative and death record data. Overall, the mean age was 53±19 years, and 59% were male. Medical records adequate to evaluate the cardiac diagnoses were available for all 134 patients.

Table 2.

Demographic information for 134 patients identified by search of EHR and death records

	Targeted ACHD	Other ACHD	No ACHD
	n = 72 (%)	n = 26 (%)	n = 36 (%)
Sex
Male	45 (63)	11 (42)	23 (64)
Female	27 (38)	15 (58)	13 (36)
Age at Death (mean±SD)	45±17	57±19	67±13
Primary ACHD Diagnosis
Eisenmenger Syndrome	12 (17)	0	0
Atrioventricular Septal Defect	4 (6)	0	0
Double Inlet Left Ventricle	4 (6)	0	0
Double Outlet Right Ventricle	1 (1)	0	0
Ebstein Anomaly	2 (3)	0	0
Hypoplastic Left Heart Syndrome	1 (1)	0	0
Transposition (L)	1 (1)	0	0
Transposition (D)	9 (13)	0	0
Tricuspid Atresia	6 (8)	0	0
Tetralogy of Fallot	17 (24)	0	0
Truncus	4 (6)	0	0
Aortic Coarctation	11 (15)	0	0
Ventricular Septal Defect	0	10 (38)	0
Patent Foramen Ovale	0	8 (31)	0
Other	0	6 (23)	0
No ACHD	0	0	36 (100)
Acquired VSD	0	0	11 (31)

Positive Predictive Value of ICD-based Criteria

Figure 1 is a flow diagram depicting the process of patient identification and evaluation. On manual EHR review, 72 (54%) of the patients coded as having moderate or greater ACHD by our list of ICD codes actually had lesions which met these criteria. This represents a positive predictive value of 0.54 for the list of codes (95% CI 0.45, 0.62).

Figure 1.

Summary of patient identification process

ES was clinically documented in 12 patients, 17% of the 72 patients with targeted ACHD. Based on prior literature, we used “cyanosis+other” as a way to identify ES. However, we found that “cyanosis+other” did not uniquely identify any patients with ES. In fact, this combination only identified 1 patient with ES, who would have been picked up by codes for other complex lesions, including “VSD+other.” In addition, this combination identified 3 patients with ACHD of moderate or great complexity who did not have ES and 3 patients with only simple ACHD whose cyanosis was probably noncardiac in origin. In contrast, all 12 patients with ES could have been identified by VSD codes. “VSD+other” identified 6 of 12 patients with ES, 2 of whom would not have been identified by the ICD code list in the absence of the “VSD+other” combination. In addition, there were 6 patients with clinically-documented ES for whom “VSD” was the only ACHD code, and who would have been missed if VSD had not been considered in isolation.

Twenty-six patients identified in our search in reality had simple ACHD lesions, but were miscoded as having more severe disease. For 8 of the 26 patients, the only lesion identified on EHR review was a patent foramen ovale (PFO) noted in an echocardiogram report. Five of these 8 were coded as having atrioventricular septal or endocardial cushion defects, all entered by the same clinician (a general cardiologist). Noncomplex VSDs were present in 14 of these 26 patients, picked up in an attempt to identify ES. The remaining 4 of the 26 patients with simple lesions had valve disease that was incorrectly coded as Ebstein’s anomaly (1 patient) or coarctation of the aorta (2 patients), or was identified by “cyanosis+other” (1 patient).

Patients with ICD-9 or ICD-10 codes for ACHD who had no ACHD by manual EHR review

Despite being coded as having ACHD, there were 36 patients for whom no ACHD was identified on manual EHR review. Acquired VSDs were present for 11 of the 36 patients (31%) in the setting of either myocardial infarction or infective endocarditis, but coded as congenital VSD. Six of 36 (17%) were coded as having atrioventricular septal or endocardial cushion defects, 5 of 36 (14%) were coded as having coarctation of the aorta, 5 of 36 (14%) were coded as having a single ventricle condition, and the remaining 9 were coded as having other conditions, none of which were present.

Each code was examined individually to identify what percent of the time the code correctly identified the specified lesion (Table 3). A single patient may have been assigned more than one code. Codes that had the highest degree of error, incorrect more than 50% of the time, were congenitally corrected transposition (745.12), endocardial cushion defect (745.60; this is a separate code from atrioventricular canal defects), and hypoplastic left heart syndrome (746.70).

Table 3.

Performance of individual ICD-9 and ICD-10 codes in this study

Diagnosis	ICD-9 (top) or ICD-10 (bottom) Code	No. patients identified by ICD code	No. (%) patients for whom code accurately identified lesion	PPV (95%CI) for code performance
Common truncus, Persistent truncus arteriosus	745.00	4	4 (100)	1 (0.4,1)
Complete/classical transposition of great vessels	745.10	26	22 (85)	0.85 (0.65,0.96)
Double outlet right ventricle	745.11	3	2 (67)	0.67 (0.09,0.99)
Corrected transposition of great vessels	745.12	16	2 (13)	0.13 (0.02,0.38)
Tetralogy of Fallot	745.20	23	22 (96)	0.96 (0.78,0.99)
Common/single ventricle	745.30	12	11 (92)	0.92 (0.62,0.99)
Endocardial cushion defects	745.60	13	4 (31)	0.31 (0.09,0.61)
Common atrioventricular canal defect; atrioventricular canal type ventricular septal defect, common atrium	745.69	7	5 (71)	0.71 (0.29,0.96)
Cor biloculare	745.70	0	0	NA
Congenital absence/atresia of pulmonary valve	746.01	7	7 (100)	1 (0.59,1)
Tricuspid atresia and stenosis, congenital	746.10	10	7 (70)	0.70 (0.35,0.93)
Ebstein’s Anomaly	746.20	3	2 (67)	0.67 (0.09,0.99)
Hypoplastic left heart syndrome	746.70	1	0	0 (0,0.98)
Coarctation of aorta	747.10	21	14 (67)	0.67 (0.43,0.85)
Anomalies of pulmonary artery (includes pulmonary atresia)	747.30	11	7 (64)	0.64 (0.31,0.89)
Total anomalous pulmonary venous return	747.41	0	0	NA
Partial anomalous pulmonary venous return	747.42	0	0	NA
Ventricular Septal Defect	745.4	25	14 (56)	0.56 (0.35,0.76)
Ventricular Septal Defect + other code 745-747	745.4 + other	65	60 (92)	0.92 (0.83,0.97)
Cyanosis + other code 745-747	782.5 + other	8	4 (50)	0.5 (0.16,0.84)
Common arterial trunk	Q200	0	0	NA
Double outlet right ventricle	Q201	0	0	NA
Double outlet left ventricle	Q202	0	0	NA
Discordant ventriculoarterial connection	Q203	1	1 (100)	1 (0.03,1)
Double inlet left ventricle	Q204	1	1 (100)	1 (0.03,1)
Discordant atrioventricular connection	Q205	0	0	NA
Atrioventricular septal defect	Q212	0	0	NA
Tetralogy of Fallot	Q213	2	2 (100)	1 (0.16,1)
Pulmonary valve atresia	Q220	0	0	NA
Ebstein’s anomaly	Q225	0	0	NA
Hypoplastic right heart syndrome	Q226	0	0	NA
Congenital malformation of tricuspid valve, unspecified	Q229	0	0	NA
Hypoplastic left heart syndrome	Q234	0	0	NA
Coarctation of the aorta	Q251	0	0	NA
Total anomalous pulmonary venous connection	Q262	0	0	NA
Partial anomalous pulmonary venous connection	Q263	0	0	NA
Ventricular Septal Defect	Q210	2	1 (50)	0.5 (0.01,0.99)
Ventricular Septal Defect + other code Q200-264	Q210 + other	3	3 (100)	1 (0.29,1)
Cyanosis + other code Q200-264	R230 + other	2	2 (100)	1 (0.16,1)

Each code is evaluated separately; patients may have been assigned more than one code.

Sensitivity of ICD-based diagnostic criteria

There were 23 patients with confirmed ACHD who were previously seen at the UWMC ACHD clinic and who were known to have died in Washington State between January 2011 and June 2016. Of these, 2 patients had an ACHD lesion that was not targeted by the ICD codes. Only 1 of the remaining 21 patients was not identified by the search of ICD-9 or ICD-10 codes and death record data, giving a sensitivity of 0.95 (95% CI 0.76, 0.99). We suspect this patient died outside of Washington state, and so death information was not captured in Washington state death records.

Inter-rater reliability

Concordance was evaluated following manual review of a 20% sample of medical records by a separate investigator. Agreement for whether subjects had “moderate or greater ACHD” was 100% (kappa of 1.0). Agreement for exact diagnosis was 88% (kappa of 0.76), indicating substantial agreement. Both reviewers found the EHR information adequate to evaluate the presence or absence of ACHD.

DISCUSSION

This study evaluated the utility of administrative data for identifying patients with ACHD of moderate or great complexity who have died. When compared to manual EHR review, ICD-based criteria had a sensitivity of 0.95, but a PPV of only 0.54, with substantial misclassification error. The relatively low PPV of 0.54 suggests that use of administrative data in isolation is suboptimal for this population, even when care is taken to search for an inclusive list of terms.

The finding of significant human error in coding that led to misclassification of patients is not surprising given the lack of formal training regarding coding practices (3, 8). Some ICD codes lack granularity, and it is often difficult to correctly assign existing codes to complex disease. It is not surprising that at least two of the codes that performed particularly poorly (congenitally corrected transposition- 745.12, endocardial cushion defect- 745.60) are codes that may have multiple interpretations or can be easily misunderstood by coders without specific training in ACHD. Results for 745.60 are particularly interesting because it is the ICD-9 code for both PFO and ASD. Although a PFO can be interpreted as a type of atrioventricular septal defect, ostium primum and secundum atrial septal defects and endocardial cushion defects have very different clinical implications from PFO. Our findings suggest the need for further education in coding practices or improved oversight by persons with coding expertise, as well as continued updates to the ICD system, to improve coding for ACHD.

In a previous study of coding for ACHD, Broberg and colleagues provided a hierarchical algorithm to aid in identification of patients with ACHD of any severity in an EHR (3). In our study, we targeted ACHD of higher severity to focus on patients at a higher risk of death due to cardiac disease, and limited our dataset to patients who died.

The absence of a specific code for ES makes identification of these patients with arguably the most severe type of ACHD difficult via administrative database search. Medical documentation for these patients can be inconsistent because many underlying lesions can lead to ES, and the clinical presentation is heterogeneous. Though cyanosis is a non-specific hallmark of ES, it is inconsistently coded. Previous studies have suggested the use of “cyanosis plus other congenital code” as a proxy for ES (3). However in our study, that approach would have only identified one of the 12 patients confirmed to have ES by manual EHR review, who would have been identified by other complex ACHD codes.

VSD is a common lesion in patients with ES. Based on the approach of combining “cyanosis plus other congenital code,” we hypothesized that “VSD plus other congenital code” could be an additional proxy for ES. This allowed for identification of 50% of the patients found to have clinically-documented ES, but only 2 patients (17%) who would not have been otherwise identified by other complex ACHD codes. A search for VSD in isolation identified 6 additional patients with ES- the other half of the group. However, the tradeoff to using this nonspecific search term was that 8 patients had a milder form of ACHD than targeted, and 13 had no ACHD at all. Furthermore, it is unlikely that the inclusion of the VSD code in isolation would allow for identification of all patients with ES, since not all patients with ES have a VSD. We considered also including atrial septal defect (ASD), either with another code or in isolation, but due to the even less specific nature of this code, we felt this would have even more frequently identified patients with non-targeted disease.

Based on the common errors identified in this study with relatively clear etiologies, several solutions can be considered to improve the identification of patients with at least moderately complex ACHD. First, 11 patients with acquired VSDs, due either to myocardial infarction or endocarditis, were erroneously included in the sample. One way to address this when creating a list of codes for search may be to exclude those patients who have either codes specifying the VSD as acquired (429.71, I51.0, I23.2), or who have myocardial infarction or endocarditis codes in addition to VSD codes. When we re-ran the search to explore this strategy, we found that all 11 of these patients would have been excluded using this method. However, 5 patients who were appropriate for the sample would have been excluded either due to having had a myocardial infarction unrelated to their VSD or having the VSD miscoded as acquired. Therefore, while this approach can improve accuracy, the consequences of missing appropriate patients should be considered. Natural language processing, if available, could be used to guide patient selection in such cases. In addition, the codes for D- and L-transposition of the great vessels were often both coded for the same patient. Since these lesions are mutually exclusive, this likely represents miscoding due to misunderstanding. If differentiating between D- and L-transposition is relevant to the particular study, use of other codes such as procedure codes could be employed since most patients with D-transposition require early corrective surgery, while most with L-transposition do not.

Our results suggest that ICD-9 codes 745.00 (common truncus), 745.20 (Tetralogy of Fallot), and 746.01 (congenital absence/atresia of pulmonary valve), may be reliable enough not to pursue manual confirmation, given their PPV of at least 0.95 with acceptable confidence intervals. However, consideration should be given to manual confirmation of codes such as 745.12 (congenitally corrected transposition), 745.60 (endocardial cushion defect), and 746.70 (hypoplastic left heart syndrome), given the high degree of error seen in their use. However, our study was small and based in a single health care system, and so this question requires further study.

This is the first study to explore the use of ICD administrative codes to identify patients with ACHD of moderate or great complexity who have died. This study has several limitations. First, generalizability may be limited since this study was performed in a single healthcare system, and, as ICD codes were found to “underperform”, this study may reflect lack of training specific to coders in this healthcare system. Second, given the short period of time studied for which ICD-10 codes were utilized, only a small number of ICD-10 codes was included in this search and we are unable to compare the accuracy of ICD-9 with ICD-10 codes. Key revisions to the ICD-10 codes that could improve accuracy include a code for “VSD as a result of myocardial infarction” and clarification of transposition of the great arteries as “discordant ventriculoarterial connection” versus “discordant atrioventricular connection.” Third, we focused on patients who have died, which may create concerns about a decedent bias for some study questions (12). Finally, although patients with moderate or great complexity ACHD have a worse prognosis as a group compared to those with less severe disease, individual patients with lesser degrees of ACHD may still have limited longevity.

In summary, we found that administrative data alone are inadequate for identification of patients with ACHD of moderate or great complexity who have died. To avoid errors, manual EHR review should be undertaken to confirm the ACHD diagnosis. The specific codes chosen to identify patients for research are likely to be very context-specific, and care must be taken to validate any approach using administrative data to identify patients with specific diagnoses. Although inclusion of less specific codes may increase the sensitivity of such codes, this can lead to misclassification and thus lower the positive predictive value. We found that our proposed approach had good sensitivity, but required manual review to ensure accurate classification.