The Impact of Differential Case Ascertainment in Clinical Registry Versus Administrative Data on Assessment of Resource Utilization in Pediatric Heart Surgery

Abstract

Background

Resource utilization in congenital heart surgery is typically assessed using administrative data sets. Recent analyses have called into question the accuracy of coding of cases in administrative data; however, it is unclear whether miscoding impacts assessment of associated resource use.

Methods

We merged data coded within both an administrative data set and clinical registry on children undergoing heart surgery (2004–2010) at 33 hospitals. The impact of differences in coding of operations between data sets on reporting of postoperative length of stay (PLOS) and total hospital costs associated with these operations was assessed.

Results

For each of the eight operations of varying complexity evaluated (total n ═ 57,797), there were differences in coding between data sets, which translated into differences in the reporting of associated resource utilization for the cases coded in either data set. There were statistically significant differences in PLOS and cost for seven of the eight operations, although most PLOS differences were relatively small with the exception of the Norwood operation and truncus repair (differences of two days, P < .001). For cost, there was a >5% difference for three of the eight operations and >10% difference for truncus repair (US$10,570; P < .01). Grouping of operations into categories of similar risk appeared to mitigate many of these differences.

Conclusion

Differences in coding of cases in administrative versus clinical registry data can translate into differences in assessment of associated PLOS and cost for certain operations. This may be minimized through evaluating larger groups of operations when using administrative data or using clinical registry data to accurately identify operations of interest.

Introduction

In the current health care environment, there is an increasing emphasis on reducing the cost of care in addition to improving clinical outcomes, or optimizing “value.” Federal legislation such as the Affordable Care Act and other initiatives aim to provide hospitals and physicians with incentives to encourage improvements in both of these domains.^1,2 Although initial efforts have focused on the adult population, these reforms will also impact pediatric care over the next several years.^3,4 However, methods to assess resource utilization in the pediatric population, particularly for common and resource intense conditions such as congenital heart disease, remain underdeveloped.

Typically, markers of resource utilization for children undergoing heart surgery, such as hospital length of stay and estimated hospital costs, are assessed using a variety of widely available administrative data sets.^5–9 Recently, it has been shown that there can be significant limitations with regard to accurate coding of cases in administrative data, which can lead to significant differences in clinical outcomes assessment between administrative and clinical registry data.^10,11 However, the degree to which these differences in case ascertainment impact assessment of associated resource utilization is unclear.

The purpose of the present study was to utilize a unique data set consisting of merged patient-level information from the Society of Thoracic Surgeons Congenital Heart Surgery (STS-CHS) database (a clinical registry) and the Pediatric Health Information Systems (PHIS) database (an administrative database) to evaluate the impact of differences in coding and classification of cases between data sets on assessment of associated resource utilization, including postoperative length of stay (PLOS) and estimated total hospital costs.

Patients and Methods

Data Source

The PHIS database is an administrative data set that collects information from the hospital bill and other resource utilization data across several US children’s hospitals.¹² Diagnoses and procedures are coded by billing personnel using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9) codes.

The STS-CHS database is the largest existing pediatric heart surgery registry and collects preoperative, operative, and outcomes data on all children undergoing heart surgery at participating centers. Diagnoses and procedures are coded by clinicians and affiliated data managers using the International Pediatric and Congenital Cardiac Code (IPCCC).^12,13

Study Population

As previously described, data on 62,052 (90%) eligible patients with 0 to 18 years of age undergoing heart surgery (with or without cardiopulmonary bypass) at 33 hospitals participating in both the STS-CHS and PHIS databases (2004–2010) were merged using the method of matching on indirect identifiers.^10,12,14,15 The resulting data set therefore contains information entered into both the PHIS and the STS-CHS databases for each patient. Patients with missing data for key study variables were excluded (n ═ 62 STS-CHS and n ═ 1,567 PHIS). In addition, we excluded those with discrepant outcomes (in-hospital mortality status, n ═ 100; and discharge date, n ═ 500) between databases.¹⁰ These latter exclusions were applied as we were primarily interested in an assessment of the impact of differences in coding of cases on assessment of resource utilization outcomes between the two data sets, rather than evaluating how differences in coding of outcomes themselves may influence resource utilization. In the present analysis, we focused on hospital survivors; thus, 2,023 patients who died in-hospital were also excluded, leaving an overall study cohort of 57,797 patients from 33 centers. For the portions of the analysis involving cost data, 8,446 patients with missing cost data were excluded. This research was not considered human subjects research by the Duke University and University of Michigan institutional review boards in accordance with the Common Rule (45 CFR 46.102(f)).

The operations evaluated in the study consisted of the eight STS benchmark operations of varying levels of complexity: ventricular septal defect (VSD) repair, tetralogy of Fallot (TOF) repair (excluding those associated with pulmonary atresia or absent pulmonary valve, or atrioventricular canal repair), complete atrioventricular canal (CAVC) repair, arterial switch operation ± VSD repair (analyzed together for the purposes of this analysis with adjustment for VSD repair as described in more detail subsequently), Fontan operation (including both lateral tunnel and extracardiac conduit ± fenestration, excluding Fontan revision), truncus arteriosus repair (excluding those requiring concomitant truncal valve repair/replacement or interrupted aortic arch repair), and the Norwood operation (including either a systemic to pulmonary artery shunt or right ventricle to pulmonary artery conduit as the source of pulmonary blood flow).¹⁶

In the STS-CHS (clinical registry) data, the operation of interest was identified using standard methodology involving assessment of the operation coded as the primary procedure for the index (first) operation of the admission.¹⁷ In the PHIS data, the two most commonly utilized methods to identify procedures in administrative data sets were used. In method 1, the individual ICD-9 procedure code for the operation of interest was used. In method 2, the Risk Adjustment in Congenital Heart Surgery, Version 1 (RACHS-1) methodology was utilized.¹⁸ As previously described, this method employs combinations of inclusionary and exclusionary ICD-9 diagnosis and procedure codes with the aim of more precisely identifying the procedure of interest.¹⁸ For method 2, additional inclusion/ exclusion codes on the STS-CHS side were applied for TOF and CAVC repair to attempt to match the RACHS-1 algorithms as closely as possible.

In addition to individual operations, we also evaluated categories of operations in order to assess whether grouping of operations of similar risk into broader categories may potentially mitigate errors associated with miscoding of individual operations.¹⁰ For this portion of the analysis, the operations identified in the administrative and clinical registry data were classified by RACHS-1 category.¹⁸ Although there are other risk stratification systems in the field, RACHS-1 is the only system that has been adapted for use with both types of data sources.

Outcomes

Outcomes included PLOS in days and total hospital costs. Costs were estimated using hospital and department-specific cost/charge ratios collected across all PHIS hospitals, adjusted for regional differences using the Centers for Medicare and Medicaid Services price—wage index, and indexed to 2010 dollars. Of note, professional fees are not included in most administrative data sets, including the PHIS database, and thus were not included in this analysis.

Analysis

As described previously, the operation performed for each patient was assessed via the information coded on that patient within the administrative data set and based on the information coded for that same patient within the clinical registry. Using these data, the cohort of patients coded as undergoing each operation of interest (in either data set) was identified (Figure 1). For example, for TOF repair, patients coded as undergoing TOF repair in the administrative data set and patients coded as undergoing TOF repair in the clinical registry were identified. For these groups of patients identified as undergoing the operation of interest in either data set, associated resource utilization in the form of PLOS and total hospital costs were calculated and described using standard summary statistics (Figure 1). Ten percent trimmed means were used as an additional measure of central tendency. To evaluate the statistical differences in resource utilization between the administrative data versus clinical registry cohorts (eg, differences in resource utilization for the patients coded as undergoing TOF repair in the administrative data set vs those coded as undergoing TOF repair in the clinical registry), negative binomial models were used (with log link function in order to account for the skewed distribution of PLOS and cost). Both PLOS and cost were evaluated as continuous variables, and group indicators were included in the models to take into account overlap between the groups of patients identified in either data set. Wald tests were constructed to compare distributions between groups. Similar analyses were performed for the RACHS-1 categories. All analyses were performed using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina). A P value <.05 was considered statistically significant.

Figure 1.

Assessment of resource utilization based on cases coded in the administrative versus clinical registry data. For the group of cases coded in either data set, cost and postoperative length of stay (PLOS) were evaluated.

Results

Study Population Characteristics

The cohort included 57,797 patients from 33 centers. Study population characteristics and overall outcomes are displayed in Table 1. The 33 included centers were diverse geographically (33.3% South, 33.3% Midwest, 21.2% West, and 12.2% Northeast) with a wide range of annual surgical volume (median 351, range 111–891 cases/year).

Table 1.

Study Population Characteristics.^a

	(N = 57,797)
Age at surgery	6.7 months (39 days-3.6 years)
Weight at surgery, kg	6.5 (3.7–14.4)
Sex, male	31,994(55.4%)
RACHS-1 categorya
1	6,450(1 1.0%)
2	18,399(32.0%)
3	16,024 (28.0%)
4	5,275 (9.1%)
5	21 (0.04%)
6	2,117(3.7%)
Unclassified	9,511 (16.5%)
Benchmark operationsb
VSD repair	4,993 (8.6%)
TOF repair	3,103 (5.4%)
CAVC repair	2,108(3.6%)
ASO	1,425 (2.5%)
ASO + VSD repair	555 (1.0%)
Fontan	3,072 (5.3%)
Truncus repair	413 (0.7%)
Norwood	1,836(3.2%)
PLOS, days	6.0(4.0–13.0)
Total cost, US$	35,721 (23,175–69,632)

Abbreviations: VSD, ventricular septal defect; TOF, tetralogy of Fallot; CAVC, complete atrioventricular canal; ASO, arterial switch operation; PLOS, postoperative length of stay; RACHS-1, Risk Adjustment in Congenital Heart Surgery, Version 1.

^aData presented as median (interquartile range) or frequency (percent).

^bAs assessed in the clinical registry.

Assessment of Resource Utilization

The number of patients undergoing each operation as assessed in the administrative data versus clinical registry is displayed in Table 2. There were differences in the number of cases identified between data sets for all operations, which were more prominent for method 1 (individual ICD-9 procedure codes to identify the operation of interest in the administrative data) compared with method 2 (where a combination of ICD-9 diagnosis and procedure codes was used to attempt to more precisely define the operation of interest in the administrative data). For method 2, there was a >10% difference in the number of cases identified between data sets for four of the eight benchmark operations.

Table 2.

Postoperative Length of Stay (in Days) for Benchmark Operations in Administrative Versus Clinical Registry Data.^a

Operation	PLOS for Operations Identified in Clinical Registry	N	PLOS for Operations Identified in Administrative Data	N	Difference in PLOS (d)	Percent Difference in PLOS (%)	P Value
Method 1—individual ICD-9 procedure codes
VSD repair	5.0 (4.0–7.0)	4,993	6.0(4.0–11.0)	10,046	1	20.0	<.0001
TOF repair	6.0 (5.0–9.0)	3,103	7.0(5.0–10.0)	3,832	1	16.7	<.0001
CAVC repair	7.0(5.0–12.0)	2,108	6.0(4.0–11.0)	3,588	−1	−14.3	.11
ASO	10.0(8.0–15.0)	1,425	1 1.0(8.0–17.0)	2,349	1	10.0	<.0001
ASO + VSD repair	1 1.0(8.0–18.0)	555	12.0(8.0–19.0)	791	1	9.1	<.01
Fontan	9.0(7.0–13.0)	3,072	9.0(7.0–13.0)	3,430	0	0.0	.28
Truncus repair	16.0(10.0–30.0)	413	14.0(8.0–27.0)	556	−2	−12.5	<.01
Norwood	NA	NA	NA	NA	NA	NA	NA
Method 2—additional inclusionary/exclusionary ICD-9 diagnosis and procedure codes
VSD repair	5.0 (4.0–7.0)b	4,993	5.0 (4.0–7.0)b	5,455	0	0.0	.047
TOF repair	6.0 (5.0–9.0)	2,787	6.0(5.0–10.0)	3,299	0	0.0	<.0001
CAVC repair	7.0(5.0–12.0)	2,039	6.0(5.0–10.0)	3,038	−1	−14.3	<.001
ASO	10.0 (8.0–15.0)b	1,425	10.0 (7.0–15.0)b	1,374	0	0.0	.054
ASO + VSD repair	1 1.0(8.0–18.0)	555	12.0(8.0–19.0)	745	1	9.1	.049
Fontan	9.0(7.0–13.0)b	3,072	9.0(7.0–13.0)b	3,015	0	0.0	<.0001
Truncus repair	16.0(10.0–30.0)	413	14.0(8.0–25.0)	504	−2	−12.5	<.001
Norwood	24.0(15.0–40.0)	1,836	22.0(13.0–37.0)	1,739	−2	−8.3	<.0001

Abbreviations: PLOS, postoperative length of stay; VSD, ventricular septal defect; TOF, tetralogy of Fallot; CAVC, complete atrioventricular canal; ASO, arterial switch operation; ICD-9, International Classification of Diseases, Ninth Revision; NA, no data.

^aData presented as median (interquartile range), PLOS is expressed in days. Reference for comparisons: clinical registry data. No data are listed for the Norwood operation for method 1, as there is no individual ICD-9 procedure code for the Norwood operation.

^bThe 10% trimmed means are included as another measure of central tendency for groups where the medians are equal: VSD repair method 2 (5.2 clinical registry and 5.3 administrative data), ASO method 2 (1 1.3 clinical registry and 1 1.0 administrative data), and Fontan method 2 (10.1 clinical registry and 9.7 administrative data).

Postoperative length of stay

Postoperative length of stay for the cases identified in the clinical registry was compared with PLOS for the cases identified in the administrative data for each operation, first using method 1 (individual ICD-9 procedure codes to identify the operation of interest in the administrative data). There were statistically significant differences in PLOS for the operations identified in the administrative versus the clinical registry data set for five of the seven benchmark operations (Table 2). However, many of the differences identified were likely not clinically meaningful (≤ 1 day); there was a difference of two days for one operation (truncus arteriosus repair).

Similar results were seen for method 2 (where a combination of ICD-9 diagnosis and procedure codes was used to define the operation of interest in the administrative data). There were statistically significant differences in PLOS for the cases identified in the administrative versus clinical data set for seven of the eight benchmark operations. Again, many of these differences were relatively small, with the exception of truncus arteriosus repair and the Norwood operation where there were differences in PLOS of two days for the cases identified in either data set.

Operations coded and classified within the RACHS-1 categories in either data set were also evaluated (Table 3). There were statistically significant differences in PLOS for the cases identified in the administrative versus clinical data set for four of the six RACHS-1 categories. Similar to the evaluation of the individual operations, many of these differences were likely not clinically meaningful. There was a difference of two days for one of the RACHS-1 categories (category 4), although this was not statistically significant (Table 3).

Table 3.

Postoperative Length of Stay (in Days) for RACHS-1 Categories in Administrative Versus Clinical Registry Data.^a

RACHS-1 Category	PLOS for Operations Identified in Clinical Registry	N	PLOS for Operations Identified in Administrative Data	N	Difference in PLOS (d)	Percent Difference in PLOS (%)	P Value
1	4.0 (3.0–5.0)	6,450	3.0 (3.0–5.0)	5,702	−1	−25.0	<.0001
2	5.0 (4.0–7.0)b	18,399	5.0 (4.0–8.0)b	17,029	0	0.0	<.0001
3	8.0(5.0–13.0)b	16,024	8.0(5.0–13.0)b	17,438	0	0.0	<.0001
4	1 1.0(7.0–20.0)	5,275	9.0(5.0–19.0)	5,469	−2	−18.2	.57
5	28.0 (22.0–39.0)	21	27.0(15.0–43.0)	59	−1	−3.6	.42
6	22.0(14.0–39.0)b	2,117	22.0(13.0–37.0)b	1,740	0	0.0	<.0001

Abbreviations: PLOS, postoperative length of stay; RACHS-1, Risk Adjustment in Congenital Heart Surgery, Version 1.

^aData are presented as median (interquartile range); PLOS is expressed in days. Reference for comparisons: clinical registry data.

^bThe 10% trimmed means are included as another measure of central tendency for groups where the medians are equal: category 2 (5.7 clinical registry and 5.9 administrative data), category 3 (9.4 clinical registry and 9.1 administrative data), and category 6 (26.0 clinical registry and 24.9 administrative data).

Cost

Total hospital costs were also compared between the cases identified in the administrative data compared to the clinical registry (Table 4). Using method 1 (individual ICD-9 procedure codes to identify the operation of interest in the administrative data), there were statistically significant differences in cost for the cases identified in the administrative versus clinical data for six of the seven benchmark operations, three of which represented a cost difference of >5%, and one a cost difference of >10% (VSD repair; Table 4).

Table 4.

Total Hospital Costs for Benchmark Operations in Administrative Versus Clinical Registry Data.^a

Operation	Cost for Operations Identified in Clinical Registry (US$)	N	Cost for Operations Identified in Administrative Data (US$)	N	Difference in Cost (US$)	Percent Difference in Cost (%)	P Value
Method 1—individual ICD-9 procedure codes
VSD repair	$26,468 ($21,057–$38,544)	4,498	$32,986 ($23,089–$62,358)	8,751	$6,518	24.6	<0001
TOF repair	$35,261 ($26,997–$53,252)	2,587	$37,213 ($27,621–$57,892)	3,180	$1,952	5.5	<0001
CAVC repair	$36,763 ($27,542–$60,261)	1,811	$33,209 ($24, 156–$56,033)	3,092	−$3,553	−9.7	.046
ASO	$70,748 ($53,777–$95,936)	1,189	$72,671 ($54,184–$101,643)	1,952	$1,923	2.7	.001
ASO + VSD repair	$74,849 ($55,786–$105,117)	467	$77,956 ($55,468–$113,795)	653	$3,107	4.2	.04
Fontan	$38,944 ($30,23 3–$56,234)	2,546	$38,917 ($29,979–$57,336)	2,845	−$27	−0.1	.02
Truncus repair	$95,605 ($64,328–$147,354)	349	$86,340 ($57,518–$143,015)	472	−$9,266	−9.7	<01
Norwood	NA	NA	NA	NA	NA	NA	NA
Method 2—additional inclusionary/exclusionary ICD-9 diagnosis and procedure codes
VSD repair	$26,468 ($21,057–$38,544)	4,498	$26,868 ($21,1 14–$39,528)	4,752	$400	1.5	.04
TOF repair	$34,263 ($26,444–$49,4907)	2,354	$36,167 ($27,093–$55,308)	2,760	$1,904	5.6	<0001
CAVC repair	$36,796 ($27,516–$60,282)	1,750	$33,690 ($24,710–$55,894)	2,631	−$3,106	−8.4	<01
ASO	$70,748 ($53,777–$95,936)	1,189	$69,743 ($53,157–$94,144)	1,148	−$1,005	−1.4	.02
ASO + VSD repair	$74,849 ($55,786–$105,117)	467	$77,570 ($55,753–$112,523)	616	$2,721	3.6	.11
Fontan	$38,944 ($30,23 3–$56,234)	2,546	$38,331 ($29,731–$55,197)	2,498	−$613	−1.6	<0001
Truncus repair	$95,605 ($64,328–$147,354)	349	$85,036 ($58,584–$141,235)	425	−$10,570	−11.1	<01
Norwood	$115,795 ($83,129–$170,593)	1,446	$110,788 ($79,022–$162,212)	1,415	−$5,007	−4.3	<0001

Abbreviations: VSD, ventricular septal defect; TOF, tetralogy of Fallot; CAVC, complete atrioventricular canal; ASO, arterial switch operation; ICD-9, International Classification of Diseases, Ninth Revision; NA, no data.

^aData are presented as median (interquartile range). Reference for comparisons: clinical registry data. No data are listed for the Norwood operation for method 1, as there is no individual ICD-9 procedure code for the Norwood operation.

Similar results were seen for method 2 (where a combination of ICD-9 diagnosis and procedure codes was used to define the operation of interest in the administrative data). Statistically significant differences in cost for the cases identified in the administrative data compared to the clinical registry data were observed for seven of the eight benchmark operations evaluated, including a >5% difference in cost for three of the eight operations and >10% difference for truncus repair (−US$10,570; P < .01; Table 4). To put some of these values into greater context, these differences would result in an underestimation of total costs associated with truncus arteriosus repair across included hospitals during the study period by US$3,688,930 if the administrative rather than clinical registry data were used to identify the cases. Similarly, there would be an underestimation of total costs associated with the Norwood operation by US$7,240,122.

Operations coded and classified within the RACHS-1 categories in either data set were also evaluated (Table 5). Although there were statistically significant differences in cost for the cases identified in the administrative versus clinical data set for four of the six RACHS-1 categories, the magnitude of these differences was relatively small and none were >5%.

Table 5.

Total Hospital Costs for RACHS-1 Categories in Administrative Versus Clinical Registry Data.^a

RACHS-1 Category	Cost for Operations Identified in Clinical Registry (US$)	N	Cost for Operations Identified in Administrative Data (US$)	N	Difference in Cost (US$)	Percent Difference in Cost (%)	P Value
1	$19,999 ($15, 193–$28,635)	5,346	$19,643 ($14,997–$27,246)	4,849	−$356	−1.8	<.0001
2	$29,350 ($22,071–$44,588)	15,854	$30,314 ($22,782–$46,127)	14,748	$964	3.3	<.0001
3	$42,602 ($28,897–$71,682)	13,744	$42,099 ($28,275–$71,1 19)	14,895	−$503	−1.2	.0001
4	$62,431 ($39,517–$103,248)	4,549	$57,214 ($29,181–$101,291)	4,660	−$5,217	−8.4	.77
5	$128,665 ($76,209–$202,214)	19	$129,799 ($89,444–$186,090)	52	$1,133	0.9	.82
6	$1 10,777 ($76,204–$167,684)	1,687	$1 10,803 ($79,035–$163,048)	1,416	$26	0.0	.03

Abbreviation: RACHS-1, Risk Adjustment in Congenital Heart Surgery, Version 1.

^aData are presented as median (interquartile range). Reference for comparisons: clinical registry data.

Comment

This large multicenter analysis demonstrates that differences in coding and classification of cases in administrative versus clinical registry data can translate into important differences in assessment of resource utilization (PLOS and cost) for certain operations. We found that these differences between data sets were minimized when evaluating groups of operations of similar risk (RACHS-1 categories) rather than individual operations.

The results of this study are similar to several previous investigations that have demonstrated differences in case ascertainment between administrative and clinical data sets in patients with congenital heart disease.^19–21 For example, in one previous study, 4,918 records in the Metropolitan Atlanta Congenital Defects Program database of patients identified as having congenital heart disease based on ICD-9 codes were recorded following medical record review using IPCCCs (the same system used in the STS-CHS database).¹⁹ The sensitivity of the ICD-9 codes for TOF was 83%, 100% for transposition of the great arteries, and 95% for hypoplastic left heart syndrome, with false positive rates were 2%, 49%, and 11%, respectively. A previous study conducted by our group was the first to evaluate how these differences in case ascertainment impacted outcomes assessment. We found that differences in the coding and classification of cases between administrative and clinical registry data sets led to important differences in the assessment of clinical outcomes.¹⁰ For example, mortality rates differed significantly for operations coded in either data set. These differences were not based on the coding of mortality, but based on differences in how operations themselves were coded and classified between data sets.¹⁰

The present study expands upon these findings and demonstrates that these issues with case ascertainment can impact the assessment of resource utilization outcomes for certain operations as well. We found that most of the differences in PLOS were relatively small (with the exception of truncus arteriosus repair and the Norwood operation), while differences in total hospital costs were more prominent, although only for certain operations. Grouping of operations into categories of similar risk (the RACHS-1 categories) appeared to mitigate many of the differences seen for the individual operations. Our linkage methodology, which results in a data set containing information as coded in the clinical registry and in the administrative data set for each patient, is unique in that it ensures that the differences found in our study are not related to the data sets containing different patients. We also excluded the few patients who had discrepant outcomes between data sets in order to isolate the impact of differences in coding and classification of cases on outcomes and ensure that any differences found were not related to differences in the coding of the outcomes themselves.

With the increased emphasis on reducing health care costs, there will undoubtedly be many further analyses of resource utilization in this population of patients. Previous reports have demonstrated that children with congenital heart defects account for the highest resource utilization among all patients with birth defects and are in the top tier of resource use across all pediatric diseases.^22,23 Our data suggest that further collaboration and linkages between clinical registry and administrative data may be the best mechanism to facilitate such analysis. This would optimize the strengths and minimize the weaknesses of each data set, by allowing the clinical data to be used for coding and identification of eligible cases, risk adjustment, and clinical outcomes assessment, and the administrative data for important resource utilization information. Efforts to expand these types of linkages are currently ongoing. Alternatively, if administrative data alone are the only type of data available, our results suggest it is advisable to focus on larger categories of operations rather than individual procedures. Additional education of administrative data coders regarding congenital heart disease may be also beneficial; however, this will not fully address the issue, as in some instances, there are simply no ICD-9 codes for certain operations, such as the Norwood operation.

Limitations

Although our study population was relatively large, our analysis was limited to centers that participated in both STS-CHS and PHIS databases during the study period. Therefore, it is possible that our findings may not be generalizable to all of the ∼ 120 pediatric heart programs in the United States.²⁴ Ongoing expansion of these linkages will allow inclusion of additional centers. Additionally, our analysis of individual operations focused on the STS benchmark operations as a first step; other operations of interest will require further study. This particular analysis evaluated the impact of differences in case ascertainment on assessment of resource utilization outcomes in the overall cohort of patients. Additional analyses are needed in order to assess whether our findings impact rankings of performance with regard to resource utilization on a hospital level.

Finally, it is unclear whether the upcoming implementation of International Classification of Diseases, Tenth Revision (ICD-10) will impact the findings of this study. Because the ICD-10 system remains largely unchanged with regard to the level of granularity of congenital heart disease coding, it is anticipated that the issues raised in our analysis will persist. In addition, it is known that in some cases, ICD-10 codes will be back coded to ICD-9 codes for the time being, as algorithms based on ICD-10 codes to identify cases currently do not exist. Ongoing work to better harmonize International Classification of Diseases, Eleventh Revision codes with the comprehensive coding system used in many congenital heart disease registries may also be useful in the future.

Conclusion

This large multicenter analysis demonstrated that differences in case ascertainment between administrative and clinical registry data for children undergoing heart surgery can translate into important differences in assessment of PLOS and cost for certain operations. Grouping of operations into categories of similar risk appeared to mitigate many of the differences seen for the individual operations. Further collaboration and linkages between clinical and administrative data sets may provide the best mechanism for accurate assessment of resource utilization in this population.

Abbreviations and Acronyms

CAVC

complete atrioventricular canal

ICD-9

International Classification of Diseases, Ninth Revision

ICD-10

International Classification of Diseases, Tenth Revision

IPCCC

International Pediatric and Congenital Cardiac Code

PHIS

Pediatric Health Information Systems

PLOS

postoperative length of stay

RACHS-1

Risk Adjustment in Congenital Heart Surgery, Version 1

STS-CHS

Society of Thoracic Surgeons Congenital Heart Surgery

TOF

tetralogy of Fallot

VSD

ventricular septal defect