Abstract
Background
As US healthcare increasingly focuses upon outcomes as a means for quantifying quality, there is a growing demand for risk models that can account for the variability of patients treated at different hospitals so that equitable comparisons between institutions can be made. We sought to apply aspects of prior risk-standardization methodology in order to begin development of a risk-standardization tool for the NCDR® IMPACT™ (Improving Pediatric and Adult Congenital Treatment) Registry.
Methods and Results
Using IMPACT, all patients undergoing diagnostic or interventional cardiac catheterization between January 2011 and March 2013 were identified. Multivariable hierarchical logistic regression was used to identify patient and procedural characteristics predictive of experiencing a major adverse event following cardiac catheterization. A total of 19,608 cardiac catheterizations were performed between January 2011 and March 2013. Amongst all cases, a major adverse event occurred in 378 (1.9%) of all cases. After multivariable adjustment, eight variables were identified as critical for risk-standardization: patient age, renal insufficiency, single-ventricle physiology, procedure-type risk group, low systemic saturation, low mixed venous saturation, elevated systemic ventricular end diastolic pressure, and elevated main pulmonary artery pressures. The model had good discrimination (C-statistic of 0.70), confirmed by bootstrap validation (validation C-statistic of 0.69).
Conclusions
Using prior risk-standardization efforts as a foundation, we developed and internally validated a model to predict the occurrence of a major adverse event following cardiac catheterization for congenital heart disease. Future efforts should be directed towards further refinement of the model variables within this large, multicenter dataset.
Keywords: heart defects, congenital; catheterization; risk factors
Despite the increased use of catheter-based techniques for the care of patients with congenital heart disease, attempts at understanding the outcomes associated with these procedures have been somewhat limited, with most prior reports limited to single or small groups of institutions1, 2 An important aspect to consider when evaluating outcomes associated with pediatric and congenital cardiac catheterization is the extent to which outcomes vary between hospitals. However, unbiased comparison between hospitals requires the application of a validated risk-standardization tool.3 Towards this end, the Congenital Heart Disease Adjustment for Risk Method (CHARM) was developed with the goal of providing a method of adjusting for case mix complexity in catheterization for congenital heart disease in order to allow for more equitable comparisons of adverse event rates.4 While the model had reasonably good discrimination, the dataset used for model development and validation incorporated data from only 8 institutions, and thus may be limited in its generalizability.
The IMPACT® Registry (IMproving Pediatric and Adult Congenital Treatment) is the largest registry, to date, collecting information on pediatric and adult patients with congenital heart disease undergoing diagnostic or interventional cardiac catheterization. IMPACT was primarily designed as a quality improvement tool to enable participating institutions to systematically evaluate their outcomes and compare them to other centers across the nation. However, the variable case-mix amongst participating centers requires development and validation of risk-standardization methodologies before the dataset can be used for benchmarking or other quality improvement initiatives. The goal of the current study is to apply aspects of prior risk-standardization methodology as a first step towards development of a risk-standardization model for the large, multicenter IMPACT Registry.
Methods
Study Population
The IMPACT Registry, part of the National Cardiovascular Data Registry (NCDR), is a U.S.-based registry collecting information on pediatric and adult patients with congenital heart disease undergoing diagnostic or interventional cardiac catheterization. Details regarding registry development and design have been previously published.5 In brief, centers performing cardiac catheterization on any pediatric patient (both with and without congenital heart disease) or adult patients with congenital heart disease are eligible for voluntary registry enrollment. Once enrolled, participating centers collect detailed information on all consecutive patients undergoing diagnostic or interventional cardiac catheterization. Collected data include information about patients’ demographics, medical history and risk factors, detailed procedural information, hemodynamic data, and information related to adverse events. More specific information is collected for six commonly performed interventional procedures: device closure of atrial septal defect (ASD), device closure of patent ductus arteriosus (PDA), pulmonary valvuloplasty, aortic valvuloplasty, angioplasty and stenting for coarctation of the aorta, and pulmonary artery stenting. The IMPACT Registry builds upon several prior pediatric and congenital cardiology database initiatives.6 The nomenclature used in the IMPACT Registry is the International Pediatric and Congenital Cardiac Code.7, 8 Data for IMPACT is collected using a standardized set of data elements and definitions and is subject to rigorous quality assurance standards consistent with other NCDR registeries.9 Only data meeting pre-specified criteria for completeness and accuracy are included in analytic datasets and used for quality reporting back to the sites. The current study used data from IMPACT v1.0.1. A comprehensive description of the IMPACT Registry version v1.0.1 data elements and definitions is available at: https://www.ncdr.com/WebNCDR/impact/home/datacollection.
Study Outcome
The primary outcome of interest was occurrence of a major adverse event, which was defined as occurrence of any of the following: cardiac arrest, tamponade (requiring pericardial drainage), embolic stroke (within 72 hours of the cardiac catheterization), device malposition or thrombus (requiring surgery), device embolization (requiring device retrieval), new requirement for dialysis, event requiring extracorporeal membrane oxygenation (ECMO), event requiring left ventricular assist device (LVAD), unplanned cardiac or vascular surgery (due to catheterization complication), or subsequent cardiac catheterization (due to catheterization complication). Unless otherwise specified, adverse events are coded up to 30 days following the catheterization procedure, aside from unplanned surgery and subsequent cardiac catheterization which are coded until the time of hospital discharge. Death was not included as part of the primary outcome because, in the current version of IMPACT, death during an episode of care cannot definitively be attributed to cardiac catheterization and could have resulted from subsequent in-hospital events (e.g. cardiac surgery) or such severe pre-procedural morbidity that the procedure might have been solely for palliative purposes. However, recognizing the clinical importance of identifying instances where death occurred following cardiac catheterization, regardless of attribution, we ran a secondary analysis including death as a major adverse event. While IMPACT does not currently allow for definitive linkage of all adverse events to the cardiac catheterization, the remainder of the adverse events selected for inclusion in this study were those that could either definitively be linked to the catheterization (e.g. device embolization, malposition, or thrombus) or those highly likely to be attributed to the cardiac catheterization procedure (e.g. embolic stroke).
From January 2011 through March 2013, there were 19,797 diagnostic or interventional cardiac catheterization procedures performed at 58 US centers. Procedures that could not be assigned to a procedural risk group (n=4)4, procedures missing data regarding single ventricle physiology (n=43), and procedures missing data on adverse events (n=142) were excluded from analysis. The final study cohort for the primary analysis included 19,608 cardiac catheterization procedures. For the secondary analysis, we redefined major adverse events to include death. For this portion of the analysis, we excluded the 1,370 instances where more than one cardiac catheterization was performed during a single hospital admission and subsequently included 18,238 episodes of care.
Study Variables for Risk Prediction
A number of baseline characteristics were screened as candidate predictors for the study outcome. These included age at the time of cardiac catheterization (categorized as neonates [<30 days], infants [≥30 days to ≤1 year], children [1≤18 years], and adults [>18 years of age]), weight, procedure status (elective, urgent, emergent, or salvage), and patient requirement for inotropes before or during the case. Additionally, presence of a genetic/congenital condition (i.e. 22q11 deletion, Alagille syndrome, Congenital Diaphragmatic Hernia, Heterotaxy syndrome, Marfan syndrome, Noonan syndrome, Rubella, Trisomy-13, Trisomy-18, Turner syndrome, Williams-Beuren syndrome), medical co-morbidities (chronic lung disease, renal insufficiency), and single ventricle physiology were evaluated for potential inclusion in the model.
Patient hemodynamic vulnerability was an additional candidate predictor variable considered for model inclusion. Hemodynamic vulnerability was defined based upon previously published, empirically derived data.4 In brief, four hemodynamic variables have been previously shown to be independently associated with experiencing a high-severity adverse event following cardiac catheterization and were used to classify a patient as hemodynamically vulnerable: systemic ventricular end diastolic pressure, systemic arterial saturation, mixed venous saturation, and main pulmonary artery pressure. Thresholds for hemodynamic vulnerability vary based upon individual patient physiology (single ventricle vs. 2-ventricle physiology) (Table 1). For the purposes of this analysis, rather than creating a composite score and categorizing patients based upon the number of hemodynamic variables present (0, 1, ≥2), as was done in the original CHARM model, each hemodynamic variable was considered separately so that the independent contribution of each risk factor could be identified. For each of the four variables, patients were classified as “yes” or “no,” depending upon whether their catheterization data met criteria for hemodynamic vulnerability. Patients missing data on one of the relevant hemodynamic parameters were placed in a third category of “missing”. We created a third category of “missing” patients, rather than exclude records with missing hemodynamic data, because excluding these records could have resulted in model bias. For example, it was unknown whether patients missing hemodynamic data were critically ill patients whose clinical status precluded a thorough hemodynamic assessment prior to pursuing a catheter-based intervention, or if they were so stable that a complete hemodynamic assessment was not considered to be clinically indicated.
Table 1.
Thresholds for Hemodynamic Vulnerability.
| Single Ventricle | Non-Single Ventricle | |
|---|---|---|
| Systemic Ventricular End Diastolic Pressure | ≥18mmHg | ≥18mmHg |
| Systemic Arterial Saturation | <78% | <95% |
| Mixed Venous Saturation | <50% | <60% |
| Main Pulmonary Artery Mean Pressure | ≥17mmHg | n/a |
| Main Pulmonary Artery Systolic Pressure | n/a | ≥45mmHg |
The last candidate variable considered for model inclusion was procedure-type risk group. Pediatric and congenital cardiac catheterization encompasses a wide variety of procedure types, each associated with varying degrees of risk. Given the broad range of procedures and the infrequency with which some of the procedures are performed, adjustment for each individual procedure type is not feasible. Procedure risk groups were created to overcome this issue and to establish a classification system whereby procedures of similar risk are grouped. A full description of procedure-type risk group development has been previously described in detail.10 In brief, the procedure-type risk groups were created using data from the Congenital Cardiac Catheterization Project on Outcomes (C3PO) and were developed based on both expert consensus and empirically derived data. Four categories of procedural risk were created (Category 1= procedures associated with lowest risk vs. Category 4=procedures associated with highest risk). The risk groups were validated within the C3PO dataset and found to have good discrimination between each of the categories. For catheterization lab visits where more than one procedure was performed, the case was classified according to the procedure of highest risk.
Statistical Analysis
Characteristics of those patients experiencing an adverse event were compared to characteristics of those not experiencing an adverse event using Student's t-test for continuous variables and chi-square or Fisher's exact test for categorical variables. For model development, hierarchical multivariable logistic regression was used to identify characteristics predictive of experiencing an adverse event, while also accounting for the clustered nature of the data (patients nested within hospitals).11 Use of hierarchical models to estimate the log-odds of experiencing an adverse event as a function of demographic and clinical variables (both fixed effects) and a random effect for each hospital allowed us to assess for hospital variation in risk-standardized adverse event rates after accounting for patient case-mix. This is the same analytic strategy used by Medicare for risk-standardized outcomes reporting.
All candidate variables were considered for model inclusion with the most clinically relevant variables selected for final model inclusion. Multicollinearity between covariates was assessed for each variable prior to model inclusion.12 A C-statistic, which quantifies the receiver operating characteristic curve (ROC), was calculated in order to assess model discrimination.13 For model validation, observed vs. predicted plots were constructed and 1000 bootstrap samples were used to derive a validation C-statistic that would correct for potential model over-fitting.14 Calibration (i.e., agreement between predicted and observed outcomes) was assessed by plotting observed rates versus mean predicted probabilities within deciles of predicted risk. For well-calibrated models these points should line along the y=x line.15 All study analyses were performed with SAS 9.3 (SAS Institute, Cary, NC) and R version 2.11.1.16 All authors have read and agree to the manuscript as written. The study was conducted on de-identified quality improvement registry data and did not meet criteria for requirement of informed consent. The IMPACT Registry's Research and Publications Committee approved the final manuscript draft.
Results
Predictors of Survival
Among the study population, a major adverse event occurred in 378 (1.9%) cases. Cardiac arrest was the most frequent adverse event to occur, occurring in 158/19,608 (0.8%) catheterization procedures. Device embolization and unplanned cardiac surgery occurred in 71/19,608 (0.4%) and 67/19,608 (0.3%) catheterization procedures, respectively. An event requiring ECMO occurred in 55/19,608 (0.3%) cases and subsequent cardiac catheterization in 57/19,608 (0.3%). The remainder of the individual adverse events occurred less often, each occurring in less than 0.2% of catheterization procedures. Tables 2 and 3 compare baseline characteristics of those experiencing an adverse event to characteristics of those not experiencing an adverse event. Neonates and infants were more likely to experience an adverse event compared to older children and adults. Patients with single ventricle physiology and renal insufficiency were more likely to experience an adverse event, whereas those with chronic lung disease or a genetic/congenital condition were no more likely to experience an adverse event. Those patients requiring inotropic support before the case were significantly more likely to experience an adverse event compared to patients without inotropic needs. In univariate analysis, procedure risk group was significantly associated with survival and, as expected, patients undergoing procedures in risk group 1 were less likely to experience an adverse event compared to those in higher risk groups.
Table 2.
Baseline Characteristics of Patients Experiencing and Not Experiencing an Adverse Event .
| Major Adverse Event |
||||
|---|---|---|---|---|
| No. (%) (N =19,608) | Yes (n=378) | No (n= 19,230) | P value | |
| Age-- no. (%) | <0.001 | |||
| <30 days | 1,226 (6.3) | 67 (17.7) | 1,159 (6.0) | |
| ≥30 days - ≤1 year | 3,751 (19.1) | 98 (25.9) | 3,653 (19.0) | |
| >1year - ≤18 years | 11,482 (58.6) | 159 (42.1) | 11,323 (58.9) | |
| >18 years | 3,149 (16.1) | 54 (14.3) | 3,095 (16.1) | |
| Male Sex-- no. (%) | 10,415 (53.1) | 222 (58.7) | 10,193 (53.0) | 0.03 |
| Race-- no. (%) | 0.99 | |||
| Black | 3,610 (18.4) | 72 (19.0) | 3,538 (18.4) | |
| White | 13,712 (69.9) | 263 (69.6) | 13,449 (69.9) | |
| Other | 1,132 (5.8) | 21 (5.6) | 1,111 (5.8) | |
| Unknown | 1,154 (5.9) | 22 (5.8) | 1,132 (5.9) | |
| Weight-- mean ±s.d.* | 31.1± 29.3 | 25.8 ± 30.1 | 31.2 ± 29.2 | <0.001 |
| Single Ventricle-- no. (%) | 3,775 (19.3) | 102 (27.0) | 3,673 (19.1) | <0.001 |
| Genetic/Congenital Condtion-- no. (%)† | 2,133 (10.9) | 43 (11.4) | 2,090 (10.9) | 0.74 |
| Chronic Lung Disease-- no. (%)‡ | 1,233 (6.3) | 28 (7.4) | 1,205 (6.3) | 0.36 |
| Renal Insufficiency-- no. (%)§ | 548 (2.8) | 26 (6.9) | 522 (2.7) | <0.001 |
| Procedure-Type Risk Group-- no. (%) | <0.001 | |||
| Risk Group 1 | 8,332 (42.5) | 84 (22.2) | 8,248 (42.9) | |
| Risk Group 2 | 6,350 (32.4) | 132 (34.9) | 6,218 (32.3) | |
| Risk Group 3 | 3,628 (18.5) | 110 (29.1) | 3,518 (18.3) | |
| Risk Group 4 | 1,298 (6.6) | 52 (13.8) | 1,246 (6.5) | |
| Procedure Status-- no. (%)∥ | <0.001 | |||
| Elective | 16,677 (85.4) | 232 (61.7) | 16,445 (85.9) | |
| Urgent | 2,363 (12.1) | 84 (22.3) | 2,279 (11.9) | |
| Emergent | 451 (2.3) | 46 (12.2) | 405 (2.1) | |
| Salvage | 40 (0.2) | 14 (3.7) | 26 (0.1) | |
| Inotrope Use Before Case—no. (%)# | 927 (4.7) | 80 (21.3) | 847 (4.4) | <0.001 |
Abbreviations: no., number; s.d, standard deviation
102 patients (0 with an adverse event) with missing information for weight
59 patients (2 with and 57 without adverse events) with missing information for genetic/congenital condition
29 patients (1 with and 28 without adverse events) with missing information on chronic lung disease
15 patients (0 with an adverse event) with missing information on renal insufficiency
77 patients (2 with and 75 without adverse events) with missing information on procedure status
86 patients (3 with and 83 without adverse events) with missing information on inotropic use
Table 3.
Hemodynamic Characteristics of Patients Experiencing and Not Experiencing an Adverse Event.
| Major Adverse Event |
||||
|---|---|---|---|---|
| No. (%) (N =19,608; SV=3,775) | Yes (n=378; SV=102) | No (n= 19,230; SV=3,673) | P value | |
| Systemic Ventricular EDP-- mean± s.d.* | 10.8 ± 4.9 | 11.5 ± 6.1 | 10.8 ± 4.9 | 0.04 |
| Cardiac Index-- mean± s.d.† | 3.8 ± 1.3 | 3.7 ± 1.4 | 3.8 ± 1.3 | 0.11 |
| Single Ventricle Hemodynamic Data | ||||
| Systemic Saturation-- mean± s.d.‡ | 83.3 ± 10.3 | 76.5 ± 16.1 | 83.5 ± 10.0 | <0.001 |
| Mixed Venous Saturation-- mean± s.d.§ | 61.3 ± 11.2 | 54.1 ± 13.9 | 61.5 ± 11.0 | < 0.001 |
| MPA Mean Pressure -- mean± s.d.∥ | 16.0 ± 7.4 | 19.0 ± 8.7 | 15.9 ± 7.3 | < 0.001 |
| Non-Single Ventricle Hemodynamic Data | ||||
| Systemic Saturation-- mean± s.d.# | 94.7 ± 7.1 | 92.4 ± 10.5 | 94.7 ± 7.0 | < 0.001 |
| Mixed Venous Saturation-- mean± s.d.** | 70.3 ± 9.4 | 65.3 ± 13.6 | 70.3 ± 9.3 | < 0.001 |
| MPA Systolic Pressure -- mean± s.d.*** | 31.5 ± 16.7 | 38.1 ± 22.6 | 31.4 ± 16.6 | < 0.001 |
| Systemic Ventricular EDP ≥ 18mmHg | 0.005 | |||
| Yes-- no. (%) | 988 ( 5.0) | 27 (7.1) | 961 (5.0) | |
| No-- no. (%) | 11,332 (57.8) | 189 (50.0) | 11,143 (57.9) | |
| Missing-- no. (%) | 7,288 (37.2) | 162 (42.9) | 7,126 (37.1) | |
| Saturation <95% (non-SV) or <78% (SV) | <0.001 | |||
| Yes-- no. (%) | 4,894 (25.0) | 128 (33.9) | 4,766 (24.8) | |
| No-- no. (%) | 12,765 (65.1) | 191 (50.5) | 12,574 (65.4) | |
| Missing-- no. (%) | 1,949 (9.9) | 59 (15.6) | 1,890 (9.8) | |
| MV Saturation <60% (non-SV) or <50% (SV) | ||||
| Yes-- no. (%) | 2,131 (10.9) | 90 (23.8) | 2,041 (10.6) | <0.001 |
| No-- no. (%) | 14,856 (75.8) | 210 (55.6) | 14,646 (76.2) | |
| Missing-- no. (%) | 2,621 (13.4) | 78 (20.6) | 2,543 (13.2) | |
| PA Systolic Pressure ≥45 (non-SV) or Mean Pressure ≥17mmHg (SV) | <0.001 | |||
| Yes-- no. (%) | 2.933 (15.0) | 85 (22.5) | 2.848 (14.8) | |
| No-- no. (%) | 13.046 (66.5) | 171 (45.2) | 12.875 (67.0) | |
| Missing-- no. (%) | 3.629 (18.5) | 122 (32.3) | 3.507 (18.2) | |
Abbreviations: no., number; s.d, standard deviation; SV, single ventricle; EDP, end diastolic pressure; MPA, main pulmonary artery; PA, pulmonary artery
7, 288 patients (162 with and 7,126 without adverse events) with missing information for systemic ventricular EDP
4,179 patients (134 with and 4,045 without adverse events) with missing information for cardiac index
301 patients (15 with and 286 without adverse events) with missing information for SV systemic saturation
476 patients (23 with and 453 without adverse events) with missing information for SV mixed venous saturation
811 patients (31 with and 780 without adverse events) with missing information for SV MPA mean pressure
1.648patients (44 with and 1604 without adverse events) with missing information for non-SV systemic saturation
2.145 patients (55 with and 2.090 without adverse events) with missing information for non-SV mixed venous saturation
2.818 patients (91 with and 2.727 without adverse events) with missing information for non-SV MPA systolic pressure
After multivariable adjustment, the eight variables included in the final model included age, single-ventricle anatomy, renal insufficiency, procedure-type risk category, low systemic arterial saturation, low mixed venous saturation, elevated systemic ventricular end diastolic pressure, and elevated main pulmonary artery systolic or mean pressure (Table 4). The model had good discrimination (C-statistic of 0.70). Model calibration was confirmed with observed vs. predicted plots, with a slope of 0.97 (standard error [SE] 0.041; p-value [for difference from 1]= 0.42) (Figure 1). For model validation, we performed a series of 1000 bootstrap samples and found that model discrimination was similar (bootstrap-corrected validation C-statistic of 0.69).
Table 4.
Model Predictors of a Major Adverse Event.
| Predictor | Odds Ratio (95% CI) |
|---|---|
| Age | |
| Neonates (<30 days) | Reference |
| Infants (≥30 days to ≤1 year) | 0.55 (0.37, 0.83) |
| Children (1≤ 18 years) | 0.67 (0.45, 0.99) |
| Adults (>18 years) | 0.81 (0.52, 1.28) |
| Single Ventricle | 1.37 (1.07, 1.75) |
| Renal Insufficiency | 2.61 (1.69, 4.01) |
| Saturation <95% (non-SV) or <78% (SV) | |
| No | Reference |
| Yes | 1.04 (0.80, 1.36) |
| Missing | 1.37 (0.92, 2.03) |
| MV Saturation <60% (non-SV) or <50% (SV) | |
| No | Reference |
| Yes | 2.20 (1.63, 2.96) |
| Missing | 1.10 (0.75, 1.61) |
| MPA Systolic Pressure ≥45 (non-SV) or MPA Mean Pressure ≥17mmHg (SV) | |
| No | Reference |
| Yes | 1.62 (1.22, 2.15) |
| Missing | 1.54 (1.13, 2.09) |
| Systemic Ventricular EDP ≥ 18mmHg | |
| No | Reference |
| Yes | 1.41 (0.92, 2.17) |
| Missing | 1.27 (0.99, 1.63) |
| Risk Group | |
| Group 1 | Reference |
| Group 2 | 2.38 (1.73. 3.28) |
| Group 3 | 2.51 (1.77. 3.55) |
| Group 4 | 3.81 (2.60. 5.59) |
Abbreviations: CI, confidence interval; SV, single ventricle; MV, mixed venous saturation; MPA, main pulmonary artery; EDP, end diastolic pressure
Figure 1.
Calibration of the Final Model (Primary Analysis) in the Derivation Cohort. The model showed excellent calibration, with slope of 0.97.
Using the same eight variables, a second multivariable model was constructed modifying the outcome of interest to include death in addition to the original adverse events (Table 5). Model discrimination for the secondary analysis was even further improved (C-statistic 0.77). Model validation was again performed using a series of 1000 boostrap samples and was similar (bootstrap-corrected validation C-statistic of 0.76). Model calibration was excellent with a slope of 1.03 (SE 0.03; p-value [for difference from 1]= 0.34).
Table 5.
Model Predictors of a Major Adverse Event (Including Death).
| Predictor | Odds Ratio (95% CI) |
|---|---|
| Age | |
| Neonates (<30 days) | Reference |
| Infants (≥30 days to ≤1 year) | 0.46 (0.33, 0.65) |
| Children (1≤ 18 years) | 0.30 (0.21, 0.43) |
| Adults (>18 years) | 0.28 (0.19, 0.43) |
| Single Ventricle | 1.40 (1.13, 1.74) |
| Renal Insufficiency | 4.89 (3.43, 6.96) |
| Saturation <95% (non-SV) or <78% (SV) | |
| No | Reference |
| Yes | 1.05 (0.84, 1.32) |
| Missing | 1.22 (0.85, 1.74) |
| MV Saturation <60% (non-SV) or <50% (SV) | |
| No | Reference |
| Yes | 2.56 (1.99, 3.30) |
| Missing | 1.63 (1.18, 2.26) |
| MPA Systolic Pressure ≥45 (non-SV) or MPA Mean Pressure ≥17mmHg (SV) | |
| No | Reference |
| Yes | 2.73 (2.15, 3.46) |
| Missing | 1.59 (1.21, 2.09) |
| Systemic Ventricular EDP ≥ 18mmHg | |
| No | Reference |
| Yes | 1.65 (1.14, 2.39) |
| Missing | 1.31 (1.06, 1.63) |
| Risk Group | |
| Group 1 | Reference |
| Group 2 | 2.18 (1.60, 2.97) |
| Group 3 | 1.97 (1.41, 2.74) |
| Group 4 | 3.01 (2.10, 4.31) |
Discussion
Using a large multicenter registry, we identified patient characteristics associated with an increased risk of experiencing an adverse event following cardiac catheterization for diagnosis or treatment of congenital heart disease. In our model, we identified eight patient characteristics critical for risk-standardization, including both procedure-type risk group and hemodynamic vulnerability as derived in the CHARM model. Importantly, while we applied aspects of prior risk-standardization methodology, we did so using the largest available dataset for pediatric and congenital cardiac catheterization and thus improved upon prior risk-standardization efforts. Ultimately, risk-standardization will be of critical importance to institutions participating in IMPACT, allowing them to compare their outcomes to other US centers after adjusting for important patient characteristics, and thus identify areas for improvement. Ultimately this could allow for improved care in the setting of catheterization for congenital heart disease.
Two prior studies have aimed to create a risk-standardization model for congenital cardiac catheterization. The first used data from a single institution to create a risk-model for preventable and possibly preventable adverse events.17 The model identified weight and procedure-type risk group as important for predicting occurrence of any preventable or possibly preventable adverse event whereas procedure-type risk group and hemodynamic vulnerability were identified as critical for predicting the occurrence of a higher-severity adverse event. As the first attempt to risk-standardize outcomes for congenital cardiac catheterization, this project laid an important foundation, however the work was considered preliminary because of its restriction to data from a single center. The CHARM model enhanced these efforts and was the first attempt at a multi-center risk-standardization model for congenital cardiac catheterization.4 The model derived new criteria for hemodynamic vulnerability and determined that procedure-type risk group, hemodynamic vulnerability, and age less than one year were critical for risk-standardization.
The model developed in this study applies aspects of the CHARM model but also significantly expands upon the work of these prior studies. First, IMPACT incorporates data from a large group of centers throughout the US and thus has improved generalizability when compared to prior risk models. Second, the standardized definitions and detailed patient, procedural, and hemodynamic information collected within IMPACT allowed us to consider multiple risk factors for model inclusion. Lastly, our study adhered to the recommended guidelines for statistical models used for publicly reported outcomes, including methodology to account for the multilevel organization of data.18 Our study identified eight unique patient characteristics that were critical for risk-standardization. Similar to prior studies, we identified procedure-type risk group and hemodynamic vulnerability as important variables to consider as part of the risk-standardization process. Additionally, we identified patient age, single ventricle physiology, and renal insufficiency as predictive of experiencing an adverse event as the result of cardiac catheterization.
Currently, institutions participating in IMPACT receive quarterly reports regarding their performance for several quality metrics including proportion of patients experiencing a major adverse event or death as a result of cardiac catheterization. However, because there is not a validated risk-standardization tool for IMPACT only crude adverse event rates are reported. While institutions can use the present quality metrics to benchmark outcomes within their own institution over time, the quality metrics are less useful for institutions trying to gauge their performance compared to other centers. Particularly in the field of pediatric and congenital cardiac catheterization, where patient heterogeneity can be substantial, adjustment for patient characteristics becomes crucial before meaningful comparisons between institutions can be made.
Our study should be interpreted in the context of several potential limitations. Our study used prior risk-standardization efforts as a foundation for the development of a risk-model specific to IMPACT. As such, our study incorporated elements from prior risk-standardization work, including the criteria for hemodynamic vulnerability and procedure-type risk group derived using data from C3PO. Given the differences in data elements and definitions (e.g. definition of major adverse event) between IMPACT and C3PO, these elements may not have optimal performance within IMPACT. For example, in our model, procedure-type risk groups 2 and 3 did not discriminate well. While the overall model had reasonable discrimination, future work to refine the procedure risk groups and criteria for hemodynamic vulnerability is crucial, particularly given that these variables are likely to be most predictive of experiencing an adverse event and therefore critical elements of the risk-standardization process. Secondly, IMPACT only collects data on in-hospital adverse events, and our risk model does not account for adverse events occurring after hospital discharge (e.g. ASD device erosion). Several adult observational registries have considered incorporating post-discharge follow-up (i.e. 30-day follow-up), and a similar process could be considered for future versions of IMPACT. Alternatively, a linkage between IMPACT and other registries that collect longitudinal data would allow for tracking of patients over time and could help in identifying patients who developed late complications that may have been related to their cardiac catheterization procedure. Thirdly, we created a risk-standardization model for occurrence of a major adverse event following pediatric and congenital cardiac catheterization, however our model did not take into account procedural success. When using our risk-model to compare rates of adverse events, it is unknown whether lower rates of adverse events at some institutions are the result of less aggressive technique and whether these same institutions have lower success rates. Ideally, outcome comparisons between institutions would incorporate measures of procedural safety as well as procedural efficacy. While the goal of our project was risk-standardization for occurrence of a major adverse event, evaluation of procedural success could be considered as part of future risk-standardization efforts. Fourth, adverse events reported to IMPACT cannot definitively be linked to the catheterization procedure. While we intentionally selected adverse events that were highly likely to be related to the procedure, there are instances where an adverse event may not have occurred as a result of cardiac catheterization. As future versions of IMPACT are developed, we may need to consider ways to modify data collection in order to more definitively link adverse events to cardiac catheterization. Lastly, given that IMPACT is a relatively new registry, our analysis preceded a formal audit of the registry data. However, given the rigorous quality assurance standards applied to the registry, we have no reason to believe that results of the audit would significantly impact our study.
Conclusion
Incorporating elements from prior risk-standardization methodology, we have developed and validated a risk-standardization tool for major adverse events following pediatric and congenital cardiac catheterization. Future efforts should be directed towards further refinement of the model variables within this large, multicenter dataset. Ultimately, risk-standardization will be of critical importance to institutions participating in IMPACT, allowing hospitals to compare their outcomes to other centers, identify gaps in practice, and improve care for patients with congenital heart disease.
Supplementary Material
Acknowledgments
Funding Sources: Dr. Jayaram received support from a T32 training grant (HL110837) from the National Heart Lung and Blood Institute.
Footnotes
Disclosures: Dr. Spertus discloses grant funding from NIH, AHA, Lilly, Amorcyte and Genentech. He serves on Scientific Advisory Boards for United Healthcare, St. Jude Medical and Genentech. He serves as a paid editor for Circulation: Cardiovascular Quality and Outcomes and has intellectual property rights for the Seattle Angina Questionnaire, Kansas City Cardiomyopathy Questionnaire, Peripheral Artery Questionnaire and an equity interest in Health Outcomes Sciences. Dr. Beekman serves as a proctor for Amplatzer Occluder Devices for St. Jude Medical, Inc. Dr. Bergersen is a consultant for 480 Biomedical.
References
- 1.Vitiello R, McCrindle BW, Nykanen D, Freedom RM, Benson LN. Complications associated with pediatric cardiac catheterization. J Am Coll Cardiol. 1998;32:1433–1440. doi: 10.1016/s0735-1097(98)00396-9. [DOI] [PubMed] [Google Scholar]
- 2.Bergersen L, Marshall A, Gauvreau K, Beekman R, Hirsch R, Foerster S, Balzer D, Vincent J, Hellenbrand W, Holzer R, Cheatham J, Moore J, Lock J, Jenkins K. Adverse event rates in congenital cardiac catheterization - a multi-center experience. Catheter Cardiovasc Interv. 2010;75:389–400. doi: 10.1002/ccd.22266. [DOI] [PubMed] [Google Scholar]
- 3.Krumholz HM, Keenan PS, Brush JE, Jr., Bufalino VJ, Chernew ME, Epstein AJ, Heidenreich PA, Ho V, Masoudi FA, Matchar DB, Normand SL, Rumsfeld JS, Schuur JD, Smith SC, Jr., Spertus JA, Walsh MN. Standards for measures used for public reporting of efficiency in health care: A scientific statement from the American Heart Association interdisciplinary council on quality of care and outcomes research and the American College of Cardiology Foundation. Circulation. 2008;118:1885–1893. doi: 10.1161/CIRCULATIONAHA.108.190500. [DOI] [PubMed] [Google Scholar]
- 4.Bergersen L, Gauvreau K, Foerster SR, Marshall AC, McElhinney DB, Beekman RH, 3rd, Hirsch R, Kreutzer J, Balzer D, Vincent J, Hellenbrand WE, Holzer R, Cheatham JP, Moore JW, Burch G, Armsby L, Lock JE, Jenkins KJ. Catheterization for Congenital Heart Disease Adjustment for Risk Method (CHARM). JACC Cardiovasc Interv. 2011;4:1037–1046. doi: 10.1016/j.jcin.2011.05.021. [DOI] [PubMed] [Google Scholar]
- 5.Martin GR, Beekman RH, Ing FF, Jenkins KJ, McKay CR, Moore JW, Ringel RE, Rome JJ, Ruiz CE, Vincent RN. The IMPACT Registry: Improving pediatric and adult congenital treatments. Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu. 2010;13:20–25. doi: 10.1053/j.pcsu.2010.02.004. [DOI] [PubMed] [Google Scholar]
- 6.Jenkins KJ, Beekman Iii RH, Bergersen LJ, Everett AD, Forbes TJ, Franklin RC, Klitzner TS, Krogman ON, Martin GR, Webb CL. Databases for assessing the outcomes of the treatment of patients with congenital and paediatric cardiac disease--the perspective of cardiology. Cardiol Young. 2008;18(Suppl 2):116–123. doi: 10.1017/S1047951108002825. [DOI] [PubMed] [Google Scholar]
- 7.Bergersen L, Everett AD, Giroud JM, Martin GR, Franklin RC, Beland MJ, Krogmann ON, Aiello VD, Colan SD, Elliott MJ, Gaynor JW, Kurosawa H, Maruszewski B, Stellin G, Tchervenkov CI, Walters HL, 3rd, Weinberg P, Jacobs JP. Report from the international society for nomenclature of paediatric and congenital heart disease: Cardiovascular catheterisation for congenital and paediatric cardiac disease (part 1 - procedural nomenclature). Cardiology Young. 2011;21:252–259. doi: 10.1017/S104795111000185X. [DOI] [PubMed] [Google Scholar]
- 8.Bergersen L, Giroud JM, Jacobs JP, Franklin RC, Beland MJ, Krogmann ON, Aiello VD, Colan SD, Elliott MJ, Gaynor JW, Kurosawa H, Maruszewski B, Stellin G, Tchervenkov CI, Walters HL, Weinberg P, Everett AD. Report from the international society for nomenclature of paediatric and congenital heart disease: Cardiovascular catheterisation for congenital and paediatric cardiac disease (part 2 - nomenclature of complications associated with interventional cardiology). Cardiol Young. 2011;21:260–265. doi: 10.1017/S1047951110001861. [DOI] [PubMed] [Google Scholar]
- 9.Messenger JC, Ho KK, Young CH, Slattery LE, Draoui JC, Curtis JP, Dehmer GJ, Grover FL, Mirro MJ, Reynolds MR, Rokos IC, Spertus JA, Wang TY, Winston SA, Rumsfeld JS, Masoudi FA. The National Cardiovascular Data Registry (NCDR) data quality brief: The NCDR data quality program in 2012. J Am Coll Cardiol. 2012;60:1484–1488. doi: 10.1016/j.jacc.2012.07.020. [DOI] [PubMed] [Google Scholar]
- 10.Bergersen L, Gauvreau K, Marshall A, Kreutzer J, Beekman R, Hirsch R, Foerster S, Balzer D, Vincent J, Hellenbrand W, Holzer R, Cheatham J, Moore J, Lock J, Jenkins K. Procedure- type risk categories for pediatric and congenital cardiac catheterization. Circ Cardiovasc Interv. 2011;4:188–194. doi: 10.1161/CIRCINTERVENTIONS.110.959262. [DOI] [PubMed] [Google Scholar]
- 11.Goldstein H. Multilevel statistical models. Edward Arnold; Wiley; London and New York: 1995. [Google Scholar]
- 12.Belsley DAK,E, Welsch RE. Regression Diagnostics : Identifying influential data and sources of collinearity. New York: 1980. [Google Scholar]
- 13.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988;44:837–845. [PubMed] [Google Scholar]
- 14.Harrell FE. Regression modeling strategies: With applications to linear models, logistic regression, and survival analysis. Springer-Verlag; New York, NY: 2001. [Google Scholar]
- 15.Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer; New York: 2009. [Google Scholar]
- 16.R Development Core Team . R: A language and environment for statistical computing. R Foundation for Statistical Computing V; Austria: 2008. ISBN 3-900051-07-0, URL http://www.R-project.org. [Google Scholar]
- 17.Bergersen L, Gauvreau K, Lock JE, Jenkins KJ. A risk adjusted method for comparing adverse outcomes among practitioners in pediatric and congenital cardiac catheterization. Congenit Heart Dis. 2008;3:230–240. doi: 10.1111/j.1747-0803.2008.00196.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Krumholz HM, Brindis RG, Brush JE, Cohen DJ, Epstein AJ, Furie K, Howard G, Peterson ED, Rathore SS, Smith SC, Jr., Spertus JA, Wang Y, Normand SL. Standards for statistical models used for public reporting of health outcomes: An American Heart Association scientific statement from the quality of care and outcomes research interdisciplinary writing group: Cosponsored by the council on epidemiology and prevention and the stroke council. Endorsed by the American College of Cardiology foundation. Circulation. 2006;113:456–462. doi: 10.1161/CIRCULATIONAHA.105.170769. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.