Developing Tools to Measure Quality in Congenital Catheterization and Interventions: The Congenital Cardiac Catheterization Project on Outcomes (C3PO)

Abstract

The broad range of relatively rare procedures performed in pediatric cardiac catheterization laboratories has made the standardization of care and risk assessment in the field statistically quite problematic. However, with the growing number of patients who undergo cardiac catheterization, it has become imperative that the cardiology community overcomes these challenges to study patient outcomes. The Congenital Cardiac Catheterization Project on Outcomes was able to develop benchmarks, tools for measurement, and risk adjustment methods while exploring procedural efficacy. Based on the success of these efforts, the collaborative is pursuing a follow-up project, the Congenital Cardiac Catheterization Project on Outcomes—Quality Improvement, aimed at improving the outcomes for all patients undergoing catheterization for congenital heart disease by reducing radiation exposure.

Introduction

In the past decade, efforts to standardize care in the pediatric cardiac catheterization laboratory have been limited by significant challenges. With the growing number of patients who undergo cardiac catheterization, it has become imperative that the cardiology community overcomes these challenges to study patient outcomes. The Congenital Cardiac Catheterization Project on Outcomes (C3PO) was launched to develop benchmarks, tools for measurement, and risk adjustment methods and explore measures of procedural efficacy. The C3PO multicenter data set made it possible to report on the incidence and specific nature of events and support the development of risk adjustment tools to account for the heterogeneity of the patient population.

Defining the Problem

Congenital cardiac catheterization encompasses a broad range of relatively rare procedures. Having low procedure-type frequencies makes it quite problematic to statistically assess the risk of these individual procedures. In addition, case-mix distribution can vary significantly among institutions and practitioners. This lack of uniformity results in different expected rates of adverse outcomes. Many pediatric cardiology catheterization labs maintain institutionally developed databases to survey patient demographics and complication rates, which are needed for internal assessment and reporting. Based on published data, complication rates in catheterization labs range from 9% to 24%, with major events occurring in 1% to 4% of cases. In the past decade, most institutions have also reported mortality rates of less than 1%.1-4 Although this information is helpful for individual institutions, small sample sizes and the discrepancies in outcome definition have constrained the ability to make comparisons among institutions and individual practitioners.

Establishing a Multi-Institutional Registry

In 2006, the first risk adjustment model for adverse events was developed from a single institutional data set at Boston Children’s Hospital. Six procedure-type risk groups were defined by categorizing 84 procedure types in similar risk groups based solely on consensus of expert opinion. Other factors were considered such as age, hemodynamic factors, and diagnosis. Further, all adverse events were classified into severity levels from 1 (none) to 5 (catastrophic), and this same nomenclature was adopted by the International Pediatric Congenital Cardiac Code (Table 1).5, 6

Table 1.

The International Pediatric Congenital Cardiac Code classifies adverse events into severity levels.

	Risk Category 1	Risk Category 2	Risk Category 3	Risk Category 4
Diagnostic Case	Age ≥ 1 month < 1 year	Age ≥ 1 month < 1 year	Age < 1 month
Valvuloplasty		Pulmonary valve ≥ 1 month	Aortic valve ≥ 1 month	Mitral valve
			Pulmonary valve < 1 month	Aortic valve < 1 month
			Tricuspid valve
Device or coil closure	Venous	PDA	Systemic surgical shunt	VSD
	collateral	ASD or PFO	Baffle leak	Perivalvar leak
	LSVC	Fontan fenestration	Coronary fistula
		Systemic to pulmonary artery collaterals
Balloon angioplasty		RVOT	Pulmonary artery < 4 vessels	Pulmonary artery ≥ 24 weeks
		Aorta dilation < 8 ATM	Pulmonary artery ≥ 4 vessels all < 8 ATM	Pulmonary vein
			Aorta > 8 ATM or CB
			Systemic artery (not aorta)
			Systemic surgical shunt
			Systemic to pulmonary collaterals
			Systemic vein
Stent placement		Systemic vein	RVOT	Ventricular septum
			Aorta	Pulmonary artery
			Systemic artery (not aorta)	Pulmonary vein
				Systemic surgical shunt
				Systemic pulmonary Collateral
Stent redilation		RVOT	Pulmonary artery	Ventricular septum
		Atrial septum	Pulmonary vein
		Aorta
		Systemic artery (not aorta)
		Systemic vein
Other	Myocardial	Snare foreign body	Atrial septostomy	Atrial septum dilation and stent
	biopsy	Transseptal puncture	Recanalization of jailed vessel in stent	Any catheterization < 4 days after surgery
			Recanalization of occluded vessel
				Atretic valve performation

RVOT indicates right ventricular outflow tract; RV, right ventricle; PA, pulmonary artery; RVOT includes RV-to-PA conduit or status after RVOT surgery with no conduit; LSVC, left superior vena cava; ATM, atmospheres; CM, cutting balloon; PDA, patent ductus arteriosus; ASD, atrial septal defect; PFO, patent foramen ovale; and VSD, ventricular septal defect.

Although development of the risk adjustment model was an important milestone in the field, it was understood that this method could not be generalized since it was derived from a single institutional data set. Therefore, a method would need to be created from a multi-institutional data set using common nomenclature to record patient and procedural characteristics and outcomes. As a result, eight institutions collaborated and launched the Congenital Cardiac Catheterization Project on Outcomes (C3PO) to establish a multicenter data set. Starting in 2007, each institution prospectively collected data on all catheterization procedures performed, including patient and procedural characteristics and adverse events. All adverse events underwent independent review for proper severity classification. Data audits were conducted throughout the project to ensure that all sites were recording accurate and complete case information.

Grant and participant funding were obtained and used to support the project. Sites benefited from on-demand report functionality, which allowed assessment of their own performance and use of the data for their own quality improvement initiatives.

Procedure-Type Risk Categories

The C3PO group focused on empirically defining the risk attributes they had previously defined by consensus. After sufficient data were collected in 2009, the collaborative began to refine procedure types and determine final categorization. The cases were parsed into the six consensus-based risk categories as defined in the single-center study. Data had shown that biopsy cases had a much lower adverse event rate than either diagnostic or interventional catheterization cases.7 On the other hand, diagnostic cases, which represented a large proportion of the procedure types, appeared to be a heterogeneous group in terms of outcome by age. Using expert opinion and empiric analysis, four final risk categories were determined, maximizing similarities within groups and discrimination between categories.

Hemodynamic Vulnerability Indicator

While it was understood that increased procedure-type risk correlated strongly with the occurrence of adverse events, the C3PO group sought to define other explanatory factors. A hemodynamic vulnerability threshold, determined solely on opinion, defined an important variable in the original single-center study. The C3PO collaborative worked to further define how hemodynamic factors, intrinsic to the patient’s status, may be associated with differences in adverse event rates. Eight candidate hemodynamic variables were considered: cardiac index, right ventricular (RV) systolic pressure, RV to systemic pressure ratio, systemic ventricle end-diastolic pressure, mixed venous saturation, systemic arterial saturation, main pulmonary artery systemic pressure, and main pulmonary artery mean pressure. Multivariable modeling yielded four indicators of hemodynamic vulnerability independently related to the occurrence of high-severity adverse events. These four indicators included systemic ventricular end-diastolic pressure ≥ 18 mm Hg, systemic arterial saturation < 95% or < 78% if single ventricle (SV), mixed venous saturation < 60% or < 50% if SV, and pulmonary artery systolic pressure ≥ 45 mm Hg or mean ≥ 17 if SV. The lack of any of these indicators, the presence of one, or the presence of two or more was associated with different expected adverse event rates.

Congenital Heart Adjustment for Risk Method: CHARM

Once the group had created the procedure-type risk categories and hemodynamic vulnerability measurement tools, they sought to create a multivariable model that could adjust for case-mix differences. They produced what is now known as CHARM/Congenital Heart Adjustment for Risk Method. Using 75% of the data collected between August 2007 and December 2009, a multivariable logistic regression model was developed for the outcome of high-severity adverse events. The model was then assessed for validation with the remaining 25% of data. The final CHARM model identified three independent variables associated with risk: procedure-type risk category, number of hemodynamic indicators, and age < one year. Using the CHARM methodology, institutions can compute the predicted probability of an adverse event (AE) for their case population. The sum of all predicted probabilities gives the expected AE rate, taking into consideration the case-mix distribution within the data set. To make a comparison between institutions or physicians at one institution, the standardized adverse event ratio (SAER) must be calculated. To compute the SAER, the observed AE rate (the number of level 3/4/5 AEs in the dataset divided by total number of cases) is divided by the expected AE rate (the expected number of AE occurrences divided by total number of cases). This can be graphically displayed as shown in Figure 1.8

Figure 1.

Sample standardized adverse event ratio (SAER) data.

Major Conclusions

Despite the complexity of modern patients and procedures, data from the C3PO database showed that in contemporary congenital cardiac catheterization and intervention, 2.1% of procedures are associated with life-threatening events, yet mortality remains uncommon at 0.28%. Clinically important high-severity adverse events (level 3, 4, or 5) occurred in 1 out of every 20 cases recorded in the multicenter dataset. There was a significant association between the type of procedure and risk for an adverse event. The group has reported on procedural outcomes for the following procedures: heart biopsy in transplant patients, hybrid procedures, pulmonary valve dilation, pulmonary artery dilation and stenting, and patent ductus arteriosus (PDA) closure.4, 9 Currently under review are additional procedure types such as atrial septal defect (ASD) closure and aortic valvuloplasty. In addition, the group has explored special characteristics in patient populations such as the adult with congenital heart disease and low birth-weight infants.10, 11 We hope to be able to provide important insight regarding the use of sedation compared to anesthesia while performing various procedure types. Finally, the robust data set supported the first analysis in this discipline of operator experience and the impact on outcomes.12

Future Directions

We anticipated a broad range of uses for the procedure-type risk categories. Registries, such as C3PO, allow institutions to track trends in provider, hospital, or national populations undergoing catheterization for congenital heart disease. Knowing which procedures and populations have the greatest risk allows procedural teams to prepare for rescue procedures such as surgical bailout or mechanical support.

Based on the success of C3PO, the group is pursuing a follow-up project, Congenital Cardiac Catheterizations Project on Outcomes – Quality Improvement (C3PO-QI); the study is aimed at improving the outcomes for all congenital heart disease patients undergoing catheterization by reducing radiation exposure and has expanded participation to include 15 sites collecting prospective data. For this effort, a new, more nimble web-based tool was constructed that features improved speed, functionality and updated nomenclature (Figure 2). Sites will have access to comparison reports between sites in addition to time series analyses and physician performance reports over time. Quality reports will include outcome metrics such as risk-adjusted adverse event ratios using CHARM. In addition, the group will be collecting efficacy information for six different procedures (aortic valvuloplasty, pulmonary valvuloplasty, ASD, PDA closure, coarctation, and transcatheter pulmonary valve [TPV] implant). Longitudinal follow-up of patients undergoing ASD closure with the atrial septal occluder and TPV replacement with the Melody valve will also allow for the study of rare adverse events associated with these devices. The collection of information specific to these procedures will enable a large cohort analysis on trends and anomalies within these specific populations.

Figure 2.

Homepage of the C3PO-QI database website.

The first quality improvement initiative of the 15-site collaboration is to reduce radiation exposure in pediatric catheterization procedures. To this end, the group has developed a key driver diagram to outline the goals and the methods by which to reduce radiation (Figure 3). The group is employing the use of PDSA methodology: 1) propose changes based on theories (Plan), 2) implement the change (Do), 3) measure or describe the effect (Study), 4) review and upgrade the process based on what is learned (Act). Using this quality improvement methodology, the multidisciplinary team participates in biweekly conference calls to assess the current status of the project and devise a plan of action for radiation reduction, which is enacted by the following meeting. Sites will share best practices to accelerate improvement within the collaborative. This iterative process will allow for improvement in chosen radiation exposure measures over time.

Figure 3.

C3PO-QI Key Driver Diagram.

Over the last decade, there have been significant advances in available tools to record and report safety and efficacy data of catheterization procedures for pediatric and adult patients with congenital heart disease. In addition to C3PO-QI, other registries are developing methods for analyzing care that will allow institutions and providers multiple platforms to use for quality improvement purposes. As interventional cardiology is a subspecialty that deals mostly with rare diseases, it is important that these initiatives lend themselves to multi-institutional collaboration and the practice of pooling data for research and quality improvement purposes. Larger registries such as IMPACT have started the process of standardizing nomenclature across the field, a prerequisite for successful multi-institutional collaboration. C3PO was able to successfully do this within its smaller, more-exclusive registry, making possible a seamless progression to C3PO-QI and other quality improvement initiatives.

Funding Statement

Funding/Support: The authors have received grant funding for the C3PO project from The Children’s Heart Foundation and the American Heart Association.