Procedural Risk in Congenital Cardiac Catheterization (PREDIC³T)

Abstract

Background

Advancements in the field, including novel procedures and multiple interventions, require an updated approach to accurately assess patient risk. This study aims to modernize patient hemodynamic and procedural risk classification through the creation of risk assessment tools to be used in congenital cardiac catheterization.

Methods and Results

Data were collected for all cases performed at sites participating in the C3PO (Congenital Cardiac Catheterization Project on Outcomes) multicenter registry. Between January 2014 and December 2017, 23 119 cases were recorded in 13 participating institutions, of which 88% of patients were <18 years of age and 25% <1 year of age; a high‐severity adverse event occurred in 1193 (5.2%). Case types were defined by procedure(s) performed and grouped on the basis of association with the outcome, high‐severity adverse event. Thirty‐four unique case types were determined and stratified into 6 risk categories. Six hemodynamic indicator variables were empirically assessed, and a novel hemodynamic vulnerability score was determined by the frequency of high‐severity adverse events. In a multivariable model, case‐type risk category (odds ratios for category: 0=0.46, 1=1.00, 2=1.40, 3=2.68, 4=3.64, and 5=5.25; all P≤0.005) and hemodynamic vulnerability score (odds ratio for score: 0=1.00, 1=1.27, 2=1.89, and ≥3=2.03; all P≤0.006) remained independent predictors of patient risk.

Conclusions

These case‐type risk categories and the weighted hemodynamic vulnerability score both serve as independent predictors of patient risk for high‐severity adverse events. This contemporary procedure‐type risk metric and weighted hemodynamic vulnerability score will improve our understanding of patient and procedural outcomes.

Nonstandard Abbreviations and Acronyms

AE

adverse event

C3PO

Congenital Cardiac Catheterization Project on Outcomes

CCISC

Congenital Cardiac Interventional Study Consortium

CRISP

Catheterization Risk Score for Pediatrics

HSAE

high‐severity adverse event

IMPACT

Improving Pediatric and Adult Congenital Treatment

Clinical Perspective

What Is New?

• Current congenital cardiac catheterization procedure categories classify catheterizations by the highest‐risk intervention performed, which does not account for added complexity when multiple interventions are performed; in this analysis, case types were created to summarize all interventions performed during a catheterization.

• The introduction of novel interventions including transcatheter pulmonary valve replacement and ductal stenting has necessitated updated case risk classification in congenital cardiac catheterization.

• Hemodynamic risk has previously been determined by the presence of an abnormal hemodynamic indicator; this study builds upon improved understanding of risk, developing a hemodynamic vulnerability score where hemodynamic indicators are weighted by associated adverse event rates.

What Are the Clinical Implications?

• The new contemporary case‐type risk categories and the weighted hemodynamic vulnerability score will be important tools that can be used to develop methodologies for risk prediction, resource planning, and operator and institutional case mix comparison as well as revised and improved risk adjustment methodology.

Congenital cardiac catheterization is a constantly evolving field with novel interventions and new technology continuously introduced into standard practice. In these cases, the broad range of heterogenous interventions, some infrequently performed, can limit meaningful comparisons.

To accurately adjust for case mix complexity, in 2011, the Congenital Cardiac Catheterization Project on Outcomes (C3PO), created 4 procedure‐type risk categories to classify cases with similar expected risk for clinically relevant adverse events (AEs). ¹ Other procedure risk category methodologies have also been developed by the CCISC (Congenital Cardiac Interventional Study Consortium) and the IMPACT (Improving Pediatric and Adult Congenital Treatment) registries. The CCISC model, Catheterization Risk Score for Pediatrics (CRISP), which was developed in 2015, uses only 3 categories of procedure risk in their model, with 78% of cases in category 1, reducing the ability to differentiate risk among the 34 procedure types captured as category 1. ² Although the IMPACT procedure risk categories were broadened in 2017 to 6 categories and included some novel procedures, the historical data set spanned January 2011 to March 2014, predating significant data collection of novel procedure outcomes. ³

Over the past decade, there have been numerous advancements in the field of congenital cardiac catheterization. The original C3PO procedure type risk groups do not account for advancements in technology and technique and the introduction of new interventions. Like the original C3PO categories, the CRISP categories, developed in 2015, do not account for novel interventions, such as transcatheter pulmonary valve replacement. Additionally, none of the existing procedure risk categories account changes in case‐level risk when >1 intervention is performed, and instead classify a case only at the level of the highest individual intervention performed. Furthermore, the field has developed a broader understanding of patient and procedural determinants of risk, such as the relative importance of different hemodynamic indicators. ² , ³ , ⁴ , ⁵ , ⁶ , ⁷ Previous consideration of hemodynamic risk used in C3PO, CRISP, and IMPACT assign equal significance to all abnormal hemodynamic variables.

Given the evolution of congenital cardiac catheterization with the introduction of novel procedures and improved understanding of risk factors, contemporary risk assessment tools are warranted to allow for accurate AE outcome reporting for institutions performing these cases. The C3PO collaborative has expanded to include 20 institutions in the United States with a wide range of institution profiles, creating a rich data source for developing of novel outcome assessment tools to further congenital cardiac catheterization. ⁸ , ⁹ , ¹⁰ , ¹¹ , ¹² The aim of this study was to improve upon and modernize available tools for assessment of procedural risk through the development of new case‐type risk category designations and the addition of an improved measure for hemodynamic vulnerability using a modern C3PO data set.

Methods

Data Source

Centers participating in the C3PO collaborative prospectively collected data on each pediatric or congenital cardiac catheterization case performed between January 1, 2014, and January 1, 2018. Institutional review board approval for this study was obtained at the sponsor site, Boston Children’s Hospital, and sought at local institutions in accordance with institutional requirements. Informed consent requirement was waived by the institutional review board given the quality improvement nature of the work. The data underlying this article will be shared on reasonable request to the corresponding author.

Data Collection

Patient characteristics included age, sex, ventricular circulation physiology (single‐ventricle/biventricular circulation), any diagnosis of a genetic syndrome (yes/no) or an associated significant noncardiac comorbidity active at the time of the catheterization procedure (yes/no) with further specification for coagulation disorder, chronic lung disease, renal insufficiency, or other. Definitions for genetic syndrome and noncardiac comorbidity were available in the C3PO User Manual. If a patient had a recent cardiac catheterization or surgery, the date of the procedure and procedure type were recorded.

Procedure baseline hemodynamic indicator variables collected included saturation measurements (systemic arterial saturation and mixed venous saturation), intracardiac and intravascular pressure measurements (end‐diastolic pressure and pulmonary artery pressure), and calculated values of Qp:Qs and pulmonary vascular resistance.

Catheterizations were broadly captured as diagnostic only, biopsy in patients following heart transplant, or interventional. Using established nomenclature of the International Pediatric and Congenital Cardiac Code, ¹³ each intervention performed as part of the procedure was recorded. Case types were used to summarize all interventions performed during a catheterization and defined on the basis of the primary and secondary intervention(s) performed to account for the increase in case complexity when multiple interventions are performed. Primary interventions were the most significant or intended intervention. Secondary interventions mostly included pulmonary artery angioplasty with or without stenting, systemic arterial or venous stenting, or aortopulmonary or venovenous collateral closure. Case types were further stratified on the basis of patient age to account for the significant differences in anticipated procedure risk profiles. Approximately 9% of cases did not meet a case‐type definition and were not included because of the infrequency of these events, limiting the ability to accurately assign a risk category. Given the limited number of these cases and/or the heterogeneity in outcomes among procedures with the same recorded intervention, these cases were excluded from further analysis.

Primary Outcome: Clinically Significant Adverse Events

The primary outcome used to determine patient outcome was the occurrence of a clinically significant adverse event, or high‐severity adverse event (HSAE), defined as a severity level 3, 4, or 5 event using established definitions for reporting of procedural complications, in harmony with the International Pediatric and Congenital Cardiac Code. ¹³ , ¹⁴ Adverse events were defined as any anticipated or unanticipated event for which patient harm could have or did occur, potentially or definitely as a consequence of the procedure performed, and assigned a graded severity from 1 to 5 according to previously established definitions (Table 1). ¹ , ¹⁵ , ¹⁶

Table 1.

Definitions for Adverse Event Severity

Severity level	Definition	Examples
Level 1: none	No harm, no change in condition, may have required monitoring to assess for potential change in condition with no intervention indicated	Balloon rupture Equipment problem
Level 2: minor	Transient change in condition, not life threatening, condition returns to baseline, required monitoring, required minor intervention such as holding a medication, or obtaining laboratory test	Groin hematoma Self‐resolving arrhythmia
Level 3: moderate	Transient change in condition may be life threatening if not treated, condition returns to baseline, required monitoring, required intervention such as reversal agent, additional medication, transfer to the intensive care unit for monitoring, or moderate transcatheter intervention to correct condition	Unstable arrhythmia with preserved blood pressure requiring intervention Vascular damage not life threatening but requiring intervention
Level 4: major	Change in the patient’s clinical condition that would be life threatening if not treated and require intense medical therapy and/or major invasive transcatheter or urgent/emergent surgical intervention to treat the condition. These conditions may also result in the need for unplanned cardiopulmonary support in the form of heart‐lung bypass (extracorporeal membrane oxygenation) to prevent a catastrophic event from occurring	Major life‐threatening vascular injury that results in cardiopulmonary collapse, need for urgent blood product administration, and/or requires a major invasive procedure to successfully treat the condition Any event requiring cardiopulmonary resuscitation Emergent surgical intervention because of device or stent embolization; and unanticipated intubation or need for cardiopulmonary support in the setting of circulatory collapse or acute respiratory failure
Level 5: catastrophic	Any death, and emergent surgery or heart‐lung bypass support (extracorporeal membrane oxygenation) to prevent death with failure to wean from bypass support	Event resulting in death

All adverse events were independently reviewed by 2 fellowship‐trained pediatric interventional cardiologists to ensure accuracy in AE reporting among institutions. Misapplication of AE severity definitions were appropriately adjusted to ensure standardized reporting of events on the basis of these established definitions.

Statistical Analysis

The outcome “adverse event” was defined at the case level, determined by the highest‐severity AE occurring during a case and summarized for each case type. Continuous variables are summarized as median (range) and categorical variables as number of occurrences (frequency as %) unless otherwise noted. A case‐type variable was created on the basis of the primary and secondary interventions performed. Rates of HSAEs were compared by patient characteristics using the chi‐square test.

Hemodynamic Vulnerability Score

All recorded hemodynamic variables were classified as normal or abnormal on the basis of previously established threshold values, ³ , ¹⁷ then stratified by single versus biventricular circulation. Each indicator variable was analyzed empirically in isolation and in varying permutations with other indicators to assess influence on rates of HSAEs. To assess the significance of each hemodynamic indicator, rates of HSAEs were estimated for cases with 1 or 2 abnormal hemodynamic indicators, and weighted point values were assigned on the basis of these rates. Indicators that did not increase risk beyond that of cases with no hemodynamic abnormality were assigned 0 points, those that moderately increased risk were assigned 1 point, and those that increased risk substantially on the basis of higher AE rate were assigned 2 points. For each case, points were summed for all hemodynamic abnormalities present to create a hemodynamic vulnerability score. Using both empiric analysis of permutations and expert opinion, score categories of 0, 1, 2, and ≥3 were generated on the basis of the composite score. The discrimination of this hemodynamic vulnerability score for predicting HSAEs was assessed using a c‐statistic; a 95% bootstrapped CI for the c‐statistic was calculated on the basis of 1000 replications.

Procedural Risk in Congenital Cardiac Catheterization (PREDIC³T) Case‐Type Risk Categories

Case‐types included in the analysis were organized on the basis of the primary outcome variable (HSAE) and were individually reviewed by the expert panel. Similar case‐type designations were combined if the expected AE rates were similar. As an initial step, k‐means clustering was used to partition the HSAE rates for each case type into 5 groups. Five possible clusters of case types were developed, each with similar discrimination for predicting the outcome (c‐statistics from logistic regression models 0.705–0.713). From these groupings, the expert panel further separated biopsy procedures (with and without coronary angiography) into a unique case‐type risk category “0,” as these are distinctly different types of catheterizations with significant variation in occurrence between centers. The discrimination of the risk categories for predicting HSAEs was quantified using a c‐statistic with a 95% bootstrapped CI based on 1000 replications.

Independence of Predictors

Relationships among patient and procedural characteristics and the outcome any severity level 3, 4, or 5 AEs were assessed using univariate logistic regression models. To assess the independence and strength of the developed categorizations, final case‐type risk categories and hemodynamic vulnerability score were tested in a stepwise‐forward multivariable model. The model was generated starting with the newly developed PREDIC³T case‐type risk categories, followed by the hemodynamic vulnerability score, and other clinical characteristics found to be important in univariate analysis were then tested for inclusion in the model, where P<0.05 was required for retention. Discrimination of the final model was evaluated using the c‐statistic.

Data Audit

In July 2019, all participating centers underwent an independent audit, verifying complete case capture and accuracy of selected data elements in a random sample of 10% of cases (maximum, 50) at each site. Audited data elements included patient age, hemodynamic values, case type, and occurrence of an AE. The audits were executed using videoconferencing to ensure that sites confirmed data using institutional medical records. The provided answers were compared with database extracts. At the conclusion of each audit, sites were asked to confirm total annual case volume to compare with the C3PO registry to verify complete annual case capture.

Results

From January 2014 through December 2017, a total of 23 119 cases met inclusion criteria and were recorded by 13 centers with average annualized case volume ranging from 195 to 1606 cases per year. Pediatric patients ≤18 years of age accounted for the majority of the cohort (88%), with ≈25% being <1 year of age (Table 2). Patients were classified as having single ventricle physiology in 20% of cases recorded. A genetic abnormality was identified in 9%, and the presence of an active noncardiac problem in 19%. Recent cardiac interventions within 90 days, either surgery or catheterization, had occurred in nearly 20% of the population. Of all catheterizations performed, 46% were defined as an interventional case, 23% as a biopsy, and 31% as a diagnostic or noninterventional case.

Table 2.

Patient and Procedural Characteristics

	Entire cohort (N=23 119) N (%) or median [IQR]
Patient characteristics
Age
≤30 d	1537 (7)
>30 d to <1 y	4417 (19)
1 to 18 y	14 492 (63)
>18 y	2673 (12)
Sex, male (n=22 904)	12 590 (55)
Single ventricle	4608 (20)
Genetic syndrome	2153 (9)
Any noncardiac problem	4331 (19)
Coagulation disorder	150 (1)
Chronic lung disease	1291 (6)
Renal insufficiency	473 (2)
Cardiac catheterization in past 90 d	4148 (18)
Cardiac surgery in past 90 d	3160 (14)
Procedural characteristics
Case type
Biopsy (+/− coronary angiography)	5303 (23)
Diagnostic	7137 (31)
Interventional	10 679 (46)
Duration of catheterization, h	1.3 [0.8, 2.0]
Abnormal hemodynamic indicator variables
Low systemic arterial saturation	6502 (28)
BiV: <95%, SV: <78%
Low mixed venous saturation	2934 (13)
BiV: <60%, SV: <50%
High pulmonary artery pressure	3091 (13)
BiV: systolic ≥45 mm Hg; SV: mean ≥17 mm Hg
High systemic ventricle EDp	1149 (5)
≥18 mm Hg
Qp:Qs	2243 (10)
>1.5
Pulmonary vascular resistance	3762 (16)
>3 iWU
Adverse events
Any adverse event	2528 (10.9)
Any level 3/4/5 adverse events	1193 (5.2)
Any level 4/5 adverse events	319 (1.4)
Level 5 adverse events	17 (0.07)

BiV indicates biventricular; EDp, end‐diastolic pressure; iWU, indexed Wood units; and SV, single ventricle.

Adverse Events

Among all cases, an AE was recorded in 10.9% of cases (n=2528), with an HSAE in 5.2% of cases (n=1193), and 17 recorded deaths (level 5) in the cohort (0.07%). Age ≤30 days had a strong association with HSAE with an incidence of 11.1% compared with age >30 days to <1 year, 1 to 18 years, and >18 years at rates of 7.2%, 3.8%, and 5.9%, respectively (P<0.001; Table 3). Broadly, all interventional catheterizations as a group had a higher HSAE rate of 8.1% compared with diagnostic and biopsy catheterizations, 3.9% and 1.1%, respectively (P<0.001). Patients with single ventricle physiology had a higher HSAE rate at 6.5% compared with non–single‐ventricle patients, 4.8% (P<0.001). Similarly, patients who had a recent cardiac surgery within 90 days of the index catheterization had an HSAE incidence of 6.0% compared with those who did not have a recent surgery (5.0%; P=0.017). There was no significant difference in incidence of HSAE in patients with a genetic syndrome or noncardiac problems such as a coagulation disorder and chronic lung disease. The most common events categorized into level 3 AEs and level 4/5 AEs are depicted in Table 4.

Table 3.

Univariate Analysis of Patient and Procedural Characteristics and Association With High Severity Adverse Events

	N (% of total) HSAE	95% CI	P value
Age			<0.001
≤30 d	171 (11.1)	9.6–12.8
>30 d to <1 y	316 (7.2)	6.4–8.0
1 to 18 y	548 (3.8)	3.5–4.1
>18 y	158 (5.9)	5.1–6.9
Sex			0.88
Male	656 (5.2)	4.8–5.6
Female	532 (5.2)	4.7–5.6
Single ventricle			<0.001
Yes	298 (6.5)	5.8–7.2
No	895 (4.8)	4.5–5.2
Genetic syndrome			0.76
Yes	114 (5.3)	4.4–6.3
No	1079 (5.2)	4.9–5.5
Any noncardiac problem			0.013
Yes	191 (4.4)	3.8–5.1
No	1002 (5.3)	5.0–5.7
Coagulation disorder			0.36
Yes	10 (6.7)	3.2–11.9
No	1183 (5.2)	4.9–5.4
Chronic lung disease			0.80
Yes	64 (5.0)	3.8–6.3
No	1129 (5.2)	4.9–5.5
Renal insufficiency			0.027
Yes	14 (3.0)	1.6–4.9
No	1179 (5.2)	4.9–5.5
Cardiac catheterization in past 90 d			0.44
Yes	224 (5.4)	4.7–6.1
No	969 (5.1)	4.8–5.4
Cardiac surgery in past 90 d			0.017
Yes	191 (6.0)	5.2–6.9
No	1002 (5.0)	4.7–5.3
Procedural characteristics			<0.001
Biopsy	56 (1.1)	0.8–1.4
Diagnostic	275 (3.9)	3.4–4.3
Interventional	862 (8.1)	7.6–8.6

HSAE indicates high‐severity adverse event.

Table 4.

Common Adverse Events Summarized by Severity Levels

	N
Level 3 adverse events
Vascular access–related complications including vessel thrombosis, vessel injury, and hemodynamically tolerated retroperitoneal hemorrhage	161
Atrial arrhythmias requiring medical and/or electrical cardioversion	139
Angioplasty‐related complications including vascular tears or vessel injury needing moderate catheterization‐based intervention such as stent placement	111
Device or stent related problem including embolization or malposition	103
Respiratory‐ or anesthesia‐related events including airway obstruction, hypoxia, postoperative stridor, or apnea	81
Catheter‐induced heart block requiring temporary intervention or observation	63
Hypotension requiring medical therapy or volume resuscitation	46
Pulmonary hemorrhage	43
Coil malposition or embolization requiring catheter retrieval or other minor catheterization‐based intervention	26
Ventricular arrhythmia not requiring cardioversion/defibrillation	18
Nonspecific ST‐T wave changes	12
Isolated central nervous system event not resulting in permanent injury	12
New valvar regurgitation not resulting in hemodynamic instability or requiring surgical intervention	12
Pulmonary edema and/or reperfusion injury	11
Bradycardia	10
Level 4 and 5 adverse events
Respiratory‐ or anesthesia‐related event resulting in clinical decompensation and needing active resuscitation	32
Ventricular arrhythmia needing resuscitation or cardioversion/defibrillation	30
Cardiac arrest requiring cardiopulmonary resuscitation or extracorporeal membrane oxygenation	29
Device or stent embolization/malposition resulting in hemodynamic compromise and/or necessitating surgical repair	29
Vascular access related complications or vessel injuries which are deemed life threatening and/or requiring surgical intervention	23
Heart block requiring cardiopulmonary resuscitation or requiring placement of a permanent pacing device	23
Angioplasty‐related complications resulting in significant vascular injury or hemodynamic instability	18
Heart perforation	17
Hypotension or depressed cardiac output deemed life threatening and requiring resuscitation	10
Central nervous system event resulting in stroke or permanent disability	7
Bradycardia deemed life threatening and requiring resuscitation	6
Pulmonary hemorrhage deemed life threatening	5

Hemodynamic Vulnerability Score

Hemodynamic indicators, stratified by single‐ or biventricular circulation, varied in their association with the outcome HSAE, ranging from 3.1% in patients with abnormal pulmonary vascular resistance and as high as 11.6% in all patients with high pulmonary artery pressure, with or without elevated vascular resistance (Table 5). Given this degree of heterogeneity in the strength of each hemodynamic variable, a hemodynamic vulnerability score was generated (0–6) on the basis of respective association with the outcome HSAE. Two points were assigned to the indicator variables with the highest HSAE rates: low systemic arterial saturation for single‐ventricle circulation and high pulmonary artery pressure for both single‐ and biventricular circulations. More moderate risk indicators were given a value of 1 point, including low systemic arterial saturation in biventricular circulation, low mixed venous saturation, high systemic ventricle end‐diastolic pressure, and high Qp:Qs. Abnormal pulmonary vascular resistance was not associated with increased HSAEs. Hemodynamic scores were generated for each case in the cohort and grouped according to a score of 0, 1, 2, and ≥3 with respective frequencies of HSAEs of 3.4%, 5%, 8.7%, and 9.5% (P<0.001; Table 6).

Table 5.

Hemodynamic Indicator Variables by Presence of HSAE

Hemodynamic indicator variables	Presence of HSAE,* n (%)	Weighted score value (0–2)
Low systemic arterial saturation
BiV (<95%)	107/2154 (5.0)	1
SV (<78%)	43/479 (9.0)	2
Low mixed venous saturation
BiV (<60%)	18/303 (5.9)	1
SV (<50%)	4/72 (5.6)	1
High pulmonary artery pressure
BiV (≥45 mm Hg)	55/495 (11.1)	2
SV (mean ≥17 mm Hg)	24/189 (12.7)	2
High systemic ventricle EDp (≥18 mm Hg)	20/431 (4.6)	1
High Qp:Qs (>1.5)	46/1089 (4.2)	1
High PVR (>3 iWU)	35/1140 (3.1)	0

BiV indicates biventricular; EDp, end‐diastolic pressure; HSAE, high‐severity adverse event; iWU, indexed Wood units; PVR, pulmonary vascular resistance; and SV, single ventricle.

^*The percent HSAE listed for each hemodynamic indicator variable in Table 5 includes only cases with a single independent abnormal indicator variable.

Table 6.

Hemodynamic Vulnerability Score

Hemodynamic vulnerability score	Presence of HSAE, n (%)	95% CI	P value
0	432/12 628 (3.4)	3.1–3.8	<0.001
1	234/4686 (5)	4.4–5.7
2	274/3135 (8.7)	7.8–9.8
≥3	253/2670 (9.5)	8.4–10.7

HSAE indicates high‐severity adverse event.

C‐statistic 0.619, 95% bootstrapped CI (0.604–0.635).

PREDIC³T Case‐Type Risk Categories

Case types with similar interventions and HSAE rates or case types in which additional interventions did not increase HSAE rates were combined into a single case type (Table 7). Thirty of 34 case types individually comprised ≤5% of the total cohort, 19 of which individually accounted for ≤1% of the cohort (Table 8). However, these 30 case types combined together accounted for 48.5% of the total cohort and 100% of all interventional cases. The frequency of reported HSAEs among all case types ranged from 0 recorded events out of 76 “Fontan fenestration or baffle leak device closure” cases to as high as 14 of 69 cases (20.3%) among “atretic valve perforation with or without valvotomy” cases. Differential HSAE rates for diagnostic cases stratified by age (≤30 days, >30 days to <1 year, and ≥1 year) range from 2.8% to 9%. Six final case‐type risk categories were created with HSAE rates of 1.1%, 2.7%, 4.2%, 7.7%, 10.8%, and 13.9% for categories 0 to 5, respectively (Table 8), with a univariate c‐statistic of 0.72.

Table 7.

PREDIC³T Case‐Type Risk Categories

	Risk category 0	Risk category 1	Risk category 2	Risk category 3	Risk category 4	Risk category 5
Diagnostic case		Diagnostic ≥1 y	Diagnostic 1 mo to <1 y	Diagnostic ≤30 d
Valvuloplasty		Pulmonary valvuloplasty >30 d	Pulmonary valvuloplasty+procedure >30 d	Pulmonary valvuloplasty +/− procedure ≤30 d		Aortic valvuloplasty +/− procedure, ≤30 d Aortic valvuloplasty +/− procedure, >30 d Mitral valvuloplasty Atretic valve perforation with or without valvuloplasty
Device or coil closure		Fontan fenestration or baffle leak device closure	ASD or PFO closure Venous collateral occlusion PDA closure	Fontan fenestration or baffle leak device closure+procedure Systemic pulmonary collateral closure +/− procedure	ASD or PFO closure+procedure	VSD closure
Balloon angioplasty and/or stent placement				Pulmonary artery (1 vessel) Pulmonary artery (1 vessel)+RVOT conduit dilation/stent Aorta (coarctation) dilation and/or stent	Pulmonary artery (1 vessel)+procedure Pulmonary artery (≥2 vessels) Pulmonary vein dilation and/or stent RVOT conduit dilation and/or stent PDA dilation and/or stent	Pulmonary artery (≥2 vessels)+RVOT +/− procedure Aorta (coarctation) dilation and/or stent+procedure
Other	Endomyocardial biopsy Endomyocardial biopsy with coronary angiography			Atrial septostomy		Atrial septostomy+procedure TPV implantation +/− procedure Atrial septum static dilation and/or stent placement

ASD indicates atrial septal defect; PDA, patent ductus arteriosus; PFO, patent foramen ovale; PREDIC³T, procedural risk in congenital cardiac catheterization; RVOT, right ventricular outflow tract; TPV, transcatheter pulmonary valve; and VSD, ventricular septal defect

C‐statistic 0.718, 95% bootstrapped CI (0.705–0.732).

Table 8.

Case‐Types by Frequency of HSAE

	Number of cases in cohort N (%)	Frequency of HSAE N (%)
Risk category 0	5303 (23)	56 (1.1)
Endomyocardial biopsy	3190 (14)	17 (0.5)
Endomyocardial biopsy with coronary angiography	2113 (9)	39 (1.9)
Risk category 1	5392 (23)	147 (2.7)
Fontan fenestration or baffle leak device closure	76 (<1)	0 (0.0)
Pulmonary valvuloplasty, age >30 d	463 (2)	10 (2.2)
Diagnostic only, age ≥1 y	4853 (21)	137 (2.8)
Risk category 2	4362 (19)	184 (4.2)
Pulmonary valvuloplasty+procedure, age >30 d	62 (<1)	2 (3.2)
ASD or PFO device closure	944 (4)	31 (3.3)
Venous collateral device or coil occlusion	416 (2)	15 (3.6)
PDA device or coil closure	1189 (5)	46 (3.9)
Diagnostic only, age >30 d to <1 y	1751 (8)	90 (5.1)
Risk category 3	3909 (17)	302 (7.7)
Pulmonary artery dilation and/or stent (only 1 vessel)	1006 (4)	69 (6.9)
Fontan fenestration or baffle leak device closure+procedure	84 (<1)	6 (7.1)
Aorta (coarctation) dilation and/or stent	564 (2)	42 (7.5)
Systemic pulmonary collateral device or coil closure +/− procedure	1024 (4)	77 (7.5)
Pulmonary valvuloplasty +/− procedure age, age ≤30 d	246 (1)	21 (8.5)
Pulmonary artery dilation and/or stent (only 1 vessel)+RVOT conduit dilation and/or stent	129 (1)	11 (8.5)
Atrial septostomy	323 (1)	28 (8.7)
Diagnostic only, age ≤30 d	533 (2)	48 (9.0)
Risk category 4	2373 (10)	257 (10.8)
Pulmonary artery dilation and/or stent (only 1 vessel)+procedure	176 (1)	18 (10.2)
ASD or PFO device closure+procedure	39 (<1)	4 (10.3)
Pulmonary vein dilation and/or stent	660 (3)	70 (10.6)
Pulmonary artery dilation and/or stent (≥2 vessels)	903 (4)	98 (10.9)
RVOT conduit dilation and/or stent	409 (2)	46 (11.3)
PDA dilation and/or stent	186 (1)	21 (11.3)
Risk category 5	1780 (8)	247 (13.9)
Aorta (coarctation) dilation and/or stent+procedure	247 (1)	30 (12.2)
Aortic valvuloplasty +/− procedure, age >30 d	226 (1)	29 (12.8)
Aortic valvuloplasty +/− procedure age ≤30 d	100 (<1)	13 (13.0)
Pulmonary artery dilation and/or stent (≥2 vessels)+RVOT and/or other procedure	212 (1)	28 (13.2)
VSD device closure	45 (<1)	6 (13.3)
Mitral valvuloplasty	75 (<1)	10 (13.3)
Atrial septostomy+procedure	72 (<1)	10 (13.9)
TPV implantation +/− procedure	679 (3)	98 (14.4)
Atrial septum static dilation and/or stent placement	55 (<1)	9 (16.4)
Atretic valve perforation +/− valvuloplasty	69 (<1)	14 (20.3)

ASD indicates atrial septal defect; HSAE, high‐severity adverse event; PDA, patent ductus arteriosus; PFO, patent foramen ovale; RVOT, right ventricular outflow tract; TPV, transcatheter pulmonary valve; and VSD, ventricular septal defect.

Multivariate Analysis of New Tools

PREDIC³T case‐type risk categories, an important feature in univariate analysis, remained an independent predictor of the primary outcome, HSAE, when tested in a stepwise‐forward modeling with the hemodynamic vulnerability score. Further addition of age categories added additional explanatory information about the risk of these important outcomes (c‐statistic of 0.74; Table 9). Other patient and procedural characteristics were not statistically significant when added to this 3‐feature model, indicating independent significance of the PREDIC³T case‐type risk categories and the hemodynamic vulnerability score. Important variables included in the 3‐feature model and their respective frequencies of HSAEs in relationship to categorical volume are depicted in the Figure.

Table 9.

Multivariate Analysis of Predictors for Outcome High‐Severity (Level 3/4/5) Adverse Events

	OR (95% CI)	P value
PREDIC3T case type risk category
0	0.46 (0.33–0.62)	<0.001
1	1.00	…
2	1.40 (1.11–1.78)	0.005
3	2.68 (2.16–3.32)	<0.001
4	3.64 (2.93–4.52)	<0.001
5	5.25 (4.23–6.53)	<0.001
Hemodynamic vulnerability score
0	1.00	…
1	1.27 (1.07–1.50)	0.006
2	1.89 (1.60–2.23)	<0.001
≥3	2.03 (1.71–2.42)	<0.001
Age
<1 mo	1.47 (1.21–1.79)	<0.001
1–11 mo	1.18 (1.01–1.39)	0.041
1–18 y	1.00	…
≥19 y	1.51 (1.25–1.82)	<0.001

c‐statistic 0.74. OR indicates odds ratio; and PREDIC³T, procedural risk in congenital cardiac catheterization.

Figure 1. Volume by risk feature and associated high‐severity adverse event rates (severity 3/4/5).

HSAE indicates high‐severity adverse event; and PREDIC³T, procedural risk in congenital cardiac catheterization.

Data Audit

All 13 centers participated in the audit, with a total of 650 cases randomly selected, not limited to cases with an AE. Case ascertainment and database recording was verified by matching case volume to institutional records with 97% to 99% agreement among the institutions. In these 650 cases, patient and procedural characteristics in the model were audited for accuracy in reporting. For case type and age, there was 100% agreement in the audited data set across centers. Among the 3900 hemodynamic indicator variables audited (6 per case), 57 were recorded incorrectly in a lower risk category, and 34 were recorded incorrectly in a higher risk category compared with the audit results. Thus, there was 97% (3809/3900) agreement in reported versus source document audited data.

There was 96% accuracy in reporting 26 of 27 HSAEs, of which 6 AEs were designated as severity level 4/5. The single event not recorded in the database was a level 4 AE related to respiratory arrest in the recovery room following catheterization.

Discussion

Using a robust multicenter data set with >20 000 prospectively gathered congenital catheterization cases, we were able to build upon previous methodology for risk assessment in congenital cardiac catheterization to meet the needs of an evolving and rapidly advancing field by modernizing procedural risk classification and developing a more robust predictive hemodynamic vulnerability scoring system. We have developed 6 procedural risk categories that identify patient risk at the case level rather than a focus on the individual interventions performed. Furthermore, the hemodynamic vulnerability scoring metric adds weighted value to each hemodynamic variable based on their relative strength at predicting the outcome, making this metric a more meaningful addition to further risk adjustment modeling. The case‐type risk categories and hemodynamic vulnerability score developed from this multi‐institutional data set provide contemporary tools for outcome assessment, procedure planning, and risk adjustment.

Congenital cardiac catheterization cases are often heterogeneous procedures, especially when infrequently performed interventions occur in groupings with other interventions. The case types developed in this study improve upon previous intervention type classification schema ¹ , ³ and aim to identify HSAEs at the case level, yielding improved catheterization risk assessment by grouping like cases to maximize the strength of outcome analysis. Previously established methodology has been used to identify events at the intervention level, establishing case‐level risk on the basis of the single most common and/or highest‐risk intervention performed. ¹ , ³ , ⁵ , ¹⁵ , ¹⁶ , ¹⁷ However, this does not account for the added risk of performing multiple interventions, which is a significant improvement this model brings to the field or account for the novel interventions introduced over the past decade. The Catheterization for Congenital Heart Disease Adjustment for Risk Method procedure‐type risk categories report a development cohort c‐statistic of 0.72, yet when the Catheterization for Congenital Heart Disease Adjustment for Risk Method categories are applied to this data set, the c‐statistic is 0.64. Additionally, by stratifying biopsy catheterizations into a unique risk category, centers that perform a disproportionate number of cardiac transplantation procedures will be more accurately assessed in future risk adjustment work. The PREDIC³T case‐type risk categories, with a univariate c‐statistic of 0.72, offer a generalizable tool that can be used to provide more meaningful outcome analysis and serve to more accurately reflect case mix complexity in a modern era of congenital interventional cardiology.

Hemodynamic vulnerability has previously been shown as a strong independent determinant of risk for patients undergoing congenital cardiac cathterization ³ , ⁵ , ¹⁷ , ¹⁸ and remains an important consideration when assessing patient risk. These prior efforts have successfully identified important hemodynamic parameters, for both single‐ and biventricular circulations, which are associated with higher risk for having a clinically important AE. ³ , ⁵ , ¹⁶ , ¹⁷ This new methodology evolves the way hemodynamic vulnerability is scored by considering the relative risk of each individual hemodynamic variable and assigns a unique score calculating a cumulative risk score that more accurately reflects patient vulnerability.

The developed PREDIC³T case‐type risk categories and hemodynamic vulnerability score will have a broad range of future applications and can be used to standardize outcome reporting and population comparisons among institutions and providers by defining case risk complexity. Additionally, it is important for institutions and operators to have a metric with which they can track population trends over time, highlighting changes in patient complexity and observed outcomes. Making these analyses with a readily available analytic tool grouping sometimes infrequently performed cases with cases of similar risk will allow for targeted risk mitigation strategies based on predicted risk complexity.

These risk categories can also assist in the development of novel outcome‐based research such as current work being done to identify risk factors for failure to rescue. ¹⁹ In the past decade, we have identified patient and procedural risk factors for experiencing an AE. However, we have yet to identify which events lead to patient harm or permanent injury, and when harm is attributable to a failure to rescue the patient from an AE. In the future, these risk categories can be paired with a metric that measures a change in a patient’s clinical status over the course of a cardiac catheterization to better understand the relationship between the case type and a change in the patient’s clinical status, evaluating both improvements as well as identifying harm. Ultimately, identification of system‐level factors and programmatic quality, rather than just individual patient‐level factors, will enable innovative strategies to mitigate risk and reduce both the occurrence of an AE and/or resulting harm, while improving clinical outcomes.

Study Limitations

Certain limitations should be considered when interpreting these findings. While C3PO is open to all institutions performing congenital cardiac catheterization, the registry does not include all centers performing cases, and outcomes may differ at nonparticipating centers. In addition, no centers with an annual volume of <200 catheterizations met inclusion criteria for this data set. C3PO is also based in the United States, which may limit the interpretability in international or low‐resource settings.

Additionally, the primary outcome for this study was HSAEs, which includes severity levels 3, 4, and 5. Based on our data audit, there is high reliability across all centers in the accuracy of AE outcome reporting, particularly of the higher severity level 4/5 AE classifications. However, there may be variation in outcome reporting for lower severity level 3 AEs, which are not always life‐threatening events. Furthermore, the 3‐feature model used here was not tested against a validation data set and requires further model development. Another important factor to consider, particularly when comparing outcomes among sites and operators, is efficacy and how it relates to safety events. Further studies should focus on identifying efficacy measures for these various case types to identify when procedural success was achieved to carefully understand the balance between accepted risk and effective procedural outcome.

Conclusions

The refinement in procedural risk assessment developed in this robust multicenter study is an important step toward the development of a modernized risk adjustment methodology to allow accurate comparison of outcomes among institutions and operators performing congenital cardiac catheterization. We have created novel PREDIC³T case‐type risk categories by stratifying case types with similar HSAE frequency. Defining procedural risk at the case level offers an enhanced ability to capture the overall complexity of each unique case rather than attributing risk to the highest risk intervention performed. Furthermore, with a better understanding of hemodynamic vulnerability we have provided an improved methodology to account for cumulative risk in patients with abnormal hemodynamic measurements, and therefore more accurately define a patient’s inherent risk to the case type being performed. We anticipate that these improvements in risk assessment tools along with the novel PREDIC³T case‐type risk categories will allow for a generalizable process to evaluate and compare outcomes, plan for appropriate resources, and predict risk in congenital cardiac catheterization.

Sources of Funding

This work was supported by the Children’s Heart Foundation.

Disclosures

None.