Variations in practice patterns and consistency with published guidelines for balloon aortic and pulmonary valvuloplasty: An analysis of data from the IMPACT® Registry

Abstract

Objectives:

To study variation in the practice of balloon aortic (BAV) and pulmonary valvuloplasty (BPV).

Background:

The IMPACT® registry provides an opportunity to study practice variation in transcatheter interventions for congenital heart disease.

Methods:

We studied BAV and BPV in IMPACT® from 1/1/2011 to 9/30/2015, using hierarchical multivariable models to measure hospital-level variation in 1) the distribution of indications for intervention; and 2) in cases with “high resting gradient” as the indication, consistency with published guidelines.

Results:

1,071 BAV cases at 60 hospitals and 2,207 BPV cases at 75 hospitals were included. The indication for BAV was high resting gradient in 82%, abnormal stress test/electrocardiogram (2%), LV dysfunction (11%), and symptoms (5%). Indications for BPV were high resting gradient in 82%, right-left shunt (6%), RV dysfunction (7%), and symptoms (5%). No association between hospital characteristics and distribution of indications was demonstrated. Among interventions performed for “high resting gradient”, there was significant adjusted hospital-level variation in the rates of cases performed consistently with guidelines. For BAV, significant differences were seen across census regions, with hospitals in the East and South more likely to practice consistently than those in the Midwest and West (p=0.005). For BPV, no association was found between hospital factors and rates of consistent practice, but there was significant inter-hospital variation (median rate ratio: 1.4 (95% CI: 1.2 – 1.6), p<0.001).

Conclusions:

There is measurable hospital-level variation in the practice of BAV and BPV. Further research is necessary to determine whether this affects outcomes or resource utilization.

CONDENSED ABSTRACT

We used the IMPACT® registry to study practice variation in the conduct of balloon aortic and pulmonary valvuloplasty in the congenital cardiac catheterization laboratory. Overall, there was substantial variation in the stated indications for intervention as well as the frequency of practice consistent with published guidelines. There were important center-level factors associated with practice variation independent of differences in patient-level factors. For balloon aortic valvuloplasty, significant practice variation occurred among geographic regions. For balloon pulmonary valvuloplasty, significant practice variation occurred among individual centers. These findings may have implications on the quality of care delivered in the pediatric cardiac catheterization laboratory.

INTRODUCTION

Practice variation is a novel outcome measure in the study of congenital heart disease and interventional cardiology, measuring the range of practice patterns across a set of hospitals and has become an important measure of quality across many fields of medicine. Reducing variability in practice has been shown to improve both resource utilization and traditional patient outcomes for adult cardiac patients in both inpatient(1, 2) and outpatient(3–8) settings, and pediatric patients with congenital heart disease(9).

In the field of pediatric or congenital interventional cardiology, practice guidelines have been published(10). Although practice variability has been described among a smaller number of centers for specific types of congenital cardiac catheterization procedures(11, 12), no studies have attempted to quantify community-level practice variation for more common interventions in this field. An obstacle to this kind of study has been the dearth of standardized data about individual procedures from a representative national sample of hospitals. The Improving Adult and Congenital Treatment® (IMPACT®) Registry, a multicenter clinical registry of catheterization procedures performed in the United States provides an opportunity to address these questions. We sought to measure variation in the practice of balloon aortic (BAV) and pulmonary valvuloplasty (BPV). These are well-established procedures and were two of the six “core procedures” identified in the initial version of the IMPACT® registry(13). As such, one might reasonably expect there to be relatively stable practice patterns between hospitals. However, we suspected that there might still be significant practice variation and sought to study this with a retrospective multicenter study.

We attempted to measured practice variation at the hospital level in two ways. First, we determined if there was variation in the distribution of stated indications for performing the interventions. Second, because the vast majority of cases of BAV and BPV are performed due to the indication of “high resting gradient,” we determined whether there was variation in consistency with published guidelines. This was accomplished by using multivariable models that adjusted for the effect of patient and procedural characteristics and allowed us to assess 1) whether hospital characteristics appeared to influence outcomes of interest; and 2) whether there was otherwise unaccounted for individual inter-hospital variation, either of which would reflect practice variation with the potential to affect clinical outcomes and healthcare resource utilization.

METHODS

Data source:

The IMPACT^® Registry is a clinical registry funded by the American College of Cardiology and managed by the National Cardiovascular Data Registry (NCDR) with data from pediatric and general hospitals performing cardiac catheterizations in children and adults with congenital heart disease(13–15). At the time of this analysis, there were 87 centers contributing data to the database distributed across the United States (Figure 1). Data are recorded using standardized data elements and definitions. The database is subject to the same rigorous quality assurance standards to which other NCDR registries are held(16), although full auditing procedures were still being developed during the timeframe of this study(16, 17). The current study used data from IMPACT v1.0.1. The project utilizes de-identified data and does not represent human subjects research in accordance with the Common Rule (45 CFR 46.102(f)).

Figure 1: The IMProving Adult and Congenital Treatment (IMPACT®) Registry.

The map demonstrates the location of 87 centers submitting data to the IMPACT® registry at the time of the study. Also shown are the divisions of the United States into four census regions (Northeast, South, Midwest, and West).

Study population and study procedures:

Subjects of all ages who underwent an isolated BAV or BPV recorded in IMPACT^® between January 1, 2011 and September 30, 2015 were included in the study. In order to limit data analysis to centers with stable reporting and performance practices, cases from hospitals contributing data for less than 6 months or at which less than 5 total BPV or BAV procedures were performed over the study period were excluded from analysis. In an attempt to identify a relatively homogeneous cohort of patients with isolated native aortic or pulmonary valve stenosis, cases were also excluded if the patient had one of several pre-specified prior operations or transcatheter interventions on the right ventricular (RV) or left ventricular (LV) outflow tracts (Supplementary Table 1), additional interventions during the same catheterization, a diagnosis of single ventricle heart disease, or other surgical or transcatheter procedure during the same admission. In addition, because the indication was vital for analysis, cases with the indication missing were also excluded.

Data collected included demographic data, pre-catheterization clinical history, indication for intervention, hemodynamics, and details about the conduct of the case. Characteristics about hospitals that were collected included center volume, percentage of procedures performed in patients older than 18 years, teaching status, census region (Northeast, Midwest, South, or West) (Figure 1), hospital type (government, private/community, or university) and hospital setting (rural, suburban, or urban).

Data analysis:

Analyses of BAV and BPV cases were performed separately. Descriptive statistics were calculated with categorical variables expressed as counts and percentages with 95% confidence intervals (CI). Continuous variables were expressed as mean ± standard deviation or median (range, interquartile range) as appropriate.

To study the distribution of stated indications for the procedure, we reported the distribution of patient- and hospital-level characteristics by indication for each procedure. Differences in the distributions of these characteristics were assessed using analysis of variance, Kruskal-Wallis, or Chi-square tests. To assess for significant practice variation, we adjusted for patient- and procedure-level characteristics using a series of hierarchical multivariable models with a random intercept for hospital to account for covariance by hospital. Patient-level covariates included as fixed effects were age, sex, prior catheterization, genetic condition, chronic lung disease, coagulopathy, diabetes mellitus, hepatic disease, renal insufficiency, sickle cell disease, prior stroke, seizure disorder, and sedation level. Sedation level was added to the model along with previously defined risk factors(18–20) because of concern that the level of anesthesia could potentially affect the measured gradient during the procedure. A series of models were created to assess for the effects of hospital characteristics [region, setting, type, procedural volume (included as both a continuous and dichotomous variable around a cut-point of 150 cases/year), proportion of adult cases, and teaching status] on outcome and to assess for additional individual inter-hospital variation. To accomplish the latter, a median rate ratio (MRR) was calculated. MRR has been used in previous studies to measure otherwise unaccounted for inter-hospital variation(6). In a sample of multiple hospitals, MRR reflects the probability that two identical patients treated at two of the sample hospitals selected at random would receive different care(3–6, 8, 21). The MRR value is always ≥1. As an example of how this value is interpreted, a MRR = 1.2 means that two identical patients would have a 20% probability of being treated differently at two randomly selected hospitals, a threshold used previously to define clinically meaningful variation(6).

For both valvuloplasty procedures, the majority of cases are performed to treat a high resting gradient. The authors of a joint American Heart Association, American Academy of Pediatrics, and Society for Cardiovascular Angiography and Interventions scientific statement(10) identify that, for BPV, a peak-to-peak gradient ≥40 mmHg is a class I indication for valvuloplasty. For BAV, a peak-to-peak gradient of ≥50 mmHg in asymptomatic patients is an indication for valvuloplasty (class I) or ≥40 mmHg in symptomatic patients or in those hoping to participate in competitive athletics or become pregnant (class IIB). For BAV, a peak-to-peak gradient <40 mmHg or a degree of aortic regurgitation warranting valve replacement or repair are situations where BAV is not recommended (class III). Neonatal interventions for “critical” AS and PS have a separate indication.

We further investigated the degree to which consistency with these recommendations varied between hospitals. Restricting analysis to patients >30 days of age and to cases where the operator identified the indication as “high resting gradient,” we assessed the likelihood of an intervention being performed in a case that was not consistent with the recommendations in the published guidelines, which we defined as a pre-intervention gradient <40 mmHg and/or ≥4+ aortic insufficiency for BAV cases. These thresholds were intentionally set as conservatively as possible to minimize the number of interventions that were deemed “inconsistent”. We then calculated a series of hierarchical multivariable models to adjust for patient and procedural characteristics and assess whether hospital characteristics affected the likelihood of performing interventions inconsistent with the published guidelines. We also sought to identify whether there was significant inter-hospital variation (measured by calculating the MRR as above) in the conduct of BAV and BPV. Neonates ≤ 30 days of age were excluded from this portion of the analysis because interventions for “critical” AS and PS are often not gradient-based, and there is no ability to denote “critical” valve stenosis in the registry.

Finally, because the reference guidelines were published in June of 2011 and there may be a delay in the dissemination and uptake of new recommendations, we developed models adjusted for patient-level factors to assess for changes in overall practice consistent with the published guidelines across calendar years.

A threshold for statistical significance was set at p<0.05. All data analyses were performed using SAS 9.4 (Cary, NC, USA)

RESULTS

Study Population:

Between 1/1/2011 and 9/30/2015, 1,071 BAV procedures (at 60 hospitals) and 2,207 BPV procedures (at 75 hospitals) meeting inclusion criteria were performed (Figure 2). These accounted for 4.3% (3278/77116) of all cases in the IMPACT® registry over this period.

Figure 2: Study cohort.

Indications for valvuloplasty:

In the BAV cohort, the indication was high resting gradient in 82% (880/1071), with a minority of subjects whose indications were abnormal stress test/electrocardiogram (2%), left ventricular (LV) dysfunction (11%), and symptoms (5%). Comparing cases across indications demonstrates significant differences in subject and hospital characteristics (Supplementary Table 2a). Subjects with symptomatic aortic stenosis or an abnormal stress test/electrocardiogram were older, heavier, and more likely to be over 18 years (all p<0.001). These cases were also more likely to have mild or greater aortic insufficiency (p<0.001). Symptomatic patients were more likely to have had a prior stroke (p<0.001). Patients with LV dysfunction were younger, lower in weight, more likely to be admitted to an ICU at the time of the procedure, and more likely to have a unicuspid aortic valve (all p<0.001). As expected, the peak-to-peak gradient was higher in patients in whom high resting gradient was the reported indication for intervention (p<0.001).

In terms of hospital characteristics, there were small magnitude differences in the distribution of indication by census region (p=0.01) and hospital setting (p=0.02). In addition, cases done for symptoms were performed at centers with a higher proportion of cases >18 years of age out of their total procedural volume and a lower overall annual procedural volume. After adjusting for patient- and procedure-level characteristics, models were calculated to assess associations between hospital level factors and the proportion of cases with high resting gradient as the indication vs. all other indications (Table 1a). No significant associations between hospital level factors and the distribution of indications were identified. In addition, there was no evidence of additional inter-hospital variation (MRR: 1.1, p=0.82).

Table 1a:

Distribution of indications for balloon aortic valvuloplasty

Variable	Comparison	RR (95% CI)	p-value	Global p-value
Site	Median rate ratio	1.1 (1.0–1.4)	---	0.82
Census Region	Northeast vs. West	1.1 (0.9–1.4)	0.55	0.56
	Midwest vs. West	0.9 (0.8–1.2)	0.54
	South vs. West	0.9 (0.8–1.1)	0.52
	Northeast vs. South	1.1 (0.9–1.4)	0.20
	Midwest vs. South	1.0 (0.8–1.2)	0.98
	Northeast vs. Midwest	1.1 (0.9–1.4)	0.21
Hospital setting	Rural vs. Suburbs	1.4 (0.8–2.2)	0.24
	Urban vs. Suburbs	1.2 (0.9–1.6)	0.20
	Rural vs. Urban	1.1 (0.7–1.8)	0.57
Teaching status	Yes vs. No	1.0 (0.8–1.4)	---	0.77
Hospital type	Private vs. University hospital	1.1 (0.9–1.2)	---	0.53
Proportion of cases in adults	Low vs. High	1.3 (0.7–2.2)	0.44	0.32
	Moderate vs. High	1.1 (0.6–1.9)	0.78
Hospital volume	Per 50 cases	1.0 (1.0–1.0)	---	0.36
	More than 150 cases/year	1.1 (0.5–2.6)	---	0.81

Each row represents a separate model adjusted for subject age, previous catheterization, genetic condition, chronic lung disease, coagulation disorder, diabetes mellitus, hepatic disease, renal insufficiency, seizures, sickle cell disease, prior stroke, sedation strategy.

The relative risk depicts the relative likelihood of having the indication of high resting gradient vs. all others.

In the BPV cohort, the indication was high resting gradient in 82% (1802/2207) of cases. The indications in the remainder of cases were right to left shunting (6%), RV dysfunction (7%), and symptoms (5%). There were significant differences in the distribution of both subject- and hospital-level characteristics across indications (Supplementary Table 2b). Relative to cases where the indication was high resting gradient, patients with symptoms or RV dysfunction were older (both in terms of age and proportion greater than 18 years of age) and heavier (p<0.001 for all). In terms of anatomy, patients with a right to left shunting had smaller pulmonary valve annulus and were more likely to have subvalvar stenosis (both p<0.001). Patients with a high resting gradient were more likely to have a “typical” form of pulmonary valve morphology (p=0.016). Patients with right to left shunting were more likely to already be admitted to an ICU, with the majority of other cases performed as either observation or outpatient procedures, and less likely to be performed as an elective case (both p<0.001).

In terms of hospital characteristics, there were small magnitude differences in the distribution of indications by census region (p=0.01). In addition, there were differences in the proportion of adults treated (p<0.001). However, in multivariable models assessing the proportion of cases with gradient as the indication (vs. all other indications), no significant associations between hospital characteristics and the distribution of indications were identified (Table 1b). There was also no evidence of additional inter-hospital variation (MRR: 1.1, p=0.66).

Table 1b:

Distribution of indications for balloon pulmonary valvuloplasty

Variable	Comparison	OR (95% CI)	p-value	Global p-value
Site	Median rate ratio	1.1 (1.0–1.2)	---	0.66
Census Region	Northeast vs. West	1.1 (0.9–1.3)	0.46	0.74
	Midwest vs. West	1.1 (0.9–1.2)	0.47
	South vs. West	1.1 (0.9–1.2)	0.27
	Northeast vs. South	1.0 (0.9–1.2)	0.86
	Midwest vs. South	1.0 (0.9–1.1)	0.69
	Northeast vs. Midwest	1.0 (0.9–1.2)	0.89
Hospital setting	Rural vs. Suburbs	0.9 (0.6–1.3)	0.53	0.62
	Urban vs. Suburbs	1.0 (0.9–1.2)	0.71
	Rural vs. Urban	0.9 (0.6–1.2)	0.36
Teaching status	Yes vs. No	1.0 (0.9–1.2)	---	0.86
Hospital type	Private vs. University hospital	1.0 (0.9–1.1)	---	0.99
Proportion of cases in adults	Low vs. High	1.0 (0.7–1.5)	0.83	0.60
	Moderate vs. High	1.0 (0.7–1.4)	0.90
Hospital volume	Per 50 cases	1.0 (1.0–1.0)	---	0.55
	More than 150 cases/year	1.0 (0.9–1.2)	---	0.97

Each row represents a separate model adjusted for subject age, previous catheterization, genetic condition, chronic lung disease, coagulation disorder, diabetes mellitus, hepatic disease, renal insufficiency, seizures, sickle cell disease, prior stroke, sedation strategy.

The relative risk depicts the relative likelihood of having the indication of high resting gradient vs. all others.

Consistency in practice:

In the subset of patients >30 days of age with high resting gradient as the indication for valvuloplasty, we measured the proportion of cases that were consistent with published recommendations. In BAV cases, 86% were performed in a manner consistent with published guidelines, compared to 66% of BPV cases (p<0.001). From the perspective of hospitals, 63% of hospitals performed >80% of their BAV cases consistently with guidelines (Figure 3a), while only 30% of hospitals performed BPV in consistent fashion with guidelines in >80% of cases (p<0.001, Figure 3b). Unadjusted differences in hospital characteristics based on whether interventions were performed consistent with the guidelines are summarized in Tables 2a and 2b. For BAV, there were significant differences in observed rates of interventions being performed consistent with the published guidelines based on hospital type (p<0.001), teaching status (p=0.04), annual procedural volume (p=0.03), and hospital census region: Northeast 96%, South 91%, Midwest 84%, and West 81%, p<0.001. For BPV, there were significant differences in the observed rates of interventions performed consistent with the guidelines based on hospital census region: Northeast 77%, South 71%, West 61%, and Midwest 59%, p<0.001.

Figure 3: Percentage of cases performed consistent with published guidelines.

This bar graph depicts the percentage of balloon aortic valvuloplasty (A) and balloon pulmonary valvuloplasty (B) cases performed consistently with published guidelines at each hospital in the study population in order of decreasing consistency. Overall, 86% of all aortic valvuloplasty cases were performed consistently with published guidelines, vs. 66% of pulmonary valvuloplasty cases. The red line marks the 80% threshold for individual hospitals. For aortic valvuloplasty, 27% of hospitals were below this threshold, while for pulmonary valvuloplasty 70% were.

Table 2a.

Unadjusted comparison of hospital characteristics based on whether balloon aortic valvuloplasty was performed consistent with the published guidelines. Cases restricted to those >30 days of age with “high resting gradient” as stated indication.

	Total BAV cases (n=672)	BAV cases performed inconsistent with guidelines (n=91)	BAV cases performed consistent with guidelines  (n=581)	p
Hospital census region  Northeast  Midwest  South  West	109 203 217 116	4 (3.7%) 33 (16.3%) 20 (9.2%) 22 (19%)	105 (96.3%) 170 (83.7%) 197 (90.8%) 94 (81%)	< 0.001
Hospital setting  Rural  Urban  Suburban	15 614 43	4 (26.7%) 80 (13%) 7 (16.3%)	11 (73.3%) 534 (87%) 36 (83.7%)	0.27
Hospital type  Private/Community  Government  University	322 27 323	39 (12.1%) 12 (44.4%) 40 (12.4%)	283 (87.9%) 15 (55.6%) 283 (87.6%)	< 0.001
Teaching institution	623	89 (14.3%)	534 (85.7%)	0.04
Annual center procedural volume	430 (270, 558)	473 (270, 544)	430 (270, 588)	0.03
Proportion of procedural volume >18 years of age	0.14 ± 0.09	0.12 ± 0.09	0.14 ± 0.09	0.098

Data presented as count (% of total) or median (25^th – 75^th %ile)

BAV = balloon aortic valvuloplasty

Table 2b.

Unadjusted comparison of hospital characteristics based on whether balloon pulmonary valvuloplasty was performed consistent with the published guidelines. Cases restricted to those >30 days of age with “high resting gradient” as stated indication.

	Total BPV cases (n = 1272)	BPV cases performed inconsistent with guidelines (n = 433)	BPV cases performed consistent with guidelines (n = 839)	p
Hospital census region  Northeast  Midwest  South  West	168 372 451 229	38 (22.6%) 154 (41.4%) 130 (28.8%) 90 (39.3%)	130 (77.4%) 218 (58.6%) 321 (71.2%) 139 (60.7%)	< 0.001
Hospital setting  Rural  Urban  Suburban	40 1129 103	15 (37.5%) 393 (34.8%) 25 (24.3%)	25 (62.5%) 736 (65.2%) 78 (75.7%)	0.09
Hospital type  Private/Community  Government  University	619 52 601	200 (32.3%) 21 (40.4%) 212 (35.3%)	419 (67.7%) 31 (59.6%) 389 (64.7%)	0.34
Teaching institution	1165	404 (34.7%)	761 (65.3%)	0.11
Annual center procedural volume	409 (257, 524)	409 (254, 515)	409 (259, 544)	0.34
Proportion of procedural volume >18 years of age	0.14 ± 0.1	0.15 ± 0.11	0.14 ± 0.1	0.3

Data presented as count (% of total) or median (25^th – 75^th %ile)

BPV = balloon pulmonary valvuloplasty

In multivariable analysis adjusted for patient- and procedure-level characteristics, the risk of performing a BAV procedure inconsistent with published guidelines was lower in the Northeast and South than either West or Midwest (overall p=0.005, Table 3a). Other hospital characteristics were not associated with the risk of valvuloplasty inconsistent with published guidelines. In terms of inter-hospital variation, the MRR was elevated at 1.8, but the association failed to attain statistical significance (p=0.07).

Table 3a:

Model for risk of balloon aortic valvuloplasty performed inconsistently with published guidelines

Variable	Comparison	RR (95% CI)	p-value	Global p-value
Site	Median rate ratio	1.8 (1.0–2.3)	---	0.07
Census Region	Northeast vs. West	0.2 (0.04–0.6)	0.005	0.005
	Midwest vs. West	0.9 (0.5–1.7)	0.76
	South vs. West	0.4 (0.2–0.9)	0.02
	Northeast vs. South	0.4 (0.1–1.3)	0.13
	Midwest vs. South	2.1 (1.1–4.1)	0.03
	Northeast vs. Midwest	0.2 (0.1–0.6)	0.006
Hospital setting	Rural vs. Suburbs	1.7 (0.3–8.9)	0.53	0.57
	Urban vs. Suburbs	0.8 (0.3–2.3)	0.70
	Rural vs. Urban	2.1 (0.5–8.3)	0.31
Teaching status	Yes vs. No	2.9 (0.7–1.7)	---	0.16
Hospital type	Private vs. University hospital	1.1 (0.7–1.7)	---	0.87
Proportion of cases in adults	Low vs. High	1.2 (0.1–10.8)	0.87	0.90
	Moderate vs. High	1.0 (0.1–9.9)	0.99
Hospital volume	Per 50 cases	1.0 (0.9–1.0)	---	0.11
	More than 150 cases/year	1.1 (0.5–2.6)	---	0.81

Each row represents a separate model adjusted for subject age, sex, previous catheterization, genetic condition, chronic lung disease, coagulation disorder, diabetes mellitus, hepatic disease, renal insufficiency, seizures, sickle cell disease, prior stroske, and sedation strategy.

The relative risk depicts the risk of performing a valvuloplasty under conditions that are inconsistent with published guidelines.

For BPV, after adjustment for patient- and procedure-level factors, no significant associations were seen between the hospital characteristics tested and risk of performing the intervention inconsistently with guidelines. There was, however, significant inter-hospital variation (MRR: 1.4, p<0.001), Table 3b.

Table 3b:

Model for risk of balloon pulmonary valvuloplasty performed inconsistently with published guidelines

Variable	Comparison	RR (95% CI)	p-value	Global p-value
Site	Median rate ratio	1.4 (1.2–1.6)	---	<0.001
Census Region	Northeast vs. West	0.7 (0.4–1.1)	0.12	0.10
	Midwest vs. West	1.0 (0.7–1.4)	0.99
	South vs. West	0.7 (0.5–1.0)	0.08
	Northeast vs. South	0.9 (0.6–1.5)	0.77
	Midwest vs. South	1.4 (1.0–1.9)	0.05
	Northeast vs. Midwest	0.7 (0.4–1.1)	0.10
Hospital setting	Rural vs. Suburbs	1.7 (0.7–4.1)	0.20	0.33
	Urban vs. Suburbs	1.4 (0.9–2.4)	0.17
	Rural vs. Urban	1.2 (0.6–2.4)	0.60
Teaching status	Yes vs. No	2.9 (0.7–12.9)	---	0.16
Hospital type	Private vs. University hospital	1.2 (0.8–2.0)	---	0.39
Proportion of cases in adults	Low vs. High	0.7 (0.3–1.5)	0.40	0.64
	Moderate vs. High	0.7 (0.3–1.5)	0.35
Hospital volume	Per 50 cases	1.0 (1.0–1.0)	---	0.38
	More than 150 cases/year	0.9 (0.7–1.2)	---	0.47

Each row represents a separate model adjusted for subject age, sex, previous catheterization, genetic condition, chronic lung disease, coagulation disorder, diabetes mellitus, hepatic disease, renal insufficiency, seizures, sickle cell disease, prior stroke, and sedation strategy.

The relative risk depicts the risk of performing a valvuloplasty under conditions that are inconsistent with published guidelines.

Because the reference guidelines were published in June of 2011 and the included cohort for study began in January of 2011, there may be concern for a delay in the dissemination and uptake of new recommendations. For this reason, changes in overall consistency in practice were evaluated across the calendar years included. For BAV, overall rates of practice inconsistent with the guidelines were: 15.9% in 2011, 13.3% in 2012, 14.4% in 2013, 13.7% in 2014, and 11.8% in 2015, p=0.95. For BPV, overall rates of practice inconsistent with the guidelines were: 23.3% in 2011, 36.9% in 2012, 33.4% in 2013, 35% in 2014, and 34.8% in 2015, p=0.23. After adjustment for patient-level factors, these differences remained insignificant.

DISCUSSION

In this multicenter retrospective study, we were able to identify significant hospital-level variation in the practice of balloon aortic and pulmonary valvuloplasty procedures that cannot be explained by patient- and procedure-level characteristics as measured in the IMPACT ® Registry. Specifically, there was significant variation in the risk of performing valvuloplasty under conditions that were inconsistent with published guidelines. BAV procedures performed at hospitals in the Midwest and West were more likely to be inconsistent with published guidelines than those in the Northeast and South, a finding that persisted after adjustment for patient- and procedure-level factors. For BPV, adjusted rates of consistency with published guidelines did not differ significantly across census regions. However, overall consistency with guidelines was much less for BPV than BAV, with significant inter-hospital variation identified clearly in BPV cases. For BAV, there was a large magnitude MRR suggestive of additional inter-hospital variation, but it was not statistically significant. These findings represent variation in practice that may affect clinical outcomes and healthcare utilization.

To date, there have been few studies assessing variability in practice and its effect on outcomes in the field of congenital cardiac catheterization. Porras et al. reported the process of imposing a Standardized Clinical Assessment and Management Plan (SCAMP) for BAV, comparing the conduct and outcomes of 23 cases performed after implementation of the SCAMP to those of 92 historical controls(22). In this series, 100% of cases performed after SCAMP implementation were in compliance with the SCAMP proposed trans-valvar gradient threshold for intervention of 35 mmHg. The proportion of cases in the historical control cohort with lower initial gradients is not reported. The authors do report that the percentage of cases with a gradient ≤50 mmHg actually increased after SCAMP implementation, which may be the result of formalizing a lower gradient for intervention. The most recent large multicenter series of BAV cases combined data from MAGIC and C3PO registries. It did not report the proportion of cases performed with initial gradients less than 40mmHg, but the reported interquartile range for initial gradient was 50 and 71, implying that 25% of cases were performed for gradients less than 50 mmHg(23).

Even less data are available regarding practice variation in BPV. No studies have attempted to impose standardized practice guidelines for BPV. To our knowledge, the only multicenter observational studies are derived from the Valvuloplasty and Angioplasty of Congenital Anomalies (VACA) Registry, reporting the outcomes of procedures performed at 28 institutions in the 1980’s(24, 25). The proportion of patients with low gradients and other aspects of practice are not reported in either study.

Although we recently reported on variability in the practice of device-closure of atrial septal defect and patent ductus arteriosus(26), the current study is the first to measure the degree of variation in practice across hospitals for transcatheter valvuloplasty procedures in children. Recommendations of the published guidelines were more consistently followed for BAV, in which 86% of all cases performed for a high resting gradient were deemed consistent with published guidelines as compared to only 66% of BPV cases performed for the same indication. In addition, clinically relevant inter-hospital variation as measured by MRR was seen in cases of BPV. It is tempting to speculate as to the ultimate reason(s) for the observed patterns of variation. We hypothesize that greater variation in BPV may reflect the perception that the risk of BPV is less than that for BAV, specifically in regards to creating valvar insufficiency. Pulmonary insufficiency is relatively well tolerated both acutely and chronically(24, 27, 28), while aortic insufficiency is poorly tolerated and has been an indication for various operative interventions(29–31). Another possibility is that the indications for BPV in the consensus guidelines include the presence of either a peak-to-peak gradient ≥40 mmHg measured in the catheterization laboratory or an echocardiographic peak instantaneous gradient ≥40 mmHg, whereas the indications given for intervention for BAV include only the catheter-derived gradient(10). Because the IMPACT® registry does not include pre-intervention gradients derived from echocardiography, we are unable to include this in our assessment of practice consistent with the guidelines for BPV. Finally, it is important to note that valve gradients (whether assessed by echocardiography or in the catheterization laboratory) are flow-dependent and thus can be affected by the patient’s physiologic state. We attempted to account for this by including the level of anesthesia in all of our models, but it is possible that this does not fully account for varying levels of physiologic arousal that could affect the measured gradients. Both of these issues could partially account for the increased between-center practice variation observed in BPV. Regardless of the cause, however, the point remains that variation in practice exists.

For BAV there were significant differences in how consistently the guidelines were followed between census regions, with hospitals in the Northeast and South more likely to have been consistent than hospitals in the Midwest and West, even after adjustment for potential differences in patient- and procedure-level factors. The etiology of this clustering is not clear. One could speculate a relationship to the local spread of trainees, but there is no way to test this hypothesis in the current dataset. There are a host of other factors (e.g. geography, referral patterns, or patterns of reimbursement) that could affect how physicians practice. These also could not be assessed as part of this study. Finally, the published guidelines are based largely on expert opinion with potential limitations in the evidence base underlying them. Thus, another potential explanation for practice inconsistent with the published guidelines is that some practitioners may simply not agree with the stated guidelines. The standardized practice pathway for BAV at a single large center which advocates a gradient threshold for intervention >35 mmHg supports this notion(22). This, though, remains a form of practice variation.

It is also challenging to measure the effect that this practice variation has on outcomes. We certainly acknowledge that, having observed a level of variation in the current series, there is no group against which to compare outcomes. However, both forms of observed practice variation have the potential to influence clinical outcomes and resource utilization. Intervening on patients with relatively low gradients represents a more aggressive practice pattern that exposes a larger number of patients to the risks of anesthesia, catheterization, and intervention. In terms of resource utilization, worse clinical outcomes and increased adverse events would certainly incur greater cost. Moreover, even if clinical outcomes are similar, unless there are dramatic improvements in outcome, referral and intervention at lower gradients inevitably increases resource utilization and healthcare spending.

There are a number of limitations to the current study. The IMPACT® registry is composed of data submitted by the staff at each hospital. Though data fields are clearly defined, local interpretation of these definitions could introduce variation inadvertently. Second, the current study cannot necessarily differentiate between the decisions of referring cardiologists and those of the interventionalist. Specifically, 1) there are no data about patients who are not referred for cardiac catheterization; and 2) less data are available on patients in whom the interventionalist chose not to intervene. For this reason we focused on the consistency between an indication for intervention cited by the interventionalist and data that they collected directly in the catheterization laboratory. Thirdly, the IMPACT® registry contains detailed information regarding patient- and procedure-level factors that might influence decisions about intervention, but the potential for unmeasured confounding remains.

Despite these limitations we conclude that there is significant hospital-level practice variation in both aortic and pulmonary balloon valvuloplasty. Further research is necessary to measure the effect this variation has on outcome and resource utilization and ultimately whether reducing practice variation improves either.

CLINICAL PERSPECTIVES.

What is known?

Reducing practice variation has been shown to improve outcomes in a number of arenas within adult cardiovascular care as well as in specific groups of patients with congenital heart disease. However, practice variation within the field of pediatric cardiac catheterization has not been explored on a national scale.

What is new?

The IMPACT® Registry, a nationwide registry of all cardiac catheterization procedures performed on children and adults with congenital heart disease, provides a unique opportunity to explore practice variation in this field. We report, for the first time, substantial practice variation in the conduct of balloon aortic and pulmonary valvulplasty (two well-established interventions in the field) independent of differences in patient-level factors.

What is next?

Further work is needed to demonstrate that this practice variation affects the quality of care and that a reduction in practice variation can improve care quality.

ABBREVIATIONS

AS

aortic stenosis

BAV

balloon aortic valvuloplasty

BPV

balloon pulmonary valvuloplasty

ICU

intensive care unit

IMPACT®

Improving Adult and Congenital Treatment

IQR

inter-quartile range

LOE

level of evidence

LV

left ventricle

MRR

median rate ratio

NCDR

National Cardiovascular Data Registry

PS

pulmonary stenosis

RV

right ventricle

SCAMP

Standardized Clinical Assessment and Management Plan