Abstract
Background:
Optimal strategies to improve national congenital heart surgery outcomes and reduce variability across hospitals remain unclear. Many policy and quality improvement efforts have focused primarily on higher risk patients and mortality alone. Improving our understanding of both morbidity and mortality and current variation across the spectrum of complexity would better inform future efforts.
Methods:
Hospitals participating in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (2014–2017) were included. Case-mix adjusted operative mortality, major complications, and post-operative length of stay (LOS) were evaluated using Bayesian models. Hospital variation was quantified by the inter-decile ratio (IDR-ratio of upper vs. lower 10%) and 95% credible intervals. Stratified analyses were performed by risk group [STS-European Association for Cardiothoracic Surgery (STAT) category], and simulations evaluated the potential impact of reductions in variation.
Results:
A total of 102 hospitals (n=84,407) were included, representing ~85% of US congenital heart programs. STAT category 1-3 (lower risk) operations comprised 74% of cases. All outcomes varied significantly across hospitals: adjusted mortality by 3-fold [upper vs. lower decile 5.0% vs. 1.6%, IDR 3.1 (2.5-3.7)], mean LOS by 1.8-fold [19.2 vs 10.5 d, IDR 1.8 (1.8-1.9)], and major complications by >3-fold [23.5% vs 7.0%, IDR 3.4 (3.0-3.8)]. The degree of variation was similar or greater for low vs. high risk cases across outcomes, e.g. ~3-fold mortality variation across hospitals for both STAT 1-3 [IDR 3.0 (2.1-4.2)] and STAT 4-5 [IDR 3.1 (2.4-3.9)] cases. High volume hospitals had less variability across outcomes and risk categories. Simulations suggested potential reductions in deaths (n=282), major complications (n=1,539), and LOS (101,183 d) over the 4-year study period if all hospitals were to perform at the current median or better, with 37-60% of the improvement related to the STAT 1-3 (lower risk) group across outcomes.
Conclusions:
We demonstrate significant hospital variation in both morbidity and mortality following congenital heart surgery. Contrary to traditional thinking, a substantial portion of potential improvements that could be realized on a national scale were related to variability among lower risk cases. These findings suggest modifications to our current approaches to optimize care and outcomes in this population are needed.
Keywords: congenital cardiac defect, outcomes research, quality of care
INTRODUCTION
Approximately 40,000 children undergo congenital heart surgery in the United States (US) each year. While overall outcomes have improved dramatically over the past 3-4 decades, numerous studies have shown persistent wide variability in outcomes across hospitals (1). For example, several studies have suggested that early mortality for patients with single ventricle heart disease undergoing the Norwood operation ranges from approximately 10% to 40% across centers (2–3). A variety of ongoing policy and quality improvement initiatives have aimed to highlight and address this variability, including efforts focused on public reporting, dissemination and implementation of best practices, and efforts exploring regionalization of care (4–10).
Traditionally, initiatives such as these have often focused on the highest risk patients and primarily on mortality as an outcome. For example, the initial quality improvement project led by the National Pediatric Cardiology Quality Improvement Collaborative (NPC-QIC) focused on single ventricle patients and interstage mortality between the Norwood operation and second stage surgery (10). In addition, the potential impact of initiatives such as regionalization of care or centers of excellence is primarily discussed in the context of neonates or those with the highest complexity disease as it has been hypothesized that this may be where the greatest potential for improvement exists (5). In part this is related to a lack of data regarding variability in outcomes beyond mortality, limited information about the extent of hospital variation in the current era as outcomes have continued to improve, and a poor understanding of the degree of variability among patients with lower risk types of congenital heart disease. Improving our understanding of current national outcomes and hospital variation across the spectrum of disease could better inform future efforts.
In this context, the purpose of the present study was to describe variation in congenital heart surgery morbidity and mortality across a large national sample spanning both low and high risk cases. In addition, we sought to evaluate the potential national implications of efforts geared toward reducing variation.
METHODS
Data Source
The Society of Thoracic Surgeons Congenital Heart Surgery Database (STS-CHSD) collects standardized peri-operative data on all patients undergoing pediatric and congenital heart surgery at participating hospitals. Data quality is optimized through data checks, site visits, and audits (11). This study was approved by the Duke University and University of Michigan institutional review boards and was not considered human subjects research in accordance with the Common Rule (45 CFR 46.102(f)). Requests to access the data utilized for this study may be sent to the STS (contact information available at: https://www.sts.org/registries-research-center/sts-research-center/access-publications).
Study Population
Patients undergoing any index cardiovascular operation with or without cardiopulmonary bypass at US centers participating in the STS-CHSD from 2014-2017 were included (n=116 centers, 95060 operations). Consistent with prior studies, the first (index) cardiovascular operation of each hospital admission was analyzed and infants <2.5 kg undergoing isolated ductus arteriosus ligation were excluded (12). We further excluded 14 hospitals that had >10% missing data for study variables, remaining patients with missing data for study predictor or outcome variables, those without an STS-European Association for Cardiothoracic Surgery (STAT) mortality score, and records with data collected under an obsolete data collection form (13). The final cohort consisted of 84,407 patients across 102 hospitals.
Outcomes
Outcomes examined included operative mortality, post-operative major complications, and post-operative length of stay (LOS). The STS-CHSD defines operative mortality as any death occurring in-hospital, and any deaths occurring after discharge within 30 days of surgery. Major complications are also defined by the STS-CHSD in a standard fashion and include renal failure requiring dialysis, permanent neurologic deficit, pacemaker, paralyzed diaphragm, mechanical circulatory support, and unplanned re-intervention, as previously described (14). These complications were previously selected and endorsed as quality metrics by the STS and Congenital Heart Surgeons Society (CHSS) based on evidence suggesting an important impact on outcome and standard definitions within the STS-CHSD (15). They have been used in multiple prior analyses (14,16,17). Major complications and LOS were evaluated in survivors in order to avoid “double counting” of events at the hospital level for a patient who, for example, had a major complication and died. A prior analysis demonstrated very high correlation between a hospital’s major complication rate overall vs. in survivors alone (0.98) and between a hospital’s mean LOS overall vs. survivors alone (0.99) (12).
Statistical Analysis
Statistical methods are described in detail in the Supplemental Material and Supplemental Tables I–IV. Briefly, the goal of the statistical analysis was to estimate hospital-specific case-mix adjusted outcome measures for the three endpoints (operative mortality, major complications, and LOS) and to evaluate the amount of hospital-level variation in these measures attributable to true differences as opposed to chance. For this purpose, we used Bayesian hierarchical generalized linear mixed models with hospital-specific random intercept parameters. The form of the model was a logistic regression for mortality and major complications and a proportional odds model for LOS. The proportional odds methodology was selected because it provides a flexible semi-parametric modeling framework and accommodates features of the LOS distribution that are not well approximated by standard parametric distributions (18). We fit these models in the overall study cohort and separately among subgroups of interest as described further below.
Case-mix adjustment was based on the previously published STS-CHSD risk model, which includes adjustment for age, weight, prematurity status, non-cardiac and genetic anomalies, procedure type, prior cardiothoracic operation, and several pre-operative factors. (19). To adjust for case-mix, we first estimated risk scores predicting each outcome in a series of non-hierarchical models using the published STS-CHSD model covariates. The resulting risk scores were used as covariates in the subsequent hierarchical models. The fit of the hierarchical models was assessed by comparing observed versus predicted outcomes overall and by comparing results of alternative modeling strategies as described further in the supplemental material.
A hospital’s risk case-mix adjusted mortality rate was calculated by multiplying the overall mortality rate in the model cohort by the ratio of the hospital’s true mortality rate according to the hierarchical model divided by the mortality rate that would be expected for a hypothetical hospital with the same case-mix and an intercept parameter equal to the population median. An analogous calculation was used for hospital-specific case-mix adjusted major complication rates and average LOS (see supplemental material for details).
Model parameters were estimated in a Bayesian statistical framework by specifying an uninformative prior probability distribution for unknown parameters. An advantage of Bayesian analysis is the ability to estimate summary measures of between-hospital variation while accounting for statistical uncertainty (20). Instead of relying on a single point estimate of each hospital’s outcome metrics, which may be highly uncertain, the Bayesian framework considers all possible estimates of each hospital’s outcome metrics and averages them appropriately. Markov Chain Monte Carlo (MCMC) simulations were used to generate representative samples of numerical estimates for each outcome metric and hospital. These simulated values were then analyzed to create the following summary measures for each of the 3 outcome metrics: (1) the average value among the 10% of hospitals with the highest values (upper decile), (2) the average value among the 10% of hospitals with the lowest values (lower decile), (3) the median value across hospitals, (4) the inter-decile ratio, which is the ratio of the average value in the upper decile vs. the lower decile, and (5) the inter-decile difference, which is the absolute difference between the upper decile and lower decile of hospitals. This approach minimizes the impact of any individual hospital or outlier. For additional perspective on the amount of between-hospital variation, we also estimated average outcomes in the highest versus lowest 25% (quartile) of hospitals (see Supplemental Material and Supplemental Table IV for details). Summary measures were reported for the overall population, as well as stratified by STAT category (STAT category 1-3 or lower risk cases vs. STAT category 4-5 or higher risk cases). While analyses of individual types of CHD diagnoses or procedures may be of interest, the wide heterogeneity of disease and small sample size for any individual lesion limit the conclusions that may be drawn both in the present study and other analyses. STAT categories were created to empirically group operations of similar risk together (minimizing within category heterogeneity and maximizing between category variation) to facilitate meaningful analyses and augment power (13). They are widely used across many congenital heart surgery analyses (1,13,21). We also performed stratified analyses by age (neonates vs. non-neonates), and center annual surgical volume using standard categories (<150, 150-349, and 350+ cases/year) (22) In addition to these summary measures, for graphical purposes we calculated estimates for individual hospitals for construction of caterpillar plots.
Finally, we used MCMC simulations to assess theoretical changes in outcome within the study population if variability were reduced and all hospitals were to perform at the current median or better. Within each MCMC iteration, we identified hospitals having hospital-specific random intercept parameters above the median. We then calculated the model-predicted total number of deaths, complications, and hospital days hypothetically if these intercept parameters were set to the median. This hypothetical prediction was compared to the observed outcome data overall and within STAT category 1-3 and STAT category 4-5. We also estimated the proportion of overall change related to STAT 1-3 vs. STAT 4-5 cases for each outcome. Other CHD studies involving simulations have focused on the volume-outcome relationship. However, our focus was on understanding potential implications regarding reductions in outcome variability directly rather than that potentially mediated by hospital volume (23).
Data are presented as point estimates with 95% Bayesian credible intervals (CrI). Unlike frequentist confidence intervals, Bayesian 95% CrIs have an intuitively direct interpretation as an interval containing the true value with 95% probability. Statistical analyses were performed using SAS v9.4 (SAS institute, Cary, NC), R version 3.5.2, WinBUGS version 1.4, Rstan version 2.19.3, and the R package “brms” (24,25).
RESULTS
Study Population Characteristics
A total of 102 hospitals (n=84,407 patients) were included. Overall study population characteristics are shown in Table 1. There were approximately three times as many STAT 1-3 (lower risk) vs. STAT 4-5 (higher risk) cases (74% vs. 26%) performed during the study period. Across hospitals, the median annual surgical volume was 170 total cases/year (interquartile range 95-292), 125 STAT 1-3 cases/year (interquartile range 77-217), and 43 STAT 4-5 cases/year (interquartile range 17-73). Unadjusted outcomes stratified by STAT category are displayed in Table 2. As anticipated, patients in STAT category 4-5 had higher observed rates of operative mortality and major complications, and longer LOS.
Table 1.
Study Population Characteristics
| Characteristic | Total N=84407 |
|---|---|
| Age at surgery | 11.8 mo (2.9 mo – 6.3 yr) |
| Neonates | 15363 (18.2%) |
| Weight at surgery | 8.4 kg (4.4-24.0) |
| Prematurity* | 8038 (19.0%) |
| Any prior cardiothoracic operation | 29980 (35.5%) |
| Any non-cardiac abnormality** | 2910 (3.5%) |
| Any chromosomal anomaly | 10520 (12.5%) |
| Any syndrome | 15927 (18.9%) |
| Any STS-CHSD pre-operative risk factor | 24291 (28.8%) |
| STAT Category | |
| 1-3 | 62622 (74.2%) |
| 4-5 | 21785 (25.8%) |
Data are displayed as median (interquartile range) or n (%)
Neonates/infants only
those included in the current STS-CHSD risk model
STS-CHSD=Society of Thoracic Surgeons Congenital Heart Surgery Database
Table 2.
Unadjusted outcomes by STAT Category
| Outcome | STAT 1-3 (n=62,622) | STAT 4-5 (n=21,785) |
|---|---|---|
| Operative mortality | (753) 1.2% | (1681) 7.7% |
| LOS | 5 (4-9) d | 15 (8-33) d |
| Major complications | (4985) 8.1% | (5005) 24.9% |
Data are displayed as median (interquartile range) or n (%)
LOS=Length of Stay
STAT=Society of Thoracic Surgeons–European Association for Cardiothoracic surgery
Hospital Variation - Overall
The distribution of outcomes across individual hospitals is displayed in Figure 1, and Table 3 contains the overall median and values among hospitals in the upper and lower deciles. Overall we found significant variation across hospitals as assessed by the ratio of upper vs. lower decile (inter-decile ratio) for all outcomes examined. This included 3-fold variation in adjusted operative mortality (inter-decile ratio 3.1, 95% CrI 2.5-3.7), >3-fold variation in adjusted major complications (inter-decile ratio 3.4, 95% CrI 3.0-3.8), and 1.8-fold variation in adjusted mean length of stay (inter-decile ratio 1.8, 95% CrI 1.8-1.9) (Table 3, Figure 1). Data from the upper and lower hospital quartiles are also presented in Supplemental Table IV.
Figure 1. Hospital variation in adjusted outcomes.
Data are displayed for operative mortality (A), length of stay (B), and major complications (C). Each hospital’s point estimate (black box) and 95% credible interval (line) are shown in descending order. The dotted line represents the population mean. Values for the upper and lower deciles and inter-decile ratio (IDR) are shown.
Table 3.
Hospital variation in adjusted outcomes by STAT category and age
| Outcome | Lower Decile | Median | Upper Decile | Inter-Decile Ratio | Inter-Decile Absolute Difference |
|---|---|---|---|---|---|
| Operative mortality (%) | |||||
| Overall | 1.6 (1.4 - 1.9) |
2.9 (2.7 - 3.1) |
5.0 (4.4 - 5.7) |
3.1 (2.5 - 3.7) |
3.4 (2.7 - 4.2) |
| Non-neonates | 0.9 (0.8 - 1.1) |
1.7 (1.6 - 1.9) |
3.0 (2.6 - 3.6) |
3.3 (2.5 - 4.3) |
2.1 (1.6 - 2.7) |
| Neonates | 4.4 (3.6 - 5.1) |
8.1 (7.4 - 8.9) |
15.3 (13.0 - 18.4) |
3.5 (2.7 - 4.7) |
10.9 (8.2 - 14.3) |
| STAT 1-3 | 0.7 (0.6 - 0.8) |
1.2 (1.1 - 1.3) |
2.1 (1.7 - 2.5) |
3.0 (2.1 - 4.2) |
1.4 (0.9 - 1.9) |
| STAT 4-5 | 4.4 (3.7 - 5.1) |
7.7 (7.1 - 8.3) |
13.4 (11.7 - 15.6) |
3.1 (2.4 - 3.9) |
9.0 (6.8 - 11.5) |
| LOS (mean, days) | |||||
| Overall | 10.5 (10.0 - 10.9) |
14.7 (14.2 - 15.3) |
19.2 (18.5 - 20.1) |
1.8 (1.8 - 1.9) |
8.8 (8.2 - 9.4) |
| Non-neonates | 8.3 (7.9 - 8.6) |
11.7 (11.2 - 12.2) |
15.7 (15.0 - 16.5) |
1.9 (1.8 - 2.0) |
7.5 (7.0 - 8.0) |
| Neonates | 19.9 (18.9 - 21.0) |
28.4 (27.1 - 29.8) |
39.6 (37.6 - 41.9) |
2.0 (1.9 - 2.1) |
19.7 (17.8 - 21.9) |
| STAT 1-3 | 7.5 (7.2 - 7.9) |
10.8 (10.3 - 11.2) |
14.8 (14.1 - 15.5) |
2.0 (1.9 - 2.0) |
7.2 (6.7 - 7.7) |
| STAT 4-5 | 19.4 (18.5 - 20.3) |
26.5 (25.5 - 27.5) |
33.7 (32.3 - 35.5) |
1.7 (1.6 - 1.8) |
14.3 (12.8 - 16.0) |
| Major complications (%) | |||||
| Overall | 6.0 (5.3 - 6.7) |
12.5 (11.6 - 13.6) |
20.3 (18.6 - 22.1) |
3.4 (3.0 - 3.8) |
14.2 (12.6 - 16.0) |
| Non-neonates | 5.3 (4.6 - 5.9) |
9.6 (8.9 - 10.3) |
15.6 (14.3 - 17.0) |
2.9 (2.6 - 3.4) |
10.3 (8.9 - 11.8) |
| Neonates | 10.0 (8.3 - 11.8) |
25.7 (23.2 - 28.4) |
46.2 (41.3 - 51.7) |
4.6 (3.9 - 5.6) |
36.2 (31.3 - 41.6) |
| STAT 1-3 | 4.3 (3.8 - 4.9) |
8.2 (7.5 - 8.8) |
13.7 (12.5 - 15.1) |
3.2 (2.7 - 3.7) |
9.4 (8.1 - 10.8) |
| STAT 4-5 | 11.9 (10.2 - 13.6) |
25.6 (23.5 - 27.9) |
41.5 (37.7 - 45.9) |
3.5 (3.0 - 4.2) |
29.5 (25.6 - 34.2) |
Data are displayed as adjusted estimates and 95% credible intervals
LOS=Length of Stay
STAT=Society of Thoracic Surgeons-European Association for Cardiothoracic surgery
Stratified Analyses by STAT Category, Age, and Hospital Volume
Stratified analyses by STAT category revealed that magnitude of variability across hospitals was similar or greater for STAT 1-3 (lower risk) vs. STAT 4-5 (higher risk) cases across all outcomes examined (Table 3, Figure 2). For example, there was approximately 3-fold variation for operative mortality in both groups [STAT 1-3: inter-decile ratio 3.0 (95% CrI 2.1-4.2) vs. STAT 4-5: inter-decile ratio 3.1 (95% CrI 2.4-3.9)]. As expected, because of higher overall event rates, the absolute differences between hospital deciles were higher for the higher risk group.
Figure 2. Variation across hospitals in adjusted outcomes stratified by risk group.
The magnitude of variation in outcome across hospitals (as assessed by the inter-decile ratio and 95% credible interval) was similar (mortality and major complications) or greater (LOS) for STAT 1-3 cases (lower risk) vs. STAT 4-5 cases (higher risk). See Table 3 for additional data. LOS=length of stay, Mortality=in-hospital mortality, Complications=major complications STAT=Society of Thoracic Surgeons-European Association for Cardiothoracic Surgery
We also conducted stratified analyses by age group. The magnitude of variability across hospitals was similar in neonates vs. non-neonates for operative mortality and LOS (Table 3). For major complications, there was greater variability in neonates (Table 3).
Finally, the relationship between annual surgical volume and hospital variation in outcome was evaluated. Across outcomes, we found less variability in outcome (lower inter-decile ratio) among high vs. low volume hospitals with the results for LOS and complications having non-overlapping 95% CrI (Table 4, Figure 3).
Table 4.
Hospital variation in adjusted outcomes by surgical volume
| Outcome | Lower Decile | Median | Upper Decile | Inter-Decile Ratio | Inter-Decile Absolute Difference |
|---|---|---|---|---|---|
| Operative mortality (%) | |||||
| Annual Volume <150 | 1.7 (1.3 - 2.0) |
3.1 (2.8 - 3.4) |
5.5 (4.6 - 6.6) |
3.3 (2.5 - 4.5) |
3.8 (2.8 - 5.0) |
| Annual Volume 150-349 | 1.6 (1.4 - 1.9) |
2.8 (2.5 - 3.1) |
4.5 (4.0 - 5.1) |
2.8 (2.3 - 3.4) |
2.9 (2.3 - 3.5) |
| Annual Volume 350+ | 1.7 (1.4 - 2.0) |
2.7 (2.4 - 3.0) |
3.9 (3.4 - 4.4) |
2.3 (1.8 - 2.8) |
2.1 (1.6 - 2.8) |
| LOS (mean, days) | |||||
| Annual Volume <150 | 10.3 (9.8 - 10.8) |
14.5 (14.0 - 15.1) |
19.7 (18.6 - 20.8) |
1.9 (1.8 - 2.0) |
9.4 (8.4 - 10.4) |
| Annual Volume 150-349 | 10.7 (10.2 - 11.2) |
15.4 (14.7 - 16.1) |
19.5 (18.6 - 20.3) |
1.8 (1.8 - 1.9) |
8.8 (8.2 - 9.4) |
| Annual Volume 350+ | 10.9 (10.5 - 11.4) |
14.1 (13.4 - 14.8) |
17.1 (16.5 - 17.8) |
1.6 (1.5 - 1.6) |
6.2 (5.8 - 6.6) |
| Major complications (%) | |||||
| Annual Volume <150 | 5.3 (4.3 - 6.2) |
12.3 (11.1 - 13.6) |
21.3 (19.0 - 23.9) |
4.0 (3.3 - 5.0) |
16.0 (13.5 - 18.7) |
| Annual Volume 150-349 | 6.6 (5.9 - 7.5) |
12.3 (11.2 - 13.5) |
19.7 (18.0 - 21.6) |
3.0 (2.6 - 3.4) |
13.1 (11.5 - 14.8) |
| Annual Volume 350+ | 7.8 (6.9 - 8.7) |
13.9 (12.4 - 15.6) |
18.2 (16.7 - 19.9) |
2.3 (2.1 - 2.6) |
10.4 (9.1 - 11.9) |
Data are displayed as adjusted estimates and 95% credible intervals. Surgical volume = average annual cases/year during the study period. Distribution of hospitals across volume groups: <150 cases/year (48 hospitals, n=16,069), 150-349 cases/year (39 hospitals, n=37,444), 350+ cases/year (15 hospitals, n=30,894).
LOS=Length of Stay
STAT=Society of Thoracic Surgeons-European Association for Cardiothoracic surgery
Figure 3. Variation in adjusted outcomes by hospital surgical volume category.
Data are displayed as adjusted estimates and 95% credible intervals for the inter-decile ratio by hospital volume category across all outcomes. High volume hospitals had less variability in outcome (lower inter-decile ratio) compared to low volume hospitals across outcomes, with the results for LOS and complications having non-overlapping 95% credible intervals. See methods and Table 4 for more information regarding volume categories: Low (<150 cases/year), Middle (150-349 cases/year), High (350+ cases/year). LOS=length of stay, Mortality=in-hospital mortality, Complications=major complications
Simulations – Theoretical Reductions in Variability
Simulations were performed to assess theoretical improvements in outcomes if variability across hospitals in the study cohort were to be reduced. We evaluated potential improvements if all hospitals in the cohort were to perform at the current median or better. During the 4-year study period we estimated 282 fewer deaths (11.6% reduction, 95% CrI 7.1-16.2), 101,183 fewer days in hospital (8.5% reduction, 95% CrI 7.0-10.1), and 1,539 fewer major complications (15.4% reduction, 95% CrI 12.3-18.7). The proportion of these reductions related to STAT 1-3 (lower risk) cases ranged from 37 - 60% across outcomes (Table 5).
Table 5.
Simulation Results - If all hospitals were to perform at median or better
| Observed number of events | Predicted number of events from simulation | Absolute Difference (observed vs. predicted) | Percent change (observed vs. predicted) | Proportion of overall change accounted for by specified STAT category* | |
|---|---|---|---|---|---|
| Operative Mortality | |||||
| Overall | 2,434 | 2,152 (2,039 – 2,260) |
282 (174 - 395) |
11.6% (7.1% - 16.2%) |
|
| STAT 1-3 | 753 | 650 (589 - 711) |
103 (42 - 164) |
13.7% (5.6% - 21.8%) |
37% (18% - 57%) |
| STAT 4-5 | 1,681 | 1,502 (1,407 – 1,593) |
179 (88 - 274) |
10.7% (5.3% - 16.3%) |
63% (43% - 82%) |
| LOS | |||||
| Overall | 1,187,392 | 1,086,209 (1,067,961 – 1,103,880) |
101,183 (83,512 – 119,431) |
8.5% (7.0% - 10.1%) |
|
| STAT 1-3 | 659,299 | 598,100 (583,515 – 612,995) |
61,199 (46,304 – 75,784) |
9.3% (7.0% - 11.5%) |
60% (52% - 69%) |
| STAT 4-5 | 528,093 | 488,109 (477,293 – 498,696) |
39,984 (29,397 – 50,800) |
7.6% (5.6% - 9.6%) |
40% (31% - 48%) |
| Major Complications | |||||
| Overall | 9,990 | 8,451 (8,120 – 8,765) |
1,539 (1,225 – 1,870) |
15.4% (12.3% - 18.7%) |
|
| STAT 1-3 | 4,985 | 4,205 (3,984 – 4,417) |
780 (568 – 1,1) |
15.6% (11.4% - 20.1%) |
51% (40% - 61%) |
| STAT 4-5 | 5,005 | 4,246 (4,000 – 4,476) |
759 (529 – 1,5) |
15.2% (10.6% - 20.1%) |
49% (39% - 60%) |
Proportion of overall absolute change accounted for by STAT 1-3 and 4-5 cases.
LOS=Length of Stay
STAT=Society of Thoracic Surgeons-European Association for Cardiothoracic surgery
DISCUSSION
These contemporary data demonstrate significant variability across US hospitals in both morbidity and mortality for children undergoing heart surgery. Importantly, this variation was evident across the spectrum of risk, and contrary to traditional thinking was not confined solely to higher risk cases. In fact, our simulations suggested that a third to more than half of the potential improvements that could be realized through reducing variability in outcomes on a national scale were related to lower risk cases, which have not been the focus of most quality improvement or health policy efforts in the field to date.
Several prior studies using a variety of different data sources have documented variability in outcomes across hospitals for patients undergoing congenital heart surgery. For example, in the Pediatric Heart Network’s Single Ventricle Reconstruction Trial which enrolled patients from 2005-2008, the rate of in-hospital death or transplant following the Norwood operation ranged from 7% to 39% across 14 trial sites analyzed (2). Analyzing the same dataset used in this study (the STS-CHSD), Jacobs and colleagues evaluated 73 hospitals from 2005-2009 and reported unadjusted in-hospital mortality rates across individual centers. Mortality ranged from 0% to 3.1% for STAT 1 cases, and from 4.8% to 50% for STAT 5 cases (21).
Several aspects of our study and findings are unique. First, the present analysis represents the most comprehensive study of variability in congenital heart surgery outcomes across the US to date, including 102 of the estimated 120 congenital heart programs in the country (26). In addition, as opposed to prior studies, we utilized risk-adjustment methods to account for the known wide differences in case-mix across centers, which is critical when assessing variability (27). We also utilized methods to better quantify the magnitude of variation while at the same time mitigating the impact of any single hospital outlier.
Our findings suggest that although recent data have shown a continued overall trend of improving outcomes for this patient population, significant variability across hospitals persists, similar to the findings from prior decades documented in the studies above (1). Further, our results suggest that variability exists not only for mortality but also for measures of morbidity (major complications and LOS). When evaluating the impact of hospital surgical volume, we found less variability in outcome among higher volume centers. Our data suggest that there is not only an overall volume-outcome relationship as shown in many prior analyses, but that outcomes at higher volume programs are also more consistent across hospitals, which has not been demonstrated previously (28–30).
Our most novel findings relate to the stratified analyses across risk groups and simulations. We found that the overall magnitude of variation across hospitals was not necessarily less for lower risk procedures (STAT 1-3) compared to higher risk procedures (STAT 4-5). This also held true when we examined mortality for neonatal cases vs. those in older age groups. Importantly, our results also demonstrated that on a national scale, lower risk STAT 1-3 cases are currently performed approximately three times more frequently as compared to STAT 4-5 cases. Thus, in our simulations we found that anywhere from a third to more than half of the potential improvements in outcome that could be achieved through reducing national variability were related to the lower risk (STAT 1-3) cases.
Taken together, our results have several important implications. In the context of efforts to reduce national variability and improve outcomes in this population, our findings suggest that lower risk patients must also be included. While many initiatives have focused on populations with the highest rates of morbidity and mortality (e.g. single ventricle patients), it is important to recognize that from a public health perspective national gains may be maximized by broadening our approach. For example, with regard to multi-center quality improvement initiatives, initial work in the field led by the National Pediatric Cardiology Quality Improvement Collaborative (NPC-QIC) focused on improving interstage mortality and other outcomes in single ventricle population following the Norwood operation (10). Recent efforts supported through Cardiac Networks United and others have included additional patients across the spectrum of complexity and may have the potential to add to these gains and augment the overall national impact. Examples include a joint Pediatric Acute Care Cardiology Collaborative (PAC3) - Pediatric Cardiac Critical Care Consortium (PC4) project which focuses on variability in peri-operative care and was successful in reducing chest tube duration and length of stay across several common “benchmark” operations spanning the spectrum of complexity (8,31,32).
The implications of our findings are also important to consider with regard to other national efforts. These include the debate around the concept of regionalization of congenital heart care, whether through reducing the number of centers by centralizing care to regional centers of excellence or collaborative models of care between larger regional centers and local sites (4,5). While regionalization has taken place in other countries, there is currently no mandate or mechanism for such a program in the US and the topic remains controversial (33). Work by our group and others has often focused primarily on the potential impacts of limiting regionalization to those with higher risk disease as this is where most of the potential benefit was thought to exist, and would minimize other disruptions such as travel for patients and families (5). However, findings from our simulations suggest that this approach might miss a substantial proportion of the potential benefits. Similar findings were also reported by Welke and colleagues. In simulations conducted using 2012 data from 36 US states this group reported that regionalization of all patients resulted in a potential relative reduction in mortality of 17.4% with 116 fewer annual deaths vs. a relative reduction of 5.9% and 39 fewer deaths if limited only to the higher risk group (23). There are many other considerations in the debate regarding the advantages and disadvantages of a regionalized system as reported on by our group and others, such as whether theoretical improvements in outcomes from simulation models are really possible to achieve, impact on patient travel, coordination, and access to care, challenges with funding other necessary programs across children’s hospitals currently supported in part by revenue from congenital heart care, and education of trainees, among many others. (4,5,34,35). The findings from the present study, along with these many other important considerations, should be taken into account in ongoing efforts among professional societies in the field considering the development of consensus standards for congenital heart care (4).
Finally, our results also have implications regarding public reporting and transparent sharing of outcomes data with the public (6,7). Our findings suggest that reporting across the spectrum of risk is important, and that families and other stakeholders should be informed that variability can exist even for lower risk procedures. In addition, while current efforts focus on mortality, our data suggests that hospital variability in outcome extends to important morbidities as well, which should also be considered for reporting (6,7).
Limitations
The limitations of our study include that we focused only on a subset of potential outcomes where robust and case-mix adjusted data are available. There are other CHD outcomes that are important including endpoints beyond the in-hospital setting. While longitudinal data on survival and other important longer-term morbidities reported on from other countries are not yet widely available in the US, initiatives to improve our capabilities to collect these critical follow-up data may make these types of analyses possible in the future (36–38). Such data may also allow more granular analysis and weighting of the relative impact of the in-hospital complications studied in the present analysis. Ongoing efforts to improve case-mix adjustment methods may also enhance our ability to understand hospital variation in the future (36). For example, we are currently investigating whether machine learning methods may allow incorporation of additional important risk factors into case-mix adjustment models and whether this impacts assessments of hospital performance. In addition, while our sub-analyses focused on certain risk groups, more granular analysis of specific congenital heart disease diagnoses or operations may be of interest. Such analyses however are limited by lack of power with which to draw meaningful conclusions, and because of this we chose to focus on STAT categories. Finally, our simulations provide theoretical estimates which may or may not correlate with actual outcomes that could be achieved through efforts geared toward reducing national variation.
Conclusions
Our study demonstrates significant variability in congenital heart surgery outcomes across US hospitals which was evident across both morbidity and mortality metrics, and not confined solely to higher risk cases. These data suggest modified approaches to efforts to reduce variation and improve national outcomes, inclusive of all risk groups, are needed.
Supplementary Material
CLINICAL PERSPECTIVE.
What is new?
This study spanning 102 hospitals (n=84,407) across the Society of Thoracic Surgeons Congenital Heart Surgery Database suggests significant hospital variation in both morbidity and mortality metrics following congenital heart surgery.
Contrary to traditional thinking, a substantial portion of the variation and potential improvements that could be realized on a national scale through reducing variability were related to lower risk cases (STAT Category 1-3).
What are the clinical implications?
Our findings suggest modified approaches to policy and quality improvement efforts to reduce variation and improve national congenital heart surgery outcomes, inclusive of all risk groups, are needed.
Acknowledgments
Funding
This study was supported by funding from the National Heart, Lung, and Blood Institute (R01HL12226; PI Pasquali). Dr. Pasquali also receives support from the Janette Ferrantino Professorship.
NON-STANDARD ABBREVIATIONS AND ACRONYMS
- US
United States
- NPC-QIC
National Pediatric Cardiology Quality Improvement Collaborative
- STS-CHSD
Society of Thoracic Surgeons Congenital Heart Surgery Database
- STAT
Society of Thoracic Surgeons-European Association for Cardiothoracic Surgery
- CHSS
Congenital Heart Surgeons Society
- LOS
Post-operative length of stay
- MCMC
Markov Chain Monte Carlo
- CrI
Credible intervals
- PAC3
Pediatric Acute Care Cardiology Collaborative
- PC4
Pediatric Cardiac Critical Care Consortium
Footnotes
Presented at the American Heart Association 2019 Sessions. Recipient of the Outstanding Research Award in Pediatric Cardiology, and Paul Dudley White International Scholar Award (top scoring abstract from the United States)
Disclosures
None
References
- 1.Jacobs JP, He X, Mayer JE, Austin EH, Quintessenza JA, Karl TR, Vricella L, Mavroudis C, O’Brien SM, Pasquali SK, et al. Mortality trends in pediatric and congenital heart surgery: An analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg. 2016;102:1345–1352. [DOI] [PubMed] [Google Scholar]
- 2.Pasquali SK, Ohye RG, Lu M, Kaltman J, Caldarone CA, Pizarro C, Dunbar-Masterson C, Gaynor JW, Jacobs JP, Kaza AK, et al. , for the Pediatric Heart Network Investigators. Variation in perioperative care across centers for infants undergoing the Norwood Procedure. J Thorac Cardiovasc Surg. 2012;144:915–921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Jacobs JP, O’Brien SM, Pasquali SK, Jacobs ML, Lacour-Gayet FG, Tchervenkov CI, Austin EH, Pizarro C, Pourmoghadam KK, Scholl FG, et al. Variation in Outcomes for Benchmark Operations: An analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg. 2011;92:2184–2192. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Backer CL, Pasquali SK, Dearani JA. Improving national outcomes in congenital heart surgery: The time has come for regionalization of care. Circulation. 2020. February 12. doi: 10.1161/CIRCULATIONAHA.119.045542. [Epub ahead of print]. [DOI] [PubMed] [Google Scholar]
- 5.Pasquali SK, Dimick JB, Ohye RG. Time for a more unified approach to pediatric healthcare policy? The case of congenital heart care. JAMA. 2015;314:1689–1690. [DOI] [PubMed] [Google Scholar]
- 6.Gaynor JW, Pasquali SK, Ohye RG, Spray TL. Potential benefits and consequences of public reporting of pediatric cardiac surgery outcomes. J Thorac Cardiovasc Surg. 2017;153:904–907. [DOI] [PubMed] [Google Scholar]
- 7.Tweddell JS, Jacobs JP, Austin EH. Are there negative consequences of public reporting? The hype and the reality. J Thorac Cardiovasc Surg. 2017;153:908–911. [DOI] [PubMed] [Google Scholar]
- 8.Gaies M, Anderson J, Kipps A, Lorts A, Madsen N, Marino B, Jacobs J, Kasnic D, Lihn S, Lannon D, et al. Cardiac Networks United: An integrated pediatric and congenital cardiovascular research and improvement network. Cardiol Young. 2019;29:111–118. [DOI] [PubMed] [Google Scholar]
- 9.Gaies M, Pasquali SK, Banerjee M, Dimick JB, Birkmeyer JD, Zhang W, Alten JA, Chanani N, Cooper DS, Costello JM, et al. Improvement in pediatric cardiac surgical outcomes through interhospital collaboration. J Am Coll Cardiol. 2019;74:2786–2795. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Anderson JB, Beekman RH, Kugler JD, Rosenthal GL, Jenkins KJ, Klitzner TS, Martin GR, Neish SR, Brown DW, Mangeot C, et al. Improvement in interstage survival in a national pediatric cardiology learning network. Circ Cardiovasc Qual Outcomes. 2015;8:428–436. [DOI] [PubMed] [Google Scholar]
- 11.Overman DM, Jacobs ML, O’Brien JE, Kumar SR, Mayer JE, Ebel A, Clarke DR, Jacobs JP. Ten Years of Data Verification: The Society of Thoracic Surgeons Congenital Heart Surgery Database Audits. World J Pediatr Congenit Heart Surg. 2019;10:454–463. [DOI] [PubMed] [Google Scholar]
- 12.O’Brien SM, Jacobs JP, Shahian DM, Jacobs JL, Gaynor JW, Romano JC, Gaies MG, Hill KD, Mayer JE, Pasquali SK. Development of a Congenital Heart Surgery Composite Quality metric: Part 2–Analytic Methods. Ann Thorac Surg. 2019;107:590–596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.O’Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro C, Welke KF, Maruszewski B, Tobata Z, Miller WK, et al. An empirically based tool for analyzing mortality associated with congenital heart surgery. J Thorac Cardiovasc Surg. 2009;138:1139–1153. [DOI] [PubMed] [Google Scholar]
- 14.Pasquali SK, Shahian DM, O’Brien SM, Jacobs ML, Gaynor JW, Romano JC, Gaies MG, Hill KD, Mayer JE, Jacobs JP. Development of a Congenital Heart Surgery Composite Quality Metric: Part 1-Background and Conceptual Framework. Ann Thorac Surg. 2019;107:583–589. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Jacobs JP, Jacobs ML, Austin EH, Mavroudis C, Pasquali SK, Lacour–Gayet FG, Tchervenkov CI, Bacha EA, del Nido PJ, Fraser CD, et al. Quality Measures for Congenital and Pediatric Cardiac Surgery. World J Pediatr Congenit Heart Surg. 2012;3:32–47. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Do N, Hill KD, Wallace AS, Vricella L, Cameron D, Quintessenza J, Goldenberg N, Mavroudis C, Karl T, Pasquali SK, et al. Shunt failure risk factors and outcomes: An analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg. 2018;105:857–864. [DOI] [PubMed] [Google Scholar]
- 17.Karamlou T, Jacobs ML, Pasquali SK, He X, Hill KD, O’Brien SM, McMullan DM, Jacobs JP. Surgeon and center volume influence on outcomes after arterial switch operation: Analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg. 2014;98:904–911. [DOI] [PubMed] [Google Scholar]
- 18.Liu Q, Shepherd BE, Li C, Harrell FE Jr. Modeling continuous response variables using ordinal regression. Stat Med. 2017;36:4316–4335. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Jacobs JP, O’Brien S, Hill K, Kumar R, Austin E, Gaynor JW, Gruber P, Jonas R, Pasquali SK, Pizarro C, et al. Refining the Society of Thoracic Surgeons Mortality Risk Model with enhanced risk adjustment for chromosomal abnormalities, syndromes, and noncardiac congenital anatomic abnormalities. Ann Thorac Surg. 2019;108:558–566. [DOI] [PubMed] [Google Scholar]
- 20.Christiansen CL, Morris CN. Improving the statistical approach to health care provider profiling. Ann Intern Med 1997;127:764–768. [DOI] [PubMed] [Google Scholar]
- 21.Jacobs JP, O’Brien SM, Pasquali SK, Jacobs ML, Lacour-Gayet FG, Tchervenkov CI, Austin EH, Pizarro C, Pourmoghadam KK, Scholl FG, et al. Variation in outcomes for risk-stratified pediatric cardiac surgical operations: An analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg. 2012;94:564–571. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Welke KF, O’Brien SM, Peterson ED, Ungerleider RM, Jacobs ML, Jacobs JP. The complex relationship between pediatric cardiac surgical case volumes and mortality rates in a national clinical database. J Thorac Cardiovasc Surg. 2009;137:1133–1140. [DOI] [PubMed] [Google Scholar]
- 23.Welke KF, Pasquali SK, Lin P, Backer CL, Overman DM, Romano JC, Karamlou T. Regionalization of Congenital Heart Surgery in the United States. Semin Thorac Cardiovasc Surg. 2020;32:128–137. [DOI] [PubMed] [Google Scholar]
- 24.Stan Development Team. 2018. RStan: The R interface to Stan. R package version 2.19.3. Avaible at: http://mc-stan.org. Accessed 7/3/2020.
- 25.Bürkner P brms: An R Package for Bayesian Multilevel Models Using Stan. J Stat Softw. 2017;80:1–28. [Google Scholar]
- 26.Morales DL, Khan MS, Turek JW, Biniwale R, Tchervenkov CI, Rush M, Jacobs JP, Tweddell JS, Jacobs ML. Report of the 2015 Society of Thoracic Surgeons Congenital Heart Surgery Practice Survey. Ann Thorac Surg. 2017;103:622–628. [DOI] [PubMed] [Google Scholar]
- 27.Pasquali SK, Wallace AS, Gaynor JW, Jacobs ML, O’Brien SM, Hill KD, Gaies MG, Romano JC, Shahian DM, Mayer JE, et al. Congenital heart surgery case mix across North American centers and impact on performance assessment. Ann Thorac Surg. 2016;102;1580–1587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Pasquali SK, Li JS, Burstein DB, Sheng S, O’Brien SM, Jacobs ML, Jaquiss RDB, Peterson ED, Gaynor JW, Jacobs JP. The association of center volume with mortality and complications in pediatric heart surgery. Pediatrics. 2012;129:e370–e376. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hirsch JC, Gurney JG, Donohue JE, Gebremariam A, Bove EL, Ohye RG. Hospital mortality for Norwood and arterial switch operations as a function of institutional volume. Pediatr Cardiol. 2008;29:713–717. [DOI] [PubMed] [Google Scholar]
- 30.Kansy A, Zue Eulenburg C, Sarris G, Jacobs JP, Fragata J, Tobota Z, Ebels T, Maruszewski B. Higher programmatic volume in neonatal heart surgery is associated with lower early mortality. Ann Thorac Surg. 2018;105:1436–1440. [DOI] [PubMed] [Google Scholar]
- 31.Bates KE, Madsen NL, Khadr L, Gao Z, Crawford K, Gaies M, Graupe M, Hanke SP, Hlavacek AM, Morell E, et al. Center variation in chest tube duration and length of stay after congenital heart surgery. Ann Thorac Surg. 2020;110:221–227. doi: 10.1016/j.athoracsur.2019.09.078. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Bates KE, Madsen NL, Khadr L, Graupe M, Hlavacek AM, Morell E, Pasquali SK, Russell JL, Schachtner SK, Tanel RE, et al. Successful reduction of postoperative chest tube duration and length of stay after congenital heart surgery: A multicenter Pediatric Acute Care Cardiology Collaborative (PAC3) and Pediatric Cardiac Critical Care Consortium (PC4) Improvement Project. Circulation. 2019;140:A11376. [Google Scholar]
- 33.Lundstrom NR, Berggen H, Bjorkhem G, Jogi P, Sunnegardh J. Centralization of pediatric heart surgery in Sweden. Pediatr Cardiol. 2000;21:353–357. [DOI] [PubMed] [Google Scholar]
- 34.Backer CL, Karamlou T, Welke KF. More Evidence for Regionalization. J Am Coll Cardiol. 2019;74:2919–2920. [DOI] [PubMed] [Google Scholar]
- 35.Burki S, Fraser CD. Larger centers may produce better outcomes: Is regionalization in congenital heart surgery a superior model? Semin Thorac Cardiovasc Surg Pediatr Card Surg Annu. 2016;19:10–13. [DOI] [PubMed] [Google Scholar]
- 36.Pasquali SK, Banerjee M, Romano JC, Normand ST. Hospital performance assessment in congenital heart surgery: Where do we go from here? Ann Thorac Surg. 2020;109:621–626. [DOI] [PubMed] [Google Scholar]
- 37.Larsen SH, Olsen M, Emmertsen K, Hjortdal VE. Interventional treatment of patients with congenital heart disease: Nationwide Danish experience over 39 years. J Am Coll Cardiol. 2017;69:2725–2732. [DOI] [PubMed] [Google Scholar]
- 38.Pasquali SK, Ravishankar C, Romano JC, Kane K, Viers S, Kennedy A, Burnham N, Lowery R, Uzark K, Retzloff L, et al. Design and initial results of a programme for routine standardised longitudinal follow-up after congenital heart surgery. Cardiol Young. 2016;26:1590–1596. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.