Abstract
Mortality after pediatric cardiac surgery varies among centers. Previous research suggests that surgical volume is an important predictor of this variation. This report characterizes the relative contribution of patient factors, center surgical volume, and a volume-independent center effect on early postoperative mortality in a retrospective cohort study of North American centers in the Pediatric Cardiac Care Consortium (up to 500 cases/center/year). From 1982 to 2007, 49 centers reported 109,475 operations, 85,023 of which were analyzed using hierarchical multivariate logistic regression analysis. Patient characteristics varied significantly among the centers. The adjusted odds ratio (OR) for mortality decreased more than 10-fold during the study period (1982 vs. 2007: OR, 12.27, 95 % confidence interval [CI], 8.52–17.66; p < 0.0001). Surgical volume was associated inversely with odds of death (additional 100 cases/year: OR, 0.84; 95 % CI, 0.78–0.90; p < 0.0001). In the analysis of interactions, this effect was fairly consistent across age groups, risk categories (except the lowest), and time periods. However, a volume-independent center effect contributed substantially more to the risk model than did the volume. The Risk Adjusted Classification for Congenital Heart Surgery, version 1 (RACHS-1) risk category remains the strongest predictor of postoperative mortality through the 25-year study period. In conclusion, center-specific variation exists but is only partially explained by operative volume. Low-risk operations are safely performed at centers in all volume categories, whereas regionalization or other quality improvement strategies appear to be warranted for moderate- and high-risk operations. Potentially preventable mortality occurs at centers in all volume categories studied, so referral or regionalization strategies must target centers by observed outcomes rather than assume that volume predicts quality.
Keywords: Congenital heart disease, Infants and children, Quality improvement, Surgical volume
Significant progress in pediatric cardiac surgery has raised expectations and generated a need to monitor performance. In adult cardiac surgery, a correlation between volume and mortality has long been recognized [21] and used to advocate for regionalization. Pediatric cardiac surgical mortality also varies among centers and in some studies correlates with volume [9, 12, 14, 27, 29], although other factors such as patient characteristics and referral patterns may be involved. This variation is important to patients, families, physicians, and policymakers [4, 5]. However, the impact of volume and other center-specific effects on pediatric cardiac postoperative mortality is incompletely understood, and additional research in this area has been encouraged [7].
Evaluating institutional performance of pediatric cardiac surgery is difficult for three reasons related to the diversity and complexity of congenital heart disease. First, coding systems do not fully capture the complexity of the diagnoses and procedures involved. Second, the wide spectrum of risk factors complicates adjustment for case mix. Third, the relative rarity of each procedure limits statistical power for detecting institution-level differences [27, 30].
The Pediatric Cardiac Care Consortium (PCCC), a multi-institutional registry, has collected patient-level data since 1982 to support quality improvement in congenital heart surgery [19]. Through 2007, the PCCC includes more than 137000 patients from centers performing up to 500 operations/year. Detailed information on invasive cardiac procedures, as well as cardiac and noncardiac diagnoses, permits reliable risk adjustment. Many previous studies have described outcomes after pediatric cardiac operations but frequently are limited to specific anomalies, individual large centers of excellence, administrative data, or short time periods and thus cannot provide a detailed longitudinal assessment using clinical outcome data.
It is important to characterize the factors affecting outcomes in centers performing up to 500 operations per year because many pediatric cardiac operations are—and will continue to be—performed in such centers [1, 3, 5, 25]. Furthermore, others have suggested that volume-related variation is most evident in this size range [29]. Therefore, we analyzed the relationship of surgical volume and other risk factors to postoperative mortality at PCCC centers and determined whether these relationships changed with time.
Methods
Data Source: The Pediatric Cardiac Care Consortium
The PCCC registry collects detailed clinical data from centers performing pediatric cardiac procedures [19]. All cardiac operations (except isolated ductal ligation in preterm infants weighing less than 2.5 kg) are reported prospectively by the centers. Diagnosis and procedure coding takes place at the core facility, and the PCCC centers retain the right to study their cases locally or via other registries. A subset of the centers likely contributed to an earlier publication [29] addressing similar questions over a shorter time period.
For this study, we excluded centers outside North America, a center that transferred patients to outlying hospitals for postoperative recovery at a rate 10-fold higher than expected, and any years that a center contributed incomplete data or performed fewer than 10 operations.
Risk Adjustment and Independent Variables
The Risk Adjusted Classification for Congenital Heart Surgery, version 1 (RACHS-1) is a validated and widely used risk-adjustment system that classifies congenital cardiac operations into six categories by expected early mortality rates [13]. Risk category 1 operations have the lowest risk of death and risk category 6 operations the highest. Because category 5 operations are very rare (~0.1 % overall), categories 5 and 6 were combined (termed “category 5&6”).
We also analyzed patient sex, age at operation (partitioned at 28 days and 1 year), year of operation (with 1982–2007 divided into five time periods for most analyses), and center annualized surgical volume (calculated for each time period as the total operations performed divided by the number of participating years of that time period). We modeled volume separately as continuous (linear on the log-odds scale) and as categorical (divided into approximate tertiles).
Patients, Procedures, and Outcomes
We calculated surgical volume using all operations performed for pediatric or adult congenital heart disease, with or without cardiopulmonary bypass, in each year or time period. After computing surgical volume, we excluded adults, hospital admissions ending in transfer to another center, and admissions containing any procedures not classifiable by the RACHS-1 system (expected to be 11–14 % of operations) from the multivariate analysis. We included patients with multiple surgical admissions, with the admissions treated as independent.
Choosing the appropriate end point after pediatric cardiac surgery is difficult [8, 11, 28, 31]. We used early postoperative mortality, defined as in-hospital death within 30 days, as the primary outcome measure. When a single hospital admission involved multiple operations, we included only one operation, chosen by highest RACHS-1 score [13] first, then by earliest date.
Study Design and Statistical Analysis
The original study design specified multivariate analysis of the relationship between surgical volume and various patient-specific factors on mortality, quantification of relative contributions to the model, and analysis of interactions. The center random effect was included per statistical routine to account for possible confounding. After seeing that the center effect contributed more to the model than the volume term, we undertook additional analyses to characterize better the behavior of the center effect.
We summarized categorical variables in frequency tables and compared them using Chi square tests. We used mixed-effect logistic regression models to compare the association between surgical volume and mortality, with and without adjustment for additional covariates. We addressed correlation within a center by including a random effect (assumed to follow a normal distribution with a mean of 0) that represented the deviation of a center from the overall adjusted log-odds of death.
In the multivariate model, we included the following predetermined clinically relevant covariates: year (partitioned into 5-year intervals), volume, center, risk category, age (partitioned into predefined age groups), and sex. We analyzed models with interaction terms to evaluate pre-specified interactions between risk category and era, volume and risk category, volume and era, and volume and age group. Each interaction model also included the main effects from the multivariate model.
We report odds ratios (ORs) for death, as defined earlier, with 95 % confidence intervals (CIs). We constructed approximate F tests for each factor in a nonlinear mixed-effects logistic regression model. We also computed approximate standard errors as well as the corresponding t statistics and p values for the factor values using the delta method. All the tests were two-sided using a cutoff of 5 % without adjustment for multiple comparisons.
To assess the relative contribution of each variable to postoperative mortality, we used the likelihood ratio test (LRT) and the Akaike information criterion (AIC) in uni-and multivariate models. In the multivariate models, we removed each variable (one at a time) to assess the increase in AIC relative to the full model [16, 24].
We performed analyses using SAS version 9.2 (SAS Institute, Cary, NC) and created figures using R version 2.14.1 (R Foundation for Statistical Computing) and Microsoft Excel.
Sensitivity Analyses
We computed the proportion of observed deaths minus the mean predicted probability of death for each center–era combination and plotted these against annualized volume to assess the potential of nonlinearity for annualized volume and to identify influential centers. We recomputed models for comparison after removing one center in which volume had dropped sharply. Considering that duration of hospitalization before death might vary by center, we repeated the multivariate model with the end point of all inhospital mortality rather than 30-day in-hospital mortality. We evaluated an analysis using only the first admission per patient to compare the results after removing potential intra-patient correlation. To assess the effect of treating age and year as categorical variables and of annualizing volume, we computed a multivariate model using age and year as continuous variables and volume based on single years. Finally, we repeated the analysis including procedures unclassifiable by RACHS-1 as a separate category in the regression model.
Results
The PCCC registry contains data from 57 participant centers for some or all of the period 1982–2007. These centers performed 118,084 operations for 90,124 patients during 112,030 hospital admissions. After applying exclusion criteria, 49 centers contributed 109,475 operations for volume calculations and 85,023 admissions for detailed statistical analysis (Table 1, Figs. 1, 2). Centers performed 11–534 cases per year overall (mean, 172; median, 154; standard deviation [SD], 96) and 13–458 cases per year when annualized by time period (mean, 161; median, 144; SD, 91) (Fig. 3) with tertile cut points of 109 and 188 cases per year. We rounded the SD to 100 cases per year when computing ORs for an approximate 1 SD increase (in the continuous–volume analyses), and categorized volume as small (10–99 cases/year), medium (100–199 cases/year), or large (≥200 cases/year) by approximate tertiles.
Table 1.
Characteristics of hospital admissions
| Raw dataseta n (%) | Analysis cohort n (%) | |||
|---|---|---|---|---|
| Total | 112,001 | (100) | 85,023 | (100) |
| Females | 51,539 | (46.0) | 39,197 | (46.1) |
| Males | 60,462 | (54.0) | 45,826 | (53.9) |
| Newborns | 18,357 | (16.4) | 14,986 | (17.6) |
| Infants | 33,643 | (30.0) | 28,783 | (33.9) |
| Children | 53,766 | (48.0) | 41,254 | (48.5) |
| Adults | 6,235 | (5.6) | – | – |
| Outcome of hospital admission | ||||
| Discharge | 103,585 | (92.5) | 79,786 | (93.8) |
| Death | 6,806 | (6.1) | 5,237 | (6.2) |
| Transfer | 1,610 | (1.4) | – | – |
| Risk category of highest-risk operation at the admission | ||||
| Unclassified | 13,171 | (11.8) | – | – |
| 1 | 20,625 | (18.4) | 17,935 | (21.1) |
| 2 | 32,338 | (28.9) | 28,389 | (33.4) |
| 3 | 35,178 | (31.4) | 29,434 | (34.6) |
| 4 | 7,688 | (6.9) | 6,563 | (7.7) |
| 5&6 | 3,001 | (2.7) | 2,702 | (3.2) |
| Operations per admission | ||||
| 1 | 106,829 | (95.4) | 82,663 | (97.2) |
| 2 | 4,486 | (4.0) | 2,148 | (2.5) |
| 3 | 565 | (0.5) | 185 | (0.2) |
| 4 | 87 | (0.1) | 24 | (< 0.1) |
| 5 | 25 | (< 0.1) | 3 | (< 0.1) |
| 6 or more | 9 | (< 0.1) | ||
Excluding 29 admissions with missing or inconsistent data
Fig. 1.
STROBE-style flow diagram. Note that all operations in eligible years (asterisk) count toward institutional volume, whereas multivariate analysis was performed on a subset, as described in the text
Fig. 2.
Annual volume and contribution of each center to the cohort. Centers are arranged (x-axis) by mean annual volume (bars, left y-axis) regardless of the number of years of participation. The percentage contribution of each center (thick and thin lines, right y-axis) accounts for length of participation
Fig. 3.
Institutional surgical volumes in the PCCC. a Annual center activity (color) for all 57 PCCC centers (y-axis) over time (x-axis). The duration of participation varied among the individual centers (NA = center’s data incomplete or unavailable for that year) and eight centers were excluded. *High transfer rate, **outside North America, ***fewer than ten cases/year. b Statistical distribution of the 49 included centers’ volumes. The x-axis indicates the years and number of centers in each time period, and the y-axis indicates surgical volume annualized by time period. Boxes represent median and IQR (interquartile range, 25–75th percentiles); whiskers represent range within 1.5 × IQR; and circles represent values outside 1.5 × IQR
For most variables, operations were not uniformly distributed among volume categories and time periods (Tables 2, 3). Younger patients and higher-risk procedures were more common at larger centers, with the largest discrepancy in risk category 5&6, comprising 1.8 % of operations at small centers, 2.6 % at medium centers, and 3.8 % at large centers (p < 0.0001).
Table 2.
Characteristics of the study cohort by volume category after removal of excluded data
| All n (%) | Volume category (cases/year within time period [TP])
|
|||||||
|---|---|---|---|---|---|---|---|---|
| Small (10–99) n (%) | Medium (100–199) n (%) | Large (> 200) n (%) | ||||||
| Center × TPs | 158 | (100) | 40 | (25.3) | 72 | (45.6) | 46 | (29.1) |
| Operations | 85,023 | (100) | 7,162 | (8.4) | 33,740 | (39.7) | 44,121 | (51.9) |
| Females | 39,197 | (46.1) | 3,331 | (46.5)a,b | 15,571 | (46.1)a,c | 20,295 | (46.0)b,c |
| Males | 45,826 | (53.9) | 3,831 | (53.5)a,b | 18,169 | (53.9)a,c | 23,826 | (54.0)b,c |
| Neonates | 14,986 | (17.6) | 1,121 | (15.7) | 5,894 | (17.5)c | 7,971 | (18.1)c |
| Infants | 28,783 | (33.9) | 2,285 | (31.9)a | 11,111 | (32.9)a | 15,387 | (34.9) |
| Children | 41,254 | (48.5) | 3,756 | (52.4) | 16,735 | (49.6) | 20,763 | (47.1) |
| Risk category | ||||||||
| 1 | 17,935 | (21.1) | 1,822 | (25.4) | 7,592 | (22.5) | 8,521 | (19.3) |
| 2 | 28,389 | (33.4) | 2,488 | (34.7)a,b | 11,322 | (33.6)a,c | 14,579 | (33.0)b,c |
| 3 | 29,434 | (34.6) | 2,246 | (31.4) | 11,505 | (34.1) | 15,683 | (35.5) |
| 4 | 6,563 | (7.7) | 480 | (6.7)a | 2,442 | (7.2)a | 3,641 | (8.3) |
| 5&6 | 2,702 | (3.2) | 126 | (1.8) | 879 | (2.6) | 1,697 | (3.8) |
| Outcome | ||||||||
| Discharge | 79,786 | (93.8) | 6,717 | (93.8) | 31,398 | (93.1) | 41,671 | (94.4) |
| Death | 5,237 | (6.2) | 445 | (6.2) | 2,342 | (6.9) | 2,450 | (5.6) |
The proportion of each variable differs significantly (Bonferroni-corrected p < 0.002) between volume categories except as indicated
No difference between small and medium centers
No difference between small and large centers
No difference between medium and large centers
Table 3.
Characteristics of the study cohort by time period (TP) after removal of excluded data
| All n (%) | TP (years)
|
|||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 (1982–1987) n (%) | 2 (1988–1992) n (%) | 3 (1993–1997) n (%) | 4 (1998–2002) n (%) | 5 (2003–2007) n (%) | ||||||||
| Center × TPs | 158 | (100) | 18 | (11.4) | 27 | (17.1) | 36 | (22.8) | 43 | (27.2) | 34 | (21.5) |
| Patients | 85,023 | (100) | 8,162 | (9.6) | 13,600 | (16.0) | 19,818 | (23.3) | 22,279 | (26.2) | 21,164 | (24.9) |
| Females | 39,197 | (46.1) | 3,775 | (46.3)a,b,c,d | 6,331 | (46.6)a,e,f,g | 9,319 | (47.0)b,e,h | 10,274 | (46.1)c,f,h,j | 9,498 | (44.9)d,g,j |
| Males | 45,826 | (53.9) | 4,387 | (53.7)a,b,c,d | 7,269 | (53.4)a,e,f,g | 10,499 | (53.0)b,e,h | 12,005 | (53.9)c,f,h,j | 11,666 | (55.1)d,g,j |
| Neonates | 14,986 | (17.6) | 1,018 | (12.5) | 2,165 | (15.9)e | 3,407 | (17.2)e | 4,190 | (18.8)j | 4,206 | (19.9)j |
| Infants | 28,783 | (33.9) | 2,214 | (27.1)a | 3,956 | (29.1)a | 6,286 | (31.7) | 7,899 | (35.5) | 8,428 | (39.8) |
| Children | 41,254 | (48.5) | 4,930 | (60.4) | 7,479 | (55.0) | 10,125 | (51.1) | 10,190 | (45.7) | 8,530 | (40.3) |
| Risk category | ||||||||||||
| 1 | 17,935 | (21.1) | 1,999 | (24.5)a,b | 3,421 | (25.2)a,e | 4,827 | (24.4)b,e | 4,509 | (20.2) | 3,179 | (15.0) |
| 2 | 28,389 | (33.4) | 2,180 | (26.7)a | 3,793 | (27.9)a | 6,441 | (32.5) | 7,990 | (35.9) | 7,985 | (37.7) |
| 3 | 29,434 | (34.6) | 3,334 | (40.8) | 5,078 | (37.3) | 6,385 | (32.2)h | 7,231 | (32.5)h | 7,406 | (35.0) |
| 4 | 6,563 | (7.7) | 547 | (6.7)a,b,c | 1,013 | (7.4)a,e,f | 1,446 | (7.3)b,e,h | 1,651 | (7.4)c,f,h | 1,906 | (9.0) |
| 5&6 | 2,702 | (3.2) | 102 | (1.2) | 295 | (2.2) | 719 | (3.6)h,i | 898 | (4)h | 688 | (3.3)i |
| Outcome | ||||||||||||
| Discharge | 79,786 | (93.8) | 7,341 | (89.9) | 12,467 | (91.7) | 18,394 | (92.8) | 21,135 | (94.9) | 20,449 | (96.6) |
| Death | 5,237 | (6.2) | 821 | (10.1) | 1,133 | (8.3) | 1,424 | (7.2) | 1,144 | (5.1) | 715 | (3.4) |
The proportion of each variable differs significantly (Bonferroni-corrected p < 0.0006) between TPs except as indicated: no pairwise difference between TPs.
1 & 2,
1 & 3,
1 & 4,
1 & 5,
2 & 3,
2 & 4,
2 & 5,
3 & 4,
3 & 5,
4 & 5
Table 4 shows the distribution of admissions and deaths by risk category for the whole study period and for the most recent era (5.2 % of overall admissions and 2.3 % of 2003–2007 admissions ending in death). Unadjusted mortality varied widely across time periods and risk categories but generally decreased throughout the study period, except in the lowest risk category, in which it has remained low and stable since the late 1980s (Fig. 4).
Table 4.
Raw data by risk category overall and for the most recent era
| Risk category | Entire study period (1982–2007)
|
Time period 5 (2003–2007)
|
||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Admissions n (%) | Deaths n (%) | Mortality (%) | Admissions n (%) | Deaths n (%) | Mortality (%) | |||||
| 1 | 17,935 | (21.1) | 99 | (2.2) | 0.6 | 3,179 | (15.0) | 14 | (2.9) | 0.5 |
| 2 | 28,389 | (33.4) | 555 | (12.6) | 2.0 | 7,985 | (37.7) | 56 | (11.6) | 0.7 |
| 3 | 29,434 | (34.6) | 1,892 | (42.9) | 6.4 | 7,406 | (35.0) | 197 | (40.6) | 2.7 |
| 4 | 6,563 | (7.7) | 951 | (21.5) | 14.5 | 1,906 | (9.0) | 115 | (23.7) | 6.0 |
| 5&6 | 2,702 | (3.2) | 918 | (20.8) | 34.0 | 688 | (3.3) | 103 | (21.2) | 15.0 |
| Total | 85,023 | (100) | 4,415 | (100) | 5.2 | 21,164 | (100) | 485 | (100) | 2.3 |
Fig. 4.
Unadjusted mortality by time and risk category. The percentage of admissions ending in death each year is shown for the overall cohort and by risk category
Table 5 shows the association of center volume, risk category, time period, age group, and sex on 30-day in-hospital mortality. Univariate analysis identified significant effects for most of the variables. Patient sex was insignificant in the univariate analysis, but females had significantly higher mortality in the adjusted multivariate model (OR, 1.20; 95 % CI, 1.13–1.29; p < 0.0001). In the multivariate model, risk category, age at operation, and time period contributed more to the prediction of death after pediatric cardiac surgery than center volume, the center random effect, or patient sex (comparing relative contributions to the logarithmic likelihood ratio Chi square of each variable; Table 6).
Table 5.
Effects on mortality by uni- and multivariate analysis
| Univariate analysis
|
Continuous-volume model
|
|||||
|---|---|---|---|---|---|---|
| OR | 95 % CI | p Value | OR | 95 % CI | p value | |
| Volume (per 100 cases/year) | 0.72 | (0.67–0.77) | < 0.0001 | 0.84 | (0.78–0.90) | < 0.0001 |
| Risk category 5&6 vs 1 | 111.9 | (89.8–139.6) | < 0.0001 | 67.0 | (53.1–84.5) | < 0.0001 |
| Risk category 4 vs 1 | 33.3 | (26.9–41.4) | < 0.0001 | 23.1 | (18.5–28.9) | < 0.0001 |
| Risk category 3 vs 1 | 13.1 | (10.6–16.1) | < 0.0001 | 10.1 | (8.2–12.5) | < 0.0001 |
| Risk category 2 vs 1 | 3.9 | (3.1–4.8) | < 0.0001 | 3.4 | (2.7–4.2) | < 0.0001 |
| Neonates vs children | 8.5 | (7.8–9.3) | < 0.0001 | 3.3 | (3.0–3.6) | < 0.0001 |
| Infants vs children | 2.5 | (2.3–2.8) | < 0.0001 | 2.3 | (2.1–2.5) | < 0.0001 |
| Neonates vs infants | 3.4 | (3.1–3.6) | < 0.0001 | 1.4 | (1.33–1.6) | < 0.0001 |
| Time period 1 vs 5 | 4.4 | (3.8–5.0) | < 0.0001 | 6.5 | (5.6–7.6) | < 0.0001 |
| Time period 2 vs 5 | 3.3 | (2.9–3.7) | < 0.0001 | 4.2 | (3.7–4.8) | < 0.0001 |
| Time period 3 vs 5 | 2.7 | (2.4–3.0) | < 0.0001 | 3.1 | (2.7–3.5) | < 0.0001 |
| Time period 4 vs 5 | 1.8 | (1.6–2.0) | < 0.0001 | 1.9 | (1.7–2.1) | < 0.0001 |
| Females vs males | 0.98 | (0.92–1.04) | 0.4993 | 1.20 | (1.13–1.29) | < 0.0001 |
OR odds ratio, CI confidence interval
Table 6.
Significance of each variable included in the final model
| Variable | DOF | Univariatea,c
|
Multivariateb,c
|
||
|---|---|---|---|---|---|
| LRT | AIC | LRT | AIC | ||
| Sex | 1 | 419 | 34,302 | 31 | 27,529 |
| Risk category | 4 | 5,648 | 29,079 | 3137 | 30,629 |
| Age group | 2 | 3,309 | 31,414 | 676 | 28,172 |
| Time period | 4 | 1,020 | 33,707 | 768 | 28,260 |
| Surgical volume | 1 | 505 | 34,216 | 25 | 27,523 |
| Center effect | 1 | 419 | 34,300 | 181 | 27,679 |
DOF degrees of freedom, LRT likelihood ratio test, AIC Akaike information criterion
For the univariate analyses, the LRT is used to compare the intercept-only model (AIC, 34,300) with the corresponding univariate model: lower AIC indicates that the model fits the data better
For the multivariate analyses, the LRT is used to compare the full model including all variables (AIC, 27,500) with a model omitting one variable at a time: higher AIC indicates that the removed variable fits the data better
For both models, higher LRT indicates a more significant impact of a variable
Adjusted postoperative mortality decreased more than 10-fold over the study period (analyzing surgical year as a categorical variable, 1982 vs. 2007: OR, 12.27; 95 % CI, 8.52–17.66; p < 0.0001) (Fig. 5a). The decrease occurred in almost all risk categories except the lowest, in which mortality has been low and stable since the early 1990s (Fig. 5b). The RACHS-1 score discriminated risk categories well, except that we did not find a statistically significant difference in risk of death between categories 1 and 2 during the last period (2003–2007; Fig. 5c).
Fig. 5.
Mortality by time and risk category. Risk-adjusted mortality (x-axis, log scale) is shown as odds ratios (circles) and 95 % confidence intervals (whiskers). a Adjusted mortality over time (reference: year 2007). b Adjusted mortality across time periods (TPs) by risk category (reference: TP 5). c Adjusted mortality across risk categories by TP (reference: risk category 1)
We evaluated center-specific mortality variation in several ways. Individual center effects, representing each center’s deviation from overall odds of death adjusted for patient factors, varied substantially but with evidence of relation to volume (Fig. 6a). Multivariate analysis including institutional volume as a continuous variable confirmed a significant inverse correlation between volume and mortality (OR, 0.84 per additional 100 operations/year; 95 % CI, 0.78–0.90; p < 0.0001). The volume–mortality relationship persisted across age groups (p = 0.3494 for interaction) but varied somewhat by risk categories (p = 0.0049 for interaction) and time periods (p = 0.0023 for interaction) (Table 7). Interestingly, volume had no effect in risk category 1 (OR, 0.99; 95 % CI, 0.79–1.25).
Fig. 6.
Individual center mortality by volume. The 49 analyzed centers are arranged by mean annual surgical volume (y-axis), with odds of mortality (x-axis, log scale) adjusted for a risk category, time period, age group, and sex, or for b risk category, time period, age group, sex, and volume. Symbols are as above, with centers having significantly increased or decreased odds of mortality highlighted respectively in red or green. For some centers, volume was adjusted up or down slightly to prevent overlap in the figure. This did not change the order of the centers or alter any center’s volume by more than two cases per year
Table 7.
The volume–mortality relationship
| OR | 95 % CI | p value | |
|---|---|---|---|
| Overall: 100 case/year increase* | 0.84 | (0.78–0.9) | < 0.0001 |
| Risk category by volume (p = 0.0049 for interaction) | |||
| 100 C/Y increase: risk category 1 | 0.98 | (0.78–1.23) | 0.8644 |
| 100 C/Y increase: risk category 2 | 0.74 | (0.66–0.83) | < 0.0001 |
| 100 C/Y increase: risk category 3 | 0.89 | (0.82–0.97) | 0.0059 |
| 100 C/Y increase: risk category 4 | 0.85 | (0.77–0.93) | 0.0012 |
| 100 C/Y increase: risk category 5&6 | 0.77 | (0.69–0.86) | < 0.0001 |
| Time period by volume (p = 0.0023 for interaction) | |||
| 100 C/Y increase: 1982–1987 | 0.83 | (0.73–0.95) | 0.0063 |
| 100 C/Y increase: 1988–1992 | 0.77 | (0.69–0.87) | < 0.0001 |
| 100 C/Y increase: 1993–1997 | 0.77 | (0.70–0.85) | < 0.0001 |
| 100 C/Y increase: 1998–2002 | 0.93 | (0.85–1.01) | 0.0994 |
| 100 C/Y increase: 2003–2007 | 0.78 | (0.69–0.88) | 0.0002 |
| Age group by volume (p = 0.3494 for interaction) | |||
| 100 C/Y increase: neonates | 0.85 | (0.79–0.93) | 0.0003 |
| 100 C/Y increase: infants | 0.84 | (0.77–0.92) | 0.0002 |
| 100 C/Y increase: children | 0.80 | (0.72–0.88) | < 0.0001 |
Values are derived from four separate models (1 main effect model and 3 interaction models), each controlling for correlation within the center but not within the patients
OR odds ratio, CI confidence interval, C/Y cases per year
After adjustment for volume, substantial variability in individual center effects persisted (Fig. 6b). The center effect followed a normal distribution (by assumption), with a mean of 0 and an SD of 0.29 (95 % CI, 0.21–0.37; p < 0.0001). The inclusion of volume in the multivariate model reduced the variability of the center effect by 20.2 %. Center-specific variation still remained significant (p < 0.0001 by LRT; Table 6).These findings demonstrate the importance of center-specific effects above and beyond institutional volume.
We performed sensitivity analyses to assess the stability of our results. We found neither statistical nor clinical differences in the conclusions of analyses that included only the first admission per patient, included operations with an undefined RACHS-1 risk category, used the end point of overall in-hospital mortality, excluded outliers, or treated year and age as continuous variables and did not annualize volume.
Discussion
Using multi-institutional data collected prospectively over 25 years of pediatric cardiac surgery, we analyzed trends in postoperative mortality and quantified the influence of risk factors, including institutional volume. Overall, survival after pediatric cardiac surgery improved substantially, consistent with a previous study in England [2]. The decrease in mortality occurred across all age and risk groups except the minimal risk category 1, which reached a plateau. Over time, the gaps between different risk categories narrowed, but the RACHS-1 score remained by far the best predictor of postoperative mortality. There is residual patient-specific risk not captured by RACHS-1, with younger age and female sex related to an increased risk of death.
We demonstrated a statistically significant protective effect of increased surgical volume on postoperative mortality. The effect was clinically relevant (relative odds reductions generally 10–30 %, similar to a previous report [22]) but modest compared with that of other variables. Others have found that the volume–mortality relationship varies substantially by patient age [21, 23] and may be attenuated [3, 7] or even absent [26] in the modern era; our analysis did not corroborate these findings. However, we did show that the volume–mortality relationship varied significantly by risk category. The complete absence of effect for the lowest risk category suggests that operations in this category are safely performed at smaller centers, consistent with most previous reports [10, 21, 29]. Even for higher-risk operations, the absolute effect of volume is relatively small at current mortality rates.
Previous reports suggest a critical threshold for surgical volume (variously 75–300 cases/year). We did not identify such a threshold within the volume range available in the PCCC, although it is possible that one exists outside this range (i.e., affecting extremely large centers). Regardless, a substantial fraction of patients do not have easy access to these highly specialized centers [1, 3, 5].
To assess the generalizability of results from the PCCC, we compared the PCCC with other data sets [25] and found that (1) case mix was fairly comparable with a wide range of published data sets (two other clinical registries, three administrative databases, and one very large center of excellence) and that (2) in recent years, PCCC centers performed about 10–30 % of the operations in the United States. These findings underscore the importance of understanding and improving outcomes from such centers.
Institutional surgical volume has been proposed as a quality indicator for pediatric cardiac surgery [18]. Our study found volume to be a relatively weak predictor of a center’s mortality rate, similar to previous reports from other databases [7, 9, 14, 29]. We demonstrated that volume-independent center-specific effects contribute more to the statistical model than volume. These findings are consistent with those of other studies [9, 23, 27, 29] suggesting that volume should not be used in isolation to predict quality at the level of individual institutions.
Regionalization has been considered as a pediatric cardiac surgery quality improvement strategy [4]. This is a complex policy issue, but if undertaken, our results suggest that centers must be targeted by direct analysis of quality rather than by volume alone. The implications for selective referral strategies [1] are similar. Our study provides reference data useful for identifying patient subgroups with the greatest potential benefit.
Study Strengths and Limitations
The strengths of our study included the large data set, the ability to characterize changes over time, the use of clinical rather than administrative data, and attention to recommendations regarding the statistical approach to volume–outcome analysis [15, 16, 22, 24].
The study limitations were mainly those typical of retrospective registry-based studies. We did not control for all patient factors, such as prematurity, weight, comorbidities, critical illness severity, and sociodemographic variables, or institutional factors such as team composition, individual surgeon training and experience, type of facility (e.g., free-standing children’s hospital, general hospital), transfusion practices, infection control, and care pathways, that may affect outcomes. Such factors should be studied in the future because they may be critical to explaining the systematic but volume-independent difference between centers but unfortunately are not available in our data set.
We grouped operations by RACHS-1 risk category and therefore cannot evaluate whether our results apply uniformly to all operations in each risk category or whether alternate approaches to risk adjustment would have affected our results, although the discrimination difference between RACHS-1, the Society of Thoracic Surgeons–European Association for Cardiothoracic Surgery (STS-EACTS) system, and the Aristotle Basic Complexity score is known to be small [20]. We calculated volume over discrete intervals, starting before and ending after each case, rather than on a rolling basis. This is unlikely to have substantially altered our findings because volumes were generally consistent over time except at one center that was therefore the subject of a sensitivity analysis.
The voluntary and changing makeup of the PCCC may limit our inference, and with only a few centers performing more than 350 operations per year, this study could not address the volume–mortality relationship at very large centers. However, a prior study including larger centers suggested that the volume–mortality effect is essentially limited to centers performing fewer than 300 operations/year [29]. Finally, unmeasured referral patterns may have confounded the results of any observational volume–outcome analysis [17], in which case, volume and mortality would indeed be correlated but not causally related.
Implications and Conclusions
Our findings have implications for future research. Studies of volume–outcome relationships and other center-specific variations set the stage for investigation of factors that may mediate these relationships and for targeted quality improvement. However, improvement efforts using mortality as the end point are hampered by limited statistical power [6]. Future research needs to identify end points that can support rapid-cycle quality improvement and incorporate other important outcomes including nonfatal complications, reoperation, neurologic and other noncardiac morbidity, and cost effectiveness.
In this large multi-institutional registry, mortality after pediatric cardiac surgery has declined substantially over the past 25 years. Mortality remains an important end point for higher-risk operations, but new end points should be developed and validated across the spectrum of pediatric cardiac operations. Center-specific variation exists even after risk adjustment, suggesting that some of the postoperative mortality is preventable, but institutional volume only partially explains this variation. Careful regionalization based on measured outcomes rather than volume may have a role in improving quality. However, we believe it may be better to identify the factors responsible for volume-independent center-specific variation and deploy them broadly to improve outcomes at centers of all sizes.
Acknowledgments
We thank the program directors and data collection coordinators from the participating PCCC institutions. Without their effort and dedication, this work could not have been completed. This study had no direct funding. Jeffrey M. Vinocur completed this work during his pediatric cardiology training, which was partially supported by a Franck Gougeon Fellowship. Mr. Gougeon was not involved in any aspect of the study. John Connett receives compensation for being a reviewer for the Journal of Cardiac Failure and honoraria for being on the boards of several NIH institutes. None of the other authors have competing interests. The following statement is required for all projects using resources from the University of Minnesota Clinical and Translational Science Institute (CTSI): This publication was supported by Grant number 1UL1RR033183 from the National Center for Research Resources (NCRR) and by Grant number 8 UL1 TR000114-02 from the National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH) to the University of Minnesota CTSI. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the CTSI or the NIH. The University of Minnesota CTSI is part of a national Clinical and Translational Science Award (CTSA) consortium created to accelerate laboratory discoveries into treatments for patients.
Contributor Information
Jeffrey M. Vinocur, Division of Pediatric Cardiology, Department of Pediatrics, University of Minnesota, East Building MB, 5th Floor, 2450 Riverside Avenue, Minneapolis, MN 55454, USA
Jeremiah S. Menk, Biostatistical Design and Analysis Center, Clinical and Translational Science Institute, University of Minnesota, Minneapolis, MN, USA
John Connett, Biostatistical Design and Analysis Center, Clinical and Translational Science Institute, University of Minnesota, Minneapolis, MN, USA. Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, USA.
James H. Moller, Division of Cardiology, Department of Medicine, University of Minnesota, Minneapolis, MN, USA. Pediatric Cardiac Care Consortium, Minneapolis, MN, USA
Lazaros K. Kochilas, Email: kochilas@umn.edu, Division of Pediatric Cardiology, Department of Pediatrics, University of Minnesota, East Building MB, 5th Floor, 2450 Riverside Avenue, Minneapolis, MN 55454, USA. Pediatric Cardiac Care Consortium, Minneapolis, MN, USA
References
- 1.Allen SW, Gauvreau K, Bloom BT, Jenkins KJ. Evidence-based referral results in significantly reduced mortality after congenital heart surgery. Pediatrics. 2003;112(1 Pt 1):24–28. doi: 10.1542/peds.112.1.24. [DOI] [PubMed] [Google Scholar]
- 2.Aylin P, Bottle A, Jarman B, Elliott P. Paediatric cardiac surgical mortality in England after Bristol: descriptive analysis of hospital episode statistics 1991–2002. BMJ. 2004;329:825. doi: 10.1136/bmj.329.7470.825. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Bazzani LG, Marcin JP. Case volume and mortality in pediatric cardiac surgery patients in California, 1998–2003. Circulation. 2007;115:2652–2659. doi: 10.1161/CIRCULATIONAHA.106.678904. [DOI] [PubMed] [Google Scholar]
- 4.Chang RK, Klitzner TS. Resources, use, and regionalization of pediatric cardiac services. Curr Opin Cardiol. 2003;18:98–101. doi: 10.1097/00001573-200303000-00004. [DOI] [PubMed] [Google Scholar]
- 5.Chang RK, Joyce JJ, Castillo J, Ceja J, Quan P, Klitzner TS. Parental preference regarding hospitals for children undergoing surgery: a trade-off between travel distance and potential outcome improvement. Can J Cardiol. 2004;20:877–882. [PubMed] [Google Scholar]
- 6.Dimick JB, Welch HG, Birkmeyer JD. Surgical mortality as an indicator of hospital quality: the problem with small sample size. JAMA. 2004;292:847–851. doi: 10.1001/jama.292.7.847. [DOI] [PubMed] [Google Scholar]
- 7.Gauvreau K. Reevaluation of the volume–outcome relationship for pediatric cardiac surgery. Circulation. 2007;115:2599–2601. doi: 10.1161/CIRCULATIONAHA.107.701722. [DOI] [PubMed] [Google Scholar]
- 8.Gibbs JL, Monro JL, Cunningham D, Rickards A. Survival after surgery or therapeutic catheterisation for congenital heart disease in children in the United Kingdom: analysis of the central cardiac audit database for 2000–1. BMJ. 2004;328:611. doi: 10.1136/bmj.38027.613403.F6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137:511–520. doi: 10.7326/0003-4819-137-6-200209170-00012. [DOI] [PubMed] [Google Scholar]
- 10.Hannan EL, Racz M, Kavey RE, Quaegebeur JM, Williams R. Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. Pediatrics. 1998;101:963–969. doi: 10.1542/peds.101.6.963. [DOI] [PubMed] [Google Scholar]
- 11.Jacobs JP, Mavroudis C, Jacobs ML, Maruszewski B, Tchervenkov CI, Lacour-Gayet FG, Clarke DR, Yeh T, Jr, Walters HL, 3rd, Kurosawa H, Stellin G, Ebels T, Elliott MJ. What is operative mortality? Defining death in a surgical registry database: a report of the STS Congenital Database Taskforce and the Joint EACTS-STS Congenital Database Committee. Ann Thorac Surg. 2006;81:1937–1941. doi: 10.1016/j.athoracsur.2005.11.063. [DOI] [PubMed] [Google Scholar]
- 12.Jenkins KJ, Newburger JW, Lock JE, Davis RB, Coffman GA, Iezzoni LI. In-hospital mortality for surgical repair of congenital heart defects: preliminary observations of variation by hospital caseload. Pediatrics. 1995;95:323–330. [PubMed] [Google Scholar]
- 13.Jenkins KJ, Gauvreau K, Newburger JW, Spray TL, Moller JH, Iezzoni LI. Consensus-based method for risk adjustment for surgery for congenital heart disease. J Thorac Cardiovasc Surg. 2002;123:110–118. doi: 10.1067/mtc.2002.119064. [DOI] [PubMed] [Google Scholar]
- 14.Karamlou T, McCrindle BW, Blackstone EH, Cai S, Jonas RA, Bradley SM, Ashburn DA, Caldarone CA, Williams WG. Lesion-specific outcomes in neonates undergoing congenital heart surgery are related predominantly to patient and management factors rather than institution or surgeon experience: a Congenital Heart Surgeons Society study. J Thorac Cardiovasc Surg. 2010;139:569–577. doi: 10.1016/j.jtcvs.2008.11.073. [DOI] [PubMed] [Google Scholar]
- 15.Laks MP, Cohen T, Hack R. Volume of procedures at transplantation centers and mortality after liver transplantation. N Engl J Med. 2000;342:1527. author reply 1528. [PubMed] [Google Scholar]
- 16.Livingston EH, Cao J. Procedure volume as a predictor of surgical outcomes. JAMA. 2010;304:95–97. doi: 10.1001/jama.2010.905. [DOI] [PubMed] [Google Scholar]
- 17.Luft HS, Hunt SS, Maerki SC. The volume–outcome relationship: practice-makes-perfect or selective-referral patterns? Health Serv Res. 1987;22:157–182. [PMC free article] [PubMed] [Google Scholar]
- 18.McDonald K, Romano P, Davies S, Haberland C, Geppert J, Ku A, Choudhry K. Measures of pediatric health care quality based on hospital administrative data: the pediatric quality indicators. 2006 http://www.qualityindicators.ahrq.gov/Downloads/Software/SAS/V31/pdi_measures_v31.pdf.
- 19.Moller JH, Hills CB, Pyles LA. A multicenter cardiac registry: a method to assess outcome of catheterization intervention or surgery. Prog Pediatr Cardiol. 2005;20:7–12. [Google Scholar]
- 20.O’Brien SM, Clarke DR, Jacobs JP, Jacobs ML, Lacour-Gayet FG, Pizarro C, Welke KF, Maruszewski B, Tobota Z, Miller WJ, Hamilton L, Peterson ED, Mavroudis C, Edwards FH. An empirically based tool for analyzing mortality associated with congenital heart surgery. J Thorac Cardiovasc Surg. 2009;138:1139–1153. doi: 10.1016/j.jtcvs.2009.03.071. [DOI] [PubMed] [Google Scholar]
- 21.Sollano JA, Gelijns AC, Moskowitz AJ, Heitjan DF, Cullinane S, Saha T, Chen JM, Roohan PJ, Reemtsma K, Shields EP. Volume–outcome relationships in cardiovascular operations: New York State, 1990–1995. J Thorac Cardiovasc Surg. 1999;117:419–428. doi: 10.1016/s0022-5223(99)70320-x. discussion 428–430. [DOI] [PubMed] [Google Scholar]
- 22.Spiegelhalter DJ. Mortality and volume of cases in paediatric cardiac surgery: retrospective study based on routinely collected data. BMJ. 2002;324:261–263. doi: 10.1136/bmj.324.7332.261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Stark JF, Gallivan S, Davis K, Hamilton JR, Monro JL, Pollock JC, Watterson KG. Assessment of mortality rates for congenital heart defects and surgeons’ performance. Ann Thorac Surg. 2001;72:169–174. doi: 10.1016/s0003-4975(01)02689-3. discussion 174–175. [DOI] [PubMed] [Google Scholar]
- 24.Thabut G, Christie JD, Kremers WK, Fournier M, Halpern SD. Survival differences following lung transplantation among US transplant centers. JAMA. 2010;304:53–60. doi: 10.1001/jama.2010.885. [DOI] [PubMed] [Google Scholar]
- 25.Vinocur JM, Moller JH, Kochilas LK. Putting the Pediatric Cardiac Care Consortium in context: evaluation of scope and case mix compared with other reported surgical datasets. Circ Cardiovasc Qual Outcomes. 2012;5:577–579. doi: 10.1161/CIRCOUTCOMES.111.964841. [DOI] [PubMed] [Google Scholar]
- 26.Welke KF, Shen I, Ungerleider RM. Current assessment of mortality rates in congenital cardiac surgery. Ann Thorac Surg. 2006;82:164–170. doi: 10.1016/j.athoracsur.2006.03.004. discussion 170–171. [DOI] [PubMed] [Google Scholar]
- 27.Welke KF, Diggs BS, Karamlou T, Ungerleider RM. The relationship between hospital surgical case volumes and mortality rates in pediatric cardiac surgery: a national sample, 1988–2005. Ann Thorac Surg. 2008 doi: 10.1016/j.athoracsur.2008.04.077. discussion 889–896. [DOI] [PubMed] [Google Scholar]
- 28.Welke KF, Karamlou T, Diggs BS. Databases for assessing the outcomes of the treatment of patients with congenital and paediatric cardiac disease: a comparison of administrative and clinical data. Cardiol Young. 2008;18(Suppl 2):137–144. doi: 10.1017/S1047951108002837. [DOI] [PubMed] [Google Scholar]
- 29.Welke KF, O’Brien SM, Peterson ED, Ungerleider RM, Jacobs ML, Jacobs JP. The complex relationship between pediatric cardiac surgical case volumes and mortality rates in a national clinical database. J Thorac Cardiovasc Surg. 2009;137:1133–1140. doi: 10.1016/j.jtcvs.2008.12.012. [DOI] [PubMed] [Google Scholar]
- 30.Welke KF, Karamlou T, Ungerleider RM, Diggs BS. Mortality rate is not a valid indicator of quality differences between pediatric cardiac surgical programs. Ann Thorac Surg. 2010;89:139–144. doi: 10.1016/j.athoracsur.2009.08.058. discussion 145–146. [DOI] [PubMed] [Google Scholar]
- 31.Williams W. Defining operative mortality: it should be easy, but is it? Ann Thorac Surg. 2006;81:1557–1560. doi: 10.1016/j.athoracsur.2006.02.025. [DOI] [PubMed] [Google Scholar]