Abstract
Background:
Patients undergoing complex pediatric cardiac surgery remain at considerable risk of mortality and morbidity, and variation in outcomes exists across hospitals. We formed the Pediatric Cardiac Critical Care Consortium (PC4) to improve the quality of care for these patients through transparent data sharing and collaborative learning between participants.
Objective:
To determine whether outcomes improved over time within PC4.
Methods:
We analyzed 19,600 hospitalizations (18 hospitals) in the PC4 clinical registry that included cardiovascular surgery from 8/2014-6/2018. The primary exposure was two years of PC4 participation; this provided adequate time for hospitals to accrue data and engage in collaborative learning. Aggregate case mix-adjusted outcomes were compared between the first two years of participation (baseline) and all months post-exposure. We also evaluated outcomes from the same era in a cohort of similar, non-PC4 hospitals.
Results:
During the baseline period there was no evidence of improvement. We observed significant improvement in the post-exposure period vs. baseline for postoperative ICU mortality [2.1% vs. 2.7%; 22% relative reduction (RR), p=0.001], in-hospital mortality (2.5% vs. 3.3%; 24% RR, p=0.001), major complications (10.1% vs. 11.5%; 12% RR, p<0.001), ICU length of stay (7.3 days vs. 7.7; 5% RR, p<0.001), and duration of ventilation (61.3 hours vs. 70.6; 13% RR, p=0.01). Non-PC4 hospitals showed no significant improvement in mortality, complications, or hospital length of stay.
Conclusions
Our analysis demonstrates improving cardiac surgical outcomes at children’s hospitals participating in PC4. This change appears unrelated to secular improvement trends, and likely reflects PC4’s commitment to transparency and collaboration.
Keywords: Cardiac Surgery, Congenital, Pediatric, Outcomes, Quality, Collaborative Learning
Condensed Abstract
We formed the Pediatric Cardiac Critical Care Consortium (PC4) to improve the quality of care for patients undergoing pediatric and congenital heart surgery through transparent data sharing and collaborative learning between participants. We analyzed 19,600 consecutive surgical hospitalizations in the PC4 clinical registry to determine if postoperative outcomes improved over time through participation in PC4. We observed statistically significant improvement in mortality (22% reduction), major complications (12% reduction), duration of mechanical ventilation (13% reduction), and critical care length of stay (5% reduction). These findings appear unrelated to secular improvement trends, and likely reflect PC4’s commitment to transparency and collaboration.
Introduction
Congenital heart disease remains the most common congenital malformation and accounts for the greatest number of infant deaths related to birth defects (1). Many children with critical pediatric and congenital cardiovascular disease undergo surgical intervention in the newborn period or later in infancy. Outcomes for cardiac surgery at children’s hospitals have improved over several decades, but these improvements appear to be leveling off (2). Mortality remains high for complex operations (3), and perioperative complications result in morbidity that can affect patients across the lifespan. Furthermore, important variation in outcomes persists across hospitals, particularly for complex surgery (3–6). These realities suggest that opportunities exist to improve the quality of care for patients with critical pediatric and congenital cardiovascular disease.
However, the optimal strategies to facilitate continued improvement remain unclear. A wealth of literature in adult cardiac surgery suggests that collaborative quality improvement represents one of the most successful strategies for reducing postoperative morbidity and mortality (7,8). These seminal efforts suggest that performance feedback to clinicians and collaborative learning among peers drive practice change and lead to higher quality care. The applicability of these methods to pediatric cardiac surgery is less certain. In contrast to adult cardiac surgery there is greater heterogeneity of diseases and operative procedures, and hospitals are geographically more distant, calling into question whether previous models of collaborative learning can be replicated.
In this context, we formed the Pediatric Cardiac Critical Care Consortium (PC4) with a vision to improve the outcomes of patients with critical cardiovascular disease. We believed that providing participating hospitals with access to timely, actionable, and transparent clinical outcome data and creating a culture of collaborative learning across PC4 would result in quality improvement. We hypothesized that clinical outcomes and resource utilization would improve across PC4 hospitals after two years of participation. We further sought to evaluate any influence of secular trends on our analysis by evaluating outcomes during the study period in a cohort of case-mix matched, non-PC4 hospitals in the Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database.
Methods
The Pediatric Cardiac Critical Care Consortium (PC4)
PC4 is a quality improvement collaborative that collects data on all patients with primary cardiac disease admitted to the cardiac intensive care unit (CICU) service of participating hospitals (9). PC4 maintains a clinical registry to support research and quality improvement initiatives. Six hospitals began contributing data in 2013, and there are now over 50 hospitals participating.
The PC4 registry shares common terminology and definitions with applicable data points from the International Pediatric and Congenital Cardiac Code (IPCCC), STS Congenital Heart Surgery Database, and American College of Cardiology Improving Pediatric and Adult Congenital Treatment (IMPACT) Registry, as previously described (9). Each hospital has a trained data manager who has completed a certification exam. The data managers collect and enter data in accordance with the standardized PC4 Data Definitions Manual. Participating hospitals are audited on a regular schedule and audit results suggest complete, accurate and timely submission of data across hospitals, with published results demonstrating a major discrepancy rate of 0.6% across 29,476 fields (10). Subsequent initial and follow-up audits suggest similar data integrity, and those hospitals that fail to meet standards for accuracy or completeness are removed from outcome reports and research analyses. This study was reviewed and approved with waiver of informed consent by the University of Michigan Institutional Review Board.
Quality improvement infrastructure within PC4
PC4 uses data from the clinical registry to promote quality improvement across hospitals through three primary mechanisms: 1) timely reporting of outcome data to participants, 2) transparency between hospitals, and 3) collaborative learning. First, cases can be submitted in real-time immediately after hospital discharge and participants have access to a web-based reporting platform that is updated each morning. This mechanism supplants traditional methods of delivering outcome data through periodic batched reporting and ensures that outcome data are up-to-date and relevant for participants. Access to these data likely creates an important Hawthorne effect for hospitals that previously had limited opportunities to critically evaluate their quality of care. When new case mix adjusted metrics are added to the reporting platform the model derivation and performance characteristics are shared with the entire collaborative to encourage use of the metric for quality improvement.
Second, the PC4 clinical champion at each hospital can identify other hospitals on the reporting platform through the hospital unblinding program which begins after the first year of participation and is contingent on passing the initial and subsequent audits. Hospitals agree to this transparency when they sign the data use agreement to participate in PC4; it is mandatory for entry into the collaborative. Participants are encouraged to identify, communicate and collaborate with high-performing hospitals in domains where they seek to improve. Similarly, hospitals that are contacted for help from another institution are expected to share insight on practices and resources that underlie their excellent performance.
Finally, PC4 promotes collaborative learning by identifying variation across hospitals in outcomes deemed important by participants, featuring panels of high-performing hospitals explaining their practices at the annual meeting, and allowing participants to share the results of local quality improvement efforts through webinars and in-person meetings. Research findings are shared with the entire collaborative via webinars on a bi-monthly basis to highlight important areas of variation in performance across hospitals.
Primary exposure and inclusion criteria
The primary exposure for this analysis was 2 years of participation in PC4. We hypothesized that outcomes would improve after two years in the collaborative because this time period would allow for meaningful data accrual, identification of areas that could benefit from quality improvement efforts by participants, adequate time to communicate with high-performing hospitals, and implementation of improvement initiatives. We selected hospitals that had at least 30 months of data at the time of analysis, thus providing ≥6 months of outcome data after the exposure period in which to measure improvement. We excluded data from one hospital that did not pass its follow-up audit during its third year of participation, leaving 18 hospitals in the analysis. We analyzed all cardiac surgical hospitalizations in the registry from August 2014 through June 2018.
Outcomes
We analyzed postoperative mortality, complications, and resource utilization. For each hospital we obtained case-mix adjusted outcomes (averaged over all patients in the hospital) based on multivariable models including patient, operative, and illness severity factors relevant to the specific metric, as previously described (Appendix) (4,11,12). In general, we selected outcomes that focus on the quality of care in the CICU since this is the primary clinical setting for data collection and quality improvement in PC4. However, we have also partnered with the congenital heart surgery community to assess and report on some metrics that are inclusive of all phases of perioperative care. Our complication metrics include those that could be considered primarily operative complications (e.g., diaphragm paralysis) and those that are impacted to a greater degree by postoperative care in the CICU (e.g., cardiac arrest).
Statistical analysis
The primary analysis compares aggregated case mix-adjusted outcomes between the first two years of participation in PC4 (baseline) and all accrued time after the two-year mark (post-exposure). Hospitals entered the collaborative at different dates, but each hospital’s outcomes over the first 2 years of participation were considered baseline for the primary analyses (Figure 1). The post-exposure period ranged from 6 to 20 months across hospitals. This study design strengthens subsequent conclusions over a simple pre-post time-series analysis because 1) any finding of improvement cannot be the result of adding hospitals with better outcomes over time, and 2) changes are much less likely to be the result of secular trends given that there are variable start times for the individual hospitals. We described and compared the patient, operative, and severity of illness measures between study periods in univariate analyses using chi-square, Fisher’s exact, or Wilcoxon rank sum test as appropriate.
Figure 1. Study Timeline.
Hospitals entered Pediatric Cardiac Critical Care Consortium at varying time points during the study period. Outcomes over the first 2 years of participation for each hospital were aggregated into the baseline period. Outcomes across hospitals after the first 2 years were aggregated into the post-exposure period.
Primary analysis
We performed case-mix adjustment during the baseline and post-exposure periods using the multivariable models described above. For categorical outcome metrics (e.g. mortality, complications) we reported the adjusted rate per CICU admission or hospitalization, as appropriate, from logistic regression models. For continuous outcome metrics (length of stay, duration of mechanical ventilation), we calculated adjusted means due to the well-recognized barriers in accurately deriving adjusted median values. These mean values represent overall observed-to-expected days or hours, respectively. We used negative binomial (length of stay) and zero-inflated negative binomial (duration of mechanical ventilation) regression models for these outcomes. Each model accounted for clustering of patients within hospitals by including a hospital-specific random effect term.
To determine whether improvement occurred with participation, we included a binary variable in our regression models for the period (baseline vs. post-exposure) in which a patient was treated based on the hospital admission date. If this exposure variable was associated with the outcome at p<0.05 then we concluded there was a statistically significant improvement in the outcome after the 2-year exposure, adjusting for case-mix differences.
Since some of the complications we evaluated may be more strongly associated with the operation and less so with postoperative care in the CICU, we also evaluated each major complication individually as described above in a multivariable logistic regression model.
Secondary analyses
One potential critique of any conclusions about the impact of PC4 participation on improved outcomes would be if hospitals were already improving during the baseline exposure period. To test for any evidence of improvement during the baseline period we performed a trend test by month during the first 2 years of participation.
To further test our a priori hypothesis that two years of participation represents the critical exposure for improvement we ran each of our regression models again but changed the time variable from the binary study period exposure to a yearly variable. If we found that there was no significant association (p<0.05) with improved outcomes in years 1 or 2, but did observe this association in years 3 and 4, then we considered that evidence supportive of our hypothesis about the primary exposure.
Finally, we sought to determine whether there was evidence for a secular trend in improvement in perioperative outcomes at hospitals not participating in PC4. This evaluation is critical in assessing the extent to which any improvements seen within PC4 hospitals reflect PC4 participation vs. the impact of other factors during the study time period. However, no database exists that allows evaluation of the exact metrics captured by PC4, or application of the same detailed case mix-adjustment methodology in a group of hospitals similar to those participating in PC4. To best approximate secular case-mix adjusted trends in peri-operative outcomes in hospitals similar to PC4 we chose to use the STS Congenital Heart Surgery Database.
We obtained aggregate data from 17 non-PC4 hospitals in the STS Database with a case mix similar to those in PC4 based on the following annual criteria: at least 200 total index operations, at least five high complexity operations [STS – European Association for Cardiothoracic Surgery (STAT) category 5] (13), and neonates comprising at least 10% of the total operations, all of which are true of the PC4 hospitals. Outcomes available in the STS Database were examined in this cohort of non-PC4 hospitals from 2014-2017; the same 17 hospitals were included for each year. This time period of available data overlapped with that of our primary analysis. We analyzed operative mortality, major complication rate, and total post-operative hospital length of stay. The STS uses risk models to perform case-mix adjusted outcomes analysis that are generally similar to those used by PC4 and include variables used within PC4 (14,15). The STS Database does not include variables to allow case-mix adjusted comparison of the non-PC4 hospitals to PC4 participants on CICU postoperative mortality, CICU length of stay, cardiac arrest, or duration of mechanical ventilation.
Only yearly aggregate raw and case-mix adjusted data were available from the STS Database for analysis. We could not perform a true difference-in-differences analysis (16) because the PC4 hospitals entered the cohort at different times compared to non-PC4 hospitals, and there was no common time point between hospital cohorts to measure pre- and post-outcomes. All analyses were performed using SAS Version 9.4 (SAS Institute, Cary, North Carolina) or STATA Version 14 (STATA, College Station, Texas).
Results
The study cohort included 19,600 surgical hospitalizations with postoperative care in the CICU during the entire study period. Table 1 shows a comparison of patient and operative characteristics between the baseline and post-exposure periods demonstrating no clinically significant differences in the study populations.
Table 1 –
Patient and operative characteristics across eras
| Characteristic | Baseline N=10,656 (54%) |
Post-Exposure N=8,944 (46%) |
P-value |
|---|---|---|---|
| Age | 0.12 | ||
| Neonate pre-term | 403 (3.8%) | 321 (3.6%) | |
| Neonate full-term | 1902 (17.9%) | 1645 (18.4%) | |
| Infant | 3385 (31.8%) | 2963 (33.1%) | |
| Child | 4389 (41.2%) | 3560 (39.8%) | |
| Adult | 577 (5.4%) | 455 (5.1%) | |
| Weight-for-age z-score | 0.34 | ||
| Normal | 7907 (74.2%) | 6685 (74.7%) | |
| Underweight | 2386 (22.4%) | 1946 (21.8%) | |
| Overweight | 363 (3.4%) | 313 (3.5%) | |
| Extracardiac anomalies, any | 1773 (16.6%) | 1586 (17.7%) | 0.04 |
| Chromosomal abnormalities/syndromes | 2255 (21.2%) | 1902 (21.3%) | 0.86 |
| STAT Mortality Category | 0.37 | ||
| 1 | 3067 (28.8%) | 2651 (29.6%) | |
| 2 | 3319 (31.2%) | 2787 (31.2%) | |
| 3 | 1416 (13.3%) | 1117 (12.5%) | |
| 4 | 2367 (22.2%) | 1961 (21.9%) | |
| 5 | 487 (4.6%) | 428 (4.8%) | |
| Preoperative CICU admission | 2097 (19.7%) | 1730 (19.3%) | 0.55 |
STAT, Society of Thoracic Surgeons-European Association of Cardiothoracic Surgery; CICU, Cardiac Intensive Care Unit
Primary analysis: Outcomes improvement over time
The Central Illustration and Figures 2, 3 and 4 show the changes in each outcome metric across time. When comparing outcomes during the baseline exposure period to the post-exposure period we observed statistically significant reductions in CICU postoperative mortality (−22% relative change, p<0.001), in-hospital mortality (−24%, p<0.001), major complications (−12%, p<0.001), CICU length of stay (−5%, p<0.001), and duration of postoperative mechanical ventilation (−13%, p=0.01).
Central Illustration: Improving Pediatric Cardiac Surgical Outcomes: Change in Postoperative Mortality Over Time.
Panel A: Improvement in cardiac surgical outcomes at pediatric cardiac critical care consortium hospitals. Panel B: Case mix adjusted mortality by quarter. Vertical line represents end of baseline period. Blue line and squares represent in-hospital mortality. Red line represents cardiac ICU postoperative mortality. The relative decrease in the post-exposure period for in-hospital mortality and cardiac ICU mortality was 24% (2.7% mortality during baseline vs. 2.1% post-exposure) and 22% (3.3% vs. 2.5%), respectively.
† p<0.05 for comparison of baseline vs. post-exposure rates. CI, confidence interval.
Gaies, M. et al. J Am Coll Cardiol. 2019;74(22):2786-95.
Figure 2. Change in postoperative complications over time.
Case mix adjusted complications by quarter. Vertical line represents end of baseline period. Red line represents all complications. Blue line represents major complications. Green line represents cardiac arrest. The relative decrease in in the post-exposure period for major complications was 12%.
† p<0.05 for comparison of baseline vs. post-exposure rate. CI, confidence interval.
Figure 3. Change in postoperative length of stay over time.
Case mix adjusted mean postoperative length of stay by quarter. Vertical line represents end of baseline period. Red line represents hospital length of stay. Blue line represents cardiac ICU length of stay. The relative decrease in cardiac ICU length of stay in the post-exposure period was 5%.
† p<0.05 for comparison of baseline vs. post-exposure duration. CI, confidence interval.
Figure 4. Change in duration of postoperative mechanical ventilation.
Case mix adjusted mean duration of postoperative mechanical ventilation by quarter. Vertical line represents end of baseline period. The relative decrease in the post-exposure period for duration of ventilation was 13%.
† p<0.05 for comparison of baseline vs. post-exposure duration. CI, confidence interval.
We examined improvement at individual hospitals for clinically significant changes in CICU postoperative mortality, CICU length of stay, and duration of mechanical ventilation, the three outcomes most directly related to CICU quality of care. For CICU postoperative mortality, 7/18 hospitals reduced their adjusted mortality by at least 1%, and none saw an increase to that degree. With regard to CICU length of stay, 8/18 hospitals reduced their mean by at least 0.5 days, and only one had an increase to that degree. Finally, for duration of mechanical ventilation, 10/18 hospitals reduced their mean duration by at least 10 hours, and none saw an increase to that degree. Improvement was not restricted to hospitals with worse-than-expected performance in the baseline period; some hospitals with baseline outcomes better than the aggregate PC4 baseline average for the three metrics above improved by 25% or more on at least one metric.
When analyzing the individual major complications, we observed significant improvement across study periods for stroke, unplanned cardiac reintervention (catheterization or surgery), use of mechanical circulatory support, diaphragm paralysis, and complete heart block requiring permanent pacemaker placement. There was no significant change in rates of seizures, intraventricular hemorrhage, acute renal failure requiring dialysis, or bleeding requiring reoperation.
Secondary analyses
We analyzed the trends in outcomes within the baseline period for each metric where we found aggregate improvement post-exposure. There was no evidence of improvement during the baseline period in any of the outcome metrics under study. When analyzing change over time by year we no found improvement in any outcome metric in years 1 or 2, but statistically significant improvement was observed starting in year 3 for each.
Table 2 shows outcomes over time at non-PC4 hospitals. There was no evidence of improvement in either major complication rate or total postoperative hospital length of stay. Trends in operative mortality were evaluated in detail, and the slope for change over time in case-mix adjusted operative mortality from linear regression analysis showed no significant difference over time (slope −0.15, 95% confidence interval −0.40 to +0.09, p=0.15).
Table 2 –
Outcomes at non-PC4 hospitals (N=17, 2014-2017)
| Year | Major complication (%) | Median postoperative hospital length of stay, days (interquartile range) | Operative mortality (%) |
|---|---|---|---|
| 2014 | 13.1 | 7 (5-15) | 3.1 |
| 2015 | 13.8 | 8 (5-16) | 3.2 |
| 2016 | 13.9 | 8 (5-16) | 2.8 |
| 2017 | 15.4 | 8 (5-16) | 2.7 |
Discussion
Our analysis provides evidence for improvement in outcomes for patients undergoing pediatric and congenital cardiac surgery at hospitals participating in PC4 (Central Illustration). These improvements occurred across a diverse set of metrics and the analysis supports our a priori hypothesis that two years of exposure to the collaborative are necessary and sufficient for collective improvement. Our study design and analyses suggest that the results are not due to improvement by hospitals during the baseline period, by adding hospitals over time with better outcomes, and, to the degree it was possible to assess with available control data, not due to secular trends in improvement (Central Illustration).
Collaborative quality improvement
We developed PC4 to follow the successful blueprint for collaborative quality improvement laid out by the Northern New England Cardiovascular Disease Study Group (NNE) (7), and subsequently replicated by several other collaboratives (17,18). In O’Connor’s landmark study from the NNE (7), the investigators engaged in an intervention that “included feedback of outcome data, training in continuous quality improvement techniques, and site visits to other medical centers,” leading to significant reduction in mortality after coronary artery bypass graft surgery.
Like the NNE, PC4 participants collect detailed clinical data on every patient treated in our CICUs, and use rigorous empirical analysis to provide timely benchmark performance data back to participants. We mandate transparency between hospitals similar to the way the NNE hospitals and surgeons welcomed one another to observe practices and find solutions to improve quality. We have promoted the culture of collaboration over competition between hospitals at every opportunity, and encouraged clinicians and researchers to work toward aggregate improvement as was the case in the NNE. Our experience mirrors that of the NNE in that we observed aggregate quality improvement without targeting any specific outcome measures or engaging in an intervention to specifically change a practice or set of practices. This analysis from PC4 demonstrates how effective the NNE model remains for improving the quality of care for surgical patients and that these principles are generalizable to pediatric cardiac surgical programs.
Independence from secular trends
Over the study period there were no formal multi-institutional quality improvement projects within or outside of PC4 targeted at the outcome metrics in this analysis. During the earliest part of our study era, the Pediatric Heart Network Collaborative Learning Study (19) - aiming to increase early extubation after infant cardiac surgery at four of the PC4 hospitals - concluded. However, the study end date preceded the PC4 participation start date for three of the four study hospitals and as such any improvements to reduce postoperative mechanical ventilation would have biased our study toward the null for this outcome. Our secondary analysis of non-PC4 hospitals from the STS Database suggested no secular trend in improvement over a similar time period. We would thus conclude that the observed improvement across all of the measured outcomes after two years of participation in PC4 remains largely due to our infrastructure and culture that includes real-time outcome reporting, transparency, and collaborative learning fostered across our network of hospitals. For example, at the 2015 annual meeting we conducted a panel where high-performing hospitals (low duration of mechanical ventilation and low extubation failure rates) presented their mechanical ventilation practices to the rest of the collaborative. This could explain, in part, how some hospitals may have positively changed mechanical ventilation practice. We also suspect that a positive Hawthorne effect contributed to the observed changes across all outcomes.
Need for focused improvement efforts
We find it insightful to examine outcomes where we observed no aggregate improvement and consider how our approach might change in the future to effect positive changes in these domains. First, despite reduction in duration of mechanical ventilation and CICU length of stay, total hospital length of stay was unchanged. It is likely that efforts to reduce total hospital length of stay require collaboration between the CICU and the acute care cardiology community that cares for postoperative patients once they transfer out of the CICU to the ward. This highlights the importance of our collaboration with the newly formed Pediatric Acute Care Cardiology Collaborative (PAC3) (20) within Cardiac Networks United (21). We also observed no change in postoperative cardiac arrest incidence across study eras. This result stands in contrast to the remainder of the study findings, which suggest general improvement in CICU quality as a result of participation in PC4, and in that context it would be reasonable to expect that cardiac arrest prevention might improve as well. In part related to our findings, we initiated a Cardiac Arrest Prevention Study in June 2018 that includes 29 hospitals from the collaborative with an aim to reduce the incidence of cardiac arrest in high-risk patients, including neonates undergoing surgery with cardiopulmonary bypass and other infants undergoing single ventricle palliation.
Finally, improvement was not uniform across all hospitals even though many demonstrated clinically significant improvement in at least one metric. While we would not expect all hospitals to take equal advantage of the quality improvement opportunities afforded by participation, it will be important to understand the reasons why certain hospitals seem to benefit more from participation in PC4 than others. One of the reasons PC4 developed the reporting and analytic infrastructure and the transparency program was to allow hospitals to decide which metrics are most important locally in order to devote improvement resources there (“bottom up”), rather than focusing exclusively on large multi-institutional projects initiated by PC4 leadership (“top down”). We hope that our efforts to facilitate a bottom up approach to quality improvement has allowed hospitals to invest in activities aimed at outcomes beyond those studied in this analysis. In the future we anticipate a mix of these strategies to realize ongoing improvements in quality for patients in our CICUs.
Limitations
The greatest potential limitation of our analysis remains our inability to assess a causal relationship, i.e. fully assess the improvement we observed across multiple outcomes against an external set of hospitals that are unexposed to PC4 participation. We did examine non-PC4 hospitals in the STS database on a limited set of metrics, and this analysis provided no evidence that secular trends in improvement explain the findings from our primary analysis. Other aspects of our study design further refute the alternative hypothesis that the improvement we demonstrated is due to widespread improvements in congenital cardiac care. Hospitals entered at different times into the collaborative, so there is no single time point where an external shock to the system could explain the results as we have analyzed them. Some study hospitals started in PC4 more than two years after the initial participants, so any general trend in improvement would have led to better outcomes in these hospitals’ baseline data and made it less likely that they would continue to improve in the post-exposure period. Thus, a concomitant secular trend in improvement would have biased our analysis toward the null. We will continue to seek opportunities to compare our results to external cohorts.
Another limitation of our analysis includes the generalizability of our findings. The group of PC4 hospitals in this study cohort all have specialized CICUs and medium-to-large volume cardiac surgical programs. It is possible that institutions like these who were early adopters of PC4 also have greater resources for quality improvement than others, and as such are in better position to take advantage of the quality infrastructure of PC4 that led to the observed aggregate improvement. As PC4 evolves to include a more diverse set of hospitals it will be crucial to determine whether certain hospitals are more or less likely to benefit from the data resources and the collaborative learning culture. Finally, some of the metrics where we observed improvement (CICU postoperative mortality, CICU length of stay, and duration of mechanical ventilation) are more tightly coupled to the CICU quality focus of PC4, whereas some of the major complications are primarily influenced by surgical care (e.g. diaphragm paralysis). It is possible that the commitment by PC4 to reporting broad-based measures of perioperative care quality is influencing intraoperative care, but further study is necessary to determine if there has been a direct impact on intraoperative teams that might explain the observed improvement.
Conclusions
We demonstrated aggregate improvement in the quality of care for patients undergoing cardiac surgery at children’s hospitals participating in PC4, which is likely a result of our commitment to transparent data sharing and collaboration. Key challenges for the future include identifying why some hospitals improve more than others, analyzing outcomes in non-surgical patients and determining if similar improvement occurs over time, and focusing quality improvement interventions on those metrics where outcomes are unchanged through participation alone. Optimizing research and quality improvement for children with critical cardiovascular disease requires multi-disciplinary participation and increased data and resource sharing between quality collaboratives.
Supplementary Material
CLINICAL PERSPECTIVES.
Competency in Practice-Based Learning and Improvement:
Collaborative and transparent data sharing between hospitals performing pediatric and congenital cardiac surgery improves clinical outcomes.
Translational Outlook:
Future studies must focus on the reasons outcomes at some hospitals improve while others do not and develop strategies beyond collaboration that improve care.
Acknowledgement:
The authors would like to acknowledge Dylan Thibault from the Duke Clinical Research Institute for data management efforts related to the non-PC4 sites.
Funding: Gaies was supported in part by funding from the National Institutes of Health/National Heart, Lung, and Blood Institute (K08HL116639). This study was supported in part by the National Heart, Lung, and Blood Institute (R01HL12226; PI Pasquali). Pasquali also received support from the Janette Ferrantino Professorship, University of Michigan, Ann Arbor, MI. The PC4 Data Coordinating Center receives funding from the University of Michigan Congenital Heart Center, CHAMPS for Mott, and the Michigan Institute for Clinical & Health Research (NIH/NCATS UL1TR002240), all University of Michigan, Ann Arbor, MI.
Disclosures: Justin Dimick and John Birkmeyer are Co-Founders and equity owners of ArborMetrix, Ann Arbor, Michigan, which provides software and analytic support to the Pediatric Cardiac Critical Care Consortium. All other authors report no COI related to this study.
Abbreviations
- CICU
Cardiac Intensive Care Unit
- NNE
Northern New England Cardiovascular Disease Study Group
- PC4
Pediatric Cardiac Critical Care Consortium
- STS
Society of Thoracic Surgeons
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Tweet: @pc4quality participation improves clinical & resource utilization outcomes after pediatric & congenital heart surgery #CHD #PedsCards #PedsICU #PedsCICU
Twitter handles: @MGaies, @skpasquali, @jdimick1, @johnbirkmeyer, @bubblesdadee, @davidscooper, @jmcostellomd, @JeffJacobs215, @Mark_Scheurer, @schws0
References
- 1.Hoffman JI, Kaplan S. The incidence of congenital heart disease. J Am Coll Cardiol 2002;39:1890–900. [DOI] [PubMed] [Google Scholar]
- 2.Jacobs JP, He X, Mayer JE Jr. et al. Mortality Trends in Pediatric and Congenital Heart Surgery: An Analysis of The Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg 2016;102:1345–52. [DOI] [PubMed] [Google Scholar]
- 3.Jacobs JP, Mayer JE Jr., Mavroudis C et al. The Society of Thoracic Surgeons Congenital Heart Surgery Database: 2017 Update on Outcomes and Quality. Ann Thorac Surg 2017;103:699–709. [DOI] [PubMed] [Google Scholar]
- 4.Tabbutt S, Schuette J, Zhang W et al. A Novel Model Demonstrates Variation in Risk-Adjusted Mortality Across Pediatric Cardiac ICUs After Surgery. Pediatr Crit Care Med 2019;20:136–142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jacobs JP, O’Brien SM, Pasquali SK et al. Variation in outcomes for benchmark operations: an analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database. Ann Thorac Surg 2011;92:2184–91; discussion 2191-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Jacobs JP, O’Brien SM, Pasquali SK et al. Variation in outcomes for risk-stratified pediatric cardiac surgical operations: an analysis of the STS Congenital Heart Surgery Database. Ann Thorac Surg 2012;94:564–71; discussion 571-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.O’Connor GT, Plume SK, Morton JR, et al. Results of a regional prospective study to improve the in-hospital mortality associated with coronary artery bypass grafting. Jama 1996;275:841–6. [PubMed] [Google Scholar]
- 8.Likosky DS, Harrington SD, Cabrera L et al. Collaborative Quality Improvement Reduces Postoperative Pneumonia After Isolated Coronary Artery Bypass Grafting Surgery. Circ Cardiovasc Qual Outcomes 2018;11:e004756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Gaies M, Cooper DS, Tabbutt S et al. Collaborative quality improvement in the cardiac intensive care unit: development of the Paediatric Cardiac Critical Care Consortium (PC4). Cardiology in the young 2015;25:951–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Gaies M, Donohue JE, Willis GM et al. Data integrity of the Pediatric Cardiac Critical Care Consortium (PC4) clinical registry. Cardiol Young 2015:1–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Alten JA, Klugman D, Raymond TT et al. Epidemiology and Outcomes of Cardiac Arrest in Pediatric Cardiac ICUs. Pediatr Crit Care Med 2017;18:935–943. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Gaies M, Werho DK, Zhang W et al. Duration of Postoperative Mechanical Ventilation as a Quality Metric for Pediatric Cardiac Surgical Programs. Ann Thorac Surg 2018;105:615–621. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Jacobs JP, Jacobs ML, Maruszewski B et al. Initial application in the EACTS and STS Congenital Heart Surgery Databases of an empirically derived methodology of complexity adjustment to evaluate surgical case mix and results. Eur J Cardiothorac Surg 2012;42:775–9; discussion 779-80. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Jacobs JP, O’Brien SM, Pasquali SK et al. The Society of Thoracic Surgeons Congenital Heart Surgery Database Mortality Risk Model: Part 2-Clinical Application. Ann Thorac Surg 2015;100:1063–8; discussion 1068-70. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.O’Brien SM, Jacobs JP, Shahian DM et al. Development of a Congenital Heart Surgery Composite Quality Metric: Part 2-Analytic Methods. Ann Thorac Surg 2019;107:590–596. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-differences approach. JAMA 2014;312:2401–2. [DOI] [PubMed] [Google Scholar]
- 17.Share DA, Campbell DA, Birkmeyer N et al. How a regional collaborative of hospitals and physicians in Michigan cut costs and improved the quality of care. Health Aff (Millwood) 2011;30:636–45. [DOI] [PubMed] [Google Scholar]
- 18.Anderson JB, Beekman RH 3rd, Kugler JD et al. Improvement in Interstage Survival in a National Pediatric Cardiology Learning Network. Circ Cardiovasc Qual Outcomes 2015;8:428–36. [DOI] [PubMed] [Google Scholar]
- 19.Mahle WT, Nicolson SC, Hollenbeck-Pringle D et al. Utilizing a Collaborative Learning Model to Promote Early Extubation Following Infant Heart Surgery. Pediatr Crit Care Med 2016;17:939–947. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Kipps AK, Cassidy SC, Strohacker CM et al. Collective quality improvement in the paediatric cardiology acute care unit: establishment of the Pediatric Acute Care Cardiology Collaborative (PAC3). Cardiol Young 2018;28:1019–1023. [DOI] [PubMed] [Google Scholar]
- 21.Gaies M, Anderson J, Kipps A et al. Cardiac Networks United: an integrated paediatric and congenital cardiovascular research and improvement network. Cardiol Young 2018:1–8. electronic publication date 12/21/2018. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.