Extubation failure rates after pediatric cardiac surgery vary across hospitals

Abstract

Objective:

Many hospitals aim to extubate children early after cardiac surgery, yet it remains unclear how this practice associates with extubation failure (EF). We evaluated adjusted EF rates and duration of postoperative mechanical ventilation (POMV) across hospitals and assessed cardiac intensive care unit (CICU) organizational factors associated with EF.

Design:

Secondary analysis of the Pediatric Cardiac Critical Care Consortium (PC⁴) clinical registry.

Setting:

PC⁴ CICUs.

Patients:

Patients with qualifying index surgical procedures from 8/2014–6/2017.

Interventions:

None.

Measurements and Main Results:

We modeled hospital-level adjusted EF rates using multivariable logistic regression. A previously validated PC⁴ model was used to calculate adjusted POMV. O/E ratios for both metrics were derived for each hospital to assess performance. Hierarchical logistic regression was used to assess the association between CICU factors and EF. Overall, 16,052 surgical hospitalizations were analyzed. Predictors of EF (p<0.05 in final case-mix adjustment model) included younger age, underweight, greater surgical complexity, airway anomaly, chromosomal anomaly/syndrome, longer cardiopulmonary bypass time, and other preoperative comorbidities. Three hospitals were better-than-expected outliers for EF (95% confidence interval (CI) around O/E<1) and three hospitals were worse-than-expected (95% CI around O/E>1). Two hospitals were better-than-expected outliers for both EF and POMV and three were worse-than-expected for both. No hospital was an outlier in opposite directions. Greater nursing hours per patient day and percent nursing staff with critical care certification were associated with lower odds of EF. CICU factors such as fewer inexperienced nurses, greater percent critical care trained attendings, CICU-dedicated respiratory therapists, and fewer patients per CICU attending were not associated with lower odds of EF.

Conclusions:

We saw no evidence that hospitals trade higher EF rates for shorter duration of POMV after pediatric cardiac surgery. Increasing specialized CICU nursing hours per patient day may achieve better extubation outcomes and mitigate the impact of inexperienced nurses.

Introduction

Pediatric patients undergoing cardiac surgery often require mechanical ventilation in the postoperative period. Many clinicians believe that minimizing the duration of postoperative mechanical ventilation (POMV) represents higher quality care (1), as mechanical ventilation is associated with important complications such as infection and airway/lung injury, increased exposure to sedatives and analgesic medications, and increased intensive care unit utilization (2, 3). In recent years, many hospitals have emphasized early extubation for children undergoing heart surgery, with some institutions now extubating a large proportion of their patients in the operating room or shortly after arrival to the pediatric cardiac intensive care unit (CICU) (4, 5). While aggressive postoperative extubation approaches may be associated with better outcomes, particularly after low- or medium-complexity operations, concern exists that this practice may be associated with increased extubation failure (EF) rates.

However, the relationship between these two metrics remains unclear, and there are limited data to support the belief that earlier extubation necessitates a trade-off for increased EF rates. Though EF and POMV have been studied separately (1, 6, 7), these two outcomes have never been compared simultaneously across a group of hospitals. We previously studied EF across a smaller cohort of both medical and surgical CICU patients and identified increasing duration of mechanical ventilation as the only independent predictor of EF (8), suggesting that more aggressive early extubation practice may actually improve the likelihood of extubation success. It remains unclear whether hospitals that extubate higher-risk patients (neonates, high-complexity operations, etc.) earlier after surgery accept higher EF rates as a consequence. The paucity of information about how these metrics relate within hospitals and the lack of well-established pediatric guidelines for timing of extubation (9) likely contribute to variation in care and outcomes of postoperative mechanical ventilation practice after pediatric cardiac surgery (1, 6, 7).

In this context, we performed an analysis using the Pediatric Cardiac Critical Care Consortium (PC⁴) clinical registry to describe the relationship between EF rates and duration of POMV in pediatric patients undergoing cardiac surgery. We measured case-mix adjusted EF rates and duration of POMV for each hospital across a wide spectrum of cardiac surgical complexity and compared the results within each center. We were particularly interested in identifying hospitals that were statistical outliers for both metrics, either in the same or opposite directions. Finally, we examined the association between EF and selected hospital characteristics.

Materials and Methods

Data source

The Pediatric Cardiac Critical Care Consortium (PC⁴) is a voluntary quality improvement collaborative that collects data on all patients with primary cardiac disease admitted to the CICU attending service of participating hospitals (10). PC⁴ maintains a clinical registry that includes data on patient demographics, comorbidities, surgical procedures/interventions, critical care therapies, and complications in order to support research and quality improvement initiatives. At the time of this analysis, 25 hospitals were submitting cases to the PC⁴ registry.

Each participating center has a trained and certified data manager who collects and enters data in accordance with the standardized PC⁴ Data Definitions Manual. The PC⁴ registry shares common terminology and definitions with applicable data points from the International Pediatric and Congenital Cardiac Code (IPCCC) (11), Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database, and American College of Cardiology Improving Pediatric and Adult Congenital Treatment (IMPACT) Registry, as previously described (10). Participating centers are audited on a regular schedule and audit results suggest complete, accurate and timely submission of data across centers, with the most recent published results demonstrating a major discrepancy rate of 0.6% across 29,476 fields (12).

The University of Michigan Institutional Review Board provides oversight for the PC⁴ Data Coordinating Center; this study was reviewed and approved with waiver of informed consent.

Inclusion and exclusion criteria

The study population included all surgical hospitalizations between August 1^st, 2014 and June 1^st, 2017. A surgical hospitalization was defined as any that included an index operation according to the Society of Thoracic Surgeons’ criteria (13). Patients were excluded from the analysis if they weighed less than 2.5kg and the only operation was ligation of isolated patent ductus arteriosus, did not receive immediate postoperative care in the CICU, or had a tracheostomy in situ at the time of the index operation.

For the analysis of EF, patients whose first planned postoperative extubation occurred in the setting of withdrawal of care or who never had a planned extubation were excluded. For calculation of POMV, patients who died before postoperative day 7 were excluded in order to align with the exclusion criteria in our previous analysis (1).

Outcome and predictor variables to calculate adjusted extubation failure rates

The primary outcome of the analysis was EF, which we defined as reintubation within 48 hours of the patient’s first planned extubation after his/her index surgical procedure. We did not consider the use of noninvasive respiratory support as part of the definition for extubation failure. Our definition is consistent with our previous analysis and the majority of literature in pediatric critical care (8, 14, 15). The aim of this study was to measure case-mix adjusted EF and profile hospital performance on this metric.

Candidate predictors included both demographic and operative variables. Patients were categorized as preterm neonate (<30 days old and <37 weeks gestation), neonate (<30 days old and ≥37 weeks gestation), infant (30–365 days), child (1–18 years), or adult (>18 years). Weight-for-age z-scores were calculated using World Health Organization or Centers for Disease Control standards, according to patient age (16). Other demographic variables included sex, prematurity (<37 weeks gestational age), and the presence of any extracardiac or chromosomal anomalies, syndromes, or airway anomalies according to IPCCC definitions (11). Airway anomalies include tracheoesophageal fistula, congenital tracheal stenosis, laryngomalacia, tracheomalacia, bronchomalacia, or other major abnormality of the larynx-trachea-bronchus. Surgical complexity was classified according to the Society of Thoracic Surgeons-European Association for Cardiothoracic Surgery (STAT) mortality categories (17). For the purposes of this study, a major preoperative risk factor was considered to be one of the following: hepatic dysfunction, chest compressions within the previous 48 hours, mechanical circulatory support (extracorporeal membrane oxygenation or ventricular assist device), shock at the time of surgery, neurological deficit, or renal dysfunction/failure.

It is important to note that the purpose of this analysis was not to identify all predictors of EF; we aimed to measure case-mix adjusted EF across hospitals. To achieve this objective, the investigative team excluded variables from the analysis that were not primarily reflective of case mix and could instead be influenced by quality of care. Potential factors associated with EF and reflective of case mix were chosen based on previous literature (8). All had to be present prior to the first planned extubation.

Statistical Analysis

Data are presented as frequency (percentage) for categorical variables and median with interquartile range for continuous variables. The univariate analyses included chi-square, Fisher’s exact test, or Wilcoxon rank sum test, used as appropriate, to determine associations between candidate predictors and EF. Those associated with the outcome p<0.1 were included in a multivariable hierarchical logistic regression model with a hospital-specific random effect to account for clustering of patients within hospitals. Variables that were significant at p<0.05 were retained in the model. We then performed bootstrap resampling (using 1000 resamples) to obtain bias-corrected 95% confidence intervals (CI) for the odds ratio associated with each covariate, and the final model included all variables whose CI did not cross 1. Based on the bootstrap resampling, we calculated a bias-corrected c-statistic for the final model.

The final case-mix adjustment model was used to calculate an observed-to-expected (O/E) ratio of EF for each hospital. The O/E ratio was defined as the observed number of patients with EF divided by the expected number calculated from the model. We empirically derived the 95% CI around the O/E ratio for each hospital from bootstrapping as described above. Statistically better-than-expected or worse-than-expected EF rates were defined as an O/E less or greater than one, respectively, with the 95% CI not crossing 1.

We previously developed and validated a metric of adjusted duration of POMV (1). We used that case-mix adjustment model to calculate expected case-mix adjusted mean POMV for each hospital in the current analytic dataset. The expected lengths of mechanical ventilation were capped at 10 days, and thus, patients with expected lengths of mechanical ventilation longer than that were excluded; we wanted to measure hospital performance for those patients whose expected duration of ventilation was below the upper 5^th percentile. Our previously-derived POMV model demonstrated better fit below the upper 5^th percentile, so we felt it more accurate to restrict analysis of hospital performance on this metric to these patients (1). Additionally, those patients with longer expected POMV times tend to have a skewed impact on a hospital’s overall O/E ratio, thus making it more accurate to compare quality among hospitals without these outliers. O/E ratios for aggregate duration of POMV at each hospital were calculated and statistical outliers were identified in an analogous method as described above for EF rates. After each institution was categorized as better-than, worse-than, or as-expected for both EF and duration of POMV, a 3×3 cross table was constructed to assess hospital performance across the two metrics.

In order to assess the relationship between CICU organizational factors and adjusted EF, we used data from the biannual PC⁴ hospital questionnaire measuring hospital and CICU resources and practice patterns. Variables assessed on this questionnaire include information on physician and nurse staffing, and bed capacity and occupancy, among other domains. We chose certain variables from the survey to analyze as potential predictors based on a priori assessment of biologic plausibility and reasonable variation across hospitals. Candidate predictors included average number of occupied beds in the CICU per day rounding team, average CICU census, average hours of nursing care per patient day, percent of CICU attending physicians board certified in pediatric critical care medicine and/or anesthesiology, CICU-dedicated respiratory therapists, physician assistant or nurse practitioner CICU coverage, proportion of nurses with <2 years experience, board-certified intensivist physician directing the CICU, in-house fellow CICU night coverage, and in-house attending physician CICU night coverage. We then fitted a hierarchical regression model with hospital fixed effects to test associations between hospital characteristics and EF, accounting for the patient and operative factors previously identified. The dependent variable was extubation failure at the patient level.

All analyses were performed using SAS Version 9.4 (SAS Institute, Cary, NC) or STATA Version 14 (Stat Corp, College Station, TX), with statistical significance at a p value of less than 0.05.

Results

Population characteristics

Across the 25 participating PC⁴ hospitals, 16,052 unique hospitalizations met inclusion criteria. Demographics and preoperative characteristics of the entire cohort are summarized in Table 1. STAT category 4 and 5 operations comprised 25.3% of all operations. Less than 5% of the cohort had a diagnosed airway anomaly. Overall in-hospital complication and mortality rates amongst CICU patients in PC⁴ over this time period were 36% and 2.9%, respectively. The median postoperative length of stay was 7 days (adjusted mean was 16 days).

Table 1.

Study population characteristics and univariate analysis of relationship with extubation failure (n = 16,052)

Characteristic	Overall Cohorta	Extubation Failure (N = 868)	Extubation Success (N=15,184)	p
Male gender	8887 (55.4%)	497 (57.3%)	8390 (55.3%)	0.24
Age	284 (84–2036)	97.5 (10–371.5)	316 (93–2141.5)	<0.001
Preterm neonate	359 (2.2%)	60 (6.9%)	299 (2.0%)
Full-term neonate	2584 (16.1%)	258 (29.7%)	2326 (15.3%)
Infant	5558 (34.6%)	332 (38.2%)	5226 (34.4%)
Child	6681 (41.6%)	194 (22.4%)	6487 (42.7%)
Adult	870 (5.4%)	24 (2.8%)	846 (5.6%)
Weight statusb				<0.001
Underweight	3780 (23.6%)	286 (33.0%)	3494 (23.0%)
Normal	11756 (73.2%)	563 (64.9%)	11193 (73.7%)
Overweight	516 (3.2%)	19 (2.2%)	497 (3.3%)
STAT categoryc				<0.001
1	4629 (28.8%)	125 (14.4%)	4504 (29.7%)
2	5110 (31.8%)	224 (25.8%)	4886 (31.2%)
3	2028 (12.6%)	104 (12.0%)	1924 (12.7%)
4	3464 (21.6%)	319 (36.8%)	3145 (20.7%)
5	595 (3.7%)	84 (9.7%)	511 (3.4%)
Not assigned	226 (1.4%)	12 (1.4%)	214 (1.4%)
Airway anomaly	697 (4.3%)	96 (11.1%)	601 (4.0%)	<0.001
Chromosomal anomaly/syndrome	3397 (21.2%)	261 (30.1%)	3136 (20.6%)	<0.001
Ventilation at time of surgery	988 (6.2%)	113 (13.0%)	875 (5.8%)	<0.001
Cardiopulmonary bypass time
No bypass time	2483 (15.5%)	147 (16.9%)	2336 (15.4%)	0.22
Bypass time, if yes	98 (66–147)	119 (84–175)	97 (65–145)	<0.001

^aN (%) or Median (Interquartile range), as appropriate.

^bWeight-for-age Z scores calculated using World Health Organization or Centers for Disease Control standards, according to patient age.

^cSociety of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Category.

Case-mix adjusted extubation failure rates across hospitals

The unadjusted extubation failure rate in the overall cohort was 5.4%. Table 1 shows the univariate associations between all candidate predictors and EF. Age at time of index operation, weight status, higher STAT category, airway anomaly, chromosomal anomaly or syndrome, mechanical ventilation at the time of surgery, and increased cardiopulmonary bypass time were all significantly associated with EF.

Table 2 shows the results of the multivariable analysis to determine a final model for calculating hospital-level case-mix adjusted EF rates. Those variables independently associated with EF (p<0.05 in final model) included younger age, underweight, greater surgical complexity, airway anomaly, chromosomal anomaly or syndrome, longer cardiopulmonary bypass time, and major preoperative comorbidities. Preterm neonate status (adjusted odds ratio 4.5, (95% CI) 3.2–6.5) and airway anomalies (2.8, 2.2–3.5) had the largest effect size of any predictors retained in the final model. The c-statistic for this model was 0.73, and the Hosmer-Lemeshow P-value was 0.09.

Table 2.

Final multivariable model for case-mix adjusted extubation failure rate

Covariates	Odds Ratio (95% CI)	p
Agea
Preterm neonate	4.5 (3.2–6.5)	<0.001
Full-term neonate	2.8 (2.2–3.5)	<0.001
Infant	1.8 (1.5–2.2)	<0.001
Adult	0.98 (0.6–1.5)	0.9
Underweight (vs. normal)	1.3 (1.1–1.6)	0.001
STATb 4–5 (vs. STAT 1–3)	1.7 (1.4–2.1)	<0.001
Airway anomaly	2.8 (2.1–3.6)	<0.001
Chromosomal anomaly or syndrome	1.4 (1.2–1.6)	<0.001
Ventilation at the time of surgery	1.2 (0.9–1.5)	0.2
Cardiopulmonary bypass time (10-minute intervals)	1.0 (1.0–1.0)	<0.001
Major preoperative risk Factors	1.5 (1.2–1.9)	0.001
Minor preoperative risk Factors	1.1 (0.9–1.3)	0.3

^aChild is reference.

^bSociety of Thoracic Surgeons-European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Category

Comparison of adjusted extubation failure rates and duration of mechanical ventilation across hospitals.

Figure 1 displays O/E ratios for both hospital performance metrics: case-mix adjusted EF rates and duration of POMV. The hospital O/E ratios for EF ranged from 0.6 to 2.1 (adjusted EF rates: 2.3% to 11.1%). For POMV, the O/E ratios ranged from 0.6 to 2.0 (adjusted mean duration of POMV: 1.6 days to 5.1 days). A 3×3 cross table comparing hospital performance on both metrics is shown in Figure 2. For adjusted EF, three hospitals performed better-than-expected and three hospitals performed worse-than expected. Two of the three hospitals with better-than-expected EF rates also had better-than-expected performance on duration of POMV. All 3 hospitals with worse-than-expected EF rates were also worse-than-expected on duration of POMV. No hospital was an outlier in opposite directions across the two metrics.

Figure 1.

Observed to expected (O/E) ratios of extubation failure rate and duration of postoperative mechanical ventilation. Hospitals are rank ordered with 1 = lowest O/E for extubation failure and 25 = highest O/E for extubation failure.

Figure 2.

Cross-table comparing hospital performance across two quality metrics, adjusted extubation failure rate and adjusted postoperative mechanical ventilation.

Organizational factors associated with extubation failure

Surveys were completed by 24 of 25 participating hospitals. Results of the hierarchical regression to determine associations between CICU staffing and resource factors with EF are shown in Table 3. Greater nursing hours per patient day and higher percentages of CICU nurses with critical care certification were associated with lower odds of EF. Fewer nurses with <2 years of experience, dedicated CICU respiratory therapists, higher percentages of critical care trained attendings, and fewer average patients per attending per day were not associated with lower odds of EF.

Table 3.

Organizational factors associated with extubation failure in multivariable analysis

Covariates	Odds Ratio (95% CI)	p
Higher nursing hours per patient day	0.95 (0.92–0.98)	0.001
Critical care registered nurse percentage	0.99 (0.98–1.00)	<0.001
Nursing staff percentage with <2 years experience	0.90 (0.84–0.97)	0.006
Dedicated respiratory therapist	1.78 (1.32–2.42)	<0.001
Percentage of critical care trained attendings (vs. ≤60%)
60%–90%	1.37 (1.03–1.82)	0.03
>90%	1.92 (1.29–2.85)	0.001
Occupied beds per day team (vs. ≤9 beds)
9–12 beds	0.88 (0.66–1.18)	0.4
>12 beds	0.50 (0.36–0.68)	<0.001
Board certified intensivist directing CICU	1.04 (0.82–1.32)	0.7
Higher CICU average occupancy	1.01 (1.00–1.01)	0.3
24/7 attending coverage	0.88 (0.62–1.26)	0.5

Discussion

We explored the relationship between EF rates and duration of POMV at the hospital-level and found that these two metrics are not inversely related as some have hypothesized. Conversely, some hospitals demonstrated both better-than-expected EF rates and duration of POMV when adjusting for case mix, suggesting that these hospitals may have optimized postoperative mechanical ventilation practices. Additionally, we found that nursing staff characteristics in the CICU including greater nursing hours per patient day and higher percentages of nurses with critical care certification are associated with improved extubation outcomes at the patient level.

Our findings align with previous literature suggesting that increased duration of POMV is associated with increased risk of EF^5,14. The study population that most closely resembles our cohort is from Mahle et al, which focused on the implementation of early extubation clinical practice guidelines after cardiac surgery in infants. That study demonstrated no statistically significant increase in EF (2.5% vs. 3.3%) as early postoperative extubation increased and median duration of POMV dropped from 21.2 hours to 4.5 hours (6). Our work was unique as it showed this relationship between EF and duration of POMV at the hospital level with a cohort of >16,000 episodes. The PC⁴ clinical registry provided us with a broader cohort of patients undergoing more complex procedures, making this the most comprehensive analysis on postoperative extubation failure in pediatric cardiac surgical patients to date.

Our study sheds light on possible opportunities to standardize and improve perioperative mechanical ventilation practices and outcomes after pediatric cardiac surgery. There was almost five-fold variation in EF rates across participating hospitals. Calculating case-mix adjusted EF rates allowed us to compare hospital performance and identify outlying hospitals on both ends of the performance spectrum. In addition to those hospitals with better-than-expected EF and duration of POMV, we found several hospitals with worse-than-expected performance on both metrics. Our experience suggests that it is possible to change practice in CICUs and improve perioperative mechanical ventilation outcomes (6). Understanding the practices at high-performing hospitals and disseminating their knowledge and strategies – particularly to lower-performing hospitals – through collaborative learning could lead to improvement in these important clinical outcomes.

The case-mix adjusted EF rate that we derived in this study should serve as a quality metric for this population moving forward, and we plan to implement it within the PC⁴ reporting platform (pc4quality.org) so that hospitals can benchmark their performance on an ongoing basis. The PC⁴ infrastructure promotes dissemination of practices between hospitals and quality improvement efforts with the potential to improve care throughout the network (10, 18). The morbidity associated with extubation failure appears significant (3, 8, 19), suggesting that ongoing examination of hospital performance in preventing failures might be an important aspect of providing high quality critical care. To our knowledge, a model for hospital-level adjusted EF rates has not been developed to date. Other critical care disciplines could adopt our methods of measuring adjusted EF rates as a step towards improving quality of care.

We gained some insight on how CICU organizational factors may impact the likelihood of EF. Increased nursing hours per patient day and increased percentage of nurses with critical care certification were the only variables associated with a lower likelihood of EF. Multiple studies have illustrated how nursing workload and experience are related to important patient outcomes for the critically ill (20–26). To our knowledge, this is the first evaluation of associations between nurse staffing and extubation failure, though previous literature suggests an association between nurse-to-patient ratios and unplanned extubation (20, 21). Interestingly, increased levels of inexperienced nurses, as measured by percentage of nurses with <2 years experience, was not associated with increased risk of EF. It may be that CICUs have developed staffing strategies that mitigate the impact of inexperience, perhaps through critical care nursing certification programs and/or increasing nursing hours per patient day, as suggested by our analysis.

Surprisingly, several organizational factors hypothesized to be associated with lower odds of extubation failure, such as a lower attending-to-patient ratio, higher percentage of CICU attendings board-certified in critical care, and a dedicated team of respiratory therapists in the CICU, were not associated with better extubation outcomes. In fact, they were associated with higher odds of extubation failure. There are no immediate explanations for these findings. As there are only 24 hospitals in this analysis, we are limited in how we model hospital factors in our hierarchical regression. We would not infer from these data that these factors are causative of extubation failure. It will be imperative to re-evaluate our findings in future analyses with a larger cohort of hospitals.

There are important limitations of our analysis to consider. When determining potential predictors of EF for our model, we were limited to candidate variables collected in the PC⁴ clinical registry. It is possible that there are other variables important for case-mix adjustment that were not measured. Additionally, PC⁴ does not contain the clinical indication for reintubation due to concerns about recording this variable with adequate validity and reliability. Similarly, PC⁴ does not collect information about hospital-specific ventilator weaning and extubation protocols. Studying the reason for reintubation and how specific practice patterns impact hospital performance on the quality metrics evaluated in this study are both prime areas for future research. Our analysis of CICU resource and staffing factors potentially impacting EF rates uses data from a PC⁴ biannual survey and is meant to describe practices that are occurring currently at that center, but it is a cross-sectional survey. Overall, we present two quality metrics that are limited to the domain of perioperative mechanical ventilation outcomes. There are several other important measures of hospital quality in children undergoing pediatric cardiac surgery that could be similarly viewed in the context of EF and duration of POMV, which we did not analyze. Therefore, our conclusions about hospital quality should be limited to this discrete area of perioperative mechanical ventilation. Finally, our methods for analyzing hospital performance on these two metrics only measure outcomes relative to what is expected for that hospital’s case-mix. Thus, ranking hospitals or directly comparing any two or more hospitals is not valid based on these methods.

Conclusions

In conclusion, we showed that certain hospitals are able to achieve low extubation failure rates and also have better-than-expected duration of postoperative mechanical ventilation in pediatric cardiac surgical patients, while others perform on the opposite end of the spectrum. Our method of evaluating these two metrics could serve as a model for assessing quality in pediatric cardiac critical care and other critical care specialties. Further, nurse staffing seemed to be the organizational factor most associated with better extubation failure rates, and it remains to be seen whether this relationship holds for other mechanical ventilation outcomes. Identifying and disseminating the key practices at high-performing hospitals that lead to better outcomes represents the next iterative step to improve outcomes for critically-ill patients.