Morbidity and Mortality Prediction in Pediatric Heart Surgery: Physiological Profiles and Surgical Complexity

Abstract

Objectives

Outcome prediction for pediatric heart surgery has focused on mortality but mortality has been significantly reduced over the last two decades. Clinical care practices now emphasize reducing morbidity. Physiology-based profiles assessed by the PRISM score are associated with new significant functional morbidity detected at hospital discharge. Our aims were to assess the relationship between new functional morbidity and surgical risk categories (RACHS and STAT), measure the performance of three-level (intact survival, survival with new functional morbidity, or death) and two-level (survival or death) PRISM prediction algorithms, and assess whether including RACHS or STAT complexity categories improves the PRISM predictive performance.

Methods

Patients (newborn to <18 years) were randomly selected from seven sites (December 2011 to April 2013). Morbidity (using the Functional Status Scale–FSS) and mortality were assessed at hospital discharge. The most recent published PRISM algorithms were tested for goodness-of-fit, and discrimination with and without the RACHs and STAT complexity categories.

Results

The mortality rate in the 1550 patients was 3.2%. Significant new functional morbidity rate occurred in 4.8%, increasing from 1.8% to 13.9% and 1.7% and 12.9% from the lowest to the highest RACHS and STAT categories, respectively. The 3-level and 2-level PRISM models had satisfactory goodness-of-fit and substantial discriminative ability. Inclusion of RACHS and STAT complexity categories did not improve model performance.

Conclusion

Both mortality and new, functional morbidity are important outcomes associated with surgical complexity and can be predicted using PRISM algorithms. Adding surgical complexity to the physiological profiles does not improve predictor performance.

Graphical Abstract

INTRODUCTION

Outcome prediction for critically ill children following congenital heart surgery has centered on operative mortality. One prominent approach uses the anatomical diagnosis and/or specific operation performed for palliation or repair as the core risk-adjustment methodology. The Risk Adjustment for Congenital Heart Surgery (RACHS) Score relied on subjective assessments of operative risk and cardiac anatomy by congenital heart surgeons and pediatric cardiologists.¹ The most recent method, the 2014 Society for Thoracic Surgery Congenital Heart Surgery Database (STS-CHSD) Mortality Risk Model, estimates risk by calculating an expected rate of mortality that accounts for the operation performed and a number of preoperative variables.^2,3 Mortality risks for individuals are computed using the risk of each combination of primary procedure, age group, and other co-factors to adjust for individual patient factors; recently these co-factors have expanded to include pre-operative ICU clinical factors and therapies.⁴ The risk for inpatient morbidity has been similarly developed.⁵ This approach is the foundation for a major quality program.^3,6

Physiology-based severity of illness methods used in adult, pediatric, and neonatal intensive care for decades have also centered on mortality.^7–10 The Pediatric Risk of Mortality (PRISM) Score is a frequently used, physiologically-based measure that assigns numeric values reflective of mortality risk to derangements of 17 commonly measured physiological variables; the PRISM score is the summation of these values while mortality risk is computed using the PRISM score and other co-factors.⁸ The numeric PRISM score is termed severity of illness.¹¹ PRISM has been a foundation of national quality programs. It has performed well in congenital heart surgery patients consistent with the observation that post-procedure physiological status reflects mortality risk.⁸ Recently, PRISM has undergone a revision of its data collection methods.^12,13 Most importantly, the PRISM outcome algorithm estimates simultaneously the risk of new functional morbidity as well as mortality at hospital discharge.¹³ PRISM algorithms are also available for estimation of mortality risk alone.¹² PRISM prediction algorithms have not been rigorously assessed in a modern cohort of congenital heart surgery patients.

A third approach for pediatric risk assessment is based on general and targeted categorical variables, and a limited set of physiological variables and therapies. The Pediatric Index of Cardiac Surgical Intensive Care Mortality (PICSIM)¹⁴ overlaps with the Pediatric Index of Mortality (PIM) which did not perform well in cardiac surgery patients.^15,16 Since most of PICSIM’s predictive power comes from the surgical complexity score, its use to assess intensive care quality is limited.¹⁷

Mortality rates in pediatric heart surgery and critical care are low and decreasing with rates being reported to be less than 4%.^2,14,18 Yet, modern risk assessment methods continue to focus on operative or intensive care mortality. In contrast, new morbidity rates assessed as functional status changes in critically ill children measured at hospital discharge are approximately twice as high as mortality rates and it has been suggested that functional morbidity is replacing mortality.¹⁹ Recently, the Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network (CPCCRN) developed a granular measure of functional morbidity that is age independent, and sufficiently rapid, accurate, and reliable for population-based outcome studies.²⁰ This method, the Functional Status Scale (FSS), is a significant improvement over common subjective scales.^21,22 Importantly, we recently demonstrated that the development of new functional status morbidities was associated with physiological status early in the ICU course in a manner that parallels the association between physiological status and mortality. Further, we demonstrated that we could simultaneously estimate the risk of both functional morbidity and mortality from data obtained in the first 4 hours of intensive care.¹³

The analyses described in this paper had three specific aims. Our first aim was to examine how the risk of developing new, significant functional morbidity was associated with levels of a physiology-based score, and with the risk categories of the RACHS and STS-CHSD mortality risk (STAT) scores. Second, we assessed the performance of the recently published three-level PRISM prediction algorithms (death, survival with new, significant functional morbidity, and survival without new, significant functional morbidity (intact survival)) and two-level prediction algorithm (survival and death) in a contemporary sample of pediatric heart surgery patients.¹³ This assessment included the performance of an objective algorithm to determine the PRISM observation time because some patients are admitted pre-operatively. Third, we assessed the potential for prediction improvement by including the risk categories from RACHS and STAT and other cardiac descriptors in the PRISM prediction equations.

METHODS

This investigation utilized the cardiovascular surgical patients in the Trichotomous Outcome Prediction in Critical Care (TOPICC) database collected by the CPCCRN. Detailed methods for the TOPICC data collection have been previously described.¹³ The central aim of TOPICC was to assess the relationship between physiological profiles and the development of functional morbidity. In brief, there were seven sites with one site composed of two institutions. Randomly selected patients, newborn to less than 18 years, admitted to participating pediatric and cardiac ICUs from December 4, 2011 to April 7, 2013 were included for analysis and stratified by hospital.¹³ Moribund patients (vital signs incompatible with life for the first two hours after ICU admission) were excluded. Only the first ICU admission during a hospitalization was included. Demographic data were obtained on admission. All participating Institutional Review Boards approved the protocol. Detailed institutional data along with other analyses have been published.^{13,21,23–25} For additional details concerning patient and site level data, outcomes, and physiological data see the supplemental materials (Supplemental Description and Details of the TOPICC Study).

Outcomes

Functional morbidity, mortality, and survival without new functional morbidity were assessed at hospital discharge. New morbidity affecting a significant decrement in functional status was assessed with the FSS for the baseline status (prior to the acute illness requiring ICU admission) and at hospital discharge. The FSS is an age-independent assessment of functional status that can be determined from the medical record or from health care providers’ input.²⁰ It was developed as a granular and objective instrument suitable for large pediatric outcome studies. The six domains (mental status, sensory, communication, motor function, feeding, and respiratory) are individually scored with a range from 1 (normal) to 5 (very severe dysfunction). The operational definitions and manual for the classifications have been published.²⁰ Newborns never achieving a stable baseline are assigned a FSS score of 6; this was operationalized by assigning a FSS of 6 to admissions to the study sites from 0 to 2 days of age and to transfers from another facility from 3–6 days of age. New morbidity was defined as an increase in the FSS score of ≥ 3 points from baseline to hospital discharge; changes of this magnitude indicate substantial worsening of functioning. Previous analysis indicated that over 95% of these children had a change of 2 or more points in a single domain, a clearly significant functional change. Functional morbidity occurs in essentially all ages and types of patients, in relatively equal proportions, and involves all FSS domains.¹⁹

Measurement of Physiological Status

Physiological status was measured with the PRISM score with a shortened time interval (2 hours prior to admission to 4 hours after admission for laboratory data and the first 4 hours of ICU care for other physiological variables). Outcome prediction using this time interval included separation of the total PRISM into neurological and non-neurological components and other patient factors.^12,13

Congenital Cardiac Conditions

Only cardiovascular surgery (CVS) patients were included in this analysis. Classifications as one or two ventricle, cyanotic or acyanotic, and by the RACHS and STAT categories were done by a cardiologist (JTB) based on the anatomical diagnosis and operative procedure in the operative report and the admission diagnostic information and blinded to the outcomes.^18,26 Operations involving combinations of procedures were assigned to the procedure with the highest mortality category. Cyanosis was based on pre-operative anatomy and description from the surgical notes. Patients were classified as one or two ventricle repair based on evidence of ventricular hypoplasia using the type of operation and operative report.

The time interval for assessing PRISM data was modified for cardiac patients < 91 days of age because some institutions admit young infants to the ICU prior to a cardiac intervention to “optimize” clinical status, and not for intensive care; in these cases, the post-intervention period more accurately reflects intensive care. However, in other infants for whom the cardiac intervention is delayed after ICU admission, the intervention is a therapy required due to failed medical management of the acute condition; in these infants, the routine PRISM data collection time interval is an appropriate reflection of critical illness. A priori, we identified infants for whom it would be more appropriate to utilize data from the 4 hours after the cardiac intervention (post-intervention time interval) and those for whom using the admission time interval was more appropriate and operationalized this decision on the conditions likely to present within the first 90 days, the time period when the vast majority of these conditions present. This approach has been detailed elsewhere (Supplemental Table 1).^12,13 We assessed the adequacy of fit, as well as performance, of the PRISM prediction models in the age groups of ≤ 90 days and > 90 days using standardized morbidity and mortality ratios.

Statistical Methods

Statistical analyses utilized SAS 9.4^® (SAS Institute Inc., Cary, NC) for descriptive statistics, model development, and fit assessment, and R 3.0.2 (The R Foundation for Statistical Computing, Vienna, Austria; http://www.wu.ac.at/statmath) for evaluation of predictive ability. The statistical analysis was under the direction of R.H.

Patient characteristics were descriptively compared and evaluated across sites using the Kruskal-Wallis test for continuous variables, and Fisher’s exact test for categorical variables.

Predicted numbers of events were calculated using probabilities from previously published models constructed from the TOPICC cohort and these calculated probabilities were used to determine the predicted outcomes in the analyses.^12,13 Goodness-of-fit of these models was assessed using the Hosmer-Lemeshow test for logistic models and an extension to three outcomes.²⁷ We treated the cardiac cohort as an independent sample in terms of applicable test degrees of freedom, as this cohort is a small subset of the entire population defined per clinical criteria, and includes “validation set” cases not used in the TOPICC model construction. To maintain the validity of the Hosmer-Lemeshow test (for which expected event counts should be ≥ 5 within most evaluated cells), subjects were sorted in order of increasing predicted probability of mortality, and then divided into risk categories each containing approximately seven expected deaths. Reported goodness-of-fit findings were robust to alternate risk category specifications.²⁷ Reported goodness-of-fit findings were robust to the number of such categories used. For reported Standardized Mortality Ratios (SMRs), the Breslow-Day method was used to calculate two-sided 95% confidence intervals.

Discrimination was assessed by two-dimensional receiver operating characteristic (ROC) curves for the survival/death model and by three-dimensional volume under the surface (VUS) for the three-level outcome. Two-dimensional ROC curves were generated, with area under the curve (AUC) calculated and its variability estimated, using the SAS logistic procedure. VUS for discriminating between the three outcomes is reported using the R_II triplet-classification rule of Mossman.²⁸ The VUS has a value of one sixth under a model with no discriminatory ability; we also report the average dichotomized c-index (the average of the areas under the curve considered over all possible ordered dichotomizations of the outcome, whose value with no model discrimination is 0.5) as an alternate summary measure of multidimensional model discriminatory ability.

For assessing whether adding a cardiac measure (RACHS, STAT, single versus two ventricle anatomy, or cyanotic versus acyanotic status) improved the predictive ability of the published PRISM models, the cardiac measure was added as a categorical predictor to a logistic model (dichotomous or trichotomous) that held each patient’s PRISM predicted outcome probabilities fixed using an offset term. The STAT mortality categories were added to our model without the use of additional pre-operative patient characteristics. This modeling used SAS PROC NLMIXED. Significance of improvement for a model including a cardiac-measure predictor was assessed by comparing its likelihood value to that of the published PRISM model applied to this population. We also quantified potential improvement in discrimination via the AUC and VUS.

RESULTS

The overall sample contained 10,078 patients of whom 1550 had a cardiac surgery. Sample characteristics at the site level and overall are shown in Supplemental Table 2 including age, age distribution, STAT categories, ICU and hospital lengths of stay, PRISM scores, outcomes, and the classifications of cyanotic or acyanotic and single or two ventricle anatomy. Of the cardiac interventions, 1199 (77.4%) had 2 ventricle anatomy and 351 (22.6%) were single ventricle patients. A total of 871 (56.2%) were acyanotic and 679 (43.8%) were cyanotic. Based on information available for the interventions performed, the RACHS score was calculable in 1447 of these cardiac patients, while the STATS categorization was achievable among 1534. Overall, the mortality rate was 3.2% and the new functional morbidity rate was 4.8%.

The new functional morbidity and mortality rates for each RACHS and STAT categories are displayed in Table 1 and illustrated for the STAT categories in Figures 1a and 1b. Overall, both the observed and predicted functional morbidity and mortality rates significantly increased with increasing RACHS and STAT categories. The only exception was the RACHS 5 category, which had too few cases for statistical stability. In particular, the new functional morbidity rates increased from 1.8% to 13.9% and 1.7% and 12.9% from the lowest to the highest severity categories for RACHS and STAT, respectively.

Table 1. Observed and Predicted Mortality and New Functional Morbidity in RACHS and STAT Categories.

Both observed and predicted mortality and functional morbidity rates increased with increasing severity categories for both systems (both p < 0.0001 Predicted new functional morbidity rates also increased with increasing severity categories (RACHS: p=0.0032; STAT: p=0.0009).

RACHS	N	Median Age (Months)	Crude Mortality N (%)	Predicted Mortality N (%)	Mortality SMR (95% CI)	Crude New Morbidity N (%)	Predicted Morbidity N (%)	Morbidity SMR (95% CI)
1	114	55	0 (0.0%)	0.7 (0.6%)	0 (NA-5.3)	2 (1.8%)	2.9 (2.5%)	0.7 (0.1–2.5)
2	585	6	10 (1.7%)	9.2 (1.6%)	1.1 (0.5–2.0)	15 (2.6%)	21.7 (3.7%)	0.7 (0.4–1.1)
3	517	9	17 (3.3%)	16.2 (3.1%)	1.1 (0.6–1.7)	24 (4.6%)	24.1 (4.7%)	1.0 (0.6–1.5)
4	149	0	8 (5.4%)	11.5 (7.7%)	0.7 (0.3–1.4)	15 (10.1%)	10.5 (7.1%)	1.4 (0.8–2.4)
5	3	0	0 (0.0%)	0.3 (11.2%)	0 (NA-10.9)	0 (0.0%)	0.3 (9.7%)	0 (NA-12.2)
6	79	0	9 (11.4%)	10.4 (13.1%)	0.9 (0.4–1.7)	11 (13.9%)	7.7 (9.7%)	1.4 (0.7–2.6)
Unable to classify	103	69	6 (5.8%)	1.5 (1.5%)	3.9 (1.4–8.5)	7 (6.8%)	3.8 (3.7%)	1.9 (0.7–3.8)
STAT
1	423	28	2 (0.5%)	3.8 (0.9%)	0.5 (0.1–1.9)	7 (1.7%)	12.7 (3.0%)	0.5 (0.2–1.1)
2	513	15	10 (1.9%)	8.2 (1.6%)	1.2 (0.6–2.2)	11 (2.1%)	18.5 (3.6%)	0.6 (0.3–1.1)
3	205	5	6 (2.9%)	6 (2.9%)	1.0 (0.4–2.2)	11 (5.4%)	9.7 (4.7%)	1.1 (0.6–2.0)
4	308	0	21 (6.8%)	20.6 (6.7%)	1.0 (0.6–1.6)	31 (10.1%)	21.1 (6.9%)	1.5 (0.99–2.1)
5	85	0	9 (10.6%)	10.7 (12.6%)	0.8 (0.4–1.6)	11 (12.9%)	8.1 (9.5%)	1.4 (0.7–2.4)
Unable to classify	16	40	2 (12.5%)	0.5 (2.8%)	4.4 (0.5–15.8)	3 (18.8%)	0.8 (5.2%)	3.6 (0.7–10.6)

Abbreviations: SMR = Standardized Morbidity/Mortality Ratio

Figure 1. a and b. Observed and Predicted New Functional Morbidity and Mortality for STAT Mortality Categories.

Both observed and predicted functional morbidity and mortality rates increased with increasing STAT mortality categories (p < 0.0001). See Table 1 for details.

Next, we tested the performance of the PRISM 3-level prediction model predicting intact survival, new functional morbidity at hospital discharge, and death. Initially, we assessed the performance of the PRISM prediction models in those ≤ 90 days of age and those > 90 days of age. The standardized morbidity and mortality ratios performed well, indicating the decision matrix for assigning the PRISM observation period was sufficient (Supplemental Table 3). In assessing the model performance, we first used the categories of RACHS (combining levels 1 with 2 and 5 with 6 due to small numbers of within-cell events) and STAT for the severity categories for the goodness-of-fit risk groups. Both RACHS and STAT (Table 1) demonstrated acceptable fit (RACHS: chi-square = 6.972, df = 8, p=0.540; STAT: chi-square = 13.558, df = 10, p=0.19) Next, we used 7 risk categories constructed with at least 7 expected mortalities in each cell to assess the goodness-of-fit for the intact survival/new morbidity/death (Table 2) and survival/death models (Table 3). Overall, for the 3-level model, 49.8 deaths were predicted and 50 were observed (standardized mortality ratio = 1.0) and 71.0 new functional morbidities were predicted and 74 were observed (standardized morbidity ratio = 0.96). The goodness-o-fit was acceptable (p = 0.31). Discriminative ability was excellent with a volume under the surface of 0.46 (versus a chance value of 0.17). The average dichotomized c-index for this population was 0.82. For the dichotomous model, 50.1 deaths were expected and 50 were observed (standardized mortality ratio = 0.86). The goodness-of-fit was acceptable (p = 0.474). The AUC of the survival/death model was 0.83.

Table 2. Goodness of Fit Test for the New Functional Morbidity-Intact Survival-Death Model.

The Hosmer-Lemeshow chi-square test statistic = 16.036 (14 degrees of freedom), p = 0.31. The volume under the surface was 0.46 (chance = 0.17). The Standardized Mortality and Morbidity Ratios were 1.00 and 0.96 respectively.

	Deaths			Morbidity
Risk Group	E	O	SMR (95% CI)	E	O	SMR (95% CI)
0	7	13	1.9 (1.0–3.2)	30	24	0.8 (0.5–1.2)
1	7	5	0.7 (0.2–1.7)	14.3	20	1.4 (0.9–2.2)
2	7	5	0.7 (0.2–1.7)	9.2	12	1.3 (0.7–2.3)
3	7	8	1.1 (0.5–2.2)	6.8	7	1.0 (0.4–2.1)
4	7	6	0.9 (0.3–1.9)	5.2	7	1.3 (0.5–2.8)
5	7.4	8	1.1 (0.5–2.1)	3.8	3	0.8 (0.2–2.3)
6	7.3	5	0.7 (0.2–1.6)	1.7	1	0.6 (0.0–3.2)
Total	49.8	50	1.0 (0.7, 1.3)	71	74	1.0 (0.8, 1.3)

O = Observed; E = Expected; SMR = Standardized Mortality Ratio

Table 3. Goodness of Fit test for the Survival-Death Model.

The Hosmer-Lemeshow chi-square test = 6.58 (7 degrees of freedom), p = 0.474. The area under the curve 0.83 (+/− 0.03). The Standardized mortality ratio was 1.0.

Risk Group	E	O	SMR (95% CI)
0	7.0	8	1.1 (0.5–2.2)
1	7.0	11	1.6 (0.8–2.8)
2	7.1	4	0.6 (0.2–1.5)
3	7.1	7	1.0 (0.4–2.0)
4	7.0	7	1.0 (0.4–2.1)
5	7.1	8	1.1 (0.5–2.2)
6	7.8	5	0.6 (0.2–1.5)
Total	50.1	50	1.0 (0.7, 1.3)

O = Observed; E = Expected; SMR = Standardized Mortality Ratio

The standardized mortality and morbidity ratios of the dichotomous and trichotomous predictors in the clinical categories of cyanotic or acyanotic and single or two ventricle lesions are shown in Table 4. The prediction performance based on standardized mortality ratios was acceptable in all groups. Finally, we assessed the potential improvement in model performance by separately adding the RACHS categories, STAT categories, cyanotic/acyanotic factor, and single/two ventricle factor to the PRISM prediction models. Table 5 displays the significance level for adding each factor, and the improvement in the VUS or AUC if the factor is added. In all cases, inclusion of the factor did not significantly improve the model performance.

Table 4.

Standardized Morbidity and Mortality Ratios for the Cyanotic/Acyanotic and One/Two Ventricle Classifications.

		Survival-Death Model			Morbidity-Intact Survival-Death Model
		Mortality			Morbidity			Mortality
Variable	N	O	E	SMR (95% CI)	O	E	SMR (95% CI)	O	E	SMR (95% CI)
Cyanotic	679	39	35.7	1.1 (0.8–1.5)	45	40.0	1.1 (0.8–1.5)	39	36.0	1.1 (0.8–1.5)
Acyanotic	871	11	14.4	0.8 (0.4–1.4)	29	31.0	0.9 (0.6–1.3)	11	13.7	0.8 (0.4–1.4)
Single ventricle	351	21	16.7	1.3 (0.8–1.9)	28	20.0	1.4 (0.9–2.0)	21	16.7	1.3 (0.8–1.9)
Two Ventricles	119 9	29	33.4	0.9 (0.6–1.2)	46	51.0	0.9 (0.7–1.2)	29	33.0	0.9 (0.6–1.3)

O = Observed; E = Expected; SMR = Standardized Morbidity/Mortality Ratio.

Table 5.

Significance of Adding RACHS, STAT, Cyanotic/Acyanotic and Single/Two Ventricle Co-Variates to the PRISM Prediction Models.

	Morbidity-Intact Survival-Death (Trichotomous) Model		Survival-Death (Binary) Model
Factor	Significance Level (3)	VUS With/Without Factor	Significance Level (3)	AUC With/Without Factor
RACHS (1)	p = 0.53	0.483 / 0.497	p = 0.78	0.854 / 0.854
STAT (2)	p = 0.16	0.472 / 0.490	p = 0.83	0.836 / 0.842
Cyanotic-Acyanotic	p = 0.75	0.457 / 0.467	p = 0.50	0.830 / 0.832
Single-two ventricle	p = 0.19	0.457 / 0.466	p = 0.37	0.830 / 0.832

¹RACHS categories 1 and 2, and categories 5 and 6, were combined to achieve sufficient numbers of outcomes in category levels to allow model convergence.

²All 5 STAT categories were used in modeling.

³For the likelihood ratio test, adding the factor to a model with outcome probabilities fit using the published model coefficients.

Abbreviations: VUS = Volume Under the Surface; AUC = Area Under the Curve.

DISCUSSION

Mortality from both pediatric heart surgery and pediatric ICUs has fallen to low rates, making mortality an insensitive outcome for care assessments and therapeutic studies without very large samples. Since much of pre-and post-operative care focuses on reducing functional morbidity as well as mortality, functional status is an important outcome. In this pediatric cardiac surgery population, the overall rate of significant, new functional morbidity was 50% higher than mortality; in the general ICU population, this rate is approximately twice as high as mortality. Importantly, the new functional morbidity risk increased over three-fold from the lowest to the highest surgical risk categories.

The PRISM models estimating functional morbidity and mortality risk performed well. Discrimination for mortality in these models is similar to the older PRISM models¹⁴ even though the observation time is substantially shorter, hospital outcome is used which has been harder to predict, only the first ICU admission is included, and the data sampling period is objectively assigned based on age and time to intervention. Importantly, the PRISM methodology was specifically developed to minimize the potential for institutional bias or “gaming” at the expense of model performance. For example, the observation time was chosen to minimize the potential for institutional care practices to affect the PRISM score,²⁹ modeling of hospital outcome was specifically chosen instead of ICU outcome to minimize the effect of premature ICU discharge with readmission, and the objective process to determine the sampling time period for heart surgery infants ≤ 90 days was created to this accommodate inter-center variability.

The discrimination is slightly less than the reported discrimination in the new STS-CHSD model and the PICSIM Score.^2,14 The PICSIM score uses post-operative therapies as well as a 12-hour post-operative sampling period for some of the physiological variables. The use of post-operative therapies in risk models can create bias. Although their inclusion would improve predictor performance, therapies are intentionally not included in the PRISM models because separating physiology from therapy allows independent assessment of the timely and appropriate use of therapy (quality of care).

Importantly, adding surgical complexity classifications to the physiology-based model did not improve model performance, indicating that the physiology-based PRISM score captured most of the information concerning surgical complexity. Since the relationship between functional morbidity and physiological status is sufficiently precise for accurate functional morbidity prediction, we believe that functional morbidity risk as well as mortality risk is reflected in large part through post-operative physiological status. However, we do not have direct confirmation of this causal relationship. Other conditions associated with congenital heart disease could be contributing to discharge functional status.^30–32

There are two general uses for prediction models such as the ones presented in this analysis. First, they can focus on evaluating of systems by adjusting for patient characteristics. Our analyses focused on this use. The advantage to the PRISM models based on post-operative physiological profiles is that they more directly assess ICU performance. Since the STS-CHSD and PICSIM model performances are based predominantly on the surgical procedure performed,^2,14 they assess risk at the time the patient enters the operating room while PRISM assesses risk when the patient enters the ICU. Methods such as the STS-CHSD better assess the whole system including the diagnostic assessment, determining the operative approach, surgical and anesthesia operative performance, and pre- and post-operative care. Therefore, the two approaches are complementary. We believe that if ICU assessment is paramount, a physiology-based approach is preferable. Second, prediction models potentially can be used at the individual patient level. Our analyses presenting performance and outcomes within subpopulations defined by various risk criteria have not been focused on this use.

There are potentially significant limitations to this analysis. First, the sample size is relatively small in comparison to other similar studies. Although the sample size is sufficient to uncover major influences on the PRISM models, it is possible that a larger sample would have uncovered other issues with significant, but weaker influences on the model. Second, it was assumed that newborns had a normal baseline functional status because they never achieved a baseline state other than their in utero condition. Although the PRISM models perform well in all age groups including neonates and young infants, we have been unable to rigorously test this assumption.

Several challenges remain in this new era of outcome assessment. First, do assessment methods change the quality of care in individual institutions? We lack sufficient evidence that the time and effort spent collecting these data are appropriately used by the participating institutions to improve care. The efforts to ensure reliable methods with relevant outcomes that are unbiased are foundational to the need to evaluate and improve care. Second, we need to better understand the relationship between hospital discharge and long-term outcomes for all types of critically ill patients, including pediatric cardiac surgery patients. Long-term outcomes are an important aspect of the effectiveness of care, but the long observation times make this difficult and challenging. A better understanding of the relationship between short-term and long-term outcomes would enable us to assess and improve short-term outcomes with the security that it would translate into improved long-term outcomes.

In summary, there is strong relationship between new, significant functional morbidity at hospital discharge and surgical complexity as well as post-operative physiological status. Since new functional morbidity is an important patient outcome that is substantially more common than mortality, it should be included as an outcome in quality and other studies for children following congenital heart surgery.

Perspective Statement.

Mortality is infrequent while new functional morbidity at hospital discharge is common after congenital heart surgery. Studies focused on mortality may miss meaningful clinical issues and require large samples. We found that new functional morbidity at hospital discharge as well as mortality increased with increasing surgical risk and can be simultaneously predicted by a physiology-based algorithm.

Central Message.

New, functional morbidity is associated with surgical complexity and can be predicted with mortality by a physiology-based algorithm.

Central Picture

New functional status morbidity increases with STAT mortality categories and can be accurately predicted.

Abbreviations

CPCCRN

Collaborative Pediatric Critical Care Research Network

FSS

Functional Status Scale

PRISM

Pediatric Risk of Mortality

RACHS

Risk Adjustment for Congenital Heart Surgery

PICSIM

Pediatric Index of Cardiac Surgical Intensive Care Mortality

TOPICC

Trichotomous Outcome Prediction in Critical Care

ROC

receiver operating characteristic

VUS

volume under the surface

AUC

area under the curve

df

degrees of freedom

Biography