Abstract
Objectives
Severity of illness measures have long been used in pediatric critical care. The Pediatric Risk of Mortality is a physiologically based score used to quantify physiologic status, and when combined with other independent variables, it can compute expected mortality risk and expected morbidity risk. Although the physiologic ranges for the Pediatric Risk of Mortality variables have not changed, recent Pediatric Risk of Mortality data collection improvements have been made to adapt to new practice patterns, minimize bias, and reduce potential sources of error. These include changing the outcome to hospital survival/death for the first PICU admission only, shortening the data collection period and altering the Pediatric Risk of Mortality data collection period for patients admitted for “optimizing” care before cardiac surgery or interventional catheterization. This analysis incorporates those changes, assesses the potential for Pediatric Risk of Mortality physiologic variable subcategories to improve score performance, and recalibrates the Pediatric Risk of Mortality score, placing the algorithms (Pediatric Risk of Mortality IV) in the public domain.
Design
Prospective cohort study from December 4, 2011, to April 7, 2013.
Measurements and Main Results
Among 10,078 admissions, the unadjusted mortality rate was 2.7% (site range, 1.3–5.0%). Data were divided into derivation (75%) and validation (25%) sets. The new Pediatric Risk of Mortality prediction algorithm (Pediatric Risk of Mortality IV) includes the same Pediatric Risk of Mortality physiologic variable ranges with the subcategories of neurologic and nonneurologic Pediatric Risk of Mortality scores, age, admission source, cardiopulmonary arrest within 24 hours before admission, cancer, and low-risk systems of primary dysfunction. The area under the receiver operating characteristic curve for the development and validation sets was 0.88 ± 0.013 and 0.90 ± 0.018, respectively. The Hosmer-Lemeshow goodness of fit statistics indicated adequate model fit for both the development (p = 0.39) and validation (p = 0.50) sets.
Conclusions
The new Pediatric Risk of Mortality data collection methods include significant improvements that minimize the potential for bias and errors, and the new Pediatric Risk of Mortality IV algorithm for survival and death has excellent prediction performance.
Keywords: critical care, intensive care, outcome prediction, pediatric critical care, pediatric intensive care, pediatric risk of mortality, pediatrics, physiologic status, quality, quality assessment, severity of illness
Severity of illness measures have been used in pediatric critical care for decades (1–4). The Pediatric Risk of Mortality (PRISM) score is a frequently used, physiologically based severity of illness measure using 17 commonly measured physiologic variables and their ranges (5). The PRISM score is a quantification of physiologic status using predetermined physiologic variables and their ranges that use categorical variables to facilitate accurate estimation of mortality risk (5). PRISM is commonly used to control for severity of illness in studies and to assess quality of care through standardized mortality ratios (SMRs). Recently, we demonstrated that physiologic status as measured with PRISM variables and their ranges is significantly associated with morbidity and mortality and could be used to simultaneously estimate morbidity and mortality risk (6).
Recently, there have been multiple changes to the data collection process for the PRISM score. First, the time period for measuring PRISM has changed. Physiologic variables are measured only in the first 4 hours of PICU care, and laboratory variables are measured in the time period from 2 hours before PICU admission through the first 4 hours (7). This time period was chosen to best separate the predictor variables from therapy while ensuring that there would be no institutional bias because of practice pattern differences in the timing and frequency of variable measurement. Second, only the first PICU admission in any hospitalization is included and outcome at hospital discharge (instead of at PICU discharge) is predicted (6). This change was made because the appropriateness of the PICU discharge decision should be included in quality assessments. Third, the institutionally based practice of admitting patients before surgery, especially cardiovascular surgery, required adjustment of the PRISM observation period because the presurgical admission period does not reflect the critical care portion of the admission if it is for observation or “optimizing” preoperative status. We developed a bias-free logic for classifying these patients (6). In addition, the relative values of physiologic instability in different systems may have drifted over time and could be assessed by adjusting for the weighting in the PRISM physiologic variable subcategories of cardiovascular, neurologic, metabolic, chemistry, and hematologic groupings. Therefore, although the PRISM score for physiologic variables and their ranges did not change (5), the prediction performance might be enhanced by assessing for this change.
Recently, the Collaborative Pediatric Critical Care Research Network (CPCCRN) conducted the Trichotomous Outcome Prediction in Critical Care (TOPICC) study. TOPICC demonstrated that physiologic status measured by PRISM physiologic variables and their ranges was associated with the risk for significant new morbidity and mortality and developed prediction algorithms for the simultaneous prediction of both significant new morbidity and mortality (6). Although we have recommended the evolution of pediatric outcome predictors to include significant morbidity and mortality, this change will take time. Therefore, using the TOPICC dataset, we revised the PRISM prediction algorithms for the dichotomous outcomes of survival versus death using the most recent changes to the collection of PRISM data. We hypothesized that these changes would not alter the predictive value of the model. This study reports the results of that analysis and opens the prediction algorithms (PRISM IV) for the public domain.
METHODS
This investigation was performed in the CPCCRN of the Eunice Kennedy Shriver National Institute of Child Health and Human Development (8). Detailed methods for the TOPICC data collection have been previously described (6). There were seven sites, and one was composed of two institutions. In brief, patients from newborn to less than 18 years were randomly selected and stratified by hospital from December 4, 2011, to April 7, 2013. Patients from both general/medical and cardiac/cardiovascular PICUs were included. Moribund patients (vital signs incompatible with life for the first 2 hr after PICU admission) were excluded. Only the first PICU admission during hospitalization was included. The protocol was approved by all participating institutional review boards. Other analyses using this database have been published (6, 7, 9, 10).
Data included descriptive and demographic information (Table 1). Interventions included both surgery and interventional catheterization. Cardiac arrest included closed chest massage within 24 hours before hospitalization or after hospital admission but before PICU admission. Admission source was classified as emergency department, inpatient unit, postintervention unit, or admission from another institution. Diagnosis was classified by the system of primary dysfunction based on the reason for PICU admission; cardiovascular conditions were classified as congenital or acquired.
TABLE 1.
Patient Descriptive Characteristics: Site Ranges and Overall Statistics (Site details Are Reported Elsewhere [6])
| Variable | Site Range | Overall |
|---|---|---|
| Sample size (n) | 1,252–1,617 | 10,078 |
| Median age (yr) | 3.2–4.1 | 3.7 |
| Primary system of dysfunction (%) | ||
| Respiratory | 20.4–43.1 | 33.5 |
| Cardiovascular disease | 14.3–38.2 | 24.1 |
| Neurologic | 15.5–24.1 | 20.1 |
| Othera | 19.7–26.4 | 22.3 |
| Admitted for postintervention care (%)b | 27.2–49.8 | 37.7 |
| Emergency admissions (%) | 47.6–70.4 | 63.6 |
| Elective admissions (%) | 29.6–52.4 | 36.4 |
| Cardiac arrest before PICU admission (%)c | 1.0–2.2 | 1.4 |
| Median Pediatric Risk of Mortality score | 0–3 | 2 |
| Median hospital length of stay (d) | 4.0–7.0 | 4.9 |
| Unadjusted mortality rate (%) | 1.3–5.0 | 2.7 |
Other includes unknown.
Interventions included operations and interventional catheterizations.
Closed chest cardiac massage occurring within 24 hr before hospital admission or during the hospitalization before the PICU admission.
The primary outcome in this analysis was hospital survival versus death.
Physiologic status was measured using the PRISM physiologic variables (5) with a shortened time interval (2 hr before PICU admission to 4 hr after admission for laboratory data and the first 4 hr of PICU care for other physiologic variables). For model building, the PRISM components were separated into cardiovascular (heart rate, systolic blood pressure, and temperature), neurologic (pupillary reactivity and mental status), respiratory (arterial Po2, pH, Pco2, and total bicarbonate), chemical (glucose, potassium, blood urea nitrogen, and creatinine), and hematologic (WBC count, platelet count, prothrombin, and partial thromboplastin time) components, and the total PRISM was also separated into neurologic and non-neurologic categories.
The time interval for assessing PRISM data was modified for cardiac patients under 91 days old because some institutions admit infants to the PICU before a cardiac intervention to “optimize” the clinical status but not for intensive care; in these cases, the postintervention period more accurately reflects intensive care. However, in other infants for whom the cardiac intervention is delayed after PICU admission or the intervention is a therapy required because of failed medical management of the acute condition, the routine PRISM data collection time interval is an appropriate reflection of critical illness. Therefore, we identified infants for whom it would be more appropriate to use data from the 4 hours after the cardiac intervention (postintervention time interval) and those for whom using the admission time interval was more appropriate. We operationalized this decision on the conditions likely to present within the first 90 days, the time period when the vast majority of these conditions present (Table 2).
TABLE 2.
Pediatric Risk of Mortality Sampling Intervals for Cardiac Patients Receiving an Intervention
| Age at admission | ICU length of Stay before Cardiac Intervention |
Intervention | Pediatric Risk of Mortality III Sampling Interval |
|---|---|---|---|
| < 24 hr | < 12 hr | Cardiac surgery or catheterization | Admission |
| 12hr to 10 d | Cardiac surgery or catheterization | Postintervention | |
| 24hr to 10 d | 0 to 10 d | Cardiac surgery or catheterization | Postintervention |
| > 10 d | Cardiac surgery or catheterization | Admission | |
| 11 to 30 d | < 48 hr | Cardiac surgery or catheterization | Postintervention |
| > 48 hr | Cardiac surgery or catheterization | Admission | |
| 31 to 90 d | < 48 hr | Cardiac surgery | Postintervention |
| < 48 hr | Cardiac catheterization | Admission | |
| > 48 hr | Cardiac surgery or catheterization | Admission | |
| > 90 d | All | Cardiac surgery or catheterization | Admission |
The admission time interval refers to the period of the 2 hr before PICU admission to 4 hr after admission for laboratory data and the first 4 hr of PICU care for other physiologic variables. The postintervention time interval refers to the first 4 hr of PICU care after a cardiac intervention (surgery or interventional catheterization but not diagnostic catheterization).
Statistical analyses used SAS 9.4 (SAS Institute Inc., Cary, NC) for descriptive statistics, model development, and fit assessment and R 3.1.1 (The R Foundation for Statistical Computing, Vienna, Austria; http://www.wu.ac.at/statmath) for evaluation of predictive ability. Patient characteristics were descriptively compared and evaluated across sites using the Kruskal-Wallis test for continuous variables and the Pearson chi-square test for categorical variables. The statistical analysis was under the direction of R.H.
The dataset was randomly divided into a derivation set (75%) for model building and a validation set (25%) stratified by the study site. Univariate mortality odds ratios were computed, and variables with a significance level of less than 0.1 were considered candidate predictors for the final model. As was the case for the previously published trichotomous (death, survival with significant new morbidity, and intact survival) model construction, a nonautomated (examined by biostatistician and clinician at each step) backward stepwise selection approach was used to select factors. Multicategorical factors (e.g., diagnostic categories) had factors combined when appropriate per statistical and clinical criteria. Clinician input was included (and paramount) in this process to ensure that the model fit was relevant and consistent with clinical information. Construction of a clinically relevant, sufficiently predictive model using predictors readily available to the clinician took precedence over inclusion based solely on statistical significance. We were cognizant of the existing trichotomous outcome model and attempted, when statistically justified, to create a compatible two-outcome model that could aid in a smooth transition to using the three-outcome approach.
Final candidate models were evaluated based on 2D receiver operating characteristic (ROC) curves (discrimination) and the Hosmer-Lemeshow goodness of fit (calibration). For the entire dataset, goodness of fit with respect to key subgroups was assessed by examining SMRs for descriptive and diagnostic categories not used in the final model. Only categories with at least 10 outcomes in observed and expected cells were used.
RESULTS
There were 10,078 patients included from the seven sites. The site ranges and summary data are presented in Table 1. Individual site data have been presented elsewhere (6). The distribution of all patient characteristics except cardiac arrest varied significantly between sites (p < 0.001). The unadjusted mortality rate was 2.7% (site range, 1.3–5.0%).
Initially, we assessed the univariate mortality odds ratios in the development dataset to select variables for inclusion into the final model building process (Supplemental Table 1, Supplemental Digital Content 1, http://links.lww.com/PCC/A203). The total PRISM physiologic variable score and each of its subcategories were statistically significant. Of the categorical variables, age, admission source, admission status, cardiac arrest with 24 hours of PICU admission, interventional classification, cancer, and primary system of dysfunction were statistically significantly associated with mortality.
The final dichotomous (survival and death) model is found in Table 3. Two of the age categories, 14 days to less than 1 month and 1 month to less than 12 months, were significant only at the level of p value less than 0.10 but were included separately to maintain a parallel structure to the trichotomous predictor and because this stratification better reflects the age categories that were significant in previous models. The area under the ROC for the development and validation sets was 0.88 ± 0.013 and 0.90 ± 0.018, respectively. The Hosmer-Lemeshow goodness of fit statistics indicated adequate model fit for both the development (p = 0.39) and validation (p = 0.50) sets (Table 4).
TABLE 3.
Final Outcome Model for Mortality From the Development Set (n = 7,560)
| Variable | Mortality Coefficient (se) | Mortality Odds Ratio (95% CI) |
|---|---|---|
| Intercept | −5.776 (0.234) | |
| Age (vs ≥ 12 mo) | ||
| 0 to < 14 d | 1.311 (0.255) | 3.708 (2.251–6.107) |
| 14 d to < 1 mo | 0.968 (0.553) | 2.632 (0.891–7.773) |
| 1 to < 12 mo | 0.357 (0.205) | 1.429 (0.956–2.135) |
| Admission source (vs operating room or postanesthesia care unit) | ||
| Another hospital | 1.012 (0.234) | 2.750 (1.739–4.349) |
| Inpatient unit | 1.626 (0.249) | 5.085 (3.124–8.278) |
| Emergency department | 0.693 (0.250) | 1.999 (1.224–3.263) |
| Cardiopulmonary resuscitation within 24 hr before PICU admission |
1.082 (0.319) | 2.949 (1.580–5.507) |
| Cancer (acute or chronic) | 0.766 (0.256) | 2.152 (1.304–3.551) |
| Low-risk systems of primary dysfunctiona | −1.697 (0.605) | 0.183 (0.056–0.600) |
| Pediatric Risk of Mortality physiologic variable scoreb | ||
| Neurologic | 0.197 (0.018) | 1.218 (1.176–1.261) |
| Nonneurologic | 0.163 (0.013) | 1.177 (1.147–1.207) |
Endocrine, hematologic, musculoskeletal, and renal systems of primary dysfunction.
For each one point Pediatric Risk of Mortality physiologic variable score increase. Neurologic components include pupillary reactivity and mental status. Nonneurologic components include heart rate, systolic blood pressure, temperature, arterial Po2, pH, Pco2, total bicarbonate, glucose, potassium, blood urea nitrogen, creatinine, WBC count, platelet count, prothrombin, and partial thromboplastin time.
TABLE 4.
Goodness of Fit for the Validation and Derivation Sets
| Derivation Set | Validation Set | |||||
|---|---|---|---|---|---|---|
| Risk Decile | n | Observed Deaths | Expected Deaths | n | Observed Deaths | Expected Deaths |
| 1 | 442 | 1 | 0.6 | 168 | 0 | 0.2 |
| 2 | 922 | 0 | 3.1 | 333 | 0 | 1.1 |
| 3 | 601 | 4 | 3.0 | 192 | 0 | 1.0 |
| 4 | 1,013 | 5 | 6.4 | 310 | 0 | 1.9 |
| 5 | 689 | 4 | 5.7 | 224 | 0 | 1.8 |
| 6 | 827 | 5 | 8.1 | 278 | 5 | 2.6 |
| 7 | 626 | 7 | 8.2 | 248 | 3 | 3.2 |
| 8 | 928 | 23 | 17.3 | 256 | 5 | 4.7 |
| 9 | 747 | 28 | 24.1 | 257 | 10 | 8.0 |
| 10 | 765 | 137 | 137.6 | 252 | 38 | 41.9 |
| Total | 7,560 | 214 | 214 | 2,518 | 61 | 66.3 |
| χ2 = 8.47 and p = 0.39 (df = 8) | χ2 = 9.32 and p = 0.50 (df = 10) | |||||
Table 5 illustrates the SMRs for common diagnostic and descriptive categories not used to develop the model. The SMRs assessing model fit within the levels of PICU type, interventional category, elective/emergency status, diagnoses of septic shock, respiratory disease, congenital cardiac conditions, and neurologic trauma were not significantly different from unity. For the levels of insurance status, the SMRs of commercial and government status were statistically different from unity.
TABLE 5.
Standardized Mortality Ratios for Diagnostic and Descriptive Categories Not Included in the Final Model for the Total Sample: Variables are Included Only For Those With at least 10 Observed and Expected Outcomes
| Variable | n | Observed Deaths (n) |
Expected Deaths (n) |
Standardized Mortality Ratio (95% CI) |
p |
|---|---|---|---|---|---|
| Payer | |||||
| Commercial | 4,168 | 72 | 101.1 | 0.71 (0.56–0.90) | 0.003 |
| Government | 5,420 | 183 | 158.5 | 1.15 (0.99–1.33) | 0.05 |
| Other/unknown | 490 | 20 | 20.7 | 0.97 (0.59–1.49) | 0.88 |
| PICU type | |||||
| Medical/surgical | 8,119 | 198 | 209.9 | 0.94 (0.82–1.08) | 0.41 |
| Cardiac | 1,931 | 77 | 69.9 | 1.10 (0.87–1.38) | 0.39 |
| Intervention category | |||||
| None | 6,281 | 217 | 222.1 | 0.98 (0.85–1.12) | 0.73 |
| Cardiac | 1,549 | 46 | 42.2 | 1.09 (0.80–1.45) | 0.55 |
| Othera | 2,248 | 12 | 16.0 | 0.75 (0.39–1.31) | 0.31 |
| Scheduled/emergency | |||||
| Scheduled | 3,667 | 49 | 53.7 | 0.91 (0.67–1.21) | 0.52 |
| Emergency | 6,411 | 226 | 226.6 | 1.00 (0.87–1.14) | 0.97 |
| Postintervention careb | |||||
| No | 6,281 | 217 | 222.1 | 0.98 (0.85–1.12) | 0.73 |
| Yes | 3,797 | 58 | 58.3 | 1.00 (0.76–1.29) | 0.97 |
| Septic shock | |||||
| Yes | 696 | 45 | 45.8 | 0.98 (0.72–1.32) | 0.91 |
| No | 9,382 | 230 | 234.6 | 0.98 (0.86–1.12) | 0.76 |
| Respiratory diseasec | |||||
| Yes | 2,781 | 87 | 77.8 | 1.12 (0.90–1.38) | 0.29 |
| No | 7,297 | 188 | 202.5 | 0.93 (0.80–1.07) | 0.30 |
| Congenital cardiovascular conditions | |||||
| Yes | 1,922 | 87 | 72.3 | 1.20 (0.96–1.49) | 0.08 |
| No | 8,156 | 188 | 208.1 | 0.90 (0.78–1.04) | 0.16 |
| Neurologic trauma | |||||
| Yes | 472 | 19 | 16.2 | 1.17 (0.71–1.83) | 0.48 |
| No | 9,606 | 256 | 264.1 | 0.97 (0.85–1.10) | 0.61 |
Other includes neurosurgical, general, otolaryngology, orthopedic, and miscellaneous surgeries.
Interventions include surgery and interventional catheterization.
Asthma, respiratory distress/failure, pneumonia, or bronchiolitis.
DISCUSSION
This revised dichotomous outcome prediction model for PRISM (PRISM IV) functioned very well with an excellent calibration and discrimination that was equivalent to the performance of the original PRISM III 12-hour model despite changes that could have impacted performance. First, using only the first PICU admission during hospitalization, combined with modeling, the outcome at hospital discharge instead of PICU discharge, was important for quality assessment because PICU discharge decision making is an important aspect of PICU quality. For example, a prematurely or inappropriately discharged PICU patient with a subsequent PICU readmission during the same hospitalization was previously credited as a good outcome for the first admission, whereas the subsequent admission had an additional mortality risk credited to their subsequent PICU admission. Therefore, the subsequent PICU admission risk was inflated although it was associated with the premature or inappropriate discharge. Previously, it had not been possible to develop a well-performing predictor using only the first PICU admission and hospital outcome, but this hurdle has been overcome to the overall benefit of model credibility. Second, changing the sampling period from the first 12 hours of care to a significantly shorter time period (2 hr before admission to 4 hr after admission for laboratory data and the first 4 hr of PICU care for other physiologic variables) better separates the PRISM score from therapies but could have resulted in an inadequate sample of physiologic data. This modification was also important because the routine of repeating preadmission laboratory data upon PICU admission, common when PRISM was initially developed, has changed in most institutions. The PRISM physiologic variables and their ranges (5) did not change, only the sampling period changed.
One other change to the PRISM model was required by a significant practice change. Admission of cardiovascular patients for “optimizing” therapy or observation before their intervention is now common in many institutions, and this necessitated a new definition of the PRISM observation period. The decision algorithms to determine the appropriate observation period were created to minimize the potential for “gaming” the observation period. These decision algorithms worked very well with excellent performance within the cardiac and the medical surgical PICUs and within the subsets of cardiac and noncardiac intervention patients. Finally, when PRISM was initially developed, the scores for physiologic derangements for each variable were calibrated to mortality odds ratios; so, the PRISM score for each physiologic variable range represented proportional risk. Over time, new therapies have evolved, and these equivalencies could have changed. We were able to test and adjust this by partitioning PRISM into its five major subcategories. The final predictor partitions the PRISM physiologic variables into the neurologic and nonneurologic components for outcome prediction.
This PRISM IV prediction algorithm based on the first 4 hours of PICU care as a predictor of survival versus death performed as well as the earlier PRISM III 12-hour prediction model, although the changes had the potential to reduce the performance. This performance is predicated on the importance of the physiologic status as the core of the conceptual framework for outcome prediction in the PICU. Recently, we demonstrated that using this core framework, we were able to extend prediction to functional status outcome and mortality. Methods dependent on categorical variables, including those using discharge diagnostic classifications, may not have similar potential to predict functional status as an outcome because they lack the conceptual framework central to pediatric intensive care, treating and maintaining physiologic stability.
The development and maintenance of PRISM and its algorithms are based on the conceptual approach that physiologic dysfunction is the core principle underlying severity of illness and can be assessed independent of computing morbidity and mortality risks. This analysis focuses on the critical details around the current data collection practices. Other categorical factors, such as age, diagnoses, or postintervention status, modify the relationship between physiologic status and risk and enable accurate and reliable estimates of mortality and morbidity risks. In order to maximize the utility of PRISM, we have not included therapies, such as mechanical ventilation for outcome prediction, because PRISM algorithms when used for quality assessment uses physiologic profiles to assess the effectiveness of therapy—conflating physiologic status with therapies would detract from the reliability of this assessment. Similarly, we have not used socioeconomic variables to enable insights into these factors after adjusting for patient status. The potential benefit of this approach is evident from the statistical significance of the SMRs for insurance status in this analysis; others have found associations of socioeconomic factors with severity of illness (11, 12).
The reference sample for this PRISM IV predictor is the PICUs in the second funding cycle of the CPCCRN. PICU quality studies using the published algorithm in this report will be able to compare themselves with the CPCCRN units (external benchmarking) and follow their own performance over time (internal benchmarking). There are advantages and disadvantages to any reference sample. A significant advantage of this sample is that the units have relatively uniform characteristics; they are all large research-oriented units in free standing or “hospital within a hospital” teaching institutions. The characteristics and the patient populations of the individual sites are clearly detailed (6). The data have been prospectively collected by dedicated staff with the rigor of National Institutes of Health–supported research and with the oversight of a data coordinating center; the data are contemporaneous, and the sample size is sufficiently large for all statistical analyses. Other reference groups may be substantially larger, including the original PRISM III sample and, more recently, the Pediatric Index of Mortality (PIM) 3 sample (5, 13). The later, in particular, is a very large sample of PICUs in the United Kingdom, Australia, Ireland, and New Zealand, but the organizational characteristics of the PIM3 PICUs have not been detailed, and presumably, there are other differences because of the regional diversity. In addition, although a very larger sample size offers statistical advantages, issues of data reliability may become important, especially if the data have been routinely collected for quality assessment or other nonresearch purposes instead of for research uses. Prediction algorithms, such as those in this article, will hopefully extend the implications of individual PICU and regional care assessments beyond the reference sample. Units using these PRISM IV algorithms may perform the same, better, or worse than this reference group, and the clear description and uniformity of the reference sample will help these sites understand their results.
Recently, we advocated for the use of a predictor that assesses survival with significant new functional morbidity, intact survival, and death for assessments of PICU care (6). As part of that effort, we developed and validated the Functional Status Scale (FSS) score, an age-independent method of measuring functional status suitable for large-scale studies; we used the FSS to assess the new morbidity rate in pediatric critical care and developed a prediction model for the simultaneous prediction of both morbidity and mortality (6, 9, 14). However, we realize that there will need to be a period of further discussion and use, as well as routine measurement of the FSS score before its acceptance. This analysis and the placement of this PRISM IV prediction algorithm in the public domain do not alter this recommendation. Hopefully, this contribution will be useful while the field of pediatric critical care considers the value of an outcome predictor of three outcomes: significant new functional morbidity, intact survival, and death.
Acknowledgments
Supported, in part, by the following cooperative agreements from the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Department of Health and Human Services: U10HD050096, U10HD049981, U10HD049983, U10HD050012, U10HD063108, U10HD063106, U10HD063114, and U01HD049934. This content is solely the responsibility of the authors and does not necessarily represent the views of the National Institutes of Health.
Dr. Pollack received grant support from the National Institute of Child Health and Human Development (NICHD), has patent applications through Children’s National Medical Center (his not-for-profit employer), and received support for article research from the National Institutes of Health (NIH). Dr. Holubkov served as a board member for Pfizer, the National Burn Association, and Fibrocell (Data Safety Board membership); consulted for St. Jude Medical, Physicians Committee for Responsible Medicine, and Covidien (biostatistical consulting); received support for article research from the NIH. Dr. Holubkov and his institution received grant support (salary support [biostatistician]) and support for travel (steering committee meeting travel) from the NIH. Dr. Funai received support for article research from the NIH. Dr. Funai and his institution received grant support from the NIH (salaries support biostatistician). Dr. Dean received support for article research from the NIH. His institution received grant support from the NIH and Health Services and Resources Administration. Dr. Berger received support for article research from the NIH. His institution received grant support from the NIH and the Association for Pediatric Pulmonary Hypertension. Dr. Wessel received support for article research from the NIH. His institution received grant support from the NIH (ongoing). Dr. Meert received consulting fees and support for article research from the NIH. Her institution received grant support from the NIH. Dr. Berg received grant support from the NICHD and received support for article research from the NIH. Dr. Carcillo received support for article research from the NIH. His institution received grant support and support for travel. Dr. Dalton lectured for rEVO Biologics, received royalties from the Society of Critical Care Medicine (SCCM) for Pediatric Multidisciplinary Board Review Book, and received support for article research from the NIH. Her institution received grant support from the NIH. Dr. Newth received support for article research from the NIH. His institution received grant support from the NIH/NICHD (Collaborative Pediatric Critical Care Research Network). Dr. Shanley received support for article research from the NIH and served as a board member for International Pediatric Research Foundation (Society for Pediatric Research Representative on Executive Committee). His institution received grant support from the NICHD (Collaborative Pediatric Critical Care Research Network [CPCCRN] grant) and the NIH (Clinical and Translational Science Award grant) and received support for travel from the NICHD (CPCCRN grant). Dr. Harrison lectured for the SCCM (board review lectures for my professional society) and received support for article research from the NIH. His institution received grant support from the NIH (CPCCRN funding and Therapeutic Hypothermia for Pediatric Cardiac Arrest trial funding via NIH). Ms Jenkins received support for article research from the NIH and disclosed government work. Dr. Tamburro received grant support from the U.S. Food and Drug Administration Office of Orphan Product Development Grant to study calfactant in pediatric stem cell transplant patients (no longer coprincipal investigator on the trial secondary to accepting a new position), received royalties from Springer Publisher (received royalties for serving as an associate editor on a pediatric critical care textbook and study guide), received support for article research from the NIH, and disclosed government work.
We thank the following members for their contribution to this study: Teresa Liu, MPH, CCRP, University of Utah (project management, Data Coordinating Center); Jean Reardon, MA, BSN, RN, Children’s National Medical Center (institutional project management, data collection); Elyse Tomanio, BSN, RN, Children’s National Medical Center (institutional project management, data collection); Morella Menicucci, MD, CCRP, Children’s National Medical Center (data collection); Fidel Ramos, BA, Children’s National Medical Center (institutional project management, data collection); Aimee Labell, MS, RN, Phoenix Children’s Hospital (institutional project management, data collection); Courtney Bliss, BS, DTR, Phoenix Children’s Hospital (data collection); Jeffrey Terry, MBA, Children’s Hospital Los Angeles (data collection); Margaret Villa, RN, Children’s Hospital Los Angeles and Mattel Children’s Hospital UCLA (institutional project management, data collection); Jeni Kwok, JD, Children’s Hospital Los Angeles and Mattel Children’s Hospital (institutional project management, data collection); Amy Yamakawa, BS, Children’s Hospital Los Angeles and Mattel Children’s Hospital UCLA (data collection); Ann Pawluszka, BSN, RN, Children’s Hospital of Michigan (institutional project management); Symone Coleman, BS, MPH, Children’s Hospital of Michigan (data collection); Melanie Lulic, BS, Children’s Hospital of Michigan (data collection); Mary Ann DiLiberto, BS, RN, CCRC, Children’s Hospital of Philadelphia (institutional project management, data collection); Carolann Twelves, BSN, RN, Children’s Hospital of Philadelphia (data collection); Monica S. Weber, RN, BSN, CCRP, University of Michigan (institutional project management, data collection); Lauren Conlin, BSN, RN, CCRP, University of Michigan (data collection); Alan C. Abraham, BA, CCRC, Children’s Hospital of Pittsburgh of University of Pittsburgh Medical Center (institutional project management, data collection); Jennifer Jones, RN, Children’s Hospital of Pittsburgh of University of Pittsburgh Medical Center (data collection); Jeri Burr, MS, RN-BC, CCRC, University of Utah (project management, Data Coordinating Center); Nichol Nunn, BS, MBA, University of Utah (project management, Data Coordinating Center); Alecia Peterson, BS, CMC, University of Utah (project management, Data Coordinating Center); and Carol Nicholson, MD (former Project Officer, Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, Department of Health and Human Services, for part of the study period).
Footnotes
Drs. Pollack and Holubkov had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Supplemental digital content is available for this article.
REFERENCES
- 1.Pollack MM, Getson PR, Ruttimann UE, et al. Efficiency of intensive care. A comparative analysis of eight pediatric intensive care units. JAMA. 1987;258:1481–1486. [PubMed] [Google Scholar]
- 2.Richardson DK, Gray JE, McCormick MC, et al. Score for neonatal acute physiology: A physiologic severity index for neonatal intensive care. Pediatrics. 1993;91:617–623. [PubMed] [Google Scholar]
- 3.Shann F, Pearson G, Slater A, et al. Paediatric index of mortality (PIM): A mortality prediction model for children in intensive care. Intensive Care Med. 1997;23:201–207. doi: 10.1007/s001340050317. [DOI] [PubMed] [Google Scholar]
- 4.Horbar JD, Onstad L, Wright E. Predicting mortality risk for infants weighing 501 to 1500 grams at birth: A National Institutes of Health Neonatal Research Network report. Crit Care Med. 1993;21:12–18. doi: 10.1097/00003246-199301000-00008. [DOI] [PubMed] [Google Scholar]
- 5.Pollack MM, Patel KM, Ruttimann UE. PRISM III: An updated Pediatric Risk of Mortality score. Crit Care Med. 1996;24:743–752. doi: 10.1097/00003246-199605000-00004. [DOI] [PubMed] [Google Scholar]
- 6.Pollack MM, Holubkov R, Funai T, et al. Simultaneous prediction of new morbidity, mortality, and survival without new morbidity from pediatric intensive care: A new paradigm for outcomes assessment. Critical Care Medicine. 2015;43:1699–1709. doi: 10.1097/CCM.0000000000001081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Pollack MM, Dean JM, Butler J, et al. The ideal time interval for critical care severity-of-illness assessment. Pediatr Crit Care Med. 2013;14:448–453. doi: 10.1097/PCC.0b013e31828a7270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Willson DF, Dean JM, Meert KL, et al. Eunice Kennedy Shriver National Institute of Child Health, and Human Development Collaborative Pediatric Critical Care Research Network: Collaborative pediatric critical care research network: Looking back and moving forward. Pediatr Crit Care Med. 2010;11:1–6. doi: 10.1097/PCC.0b013e3181c01302. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Pollack MM, Holubkov R, Funai T, et al. Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network: Pediatric intensive care outcomes: Development of new morbidities during pediatric critical care. Pediatr Crit Care Med. 2014;15:821–827. doi: 10.1097/PCC.0000000000000250. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Pollack MM, Holubkov R, Funai T, et al. Relationship between the functional status scale and the pediatric overall performance category and pediatric cerebral performance category scales. JAMA Pediatr. 2014;168:671–676. doi: 10.1001/jamapediatrics.2013.5316. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Epstein D, Reibel M, Unger JB, et al. The effect of neighborhood and individual characteristics on pediatric critical illness. J Community Health. 2014;39:753–759. doi: 10.1007/s10900-014-9823-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Naclerio AL, Gardner JW, Pollack MM. Socioeconomic factors and emergency pediatric ICU admissions. Ann N Y Acad Sci. 1999;896:379–382. doi: 10.1111/j.1749-6632.1999.tb08147.x. [DOI] [PubMed] [Google Scholar]
- 13.Straney L, Clements A, Parslow RC, et al. ANZICS Paediatric Study Group and the Paediatric Intensive Care Audit Network. Paediatric index of mortality 3: An updated model for predicting mortality in pediatric intensive care*. Pediatr Crit Care Med. 2013;14:673–681. doi: 10.1097/PCC.0b013e31829760cf. [DOI] [PubMed] [Google Scholar]
- 14.Pollack MM, Holubkov R, Glass P, et al. Eunice Kennedy Shriver National Institute of Child Health and Human Development Collaborative Pediatric Critical Care Research Network: Functional Status Scale: New pediatric outcome measure. Pediatrics. 2009;124:e18–e28. doi: 10.1542/peds.2008-1987. [DOI] [PMC free article] [PubMed] [Google Scholar]