Prognostic scoring in critically ill children: What to predict?

Severity-of-illness scoring systems have been widely used in pediatric intensive care units (PICUs) to quantify patient outcomes. These scoring systems can be used for internal and external benchmarking to assess quality of care, and for identifying and stratifying patients enrolled in clinical trials. Most systems use mortality as the outcome measure to be predicted. Alternatively, a surrogate outcome with a higher incidence than mortality may be selected as an outcome parameter for scoring systems. Organ dysfunction could well be such a surrogate and has been used by Leteurtre and colleagues.¹

The scoring systems most frequently used in PICUs are the Pediatric Risk of Mortality (PRISM) and the Paediatric Index of Mortality (PIM) scores. The latest version of the Pediatric Risk of Mortality (PRISM-III) was derived from data from more than 11 000 patients in 32 PICUs and was recalibrated on more than 20 000 patients.² It uses physiologic, laboratory and diagnostic data to predict mortality after 12 or 24 hours in intensive care. The latest version of the Paediatric Index of Mortality (PIM2) was derived from data from about 20 000 patients in 14 PICUs and uses physiologic and laboratory data upon admission to the intensive care unit.³ Essentially different from the Pediatric Risk of Mortality score, the Paediatric Index of Mortality originated from the concept of eliminating lead-time bias (i.e., the influence that management before admission to the intensive care unit might have on the physiologic variables during the first 24 hours after admission to the unit). Furthermore, it is less likely to be influenced by the quality of management in the PICU during the first 24 hours.

The problem is that mortality may not be the best outcome parameter, for two important reasons. First, because of the advanced improvement in pediatric critical care, rates of death are usually as low as 5%–10%.⁴ Second, mortality in the PICU is influenced substantially by the way of dying: withdrawal or withholding of life-sustaining treatment is not uncommon in pediatric critical care.⁵

Organ dysfunction is the parameter used in the Pediatric Logistic Organ Dysfunction (PELOD) scoring system.⁶ Unlike the Pediatric Risk of Mortality and the Paediatric Index of Mortality scores, the PELOD score is derived from a combination of data from a relatively small sample (about 600 patients) admitted to three PICUs and variables suggested by expert opinion. The authors did an external validation of the PELOD score among 1806 critically ill children admitted to seven European PICUs and concluded that it was a good predictor of mortality.⁷ Nevertheless, several years later, the authors recognized that their external validation was erroneous because their model had poor calibration.⁸ This lack of good calibration was recently confirmed by data from nearly 1500 children admitted to two PICUs in Brazil and Argentina.⁹ In both this study and the original report of external validation, mortality was underpredicted by the score among patients at lower risk of death and overpredicted among those at higher risk.

Despite this major issue, Leteurtre and colleagues have reanalyzed the data that they used to validate the PELOD score in order to describe temporal changes in the score in relation to mortality.¹ They have attempted to quantify the subjective feeling of physicians that a patient is likely to die if his or her organ function deteriorates or fails to improve. Bearing in mind the poor calibration of the PELOD score and several questions regarding the methodology of the study, the PELOD score cannot be regarded as a suitable surrogate parameter for death. First, and perhaps most important, confounding by indication was introduced because variables were measured only if requested by the attending physician. Second, the most abnormal value was used to calculate the score. But does this truly reflect deterioration of organ function? Third, the rate of death was low (6.4%), and the severity of illness of the majority of patients appeared to be only moderately severe (median Pediatric Risk of Mortality score of 6). This suggests that their study may be underpowered to detect clinically relevant and statistically significantly differences, especially among patients with a high PELOD score during the first day in intensive care. Fourth, nearly half of their patients were admitted postoperatively, a different population than that of patients admitted nonelectively to the PICU.

If organ dysfunction is not a suitable surrogate parameter for death, should we seek alternative parameters or should we look for alternative prognostic outcome criteria in pediatric critical care? Although the idea of pediatric critical care is to prevent death, the functional outcome of the patient should be at least preserved, if not improved. Treatments need to be directed toward the most optimal outcome with respect to the patient’s physical, psychological and social well-being. Unfortunately, tools for measuring these outcomes are crude, and it is unknown when to assess these outcomes after discharge from the PICU.¹⁰ Nevertheless, first attempts toward developing scoring systems for predicting functional outcome in critically ill children have recently been reported. Pollack and coworkers have developed and externally validated the Functional Status Scale, a score based on the patient’s mental status, sensory functioning, ability to communicate, motor functioning, ability to feed and respiratory status.¹¹ External validation in other patient populations is awaited.

Predicting outcome in pediatric critical care is common. However, what to predict is not clear. Should it be mortality or organ dysfunction? Or perhaps it should be functional status? I feel this outcome is equally important and deserves to be the way forward in pediatric critical care.

Key points

• Scoring systems to predict mortality are widely used in pediatric critical care.

• Mortality may not be the best outcome parameter to predict.

• Surrogate parameters should be pursued, including the functional status of the child.