Validation of Pediatric Index of Mortality-2 Scoring System in a Single Pediatric Intensive Care Unit in Iran

Abstract

Objective

A study to validate and calibrate Pediatric Index of Mortality-2 (PIM2) in children admitted to our pediatric intensive care unit (PICU).

Methods

This is a prospective cohort study performed in Bahrami Children's Hospital affiliated to Tehran University of Medical Sciences. We studied the patients admitted to PICU from May 2007 to November 2008. Clinical measures were identified upon arrival in PICU. We used PIM2 score and logistic regression analysis to compare expected mortality risk with observed mortality rate. Receiver operating characteristics (ROC) curve analysis was done and standardized mortality ratio was calculated. PIM2 Index assessment was performed by use of Hosmer and Lemeshow goodness-of-fit test.

Findings

240 patients were included in this study. The model fit was achieved adequately (P value=0.741). The area under the ROC curve was 0.795 (0.715-0.875 for 95% confidence interval) and standardized mortality ratio was 1.8 (1.28-2.465 for 95% confidence interval) High-risk group diagnosis with adjusted odds ratio (AOR)=14.75, pupil reaction to light (AOR=0.13) and duration of stay in PICU (AOR=1.03) had significant statistical association to pediatric mortality.

Conclusion

PIM2 is a good index for prediction of mortality in our pediatric intensive care unit. This study revealed that there is significant statistical association between the children mortality and the length of hspita;ization, pupillary light reflex and the risk level category on admission.

Introduction

The purpose of establishing a pediatric intensive care unit (PICU) is to upgrade the quality of services and obtaining the best results and better outcomes for the severely ill children. One of the ways to achieve that goal is to predict the mortality risk of the patients admitted to the PICU to provide them with the best care available. There are some scoring systems designed to predict the patients’ mortality risk. Using these systems, one is able to assess the severity of the disease, planning for triage, treatment options, clinical progress and the outcome of patients. These are also considered to measure the quality control criteria and cost analysis [1–4]. From 1980 onwards, the systems have been used including the GCS (Glasgow coma scale), MPM (mortality prediction model), PRISM (pediatric risk of mortality) and PIM (pediatric index of mortality)[5–11].

In the recent two decades, the American rating system “PRISM” and the European system “PIM” are known as the two successful indices in many countries. But they are not readily available for many health-care systems due to their high cost or large amount of information needed to be supplied. Normally, the guidelines of providing cares in the PICU are designed upon the possibilities and limitations in each country so that there are many reconstructed criteria extracted through similar studies[12, 13]. Thus, we decided to make validation of PIM2 score according to the current available facilities to use instead in a way that the new scoring index could be accessible and as such functional and valuable.

Subjects and Methods

This study was conducted on patients admitted to PICU of Bahrami Children's Hospital affiliated to Tehran University of Medical Sciences from May 2007 to November 2008. It is a Cohort study. The sampling was simplified and randomized. The inclusion criteria were: any child admitted to PICU aged 1 month to 16 years. The exclusion criteria were: admitted patients died within first 24 hours after admission to PICU and patients transferred, for any reason, to other hospitals. Patients’ data was recorded in a questionnaire including demographic information, main diagnosis in PICU (defined as the primary cause of admission in PICU), outcome (death or discharge or transfer to another department), and variables of PIM2. Based on the PIM2 point system, the disease was considered as high risk or low risk disease. The high-risk diseases were cardiac arrest before admission to PICU, after first induction of chemotherapy in leukemia or lymphoma, spontaneous cerebral hemorrhage, cardiomyopathy or myocarditis, hypoplastic left heart syndrome, HIV infection and liver failure. Neurodegenerative diseases, asthma, bronchiolitis, croup, obstructive sleep apnea and diabetic ketoacidosis were included in the low-risk group.

The data collected based on the patient's medical history, physical examination and laboratory findings which were recorded by the physician during first visit (immediately after admission to PICU). The completed questionnaire was then evaluated and possible defects were eliminated. In order to calculate the predicted risk of death for each patient, logit was calculated by multiplying fixed coefficients of the logistic regression (Table 1) by variables (X ₁ ,X ₂ ,….X ₁₀) and adding them to the pre-determined constant values which gives:

Table 1.

Coefficient of variables and constant value for Pediatric Index of Mortality-2 formula

Variable	Constant Ratio
Absolute (Systolic blood pressure-120) (mmHg)	0.01395
Pupillary light reflex (yes/no)	3.0791
100 × FiO 2 /PaO 2	0.2888
Additional alkali (mmHg)	0.104
Need for mechanical ventilation in the first hour (yes/no)	1.3352
Type of admission (elective/non-elective)	-0.9282
Hospitalization following surgery (yes/no)	-1.0244
Hospitalization following cardiac bypass (yes/no)	0.7507
With high risk disease (yes/no)	1.6829
With low risk disease (yes/no)	-1.577
Constant value	-4.8841

Logit=constant + AX₁+ BX₂+ CX₃+ …

Thus for the quantitative variables, absolute values and for the qualitative variables (dichotomy), either number zero or 1 was used, and the formula calculated. For example, if the infant had a systolic blood pressure of 70 mmHg, 50 was multiplied by the related constant ratio of 0.01395, or if there was no pupillary light reflex, 1 was multiplied by the related constant ratio of 3.0791. Additional amounts of alkali, PaO₂ and FiO₂ were obtained according to laboratory findings.

In order to calculate the possibility of death, the logit value was obtained from the formula:

P=e ^logit /(1 + e ^logit ), where e=2.7183

The collected information was complied with related database and PIM2 was analyzed using chi-square, logistic regression and Hosmer-Lemeshow test.

To calculate SMR (standardized mortality ratio), the probability of death for each child was determined based on PIM2 model and total amount of probability of death in the studied population was obtained. Then SMR was calculated by dividing the numbers of deaths occurred to the expected. Since logistic regression is the main method for statistical analysis, 15 samples were chosen for each variable. Overall, because 15 variables were studied, at least 225 (15×15) of cases were considered.

Statistical analysis was performed using SPSS (Ver. 16). The use of persons’ data has been handled in accordance with the rules of the ethical review board of Tehran University of Medical Sciences. We obtained prior informed written consent from the patients’ parents and patient anonymity is preserved.

Findings

240 patients were studied. Among them 150 were (62.5%) boys and 90 (37.5%) girls. Table 2) shows detailed characteristics of the patients.

Table 2.

Characteristics of patients under study when admitted to pediatric intensive care unit

Characteristic	SD ± Average	Frequency
Age (month)	31.427(42.7978)	-
Systolic blood pressure (mmHg)	95.33(20.544)	-
Heart rate (per minute)	138.03(26.434)	-
Axillary temperature (°C)	37.472(0.7634)	-
Respiratory rate (per minute)	39.93(12.896)	-
[FiO 2 /PaO 2 ]×100	73.0551(62.97758)	-
Additional alkali	-5.791(9.6331)	-
Length of stay in PICU	7.12(11.836)	-
Normal pupillary reflex to light	-	230(95.8%)
Need for mechanical ventilation in the first hour	-	24(10%)
Elective admission	-	2(0.8%)
Hospitalization following surgery	-	25(10.4%)
Hospitalization following cardiac bypass	-	1(0.4%)
With high risk disease	-	39(16.3%)
With low risk disease	-	36(15%)

SD: Standard Deviation

230 (95.8%) patients had normal pupillary light reflex and 24 (10%) required mechanical ventilation in the first hour. Only 2 (0.8%) patients were hospitalized electively and 25 (10.4%) were admitted to PICU following surgery. Only one (0.4%) patient was hospitalized in PICU following cardiac bypass. In this study 39 (16.3%) high risk patients and 36 (15%) low risk patients were admitted to PICU, and others did not have any high or low risk diseases (based on PIM2 definition).

During the study, 36 (15%) patients died after admission to PICU. First, the effect of variables was analyzed individually by the univariate logistic regression method (Table 3).

Table 3.

Results of univariate analysis of studied variables

Variable	Odds Ratio	P Value	Interval for Lower	Odds RatioHigher
Sex (female/male)	0.769	0.5	0.36	1.62
Age (month)	0.999	0.8	0.99	1.01
Primary systolic blood pressure	0.999	0.9	0.98	1.02
FiO 2	1.044	<0.001	1.02	1.06
PaO 2	1.007	<0.001	1.00	1.01
FiO 2 /PaO 2 ratio	1.005	0.03	1.00	1.01
Additional amount of alkali	1.003	0.8	0.97	1.04
Length of stay in PICU (day)	1.031	0.02	1.00	1.06
Pupillary reflex to light (yes/no)	0.104	0.001	0.03	0.39
Need for mechanical ventilation in the first hour	12.345	<0.001	4.90	31.09
Admission (elective/non-elective)	0.845	0.7	0.80	0.89
Hospitalization following surgery (yes/no)	0.726	0.6	0.21	2.56
Hospitalization following cardiac bypass (yes/no)	0.151	0.1	0.11	0.20
With high risk disease (yes/no)	16.047	<0.001	7.05	36.54
With low risk disease (yes/no)	0.824	0.004	0.77	0.88

Then Hosmer-Lemeshow test was done for the desired variables in PIM2 and the results showed that this is an appropriate variable-based model (P value=0.2 and Chi-Square=0.741). There was no statistical significant difference between occurred death and expected death based on the model built with PIM2 variables.

In the next step to assess the probability of death based on PIM2 index, ROC curve analysis was performed (Fig 1). Area under the curve (AUC) of 0.795 was achieved with a confidence interval of 95% (0.715-0.875).

Regarding 36 deaths in 240 cases studied, the average probability of death and its confidence interval of 95% was 0.15 (0.1073-0.2016). We expected 20 cases of death based on PIM2 model which would make an average of 0.083 (0.052-0.126) for probability of death and its 0.95% confidence interval.

Finally, by dividing the assigned number of deaths to the expected number of deaths based on PIM2 model, Standardized Mortality Ratio (SMR) and its confidence interval of 0.95% was obtained to be 1.8 (1.28-2.46). Therefore, death occurs 1.8 times more in comparison to what was expected with PIM2 model. Using the variables which had P-value <2 we tried to plan the conclusive model through Forward Stepwise method.

Using the multivariate analysis through Forward Stepwise method revealed that high-risk group diagnosis, the length of PICU stay and pupillary light reflex have significant association with pediatric mortality.

In order to determine the goodness of fit, the Hosmer and Lemeshow showed that the above model (final model) is the appropriate model. In other words according to this model there is no difference between the number of death outcome and the expected cases of death (P-value=0.482; Chi-square=5.494).

Discussion

Various studies have suggested that Pediatric Index of Mortality having eight variables is an appropriate measure to estimate the probability of death of patients in PICU [3, 14]. A newer edition of this index called PIM2 has been proposed which utilizes 10 variables [5].

We had 36 (15%) deaths among 240 studied patients, and expected using PIM2 a mortality rate of 20 (8.3%). We studied the 10-variable model of PIM2 using Hosmer-Lemeshow test. There was no significant difference between occurred and expected death in the built model (P-value=0.7, χ²=5.161). Calculating probability of death, we obtained the AUC=0.795 with the confidence interval of 95% (0.71-0.87) by ROC curve analysis. There was also a Standardized Mortality Ratio 1.8 (1.28-2.46) with the confidence interval of 95%. Therefore considering the proximity of the area under the ROC curve to 0.8 and obtained SMR, this model can be an appropriate one.

Shann and colleagues in Australia obtained the following amounts for the area under the curve of PIM model in eight studied hospitals: 0.80, 0.85, 0.86, 0.89, 0.91, 0.92, and 0.92[8]. Also in Australia, Slater and colleagues calculated an area under the ROC curve of 0.90 (0.89-0.92)[15].

In comparison to our study that focused on length of PICU stay, pupillary light reflex and the risk level category (high risk or low risk) on admission, the Bains HS and Kumar Soni study demonstrates temperature, oxygen saturation and respiratory rate to be significantly associated with mortality[12]. Hooman N, et al focused on the plasma level of an indicator (uric acid) as a mortality predictor of PICU patients[13].

In 2002 through a cohort study, Gemke and colleagues revealed that 20 patients out of 303 (6.6%) died. The expected mortality rate after 24 hours using the PRISM Ш index was 6.95% and SMR was 0.95 (0.67-1.22). Expected mortality rate using PIM2 index was 7.5% and calculated SMR was 0.88 (0.55-1.20). The level under ROC curve was 0.78 (0.67-0.89) for PRISM Ш index while it was 0.74 (0.63-0.85) for PIM2[16].

Among all reported experiments about the usage of mortality prediction systems in Iran, Kadivar and colleagues’ study is considerable. They studied 205 patients in the PICU of children's Medical Center in Tehran using the primary PRISM (14 variables) during a six month period. The mortality rate was 21.5% among hospitalized patients and PRISM could predict most of the deaths[17]. Thus, the relative frequency of death in our study was 6.5% less than that.

So far, the relative frequency of mortality in our study is different from that of the others; therefore, we found it useful for practical comparing of expected death and SMR index. The expected mortality in our study was 8.3%, which was 6.7% lower than the observed death and SMR was 1.8 (1.28-2.465) which was 80% higher. These findings are suitable markers for the authorities of Bahrami Children's Hospital to make a further review on this issue.

In our study, we realized that the variables “being in the high risk group”, “pupillary light reflex” and “length of admission” have significant association with the probability of death.

Conclusion

It can be concluded that the evaluated PIM2 index is an appropriate method of proper estimation for the probability of death for hospitalized patients in PICU and is applicable in our children's hospitals. Our hospital did not accept head trauma patients and did not provide neurosurgery service. Thus patients who need any kind of neurosurgical care were not admitted to the hospital and not included in the study. This can be considered as a limitation of the study.

Conflict of Interest

None