Validity of Pediatric Early Warning Score in Predicting Unplanned Pediatric Intensive Care Unit Readmission

Abstract

Despite the fact that unscheduled readmission to pediatric intensive care units (PICUs) has significant adverse consequences, there is a need for a predictive tool appropriate for use in the clinical setting. The aim of this study was to assess the ability of the modified Brighton pediatric early warning score (PEWS) to identify children at high risk for early unplanned readmission. In this retrospective cohort study, all patients aged 1 month to 18 years of age discharged from PICUs of two tertiary children's hospitals during the study interval were enrolled. Apart from demographic data, the association between PEWS and early readmission, defined as readmission within 48 hours of discharge, was analyzed by multivariable logistic regression. From 416 patients, 27 patients had early PICU readmission. Patients who experienced readmission were significantly younger than the controls. (≤12 months, 70.4 vs. 39.1%, p  = 0.001) Patients who were admitted from the emergency room (66.7 and 33.3% for emergency department (ED) and floor, respectively, p  = 0.012) had higher risk of early unplanned readmission. PEWS at discharge was significantly higher in patients who experienced readmission (3.07 vs. 0.8, p  < 0.001). A cut-off PEWS of 2, with sensitivity 85.2% and specificity 78.1%, determined the risk of unplanned readmission. Each 1-point increase in the PEWS at discharge significantly increases the risk of readmission (odds ratio [OR] = 3.58, 95% confidence interval [CI]: [2.42–5.31], p  < 0.001). PEWS can be utilized as a useful predictive tool regarding predicting unscheduled readmission in PICU. Further large-scale studies are needed to determine its benefits in clinical practice.

Introduction

Children who are transferred from the intensive care unit (ICU) to the pediatric wards are more vulnerable to clinical deterioration and life-threatening events. ^{1 2} Due to the constant pressure to access ICU beds, discharge decisions are often affected by external, nonclinical factors rather than the absolute readiness and preparation of the patients for transferring to the ward. Even patients whom have their clinical status accurately assessed to determine the optimal time for discharge have a remarkably high rate of emergent, unplanned readmissions from the floor or from home. ^{3 4} Regarding highly significant adverse outcomes in readmitted patients, identifying an effective clinical scoring system to screen high-risk patients would be a great achievement. There are several studies available in adult-based literatures which describe risk assessment tools for detecting high-risk patients for early readmission to ICU _. ^{5 6 7 8 9} Despite the availability of studies in the pediatric population which describe parameters associated with PICU readmission, ^{4 10} few studies have developed an assessment tool in predicting the risk of unscheduled PICU readmission. ^{5 11 12}

Pediatric early warning score (PEWS) has been developed by expert consensus to provide a reproducible objective tool for early identification of hospitalized patients at risk for clinical deterioration. Among variable pediatric scoring systems worldwide, the Brighton PEWS is considered the most flexible and quickly performed one which consists of the following three prominent domains: (1) cardiovascular status, (2) respiratory component, and (3) the child behavioral component. ¹³ While there is widespread literature available regarding PEWS in inpatient setting, ¹³ limited previously published studies evaluated the utility of PEWS in determining unplanned PICU readmissions. Here in this study, we sought to evaluate the validity of PEWS as a risk assessment tool to identify patients who are at greater risk of unscheduled PICU early readmission.

Materials and Methods

Study Design and Population

This is a retrospective, cohort study conducted in the pediatric ICU (PICU) of Children's Medical Center and Bahrami Children hospital which are two main tertiary centers with more than 1,600 pediatric admissions per year in Tehran, Iran.

A 10-bed mixed medical and surgical PICU in Bahrami Children's hospital and three 10-bed mixed medical and surgical PICUs in Children's Medical Center were enrolled in the study. All eligible participants aged 1 month to 18 years who discharged from PICU of mentioned hospitals from September 1, 2019 through March 1, 2020 were enrolled. The majority of patients transferred out of the PICU continued to receive their medical therapies in other units with the hospital, and only a minority of patients were discharged to home. Inclusion criteria included unplanned early readmission to PICU. Exclusion criteria included transfer out to a separate facility or patient death during ICU admission.

This study was ethically approved by the Research Deputy and Ethics Committee of the Tehran University of Medical Science. As it was a descriptive study and there was no need of any medical intervention, informed consent was waived. All patients' information got anonymized before any analysis.

Data Collection

Demographic and medical features of each study participant including age, gender, body weight, primary underlying condition, gastrostomy and tracheostomy tube utilization, long-term use of noninvasive ventilation (continuous positive airway pressure [CPAP] or bilevel positive airway pressure [BiPAP]), PICU admission source, and discharge characteristics including discharge time of the day and hospital and PICU length of stay were recorded in a designed questionnaire. Moreover, for each study participant, a standardized questionnaire was designed for evaluating PEWS by a physician or nurse who was expert in the field, within 4 hours of discharge or transfer to the floor. Participating physicians and nurses received didactic training sessions and interactive training programs within the hospital setting.

Table 1 describes PEWS components used in our study. Score components that were not consistently documented were removed from our study leading to the “modified Brighton PEWS.”

Table 1. The Brighton pediatric early warning score components.

Components	Score
	0	1	2	3
Behavior	Playing/appropriate	Sleeping	Irritable	Lethargic/confused or reduced response to pain
Cardiovascular	Pink or capillary refill 1–2 seconds	Pale or capillary refill 3 seconds	Gray or capillary refill 4 seconds or tachycardia of 20 above normal rate	Gray and mottled or capillary refill 5 seconds or above or tachycardia of 30 above normal rate or bradycardia
Respiratory	Within normal parameters, no retractions	>10 above normal parameters using accessory muscles or 30+ %FiO 2 or 3+ L/min	>20 above normal parameters and retractions or 40+ %FiO 2 or 6+ L/min	Five below-normal parameters with retractions and grunting or 50% FiO 2 or 8+ L/min

Note: A maximum of 3 points can be assigned for each of three main components (respiration, circulation, and behavior/”disability”). In addition, 2 additional points can be awarded if either continuous inhalation medications or continuous positive airway pressure (CPAP) treatments are being administered, and 2 additional points for the presence of persistent postoperative vomiting. Hence, due to restricted access of clinical information researchers were forced to withdraw the last two mentioned components.

Statistical Analysis

All statistical analyses were performed by SPSS statistical software (version 24.0.0: PASW, SPSS Inc., Chicago, Illinois, United States). Kolmogorov–Smirnov test was performed to find normality. We utilized descriptive statistics including frequencies and percentages for categorical variables. Also, Fisher's exact test and the Mann–Whitney U -test were used for comparison of qualitative and quantitative variables, respectively.

Sample size was determined according to a power calculation of 80% and an α-error of 0.05 using previously published data in similar studies in the pediatric population. ^{11 12} Estimated odds ratios (ORs) with 95% confidence intervals (95% CIs) and p -values of <0.05 were used to assess the statistical significance of the correlations and associations between indicators. For total score comparison, normality of variables was analyzed using the Kolmogorov–Smirnov test. Due to abnormal distribution of data, a nonparametric Mann–Whitney test was utilized. We also performed χ 2 test for other variables.

Characteristics contributing to unplanned readmission were subjected to a deeper analysis by direct logistic regression. Analysis was performed to assess the impact of predictors such as respiratory, cardiovascular, and behavioral components and total PEWS in the determination of final risk of unplanned readmission ( Table 4 ).

Table 4. PEWS components and their effect on unplanned readmission using logistic regression.

Variables of PEWS	B	SE	Sig	OR	95% CI
Respiratory component	1.03	0.26	<0.001	2.80	1.69	4.64
Cardiovascular component	1.41	0.47	0.002	4.11	1.65	10.25
Behavioral component	2.03	0.36	<0.001	7.64	3.77	15.49
Total score	1.28	0.20	<0.001	3.58	2.42	5.31

Abbreviations: B, β-coefficient; CI, confidence interval; OR, odds ratio; PEWS, pediatric early warning score; SE, standard error; Sig, significance probability.

The frequency distribution plot of PEWS revealed that most readmitted patients had PEWS of greater than 2 at discharge which contrasted with nonreadmitted patients had PEWS of 0 and 1. A receiver operating characteristic (ROC) curve was established with associated coordinates, and the Youden index ( J ) method was utilized to define the optimal cut-off as the point maximizing the Youden function which is the difference between true-positive and false-positive rates over all possible cut-off values. ROC and area under the curve (AUC) were calculated to determine the discrimination ability of PEWS.

Results

Descriptive Statistics

A total of 416 patients were enrolled over the study period. No patient was excluded due to established criteria. Out of 416 patients, 27 patients (6.49%) experienced early unplanned readmission to PICU.

Demographics and Admission Characteristics

Patients in the early readmission group were significantly younger than nonreadmitted patients (≤12 months, 70.4 vs. 39.1%, p  = 0.001) and had lower weight on admission (<10 kg, 77.8 vs. 48.8%, p  = 0.002). There was no significant association between readmission rate in 48 hours and gender distribution.

Our results indicated that the source of admission was also associated with the readmission rate. Among readmitted patients, those who admitted from the emergency department (ED) account for the majority of cases. Meanwhile, children admitted from the operating room and transferred from an outside hospital floor or ICU had the lowest rate of readmission (66.7 and 33.3% for ED and floor, respectively, p  = 0.012; Table 2 ).

Table 2. Demographic characteristics, PICU admission source and underlying diseases of patients.

Patient characteristic	Non-readmitted ( n  = 389) n (%)	Re admitted ( n  = 27) n (%)	p -Value
Age			0.001
1–12 months	152 (39.1)	19 (70.4)
12 months–5 years	149 (38.3)	5 (18.5)
5–10 years	54 (13.9)	0
10 years or more	34 (8.7)	3 (11.1)
Gender			0.603
Male	207 (53.2)	17 (63)
Female	181 (46.5)	10 (37)
Weight (kg)			0.002
Less than 10	190 (48.8)	21 (77.8)
11–24	141 (36.3)	2 (7.4)
25 or more	58 (14.9)	4 (14.8)
PICU admission source			0.012
OR	81 (20.8)	0
ED	155 (39.8)	18(66.7)
Floor	140 (36)	9(33.3)
Another hospital floor/ICU	13 (3.3)	0
Underlying Condition
Congenital anomaly	35 (9)	4(14.8)	0.132
Endocrine disorders	50 (12.9)	0	0.340
Gastroenterology disorders	19 (4.9)	3 (11.1)	0.976
Hematology/oncology	47 (12.1)	0	0.881
Immunology/rheumatology/infectious disorders	26 (6.7)	5 (18.5)	0.061
Metabolic disorders	4 (1)	1 (3.7)	0.005
Neurologic disorders	40 (10.3)	1 (3.7)	0.730
Neurosurgery	9 (2.3)	0	0.048
Respiratory disorders	121 (31.3)	11 (40.7)	0.146
Postoperative	38 (9.8)	2 (7.4)	0.019

Abbreviations: ED, emergency department; ICU, intensive care unit; OR, operating room; PICU, pediatric ICU.

Note: p -Value was calculated using the Fisher's exact test.

Results indicated that underlying disease at admission was also associated with readmission. Among readmitted patients, those with respiratory illness account for the majority, 40.7% of the readmitted patients.

Meanwhile, patients with neurosurgical problems who were admitted in PICU during their postoperative period had the lowest rate of readmission. Chronic underlying conditions, such as metabolic diseases, significantly increase the risk of readmission (3.7 vs. 1%, p  = 0.005; Table 2 ).

Medical and Surgical Procedures during Pediatric Intensive Care Unit Admission

During ICU admission, surgical procedures including insertion of a gastrostomy tube were performed in 11.1% of patients in the readmitted group compared with 4.8% in the nonreadmitted group ( p  = 0.000). Additionally, tracheostomy usage rates were higher among readmitted patients when compared with nonreadmitted patients (3.7 vs. 0.9%, p  = 0.012). The use of noninvasive ventilation (BiPAP or CPAP) was also significantly more prevalent in the readmitted group (18.5 vs. 4.9%, p  = 0.003).

Discharge Characteristics

According to our data, no significant difference was found for the risk of unplanned readmission between regular hours versus call-time hours transfer of patients ( p  = 0.095).

There was a strong association between the requirement of oxygen more than 21% at discharge, with the rate of readmission (1.8 vs. 14.8% for dependency to FiO ₂ >50% or 8+ L/min in readmitted and nonreadmitted, respectively, p  < 0.001). Moreover, behavioral status is also significantly correlated with readmission risk 22.2 vs. 1.3% of patients with irritability at discharge in readmitted and non-readmitted groups, respectively, p  < 0.001).

Patients with readmission experienced more tachypnea/bradypnea at discharge in comparison to the nonreadmitted group. (51.8 vs. 24.2% tachypnea in readmitted and nonreadmitted respectively, p  < 0.001). Additionally, patients in the readmitted group appear to have longer capillary refill time and have pale skin color (33.3 vs. 12.6% in readmitted and nonreadmitted group, p  = 0.01; Table 3 ).

Table 3. Discharge characteristics.

	Not-readmitted ( n  = 389) n (%)	Readmitted ( n  = 27) n (%)	p -Value
Discharge time of day			0.095
Regular hours	329 (84.6)	20 (74.1)
Call-time hours	54 (13.9)	7 (25.9)
Oxygen requirement at discharge			<0.001
21%< FiO 2 <30%	363 (93.3)	13 (48.1)
FiO 2 > 30% or 3+ L/min	6 (1.5)	1 (3.7)
FiO 2 > 40% or 6+ L/min	13 (3.3)	9 (33.3)
Fio 2 > 50% or 8+ L/min	7 (1.8)	4 (14.8)
Behavioral status at discharge			<0.001
Playing/appropriate	358 (92)	14 (51.9)
Sleeping	26 (6.7)	3 (11.1)
Irritable	5 (1.3)	6 (22.2)
Lethargic/confused/reduced response to pain	0	4 (14.8)
Respiratory status at discharge			<0.001
Normal RR/no retraction	273 (70.2)	8 (29.6)
RR >10 above age appropriate RR or using accessory muscles	73 (18.8)	4 (14.8)
> 20 above age appropriate RR and retractions	21 (5.4)	10 (37)
Five below-age appropriate RR with retractions	22 (5.7)	5 (18.5)
Cardiovascular status at discharge			0.011
Pink or capillary refill 1–2 seconds	333 (85.65)	18 (66.7)
Pale or capillary refill 3 seconds	49 (12.6)	9 (33.3)
Gray or capillary refill 4 second or tachycardia of 20 normal age appropriate	7 (1.8)	0
Gray and mottled or capillary refill 5 seconds or above or tachycardia of 30 above normal rate or bradycardia	0	0

Abbreviation: RR, risk ratio.

Note: p -Value was calculated using the Fisher Exact test.

Outcomes

Compared with nonreadmitted patients, those who were readmitted to PICU showed significantly worse clinical outcomes. Readmitted patients showed higher mortality rate (51.9 vs. 0.3%, p <0.001). Moreover, readmitted patients revealed to have longer hospital length of stay (22.31 vs. 8.67, p <0.001) and longer mean total PICU occupancy represented as the combined hospital length of stay for both first and second admissions (14 vs. 4.88, p <0.001).

Pediatric Early Warning Score and Its Characteristics

Each component of PEWS for both readmitted and nonreadmitted patients was analyzed separately to evaluate the risk of unplanned readmission. Tachypnea/bradypnea at discharge is a significant factor in increasing the risk of unplanned readmission (OR = 2.8, 95% CI: [1.69–4.64], p <0.001). Abnormal heart rate (age appropriate tachycardia) at discharge is also attributable to increased risk of unplanned readmission. (OR = 4.11, 95% CI: [1.65–10.25], p  = 0.002). Inappropriate behavioral component at discharge is another correlated factor with unplanned readmission. (OR = 7.64, 95% CI: [3.77–15.49], p <0.001; Table 4 ).

As a side note, readmitted patients have a higher median PEWS score when compared with the nonreadmitted group prior to PICU discharge (3.07 vs. 0.8, p  < 0.001). The most common score in nonreadmitted and readmitted groups was 0 and 2, respectively. All of the nonreadmitted patients had PEWS of 3 and less, while 59.2% of readmitted patients had scores of equal or greater than 3 at discharge ( Table 5 ). Each 1-point increase in the PEWS at discharge significantly increases the risk of readmission (OR = 3.58, 95% CI: [2.42–5.31], p <0.001). Frequency distribution plot of PEWS in readmitted and nonreadmitted patients was also designed.

Table 5. Distribution of PEWS in readmitted and nonreadmitted patients.

PEWS within 4 hours of transfer to floor	Non-readmitted ( n  = 389) n (%)	Readmitted ( n  = 27) n (%)	p -Value
0	207 (53.2)	2 (7.4)	<0.001
1	92 (23.7)	2 (7.4)
2	51 (13.1)	7 (25.9)
3	1 (0.3)	6 (22.2)
4	0	4 (14.8)
5	0	3 (11.1)
6	0	3 (11.1)
Mean	0.8	3.07

Abbreviation: PEWS, pediatric early warning score.

Sensitivity and specificity for each possible cut-off point related to PEWS were calculated and the highest discrimination ability of PEWS value was identified as well. Calculating the cut-off values for PEWS at PICU discharge, the best combination of sensitivity and specificity was determined at a cut-off PEWS of 2 (sensitivity, 85.2% and specificity, 78.1%). PEWS of 2 or more had maximum discrimination ability to distinguish high-risk patients for readmission that was achieved by ROC curves and AUC. AUC was calculated 0.865 ( Fig. 1 ).

Fig. 1.

Receiver operating characteristic curve of 0.86 related to the study model which is indicative of the reliability of PEWS as a predictive screening tool. PEWS, pediatric early warning score.

Discussion

Not only early unplanned PICU readmission is strongly associated with a higher rate of morbidity and mortality but also the economic burden imposed to hospital and health care systems cannot be ignored. ¹⁴ Unplanned readmissions to PICU has nowadays become the focus of health care quality, as it might reflect the efficacy of health care system or patients' safety. Moreover, the rate of readmissions to ICU within 48 hours of discharge or transfer from ICU is recognized as an indicator of the quality of critical care. ¹⁵

Currently, there is no standardized screening tool to determine the risk of PICU unplanned readmission in children. The main goal of our study is to establish an objective screening tool to predict unscheduled readmission for pediatric intensivists in clinical practice. The ability to identify patients at high risk for unplanned readmission, which mostly includes patients with multiple comorbidities, could potentially promote the argument for establishing “intermediate care units” to avoid predictable unplanned readmissions.

The PICU readmission rate is quite variable in different studies in literature. The overall rate of unplanned readmission in pediatric ICUs is 3 to 20% which, a wide range of prevalence, may be due to the inclusion of all kinds of readmission regardless of time and nature. ^{11 16 17} According to our study, 6.49% of our patients experienced early unplanned readmission to PICU within 48 hours. Earlier studies of PICU readmission reported higher incidence of readmission in younger children and children with lower weight at admission, ^{10 12 16 18} similarly, our findings are consistent with the previous studies.

In current study, we found a higher risk of readmission in patients admitted from the ED in comparison to the operation room. Our results were compatible with previous studies by Edwards et al ⁴ and Khan et al, ¹⁹ while some other studies did not approve the results. ^{6 20} Primary diagnosis of respiratory diseases accounts for the majority of readmitted patients. This finding was in line with studies by Chrusch et al ²¹ and Khan et al ¹⁹ in both pediatric and adult populations.

Our data corroborate the findings from previous studies by Hua et al ²² , and Edwards et al ⁴ , stating that nonoperated patients were more likely to require readmission. In our data, the patients who had a neurosurgery procedure before admission to the ICU were less likely to be readmitted to the PICU. Although our readmitted patients did have a higher rate of surgical procedures being done during the PICU stay, patients initially brought from the operating room did not have a higher readmission rate. This finding may indicate that these surgical patients only require acute care after surgery.

On the side note, presence of a chronic genetic/metabolic condition may lead to the increased risk of unplanned readmission according to our data which was in concordance with most of the previous studies. ^{16 17 19}

Our analysis confirmed that the existence of any chronic respiratory condition which mandates the usage of noninvasive ventilation equipment, such as BiPAP/CPAP, would also be independently associated with higher risk of readmission. This might be related to lack of knowledge of health care providers in lower levels of care regarding instrument instruction and medically complex care. This finding is compatible with previous reports by Bernard and Czaja ³ and Mandell et al. ¹¹ The challenge of accommodating all patients who need critical care would be a constant pressure that would lead to premature discharge of ineligible patients from PICU. ¹⁰ Usually, discharges in call-time hours and weekend days were associated with a higher risk of readmission according to previous data. ^{10 11 18 23} Our data did not approve the above findings which might probably be due to a relatively small sized population.

Vital signs and oxygen requirements more than 21% at discharge are of utmost importance as potential factors in predicting unplanned readmission. We could establish a significant correlation between the oxygen requirement of more than 21% at discharge and increased risk of unplanned readmission which is in line with several previous studies. ^{5 10 11}

On the other hand, according to our findings, patients experiencing abnormalities in respiratory and heart rates at discharge will be susceptible to unplanned PICU readmission. Signs of behavioral changes had been associated with higher risk of unplanned readmission as well. Some pediatric studies in the literature have investigated the role of vital signs at discharge by using screening tools like pediatric logistic organ dysfunction (PELOD). ¹⁸ Acute physiology and chronic health evaluation (APACHE2) ²⁴ and simplified adult physiology score (APS) ^{3 23} had similarly been utilized as adjustment tools in several adult investigations. Among the many pediatric scoring systems that have been developed worldwide, the PEWS score is one of the simplest and most flexible systems that can be quickly performed and that is not age specific. Either using the screening tools or not, vital signs at discharge are meticulously important in distinguishing the potential risk of readmission.

Longer hospital/PICU stay and higher mortality rates are among those well-known complications of unplanned readmission. Longer PICU length of stay, a consequence of unplanned readmission, should be addressed as a critical issue in the context of resource utilization since by nature the PICU is a resource intensive unit. Most of the previous studies in the pediatric and adult population are in line with the mentioned fact. ^{4 10 12 18 24} Our study supports the findings which all contribute to worse clinical outcomes.

Several guidelines have been proposed in the literature describing discharge criteria of critically ill patients to the ward, ^{25 26} although most of the recommended criteria are nonspecific and are subjective. Few risk-predictive tools in pediatric population have been described in identifying risk factors for unplanned readmission, ^{5 11} while their validation and discrimination ability are questionable.

To the best of our knowledge, our study is one of the most detailed investigations describing an association between PEWS subscores and PICU readmission. PEWS is an objective index which is capable of distinguishing patients who are experiencing clinical deterioration in the hours preceding a sentinel event. Moreover, this score can identify patients who are at risk of code blue events at least 1 hour before deterioration of their clinical status. ²⁷ According to our findings, the mean PEWS of readmitted patients were significantly higher in comparison to nonreadmitted groups. At a cut-off PEWS of 2, maximum discrimination ability was identified with sensitivity 85.5% and specificity 78.1%. In the study by Mandell et al, the cut-off scores were not sensitive or specific enough to be clinically useful. Similar to a study in 2018 by Kroeger et al, ²⁸ we found an area under the ROC curve of 0.86, revealing that the PEWS is a reliable tool at predicting PICU readmission.

Limitations

There are a few limitations in this study. First, this study had a small number of readmitted patients and may benefit from further establishing the validity of PEWS in larger pediatric populations. Second, we did not have access to electronic databases; therefore, we collected our data from the clinical records which might cause some unpreventable information loss. Finally, PEWS screening tool includes a broad scoring system which might result in not particularly sensitive results while some components of PEWS consist of subjective measures.

Conclusion

Due to limited sources of critical care beds in most of the world, identifying a user friendly, reliable, and practical tool in detecting patients with higher risk for early unscheduled readmission seems to be beneficial. Considering a pretransfer or predischarge PEWS calculation, promises a highly valid and reliable score in predicting unplanned PICU readmission. Higher PEWS score at discharge is a proxy tool for recognizing sicker patients who might need prompt medical attention and may not be eligible for transfer. Patients with high PEWS at discharge may benefit from continuous monitoring of PEWS trends, while on the wards, as a mean of potential early detection of clinical deterioration. Larger multicenter prospective studies are warranted to evaluate the reliability and efficacy of the designated scoring system.