Modification and assessment of the Bedside Pediatric Early Warning Score in the pediatric allogeneic hematopoietic cell transplant population

Abstract

Objectives

To determine the validity of the Bedside Pediatric Early Warning Score (PEWS) system in the hematopoietic cell transplant (HCT) population, and to determine if the addition of weight gain further strengthens the association with need for pediatric intensive care unit (PICU) admission.

Design

Retrospective cohort study of pediatric allogeneic HCT patients from 2009–2016. Daily PEWS and weights were collected during hospitalization. Logistic regression was used to identify associations between maximum PEWS or PEWS plus weight gain and the need for PICU intervention. The primary outcome was need for PICU intervention; secondary outcomes included mortality and intubation.

Setting

A large quaternary free-standing children’s hospital

Patients

102 pediatric allogeneic HCT recipients.

Interventions

None

Measurements and Main Results

Of the 102 HCT patients included in the study, 29 were admitted to the PICU. The median peak PEWS score was 11 (IQR 8, 13) in the PICU admission cohort; compared to 4 (IQR 3, 5) in the cohort without a PICU admission (p<0.0001). PEWS ≥8 had a sensitivity of 76% and a specificity of 90%. The area under the receiver operating characteristics (ROC) curve was 0.83. There was a high negative predictive value at this PEWS of 90%. When PEWS ≥8 and weight gain ≥7% were compared together the area under the ROC curve increased to 0.88.

Conclusions

In this study a PEWS ≥ 8 was associated with PICU admission, having a moderately high sensitivity and high specificity. This study adds to literature supporting PEWS monitoring for HCT patients. Combining weight gain with PEWS improved the discriminative ability of the model to predict the need for critical care, suggesting that incorporation of weight gain into PEWS may be beneficial for monitoring of HCT patients.

Introduction

Timely identification of a patient who requires critical care can improve outcomes and overall mortality (1–4). The converse of this has also been demonstrated in studies; late identification of patients requiring critical care leads to worse outcomes (5–6). As a result, multiple scoring systems have been developed and validated in the pediatric population to better identify patients at higher risk for needing critical care intervention (7–14). One such validated scoring system is the Bedside Pediatric Early Warning Score (PEWS). This scoring system was developed based on seven variables and has been validated in a multi-centered trial. It is a predictive scoring system for general pediatric patients who are at risk for cardiopulmonary arrest, and a score of ≥8 has been previously established as being associated with the need for critical care interventions (7,15). Some pediatric hospitals use PEWS to monitor patients on the general ward for clinical deterioration. We sought to better understand the utility of PEWS on the hematopoietic cell transplant (HCT) unit.

The HCT patient is at high-risk for needing critical care and at high risk for respiratory failure. A recent study suggested that there may be worse outcomes in mechanically ventilated HCT patients who have delays in care (16). However, as these children are chronically ill and more complex than the general hospitalized pediatric population, it can be difficult to determine who will progress to become critically ill. Furthermore, these patients often have significant emotional and social bonds with the transplant medical care team making transfer to the PICU a more stressful situation. The social support system often found on the HCT are incredibly important. Therefore, models to improve both early identification and accurate discrimination would be useful in this population. This would aid in reducing unnecessary PICU transfers while also identifying critical illness in a timely fashion. Currently there is concern that scoring systems developed for the general population may not be as applicable in the HCT patient population due to these patients having higher baseline PEWS and therefore less discriminative ability. Scoring systems developed for the general population may be improved with specific considerations for the HCT population. A known complication in the HCT population is fluid overload, which has been demonstrated in multiple studies (17, 18). Fluid overload status can be assessed by percent weight change. Increased percent weight change is associated with increased mortality and places these patients at higher odds for respiratory failure (19, 20). Given this information this study aimed to assess the validity of Bedside PEWS scoring systems to identify those patients at need of critical care intervention during their transplant admission, and to determine if a modified version of the Bedside PEWS that includes weight as a variable is more predictive of the need for PICU admission or is more highly associated with increased mortality in this select patient population.

Materials and Methods

A single center retrospective cohort study of pediatric and young adult allogeneic HCT recipients admitted between July 2009 and June 2016 was performed. Institutional Review Board approval was obtained prior to data collection. Inclusion criteria for the study were age less than 21 years and admission for an allogeneic stem cell transplant. All indications for transplant were included in the study. Exclusion criteria included patients who expired prior to critical care intervention, patients already in the PICU at time of transplant, and patients with missing vital signs prior to PICU admission.

Transplant admissions were examined. Data collected included age, sex, ethnicity, diagnosis leading to transplant, transplant source, transplant type, daily weights, daily PEWS, PEWS prior to PICU admission, PICU length of stay, critical care interventions, hospital length of stay, and disposition. The PEWS score is composed of seven variables: heart rate, systolic blood pressure, respiratory rate, capillary refill, respiratory effort, oxygen saturation, and oxygen therapy. For those who did not have a PEWS calculated, these variables were collected to calculate PEWS for all time points.

Univariate predictors of the three outcomes, PICU admission, mortality, and intubation, were assessed using basic statistics; percentages and Chi-squared or Fisher’s Exact tests for categorical data, and medians, IQRs and Wilcoxon Rank Sum tests for continuous data. The primary outcome was PICU admission and secondary outcomes were mortality and intubation. Logistic regression modeling was conducted to determine the odds of each outcome based on PEWS score, both with and without weight gain. Receiver operating characteristic (ROC) curves were conducted for each model. The area under the curve (AUC) of ROC curves was calculated to assess the predictive power of the models. Prior to modeling, an optimal cut point (≥7%) was found for weight gain, and a literature value (≥8) was used to dichotomize the PEWS score. Sensitivity, specificity, positive and negative predictive values were also calculated for each of the three predictors. The statistical software used for analysis was SAS Proprietary Software, Version 9.4 (Copyright © 2002–2012 by SAS Institute Inc., Cary, NC, USA.).

Results

During the study period, 104 patients were admitted for allogeneic HCT. Two patients were excluded from this study both due to being in the PICU at the time of transplant. The remaining 102 patients were used in the analysis. Of the 102 patients, 29 (28%) required PICU admission.

The characteristics of those needing PICU admission compared to those without a PICU admission are described in Table 1. Patients requiring PICU admission were more likely to have unrelated donors, mismatched donors, or received an umbilical cord blood transplant compared to those never admitted to the PICU.

Table 1.

Demographics for study cohort and assessed by need for PICU admission

Patient Characteristics	Entire Cohort (n=102)	PICU admission (n=29)	No PICU admission (n=73)	P value
Gender (male)	63 (61.8%)	17 (58.6%)	46 (63%)	0.68
Age at transplant	7.2 (3.0, 13.8)	4.6 (0.8, 12.4)	7.5 (3.8, 13.8)	0.10
Race
Caucasian	81 (79.4%)	22 (75.9%)	59 (80.8%)	0.77
African American	6 (5.9%)	2 (6.9%)	4 (5.5%)
Other	4 (3.9%)	0 (0.0%)	4 (5.5%)
Ethnicity
Hispanic	11 (10.8%)	5 (45.4%)	6 (54.5%)	0.29
Non-Hispanic	91 (89.2%)	24 (82.7%)	67 (91.8%)
Transplant Reason
ALL	26 (25.5%)	9 (31.0%)	17 (23.3%)	0.18
AML	25 (24.5%)	4 (13.8%)	21 (28.8%)
MDS	6 (5.9%)	0 (0.0%)	6 (8.2%)
Immunodeficiency	15 (14.7%)	7 (24.1%)	8 (11%)
Bone Marrow Failure	6 (5.9%)	1 (3.5%)	5 (6.8%)
Metabolic/Genetic	2 ( 2%)	0 (0.0%)	2 (2.7%)
NHL	2 (2%)	1 (3.5%)	1 (1.4%)
Other	17 (16.7%)	7 (24.1%)	10 (13.7%)
Hemoglobinopathy	3 (2.9%)	0 (0.0%)	3 (4.1%)
1st transplant	83 (81.4%)	22 (75.9%)	61 (87.1%)	0.35
Related Donor	36 (35.3%)	3 (10.3%)	33 (45.2%)	0.001
Matched Donor	71 (69.6%)	17 (58.9%)	54 (74%)	0.1284
Mismatched donor	31 (30.4%)	12 (41.4%)	19 (26%)
Source of Transplant
Cord Blood	41 (40.2%)	18 (62.1%)	23 (31.5%)	0.01
Peripheral blood	5 (4.9%)	1 (3.4%)	4 (5.5%)
Bone Marrow	56 (54.9%)	10 (34.5%)	46 (63%)

PICU=Pediatric intensive care unit, ALL=Acute lymphoid leukemia, AML=acute myelogenous leukemia, MDS=myelodysplastic syndrome, NHL=Non-Hodgkin lymphoma

The maximum PEWS in the 24 hours preceding PICU admission were compared to the maximum PEWS during the hospitalization for the patients without a PICU admission. The median peak PEWS was 11 (IQR 8, 13) in the PICU admission cohort; compared to 4 (IQR 3, 5) in the cohort without a PICU admission (p<0.0001). Based on previous studies in the general pediatric population, PEWS ≥8 were analyzed. Of the patients who went to the PICU, 78.6% of the patients had a PEWS ≥8. Of the patients who did not go to the PICU, only 9.6% of them had a PEWS ≥8 (p<0.0001). A PEWS score ≥ 8 was able to accurately discriminate those who required PICU admission (AUC 0.83).

Next, percent weight gain during the hospitalization was evaluated. Patients with a weight change ≥ 7% were more likely to require PICU admission. Of the 29 patients admitted to the PICU 12 (41.4%) of them had a weight change ≥ 7%, whereas of the 73 patients not admitted to the PICU only 13 (17.8%) had a weight change ≥ 7% (p-value=0.015). A 7% weight gain was chosen based on sensitivity analysis.

PEWS score and weight gain were analyzed together to determine the predictive value of this model for the need for PICU admission. The addition of ≥ 7% weight gain to the PEWS score improved the model’s discriminative ability to accurately predict those who needed PICU admission (AUC 0.88). Sensitivity, specificity, positive predictive value, and negative predictive value were then calculated for PEWS ≥8 and the combination of PEWS ≥8 and increase in weight ≥7%. The results of these are in Table 2. Of note, the combination increased both the positive predictive value and the specificity in relation to the need for PICU admission.

Table 2.

Discrete statistics for PEWS ≥8 and PEWS ≥8 + increase in weight ≥7%

Statistical Measures	PEWS ≥8	PEWS ≥8 and Increase in Weight ≥7%
Sensitivity	75.86	27.59
Specificity	90.41	98.63
Positive Predictive Value	75.86	88.89
Negative Predictive Value	90.41	77.42

Patients requiring PICU admission had higher rates of veno-occlusive disease (34.5% vs 8.2% p<0.0001), graft versus host disease (41.4% vs 19.2% p=0.02), and positive blood, urine, respiratory, or viral cultures (86.2% vs 61.6% p=0.02). Of the subjects admitted to the PICU, 13 patients died during their PICU admission for a mortality of 45%. The average PICU length of stay was 15.4 days. Of those admitted to the PICU, 17 required non-invasive ventilation (59%), 19 required intubation (66%), 8 required dialysis (28%), and 12 required vasopressors (41%).

Multivariate analysis incorporating demographics and transplant characteristics from Table 1 with a p value < 0.20 combined with the PEWS score, weight gain, and positive blood culture was then performed. We chose the model which had the best predictive value by AUROC curve that incorporated longitudinally monitored risk factors for PICU admission. The results are summarized in table 3. The best predictive model was PEWS ≥8, weight gain ≥7%, and any positive culture as evidenced by an AUC of 0.90.

Table 3.

Univariate and Multivariate Analyses

Logistic Model	OR (95% CI)	p-value	AUC
PEWS ≥ 8	29.6 (9.4, 93.9)	<0.0001	0.83
Change in Weight ≥ 7%	3.3 (1.3, 8.4)	0.0150	0.62
PEWS ≥ 8 +	36.3 (10.1, 130.5)	<0.0001	0.88
Change in Weight ≥ 7%	5.0 (1.3, 19.7)	0.0198
PEWS ≥ 8 +	37.5 (9.9, 141.8)	<0.0001	0.90
Change in Weight ≥ 7% +	5.0 (1.2, 17.9)	0.0279
Any Positive Culture	4.2 (0.9, 20.1)	0.0758

OR=Odds Ratio, CI=Confidence Interval, AUC=Area under the curve, PEWS=Pediatric Early Warning Score

One of the multivariate models tested which included PEWS, positive blood culture, and unrelated donor status, had an essentially identical AUC of the presented model above. The model presented above was chosen as these are more likely to be modifiable or treatable risk factors. However, to ensure PEWS was still useful regardless of donor status we did a sub-analysis of those who had received a HCT from an unrelated donor. The median peak PEWS in those that were admitted to the PICU was 10 (7, 13) compared to those who were not admitted to the PICU with a median of 4 (3, 5), p < 0.0001. Also, more patients who were admitted to the PICU had a peak PEWS ≥8 (71.4% vs 5.3%, p <0.0001).

Sub-analysis was then undertaken to further characterize those patients admitted to the PICU. Median peak PEWS were compared in the intubated patients versus the non-intubated patients, and in those patients who died during hospitalization and those who survived. The median peak PEWS score for those requiring intubation (n=19) was significantly higher at 11 (9.0, 14.0) compared to those who were not intubated (n=83) 4 (3.0, 6.0), p<0.0001. The median peak PEWS score of survivors (n=89) was 4 (3, 6) versus those who died (n=13) 12 (9, 13), p<0.0001.

Discussion

The HCT patient population is a unique, high-risk group with frequent need for escalation of care and higher mortality rates. Generally, patients with worsening clinical status have better outcomes with earlier identification and treatment. Our study validated the bedside PEWS scoring system for the pediatric allogeneic HCT population and found that the addition of weight gain of ≥ 7% and positive blood culture to the model improved the association with an AUC of 0.90. Our study demonstrates both the validity of the PEWS scoring system in this population, and proposes the addition of another key variable, percent weight gain, to further strengthen the association. While the positive blood culture strengthened the AUC of the model, this variable was not significant in multivariate testing with a p-value of 0.08. The addition of positive blood culture also did not really change the odds ratios of the other two variables, questioning the importance of this variable in our model. Our study also supports the idea that a higher PEWS score prior to PICU admission is associated with a higher likelihood of requiring intubation.

Early identification of clinically deteriorating patients in need of intervention is critical. Clinically there has been concern about the discriminative ability of PEWS in the HCT patient population given the thought that these patients would score higher and thus be more difficult to interpret which patients would need critical care intervention. A recent study demonstrated that the Children’s Hospital Early Warning Score developed by Boston Children’s hospital, highly correlated with the need for unplanned PICU admission in hospitalized HCT and oncology patients (21). This study also demonstrated that a higher score was associated with increased mortality. Our study provides further evidence of the use of early warning scores as our study validated a different pediatric early warning score, the Bedside PEWS. We found that a PEWS ≥ 8 was highly associated with the need for PICU admission. Interestingly, we also found that patient’s not requiring PICU admission had a much lower maximum PEWS. This directly contrasts the current notion that the HCT patients would have higher baseline PEWS and therefore have more difficulty interpreting which patients truly need critical care intervention. The Bedside PEWS scoring system may be advantageous compared to other scoring systems in its simplicity. The bedside PEWS only relies on vital signs and routinely obtained nursing assessments, potentially allowing for easier implementation. It does not incorporate behavior/neurologic assessments that may be difficult to routinely conduct at night for children hospitalized for prolonged periods of time. Additionally, it does not rely on subjective feelings of concern in either staff or family members.

It is known that the HCT population has many risk factors for fluid overload. They often require large volume medications and blood product administration. Additionally, they are at risk to have underlying renal injury or dysfunction from previous treatment toxicity (22). To try to strengthen the association between PEWS and PICU admission in this patient population, percent weight gain was added to the Bedside PEWS. The addition of ≥ 7% weight gain to the Bedside PEWS score improved the ability of the model to accurately predict those who needed PICU admission as evidenced by an increase in the AUC to 0.88.

While it is was demonstrated that multiple transplant factors can influence need for PICU admission (donor characteristics, transplant source, etc.) we chose to only use modifiable variables in our multivariate analysis. In doing this we found the best predictive model for those patients requiring PICU admission to be the combination of PEWS ≥8, weight gain ≥7%, and any positive culture (AUC 0.90). The significance of this model compared to a model including transplant characteristics is that all of our variables included can be targeted and potentially mitigated by clinical interventions, i.e. more aggressive diuresis, or earlier administration of antibiotics in a patient with concern for infection.

The importance of identifying those patients requiring mechanical ventilation in this subset is extremely important. Studies have shown that this particular subset of patients is at higher risk of mortality when intubated compared to non-HCT patients, and that a delay in mechanical ventilation in this population may be associated with worse outcomes (16, 23). Our sub-analysis demonstrated that the patients requiring intubation had a higher median PEWS score prior to PICU admission compared to those who did not require intubation. Using PEWS in this patient population may help identify which patients are at higher risk for intubation and therefore help shorten the time to critical care intervention.

There are some limitations to our study. One such limitation is sample size. It is evident that many factors influence the need for PICU admission in this patient population. VOD, GVHD, and infection were all statistically significant independent variables associated with PICU admission. Our sample size limited the statistical ability to control for all of these variables. However, since the PEWS score is being used as a surveillance tool for those in need of PICU admission, the cause of the increased score is less important; whether it be infection, VOD, or some other entity. Another possible limitation is the retrospective nature of this study. The decision to transfer a patient to the PICU was variable. There is no set criteria for which patient gets admitted to the PICU during this study period. Patients are admitted to the PICU based on the evaluation by the PICU staff called to evaluate the patient. While there could be some variability in the decision to admit certain patients, the PICU intervention data that was collected further demonstrates that the children admitted to the PICU were at high risk for mortality and/or the need for critical care interventions.

Conclusions

This study demonstrates the utility of PEWS in this high-risk patient population. A PEWS ≥ 8 is associated with the need for PICU admission. This study also demonstrates that including percent weight gain with a cutoff of 7% change, strengthens the association with the need for PICU admission. Furthermore, a higher PEWS is associated with the need for intubation. This study both adds to current literature in supporting the use of PEWS monitoring in this patient population, and suggests the addition of a percent weight change as a variable to the current monitoring system in this high-risk patient population.