Effect of a Pediatric Early Warning System on All-Cause Mortality in Hospitalized Pediatric Patients

Key Points

Question

Does the implementation of the Bedside Paediatric Early Warning System (BedsidePEWS) reduce hospital mortality compared with no severity of illness score?

Findings

In this cluster randomized trial that included 21 hospitals, 144 539 patient discharges, and 559 443 patient-days, implementation of the BedsidePEWS compared with usual care did not significantly decrease all-cause mortality among hospitalized pediatric patients (1.93 per 1000 discharges vs 1.56 per 1000 discharges, respectively).

Meaning

This study does not support the use of the BedsidePEWS to reduce hospital mortality.

Abstract

Importance

There is limited evidence that the use of severity of illness scores in pediatric patients can facilitate timely admission to the intensive care unit or improve patient outcomes.

Objective

To determine the effect of the Bedside Paediatric Early Warning System (BedsidePEWS) on all-cause hospital mortality and late admission to the intensive care unit (ICU), cardiac arrest, and ICU resource use.

Design, Setting, and Participants

A multicenter cluster randomized trial of 21 hospitals located in 7 countries (Belgium, Canada, England, Ireland, Italy, New Zealand, and the Netherlands) that provided inpatient pediatric care for infants (gestational age ≥37 weeks) to teenagers (aged ≤18 years). Participating hospitals had continuous physician staffing and subspecialized pediatric services. Patient enrollment began on February 28, 2011, and ended on June 21, 2015. Follow-up ended on July 19, 2015.

Interventions

The BedsidePEWS intervention (10 hospitals) was compared with usual care (no severity of illness score; 11 hospitals).

Main Outcomes and Measures

The primary outcome was all-cause hospital mortality. The secondary outcome was a significant clinical deterioration event, which was defined as a composite outcome reflecting late ICU admission. Regression analyses accounted for hospital-level clustering and baseline rates.

Results

Among 144 539 patient discharges at 21 randomized hospitals, there were 559 443 patient-days and 144 539 patients (100%) completed the trial. All-cause hospital mortality was 1.93 per 1000 patient discharges at hospitals with BedsidePEWS and 1.56 per 1000 patient discharges at hospitals with usual care (adjusted between-group rate difference, 0.01 [95% CI, −0.80 to 0.81 per 1000 patient discharges]; adjusted odds ratio, 1.01 [95% CI, 0.61 to 1.69]; P = .96). Significant clinical deterioration events occurred during 0.50 per 1000 patient-days at hospitals with BedsidePEWS vs 0.84 per 1000 patient-days at hospitals with usual care (adjusted between-group rate difference, −0.34 [95% CI, −0.73 to 0.05 per 1000 patient-days]; adjusted rate ratio, 0.77 [95% CI, 0.61 to 0.97]; P = .03).

Conclusions and Relevance

Implementation of the Bedside Paediatric Early Warning System compared with usual care did not significantly decrease all-cause mortality among hospitalized pediatric patients. These findings do not support the use of this system to reduce mortality.

Trial Registration

clinicaltrials.gov Identifier: NCT01260831

This cluster randomized trial evaluates the effect of the Bedside Paediatric Early Warning System, a system of care that guides escalation and de-escalation of care for hospitalized children, on all-cause hospital mortality.

Introduction

The prevention of near and actual cardiopulmonary arrest is a fundamental element of patient safety. Prevention is contingent on the timely identification, referral, and treatment of hospitalized children who are clinically deteriorating. Implementation of severity of illness scores or criteria with or without a rapid response team may facilitate timely admission to the intensive care unit (ICU) and the delivery of critical care to improve patient outcomes. To date, interventional studies in children have used before and after designs that may be confounded by temporal trends and limited by a predominance of single-center evaluations.

The Bedside Paediatric Early Warning System (BedsidePEWS) is a documentation-based system of care composed of a validated severity of illness score, an interprofessionally designed documentation record, and multidomain recommendations for care escalation and de-escalation among hospitalized patients from term to 18 years of age.

A cluster randomized clinical trial was designed and conducted to test the hypothesis that implementation of BedsidePEWS would reduce rates of all-cause hospital mortality and significant clinical deterioration among hospitalized children compared with usual care at hospitals without an early warning score.

Methods

Study Design

A detailed description of the study rationale, design, interventions, and outcomes was described previously and appears in Supplement 1 and Supplement 2. Research ethics board approval was obtained at each participating hospital. The need for written informed consent for patient or clinician participation was waived in all jurisdictions. This 21-center cluster randomized clinical trial was coordinated by the Center for Safety Research at the Hospital for Sick Children, in Toronto, Ontario, Canada, and was overseen by the study executive steering committee and the Canadian Critical Care Trials Group.

Eligibility criteria were based on hospital, inpatient unit, and patient criteria. Hospitals were included if they had a pediatric ICU and if their hospital leadership agreed to randomization. Hospitals with rapid response teams were eligible to participate. Eligible inpatient units were defined as areas in which care was provided to pediatric inpatients, including emergency departments that used inpatient documentation records to care for admitted patients.

Hospitals were ineligible if they were already using a severity of illness score on inpatient units, or were planning to introduce or discontinue a rapid response team. Ineligible inpatient units were ICUs, areas designated for anesthesiologist-supervised procedures, and high-dependency units in which critical care physicians supervised care.

Infant (gestational age ≥37 weeks) to adolescent (aged ≤18 years) patients were included if they had received care in 1 or more eligible inpatient units. Participating hospitals were randomized in a 1:1 ratio to either the BedsidePEWS intervention or usual care using a computer-generated random sequence. Randomization was within 2 strata of hospital size (<200 and ≥200 eligible inpatient ward beds). A block size of 4 was used for both strata, and was only known by the study statistician. Randomization and disclosure of the resulting site allocation occurred during the second week of data collection at each site.

During the first 26 weeks, an interprofessional team at the Center for Safety Research supported implementation teams at hospitals randomized to the intervention as they led local preparations. The implementation of the intervention involved several steps. First, the inpatient unit–based vital sign documentation was changed to the paper-based BedsidePEWS documentation record. The chart designed by health care professionals enabled documentation and graphical presentation of individual vital signs, and manual calculation of the BedsidePEWS score. The BedsidePEWS score ranges from 0 to 26, with higher scores indicating greater severity of illness. Five age-specific paper documentation records for each of the age groups of the BedsidePEWS score were introduced at each hospital. Second, the language of the BedsidePEWS recommendations for care escalation and de-escalation were revised to match the local vernacular. These recommendations encompassed multiple domains including documentation frequency, physician review, and ICU consultation and were printed on each documentation record. Third, locally relevant education programs were developed and delivered 1 to 2 months prior to clinical implementation, which began during week 26 as the study run-in phase.

Implementation involved the routine use of the BedsidePEWS documentation record and the score-matched care recommendations for all patients admitted to eligible inpatient unit beds. The BedsidePEWS intervention was continued throughout the 52-week intervention period (Figure 1).

Figure 1. Hospital Site and Patient Flow Through Study.

^aEnrollment began in 2011 for 8 hospitals, in 2012 for 10 hospitals, and in 2013 for 5 hospitals. Disclosure of randomization to the hospital occurred during the second week of the 26-week baseline period.

^bHospitals randomized to the BedsidePEWS intervention collected data during the 26-week baseline period as they prepared for implementation of the intervention. During the 5-week run-in phase, adherence to vital sign documentation was assessed and reported to the study executive steering committee. Implementing hospitals required a minimum of 80% adherence to documentation standards and the majority vote of the executive steering committee to move into the intervention period.

^cHospitals randomized to usual care collected data during the 26-week baseline period, did not collect data during the 5-week run-in phase, and then resumed data collection for the 52-week intervention period.

^dDid not include beds in the intensive care unit.

^eDischarges were all eligible patients discharged from the hospital and included the patients who died. Patients who were in the hospital at the end of the study period were regarded as discharged.

^fUnable to adhere to implementation timelines specified by the study and were excluded before implementation. These hospitals were regional pediatric centers. One hospital had a rapid response team and no extracorporeal membrane oxygenation and the other did not have a rapid response team but had extracorporeal membrane oxygenation. The numbers of patient-days and patient discharges reported reflect the amount of data received from each hospital before they withdrew from the study.

^gThe 5-week run-in phase included weekly assessment beginning during the second week of implementation.

Hospitals with usual care (control) did not receive a new prospective severity score to help identify children at risk of significant clinical deterioration during the study, but continued to use preexisting rapid response teams, ICU consultation mechanisms, and resuscitation teams.

During the run-in phase, adherence to the BedsidePEWS documentation recommendations was assessed by review of documentation frequency and scoring accuracy. The study executive steering committee reviewed weekly adherence data leading to either extension of the 5-week run-in phase or transition to active data collection during the intervention period. Ongoing review of adherence included practice audits with local reporting at implementation hospitals.

Hospitals randomized to usual care resumed data collection after a 5-week hiatus. Documentation practices were reviewed to reflect implementation fidelity at BedsidePEWS hospitals and documentation practice at control hospitals. Five patients who had been on an eligible inpatient ward for more than 24 hours were randomly selected from each hospital during each study week. The number of documented assessments were abstracted for each of 7 clinical observation types, and the number of clinical observation types in the most recent set of clinical observations was counted.

The analysis of clinical documentation completeness was revised from the protocol-specified method before the analyses were conducted. Instead of evaluating all sets of documented clinical observations, the last 1 was selected. The threshold number of observations within that set of observations was reduced from 7 to 5 to be consistent with the minimum number of observations recommended to calculate a BedsidePEWS score.

Study Outcomes

The primary outcome was all-cause hospital mortality. This included deaths among children with do-not-resuscitate (DNR) orders because the DNR order reflects current expectations of outcome rather than the preventability of the clinical events that preceded the DNR order. Among hospitalized pediatric patients, the majority of deaths occur remote from the clinical deterioration event (within days), and there is a relatively short period (within hours) between following the DNR order and death. Thus, the placement of a DNR order may not provide a good separation between potentially preventable and unpreventable death.

The main secondary outcome was the significant clinical deterioration event. This measure of late ICU admission was a composite outcome that occurred among patients without DNR orders and was composed of 1 or more of the following: death before ICU admission; provision of cardiopulmonary resuscitation, tracheal intubation, administration of vasoactive medication, or provision of fluid boluses of 60 mL/kg or greater within the 12 hours before ICU admission; tracheal intubation, cardiopulmonary resuscitation, initiation of extracorporeal membrane oxygenation, or death within the first hour of ICU admission.

The study protocol included 21 other prespecified outcomes. The outcomes included mortality without a DNR, ICU mortality (overall and after urgent ICU admission), potentially preventable cardiac arrest, unplanned ICU readmission and hospital readmission within 48 hours, predicted mortality using severity of illness at ICU admission, organ dysfunction in the ICU, and ventilator-free days in the ICU among patients urgently admitted to ICU.

Urgent ICU admission was defined as (1) transfer to the ICU within 6 hours of the transfer decision from an eligible inpatient unit and (2) transfer initiated while an eligible patient was in the operating room. This definition intentionally incorporated patients in which preoperative review may have identified candidates for elective postoperative ICU admission, and recognizing that unexpected intraoperative complications (in previously well patients) that lead to ICU admission are uncommon.

The potential preventability of cardiac arrest was defined as the degree to which events may have been avoided given the application of reasonable current standards of practice by an average practitioner and system anticipated to manage the condition in question. The assessment method was based on a validated approach used to evaluate clinical data describing cardiac arrest and other adverse events. Blinded review by 2 independent experts resulted in either initial agreement or discussion leading to a consensus rating. Assessment ratings of preventable by greater than 50% were deemed potentially preventable.

For each urgent ICU admission, the revised Paediatric Index of Mortality (PIM2) score, organ dysfunction using the Paediatric Logistic Organ Dysfunction score, and ICU mortality were determined. If a patient had more than 1 admission, the mean was used. Ventilator-free days were determined for the 28 days after the first ICU admission during each study period. Complementing the above prespecified outcomes, we report rates of ward-based cardiac arrest and urgent ICU admission, and the PIM2 score at ICU admission. These outcomes were not prespecified.

The process of care outcomes were immediate calls for a physician, immediate calls for the resuscitation team, consultations to the ICU or rapid response team (response within 15 minutes), and documentation of clinical observations. Resource use among patients after urgent ICU admission was assessed by the ICU length of stay and the days in which mechanical ventilation, high-frequency oscillatory ventilation, hemodialysis, extracorporeal membrane oxygenation, and nitric oxide were used. Definitions of these outcomes appear in eTable 1 in Supplement 3. The perceptions of frontline staff and administrators were sought using surveys and are not reported herein. Data were collected by trained research coordinators.

Sample Size and Assumptions

Power calculations were based on data from 2007 to 2009 using an established method for cluster randomized trials. We assumed a baseline all-cause hospital mortality of 5.1 deaths per 1000 hospital discharges and estimated a mortality reduction of 1 per 1000 hospital discharges (from 5.1 to 4.1) to be sufficient to change practice. Given a κ of 0.15 (intercluster coefficient of variation), inclusion of 20 hospitals randomized in a 1:1 ratio with an average of 119 beds, occupancy of 0.90, and average hospital stay of 4 days could show an absolute risk reduction of 0.9 per 1000 hospital discharges for mortality with 2-sided type I error probability of .05 and 80% power. Assuming attrition of 1 to 2 hospitals, we planned to enroll 22 hospitals.

Data Analyses

Demographic and unadjusted outcomes data are reported using descriptive statistics, medians with interquartile ranges, means and SDs, as proportions with 95% CIs, and as rate differences with 95% CIs. Patient outcomes are expressed as rates per 1000 eligible patients discharged from the hospital (all-cause mortality, hospital readmission), per 1000 ICU discharges (ICU mortality), per 1000 patient-days in the ICU (ICU outcomes), and per 1000 patient-days on eligible inpatient units (other outcomes). Outcomes are reported for the baseline and intervention periods for each hospital as recommended for cluster randomized trials using odds ratios (ORs) or rate ratios and between-group rate differences with 95% CIs.

Generalized estimating equation models with an exchangeable correlation structure and grouping by center were used to compare outcomes between centers assigned to BedsidePEWS and usual care. Binary outcomes used a logistic model, count outcomes used a Poisson model with patient-days as an offset, and continuous outcomes used a Gaussian model. To estimate absolute differences in rates and proportions, the logistic and Poisson models were fitted using an identity link function. If the generalized estimating equation model with the identity link did not converge, the differences and their 95% CIs were calculated using the corresponding model without adjusting for baseline.

Hospital-level data were used for mortality, readmission and resuscitation team calls, stat calls, and ICU consultation. In each analysis, the 2 predictors were a binary variable for intervention and the center’s baseline summary value of the corresponding outcome. Individual-level data were used for analyses of urgent ICU admission outcomes and accounted for clustering within center. An interim analysis was neither planned nor performed.

A 2-sided P value of .05 was regarded as significant. Post hoc adjustment for multiple comparisons of the 21 prespecified outcomes used the method of Holm. The assumptions supporting the trial sample size calculation and those found after its conduct were tabulated to enable post hoc comparison. The analytic team were not blinded to the treatment allocation. Four planned subgroup analyses including hospitals with and without rapid response teams were described in the protocol, but are not reported herein.

Results

Thirty-four hospitals were screened for enrollment in this study. Twenty-three hospitals met the eligibility criteria. The 21 hospitals that completed the study were located in Belgium, Canada, England, Ireland, Italy, New Zealand, and the Netherlands and had a total of 2085 eligible inpatient unit beds (Figure 1). Hospitals had a range of pediatric services including cardiopulmonary bypass, solid organ transplantation, and bone marrow transplantation, and all had pediatric trainee physicians and continuous in-house physician staffing (Table 1). Because only 3 hospitals had more than 200 eligible inpatient beds, the planned stratification was removed from the statistical analysis.

Table 1. Baseline Characteristics of Intervention and Control Hospitals (N = 21).

Characteristic	BedsidePEWS	Usual Care
No. of hospitalsa	10	11
No. of total beds	5172	3169
No. of pediatric bedsb	937	1148
<200, No. (%)	8 (80)	10 (91)
≥200, No. (%)	2 (20)	1 (9)
Hospital Services, No. (%)
Rapid response team	5 (50)	4 (36)
Affiliated with university	8 (80)	11 (100)
Emergency department	9 (90)c	11 (100)
Cardiopulmonary bypass	5 (50)	7 (64)
Extracorporeal membrane oxygenation	5 (50)	8 (73)
Solid organ transplant	5 (50)	7 (64)
Bone marrow transplant	4 (40)	6 (55)
Hospital Staffing
Most senior ward physician in-house overnight, No. (%)d
Pediatric-trained staff physician	4 (40)	3 (27)
Fellow	1 (10)	3 (27)
Resident	5 (50)	5 (45)
Most senior ICU physician in-house overnight, No. (%)
ICU staff physician	2 (20)	1 (9)
ICU fellow	5 (50)	3 (27)
Resident	1 (10)	7 (64)
Emergency physicians available overnight, No. (%)e	8 (80)	8 (73)
No. of full-time equivalent nurses
<0.5	169	117
0.5-0.9	467	962
>0.9	749	1070
Hospital Volume, No.
Patient dischargesf	26 664	46 718
Patient-days	129 700	162 497
ICU patient dischargesf	1859	2599

Abbreviations: BedsidePEWS, Bedside Paediatric Early Warning System; ICU, intensive care unit.

^aHad a pediatric ICU, pediatric trainees, staff physicians for the pediatric ICU, and pediatric surgeons. All hospitals remained eligible throughout the course of the study.

^bExcluded ICU beds.

^cOne hospital without an emergency department provided care to specialized patient populations and accepted patients from other facilities.

^dAll had 1 or more physicians in-house continuously.

^eIncluded pediatric emergency physicians or other pediatric trainees.

^fPatients who were inpatients at the end of the baseline or intervention periods were regarded as discharged from the hospital.

Enrollment was initiated on February 28, 2011, and ended on June 21, 2015. Follow-up was completed on July 19, 2015. There were 73 382 hospital discharges and 292 197 patient-days during the baseline period and 144 539 hospital discharges and 559 443 patient-days during the intervention period. There was no loss to follow-up for study events.

The frequency of documentation increased at the BedsidePEWS hospitals for 5 of the 7 clinical observation types reviewed (Table 2). At the BedsidePEWS hospitals compared with the usual care hospitals, the difference in the number of documented observations within 24 hours increased for respiratory rate by a mean of 0.85 (95% CI, 0.02-1.68; P = .05); systolic blood pressure, 1.12 (95% CI, 0.59-1.65; P < .001); transcutaneous oxygen saturation, 1.06 (95% CI, 0.27-1.85; P = .009); respiratory effort, 4.67 (95% CI, 3.23-6.12; P < .001); and capillary refill, 4.65 (95% CI, 3.49-5.80; P < .001). The overall proportion of sets of clinical observations with 5 or more of the 7 clinical observation types reviewed increased by 38.1% (95% CI, 20.8%-55.4%) at BedsidePEWS hospitals compared with usual care hospitals (P < .001) (eFigure 2 in Supplement 3).

Table 2. Measurements to Assess the Completeness of the Documented Clinical Observations Among Randomly Selected Patients^a.

	Mean No. of Measurements (SD)b				Between-Group Mean Difference (95% CI)c	P Value	Adjusted P Valued
	BedsidePEWS		Usual Care
	Baseline Period	Intervention Period	Baseline Period	Intervention Period
Total No. of patient assessments	1270	2588	1419	2832
Heart rate	6.97 (5.1)	7.40 (4.8)	6.48 (4.9)	6.45 (4.7)	0.58 (−0.11 to 1.26)	.10	.30
Respiratory rate	6.99 (5.5)	7.38 (4.9)	5.59 (4.2)	5.53 (4.0)	0.85 (0.02 to 1.68)	.05	.18
Systolic blood pressure	4.09 (4.2)	5.05 (3.5)	3.58 (4.5)	3.59 (3.6)	1.12 (0.59 to 1.65)	<.001	<.001
Transcutaneous oxygen saturation	6.64 (5.3)	7.30 (5.0)	5.29 (5.4)	5.21 (5.3)	1.06 (0.27 to 1.85)	.009	.04
Respiratory effort	1.99 (3.6)	7.16 (4.7)	2.80 (3.3)	3.00 (3.6)	4.67 (3.23 to 6.12)	<.001	<.001
Capillary refill	1.96 (3.5)	6.66 (4.4)	1.48 (3.0)	1.66 (3.1)	4.65 (3.49 to 5.80)	<.001	<.001
Oxygen therapy	6.80 (5.2)	7.30 (4.8)	5.83 (5.6)	6.09 (5.7)	0.37 (−0.71 to 1.46)	.50	>.99
Observation sets with ≥5 vital signs, No. (%)	960 (75.6)	2563 (99.0)	883 (62.2)	1725 (60.9)	38.1 (20.8 to 55.4)e	<.001	<.001

Abbreviation: BedsidePEWS, Bedside Paediatric Early Warning System.

^aAt each hospital during each study week, 5 patients admitted to a ward for at least 24 hours were randomly selected for documentation review. Study coordinators abstracted the number of documented measurements during the 24 hours before assessment for each of the 7 clinical observations in this Table and abstracted the number of clinical observation types included in the last set of clinical observations. A set of clinical observations was regarded as those that were documented as being from the same time. There were 8190 (21 sites × 5 patients per week × [26 + 52] weeks) case report forms anticipated and 8109 (99%) were obtained. The missing data may reflect the challenges of finding 5 randomly selected patients admitted for more than 24 hours at a smaller hospital, and in some cases, the fifth patient may have been unintentionally missed.

^bUnless otherwise indicated.

^cCalculated from a generalized estimating equation linear regression model that adjusted for clustering by hospital using an exchangeable correlation structure. The BedsidePEWS group and the hospital baseline mean value were used as predictors.

^dAdjustment for multiple comparisons was performed using the method of Holm.

^eCalculated using a generalized estimating equation regression model with a binomial variance and a linear link function. The BedsidePEWS group and the hospital baseline proportion were the only covariates.

Primary Outcome

For the primary outcome of all-cause mortality, there were 244 deaths that occurred at the hospital, corresponding to 1.69 per 1000 patient discharges and including 155 deaths (63.5%) after DNR orders (Table 3). During the baseline period, there were no deaths at 4 hospitals. During the 52-week intervention period, there were fewer than 10 deaths at 13 hospitals (Figure 2 and eTable 2 in Supplement 3). Hospital mortality was 1.93 per 1000 patient discharges at the BedsidePEWS hospitals compared with 1.56 per 1000 patient discharges at usual care hospitals.

Table 3. Study Outcomes for Bedside Paediatric Early Warning System (BedsidePEWS) vs Usual Care.

	BedsidePEWS				Usual Care				Adjusted Between-Group Rate Difference (95% CI)a	Adjusted Ratio (95% CI)	P Value
	Baseline Period		Intervention Period		Baseline Period		Intervention Period
	No. of Events	Rate	No. of Events	Rate	No. of Events	Rate	No. of Events	Rate
Patient dischargesb	26 664		50 173		46 718		94 366
Patient-daysb	129 700		251 859		162 497		307 584
Primary Outcome
All-cause hospital mortalityc	52	1.95	97	1.93	61	1.31	147	1.56	0.01 (−0.80 to 0.81)	OR, 1.01 (0.61 to 1.69)d	.96
All-cause hospital mortality without a DNR orderc	26	0.98	42	0.84	16	0.34	47	0.50	0.36 (−0.53 to 1.25)	OR, 2.05 (0.64 to 6.61)d	.23
Secondary Outcome
Significant clinical deterioration evente	80	0.62	127	0.50	144	0.89	259	0.84	−0.34 (−0.73 to 0.05)	RR, 0.77 (0.61 to 0.97)f	.03
Post hoc Outcomesg
ICU mortalityh	33	17.75	56	16.92	34	13.08	91	17.90	−3.01 (−12.26 to 6.25)	OR, 0.89 (0.51 to 1.57)d	.69
ICU mortalityc	33	1.24	56	1.12	34	0.73	91	0.96	−0.11 (−0.73 to 0.51)	OR, 0.95 (0.48 to 1.86)d	.88
Cardiac arreste	15	0.12	27	0.11	18	0.11	32	0.10	0 (−0.06 to 0.07)	RR, 1.02 (0.65 to 1.62)f	.92
Potentially preventable cardiac arreste,i	11	0.08	21	0.08	12	0.07	29	0.09	−0.02 (−0.07 to 0.02)	RR, 0.87 (0.49 to 1.54)f	.62
Immediate call for resuscitation teame	64	0.49	126	0.50	97	0.60	179	0.58	0.02 (−0.07 to 0.10)	RR, 0.98 (0.82 to 1.17)f	.83
Immediate call for physiciane,j	1007	7.76	1727	6.86	844	5.19	1157	3.76	3.10 (−1.92 to 8.11)	RR, 1.17 (0.73 to 1.88)f	.52
Urgent (<15 min) ICU consultatione	478	3.69	1015	4.03	928	5.71	1694	5.51	0.16 (−0.57 to 0.89)	RR, 1.05 (0.85 to 1.30)f	.64
Urgent ICU admissione	469	3.62	828	3.29	652	4.01	1178	3.83	−0.18 (−0.67 to 0.30)	RR, 0.95 (0.82 to 1.09)f	.45
ICU readmission <48 hh	64	34.43	94	28.40	73	28.09	108	21.25	4.95 (−1.62 to 11.52)	OR, 1.11 (0.77 to 1.61)d	.58
Hospital readmission <48 hc	101	3.79	170	3.39	201	4.30	387	4.10	−0.71 (−4.92 to 3.49)	OR, 0.93 (0.61 to 1.41)d	.74

Abbreviations: DNR, do-not-resuscitate; ICU, intensive care unit; OR, odds ratio; RR, rate ratio.

^aCalculated using binomial and Poisson generalized estimating equation models with an identity link function and adjustment for baseline values. For significant clinical deterioration, cardiac arrest, immediate call for physician, and hospital readmission, the model was not adjusted for baseline values.

^bData were used for the rate calculations.

^cRates expressed per 1000 patient discharges from the hospital.

^dLogistic regression was used. Analyses included adjustment for baseline event rates and used the generalized estimating equation approach to group data by center to account for clustering.

^eRates expressed per 1000 patient-days.

^fPoisson regression was used. Analyses included adjustment for baseline event rates and used the generalized estimating equation approach to group data by center to account for clustering.

^gThe analyses of these outcomes should be regarded as exploratory. There was adjustment for multiple comparisons using the Holm method and yielded P values of >.99 for these 10 outcomes.

^hRates expressed per 1000 discharges from the ICU.

ⁱAssessed by 2 blinded expert reviewers using a 6-point scale. A rating of 4 indicates more than likely preventable. Weighted κ, 0.35 (95% CI, 0.15-0.51) for agreement of initial reviewer ratings. Discussion between the 2 reviewers increased agreement such that arbitration by the third reviewer was used for only 5 events. There were 21 (77.8%) cardiac arrest events rated as potentially preventable at the BedsidePEWS hospitals vs 29 (90.6%) events at the control hospitals.

^jDetermined from multiple potentially overlapping sources (eg, switchboard paging logs, ward clerk documentation, review of patients with events, and other hospital reports). The customized approach taken at each hospital was developed as part of site coordinator training provided by the coordinating center and was applied consistently throughout the study at each site.

Figure 2. Mortality by Hospital.

All-cause hospital mortality rates during the baseline and intervention periods are presented by hospital. Each circle represents a hospital. The circle center reflects the coordinates of the baseline and intervention mortality rates. The colored lines represent the linearized fitted relationships between mortality during the baseline and intervention periods for the BedsidePEWS intervention hospitals (dashed orange line) and usual care hospitals (solid blue line). The estimated difference in the slopes between the BedsidePEWS (slope = 0.57) and the usual care group (slope = 0.53) was not statistically significantly different from 0 (P = .94). In the analysis, the slopes were assumed to be equal. In Supplement 3, eFigure 1 provides linkage of these hospital-level mortality data to additional information about the individual hospitals contained in eTables 2 and 3.

^aThe circle size is proportional to the number of discharges during the intervention period of that hospital. Thus hospitals with larger circles are contributing more data and will have narrower 95% CIs for the true values of their mortality rates.

The primary analysis found no significant differences between the BedsidePEWS and usual care hospitals (adjusted between-group rate difference, 0.01 [95% CI, −0.80 to 0.81 per 1000 patient discharges]; adjusted OR, 1.01 [95% CI, 0.61 to 1.69]; P = .96). Hospital mortality among patients without a DNR was 0.84 per 1000 discharges at the BedsidePEWS hospitals compared with 0.50 per 1000 discharges at the usual care hospitals (adjusted between-group rate difference, 0.36 [95% CI, −0.53 to 1.25 per 1000 patient discharges]; adjusted OR, 2.05 [95% CI, 0.64 to 6.61]; P = .23).

Secondary Outcome

There were 386 (127 at BedsidePEWS hospitals vs 259 at usual care hospitals) significant clinical deterioration events (Table 3). This corresponded to rates of 0.50 per 1000 patient-days at BedsidePEWS hospitals compared with 0.84 per 1000 patient days at usual care hospitals (P = .03). The baseline-adjusted rate ratio was 0.77 (95% CI, 0.61 to 0.97) and the adjusted between-group rate difference was −0.34 (95% CI, −0.73 to 0.05) events per 1000 patient-days. Significant clinical deterioration events comprised 15.3% of urgent ICU admissions of eligible patients at BedsidePEWS hospitals and 22.0% at usual care hospitals and included 59 cardiac arrest events and 8 deaths before transfer to the ICU.

Other Outcomes

There were no significant differences in the rates of cardiac arrest, potentially preventable cardiac arrest, unplanned ICU readmission, or hospital readmission (Table 3); however, individual hospital rates were low (eTable 3 in Supplement 3).

The 1653 patients with urgent ICU admission in the per-patient analysis remained in ICU for 15 212 days and received mechanical ventilation for 6400 days. There were no significant between-group differences for the severity of illness at ICU admission, organ dysfunction, ventilator-free days, and resource use (eTable 4 in Supplement 3).

There were 109 deaths during the course of urgent ICU admission (42 [6.1%] in BedsidePEWS hospitals vs 67 [6.9%] in usual care hospitals; adjusted between-group difference, −1.55% [95% CI, −4.90% to 1.80%], P = .36). The mortality rate predicted by the PIM2 score was 5.5% at the BedsidePEWS hospitals vs 4.6% at usual care hospitals (adjusted between-group difference, 0.69% [95% CI, −0.54% to 1.92%], P = .27).

There were 2884 calls for immediate physician review, 2709 urgent (<15 minutes) ICU consultations, and 305 immediate calls for resuscitation teams (Table 3). There were no significant differences in calls for immediate physician review (adjusted between-group rate difference, 3.10 [95% CI, −1.92 to 8.11 per 1000 patient-days]; adjusted rate ratio, 1.17 [95% CI, 0.73 to 1.88]; P = .52), in immediate calls for the resuscitation team (adjusted between-group rate difference, 0.02 [95% CI, −0.07 to 0.10 per 1000 patient-days]; adjusted rate ratio, 0.98 [95% CI, 0.82 to 1.17]; P = .83), or for urgent ICU consultation (adjusted rate ratio, 1.05 [95% CI, 0.85 to 1.30]; adjusted between-group rate difference, 0.16 [95% CI, −0.57 to 0.89], P = .64).

Discussion

In this international cluster randomized trial comparing implementation of the BedsidePEWS intervention vs usual care, the BedsidePEWS intervention did not significantly decrease all-cause mortality among hospitalized pediatric patients. The 95% CI for the between-group difference excludes a difference of greater than 0.8 deaths per 1000 hospital discharges in either direction (47% relative change). Exploratory analyses did not find significant reductions in mortality without DNR orders or ICU mortality. Together these findings do not support the use of the BedsidePEWS intervention to reduce mortality.

Despite the observed rate of all-cause hospital mortality being lower than anticipated (1.69 vs 5.10 per 1000 patient discharges), the observed variability between hospitals (0-5.21 per 1000 discharges) was similar to that anticipated before the trial (eTable 5 in Supplement 3). This may reflect a floor effect of 0 mortality that reduced between-hospital variability. There were 4 hospitals that had no deaths during the baseline period, and 1 had no deaths during the intervention period. If hospitals with higher expected mortality had been enrolled, variability may have been reduced; however, exclusion of smaller regional pediatric centers would have reduced the generalizability of the trial results.

The composite outcome used to measure late ICU admission (significant clinical deterioration events) was significantly reduced in hospitals implementing the BedsidePEWS intervention. This isolated positive finding was not accompanied by significant effects on cardiac arrest, urgent ICU admission, mortality after urgent ICU admission, risk-adjusted ICU mortality, or ICU resource use. These latter findings contrast with the data underpinning the Society of Critical Care Medicine 2016 guidelines for adult ICU admission and suggestions that other measures of timeliness of ICU admission among pediatric patients are associated with ICU outcomes. It is possible that among urgent pediatric ICU admissions, late ICU admissions may constitute too small a proportion to modify overall mortality or ICU resource use, or that the indication for cardiorespiratory intervention was a greater determinant of patient outcome than the time that the intervention was initiated relative to ICU admission.

Exploratory analyses found no significant between-group difference in cardiac arrest among patients without a DNR order. Rates were similar to previous single-center reports. Eight cardiac arrests (14%) resulted in death before ICU admission, and most were judged to be potentially preventable (78% in the BedsidePEWS intervention group vs 91% in the usual care control group). Realizing the potential to prevent these rare serious events and to improve overall patient outcomes may require other cardiac arrest prevention strategies that operate in other hospital areas, and may include increased human health resources, monitoring, and educational interventions.

The strengths of the study include the cluster randomized design, the large size of the trial, the geographic diversity of participating hospitals, complete follow-up of clinically relevant outcome measures, demonstration that the intervention changed practice, and the use of robust processes to ensure the integrity of the study data, analyses, and interpretations.

Limitations

This study has several limitations. First, temporal reductions in mortality provided the rationale for the cluster randomized design; however, the observed mortality was lower than the conservative estimates used in planning, and cardiac arrest was infrequent (eTable 4 in Supplement 3). Enrollment of more hospitals with higher event rates, less diverse characteristics, or from fewer countries may have increased the precision of the results.

Second, descriptions of the 144 539 enrolled patients were not collected. This may limit the confident generalization of the study results to hospitals providing a different range of inpatient services, and to hospitals caring for pediatric patients of different ages than the regional centers studied.

Third, the study was not blinded. Unmeasured quality initiatives may have narrowed observed differences along with the BedsidePEWS implementation training of frontline staff that began during the baseline period.

Fourth, the initial agreement between reviewers was low for the rating of potentially preventable cardiac arrest. The assessment method used was based on validated adverse event evaluation methods that were modified to increase reviewer blinding and were supported by reviewer training. Low initial agreement may reflect the subjectivity of preventability review. In addition, there was a loss of context associated with presentation of the consistently abstracted data, which was used to ensure reviewers remained blind to randomization group.

Fifth, BedsidePEWS is a complex health care intervention that required the actions of multiple persons and teams, with the intent of becoming embedded in social systems. The evaluation of adherence was mechanistic, focusing on documentation rather than effects on clinical communication and culture. The 2 sites that withdrew from the study were both randomized to the BedsidePEWS intervention. The extended run-in phase lasting 1, 5, and 6 weeks in 3 of the BedsidePEWS hospitals may have biased the results. However, this also illustrates the practical challenges of conducting clinical trials that randomize the routine business of hospital inpatient units to complex health care interventions.

Sixth, the inclusion of mortality with DNR orders was a pragmatic decision that reflected assumptions that such orders may occur after a preventable clinical deterioration. Uncertainty about the validity of this assumption provided rationale for the conduct of the sensitivity analysis of mortality after DNR order, and remains a potential limitation of all-cause hospital mortality.

Seventh, the generalizability of these results to other less well-developed or less robustly implemented early warning scoring systems cannot be assumed. In this trial, implementation was enabled by local teams that were closely overseen by the Center for Safety Research and a variety of hospital-specific strategies were used to maintain adherence.

Conclusions

Implementation of the Bedside Paediatric Early Warning System compared with usual care did not significantly decrease all-cause mortality among hospitalized pediatric patients. These findings do not support the use of this system to reduce mortality.

Supplement 1.

Trial protocol

Supplement 2.

Statistical analysis plan

Supplement 3.

eFigure 1. Hospital mortality per thousand hospital discharges

eFigure 2. Documented clinical observations in randomly selected patients

eTable 1. Study outcome definitions

eTable 2. Hospital level outcomes: mortality, mortality without ‘do not resuscitate’ order

eTable 3. Hospital-level outcomes: significant clinical deterioration, cardiac arrest and potentially preventable cardiac arrest

eTable 4. Per patient analysis of ICU resource utilisation

eTable 5. Study sample pre-trial assumptions and post-trial calculations

Section Editor: Derek C. Angus, MD, MPH, Associate Editor, JAMA (angusdc@upmc.edu).