Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards

Abstract

Background

Early warning systems (EWS) and rapid response systems (RRS) have been implemented internationally in acute hospitals to facilitate early recognition, referral and response to patient deterioration as a solution to address suboptimal ward‐based care. EWS and RRS facilitate healthcare decision‐making using checklists and provide structure to organisational practices through governance and clinical audit. However, it is unclear whether these systems improve patient outcomes. This is the first update of a previously published (2007) Cochrane Review.

Objectives

To determine the effect of EWS and RRS implementation on adults who deteriorate on acute hospital wards compared to people receiving hospital care without EWS and RRS in place.

Search methods

We searched CENTRAL, MEDLINE, Embase and two trial registers on 28 March 2019. We subsequently ran a MEDLINE update on 15 May 2020 that identified no further studies. We checked references of included studies, conducted citation searching, and contacted experts and critical care organisations.

Selection criteria

We included randomised trials, non‐randomised studies, controlled before‐after (CBA) studies, and interrupted time series (ITS) designs measuring our outcomes of interest following implementation of EWS and RRS in acute hospital wards compared to ward settings without EWS and RRS.

Data collection and analysis

Two review authors independently checked studies for inclusion, extracted data and assessed methodological quality using standard Cochrane and Effective Practice and Organisation of Care (EPOC) Group methods. Where possible, we standardised data to rates per 1000 admissions; and calculated risk differences and 95% confidence intervals (CI) using the Newcombe and Altman method. We reanalysed three CBA studies as ITS designs using segmented regression analysis with Newey‐West autocorrelation adjusted standard errors with lag of order 1. We assessed the certainty of evidence using the GRADE approach.

Main results

We included four randomised trials (455,226 participants) and seven non‐randomised studies (210,905 participants reported in three studies). All 11 studies implemented an intervention comprising an EWS and RRS conducted in high‐ or middle‐income countries. Participants were admitted to 282 acute hospitals. We were unable to perform meta‐analyses due to clinical and methodological heterogeneity across studies. Randomised trials were assessed as high risk of bias due to lack of blinding participants and personnel across all studies. Risk of bias for non‐randomised studies was critical (three studies) due to high risk of confounding and unclear risk of bias due to no reporting of deviation from protocol or serious (four studies) but not critical due to use of statistical methods to control for some but not all baseline confounders. Where possible we presented original study data which reported the adjusted relative effect given these were appropriately adjusted for design and participant characteristics. We compared outcomes of randomised and non‐randomised studies reported them separately to determine which studies contributed to the overall certainty of evidence. We reported findings from key comparisons.

Hospital mortality

Randomised trials provided low‐certainty evidence that an EWS and RRS intervention may result in little or no difference in hospital mortality (4 studies, 455,226 participants; results not pooled). The evidence on hospital mortality from three non‐randomised studies was of very low certainty (210,905 participants).

Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)

One randomised study showed that an EWS and RRS intervention probably results in no difference in this composite outcome (adjusted odds ratio (aOR) 0.98, 95% CI 0.83 to 1.16; 364,094 participants; moderate‐certainty evidence). One non‐randomised study suggests that implementation of an EWS and RRS intervention may slightly reduce this composite outcome (aOR 0.85, 95% CI 0.72 to 0.99; 57,858 participants; low‐certainty evidence).

Unplanned ICU admissions

Randomised trials provided low‐certainty evidence that an EWS and RRS intervention may result in little or no difference in unplanned ICU admissions (3 studies, 452,434 participants; results not pooled). The evidence from one non‐randomised study is of very low certainty (aOR 0.88, 95% CI 0.75 to 1.02; 57,858 participants).

ICU readmissions

No studies reported this outcome.

Length of hospital stay

Randomised trials provided low‐certainty evidence that an EWS and RRS intervention may have little or no effect on hospital length of stay (2 studies, 21,417 participants; results not pooled).

Adverse events (unexpected cardiac or respiratory arrest)

Randomised trials provided low‐certainty evidence that an EWS and RRS intervention may result in little or no difference in adverse events (3 studies, 452,434 participants; results not pooled). The evidence on adverse events from three non‐randomised studies (210,905 participants) is very uncertain.

Authors' conclusions

Given the low‐to‐very low certainty evidence for all outcomes from non‐randomised studies, we have drawn our conclusions from the randomised evidence. This evidence provides low‐certainty evidence that EWS and RRS may lead to little or no difference in hospital mortality, unplanned ICU admissions, length of hospital stay or adverse events; and moderate‐certainty evidence of little to no difference on composite outcome. The evidence from this review update highlights the diversity in outcome selection and poor methodological quality of most studies investigating EWS and RRS. As a result, no strong recommendations can be made regarding the effectiveness of EWS and RRS based on the evidence currently available. There is a need for development of a patient‐informed core outcome set comprising clear and consistent definitions and recommendations for measurement as well as EWS and RRS interventions conforming to a standard to facilitate meaningful comparison and future meta‐analyses.

Plain language summary

Checklists and specialist teams to recognise and manage sick people in hospital

What was the aim of the review?

Adults in hospital who become unwell need to be identified quickly by clinical staff as requiring help. One way to do this is for doctors and nurses working on hospital wards to use a checklist of vital signs (e.g. blood pressure, pulse) to help them to recognise signs that patients are getting worse. These checklists are then used to refer patients to specialist teams of doctors and nurses for rapid assessment and treatment. We conducted this review to understand if using a checklist and referring to a specialist team reduces the number of deaths, unplanned intensive care unit (ICU) admissions/readmissions, hospital length of stay, and cardiac or respiratory arrest compared to hospital wards without these resources.

Key messages

We found evidence that checklists to help doctors and nurses recognise and refer patients who get worse early to specialist teams for management may result in little or no difference to the number of deaths, unplanned ICU admissions, length of hospital stay or cardiac arrests in hospital.

What did we study in the review?

This Cochrane Review presents what we know from research on the effect of hospital checklists to help doctors and nurses recognise and refer patients who are getting worse in hospital to specialist teams for help. Research has shown that patients in acute hospital wards often show early signs and symptoms, such as changes in breathing and pulse, when their condition is getting worse. It is thought that if hospital staff could identify and refer those patients who are getting worse earlier to specialist teams with appropriate knowledge and skills in acute care (active but short‐term treatment), then there would be fewer deaths, unplanned ICU admissions, cardiac arrests and reduced length of stay.

What were the main results of the review?

We identified 11 studies. In total, 666,131 participants in 282 hospitals were included from seven middle‐ to high‐income countries. The implementation of a checklist to recognise deteriorating patients and a specialist team to rapidly assess and treat presenting symptoms make little or no difference to deaths, unplanned ICU admissions, length of hospital stay, or cardiac and respiratory arrests compared to the wards or hospitals without access to the checklist and specialist team. No studies reported ICU readmission rates.

How up‐to‐date is this review?

The review authors searched for studies published up to May 2020.

Summary of findings

Summary of findings 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deterioration on acute hospital wards.

Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deterioration on acute hospital wards
Patient or population: adult on acute hospital wards at risk of clinical deterioration Settings: acute wards of hospital in the UK, US, Australia and the Netherlands Intervention: RRS that included an EWS Comparison: usual care without any form of EWS or RRS
Outcome	Study	Relative effect* (95% CI)	No of participants/admissions	Certainty of the evidence ‡ (GRADE)	Impact
Hospital mortality – randomised trials (follow‐up: 3–16 months)	Priestley 2004	aOR 0.52 (0.32 to 0.85)	2792	⊕⊕⊝⊝ Lowa,b	EWS and RRS may result in little or no difference in mortality among adults at risk of deterioration.
	Hillman 2005	aOR 1.03 (0.84 to 1.28)	364,094
	Jeddian 2016	aOR 1.02 (0.68 to 1.55)	18,684
	Haegdorens 2018	aOR 0.82 (0.34 to 1.95)	69,656
Hospital mortality – non‐randomised studies (follow‐up: 5–48 months)	Lighthall 2010	No relative differences are provided for this study, and therefore no data is presented.	—	⊕⊝⊝⊝ Very lowb,c,d	—
	Rothberg 2012	No relative differences are provided for this study, and therefore no data is presented.	—
	Davis 2015	No relative differences are provided for this study, and therefore no data is presented.	—
	Ludikhuize 2015	aOR 0.85 (0.64 to 1.00)	57,858
	Chen 2016	aRR 0.81 (0.76 to 0.86)	—
	Menon 2018	RR 1.0 (0.03 to 29.8)	18,954
Composite outcome – randomised trials Composite outcome of incidence (events divided by number of eligible participants admitted to the hospital during the study period) of cardiac arrests without a pre‐existing NFR order, unplanned ICU admissions, and unexpected deaths (deaths without a pre‐existing NFR order) (follow‐up: 6 months)	Hillman 2005	aOR 0.98 (0.83 to 1.16)	364,094	Moderatea	EWS and RRS probably results in no difference in the number of participants experiencing the composite outcome.
Composite outcome – non‐randomised studies Composite endpoint of cardiopulmonary arrest, unplanned ICU admission or death per 1000 admitted patients (follow‐up: 5 months)	Ludikhuize 2015	aOR 0.85 (0.72 to 0.99)	57,858	⊕⊕⊝⊝ Lowc	—
Unplanned ICU admission – randomised trials (follow‐up: 3–16 months)	Hillman 2005	aOR 1.04 (0.89 to 1.21)	364,094	⊕⊕⊝⊝ Lowa,b	EWS and RRS may result in little or no difference in unplanned ICU admission rates.
	Jeddian 2016	aOR 1.15 (0.64 to 2.09)	18,684
	Haegdorens 2018	aOR 1.23 (0.91 to 1.65)	69,656
Unplanned ICU admission – non‐randomised trial (follow‐up: 5 months)	Aitken 2015	No relative differences provided and no data presented.	—	⊕⊝⊝⊝ Very lowb,c,d	—
	Ludikhuize 2015	aOR 0.88 (0.75 to 1.02)	57,858
ICU readmission – randomised trials	—	—	—	—	No randomised trials reported rates of ICU readmissions.
ICU readmission – non‐randomised studies	—	—	—	—	No non‐randomised studies reported rates of ICU readmissions.
Length of hospital stay – randomised trials (follow‐up: 3 months)	Priestley 2004	HR 0.91 (0.83 to 0.98)	2733	⊕⊕⊝⊝ Lowa,d	EWS and RRS may have little or no effect on hospital length of stay.
	Jeddian 2016	aRGM 1.00 (0.97 to 1.03)	18,684
Length of hospital stay – non‐randomised studies	—	—	—	—	No non‐randomised studies reported length of hospital stay.
Adverse events (unexpected cardiac or respiratory arrest) – randomised trials (follow‐up: 3–16 months)	Hillman 2005	aOR 0.94 (0.79 to 1.13)	364,094	⊕⊕⊝⊝ Lowa,b	EWS and RRS may result in little or no difference in adverse events among adults at risk of deterioration.
	Jeddian 2016	aOR 1.00 (0.69 to 1.48)	18,684
	Haegdorens 2018	aOR 0.71 (0.33 to 1.52)	69,656
Adverse events (unexpected cardiac or respiratory arrest) – non‐randomised studies (follow‐up: 5–48 months)	Lighthall 2010	No relative differences provided and no data presented.	—	⊕⊝⊝⊝ Very lowb,c,d	—
	Rothberg 2012	aOR 1.00 (0.69 to 1.48)	—
	Aitken 2015	No relative differences provided and no data presented.	—
	Davis 2015	No relative differences provided and no data presented.	134,093
	Ludikhuize 2015	aOR 0.61 (0.39 to 0.94)	57,858
	Chen 2016	aRR 0.55 (0.48 to 0.62)	—
	Menon 2018	No relative differences provided and no data presented.	18,954
*The absolute effect is not presented due to lack of unadjusted data. ‡GRADE Working Group grades of evidence
GRADE Working Group grades of evidence High: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low. Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate. Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high. Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high. ‡ Substantially different = a large enough difference that it might affect a decision. aOR: adjusted odds ratio; aRGM: adjusted ratio of geometric means; aRR: adjusted risk ratio; CI: confidence interval; EWS: early warning system; HR: hazard ratio; ICU: intensive care unit; NFR: not‐for‐resuscitation; RR: risk ratio; RRS: rapid response system.

^aDowngraded one level for risk of bias: concerns regarding performance bias or contamination bias, or both.
^bDowngraded one level for imprecision of the direction of the effect: due to wide RR interpreted as less than 0.75 to greater than 1.25 which includes appreciable beneficial or harmful effects.
^cDowngraded one level for risk of bias: serious or critical risk of bias due to inadequate adjustment for confounders in observational studies assessed with Risk Of Bias in Non‐randomised Studies – of Interventions (ROBINS‐I) tool.
^dDowngraded one level for inconsistency: as unable to calculate effect size for all studies.

Background

Description of the condition

Patients who deteriorate on acute hospital wards often exhibit early physiological warning signs (such as changes in respiratory rate, heart rate, blood pressure, level of consciousness or urine output) prior to cardiac arrest, death or the need for emergent intensive care unit (ICU) admission (Kause 2004; Fujiwara 2016). These abnormal physiological changes are observable in documented vital signs one to four hours prior to cardiac arrest. An increased number of abnormal vital signs is associated with increased hospital mortality (Andersen 2016). Early identification of changes in vital signs can assist in the detection of people at risk for cardiac arrest, or in the early stages of deterioration, averting the occurrence of serious adverse events. However, these changes in vital signs are often missed, misinterpreted or mismanaged (Donaldson 2014). As a result, many acutely ill people on general hospital wards receive suboptimal care (McQuillan 1998; NCEPOD 2005). Suboptimal care is defined as delays in diagnosis, treatment or referral, suboptimal assessment, or inappropriate or inadequate treatment (Quirke 2011). Evidence suggests that up to 20% to 30% of hospital ward patients receive substandard care prior to ICU admission, with up to 70% of adverse events being potentially avoidable (Vlayen 2012; Garry 2014).

Description of the intervention

Early warning systems (EWS) and rapid response systems (RRS) have been implemented into practice internationally to improve the early identification of physiological instability and to initiate an early response mechanism to manage deteriorating patients on acute hospital wards. These systems were developed through expert consensus as a solution to address suboptimal ward‐based care and to reduce preventable hospital deaths. Essential features of RRS consist of three inter‐related components: an afferent arm, which is the crisis detection and response triggering mechanism; an efferent arm, which provides competent, skilled personnel and resources at the bedside to initiate an appropriate level of response; and a governance and audit process to co‐ordinate human and financial resources, evaluate and prevent future adverse events, and to ensure sustainability of RRS (DeVita 2006). Implementation of all three RRS components ensures deteriorating hospital patients are identified early through the use of EWS serving as track and trigger tools; patients are referred early to a more appropriate level of care using protocolised graded response strategies; and management is by professionals with specialist knowledge and skills in acute and critical care.

Crisis detection in the afferent arm of the RRS is facilitated by the use of a structured EWS tool to detect patient deterioration. These are known as 'track and trigger' systems. These systems include assessment tools incorporated into bedside observation charts on acute hospital wards that monitor physiological parameters such as systolic blood pressure, heart rate, respiratory rate, urinary output, temperature and level of consciousness (RCP 2017). 'Track' refers to the physiological monitoring of observations visually represented as trends on the observation chart. 'Trigger' refers to a threshold figure or criteria indicating impending clinical instability and requiring initiation of an early response to prevent an adverse event. The purpose of track and trigger tools is to provide standardisation, thus removing variation associated with individual clinical decision‐making.

A number of track and trigger tools exist. These are either based on exceeding any one of a set physiological criterion (single parameter systems or calling criteria) or on the allocation of points to physiological observations which are summed to give a score (aggregate weighted scoring systems including EWS, and the National Early Warning System (NEWS2) (RCP 2017)). All tools have both objective physiological criteria and subjective calling criteria. Standard practice for tracking objective physiological criteria include the monitoring of heart rate, respiratory rate, systolic blood pressure, level of consciousness, oxygen saturation and temperature, with a minimum frequency of 12‐hourly observation recording (NICE 2007; ACSQHC 2017). The subjective calling criteria allow the nurse or doctor to call for help if they have concerns regardless of physiological criteria. The type of track and trigger system selected is associated with the RRS model utilised within the hospital. Guidelines for standard practice in Australia, UK, Ireland, and the Netherlands recommend that all hospital patients should have physiological observations recorded on admission, and should have a clear written plan indicating the parameters, frequency of observations to be monitored and trigger thresholds for referral (NICE 2007; VMS 2008; DoH 2013; ACSQHC 2017).

All track and trigger tools have a predetermined physiological threshold or trigger score that assists hospital staff in recognising clinical deterioration, and aids decision‐making in terms of the type of response required. These criteria or scores provide a mechanism for early intervention and treatment initiation. A rising score indicates substantial deterioration and the need to trigger the RRS. If trigger criteria are met, or threshold value is reached, a predefined response strategy or referral protocol is initiated. A graded response strategy has been adopted in Australia, UK, and Ireland as best practice (NICE 2007; DoH 2013; ACSQHC 2017). Three levels of response (low, medium and high score groups) with maximum response times are specified in the UK (NICE 2007); a two‐tier response (ICU outreach nurses or specialist team) is used in Australia. These response levels allow actions to be tailored to the level of response required, provide a clear indication of the action and decisions to be taken to initiate the response for help, and provide standardisation of practice (NICE 2007; DoH 2013; ACSQHC 2017). Thus, appropriate and competent staff are alerted to changes in a patient's physiological condition to manage the episode of instability.

To provide an effective RRS response, all staff must be competent in assessing and instituting clinical interventions at varying levels. The provision of staff training and education in acute care ensures that staff on acute hospital wards are competent to recognise, refer and manage patient deterioration appropriately. Training and education vary across countries and correlate with the RRS model adopted. In Australia, UK and Ireland, several courses provide a flexible model of training using simulation, scenario‐based training, online and induction training modules for staff (Smith 2002; HSE 2011; ACT 2016). Training programme content is based on the principles of the Primary Survey Model training of Airway, Breathing, Circulation, Disability and Exposure (ABCDE) assessment process (Smith 2004).

A key component of the RRS is the interprofessional critical care teams, called critical care outreach teams (CCOTs) in the UK (NORF 2003), medical emergency teams (METs) in Australia (Lee 1995) (with or without ICU liaison nurses (ICULNs) (Eliott 2008), and rapid response teams (RRTs) in the US and Canada (Berwick 2006). These specialist teams respond to calls from ward staff following identification of patients at risk based upon the trigger score or calling criteria as part of the standardised escalation protocol. These teams are knowledgeable, experienced critical care staff competent in managing patient deterioration. CCOTs and ICULNs also provide follow‐up care for discharged ICU patients and provide staff training and education on aspects of critical care as an integral part of their role (DoH 2000; ICS 2002; DeVita 2006). The number of interprofessional members involved, and their availability (five days a week or 24 hours a day, seven days a week), depends on the funding model. CCOTs and ICULNs potentially avert the need for ICU admissions, enable more timely ICU discharges, and provide educational support to extend the skills of acute ward staff in identifying and managing deteriorating patients (DoH 2000; ICS 2002).

How the intervention might work

The implementation of EWS and RRS uses a systems wide approach to ensure all patients in hospital who deteriorate are managed appropriately regardless of location. All patients have their vital signs recorded regularly using track and trigger tools providing an early indicator of deterioration and the need for intervention. A rising aggregate score or meeting set physiological calling criteria triggers a graded, protocolised response by a specialist team. This ensures that the patient is managed appropriately by personnel with specified competencies and skills. The inter‐related components of the EWS and RRS ensure that patient deterioration is recognised, referred and managed commensurate with the level of need required to prevent death, unplanned ICU admission, and cardiac arrest, and to reduce length of hospital stay.

Why it is important to do this review

In 2007, we published a systematic review examining the effectiveness of EWS and RRS on patient outcomes on acute hospital wards (McGaughey 2007) based on the methods in our published protocol (McGaughey 2005). This review identified only two studies with medium risk of bias (Priestley 2004; Hillman 2005), which limited this review's conclusions.

Therefore, this update was needed to ascertain whether the implementation of EWS and RRS reduces mortality, composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death), unplanned ICU admissions, ICU readmissions, length of hospital stay or adverse events (unexpected cardiac or respiratory arrest) of deteriorating patients on acute hospital wards when compared to patients in hospital wards without an EWS and RRS. It is important to reassess the body of evidence and update this review. This is the first update of a previous Cochrane Review (McGaughey 2007).

Objectives

To determine the effect of EWS and RRS implementation on adults who deteriorate on acute hospital wards compared to people receiving hospital care without EWS and RRS in place.

Methods

Criteria for considering studies for this review

Types of studies

We considered all randomised trials, non‐randomised studies, controlled before‐after studies (CBAs) and interrupted time‐series (ITS) designs of an EWS and RRS that meet the most recent explicit inclusion and quality criteria of the Cochrane Effective Practice and Organisation of Care (EPOC) Group (EPOC 2017a). CBA studies were eligible if they had at least two intervention and two control sites; ITS studies with a clearly defined point in time when the intervention occurred, and a minimum of three data points before and after the intervention. Non‐randomised studies are typically used to evaluate the effectiveness of this type of intervention due to the difficulty in randomising acutely deteriorating patients to an EWS or RRS.

Types of participants

We considered studies enrolling all patients who experienced a deterioration in health and were on an acute adult hospital ward.

Types of interventions

We considered studies that included an intervention with both components of an EWS and RRS in an acute hospital setting to identify deteriorating patients on acute wards and compared to usual care on acute wards without an EWS and RRS. Track and trigger tools included any aggregate scoring systems (EWS, Modified Early Warning Systems (MEWS), NEWS, Patient at Risk (PAR) scores) and single parameter calling criteria as defined by the individual RRS. RRS included CCOT, MET or RRT.

Types of outcome measures

Primary outcomes

• Hospital mortality

Secondary outcomes

• Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)

• Unplanned ICU admission

• ICU readmission

• Length of hospital stay

• Adverse events (defined as an unexpected cardiac or respiratory arrest)

Search methods for identification of studies

An EPOC information specialist developed the search strategy in consultation with the review authors. The strategy was based on a citation search of two included studies from the original review and studies from a scoping search using MeSH terms and free‐text terms related to EWS and RRS. The search strategy was translated into each database using the appropriate controlled vocabulary. We applied no language restrictions.

Electronic searches

The EPOC Information Specialist peer reviewed the searches used to conduct the previous version of this review and revised the searches for this update. Searches were restricted to retrieve material published since the date of the last search for the previous version of this review (McGaughey 2007). Full search strategies are available in Appendix 1.

We searched the following databases for primary studies to 28 March 2019:

• Cochrane Central Register of Controlled Trials (CENTRAL; 2019, Issue 3) in the Cochrane Library;

• MEDLINE Ovid (including Epub Ahead of Print, in process and other Non‐Indexed Citations and Versions);

• Embase Ovid;

• CINAHL EBSCO (Cumulative Index to Nursing and Allied Health Literature).

Searching other resources

We conducted a search for references citing the included studies in this review (also on 28 March 2019) using Science Citation Index (Web of Science, Clarivate). These citation searches were not restricted by date.

We searched for ongoing studies on ClinicalTrials.gov (www.clinicaltrials.gov) and the World Health Organization International Clinical Trials Registry Platform Search Portal (ICTRP) (www.who.int/trialsearch).

We searched reference lists of all included studies and relevant reviews for additional references. Where relevant, we contacted authors of potentially eligible papers for further published and unpublished data. We contacted the following nine professional critical care organisations to identify any additional or ongoing studies: British Association of Critical Care Nurses (BACCN), World Federation of Critical Care Nurses (WFCCN), European federation of Critical Care Nurses association (EfCCNa), UK Critical Care Research Group (UKCCRG), UK Critical Care Nurses Network, European Society of Intensive Care Medicine (ESICM), UK Intensive Care Society (ICS), UK National Outreach Forum (NorF), International Liaison Committee on Resuscitation (ILCOR), and Education, Implementation & Teams (EIT) Taskforce of ILCOR.

Data collection and analysis

We followed standard methods of Cochrane for conducting a systematic review as described in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011) and adhered to EPOC methods (EPOC 2017b).

Selection of studies

Three review authors (JM and PVB or DH) independently screened the results of the database searches using Covidence to identify potentially relevant studies. Two review authors (JM and PVB) independently assessed the full‐text records of potentially relevant studies. We resolved disagreement or lack of consensus between review authors regarding study inclusion or exclusion through discussion with another review author (LR). We circulated included studies to nine professional organisations to identify any additional published or unpublished studies that met our inclusion criteria (see Searching other resources). We documented the selection process with a PRISMA flow chart (Figure 1) and produced a Characteristics of included studies table. All studies that initially appeared to meet our inclusion criteria but were subsequently excluded are detailed in the Characteristics of excluded studies table.

1.

Study flow diagram

Data extraction and management

Two review authors (JM, PVB) independently extracted data from each study using an adapted EPOC data collection tool (EPOC 2017c). We checked the data for any discrepancies and collated them. We resolved any discrepancies through discussion until review authors reached consensus. We corresponded with nine study investigators for further methodological data. Four replied providing the requested information. We extracted important information with respect to study characteristics, participant characteristics, intervention characteristics and outcome measures.

Assessment of risk of bias in included studies

Randomised trials

Two review authors (JM and PVB) independently assessed risk of bias of randomised trials using the Cochrane risk of bias tool according to the six domains outlined in the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2011): random sequence generation, allocation concealment, blinding of participants and healthcare personnel, blinding of outcome assessors, incomplete outcome data, selective reporting and other sources of bias. Risk of bias judgements were categorised as low risk of bias (judged at low risk of bias for all domains for this result), unclear risk of bias (judged at unclear risk of bias for one or more key domains for this result), and high risk of bias (judged at high risk of bias in one or more key domains for this result). We resolved discrepancies in ratings by discussion. We assessed the methodological information and entered it into the risk of bias table for each included trial.

Non‐randomised studies

Two review authors (LR and PVB) independently assessed the methodological quality of each CBA and ITS study, rather than specific outcome results,  using the eight domains of the Cochrane ROBINS‐I tool (Sterne 2016). These eight domains include bias due to confounding, bias in selection of participants into the study, bias in classification of interventions, bias due to deviations from intended interventions, bias due to missing data, bias in measurement of outcomes, bias in selection of the reported result and overall bias.

We considered the following as possible sources of confounding.

• Participant characteristics: pre‐existing comorbidity, age, severity of illness during ICU admission.

• Organisational characteristics: ICU and hospital census, staffing levels, patient acuity/case‐mix on hospital wards, ward culture and organisational practices, medical and nursing education levels; ICU discharge policies, ICU readmission policies, and overall predilection to readmit to ICU.

• ICU admission/readmission rates.

• Prevalence of patients with do not resuscitate status.

• Seasonal variation in patient census.

We considered the following co‐interventions as likely to have impact on outcomes.

• Patient safety initiatives within the organisation.

• Cardiac arrest team in place as well as the outreach/RRT.

• Education provided to staff specific to management of a deteriorating patient.

• Policy changes to ICU readmission practices.

Risk of bias judgements were categorised as 'low risk of bias' (comparable to a well‐performed randomised trial); 'moderate risk of bias' (sound evidence for a non‐randomised study but cannot be considered comparable to a well‐performed randomised trial); 'serious risk of bias' (some important problems); 'critical risk of bias' (too problematic to provide any useful evidence and should not be included in any synthesis); and 'no information' on which to base a judgement about risk of bias. We retained studies at critical risk of bias as our aim was to review all the available evidence. Review authors resolved discrepancies in certainty ratings by discussion. We assessed the methodological information and entered it into the risk of bias table for each included study, and we provided an additional overall table for all ROBINS‐I assessments.

Measures of treatment effect

Randomised trials

We report adjusted relative effects and absolute differences following standardisation across all studies for rates per 1000 admissions. We reported length of stay using hazard ratios (HR) (Priestley 2004) or adjusted ratio of geometric means (aRGM) (Jeddian 2016).

Non‐randomised studies

We reported adjusted relative effects and absolute differences where we were able to standardise studies for rates per 1000 admissions/discharge. We calculated 95% confidence intervals (CIs) when data were available.

Unit of analysis issues

Randomised trials

Unit of analysis was the cluster for all randomised controlled trials. Clusters were hospitals (Hillman 2005) or wards (Priestley 2004; Jeddian 2016; Haegdorens 2018). We examined cluster randomised controlled trials for unit of analysis errors with none detected.

Non‐randomised studies

The unit of analysis for four non‐randomised studies was individual patient data (Rothberg 2012; Aitken 2015; Davis 2015; Menon 2018) and the cluster at ward level (Lighthall 2010) or hospital level (Ludikhuize 2015; Chen 2016) for the remaining studies. We identified no unit of analysis issues with two ITS studies (Rothberg 2012; Chen 2016) or two multicentre CBA studies (Davis 2015; Ludikhuize 2015).

We reanalysed three single‐centre CBAs (Lighthall 2010; Aitken 2015; Menon 2018), which provided data at three time points before and after the implementation of the intervention using ITS methods taking into account unit of analysis issues. These studies provided data on the following: hospital mortality per 100 discharges and cardiac arrest per 1000 discharges (Lighthall 2010); unplanned ICU admissions and cardiac arrest rates mean per month (Aitken 2015); and hospital mortality and code blue per 1000 hospital admissions (Menon 2018).

To perform the ITS analyses, we received unpublished data from study authors on mortality per 100 discharges (Lighthall 2010), and code blue per 1000 hospital admissions (Menon 2018). For Aitken 2015, we used published data and the software Engauge (digitizer.sourceforge.net/) to enable ITS analyses of the outcomes of unplanned admission and cardiac arrest. Similarly, we used this method to examine cardiac arrest per 1000 discharges in Lighthall 2010. We reanalysed data using segmented regression analysis with Newey‐West autocorrelation adjusted standard errors with lag of order 1 (Newey 1987). In this analysis we found Lighthall 2010 had employed appropriate ITS analysis methods, not limited to pre–post comparisons. For Lighthall 2010 and Menon 2018, we reported standardised rates per 1000 admissions/discharges for hospital mortality and adverse events with the absolute effect or reanalysed ITS data. Data from Aitken 2015 could not be standardised as the number of admissions was not provided and, therefore, we reported author findings or reanalysed ITS data. These reanalyses did not adjust for confounding.

Dealing with missing data

We contacted authors regarding missing data via several strategies namely, the corresponding author contact email on the publication, checking institutional email addresses using Google, and via ResearchGate (ResearchGate 2017). We planned to assign studies to the Characteristics of studies awaiting classification table when there was insufficient information to determine the study design and efforts to contact the authors were unsuccessful.

Assessment of heterogeneity

The small number of included studies meant we did not examine statistical heterogeneity between trials by inspecting forest plots or quantify the impact of heterogeneity using the I² statistic (Deeks 2011). As a result, we explored possible causes qualitatively such as design, quality of studies, differences in the participants, interventions, settings and control groups. We used the TIDieR (Template for Intervention Description and Replication) checklist to compare interventions across studies  (Hoffman 2014).

Assessment of reporting biases

We were unable to construct a funnel plot for the primary outcome against trial precision (standard error) as there were insufficient trials to conduct meta‐analyses (Sterne 2011), or test formally for asymmetry.

Data synthesis

We summarised search results in a PRISMA study flow diagram (Moher 2009). We did not perform meta‐analyses due to clinical and methodological heterogeneity of included studies, and, therefore, reported data narratively. For each included cluster randomised study, we reported the published adjusted relative effects as these were appropriately adjusted for design and patient characteristics (Table 1). 

For each non‐randomised study, we considered data provided per 1000 admissions (Rothberg 2012; Ludikhuize 2015; Chen 2016; Menon 2018); for two studies, we considered data per 100 (Lighthall 2010) or 1000 discharges (Davis 2015). For Lighthall 2010, we standardised per 100 discharges to per 1000 discharges. We reported standardised rates per 1000 admissions or discharges for mortality, unplanned ICU admission and adverse events with the relative effect where data were available or as reanalysed ITS data. Where rates per 1000 admissions or discharges were not provided, we used the total number of events and the total number of admissions or discharges to calculate a rate per 1000.

In addition, we calculated the absolute effects based on the original study data and presented these unadjusted risk differences for non‐randomised studies as there were differences in effect due to lack of unadjusted data as a result of adjustment for multiple time points (Table 2). When possible (i.e. study participant numbers were available), we calculated 95% CIs for risk differences using the Newcombe and Altman method (Newcombe 2000). We were unable to standardise reanalysed ITS data or calculate absolute difference for unplanned ICU admissions and cardiac arrests in one single‐centre CBA study (Aitken 2015). If available, we reported the published relative effects for non‐randomised studies or reanalysed ITS data for three single‐centre CBA studies (Lighthall 2010; Aitken 2015; Menon 2018).

1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deterioration on acute hospital wards – absolute effects.

Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deterioration on acute hospital wards
Patient or population: adults on acute hospital wards at risk of clinical deterioration Settings: acute wards of hospital in the UK, US, Australia and the Netherlands Intervention: RRS that included an EWS Comparison: usual care without any form of EWS or RRS
Outcomes	Study	Absolute effect* (95% CI)		No of participants/admissions	Reanalysed ITS data
		Assumed risk	Corresponding risk
		Without EWS and RRS	With EWS and RRS
Hospital mortality – randomised trials (follow‐up: 3–16 months)	Priestley 2004	65.1 per 1000	49.1 per 1000	2792	—
		16.0 more participants per 1000 died without EWS (95% CI 6 to 26)			—
	Hillman 2005	1.18 per 1000	1.06 per 1000	364,094	—
		0.12 more participants per 1000 died without EWS (95% CI 0 to 0)			—
	Jeddian 2016	47.4 per 1000	35.3 per 1000	18,684	—
		12.2 more participants per 1000 died without EWS (95% CI 6 to 18)			—
	Haegdorens 2018	1.5 per 1000	0.7 per 1000	69,656	—
		0.8 more participants per 1000 died without EWS (95% CI 0 to 1)			—
Hospital mortality – non‐randomised studies (follow‐up: 5–48 months)	Lighthall 2010	27.1 per 1000	22.4 per 1000		0.19 (−0.83 to 1.22)
		—
	Rothberg 2012	22 per 1000	22 per 1000
		—
	Davis 2015	0.21 per 1000	0.17 per 1000	134,093	—
		0.04 more participants per 1000 died without EWS (95% CI −0.1 to 0.3)
	Ludikhuize 2015	20.4 per 1000	17.7 per 1000	57,858
		2.7 more participants per 1000 died without EWS (95% 0.5 to 4.9)
	Chen 2016	18 per 1000	14 per 1000	—	—
		3.4 more participants per 1000 died without EWS (P < 0.001)
	Menon 2018	4.9 per 1000	2.4 per 1000	18,954	—
		2.5 more participants per 1000 died without EWS (95% CI 0.7 to 4.6)
Composite outcome – randomised trials Composite outcome of incidence (events divided by number of eligible participants admitted to the hospital during the study period) of cardiac arrests without a pre‐existing NFR order, unplanned ICU admissions and unexpected deaths (deaths without a pre‐existing NFR order) (follow‐up: 6 months)	Hillman 2005	5.9 per 1000	5.3 per 1000	364,094	—
		0.6 more participants per 1000 experienced the composite outcome without EWS (95% CI −0.3 to 1.4)
Composite outcome – non‐randomised studies Composite endpoint of cardiopulmonary arrest, unplanned ICU admission or death (follow‐up: 5 months)	Ludikhuize 2015	37.1 per 1000	32.9 per 1000	57,858	—
		4.2 more participants per 1000 experienced the composite outcome without EWS (95% CI 1.2 to 7.2)
Unplanned ICU admission – randomised trials (follow‐up: 3–16 months)	Hillman 2005	4.7 per 1000	4.2 per 1000	364,094
		0.5 more participants per 1000 unplanned ICU admission without EWS (95% CI −0.2 to 1.2)
	Jeddian 2016	12.3 per 1000	12.8 per 1000	18,684	—
		0.5 fewer participants per 1000 experienced unplanned ICU admission without EWS (95% CI −2.9 to 3.7)
	Haegdorens 2018	6.5 per 1000	10.3 per 1000	69,656	—
		3.8 fewer participants per 1000 experienced unplanned ICU admission without EWS (95% CI 2.3 to 5.0)
Unplanned ICU admission – non‐randomised studies (follow‐up: 5 months)	Aitken 2015	NR	NR	—	−6.52 (−15.24 to 2.20)
		—
	Ludikhuize 2015	19.8 per 1000	17.1 per 1000	57,858	—
		2.6 more participants per 1000 experienced unplanned ICU admission without EWS (95% CI 0.5 to 4.9)
ICU readmission – randomised trials	No randomised trials reported on the rates of unplanned ICU readmission.
ICU readmission  – non‐randomised studies	No non‐randomised studies reported on the rates of unplanned ICU readmission.
Length of hospital stay – randomised trials	No randomised trials reported length of hospital stay.
Length of hospital stay – non‐randomised studies (follow‐up: 3 months)	Priestley 2004	NR	NR	2733	—
		—
	Jeddian 2016	Median 6 days (IQR 3 to 10)	Median 4 days (IQR 2 to 8)	18,684	—
		Length of stay was reduced by 2 days with EWS
Adverse events (unexpected cardiac or respiratory arrest) – randomised trials (follow‐up: 3–16 months)	Hillman 2005	1.6 per 1000	1.3 per 1000	364,094	—
		0.3 more participants per 1000 experienced an adverse event without EWS (95% CI −0.1 to 0.8)			—
	Jeddian 2016	36.1 per 1000	48.6 per 1000	18,684	—
		12.5 fewer participants per 1000 experienced an adverse event without EWS (95% CI 9.5 to 15.5)			—
	Haegdorens 2018	1.3 per 1000	1.0 per 1000	69,656	—
		0.3 more participants per 1000 experienced an adverse event without EWS (95% CI −0.7 to 1.4)			—
Adverse events (unexpected cardiac or respiratory arrest) – non‐randomised studies (follow‐up: 5–48 months)	Lighthall 2010	10.1 per 1000	4.4 per 1000	—	20.3 (−18.56 to 59.19)
		5.7 more participants per 1000 experienced an adverse event without EWS
	Rothberg 2012	4.7 per 1000	3.1 per 1000	—	—
		1.6 more participants per 1000 experienced an adverse event without EWS
	Aitken 2015	NR	NR	—	1.29 (−8.49 to 5.91)
		—
	Davis 2015	2.7 per 1000	1.1 per 1000	134,093	—
		1.6 more participants per 1000 experienced an adverse event without EWS (95% CI 1.1 to 2.2)
	Ludikhuize 2015	1.9 per 1000	1.2 per 1000	57,858	—
		0.7 more participants per 1000 experienced an adverse event without EWS (95% CI 0.1 to 1.4)
	Chen 2016	2.4 per 1000	1.3 per 1000	—	—
		1.1 more participants per 1000 experienced an adverse event without EWS
	Menon 2018	6.8 per 1000	31 per 1000	18,954	3.44 (−10.35 to 17.23)
		3.7 more participants per 1000 experienced an adverse event without EWS (95% CI 1.5 to 6.0)
*The risk WITHOUT the intervention is based on the USUAL CARE control group risk IN EACH STUDY. The corresponding risk WITH the intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence interval).

CI: confidence interval; EWS: early warning system; ICU: intensive care unit; IQR: interquartile range; NFR: not‐for‐resuscitation; NR: not reported; RRS: rapid response system.

Subgroup analysis and investigation of heterogeneity

The small number and heterogeneity of included studies meant it was not possible to conduct subgroup analyses of nurse‐led outreach versus physician‐led RRS, or by study design (cluster randomised trial, parallel group randomised trial, CBA and ITS studies).

Sensitivity analysis

Given the small number of included studies, a sensitivity analysis to explore the impact of methodological quality was not possible.

Summary of findings and assessment of the certainty of the evidence

We created a summary of findings table using the GRADE approach (Guyatt 2008), and following specific guidance developed by EPOC (EPOC 2017d). We included in the summary of findings table the outcomes of hospital mortality, composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death), unplanned ICU admissions, ICU readmissions, length of hospital stay and adverse events (unexpected cardiac arrest or respiratory arrest). Previous systematic reviews combined data from randomised trials and non‐randomised studies (Chan 2010; Winters 2013; Maharaj 2015; de Jong 2016; Solomon 2016) however, we found no manner of easily interpretable data presentation.

To assess the certainty of evidence for each outcome, we used five GRADE considerations (risk of bias, inconsistency, indirectness, imprecision and publication bias). Two review authors (JM, LR) independently assessed the certainty of evidence (high, moderate, low or very low) following completion of the 'Calculation of GRADE ratings worksheet' (Table 3). We considered evidence from randomised trials as high certainty but downgraded the evidence one level for serious (or two levels for very serious) limitations. Justification for decisions to downgrade ratings are provided in the footnotes of Table 1.

2. Calculation of GRADE ratings worksheet.

Number of studies	Design	Risk of bias	Inconsistency	Indirectness	Imprecision	Publication bias	Certainty‡ (overall score)
Hospital mortality – randomised trials
Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens 2018	4 RCT (4)	Serious (−1)	Not serious	Not serious	Serious (−1)	Not serious	⊕⊕⊝⊝ Lowa,b
Hospital mortality – non‐randomised studies
Lighthall 2010; Rothberg 2012; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018	4 CBA 2 ITS (3)	Serious (−1)	Serious (−1)	Not serious	Serious (−1)	Not serious	⊕⊝⊝⊝ Very lowb,c,d
Composite outcome – randomised trials
Hillman 2005	1 RCT (4)	Serious (−1)	None	Not serious	None	Not serious	⊕⊕⊕⊝ Moderatea
Composite outcome – non‐randomised studies
Ludikhuize 2015	1 CBA (3)	Serious (−1)	None	Not serious	None	Not serious	⊕⊕⊝⊝ Lowc
Unplanned ICU admission – randomised trials
Hillman 2005; Jeddian 2016; Haegdorens 2018	3 RCT (4)	Serious (−1)	Not serious	Not serious	Serious (−1)	Not serious	⊕⊕⊝⊝ Lowa,b
Unplanned ICU admission – non‐randomised studies
Aitken 2015; Ludikhuize 2015	CBA reanalysed as ITS 1 ITS (3)	Serious (−1)	Serious (−1)	Not serious	Serious (−1)	Not serious	⊕⊝⊝⊝ Very lowb,c,d
ICU readmission – randomised trials	Not reported
ICU readmission – non‐randomised studies	Not reported
Length of hospital stay – randomised trials
Priestley 2004; Jeddian 2016	2 RCT (4)	Serious (−1)	Serious (−1)	Not serious	Not serious	Not serious	⊕⊕⊝⊝ Lowa,d
Length of hospital stay – non‐randomised studies	Not reported
Adverse events – randomised trials
Hillman 2005; Jeddian 2016; Haegdorens 2018	3 RCTs (4)	Serious (−1)	Not serious	Not serious	Serious (−1)	Not serious	⊕⊕⊝⊝ Lowa,b
Adverse events – non‐randomised studies
Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018	5 CBA 2 ITS (3)	Serious (−1)	Serious (−1)	Not serious	Serious (−1)	Not serious	⊕⊝⊝⊝ Very lowb,c,d
‡GRADE Working Group grades of evidence High: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low. Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate. Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high. Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high. ‡Substantially different = a large enough difference that it might affect a decision.
CBA: controlled before‐after study; ICU: intensive care unit; ITS: interrupted time series; RCT: randomised controlled trial.

^aDowngraded one level for risk of bias: concerns regarding performance bias, contamination bias, or both.
^bDowngraded one level for imprecision of the direction of the effect: due to wide risk ratio interpreted as less than 0.75 to greater than 1.25, which includes appreciable beneficial or harmful effects.
^cDowngraded one level for risk of bias: serious or critical risk of bias due to inadequate adjustment for confounders in observational studies assessed with the Risk Of Bias in Non‐randomised Studies – of Interventions (ROBINS‐I) tool.
^dDowngraded one level for inconsistency: as unable to calculate effect size for all studies.

For imprecision, we downgraded certainty of evidence one level due to wide 95% CIs defined as less than 0.50 to more than 1.10 (Higgins 2020). For inconsistency, we downgraded certainty of evidence one level when we were unable to determine or calculate the effect size of the outcome for all studies.

For non‐randomised studies, we downgraded all studies one level due to risk of bias associated with no randomisation. We further downgraded certainty of evidence one level for serious or critical risk of bias due to confounding as assessed using the ROBINS‐I tool (Table 4). The ROBINS‐I tool was designed to be applied to individual results, however in the absence of any meta‐analysis we applied it to each study as a whole. Prior to future updates we will reconsider this aspect of the protocol and look to apply the tool as it was designed to specific results from each study.

3. Quality assessment judgements (ROBINS‐I) of non‐randomised studies.

Lighthall 2010
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Serious risk	Comment: serious but not critical risk due to use of statistical methods to control for some, but not all, baseline confounders.
D2: selection bias	Low risk	Comment: no evidence of selective recruitment.
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from protocol not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Comment: outcomes were objective and not subject to interpretation.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting.
Overall risk of bias	Serious risk	Comment: due to serious risk of confounding and unclear risk of protocol deviation.
Rothberg 2012
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Serious risk	Comment: serious but not critical risk due to use of statistical methods to control for some, but not all, baseline confounders.
D2: selection bias	Low risk	Comment: no evidence of selective recruitment.
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from protocol not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Comment: outcomes were objective and not subject to interpretation.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting.
Overall risk of bias	Serious risk	Comment: serious but not critical risk due to use of statistical methods to control for some, but not all, baseline confounders.
Aitken 2015
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Critical risk	Comment: there was no adjustment for patient or organisational characteristics. No baseline confounders were measured, or analyses performed for confounding. Authors used process control methods, which helps to at least assess for time‐varying confounding but there are no analysis methods that control for confounding.
D2: selection bias	Low risk	Comment: participant characteristics were not used as selection criteria after the intervention. No evidence of selective recruitment.
D3: bias in classification of interventions	Low risk	Comment: interventions were defined in terms of time of exposure – either before or after the intervention was implemented.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from intended intervention not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Comment: measured outcomes including cardiac arrest, death and unplanned ICU admission were all objective and not subject to measurement bias.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective outcome reporting.
Overall risk of bias	Critical risk
Davis 2015
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Critical risk	Comment: there was no attempt to control for patient‐associated confounders such as acuity, comorbidity or age. Changes in ICU admission/readmission rates could have confounded results, i.e. fewer non‐ICU cardiac arrest because more early ICU admissions (non‐RRT initiated); however, this was not reported on or adjusted for. No evidence of analyses controlling for confounding. There were no postintervention variables that need to be controlled for.
D2: selection bias	Low risk	Comment: no evidence of selective recruitment.
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from intended intervention not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Comment: cardiac arrest rates and mortality were objective outcomes and were not subject to measurement bias.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting.
Overall risk of bias	Critical risk	Comment: Rated as critical due to high risk if confounding bias and unclear risk of bias due to no reporting of deviation from protocol.
Ludikhuize 2015
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Serious risk	Comment: serious but not critical risk due to use of statistical methods to control for some, but not all, baseline confounders.
D2: selection bias	Low risk	Comment: no evidence of selective recruitment
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from protocol not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Comment: outcomes were objective and not subject to interpretation.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting.
Overall risk of bias	Serious risk	Comment: serious but not critical risk due to use of statistical methods to control for some, but not all, baseline confounders.
Chen 2016
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Serious risk	Comment: evaluated changes in baseline characteristics by calendar year to account for hospital cluster effect and assessed intervention effect using regression to measure monthly trends. Analysis adjusted for year, age, sex, marital status, country of birth, socioeconomic status, hospital (urban vs rural), health insurance and major hospital peer groups. There were no postintervention variables that need to be controlled for.
D2: selection bias	Low risk	Comment: population‐based study that used NSW APDC database to obtain data on all admissions to 232 hospitals between 2007 and 2013.
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from intended intervention not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Quote: "Patient outcomes and other related variables were derived from the NSW APDC database, which includes demographic and diagnostic information for each public and private hospital admission episode". Comment: databases coded hospital mortality and cardiac arrest rates as objective outcomes and were not subject to measurement bias.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting.
Overall risk of bias	Serious risk	Comment: rated as serious due to high risk of confounding bias due to use of statistical methods to control for some, but not all, baseline confounders.
Menon 2018
Bias	Authors' judgement	Support for judgement
D1: confounding bias	Critical risk	Comment: there was no attempt to control for patient‐associated confounders such as acuity, comorbidity, age or time variation. No evidence of analyses controlling for confounding. There were no postintervention variables that need to be controlled for.
D2: selection bias	Low risk	Comment: no evidence of selective recruitment.
D3: bias in classification of interventions	Low risk	Comment: intervention status was clearly defined in terms of time of exposure, i.e. admission to hospital before or after the RRS.
D4: bias due to deviation from intended intervention	No information	Comment: deviations from intended intervention not reported.
D5: bias due to missing data	Low risk	Comment: no evidence of missing data.
D6: bias in measurement of outcomes	Low risk	Code Blue events recorded before and after MET. Comment: Code Blue rates were objective outcomes and were not subject to measurement bias.
D7: bias in selection of reported outcomes	Low risk	Comment: no evidence of selective reporting. Deviations from protocol not reported.
Overall risk of bias	Critical risk	Comment: rated as critical due to high risk of confounding bias due lack of statistical methods to control for baseline confounders.

ICU: intensive care unit; MET: medical emergency team; NSW APDC: New South Wales Admitted Patient Data Collection; ROBINS‐I: Risk Of Bias in Non‐randomised Studies – of Interventions; RRS: rapid response system.

Results

Description of studies

We identified 11 studies comprising four randomised trials and seven non‐randomised studies.

Results of the search

Figure 1 shows the PRISMA flow diagram for screening, selection and assessment of studies.

We identified 5370 studies from all possible sources following the removal of duplicates and excluded 5262 studies based on review of title and abstract. We reviewed 108 full‐text articles and identified nine new studies  meeting our inclusion criteria (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Jeddian 2016; Haegdorens 2018; Menon 2018). We previously identified two studies in our original review (Priestley 2004; Hillman 2005) (see Characteristics of included studies table).

Included studies

Of the 11 studies meeting review inclusion criteria, two were randomised trials included in the previously published review (Priestley 2004; Hillman 2005), two were new randomised trials (Jeddian 2016; Haegdorens 2018), and seven were non‐randomised studies (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018). We obtained missing data  from authors on  mortality  (Davis 2015), study participant numbers (Hillman 2005), unplanned ICU arrest incidence (Lighthall 2010; Menon 2018), and further additional information from published supplementary files (Chen 2016; Davis 2015; Ludikhuize 2015; Jeddian 2016). Details of are provided in the Characteristics of included studies table, Table 5, and are briefly summarised below. Table 6 shows the comparison of interventions using the TIDieR checklist. Table 7 provides data on the clinical heterogeneity of the type of EWS and RRS.

4. Summary of included study characteristics.

Study ID	Design		Country	Participants (before)	Participants (after)	Hospital	Intervention	Before	After	Timeframe	Outcomes
Randomised trials
Priestley 2004	Prospective step wedge RCT		UK	3,391	3,090	800‐bedded general hospital (Dataset 2: 2903 participants)	PAR score, education, 24/7 CCOT	4 weeks	28 weeks	32 weeks	Primary: in‐hospital deaths, LOS
Hillman 2005	Cluster RCT		Australia	68,376	56,756	23 public hospitals (total 125,132 participants)	Calling criteria, education, MET	2 months	6 months	12 months	Primary: composite outcome (CAs, unplanned ICU admissions, unexpected deaths) Secondary outcome: CAs, unplanned ICU admissions, unexpected deaths
Jeddian 2016	Step wedge cluster RCT		Iran	10,880	7,802	1 university and public teaching hospital (total 18,682 participants)	Calling criteria, education CCOT	12 weeks	12 weeks	July 2010–December 2011	Primary: in‐hospital mortality, CAs Secondary: LOS, ICU admissions
Haegdorens 2018	Step wedge cluster RCT		Belgium	35,389	34,267	7 hospitals (total 69,656 participants)	NEWS, SBAR, training, 24/7 MET	4 months	16 months	October 2013–May 2015	Primary: unexpected deaths, CAs, unplanned ICU admissions Secondary: total ward mortality, ward mortality without NFR code, hospital mortality
Non‐randomised studies
Lighthall 2010		Single‐centre CBA, reanalysed ITS	USA	NR	NR	Tertiary medical centre 240 beds (total number of participants NR)	Calling criteria, education, high capability MET	9 months (CA), 42 months (mortality)	27 months	June 2005–September 2007	Hospital mortality, CAs
Rothberg 2012		ITS	USA	NR	NR	670‐bed tertiary care referral centre (total number of participants NR)	Calling criteria, education, MET	24 months (January 2004–December 2006)	41 months (July 2006–December 2009)	January 2004–December 2009	Hospital mortality, CAs
Aitken 2015		Quasi‐experimental, pretest–post‐test single centre CBA, reanalysed ITS	Australia	NR	NR	750 bed tertiary hospital (total number of participants NR)	MEC calling criteria, education, 2‐tier RRS (ICUON, 24 hours per day/7 days per week RRT)	20 months (ICU admission and CA); prevalence: 1 day prior to implementation	48 months (ICU admission and CAs); Prevalence: 8 months following	January 2008–October 2013 (ICU admission and CAs)	Unplanned ICU admissions, CAs, prevalence of deteriorating patients, staff satisfaction
Davis 2015		2‐centre CBA	USA	40,806	93,287	Primary medical centre (392 beds) and sister campus (119 beds) (total 134,093 participants)	Calling criteria, training, RRT	28 months	48 months	July 2005–June 2011	CAs (non‐ICU), hospital mortality
Ludikhuize 2015		Prospective, pragmatic before‐after multicentre study	Netherlands	28,298	29,560	2 large university hospitals (882– 1000 beds), 8 teaching hospitals (359–1070 beds) and 2 regional hospitals (290–325 beds) (total 57,858 participants)	MEWS, SBAR, education, RRT	5 months	5 months	April/July 2009–August/November 2011	Primary: composite outcome (CAs, unplanned ICU admission, death) Secondary: CAs, unplanned ICU admission, death
Chen 2016		ITS	Australia	2,602,351	3,033,596	232 hospitals in NSW (total 9,799,081 participants pre, run‐in, post)	Calling criteria, education, RRT	3 years	3 years	January 2007–December 2013	Primary: IHCA rate, IHCA‐related mortality rate, hospital mortality rate, FTR, DLMDRG, 1‐year postdischarge mortality
Menon 2018		Prospective single‐centre CBA, reanalysed ITS	India	7495	11,459	226‐bed tertiary academic hospital (total 18,954 participants)	AWC calling criteria, education, MET	12 months	24 months	April 2013–March 2016	Primary: CAs (non‐ICU), mortality

AWC: Amrita Warning Criteria; CAs: cardiac arrests; CBA: controlled before‐after study; CCOT: Critical Care Outreach Team; CCU: critical care unit; FTR: failure‐to‐rescue; DLMDRG: deaths in low mortality diagnostic related groups; NFR: not‐for‐resuscitation; ED: emergency department; EWS: early warning system; ICU: intensive care unit; ICUON: intensive care unit outreach nurse;  IHCA: in‐hospital cardiac arrest; ITS: interrupted time series; LOS: length of stay; MEC: medical emergency calling criteria; MET: medical emergency team; MEWS: Modified Early Warning Score; NEWS: National Early Warning Score; NR: not reported; NSW: New South Wales; PAR: Patient At Risk score; RCT: randomised controlled trial; RRS: rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Response communication tool.

5. TIDieR framework of intervention description.

Study ID

EWS

Criteria

Type of RRS

Availability

RRS process

Training

Duration

Content

Trainer/educator

Mode

Location

Intervention modified

Reminders

Organisational structures

Arrest team

Randomised trials

Priestley 2004

PAR score

Objective criteria and concern

CCOT

24 hours/7 days

PAR referral to CCOT and admitting consultant, graded response strategy, support and advise staff or transfer

Training for all ward nurses and doctors, emphasis on sharing skills and collaboration

4 weeks

Management of critically ill patients and PAR score

CCOT

NR

NR

NR

NR

Obtained funding for CCOT with requirement for operationalisation by April 2001

NR

Hillman 2005

Calling criteria

Objective criteria and concern

MET

NR

Staff education and reminders regarding calling criteria, standardised implementation strategy

Standardised education strategy for all nursing and medical staff

4 months before implementation

Calling criteria, importance of criteria to identify patients at risk, need to call quickly, how to call MET

NR

Lectures, MET videotape, booklets

NR

NR

Identification badges and posters with list of calling criteria

Management, ethics and resuscitation committee approval obtained from control hospitals

Arrest team maintained in control hospitals and MET equivalent to pre‐existing arrest team

Jeddian 2016

Calling criteria

Objective criteria and concern

CCOT

NR

Patient showing no improvement after 30 minutes were referred to team, CCOT assessed using composite scoring system and managed or transferred

Training for ward nurses

8 weeks

Assessment, identification, and management of acutely ill patients

NR

NR

NR

NR

NR

Overseen by committee

NR

Haegdorens 2018

NEWS and SBAR

Objective criteria and concern

MET

24 hours/7 days

Team‐directed implementation strategy, graded response protocol, maximum waiting times, contact numbers, backup procedures

Training for ward nurses

4 hours

Measurement and interpretation of vital signs, clinical observation, communication skills, and practical tips and tricks in handling NEWS and SBAR

2 experienced practising nurses

Interactive training

NR

NR

Website, knowledge test for nurses

Project manager per hospital to ensure uniformity

NR

Non‐randomised studies

Lighthall 2010

Calling criteria

Objective criteria and concern

Physician‐led, high capability MET

NR

eTeam paged and primary care team summoned

In‐service education for nurses, medical and surgical departments

4 months before operation

NR

NR

NR

NR

NR

Posters and identification badge holders with calling criteria

Steering committee of major academic and clinical service departments established criteria, team composition and operation; eTeam steering and resuscitation committee meetings including physician and nurse leaders to discuss event rates, provider conduct, efficacy and patient outcomes

Arrest team in place at time

Rothberg 2012

Calling criteria

Objective criteria and concern

MET

24‐hour coverage by hospitalists

MET activation via pager

Hospital‐wide education of unit staff, all levels of management and physicians

NR

Purpose of team and how to activate the team

NR

Meetings, email, communication boards, posters

NR

Team composition and protocol adjustments using PDSA cycles

Emails, posters, pocket cards, activation number on all ward telephones

MET implemented in accordance with IHI recommendations, oversight committee met biweekly

Separate 'code' team

Aitken 2015

MEC

Objective criteria and concern

2‐tier RRS (ICUON, RRT)

24 hours/7 days (ICUON provided 7–11 p.m. service, RRT 8–5 p.m., out‐of‐hours service 11 p.m.–8 a.m.)

ICUON activated via direct call for concern, RRT via switchboard and pager system for physiological criteria

Extensive in‐service training for CCU, ED, ward nurses, consultants, registrars

NR

Tailored sessions for various stakeholders

NR

NR

NR

NR

Posters, lanyards and screen savers

Project plan developed, physical resources organised, policies revised; evaluation and meetings with stakeholder groups

Replaced MEC system and arrest team

Davis 2015

Calling criteria

Objective criteria and nurse or family concern

RRT

NR

Charge nurse conducts 'rounds' each shift, algorithm for arrest and non‐arrest, RRT activation criteria and RRT nurse responds with code blue physician lead available

RRT education on patient surveillance and recognition of patient deterioration

Several hours as part of annual training

Patient surveillance, resuscitation training, recognition of deterioration

Critical care physicians and RRT/Code Blue nurses

Conceptual model

NR

Frequently modified based on institutional CQI data

NR

Hospital‐wide advanced resuscitation training curriculum based on 5 key components

NR

Ludikhuize 2015

MEWS and SBAR

Objective criteria and concern

RRT

NR

MEWS > 3, Call physician using SBAR, Physician responds within 30 minutes, no effect to therapy 60 minutes, call RRT

Education to each carer

EWS and SBAR 7 months before implementation

Trained using standardised tool kits

NR

NR

NR

NR

Plasticised handheld cards, posters, in patient charts, feedback session and face‐to‐face communication

No details

NR

Chen 2016

Calling criteria

Objective criteria and nurse or family concern

RRT

NR

2‐tier response strategy (yellow red red), response strategy with minimum skills and minimum response times

Education programme aimed at all staff

NR

BTF programme

NR

NR

NR

NR

NR

State‐wide government led 5 component strategy: EWS, RRS, education, governance structure and minimum dataset

NR

Menon 2018

AWC

Objective criteria

MET

NR

Dedicated telephone number, activate via hospital intercom system, pager to MET member, respond × 15 mins

Education and training sessions

1 month

NR

Senior physicians and nurse administrators

NR

NR

NR

Concise pocket MET card, workflow handouts

Expert review by clinicians and senior administrators developed calling criteria (AWC) and MET process

Not stated. In place prior to implementation

AWC: Amrita Warning Criteria; BTF: Between the Flags; CCOT: critical care outreach team; CCU: coronary care unit; CQI: continuous quality improvement; ED: emergency department; EWS: Early Warning Score; ICUON: intensive care unit outreach nurse; IHI: Institute for Health Improvement; MEC: Medical Emergency Calling criteria; MET: medical emergency team; MEWS: Modified Early Warning Score; min: minute; NEWS: National Early Warning Score; NR: not reported; PAR: Patient at Risk score; PDSA: Plan, Do, Study, Act cycle; resus: resuscitation; RCT: randomised controlled trial; RRS: rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Response communication tool.

6. Clinical heterogeneity.

Study ID	Design	EWS trigger mechanism	Type of RRS	Composition
Randomised trials
Priestley 2004	Prospective step wedge cluster RCT	Aggregated weighted score	Nurse‐led CCOT	Led by a nurse consultant with a team of nurses supported by physicians when required.
Hillman 2005	Cluster RCT	Single parameter criteria	Medical‐led MET	Varied across participating hospitals with study protocol requiring at least the equivalent of pre‐existing cardiac arrest team consisting of a physician, and either an ICU or ED nurse.
Jeddian 2016	Step wedge cluster RCT	Single parameter criteria	Nurse‐led CCOT	6 experienced ICU nurses.
Haegdorens 2018	Step wedge cluster RCT	Aggregated weighted score	Medical‐led MET	Physician and nurse.
Non‐randomised studies
Lighthall 2010	Single‐centre controlled before‐after	Single parameter criteria	Medical‐led MET	A high capability, physician led e‐Team.
Rothberg 2012	ITS	Single parameter criteria	Medical‐led MET	An ICU nurse, respiratory therapist, intravenous therapist, and the patient's physician, surgeon or resident.
Aitken 2015	Quasi‐experimental, single‐centre pretest–post‐test	Single parameter criteria	Nurse‐led RRT	ICU outreach nurse and interprofessional team with an after‐hours team comprising medical resident, ICU junior registrar, ICUON, and a CCU or ED nurse.
Davis 2015	2‐centre controlled before‐after	Single parameter criteria	Nurse‐led RRT	A dedicated ICU nurse, respiratory therapist and charge nurse on each inpatient unit.
Ludikhuize 2015	Prospective, pragmatic controlled before‐after multicentre study	Aggregated weighted score	Medical‐led RRT	An ICU nurse and a physician.
Chen 2016	ITS	Single parameter criteria	Medical‐led RRT	ICU clinicians with advanced life support skills.
Menon 2018	Prospective, single‐centre controlled before‐after	Single parameter criteria	Medical‐led MET	Physician‐led team with charge nurse, nursing supervisor, phlebotomist, respiratory therapist and ECG technician.

CCOT: critical care outreach team; CCU: coronary care unit; ECG: electrocardiogram; ED: emergency department; EWS: early warning system; ICU: intensive care unit; ICUON: intensive care unit outreach nurse;  ITS: interrupted time series; MET: medical emergency team; RCT: randomised controlled trial; RRS: rapid response system; RRT: rapid response team.

Three randomised trials (Priestley 2004; Hillman 2005; Haegdorens 2018), and two non‐randomised studies (Aitken 2015; Chen 2016), reported funding. All were investigator‐led academic or government grants.

Randomised trials

Study design

We included one prospective cluster randomised trial (with the clusters at hospital level) conducted over 12 months in general inpatient wards in 23 Australian hospitals (Hillman 2005). This trial adjusted for individual (age/sex) and cluster (bed number/hospital) characteristics and reported an intraclass cluster coefficient (ICC) for primary and secondary outcomes. We included three prospective stepped‐wedge cluster randomised trials. These trials introduced a CCOT in 16 acute general wards in one UK hospital (Priestley 2004); a CCOT across 13 general wards in one Iranian hospital (Jeddian 2016); and a MET across 28 medical or surgical wards in seven Belgian hospitals (Haegdorens 2018).

All prospective stepped‐wedge cluster randomised trials adjusted for cluster (ward/bed number) and time trends. Haegdorens 2018 applied generalised linear mixed models (GLMM) as cluster sizes varied. Jeddian 2016 used latent ICC.

Controls in these four trials comprised randomly assigned hospitals that did not receive MET education at any stage (Hillman 2005); and wards randomised to sequential roll out of the EWS and RRS intervention with baseline, intervention and postintervention phases at different time points (Priestley 2004; Jeddian 2016; Haegdorens 2018).

Participants

Two studies did not report age inclusion criteria (Priestley 2004; Jeddian 2016). The remaining studies excluded participants aged under 14 years (Hillman 2005) or under 17 years (Haegdorens 2018) admitted to participating wards. The mean age of participants ranged from 43 to 65 years across pre‐intervention and postintervention groups with the percentage of males ranging from 39% to 54%. In the four randomised trials, allocation of participants to the intervention or control groups was based on: single‐centre admission ward (Priestley 2004 with 16 wards; Jeddian 2016 with 13 wards); across 28 wards of seven hospitals (Haegdorens 2018); or using matched hospitals (Hillman 2005 with 23 hospitals).

Sample sizes of randomised trials ranged from 2733 (Priestley 2004) to 364,094 (Hillman 2005) participants.

Settings

Studies were conducted in the UK (Priestley 2004), Australia (Hillman 2005), Belgium (Haegdorens 2018), and Iran (Jeddian 2016). Therefore, all studies were set in middle‐ to high‐income countries.

Two studies were single‐centre general (Priestley 2004) or university‐affiliated (Jeddian 2016) hospitals. Haegdorens 2018 included seven acute care hospitals with at least 850 admissions per year. Hillman 2005 included 23 public hospitals with greater than 20,000 admissions per year (11 control hospitals, median bed number 315, interquartile range (IQR) 229 to 400) and 12 MET hospitals (median bed number 364, IQR 182 to 457).

Hospital wards included general wards (Hillman 2005; Jeddian 2016), medical and surgical wards (Priestley 2004; Haegdorens 2018), and elderly medicine wards (Priestley 2004). One study excluded patient data from emergency departments (EDs), ICUs, ICU‐supervised high dependency units, operating theatres and postoperative recovery (Hillman 2005). Exclusion criteria were either not stated (Priestley 2004; Haegdorens 2018), or there were no patient exclusion criteria (Jeddian 2016).

Interventions

EWS interventions comprised the PAR score (Priestley 2004), NEWS and SBAR (Situation, Background, Assessment, Recommendation) communication tool (Haegdorens 2018), or specific calling criteria (Hillman 2005; Jeddian 2016). PAR score is a multiple parameter scoring system, NEWS is an aggregate‐weighted scoring system, and calling criteria are single parameter systems for a response to patient deterioration.

RRS interventions included a CCOT (Priestley 2004; Jeddian 2016), or a MET (Hillman 2005; Haegdorens 2018). Two studies introduced the CCOT/MET on a 24 hour, seven day per week basis (Priestley 2004; Haegdorens 2018). Two studies did not report team availability (Hillman 2005; Jeddian 2016) (Table 6).

Team composition varied across the four trials. One study included a CCOT led by a nurse consultant with a team of nurses supported by physicians when required (Priestley 2004); one study used a nurse‐led CCOT consisting of six experienced ICU nurses (Jeddian 2016); two studies used a medical‐led MET comprising a physician, and either an ICU or ED nurse (Hillman 2005); or physician and a nurse (Haegdorens 2018). At the time of RRT implementation, two studies indicated the cardiac arrest team was retained (Hillman 2005; Haegdorens 2018), two studies did not report this characteristic (Priestley 2004; Jeddian 2016).

All randomised trials compared implementation of the EWS and RRS intervention with control hospitals or wards that did not have EWS and RRS. All studies used an educational strategy to introduce the RRS intervention with the training implementation phase varying from four hours (Haegdorens 2018), one month (Priestley 2004), two months (Jeddian 2016), and four months (Hillman 2005). Training consisted of in‐service educational programmes on recognition of patient deterioration (Priestley 2004; Jeddian 2016), and EWS or calling criteria (Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens 2018). There was training for ward nurses (Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens 2018) and doctors (Priestley 2004; Hillman 2005). Hillman 2005 indicated education focused on when and how to call the outreach team and not on the treatment of sick patients. Delivery of the education was undertaken using lectures, video tapes and booklets (Hillman 2005), or via interactive training (Haegdorens 2018). Two studies emphasised sharing skills between teams and ward staff (Priestley 2004; Jeddian 2016). To reinforce implementation, two studies employed reminders in the form of posters (Hillman 2005; Haegdorens 2018), identification badges (Hillman 2005), champions (Haegdorens 2018), or made resources available via a study website including a knowledge test for nurses (Haegdorens 2018).

Outcomes

The unit of analysis was the cluster at the hospital level for one study (Hillman 2005), and at the ward level for remaining studies. The baseline data collection period was one to four months with postintervention implementation data collected at three to 16 months (Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens 2018).

Selection of outcomes varied across studies. Primary outcomes included rate of hospital deaths (Priestley 2004; Jeddian 2016) or unexpected deaths (Haegdorens 2018); composite outcome of cardiac arrests without a pre‐existing not‐for‐resuscitation (NFR) order, unplanned ICU admissions, and unexpected deaths (deaths without a pre‐existing NFR order) (Hillman 2005); rate of unplanned ICU admissions (Haegdorens 2018); length of hospital stay (Priestley 2004); and rate of cardiac arrests (Jeddian 2016; Haegdorens 2018). Two studies reported individual components of their composite endpoint (cardiac arrests, unplanned ICU admissions and unexpected deaths) (Hillman 2005) or length of stay and ICU admission (Jeddian 2016) as secondary outcomes.

Non‐randomised studies

Study design

We included seven non‐randomised studies: three single‐centre prospective CBAs  analysed as ITS (Lighthall 2010; Aitken 2015; Menon 2018); one two‐centre CBA (Davis 2015); one prospective, multicentre CBA (Ludikhuize 2015); and two ITS studies (Rothberg 2012; Chen 2016). For the single‐centre studies, we used the data provided per 1000 admissions (Menon 2018), standardised published data to per 1000 discharges when the study had employed appropriate ITS analysis (Lighthall 2010), or reported reanalysed ITS results for absolute difference when reanalysed ITS data could not be standardised or relative effect could not be calculated (Aitken 2015). In the pre‐intervention period, five studies used wards as controls (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Menon 2018); and two studies used hospitals as controls (Ludikhuize 2015; Chen 2016). Three studies described usual care to comprise availability of a cardiac arrest team (Lighthall 2010; Rothberg 2012; Menon 2018); one study described usual care as access to a medical registrar using MET criteria and a cardiac arrest team (Aitken 2015). Three studies did not describe usual practice prior to the implementation of RRS (Davis 2015; Ludikhuize 2015; Chen 2016).

Participants

Three studies reporting patient inclusion criteria included participants aged 18 years and over admitted to hospital wards (Ludikhuize 2015; Chen 2016; Menon 2018). Five studies enrolled people admitted to all acute general wards (Lighthall 2010; Rothberg 2012; Davis 2015; Chen 2016; Menon 2018). Two studies enrolled all people admitted from four (Ludikhuize 2015) or six wards (Aitken 2015) in the participating hospital(s).

Sample sizes ranged from 18,954 (Menon 2018) to 9,799,081 participants (Chen 2016). Three studies did not report participant numbers (Lighthall 2010; Rothberg 2012; Aitken 2015). Two studies providing participant demographics reported mean age ranged from 62 to 65 years (Lighthall 2010; Ludikhuize 2015). Two studies reported the proportion of participants across age categories (aged less than 18 to greater than 75 years) (Chen 2016; Menon 2018). The percentage of men ranged from 47% to 96% (Lighthall 2010).

Settings

Three studies were conducted in the US (Lighthall 2010; Rothberg 2012; Davis 2015), two in Australia (Aitken 2015; Chen 2016), one in the Netherlands (Ludikhuize 2015), and one in India (Menon 2018). Therefore, all studies took place in middle‐ to high‐income countries.

Most studies were single centre, tertiary (Aitken 2015; Menon 2018) or university‐affiliated (Lighthall 2010; Rothberg 2012) hospitals. Multicentre studies included two (Davis 2015), 12 (Ludikhuize 2015), or 232 (Chen 2016) participating hospitals. All studies differed in hospital size and number and type of wards involved. Hospital bed size in single‐centre studies ranged from a 226‐bed tertiary academic hospital (Menon 2018) to a 750‐bed tertiary hospital (Aitken 2015). Multicentre studies included two participating hospitals with an inpatient primary medical centre (392 beds) and all units in a sister campus (119 beds) (Davis 2015); two large university hospitals (number of beds 882 to 1000), eight large teaching hospitals (number of beds 359 to 1070) and two smaller regional hospitals (number of beds 290 to 325) (Ludikhuize 2015); and 232 hospitals in New South Wales (Chen 2016).

Two studies included acute surgical or medical wards (Ludikhuize 2015; Menon 2018). Three studies included admissions to all hospital wards (Rothberg 2012; Davis 2015; Chen 2016). Patient data from EDs (Lighthall 2010; Aitken 2015; Davis 2015) and ICU (Aitken 2015) were excluded in three studies or were not reported (Rothberg 2012; Ludikhuize 2015; Chen 2016; Menon 2018). Lighthall 2010 excluded ED cardiac arrest events but included deaths in ED, ICU and general wards. All studies excluded hospice or nursing units within the hospital.

Interventions

EWS interventions comprised the MEWS, an aggregated weighted score (Ludikhuize 2015) or single parameter calling criteria (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016; Menon 2018). In six studies, EWS interventions included objective physiological criteria and the subjective criterion of staff concern or worry to trigger a call for help (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016). The Amrita Warning Criteria developed in India and used in one study did not include a subjective criterion of staff concern (Menon 2018). Two studies included participant/family concern as a criterion to escalate a call for help (Davis 2015; Chen 2016).

RRS interventions included RRT in four studies (Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016), or MET in three studies (Lighthall 2010; Rothberg 2012; Menon 2018). Two studies introduced the RRS intervention on a 24 hour, seven day‐a‐week basis (Rothberg 2012; Aitken 2015). The remaining five studies did not state team availability (Lighthall 2010; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018). At the time of RRS implementation, the cardiac arrest team was retained (Lighthall 2010; Rothberg 2012; Davis 2015), considered as replaced by the RRS (Aitken 2015), or availability was not stated (Ludikhuize 2015; Chen 2016; Menon 2018).

Team composition varied with teams comprising a medical‐led MET (Lighthall 2010; Rothberg 2012; Menon 2018); medical‐led RRT (Ludikhuize 2015; Chen 2016), or nurse‐led RRT (Aitken 2015; Davis 2015). Team membership also demonstrated substantial heterogeneity (Table 7).

All seven non‐randomised studies compared the EWS and RRS intervention with a control group comprising standard practice of acute care management before EWS and RRS implementation (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018). Intervention hospitals/wards all used an educational strategy to introduce the EWS and RRS intervention, with length of the training implementation phase ranging from several hours as part of annual training (Davis 2015), one month (Menon 2018), four months (Lighthall 2010), to seven months for the implementation of EWS and SBAR as part of the first phase of a two‐stage RRS strategy (Ludikhuize 2015). Three studies did not describe the training timeframe (Rothberg 2012; Aitken 2015; Chen 2016). Training consisted of in‐service educational programmes on calling criteria (Lighthall 2010), purpose of RRT and how to activate team (Rothberg 2012), recognition of patient deterioration and annual advanced resuscitation training programme (Davis 2015), MEWS and SBAR communication tool (Ludikhuize 2015), and standardised state‐wide Between the Flags programme (Chen 2016). Two studies reported educational sessions were tailored for coronary care unit (CCU), ED, ward nurses, consultants and registrars (Aitken 2015), or were trained using standardised tool‐kits (Ludikhuize 2015). Two studies did not report on educational content (Aitken 2015; Menon 2018). One study reported the mode of delivery was via meetings, email, communication boards and posters (Rothberg 2012). To reinforce implementation, four studies employed reminders in the form of posters (Lighthall 2010; Rothberg 2012; Aitken 2015; Ludikhuize 2015), identification badge holders with calling criteria (Lighthall 2010), pocket cards (Rothberg 2012; Ludikhuize 2015; Menon 2018), calling criteria lanyards and screen‐saver reminders (Aitken 2015), workflow handouts (Menon 2018), and activation number on all ward telephones (Rothberg 2012).

Outcomes

Individual patient data (i.e. patient admissions to study wards) was the unit of analysis in four studies (Rothberg 2012; Aitken 2015; Davis 2015; Menon 2018). In the remaining studies, the unit of analysis was the cluster at either the hospital (Ludikhuize 2015; Chen 2016), or ward level (Lighthall 2010).

The baseline data collection period was from five months (Ludikhuize 2015), nine months (Lighthall 2010), 12 months (Menon 2018,) and beyond (Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016). Studies reported postintervention implementation data at five months (Ludikhuize 2015), 24 to 27 months (Lighthall 2010; Menon 2018), or 36 months and beyond (Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016).

Selection of outcomes varied across studies. The primary outcome of non‐randomised studies included mortality rates (Lighthall 2010; Rothberg 2012; Davis 2015; Chen 2016; Menon 2018); composite endpoint comprising cardiopulmonary arrest, unplanned ICU admission, or death (Ludikhuize 2015); incidence of unplanned ICU admission (Aitken 2015; Davis 2015); and cardiac arrest rates (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016; Menon 2018). One study reported individual components of their composite endpoint (cardiopulmonary arrest, unplanned ICU admission or death) as secondary outcomes (Ludikhuize 2015). Other outcomes included staff satisfaction, prevalence of deteriorating patients (Aitken 2015); patient acuity, hospital discharge (Davis 2015); Charlson Comorbidity Index (Lighthall 2010); number of RRS calls (Rothberg 2012; Davis 2015; Menon 2018); MET process variables and staff questionnaire (Menon 2018). No studies reported ICU readmission rates.

Excluded studies

Ninety‐seven studies did not meet our inclusion criteria and are included in the Characteristics of excluded studies table. The primary reasons for exclusion was study design due to an inadequate number of data points before and after the intervention to meet the EPOC criteria for CBAs, followed by failing to include both EWS and RRS as components of the intervention.

One author replied to a request for further information (Rashid 2014), and we excluded the study as a retrospective single‐centre CBA. We excluded one paper, which we translated from Czech to English, based on study design criteria  (Martin 2009). Requests for additional studies from professional organisations identified three studies from the European federation of Critical Care Nurses association (EfCCNa) (Laurens 2011; Chen 2015; Petersen 2016), and one study from Critical Care National Network Nurse Leads Forum (CC3N) (Churpek 2017). We excluded all four studies, two based on design criteria (Laurens 2011; Chen 2015), and two as the wrong intervention (Petersen 2016; Churpek 2017).

Risk of bias in included studies

Randomised trials

Risk of bias for the four randomised trials is presented in the Characteristics of included studies table detailing our judgements for each domain (see Figure 2).

2.

Risk of bias summary: review authors' judgements about each risk of bias item for each included randomised study.

Allocation

We considered three studies at low risk of bias for random sequence generation as they undertook block randomisation using computer software stratifying hospitals by teaching status and number of beds (Hillman 2005); they conducted randomisation at a fixed time point independent of the trial team (Jeddian 2016), or an individual not involved in further study conduct performed computerised randomisation (Haegdorens 2018). One study did not describe the method of random sequence generation explicitly and was at unclear risk of bias (Priestley 2004).

We considered two studies at low risk of bias for allocation concealment as an independent statistician concealed allocation from investigators/participating hospitals (Hillman 2005), or information on ward sequence was revealed two to three days before the transition period commenced (Jeddian 2016). We considered two studies as unclear risk of bias (Priestley 2004; Haegdorens 2018) as the method of allocation concealment was not described explicitly.

Blinding

We rated performance bias in all randomised trials at high risk of bias as blinding of participants and personnel to group allocation was not feasible and therefore not performed.

We rated all studies at low risk of bias for blinding of outcome assessment as outcomes were objective measurements collected by trained data collectors/independent data team (Hillman 2005), or used hospital databases (Priestley 2004; Jeddian 2016), or both (Haegdorens 2018).

Incomplete outcome data

We rated three studies at low risk of bias as no study reported incomplete data or loss to follow‐up (Priestley 2004; Hillman 2005; Jeddian 2016); and one study at high risk of bias as 50% of enrolled hospitals withdrew from the study (Haegdorens 2018). Three studies reported using intention‐to‐treat analyses (Priestley 2004; Hillman 2005; Jeddian 2016); the remaining study did not report this (Haegdorens 2018).

Selective reporting

We identified two published trial registrations  with the same outcomes identified a priori as published in the final report (Jeddian 2016, IRCT201107187053N1; Haegdorens 2018, NCT01949025). For the two earlier studies for which we were unable to identify a trial registration or published protocol, we considered one study included all expected outcomes and rated as low risk of bias (Hillman 2005); the remaining study was at high risk of bias as authors stated findings were confirmed using data set three but provided no details or data for data set three to confirm findings (Priestley 2004).

Other potential sources of bias

We rated three studies at unclear risk of bias as there was potential for contamination due to publicity of the intervention across control wards (Priestley 2004; Jeddian 2016; Haegdorens 2018); and the remaining study at high risk of contamination bias as the benefits of the MET system were widely reported in the media, which may have impacted on the behaviour of the control hospitals (Hillman 2005).

Non‐randomised studies

The domain‐level judgements for risk of bias for all seven non‐randomised studies are presented (see Figure 3). Further details and our supporting statements for domain‐level judgments for risk of bias are available (Table 4) and summarised (Table 8). 

3.

Risk of bias summary: review authors' judgements about each risk of bias item for each included non‐randomised study.

7. Quality assessment summary (ROBINS‐I) of non‐randomised studies.

Study ID	D1: confounding	D2: selection	D3: intervention classification	D4: intervention deviation	D5: missing data	D6: outcome measurement	D7: result reporting	Overall
Lighthall 2010	Serious	Low	Low	No information	Low	Low	Low	Serious
Rothberg 2012	Serious	Low	Low	No information	Low	Low	Low	Serious
Aitken 2015	Critical	Low	Low	No information	Low	Low	Low	Critical
Davis 2015	Critical	Low	Low	No information	Low	Low	Low	Critical
Ludikhuize 2015	Serious	Low	Low	No information	Low	Low	Low	Serious
Chen 2016	Serious	Low	Low	No information	Low	Low	Low	Serious
Menon 2018	Critical	Low	Low	No information	Low	Low	Low	Critical

Low risk of bias: comparable to a well‐performed randomised trial.
Moderate risk of bias: sound evidence for a non‐randomised study but cannot be considered comparable to a well‐performed randomised trial.
Serious risk of bias: some important problems; critical risk of bias: too problematic to provide any useful evidence and should not be included in any synthesis.
No information: no information on which to base a judgement about risk of bias.

ROBINS‐I: Risk Of Bias in Non‐randomised Studies – of Interventions.

Confounding bias

We established the following confounding variables a priori: pre‐existing comorbidity; age; severity of illness during ICU admission; ICU and hospital census; staffing levels; patient acuity/case‐mix on hospital wards; ward culture and organisational practices; medical and nursing education levels; ICU discharge policies, ICU readmission policies, and overall predilection to readmit to ICU; ICU admission/readmission rates; prevalence of patients with NFR order; seasonal variation in patient census; patient safety initiatives within the organisation; cardiac arrest team in place as well as the outreach/RRT; education provided to staff specific to management of a deteriorating patient; policy changes to ICU readmission practices.

We considered three studies at critical risk of confounding bias (Aitken 2015; Davis 2015; Menon 2018), and four studies at serious risk  due to use of statistical methods to control for some but not all baseline confounders (Lighthall 2010; Rothberg 2012; Ludikhuize 2015; Chen 2016).

Selection bias

We considered all seven studies at low risk of selection bias as we saw no evidence of selective recruitment (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018). All admitted patients were included in the seven studies based on time of exposure, that is, before or after the intervention commenced as opposed to participant characteristics observed after the commencement of the intervention. For each participant, start of follow‐up and start of intervention coincided.

Bias in classification of interventions

We considered all seven studies at low risk of bias in classification of interventions  as intervention status was clearly defined in terms of time of exposure, that is, admission to hospital before or after the EWS and RRS intervention was introduced (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Bias due to deviations from intended interventions

For all seven studies, we were unable to assess if there was deviation from the intended intervention due to lack of reporting (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Bias due to missing data

We considered all seven studies at low risk of bias due to missing data (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Bias in measurement of outcomes

Measured outcomes including cardiac arrest, death and unplanned ICU admission are all objective and not subject to measurement bias. Therefore, we considered all seven studies at low risk of outcome measurement bias (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Bias in selection of the reported result

We found no evidence of selective reporting and rated risk as low for all seven studies (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Overall risk of bias

We rated three studies at critical overall risk of bias (Aitken 2015; Davis 2015; Menon 2018), and four studies at serious risk of bias (Lighthall 2010; Rothberg 2012; Ludikhuize 2015; Chen 2016).

Effects of interventions

See: Table 1

See: Table 1.

We present effect sizes for each study for the following outcomes.

Hospital mortality

Ten studies reported on mortality.

Randomised trials

We identified four studies recruiting 455,226 participants reporting on hospital mortality. The measurement of hospital mortality varied across studies; Priestley 2004 and Jeddian 2016 reported hospital mortality without exclusions while Haegdorens 2018 and Hillman 2005 reported unexpected mortality as either NFR order, palliative or terminal care, family attending during the process of dying, cessation or limiting of active therapy in untreatable disease' (Haegdorens 2018) or as all inpatient deaths without a pre‐existing NFR order (Hillman 2005).

One study identified a reduction in hospital mortality (adjusted odds ratio (aOR) 0.52, 95% CI 0.32 to 0.85) (Priestley 2004). The remaining three studies identified no effect on hospital mortality (aOR 1.03, 95% CI 0.84 to 1.28 (Hillman 2005); aOR 1.02, 95% CI 0.68 to 1.55 (Jeddian 2016); aOR 0.82, 95% CI 0.34 to 1.95 (Haegdorens 2018)).

All randomised trials adjusted for cluster (ward/bed number); three studies adjusted for time trends (Priestley 2004; Jeddian 2016; Haegdorens 2018).

The implementation of EWS and RRS may result in little or no difference in hospital mortality (low‐certainty evidence).

Non‐randomised studies

We identified six studies, three recruited 210,905 participants (number of participants in remaining three studies was unclear due to reporting bed occupancy (Lighthall 2010);  number of participants not provided (Rothberg 2012); and participants grouped annually (Chen 2016)).

One ITS study identified a reduction in hospital mortality (adjusted risk ratio (aRR) 0.81, 95% CI 0.76 to 0.86 (Chen 2016)). Two studies showed no effect (aOR 0.85, 95% CI 0.64 to 1.00 (Ludikhuize 2015); risk ratio (RR) 1.0, 95% CI 0.03 to 29.8 (Menon 2018)). We were unable to calculate the relative effect on mortality for the remaining three studies (Lighthall 2010; Rothberg 2012; Davis 2015). Reanalysed ITS data for one single‐centre CBA showed an absolute increase of 0.19 deaths per 100 discharge at 24 months' postintervention (0.19, 95% CI −0.83 to 1.22; P = 0.02; Lighthall 2010).

We are uncertain whether this intervention reduces mortality because the certainty of this evidence was very low (Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018).

Non‐randomised studies adjusted for time (Rothberg 2012; Davis 2015; Chen 2016), patient acuity (Davis 2015); individual characteristics (age, sex, marital status, country of birth, socioeconomic status, private health insurance), geographical locations of hospitals (urban versus rural), hospital type (Chen 2016);  time trends and seasonality (time series data), CCI score and periodic or secular variations (Lighthall 2010); and age, sex, hospital and admission type (Ludikhuize 2015).

Composite outcome

Two studies reported on a composite outcome as either the composite outcome of incidence (events divided by number of eligible patients admitted to the hospital during the study period) of cardiac arrests without a pre‐existing NFR order, unplanned ICU admissions, and unexpected deaths (deaths without a pre‐existing NFR order) (Hillman 2005) or composite endpoint of cardiopulmonary arrest, unplanned ICU admission, or death per 1000 admitted patients (Ludikhuize 2015).

Randomised trials

One study recruited 364,094 participants and found no effect in composite outcome during the six‐month study period (aOR 0.98, 95% CI 0.83 to 1.16; Hillman 2005), adjusting for individual (sex, age) and cluster (bed number, hospital teaching status) characteristics (ICC 0.067, 95% CI 0.05 to 0.08). The implementation of EWS and RRS probably results in little or no difference in composite outcome (moderate‐certainty evidence).

Non‐randomised studies

One prospective, multicentre non‐randomised study recruited 57,858 participants and found a reduction in composite outcome (aOR 0.85, 95% CI 0.72 to 0.99; Ludikhuize 2015), adjusting for individual (age, sex), emergency admission and individual hospital characteristics. The implementation of EWS and RRS may result in little or no difference in composite outcome (low‐certainty evidence).

Unplanned intensive care unit admission

Five studies reported unplanned ICU admissions as a primary (Aitken 2015; Haegdorens 2018) or secondary outcome (Hillman 2005; Ludikhuize 2015; Jeddian 2016).

Randomised trials

We identified three studies recruiting 452,434 participants. All three studies identified no effect on the incidence of unplanned ICU admissions (aOR 1.04, 95% CI 0.89 to 1.21 (Hillman 2005); aOR 1.15, 95% CI 0.64 to 2.09 (Jeddian 2016); aOR 1.23, 95% CI 0.91 to 1.65 (Haegdorens 2018)), adjusting for cluster (ward/bed number) and time trends (Hillman 2005; Jeddian 2016; Haegdorens 2018). RRS and EWS intervention may result in little or no difference in unplanned ICU admissions (low‐certainty evidence).

Two studies reported ICC (0.09, 95% CI 0.076 to 0.12 (Hillman 2005); 0.01, 95% CI 0.0 to 0.26 (Jeddian 2016)).

Non‐randomised studies

We identified two studies, one reported recruiting 57,858 participants. One prospective, multicentre CBA found no effect in incidence of unplanned ICU admission (aOR 0.88, 95% CI 0.75 to 1.02, Ludikhuize 2015). Reanalysed ITS data from a single‐centre CBA showed no evidence of change in trend due to the intervention (P = 0.10), with an absolute decrease of 6.52 unplanned ICU admissions per month after 24 months intervention (−6.52, 95% CI −15.24 to 2.20; Aitken 2015). We were unable to calculate the absolute difference or standardise reanalysed ITS data in this study as the number of admissions was not provided (Aitken 2015). Therefore, we do not know whether EWS and RRS reduces unplanned ICU admissions (very low‐certainty evidence).

Ludikhuize 2015 adjusted for individual (age, sex), emergency admission and individual hospital characteristics; Aitken 2015 adjusted for time differences.

Intensive care unit readmission

No studies reported ICU readmission rates and therefore the effect on ICU readmission of an EWS and RRS intervention is unknown.

Length of hospital stay

Two randomised trials reported length of hospital stay (Priestley 2004; Jeddian 2016). No non‐randomised studies reported this outcome.

Randomised trials

We identified two studies recruiting 21,417 participants. Priestley 2004 reported an increase in mean length of hospital stay  in the EWS and RRS intervention arm compared to control (hazard ratio (HR) 0.91, 95% CI 0.83 to 0.98). However, there was no difference following adjustment for clustering and in further sensitivity analyses examining ward characteristics and time trends. Jeddian 2016 found a reduction in the median days of hospital stay (6, IQR 3 to 10 with intervention versus 4, IQR 2 to 8 with no intervention; aRGM 1.00, 95% CI 0.97 to 1.03). However, there was no difference when adjusted for individual (age, sex, Simplified Acute Physiology Score (SAPS II), admission type), cluster (ward) and time effects (aRGM 1.00, 95% CI 0.97 to 1.03; ICC 0.10, 95% CI 0.01 to 0.18). Therefore, an EWS and RRS intervention may result in little or no difference in hospital length of stay (low‐certainty evidence).

Non‐randomised studies

No non‐randomised studies reported length of hospital stay.

Adverse events (unexpected cardiac or respiratory arrest)

Ten studies measured cardiac arrest rates as either a primary (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016; Jeddian 2016; Menon 2018; Haegdorens 2018) or secondary outcome (Hillman 2005; Ludikhuize 2015).

Randomised trials

We identified three studies recruiting 452,434 participants. All three studies showed no difference in adverse event rates when adjusted for cluster (ward/bed number) and time effects (aOR 0.94, 95% CI 0.79 to 1.13 (Hillman 2005); aOR 1.00, 95% CI 0.69 to 1.48 (Jeddian 2016); aOR 0.71, 95% CI 0.33 to 1.52 (Haegdorens 2018)). The implementation of EWS and RRS may result in little or no difference to adverse events (low‐certainty evidence).

Two studies reported ICC (0.02, 95% CI 0.01 to 0.07 (Hillman 2005); 0.06, 95% CI 0.00 to 0.12 (Jeddian 2016)).

Non‐randomised studies

We identified seven studies, three reported recruiting 210,905 participants and the number of participants in remaining four studies was unclear due to reporting bed occupancy (Lighthall 2010), number of admissions/participants not reported (Rothberg 2012; Aitken 2015), and number of participants grouped yearly (Chen 2016).

Two studies reported a reduction in adverse events (aOR 0.61, 95% CI 0.39 to 0.94 (Ludikhuize 2015); aRR 0.55, 95% CI 0.48 to 0.62 (Chen 2016)). One study reported no effect on adverse event rates (aOR 1.00, 95% CI 0.69 to 1.48; Rothberg 2012).

Reanalysed data from single‐centre CBA studies showed no evidence of change in trend due to the EWS and RRS intervention  24 months postintervention (20.3, 95% CI −18.56 to 59.19; P = 0.62 (Lighthall 2010); −1.29, 95% CI −8.49 to 5.91; P = 0.58 (Aitken 2015); 3.44, 95% CI −10.35 to 17.23; P = 0.25 (Menon 2018)). As the certainty of this evidence was very low, we do not know whether this intervention reduces cardiac arrests.

Non‐randomised studies adjusted for time (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016; Menon 2018), patient acuity (Davis 2015); individual (age, sex, marital status, country of birth, socioeconomic status, private health insurance), geographical area of hospitals (urban versus rural), major hospital peer groups (Chen 2016); and age, sex, hospital and admission type (Ludikhuize 2015).

Reanalysed data

The cluster randomised study results are reported as the published adjusted relative effects as these were appropriately adjusted for design and patient characteristics (Table 1). The reanalysed absolute effects are presented in a separate table as there were noticeable differences in effect sizes between the relative effect and absolute effect (Table 2). These differences were attributed to errors as the absolute effect values were unadjusted and the ICC was not derived from the studies.

Non‐randomised study results provide relative effects obtained from the original study data where possible (Table 1) or as reanalysed ITS data for three single‐centre CBA studies (Lighthall 2010; Aitken 2015; Menon 2018) (Table 2). There were no data where relative differences could not be obtained from study authors or calculated from the original study data.

Discussion

Summary of main results

We identified 11 studies meeting our inclusion criteria and reporting on the effectiveness of an EWS and RRS intervention for reducing hospital mortality, composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death), unplanned ICU admissions, length of hospital stay and adverse events (unexpected cardiac arrest or respiratory arrest). We found no studies reporting on ICU readmission rates.

Meta‐analyses across studies were not possible due to clinical heterogeneity in terms of the intervention, setting and study designs. From randomised trials, we found low‐certainty evidence that an EWS and RRS intervention may result in little or no difference in hospital mortality, unplanned ICU admission rates, hospital length of stay or adverse events; and moderate‐certainty evidence of little to no effect on composite outcome.

From non‐randomised studies, we found very low‐certainty evidence of little to no effect of EWS and RRS interventions on hospital mortality, unplanned ICU admissions or adverse events. We found low‐certainty evidence of little to no effect on composite outcome. No non‐randomised studies reported hospital length of stay or ICU readmissions and, therefore, we are unable to make a statement as to the effect of EWS and RRS interventions on these outcomes.

See Table 1 for more information on our judgements for the certainty of evidence for our outcomes of interest.

Overall completeness and applicability of evidence

The 11 included studies were conducted in large urban hospitals in middle‐ or high‐income countries, with diverse patient populations. Therefore, we can make no comment on the applicability of our findings to hospitals in lower‐income countries. These 11 studies highlight heterogeneity in EWS systems between aggregated weighted scores and single parameter criteria, RRS leadership and composition, RRS dose and implementation processes. These factors and the variation in context and adherence to EWS and RRS make generalisations difficult. The choice of EWS tool was linked to the type of RRS specialist team used (i.e. CCOT, RRT or MET). In general, aggregate scoring systems were used by CCOTs; single parameter systems were used by METs/RRTs. Single parameter criteria were more commonly used as the EWS trigger mechanism, despite being less accurate than aggregated weighted scores (Churpek 2016; Smith 2016). Further evidence is required to understand the effect of the type of trigger mechanism used on patient outcomes.

RRS composition and number of individuals in the team varied across countries with either a nurse‐led, or physician‐led team supported by various other supporting team members. The impact of the team composition is unclear. We had planned a subgroup analysis of nurse‐led outreach versus medical‐led MET. However, this was not possible due to the limited number of studies. Most studies did not state the availability of the team. Structure and content of ward staff training also varied across studies. To inform further understanding of the effectiveness of EWS and RRS interventions, standardisation of practices and guidelines for the composition and dose of RRS would allow for more robust comparisons.

Due to strict inclusion criteria relating to study design, that is, exclusion of single‐centre CBA without three data points before and after intervention implementation, we were only able to identify a small number of relevant studies. Despite these studies including 666,131 participants, no strong recommendations as to the effectiveness of EWS and RRS interventions can be made based on the evidence currently available. Inconclusive findings from the review limit the applicability of the findings. Our inclusion of non‐randomised studies provided an important understanding of the limitations of combining study designs, as results differed from the results of randomised trials (Reeves 2020). No studies evaluated costs and future work should determine the economic implications of implementing RRS.

The evidence from this review update highlights some diversity in outcome selection and moderate‐to‐poor methodological quality of studies investigating EWS and RRS interventions. The moderate‐ to very low‐certainty evidence makes it difficult to draw conclusions regarding the effectiveness of EWS and RRS. The inconsistent implementation of EWS and RRS criteria, team composition, dose and processes require further description to understand intervention fidelity across contexts and countries. There is a need for development of a patient‐informed core outcome set comprising clear and consistent definitions (COMET Initiative), and recommendations for measurement as well as EWS and RRS interventions conforming to a standard to facilitate meaningful comparison and future meta‐analyses. Further studies utilising alternative designs to randomised trials is warranted to determine the effectiveness of EWS and RRS.

Quality of the evidence

We judged the certainty of evidence from randomised controlled trials as low for hospital mortality, unplanned ICU admission, length of stay and adverse events, and moderate for the composite outcome (Table 1). We judged the certainty of evidence from non‐randomised studies as very low for hospital mortality, unplanned ICU admissions and adverse events, and low certainty for the composite outcome (Table 1). Because we applied the ROBINS‐I tool to studies as a whole rather than to specific results as is recommended, the bias assessments may not accurately reflect bias of the individual results of the studies.

Potential biases in the review process

We believe the potential for bias in our review process is low. We adhered to procedures outlined by Cochrane (Higgins 2011); used a comprehensive search strategy designed to minimise the risk of language and publication bias; and conducted independent study inclusion screening, data extraction, risk of bias assessment and grading of evidence certainty by two review authors. We made modifications to the review from the original protocol (see Differences between protocol and review), but we do not believe these modifications introduced bias into the review process.

Agreements and disagreements with other studies or reviews

We identified seven systematic reviews on EWS and RRS interventions. Comparison of our findings with three reviews highlight similar issues of poor‐quality studies with high risk of bias and the variability of evidence on patient outcomes (Chan 2010; McNeill 2013; Tirkkonen 2017). Four other systematic reviews undertook meta‐analyses including randomised trials, prospective and retrospective observational studies (before and after) and ITS designs. These broader inclusion criteria in terms of study design, single centre studies and adult / paediatric populations resulted in inclusion of 26 studies (Winters 2013), 29 studies (Maharaj 2015), 32 studies (de Jong 2016), and 30 studies (Solomon 2016). These four systematic reviews all reported that an EWS and RRS intervention reduces hospital mortality (Winters 2013; Maharaj 2015; de Jong 2016; Solomon 2016), and cardiac arrests (Winters 2013; Maharaj 2015; Solomon 2016).

A major difference of our review was the strict inclusion criteria in terms of study design and adult population limiting the number of included studies. Further, we chose not to meta‐analyse data from studies that we considered to have considerable heterogeneity in terms of the structure and composition of the study interventions. As a result, the findings of our review update are based on studies with more rigorous designs and findings that are more cautious given the limited number of high‐quality studies.

Authors' conclusions

Implications for practice.

There is low‐ to moderate‐certainty evidence from randomised controlled trials that early warning system (EWS) and rapid response system (RRS) interventions have little to no effect on hospital mortality, composite outcome, unplanned ICU admission rates, hospital length of stay or adverse events. However, the widespread adoption of EWS and RRS suggests there are perceived benefits in practice that currently are not evidenced in measured objective patient outcomes. For example, having the ability to call a specialist team for urgent review may provide benefits for ward staff in terms of reduced stress, and improved well‐being. This may lead to improved staff retention. To ascertain the perceived benefits for healthcare staff, patients, relatives and organisations, further research is required that measures staff‐, patient‐ and relative‐reported outcomes as well as a better understanding of effective processes to implement and sustain an EWS and RRS intervention.

Implications for research.

EWS and RRS have been implemented internationally with no evidence of cost‐effectiveness. Future research studies need to build in an economic evaluation of EWS and RRS to ensure the implementation of EWS and RRS is cost‐effective given the low‐ to moderate‐certainty evidence from randomised controlled trials that EWS and RRS interventions have little to no effect on hospital mortality, unplanned ICU admission rates, hospital length of stay or adverse events.

The variation in context, adherence, and the type and combination of EWS and RRS interventions restricts direct comparisons. Further, evidence suggests EWS with standardised protocols are being implemented in hospitals within existing referral systems (i.e. without the implementation of an RRS). Implementation of the afferent arm of the RRS without the efferent arm may be due to a lack of equivocal evidence or based on economic reasons. Interpreting the effect of multiple interventions introduced at the same time is difficult and further understanding of the components of the afferent and efferent arm for effective RRS implementation is required. Additionally, evidence underpinning the process and context of EWS and RRS implementation suggests that staffing ratios and resources impact on the fidelity of the intervention and the effect on measured outcomes (McGaughey 2017a; McGaughey 2017b). Therefore, future studies of EWS and RRS interventions should be accompanied by a process evaluation (Moore 2015) using a framework that helps understand the contextual drivers of the implementation process.

The potential risk of contamination bias is an issue with EWS and RRS research. This was specifically highlighted by Sandroni 2015 with reference to the Hillman 2005 study which noted that although authors made every effort to prevent contamination, the benefits of the RRS system were reported in the media during the study period which could have affected personal behaviour in the control hospitals and explain why the rates of in‐hospital cardiac arrest decreased more in the hospitals of the control group than in those of the interventional group between the two study periods. Large randomised trials of complex interventions with an embedded process evaluation may provide insight into the potential risks and strategies that need to be considered and monitored to mitigate against contamination in the conduct of trials (Robinson 2020).

In summary, research implications arising from this review include the need to:

•  understand decisions regarding why different afferent and efferent components are adopted in practice;

• measure fidelity of future EWS and RRS intervention trials;

• determine the most effective dose and reach of EWS and RRS interventions;

• establish consensus on a core outcome measurement set to facilitate comparisons of outcomes across studies; and

• conduct cost‐effectiveness studies.

What's new

Date	Event	Description
29 October 2020	New citation required and conclusions have changed	This is the first update of the Cochrane Review originally published in 2007. We conducted a new search and updated other content according to the methodological expectations of Cochrane Intervention Reviews (MECIR). The review now includes 11 studies.

History

Protocol first published: Issue 4, 2005
Review first published: Issue 3, 2007

Date	Event	Description
15 May 2020	New search has been performed	Updated MEDLINE search identified 163 studies which were screened with no further studies meeting our inclusion criteria identified.
12 November 2008	Amended	Minor changes
22 April 2008	Amended	Converted to new review format.
23 May 2007	New citation required and conclusions have changed	Substantive amendment

Notes

Prior to submission for publication, an updated MEDLINE search was run (see Appendix 1). This search identified 163 studies, which were screened with no further studies meeting our inclusion criteria. Consequently, the network editor approved publication without an update search for this update. 

Appendices

Appendix 1. Search strategies

MEDLINE (Ovid)
Search date: 28 March 2019

includes In‐Process & Other Non‐Indexed Citations

No.	Search terms	Results
1	hospital rapid response team/	662
2	(critical care adj2 outreach).ti,ab.	131
3	(rapid response adj2 (team? or system?)).ti,ab.	963
4	medical emergency team?.ti,ab.	487
5	code team*.ti,ab.	67
6	crash team*.ti,ab.	7
7	medical crisis team?.ti,ab.	2
8	hospital at night.ti,ab.	42
9	patient at risk.ti,ab.	1034
10	"track and trigger".ti,ab.	98
11	or/1‐10	2878
12	(warning system? or warning scor* or early warning).ti,ab.	5888
13	exp intensive care units/	76387
14	emergencies/	38957
15	emergency medicine/	12628
16	exp emergency medical services/	128798
17	exp critical care/	53880
18	critical illness/	25433
19	((critical* or acute*) adj ill*).ti,ab.	51970
20	((acute or intensive or critical) adj care).ti,ab.	163347
21	emergen*.ti,ab.	360711
22	(clinical* adj deteriorat*).ti,ab.	4310
23	exp monitoring, physiologic/	163747
24	exp vital signs/	396027
25	monitor*.ti,ab.	729007
26	vital sign?.ti,ab.	12593
27	or/13‐26	1779978
28	12 and 27	2662
29	11 or 28	5339
30	randomized controlled trial.pt.	478827
31	controlled clinical trial.pt.	92995
32	multicenter study.pt.	247591
33	pragmatic clinical trial.pt.	1011
34	(randomis* or randomiz* or randomly).ti,ab.	819388
35	groups.ab.	1894087
36	(trial or multicenter or multi center or multicentre or multi centre).ti.	232146
37	(intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or repeated measur*).ti,ab.	8881821
38	non‐randomized controlled trials as topic/	469
39	interrupted time series analysis/	549
40	controlled before‐after studies/	380
41	or/30‐40	9907346
42	exp animals/	2.2E+07
43	humans/	1.8E+07
44	42 not (42 and 43)	4563407
45	review.pt.	2494049
46	meta analysis.pt.	98900
47	news.pt.	194141
48	comment.pt.	761592
49	editorial.pt.	485437
50	cochrane database of systematic reviews.jn.	14093
51	comment on.cm.	761534
52	(systematic review or literature review).ti.	127756
53	or/44‐52	8188805
54	41 not 53	6954531
55	29 and 54	2207

 

MEDLINE (Ovid)

Updated search date: 15 May 2020

1	hospital rapid response team/	297
2	(critical care adj2 outreach).ti,ab.	16
3	(rapid response adj2 (team? or system?)).ti,ab.	326
4	medical emergency team?.ti,ab.	103
5	code team*.ti,ab. (14)	14
6	crash team*.ti,ab.	0
7	medical crisis team?.ti,ab.	0
8	hospital at night.ti,ab.	10
9	patient at risk.ti,ab.	112
10	"track and trigger".ti,ab.	36
11	1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10	665
12	(warning system? or warning scor* or early warning).ti,ab.	1641
13	exp intensive care units/	17464
14	emergencies/	3128
15	emergency medicine/	2261
16	exp emergency medical services/	26195
17	exp critical care/	8463
18	critical illness/	7433
19	((critical* or acute*) adj ill*).ti,ab.	9708
20	20 ((acute or intensive or critical) adj care).ti,ab.	30725
21	emergen*.ti,ab.	69753
22	(clinical* adj deteriorat*).ti,ab.	755
23	exp monitoring, physiologic/	26102
24	exp vital signs/	33510
25	monitor*.ti,ab.	121303
26	vital sign?.ti,ab.	2892
27	or/13‐26	271347
28	12 and 27	839
29	11 or 28	1417
30	randomized controlled trial.pt.	78514
31	controlled clinical trial.pt.	2784
32	multicenter study.pt.	62265
33	pragmatic clinical trial.pt.	844
34	(randomis* or randomiz* or randomly).ti,ab.	155702
35	groups.ab.	316764
36	(trial or multicenter or multi center or multicentre or multi centre).ti.	53468
37	(intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or repeated measur*).ti,ab.	1402946
38	non‐randomized controlled trials as topic/	533
39	interrupted time series analysis/	570
40	controlled before‐after studies/	293
41	or/30‐40	1545468
42	exp animals/	2702941
43	humans/	2290095
44	42 not (42 and 43)	412846
45	review.pt.	355413
46	meta analysis.pt.	42457
47	news.pt.	12275
48	comment.pt.	158816
49	editorial.pt.	81909
50	cochrane database of systematic reviews.jn.	1362
51	comment on.cm.	158815
52	(systematic review or literature review).ti.	52187
53	or/44‐52	998690
54	41 not 53	1109544
55	29 and 54	720
56	limit 55 to (yr="2019 ‐Current")	163

 

Embase (Ovid)

Search date: 28 March 2019

No.	Search terms	Results
1	rapid response team/	1972
2	(critical care adj2 outreach).ti,ab.	250
3	(rapid response adj2 (team? or system?)).ti,ab.	1492
4	medical emergency team?.ti,ab.	717
5	code team*.ti,ab.	175
6	crash team*.ti,ab.	12
7	medical crisis team?.ti,ab.	2
8	hospital at night.ti,ab.	64
9	patient at risk.ti,ab.	1473
10	"track and trigger".ti,ab.	146
11	or/1‐10	4797
12	(warning system? or warning scor* or early warning).ti,ab.	7773
13	intensive care unit/	135245
14	emergency/	51406
15	emergency medicine/	38107
16	emergency health service/	88646
17	exp intensive care/	638450
18	critical illness/	27067
19	((critical* or acute*) adj ill*).ti,ab.	76379
20	((acute or intensive or critical) adj care).ti,ab.	237772
21	emergen*.ti,ab.	496642
22	(clinical* adj deteriorat*).ti,ab.	7025
23	exp physiologic monitoring/	5375
24	vital sign/	19753
25	monitor*.ti,ab.	993187
26	vital sign?.ti,ab.	24569
27	or/13‐26	2162766
28	12 and 27	3803
29	11 or 28	8238
30	randomized controlled trial/	538031
31	controlled clinical trial/	458827
32	quasi experimental study/	5390
33	pretest posttest control group design/	374
34	time series analysis/	22533
35	experimental design/	16656
36	multicenter study/	209033
37	(randomis* or randomiz* or randomly).ti,ab.	1135425
38	groups.ab.	2593764
39	(trial or multicentre or multicenter or multi centre or multi center).ti.	319997
40	(intervention? or effect? or impact? or controlled or control group? or (before adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or time series or time point? or repeated measur*).ti,ab.	11207074
41	or/30‐40	12508639
42	(systematic review or literature review).ti.	152716
43	"cochrane database of systematic reviews".jn.	13125
44	exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or animal tissue/ or animal cell/ or nonhuman/	25551124
45	human/ or normal human/ or human cell/	19467168
46	44 not (44 and 45)	6138161
47	42 or 43 or 46	6302571
48	41 not 47	9604704
49	29 and 48	4269

 

CENTRAL (in the Cochrane Library)

Search date: 28 March 2019

No.	Search terms	Results
#1	[mh "hospital rapid response team"]	13
#2	("critical care" near/2 outreach):ti,ab	7
#3	("rapid response" near/2 (team* or system*)):ti,ab	43
#4	medical emergency team*:ti,ab	28
#5	code team*:ti,ab	5
#6	crash team*:ti,ab	0
#7	medical crisis team*:ti,ab	0
#8	hospital at night:ti,ab	4
#9	patient at risk:ti,ab	298
#10	track and trigger:ti,ab	7
#11	{or #1‐#10}	391
#12	(warning NEXT system* or warning NEXT scor* or early NEXT warning*):ti,ab	173
#13	[mh "intensive care units"]	3233
#14	[mh emergencies]	1095
#15	[mh "emergency medicine"]	243
#16	[mh "emergency medical services"]	3494
#17	[mh "critical care"]	1911
#18	[mh "critical illness"]	1829
#19	((critical* or acute*) next ill*):ti,ab	5627
#20	((acute or intensive or critical) next care):ti,ab	14638
#21	emergenc*:ti,ab	18503
#22	(clinical* next deteriorat*):ti,ab	391
#23	[mh "monitoring, physiologic"]	11469
#24	[mh "vital signs"]	35118
#25	monitor*:ti,ab	55068
#26	(vital NEXT sign*):ti,ab	6109
#27	{or #13‐#26}	128421
#28	#12 and #27	94
#29	#11 or #28	476

 

CINAHL

Search date: 28 March 2019

No.	Search terms	Results
S1	(critical care N2 outreach)	135
S2	(rapid response N2 (team* or system*))	1,009
S3	medical emergency team*	545
S4	code team*	218
S5	crash team*	16
S6	medical crisis team*	17
S7	"hospital at night"	35
S8	"patient at risk"	1,268
S9	"track and trigger"	66
S10	(warning system* or warning scor* or early warning)	2,017
S11	S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10	5,009
S12	S11 Limiters ‐ Exclude MEDLINE records	1,955
S13	S12 Limiters ‐ Published Date: 20160101‐20191231	908
S14	PT randomized controlled trial	87,226
S15	PT clinical trial	86,261
S16	PT research	1,957,254
S17	(MH "Randomized Controlled Trials")	80,176
S18	(MH "Clinical Trials")	143,516
S19	(MH "Intervention Trials")	8,000
S20	(MH "Nonrandomized Trials")	389
S21	(MH "Experimental Studies")	23,330
S22	(MH "Pretest‐Posttest Design+")	38,767
S23	(MH "Quasi‐Experimental Studies+")	13,090
S24	(MH "Multicenter Studies")	105,085
S25	(MH "Health Services Research")	12,992
S26	TI ( randomis* or randomiz* or randomly) OR AB ( randomis* or randomiz* or randomly)	247,332
S27	TI (trial or effect* or impact* or intervention* or before N5 after or pre N5 post or ((pretest or "pre test") and (posttest or "post test")) or quasiexperiment* or quasi W0 experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or "time series" or time W0 point* or repeated W0 measur*) OR AB (trial or effect* or impact* or intervention* or before N5 after or pre N5 post or ((pretest or "pre test") and (posttest or "post test")) or quasiexperiment* or quasi W0 experiment* or pseudo experiment* or pseudoexperiment* or evaluat* or "time series" or time W0 point* or repeated W0 measur*)	1,670,088
S28	S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR S24 OR S25 OR S26 OR S27	2,701,725
S29	S13 AND S28	557

 

WHO International Clinical Trials Registry Platform (ICTRP)

Search date: 28 March 2019

Search terms	Results
critical care outreach	0
rapid response team	13
medical emergency team	13
medical crisis team	0
track and trigger	7
warning system	24
warning score	27
early warning	105
Total	189

 

ClinicalTrials.gov

Search date: 28 March 2019

Field	Searches terms
Other terms	“critical care outreach” OR “rapid response team” OR “medical emergency team” OR “medical crisis team” OR "track and trigger" OR "warning system" OR "warning score" OR "early warning"

Data and analyses

Comparison 1. Early warning systems (EWS) and rapid response systems (RRS) versus usual care.

Outcome or subgroup title	No. of studies	No. of participants	Statistical method	Effect size
1.1 Hospital mortality	10		Other data	No numeric data
1.1.1 Randomised controlled trials (RCT)	4		Other data	No numeric data
1.1.2 Non‐randomised controlled trials	6		Other data	No numeric data
1.2 Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)	2		Other data	No numeric data
1.2.1 Randomised controlled trials (RCT)	1		Other data	No numeric data
1.2.2 Non‐randomised controlled trials	1		Other data	No numeric data
1.3 Unplanned intensive care unit admission	5		Other data	No numeric data
1.3.1 Randomised controlled trials (RCT)	3		Other data	No numeric data
1.3.2 Non‐randomised controlled trial	2		Other data	No numeric data
1.4 Length of hospital stay	2		Other data	No numeric data
1.4.1 Randomised controlled trials (RCT)	2		Other data	No numeric data
1.5 Adverse events (unexpected cardiac or respiratory arrest)	10		Other data	No numeric data
1.5.1 Randomised controlled trials (RCT)	3		Other data	No numeric data
1.5.2 Non‐randomised controlled trials	7		Other data	No numeric data

1.1. Analysis.

Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 1: Hospital mortality

Hospital mortality
Study	Absolute effect (EWS and RRS)	Absolute effect (control)	Relative effect (95% CI)	No. participants/admissions	Notes
Randomised controlled trials (RCT)
Haegdorens 2018	0.7/1000	1.5/1000	aOR 0.82 (0.34 to 1.95)	69,656	0.8 more participants per 1000 died without EWS and RRS (95% CI 0.4 to 1.1) Step wedge cluster RCT; NEWS, medical‐led MET
Hillman 2005	1.1/1000	1.2/1000	aOR 1.03 (0.84 to 1.28)	364,094	0.1 more participants per 1000 died without EWS and RRS (95% CI −0.2 to 0.5) Cluster RCT; calling criteria, medical‐led MET
Jeddian 2016	35.3/1000	47.4/1000	aOR 1.02 (0.68 to 1.55)	18,684	12.2 more participants per 1000 died without EWS and RRS (95% CI 6.4 to 18.1) Step wedge cluster RCT; calling criteria, nurse‐led CCOT
Priestley 2004	49.1/1000	65.1/1000	aOR 0.52 (0.32 to 0.85)	2792	16.0 more participants per 1000 died without EWS and RRS (95 % CI 6.0 to 26.0) Prospective step wedge RCT, absolute effect based on Dataset 1 and 2 patients combined; PAR score, nurse‐led CCOT
Non‐randomised controlled trials
Chen 2016	14.5/1000	17.8/1000	aRR 0.81 (0.67 to 0.86)	NR	ITS, number of participants grouped yearly and not possible to determine number of participants or CI; calling criteria, medical‐led RRT
Davis 2015	0.2/1000	0.2/1000	—	134,093	0.04 more participants per 1000 died without EWS and RRS (95% CI −0.1 to 0.3) Number of participants unclear and CI calculated from Table 2 2 centre before‐after; calling criteria, nurse‐led RRT
Lighthall 2010	22.4/1000	27.1/1000	—	NR	Single‐centre before‐after; calling criteria, medical‐led MET Number of participants unclear and absolute effect based on 'bed occupancy'
Ludikhuize 2015	17.7/1000	20.4/1000	aOR 0.85 (0.64 to 1.00)	57,858	Prospective, pragmatic before‐after multicentre study; EWS, medical‐led RRT
Menon 2018	2.4/1000	4.93/1000	RR 1.0 (0.03 to 29.8)	18,954	2.49 more participants per 1000 died without EWS and RRS (95% CI 0.7 to 4.6) Prospective single‐centre before‐after; calling criteria, medical‐led MET
Rothberg 2012	22/1000	22/1000	—	NR	ITS. Number of participants unclear; calling criteria, medical‐led MET

1.2. Analysis.

Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 2: Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)

Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)
Study	Absolute effect (EWS and RRS)	Absolute effect (control)	Relative effect (95% CI)	No. participants/admissions	Notes
Randomised controlled trials (RCT)
Hillman 2005	5.3/1000	5.9/1000	aOR 0.98 (0.83 to 1.16)	364,094	0.6 more participants per 1000 experienced the composite outcome without EWS (95% CI −0.3 to 1.4) Cluster RCT; calling criteria, medical‐led MET
Non‐randomised controlled trials
Ludikhuize 2015	32.9/1000	37.1/1000	aOR 0.85 (0.72 to 0.99)	57,858	4.2 more participants per 1000 experienced the composite outcome without EWS (95% CI 1.2 to 7.2) Prospective, pragmatic before‐after multicentred trial; EWS, medical‐led RRT

1.3. Analysis.

Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 3: Unplanned intensive care unit admission

Unplanned intensive care unit admission
Study	Absolute effect (EWS and RRT)	Absolute effect (control)	Relative effect	No. participants/admissions	Notes
Randomised controlled trials (RCT)
Haegdorens 2018	10.3/1000	6.5/1000	aOR 1.23 (0.91 to 1.65)	69,656	3.8 fewer participants per 1000 experienced unplanned ICU admission without EWS and RRS (95% CI 2.3 to 5.0) Step wedge cluster RCT; NEWS, medical‐led MET
Hillman 2005	4.2/1000	4.7/1000	aOR 1.04 (0.89 to 1.21)	364,094	0.49 more participants per 1000 unplanned ICU admission without EWS and RRS (95% CI −0.2 to 1.2) Cluster RCT; calling criteria, medical‐led MET
Jeddian 2016	12.8/1000	12.3/1000	aOR 1.15 (0.64 to 2.09)	18,684	0.5 fewer participants per 1000 experienced unplanned ICU admission without EWS and RRS (95% CI −2.9 to 3.7) Step wedge cluster RCT; calling criteria, nurse‐led CCOT
Non‐randomised controlled trial
Aitken 2015	NR	NR	−6.52 (−15.24 to 2.20)	NR	Relative effect calculated from reanalysed ITS, absolute difference could not be calculated and reanalysed ITS could not be standardised as number of admissions not provided Quasi‐experimental, single‐centre pretest–post‐test, reanalysed ITS; calling criteria, nurse‐led RRT
Ludikhuize 2015	17.1/1000	19.8/1000	aOR 0.88 (0.75 to 1.02)	57,858	2.6 more participants per 1000 experienced unplanned ICU admission without EWS and RRS Prospective, pragmatic before‐after multicentre trial; EWS, medical‐led RRT

1.4. Analysis.

Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 4: Length of hospital stay

Length of hospital stay
Study	Absolute effect (EWS and RRT)	Absolute effect (control)	Relative effect	No. participants/admissions	Notes
Randomised controlled trials (RCT)
Jeddian 2016	Median (IQR) 4 (2 to 8) days	Median (IQR) 6 (3 to 10) days	aRGM 1.00 (0.97 to 1.03)	18,684	Length of stay was reduced by 2 days with EWS Step wedge cluster RCT; calling criteria, nurse‐led CCOT
Priestley 2004	NR	NR	HR 0.91 (0.83 to 0.98)	2733	Prospective step wedge RCT; PAR score, nurse‐led CCOT

1.5. Analysis.

Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 5: Adverse events (unexpected cardiac or respiratory arrest)

Adverse events (unexpected cardiac or respiratory arrest)
Study	Absolute effect (EWS and RRS)	Absolute effect (control)	Relative effect (95% CI)	No. participants/admissions	Notes
Randomised controlled trials (RCT)
Haegdorens 2018	1.0/1000	1.3/1000	aOR 0.71 (0.33 to 1.52)	69,656	0.3 more patients per 1000 experienced an adverse event without EWS and RRS (95% CI −0.7 to 1.4) Step wedge cluster RCT; NEWS, medical–led MET
Hillman 2005	1.3/1000	1.6/1000	aOR 0.94 (0.79 to 1.13)	364,094	03 more patients per 1000 experienced an adverse event without EWS and RRS (95% CI −0.1 to 0.8) Cluster RCT; calling criteria, medical‐led MET
Jeddian 2016	48.6/1000	36.1/1000	aOR 1.00 (0.69 to 1.48)	18,684	12.5 fewer participants per 1000 experienced an adverse event without EWS and RRS (95% CI 9.5 to 15.5) Step wedge cluster RCT; calling criteria, nurse‐led CCOT
Non‐randomised controlled trials
Aitken 2015	NR	NR	−1.29 (−8.49 to 5.91)	NR	Quasi‐experimental, single‐centre pretest–post‐test; reanalysed ITS; calling criteria, nurse‐led RRT
Chen 2016	1.3/1000	2.4/1000	aRR 0.55 (0.48 to 0.62)	NR	1.1 more participant per 1000 experienced an adverse event without EWS and RRS ITS; calling criteria, medical‐led RRT
Davis 2015	1.1/1000	2.7/1000	NR	134,093	1.6 more participants per 1000 experienced an adverse event without EWS (95% CI 1.1 to 2.2) 2 centre before‐after; calling criteria, nurse‐led RRT
Lighthall 2010	4.4/1000	10.1/1000	20.3 (−18.56 to 59.19)	NR	5.7 more participants per 1000 experienced an adverse event without EWS and RRS Single‐centre before‐after; reanalysed ITS; calling criteria, medical‐led MET
Ludikhuize 2015	1.2/1000	1.9/1000	aOR 0.61 (0.39 to 0.94)	57,858	0.7 more participant per 1000 experienced an adverse event without EWS and RRS (95% CI 0.1 to 1.4) Prospective, pragmatic before‐after multicentre trial; EWS, medical‐led RRT
Menon 2018	3.1/1000	6.8/1000	3.44 (−10.35 to 17.23)	18,954	Prospective single‐centre before‐after; reanalysed ITS; calling criteria, medical‐led MET
Rothberg 2012	3.1/1000	4.7/1000	aOR 1.00 (0.69 to 1.48)	NR	1.6 more participants per 1000 experienced an adverse event without EWS and RRS ITS, number of participants unclear; calling criteria, medical‐led MET

Characteristics of studies

Characteristics of included studies [ordered by study ID]

Aitken 2015.

Study characteristics
Methods	Quasi‐experimental pretest–post‐test design
Participants	Patients on 6 randomly selected wards for point prevalence assessment. Inpatients across all hospital wards for incidence of ICU unplanned admission and cardiac arrest and random selection of patients for chart review on predetermined dates. Age: NR Sex: NR Setting: tertiary metropolitan hospital (750 bed), Australia
Interventions	Intervention: a 2‐tier RRS consisting of ICUON and RRT, MEC criteria Team composition: ICUON to assist with stabilisation and care of deteriorating patients; multidisciplinary RRT. After hours, the team included medical resident, ICU junior registrar, ICUON and a CCU or ED nurse Timeline: point prevalence assessment 1 day prior to and 8 months following RRS implementation; incidence of ICU unplanned admissions and cardiac arrests 20 months prior and 48 months after RRS implementation Comparison: standard ward care included MEC calling criteria, medical registrar assistance, cardiac arrest team
Outcomes	Primary outcomes: prevalence of deteriorating patients, incidence of unplanned admission to ICU and cardiac arrests (mean/month), staff satisfaction
Notes	Funded by Princess Alexandra Hospital Private Practice Trust Fund
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Chen 2016.

Study characteristics
Methods	Interrupted time series (population based)
Participants	Participants: all patients admitted to hospital Age groups: before: ≥ 18  to  < 35 years: 18.4%; ≥ 35 and < 55 years: 22.8%; ≥ 55 to < 75 years: 32.2%; ≥ 75 years: 26.6%; after ≥ 18 to < 35 years: 16.7%; ≥ 35 to < 55 years: 21.7%; ≥ 55 to < 75 years: 33.4%; ≥ 75 years: 28.2% Sex: before 1,232,046 (47.3%) male; after 714,346 (48.1%) male Setting: 232 public hospitals across NSW, Australia (principal group: > 25,000 admissions; major hospitals: > 10,000 and < 25,000 admissions; district: > 2000 and < 10,000 admissions; hospitals: < 2000 admissions; all other public health facilities including subacute and residential: 5,635,947
Interventions	Intervention: standardised, hospital‐wide RRT and single parameter EWS Team composition: ICU clinicians Timeline: baseline 2007–2009; run in 2010; after 2011–2013 Comparison: retrospective data (admissions database)
Outcomes	Primary outcomes: IHCA rate (per 1000 admissions), IHCA related mortality rate Hospital mortality rate (per 1000 admissions), failure to rescue, deaths in low mortality diagnostic groups, 1 year postmortality of IHCA
Notes	Clinical Excellence Commission implemented a standardised RRS in the state of NSW in January 2010. Funded by National Health and Medical Research Council of Australia.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Davis 2015.

Study characteristics
Methods	2 centre controlled before‐after study
Participants	Participants: inpatients across all units Age: NR Sex: NR Setting: primary medical centre (392 beds) and sister campus (119 beds) (134,093 participants), US
Interventions	Intervention: RRT and single‐parameter EWS Team composition: dedicated critical care nurse and respiratory therapist. The third member of the team was the unit charge nurses who was not a dedicated primary responder but acted only if the response was activated in their specific unit. Timeline: non‐ICU CPA: July 2005–June 2011; hospital discharge and mortality: July 2006–June 2011; Code Blue activation: July 2005–June 2011 Comparison: retrospective baseline data (electronic records)
Outcomes	Primary outcomes: number and year‐over‐year Code Blue and RRT activations over time, hospital discharge (per 1000 discharges), overall hospital mortality (%), patient acuity (CMI), yearly incidence of non‐CPAs, incidence of ICU CPAs
Notes	No funding stated
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Haegdorens 2018.

Study characteristics
Methods	Stepped wedge cluster randomised controlled trial
Participants	Participants: inpatients on 28 wards Age: intervention: mean 59.9 (SD 18.2) years; control: mean 58.9 (SD 18.6) years Sex: intervention: 35,389 (51%) male; control: 34,267 (49%) male Setting: 7 hospitals (> 850 admissions per year) across Belgium (69,656 participants)
Interventions	Intervention: MET and NEWS Team composition: NR Timeline: baseline (T0) 4 months; postintervention period (T1–T4) 16 months Control: 28 matched and paired surgical/medical wards moved from control (standard care) to implementation via training period
Outcomes	Primary outcomes: unexpected death, cardiac arrest with CPR, unplanned ICU admission (per 1000 admissions)
Notes	Belgium Federal Government sponsorship. Trial registration: NCT01949025
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Comment: 1 surgical and 1 medical ward per hospital were randomly paired and assigned as a block to the intervention. In total, 56 wards were enrolled and randomly allocated to 4 groups using computerised randomisation which as performed by KW who was not involved in the further conduct of the study.
Allocation concealment (selection bias)	Low risk	Comment: communication with authors indicated allocation concealment process was undertaken by KW using randomised computer process. Trial registration indicated allocation as randomised.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: unable to blind staff as received training/education prior to implementing NEWS/SBAR.
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Comment: hospitals were blinded for the collection date of comorbidity. Databases from each hospital and used a standardised electronic checklist to collect objective data (unexpected death, cardiac arrest, unplanned ICU admission). The researchers reviewed each patient record in case of a crude outcome Indicator. When uncertain 2 independent researchers (FH and MM) agreed.
Incomplete outcome data (attrition bias) All outcomes	High risk	Quote: "… seven hospitals were excluded from the analysis because of unavailable data". Comment: patients with incomplete data were excluded prior to data analysis (Figure 1 Consort trial profile, p 4).
Selective reporting (reporting bias)	Low risk	Comment: no evidence of selective reporting. All outcomes stated were reported.
Other bias	Unclear risk	Comment: potential risk of contamination across wards.

Hillman 2005.

Study characteristics
Methods	Prospective cluster randomised controlled trial
Participants	Participants: inpatients on general wards Age: intervention: mean 55.4 (SD 19.9) years; control: mean 56.9 (SD 20.8) years Sex: intervention: 33,965 (50%) male; control: 26,775 (47%) male Setting: 23 hospitals in Australia, 12 intervention hospitals (68,376 participants, median 364 bed) and 11 control hospitals (56,756 participants, median 315 bed)
Interventions	Intervention: hospital‐wide MET and single parameter calling criteria Team composition: staff designated to form the MET varied between participating hospitals, with a minimum of 1 doctor and a nurse from ED or ICU to meet protocol requirement Timeline: 2‐month baseline period, 4‐month implementation period and 6‐month after period in both control and intervention hospitals Control: randomised controlled hospitals did not receive MET education and cardiac arrest teams continued unchanged during the implementation and study period
Outcomes	Primary outcome: composite outcome of the incidence (events divided by the number of eligible patients admitted to the hospital during the study period) of cardiac arrests without a pre‐existing NFR order, unplanned ICU admission, and unexpected deaths (deaths without a pre‐existing NFR) (per 1000 admissions) Secondary outcomes: individual patient data (cardiac arrests without pre‐existing NFR, unplanned ICU admissions, unexpected deaths and length of hospital stay per 1000 admissions)
Notes	If a patient had > 1 event during their hospital stay, only 1 event was included in the composite measure. Funded by grants from the Australian National Health and Medical Research Council, the Australian Council for Quality and Safety in Healthcare, and the Australian and New Zealand Intensive Care Foundation.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Comment: participating hospitals identified via yearbook and those which met the criteria were randomised using SAS version 6.12 blocked by the number of beds and stratified by teaching status.
Allocation concealment (selection bias)	Low risk	Quote: "… an independent statistician (who had no other involvement in the study) randomly assigned hospitals to receive standardised MET implementation or to be controls" (p 2091). Comment: randomisation was concealed from investigators and hospitals.
Blinding of participants and personnel (performance bias) All outcomes	High risk	Comment: blinding unfeasible as 4‐month implementation period (MET educational strategy, regular reminders, poster, name badges) in intervention hospitals. Comment: no blinding, but outcome unlikely to be influenced
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Comment: objective outcomes measured (cardiac arrest, ICU admissions, unexpected death). Optical automated scanning data entry.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: no hospitals lost to follow‐up (Figure Trial Profile, p 2093).
Selective reporting (reporting bias)	Low risk	Comment: protocol unavailable but no evidence of selective reporting.
Other bias	High risk	Quote: "Control hospitals did not receive any education about the MET at any stage' (pg 2091); 'study was not publicised in the control hospitals" (p 2092). Comment: there was potential for risk of contamination as the benefits of the MET system were widely reported in the media, which may have impacted on the behaviour of the control hospitals.

Jeddian 2016.

Study characteristics
Methods	Stepped wedge cluster randomised trial
Participants	Participants: all patients on 13 adult general wards Age: intervention: mean 43 (SD 19) years; unexposed: mean 44 (SD 20) years Sex: intervention: 4266 (39%) male; unexposed: 3732 (48%) male Setting: university and public teaching hospital (18,682 participants, 800 bed), Iran
Interventions	Intervention: CCOS and single parameter calling criteria Team composition: 6 experienced ICU nurses Timeframe: baseline for 3 periods (12 weeks), roll‐out 2 wards every 2 periods (6 steps of 8 weeks each), transition phase for each ward (8 weeks), and postintervention data collection for 3 periods (12 weeks). Total of 18 periods (72 weeks) Control: 13 wards moved from control (usual care) to intervention via training period
Outcomes	Primary outcomes: in‐hospital mortality (%), CPR (number, %) Secondary outcomes: length of stay (median days), ICU admission (%)
Notes	No funding reported. Trial registration: IRCT201107187053N1
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Low risk	Quote: "Randomisation was carried out at a fixed point in time independent of the trial team" (p 214).
Allocation concealment (selection bias)	Low risk	Quote: "… information on ward sequence was revealed 2 to 3 days before start of the transition period" (p 214).
Blinding of participants and personnel (performance bias) All outcomes	High risk	Quote: "The intervention was delivered without blinding" (p 214).
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Quote: "Mortality and length of stay data were obtained from the hospital electronic information systems. Data on cardiopulmonary resuscitation and admissions to the intensive care unit were obtained from nursing office and CCOT records by the CCOT in exposed wards and by the independent data team in unexposed wards. For these outcomes, data collection was, therefore, not blind to exposure status". Comment: objective outcome measures recorded by staff in exposed wards and independent data team in unexposed wards.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Quote: "The primary analysis was by intention to treat … For the fully adjusted analysis, less than 2% of patients had incomplete data so the missing data methods were not warranted" (p 214).
Selective reporting (reporting bias)	Low risk	Comment: protocol unavailable and all outcomes stated are reported.
Other bias	Unclear risk	Comment: potential risk of contamination across wards.

Lighthall 2010.

Study characteristics
Methods	Single‐centre controlled before‐after study
Participants	Participants: adult inpatients in all acute care settings of an American veteran hospital Age: before: 65.2 (SEM 0.08) years; after 65.5 (SEM 0.08) years Sex: before 96% male; after: 96% male Setting: tertiary medical centre (240 bed), US
Interventions	Intervention: high capability MET and single parameter EWS Team composition: physician led MT Timeframe: before 9 months and 27 months after (analysed in 9‐month blocks); mortality 27 months after Comparison: prospective baseline cardiac arrest data 9 months before, retrospective baseline mortality data 3.5 years before
Outcomes	Primary outcomes: mortality rates (per 100 discharges), cardiac arrests rates (per 1000 discharges)
Notes	Cardiac arrests in the ED and areas not served by the MET were not included in the analysis. No funding reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Ludikhuize 2015.

Study characteristics
Methods	Prospective, pragmatic before‐after multicentre trial
Participants	Participants: adult patients admitted to COMET study wards Age: before: mean 62.2 (SD 18) years; after: mean 62.3 (SD 18) years Sex: before: 49.2% male; after 50.1% male Setting: 2 large university hospitals (882–1000 beds), 8 teaching hospitals (359–1070 beds), 2 regional hospitals (290–325 beds) (54,479 participants; Table 3), the Netherlands
Interventions	Intervention: RRT and MEWS Composition: the RRT included both an ICU nurse and a physician who was at least trained in fundamental critical care Timeframe: baseline 5 months, MEWS and SBAR implementation (7 months), RRT implementation (12 months), after 5 months (same months of year as the before period) Comparison: prospective baseline data
Outcomes	Primary outcome: composite endpoint of cardiopulmonary arrest, unplanned ICU admission or death per 1000 admissions Secondary outcomes: individual patient data (cardiopulmonary arrest, unplanned ICU admission, or death per 1000 admissions)
Notes	Patients who were readmitted to the hospital were not excluded from the analysis. ICU admission did not include medium care or other high‐dependency units. Intensive care was defined according to the criteria from the Dutch National Intensive Care Evaluation registry. No funding reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Menon 2018.

Study characteristics
Methods	Prospective before‐after study
Participants	Participants: all patients Age: before: 11 at 18–40 years, 30 at 41–70 years, 10 at > 70 years; after: 6 at 18–40 years, 41 at 41–70 years, 16 at > 70 years Sex: before: 35 (68.6%) male; after 40 (66%) male Setting: tertiary academic hospital (28,053 participants, 226 bed), India
Interventions	Intervention: physician‐led MET and AWC Team composition: MDT lead by internal medicine, charge nurse, nursing supervisor, phlebotomist, respiratory therapist, ECG technician Timeframe: pre‐implementation phase January–March 2013 (AWC and MET), 1‐month education, before period April 2013–March 2014, after April 2014–March 2016 Comparison: prospective baseline data
Outcomes	Primary outcomes: non‐ICU cardiac arrests (per 1000 admissions),  Code Blue mortality (per 1000 admissions) Secondary outcomes: MET activations and process (number), staff questionnaire
Notes	No funding reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

Priestley 2004.

Study characteristics
Methods	Prospective stepped wedge randomised trial
Participants	Participants: all patients admitted to 16 acute adult wards Age: intervention: mean 65.2 (95% CI 64.3 to 66.2) years; control: mean 57.4 (95% CI 56.3 to 58.5) years Sex: intervention: 804 (54.7%) male; control: 611 (43.1%) male Setting: general hospital (6481 participants, 800 bed), UK
Interventions	Intervention: CCOT and PAR score Team composition: nurse consultant with a team of experienced nurses and medical support as required Timeframe: 4 weeks' training each ward after which CCOS was fully operational, phased implementation over 32‐week period Control: control wards moved from control to intervention wards via the training period
Outcomes	Primary outcome: rate of in‐hospital deaths (number), length of hospital stay (mean)
Notes	A small number of participants were admitted to the hospital more than once during the study period and were included more than once in the sample. Funded by the York Research Innovation Fund (York Hospitals NHS Trust).
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	Comment: the process of randomisation for the ward pairings and order of introduction to the 4 groups of wards was not specified. Probably done. Quote: "One from each pair of wards was randomised to the earlier phase of outreach introduction"; "The order of introduction to the four groups of wards was also randomly determined".
Allocation concealment (selection bias)	Unclear risk	Comment: randomisation was done by a study investigator and the method of allocation is unclear. Quote: "Randomisation was done by DR alone, based on ward pairings and risk estimates provided by the rest of the study team".
Blinding of participants and personnel (performance bias) All outcomes	High risk	Quote: "No blinding was possible".
Blinding of outcome assessment (detection bias) All outcomes	Low risk	Comment: objective outcomes measured that were unlikely to be influenced by lack of blinding, e.g. in‐hospital mortality, length of hospital stay.
Incomplete outcome data (attrition bias) All outcomes	Low risk	Comment: patients with incomplete data excluded prior to data analysis. All patients were included in the analysis (data set 1).
Selective reporting (reporting bias)	High risk	Comment: authors stated findings were confirmed using data set 3 but no details or data for data set 3 were provided to confirm findings. No protocol available and all outcomes stated are reported.
Other bias	Unclear risk	Comment: potential risk of contamination across wards.

Rothberg 2012.

Study characteristics
Methods	Interrupted time series
Participants	Participants: all hospitalised patients Age: NR Sex: NR Setting: tertiary care referral centre (154,382 admissions; 670 bed), US
Interventions	Intervention: hospitalist‐led MET and single parameter EWS Team composition: critical care nurse, respiratory therapist, intravenous therapist and patient's physician or ICU physician as back up team member Timeframe: before 24 months (January 2004–December 2006), implementation period first quarter and second quarter 2006, after 41 months (July 2006–December 2009) Control: retrospective data
Outcomes	Primary outcomes: MET calls and process (n), Cardiac arrests (per 1000 admissions), Code deaths (per 1000 admissions), hospital mortality (per 1000 admissions)
Notes	A separate "code" team for cardiovascular arrests included ICU medical resident and intern, critical care nurse, anaesthetist, respiratory therapist, staff nurse and house supervisor. No funding reported.
Risk of bias
Bias	Authors' judgement	Support for judgement
Random sequence generation (selection bias)	Unclear risk	See Table 4.

APDC: Admitted Patient Data Collection; AWC: Amrita Warning criteria; CCOT: critical care outreach team; CCU: coronary care unit; CMI: Case Mix Index; COMET: Cost and Outcomes analysis of MET; CPA: cardiopulmonary arrest; CPR: cardiopulmonary resuscitation; ECG: electrocardiograph; ED: emergency department; EWS: early warning system; ICU: intensive care unit; ICUON: intensive care unit outreach nurse; IHCA: in‐hospital cardiopulmonary arrest; MDT: multidisciplinary team; MEC: Medical Emergency Criteria; MET: medical emergency team; MEWS: modified early warning system; NEWS: National Early Warning System; NFR: not‐for‐resuscitation; NR: not reported; RRS: rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Recommendation communication tool; SD: standard deviation; SEM: standard error of the mean.

Characteristics of excluded studies [ordered by study ID]

Study	Reason for exclusion
Al Qahtani 2013	Wrong study design.
Aneman 2015	Before‐after study with historical control group.
Austin 2014	Wrong study design.
Bannard‐Smith 2016	Wrong study design.
Barrett 2010	Wrong study design.
Barwise 2014	Wrong study design.
Barwise 2016	Wrong study design.
Beitler 2010	Before‐after study with historical control group.
Beitler 2011	Before‐after study with historical control group.
Bergamasco 2017	Wrong setting.
Bingham 2015	Wrong study design.
Bittman 2020	Wrong intervention.
Blotsky 2012	Wrong study design.
Bucknall 2013	Wrong study design.
Butcher 2010	Wrong study design.
Cabrini 2012	Wrong study design.
Calzavacca 2010	Wrong study design.
Cerchiari 2010	Wrong outcome.
Chan 2008	Wrong study design.
Chen 2009	Post‐hoc analysis.
Chen 2014a	Wrong study design.
Chen 2014b	Wrong study design.
Chen 2015	Retrospective data from MERIT study.
Choi 2011	Wrong study design.
Churpek 2017	Wrong intervention (qSOFA vs EWS).
Considine 2019	Wrong design.
Danesh 2012	Wrong study design.
Danesh 2019	Wrong intervention.
Fernando 2018	Wrong design.
Fernando 2019	Wrong design.
Frost 2015	Wrong study design.
Gagne 2018	Wrong intervention.
Galhotra 2010	Wrong study design.
Gao 2007	Wrong study design.
Gilmore 2014	Wrong study design.
Goncales 2012	Wrong study design.
Gray 2011	Wrong study design.
Harm 2012	Wrong study design.
Harrison 2010	Before‐after study with historical control group.
Hassan 2015	Wrong study design.
Hatlem 2011	Wrong study design.
Hellervik 2012	Wrong study design.
Jaderling 2013	Wrong study design.
Javeri 2013	Wrong study design.
John 2010	Wrong study design.
Jokela 2015	Wrong study design.
Jones 2005	Before‐after study. Historical controls.
Jones 2008	Wrong study design.
Karpman 2013	Wrong study design.
Karvellas 2012	Wrong study design.
Kawaguchi 2015	Wrong study design.
Kim 2013	Wrong study design.
Kodama 2013	Wrong study design.
Kodama 2014	Wrong study design.
Kollef 2014	Wrong intervention.
Konrad 2010	Wrong study design.
Laurens 2011	Cohort before‐after study in single hospital.
Lawless 2013	Wrong study design.
Lee Ekblad 2012	Wrong study design.
Le Guen 2015	Wrong study design.
Liberti 2012	Wrong study design.
Lim 2011	Wrong study design.
Lorencio Cardenas 2014	Wrong study design.
Ludikhuize 2013	Wrong intervention.
Ludington 2011	Wrong study design.
Magnuson 2015	Wrong study design.
Martin 2009	Wrong study design.
Martin 2017	Wrong design.
McDonnell 2013	Wrong outcomes.
Mitchell 2010	Wrong study design.
Mullany 2016	Wrong design.
O'Connell 2016	Wrong design.
Palomba 2014	Wrong study design.
Panico 2011	Wrong study design.
Petersen 2016	Wrong intervention (EWS only).
Pirret 2015	Wrong study design.
Pisa 2012	Wrong study design.
Prchalova 2013	Wrong study design.
Radeschi 2014	Wrong study design.
Rashid 2014	Wrong study design.
Reilly 2010	Wrong study design.
Sabahi 2012	Wrong study design.
Salvatierra 2014	Wrong study design.
Santamaria 2011	Wrong study design.
Sebat 2018	Wrong design.
Simmes 2012	Wrong study design.
Simmes 2015	Wrong study design.
Smith 2013	Wrong study design.
So 2018	Wrong design.
Stelfox 2012	Wrong study design.
Stewart 2014	Wrong study design.
Sutherasan 2018	Wrong design.
Taguti Pda 2013	Wrong study design.
Tridente 2015	Wrong study design.
Trottier 2010	Wrong study design.
White 2016	Wrong design.
Yu 2016	Wrong intervention

EWS: early warning system; MERIT: Multi‐center ESG Randomized Interventional Trial; qSOFA: quick sepsis‐related organ failure assessment.

Differences between protocol and review

We updated the review title to reflect international terminology, refined the search strategy to increase precision and updated the eligibility criteria of study designs to align with  Effective Practice and Organisation of Care (EPOC) study design inclusion criteria. This updated review includes the main author of initial review (JM) with the addition of three new authors (LR, PVB, DF).

We updated the databases searched and the search strategies. We revised the types of outcome measures to differentiate between primary and secondary outcomes, reflect the change in the use of outcome terms and to include 'composite outcomes' given this was a decision taken at the initial review and not subsequently clarified. We added a description of the 'Unit of analysis' under the methods section to reflect Cochrane's recommendations (Higgins 2011) and Cochrane EPOC (EPOC 2017e) for conducting and reporting analysis in reviews and added a summary of findings table and GRADE ratings, which were not a requirement when we published the original protocol and review.

We revised the data analysis for this updated review from the original protocol. We planned to perform quantitative analysis using an intention‐to‐treat analysis, utilising risk ratio (RR) with 95% confidence intervals as our measure of effect for both individual and pooled effect estimates, but, due to the heterogeneity of outcome measures, this was not possible. We planned to undertake meta‐analysis by computing a pooled RR using a random effects model and adjust study‐level findings for the study design effect by adopting the approach recommended by Cochrane (design effect = 1 + (M – 1) × ICC where M is the mean cluster size and ICC is the intracluster correlation coefficient), but, due to statistical and clinical heterogeneity across studies, this was not possible.

Contributions of authors

JM led the writing of the protocol and all previous review authors (Fiona Alderdice, Rob Fowler, Atul Kapila, Marianne Moutray) provided comment and feedback. 

For the updated review

JM and PVB screened records for eligibility, extracted data and assessed the certainty of randomised trials.

LR was an independent arbitrator and resolved disagreement. 

PVB and LR independently assessed and agreed the quality of non‐randomised studies. 

JM undertook the analysis, interpreted the results and wrote the review. 

PVB and LR contributed to the interpretation of results and writing of the review. 

DF provided statistical advice.

Sources of support

Internal sources

• Queens University Belfast, UK

External sources

• The initial review was funded by a Research and Development Office and Health Research Board All Ireland Cochrane Fellowship, UK

Declarations of interest

JM: none.

PVB: none.

DF: none.

LR: none.

Edited (conclusions changed)