Table 3.

Retrospective studies evaluating scoring tools.

Author, year, and country	Settings	Study aim	Scoring tools	Prediction event	Key findings
Lighthall et al, 2009 [67], United States	1089 patients, 1 hospital, 2006	To evaluate vital signs and association with critical events	METa call criteria	CAb, ICUc transfer or death	Even a single recording of an abnormal vital sign increases the risk of critical events in hospitalized patients.
Huh et al 2014, [41], Korea	3030 events, 1 hospital, 2008-2010	To evaluate the efficacy of screening triggered alerts for MET management	Medical alert system criteria	MET activation	The automatic alert system triggers, along with a skilled intervention team, were successful in managing the MET
Romeo-Brufau et al, 2014 [54], United States	34,898 patients, 2 hospitals, 2011	Comparative analysis of the performance of common EWSd methods and how they would function if automated	MEWSe, SEWSf, GMEWSg, Worthing, ViEWSh, NEWSi	Resuscitation call, RRSj activation or ICU transfer	The evaluated scores did not offer good predictive capabilities for an automated alarm system. Positive predictive values ranged from <0.01-0.21, and sensitivity ranged from 0.07-0.75.
Yu et al, 2014 [59], United States	328 cases, 328 controls, 1 hospital, 2009-2010	To compare the ability of 9 risk prediction scores in detecting clinical deterioration among non-ICU ward patients	SOFAk, PIROl, ViEWS, SCSm, MEDSn, MEWS, SAPS IIo, REMSp, APACHE IIq	Critical care consult, ICU transfer or death	Prediction scores can be used to estimate a ward patient’s risk of clinical deterioration, with good discriminatory ability comparable with that of existing track-and-trigger systems. 0-12 hours before clinical deterioration, 7 of 9 scores performed with acceptable discrimination (AUCr>0.70).
Wengerter et al, 2018 [57], United States	217 cases, 868 controls, 1 hospital, 2013-2015	To evaluate whether Rothman Index variability can predict RRTs activation in surgical patients	Rothman Index	RRT activation, mortality	Rothman Index variability predicted likelihood of RRT activation.
Bedoya et al, 2019 [26], United States	85,322 patients, 2 hospitals, 2014-2016	To determine the effectiveness of NEWS implementation on predicting and preventing patient deterioration	NEWS	ICU transfer or death	No change after implementing NEWS. At both academic and community hospitals, NEWS had poor performance characteristics and was generally ignored by nursing staff.
Heller et al [39], 2020, Germany	3827 patients, 2 wards, 2016-2017	To develop a prediction model for Code Blue, using EMRt data, and compare with MEWS	MEWS with paging functionality	CA or ICU transfer	The rate of CA and ICU transfers significantly decreased after implementing MEWS with paging functionality.

^aMET: medical emergency team.

^bCA: cardiac arrest.

^cICU: intensive care unit.

^dEWS: early warning score.

^eMEWS: Modified Early Warning Score.

^fSEWS: Standardized Early Warning Score.

^gGMEWS: Global Modified Early Warning Score.

^hViEWS: VitalPAC Early Warning Score.

ⁱNEWS: National Early Warning Score.

^jRRS: rapid response system.

^kSOFA: Sequential Organ Failure Assessment.

^lPIRO: Predisposition, Infection, Response, Organ, Dysfunction Score.

^mSCS: simple clinical score.

ⁿMEDS: Mortality in Emergency Department Sepsis.

^oSAPS II: Simple Acute Physiology Score II.

^pREMS: Rapid Emergency Medicine Score.

^qAPACHE II: Acute Physiology and Chronic Health Evaluation Score II.

^rAUC: area under the receiver operating characteristic curve.

^sRRT: rapid response team.

^tEMR: electronic medical record.