Table 2.

Studies conducted in Chicago hospitals.

Author and year	Study design	Setting	Study aim	Model type or used	Prediction event	Key findings
Churpek et al, 2012 [31]	RSa: tool development and evaluation	47,427 patients, 1 hospital, 2008-2011	To develop a CARTb score and compare with the MEWSc	Logistic regression	CAd or transfer to ICUe	The CART score more accurately predicted cardiac arrest than the MEWS. Model AUCf 0.84
Churpek et al, 2013 [29]	RS: tool development	59,643 patients, 1 hospital, 2008-2011	To assess the impact of outcome selection on the performance of prediction algorithms	Logistic regression (4 models)	CA, transfer to ICU, death, all combined	Mortality is the easiest outcome to predict (AUC range 0.73-0.82), and ICU transfer is the most difficult.
Churpek et al, 2014 [30]	RS: tool development	59,301 patients, 1 hospital, 2008-2011	To derive and validate a prediction model for CA	Logistic regression	CA and transfer to ICU	The model can simultaneously predict the risk of CA and ICU transfer and was more accurate than ViEWSg. Model AUC 0.88 for CA, 0.77 for ICU
Churpek et al, 2014 [33]	RS: tool development	269,999 admissions, 5 hospitals, 2008-2011	To develop and validate eCARTh score using commonly collected EMRi data	Survival analysis	CA, transfer to ICU, or death	eCART score was more accurate than MEWS for detecting CA, ICU transfer, or death. Model AUC 0.83 for CA, 0.75 for ICU transfer, 0.93 for death and 0.77 all combined
Somanchi et al, 2015 [56]	RS: tool development	133,000 patients, 4 hospitals, 2006-2011	To develop a prediction model for Code Blue, using EMR data, and compare with MEWS	SVMj and logistic regression	Code blue event in the next x hours	The model was able to predict Code Blue with ~80% recall and 20% false positive rate 4 hours ahead of the event. It out-performed MEWS.
Churpek et al, 2016 [32]	RS: tool development and evaluation	269,999 patients, 5 hospitals, 2008-2013	To compare the accuracy of different techniques for detecting clinical deterioration on the wards	Logistic, decision trees, SVM, K-NNk, neural net, MEWS	CA, transfer to ICU, or death	This multicenter study showed that several machine learning methods can more accurately predict clinical deterioration than logistic regression.
Kang et al, 2016 [66]	Prospective study: feasibility study	3889 admissions, 3 wards, 2013-2014	To assess the feasibility of a real-time risk stratification tool	eCART	CA, transfer to ICU, or death	eCART score identified more CA and ICU transfers, many hours in advance, compared with standard RRTl activation.
Green et al, 2018 [37]	RS: tool evaluation	107,868 admissions, 5 hospitals, 2008-2013	To compare the BTFm calling criteria to MEWS, NEWSn and eCART score	BTF, NEWS, MEWS, eCART	CA, transfer to ICU, or death (24 hours)	eCART was more accurate than BTF, MEWS, NEWS for predicting the composite outcome of CA, ICU transfer and death. eCART AUC 0.80 (0.79-0.80)
Bartkowiak et al, 2019 [25]	RS: tool evaluation	32,537 admissions, 1 hospital, 2008-2016	To assess the accuracy of three EWSo postoperatively	NEWS, MEWS, eCART	CA, ICU transfer or ward, or death	The eCART score was the most accurate followed by MEWS. Maximum respiratory rate was the most predictive vital sign.
Mayampurath et al, 2019 [48]	RS: tool development	115,825 admissions, 1 hospital, 2008-2016	To develop a model from visual timelines to predict mortality	Convolutional neural network	Death	The model was more accurate than MEWS and SOFAp, validation model AUC 0.91, and visual timelines enabled interpretation of a deep neural network.

^aRS: retrospective study.

^bCART: Cardiac Arrest Risk Triage.

^cMEWS: Modified Early Warning Score.

^dCA: cardiac arrest.

^eICU: intensive care unit.

^fAUC: area under the receiver operating characteristic curve.

^gViEWS: VitalPAC Early Warning Score.

^heCART: electronic Cardiac Arrest Risk Triage.

ⁱEMR: electronic medical record.

^jSVM: support vector machine.

^kK-NN: K-nearest neighbors.

^lRRT: rapid response team.

^mBTF: Between the Flags.

ⁿNEWS: National Early Warning Score.

^oEWS: early warning score.

^pSOFA: Sequential Organ Failure Assessment.