Abstract
Aim
Maintaining rapid response team (RRT) response quality is difficult. A system that supports RRT assessment could potentially contribute to medical safety. Although rapid response system (RRS) triggers have been well‐studied, studies on the prediction models of short‐term prognosis after RRS activation are scarce. We aimed to develop a model to predict short‐term outcomes after RRS activation using machine learning.
Methods
This retrospective cohort study used the In‐Hospital Emergency Registry in Japan, a multicentre RRS online registry. We collected data on patient demographics, treatment before RRS, RRT calls, and physiological parameters. The outcome was death within 24 h after RRS calls or unplanned transfers to an intensive care unit. To develop the eXtreme Gradient Boosted Tree Classifier (XGB) and Random Forest (RF) algorithms, a logistic regression (LR) algorithm was used. For model comparison, receiver‐operating area under the curve (AUC) was evaluated and compared with those of the National Early Warning Score (NEWS) and Modified Early Warning Score (MEWS).
Results
5414 cases were included in the study. The outcome occurred in 28.4% of the cases. The XGB model showed the highest AUC (0.798) compared to the RF model (0.796), LR model (0.785), NEWS (0.696), and MEWS (0.660). The most weighted feature in the XGB model was doctor activation, followed by hypotension as the activation criteria and usage of oxygen.
Conclusions
We developed the first machine learning model for short‐term prognosis after RRS. This model has the potential to support decision‐making by RRT.
Keywords: early warning score, machine learning, medical emergency team, rapid response system
In this study, we developed a machine learning model for short‐term prognosis after RRS activation. The XGBoost model showed better prediction than widely used EWSs. The most weighted feature in the XGB model was doctor activation, followed by hypotension as the activation criteria and usage of oxygen.
INTRODUCTION
The rapid response system (RRS) has become the standard international system for preventing death from unexpected cardiac arrests. 1 , 2 , 3 , 4 Recently, the development of better prediction models for adverse events has become an active research area related to RRS. Machine learning models show better prediction than the widely used early warning scores (EWSs), such as the National EWS (NEWS) or the modified EWS (MEWS). 5 , 6 , 7 , 8 , 9 , 10 , 11
Although developing better track‐and‐trigger systems is important for improving the quality of in‐hospital emergency systems, saving patients from preventable death requires adequate treatment provided by the medical emergency team (MET) and rapid response team (RRT). The importance of accurately assessing severity and providing care levels that match that severity has been highlighted. 12 , 13 Ideally, once the RRS is activated, patients should receive the best treatment. However, maintaining the quality of RRT/MET response at a high level can be challenging because of less trained new employees and a lack of human resources.
While the assessment by frontline medical staff is important, the use of a model to predict short‐term prognosis after RRS may prove beneficial in supporting the METs/RRTs while making decisions regarding the transportation of patients to the intensive care unit (ICU) or talking about end‐of‐life issues. In addition, when the MET/RRT is led by resident doctors, advanced practice providers (APPs), and registered nurses (RNs), they need to consult a senior physician or employ the escalation protocol if they encounter critically ill patients. In such cases, a model may help the MET/RRT make decisions with confidence. Although many studies have reported on the criteria before RRS calls, studies on the threshold for escalating protocols after RRS calls are limited. Thus, the factors associated with poor prognosis in those requiring RRS activation remain unclear.
RRS is a standard system to ensure patient safety in advanced countries such as the U.S., the U.K. and Australia. 1 , 3 , 14 , 15 , 16 , 17 However, some countries like Japan do not have well‐developed in‐hospital emergency systems and thereby struggle with a low RRS call rate or malfunctional RRS. 18 Automatic track‐and‐trigger systems using traditional EWS or machine learning EWS are expected to be a solution for increasing the call rate. Mature facilities have over 25 RRS cases every 1000 admissions. 2 , 19 , 20 , 21 , 22 , 23 This makes training healthcare providers to handle the increase in RRS calls challenging. Thus, a system that supports the assessment of inexperienced MET/RRT responses could potentially improve medical safety.
Therefore, this study aimed to develop a model for predicting short‐term outcomes after RRS activation using machine learning, which was recently reported to provide good predictions of patient deterioration. 5 , 6 , 8 , 24 , 25
METHODS
Study design and participants
This retrospective cohort study used the In‐Hospital Emergency Registry in Japan (IHER‐J), a unique Japanese multicentre RRS online registry created in 2014 with the goal of reducing in‐hospital cardiac arrests and providing high‐quality medical care using the RRS. 26 The study population consisted of RRS cases registered on the IHER‐J between November 2017 and July 2021. We excluded patients who were <16 years old, had incomplete outcome data, and required cardiopulmonary resuscitation upon RRT/MET arrival. The excluded cases are shown in Figure 1. All data used in this analysis were retrieved from the IHER‐J.
FIGURE 1.
Flow chart of study. a: death within 24 hours after RRS calls or unplanned ICU transfer.
This study was approved by the Institutional Research Board Committee of the St. Marianna University Hospital (#5552, Kawasaki, Japan). The need for written informed consent was waived because this study used an anonymous case registry and was conducted by a patient safety committee.
Outcomes
The outcome of this study was a composite of death within 24 h after RRT/MET call or unplanned transfers to an ICU soon after RRT/MET intervention. This outcome definition is consistent with previous RRT/MET and NEWS research. 5 , 24 , 27
Collected variables
We collected the variables obtained on the timing of RRT/MET arrival. The variables used for this analysis were as follows: patient demographic data (age, sex, admission status, primary disease, admitted department, primary team, consciousness level, cerebral performance category, and code status), treatment before RRT/MET (oxygen supplementation, ICU discharge within 72 h, anaesthesia within 24 h, and operation within 7 days), data related to the RRT/MET call (activator, activated location, activated time, and activation criteria), and physiological parameters at the time of the RRT/MET call (consciousness level, systolic blood pressure, diastolic blood pressure, heart rate, respiratory rate, body temperature, and oxygen saturation). We used patient information at RRT/MET arrival as the data input to predict future outcomes of mortality and ICU transfer.
Feature selection
All collected variables were evaluated and coded as binary variables. For continuous variables, categorisations were determined based on the clinical meaning and data volumes. All the generated features were evaluated based on their correlations with the target outcome. The chi‐square test and calculation of sensitivity, specificity, positive predictive value (PPV), and negative predictive value were performed for each binary variable. Variables with fewer than 50 cases were omitted because of the possibility of inaccuracy. The most dominant or clinically normal range variables in each category were selected as default. The default variables were omitted from the model development. There were 156 initial features, and the final number of selected features was 130.
Model training
Once the features were created, we used DataRobot (Datarobot Inc., Boston, Massachusetts, MA, the U.S.) to train a wide range of 33 machine learning models and selected the best performing models using the area under the curve (AUC). DataRobot is a cloud‐based automated artificial intelligence (AI) platform with hundreds of machine learning algorithms that can be quickly applied to different types of users and data. This platform provides a complete set of testing, diagnosis, and evaluation tools for AI model development. All the models were trained on the ‘DataRobot.com’ cloud platform. The dataset was split into training (80%) and holdout (20%) sets. Through the model selection process, the eXtreme Gradient Boosted Tree Classifier (XGB) and Random Forest (RF) algorithms were selected as the final machine learning models. A logistic regression (LR) model was developed using the same data for comparison.
External model testing and model comparison
The holdout dataset (20%) was used for model testing. The AUC, sensitivity, and PPV were evaluated. Furthermore, sensitivity and PPV at low alert rates (5%, 10%, and 30%) were calculated, which is an important parameter for understanding model performance among patients who were categorised as having a higher risk of composite outcome.
To compare the accuracy of the model with widely used EWSs, AUCs were evaluated using the NEWS and MEWS. We calculated these EWSs using all datasets, and the missing data were imputed as normal values.
Feature impact
To understand the model, we measured the feature impact on the machine learning model by calculating the permutation feature importance, which is the increase in the prediction error of the model after permuting each feature.
Statistical analyses
All data manipulations, except machine learning model development, were carried out using the R software (version 3.6.1.17; R Foundation for Statistical Computing, Vienna, Austria).
RESULTS
Study population
A total of 5948 patients from 34 institutions who received RRS activations were registered during this period, and 5414 cases were eligible for this analysis (Figure 1). Patient characteristics are shown in Table 1. Four percent of the patients died within 24 h after the RRT/MET call, and an unexpected ICU transfer occurred in 25.4% of the cases. The outcome occurred in 28.4% of the cases (event group), and 71.6% of patients were without the outcome (non‐event group).
TABLE 1.
Patient characteristics.
| All (n = 5414) | Event a (n = 1540) | Non‐event (n = 3874) | |
|---|---|---|---|
| Age, median (IQR) | 74.0 (61.0–82.0) | 73.0 (63.0–81.0) | 74.0 (61.0–82.0) |
| Male (%) | 3298 (60.9) | 1129 (73.3) | 2169 (56.0) |
| Patient status (%) | |||
| Impatient | 4225 (78.0) | 1407 (91.4) | 2818 (72.7) |
| Outpatient | 1102 (20.4) | 133 (8.6) | 969 (25.0) |
| Other | 87 (1.6) | 0 (0.0) | 87 (2.2) |
| Primary disease b (%) | |||
| OHCA | 12 (0.2) | 4 (0.3) | 8 (0.2) |
| Trauma | 326 (6.0) | 45 (2.9) | 281 (7.3) |
| Infection | 339 (6.3) | 141 (9.2) | 198 (5.1) |
| Obstetrics | 56 (1.0) | 21 (1.4) | 35 (0.9) |
| Malignancy | 850 (15.7) | 303 (19.7) | 547 (14.1) |
| Medicine (cardiac) | 532 (9.8) | 128 (8.3) | 404 (10.4) |
| Medicine (non‐cardiac) | 1971 (36.4) | 635 (41.2) | 1336 (34.5) |
| Surgery (cardiac) | 226 (4.2) | 65 (4.2) | 161 (4.2) |
| Surgery (non‐cardiac) | 1272 (23.5) | 342 (22.2) | 930 (24.0) |
| Other | 379 (7.0) | 60 (3.9) | 319 (8.2) |
| Admitted department c (%) | |||
| Medical | 2124 (50.3) | 781 (55.5) | 1343 (47.7) |
| Surgical | 1762 (41.7) | 511 (36.3) | 1251 (44.4) |
| OB/GYN | 120 (2.8) | 51 (3.6) | 69 (2.4) |
| Psychiatric | 54 (1.3) | 9 (0.6) | 45 (1.6) |
| Other | 164 (3.9) | 54 (3.8) | 110 (3.9) |
| Consciousness d (%) | |||
| Alert | 3502 (64.7) | 910 (59.1) | 2592 (66.9) |
| Response to voice | 1158 (21.4) | 380 (24.7) | 778 (20.1) |
| Response to pain | 217 (4.0) | 76 (4.9) | 141 (3.6) |
| Unresponsive | 107 (2.0) | 46 (3.0) | 61 (1.6) |
| CPC e (%) | |||
| 1 | 3880 (71.8) | 1086 (70.7) | 2794 (72.2) |
| 2 | 792 (14.7) | 245 (16.0) | 547 (14.1) |
| 3 | 627 (11.6) | 172 (11.2) | 455 (11.8) |
| 4 | 103 (1.9) | 32 (2.1) | 71 (1.8) |
| 5 | 2 (0.0) | 1 (0.1) | 1 (0.0) |
| ICU discharge within 72 h (%) | 258 (4.8) | 98 (6.4) | 160 (4.1) |
| Anaesthesia within 24 h (%) | 327 (6.0) | 114 (7.4) | 213 (5.5) |
| Post operation state f (%) | 659 (12.2) | 186 (12.1) | 473 (12.2) |
| Emergency operation (%) | 167 (3.1) | 59 (3.8) | 108 (2.8) |
| Activator g (%) | |||
| Physician | 1238 (23.2) | 579 (38.2) | 659 (17.2) |
| Nurse | 3809 (71.3) | 914 (60.3) | 2895 (75.7) |
| Medical staff | 180 (3.4) | 17 (1.1) | 163 (4.3) |
| Other | 113 (2.1) | 5 (0.3) | 108 (2.8) |
| Code status h (%) | |||
| Full | 2914 (84.2) | 996 (88.2) | 1918 (82.3) |
| DNAR | 426 (12.3) | 101 (8.9) | 325 (13.9) |
| LOMT | 120 (3.5) | 32 (2.8) | 88 (3.8) |
| Consciousness at RRS activation i (%) | |||
| Alert | 2385 (46.7) | 467 (32.3) | 1918 (52.4) |
| Response to voice | 1668 (32.7) | 520 (35.9) | 1148 (31.4) |
| Response to pain | 441 (8.6) | 178 (12.3) | 263 (7.2) |
| Unresponsive | 614 (12.0) | 283 (19.5) | 331 (9.0) |
| Vital signs j (IQR) | |||
| Systolic blood pressure | 113.0 (88.0–140.0) | 108.0 (79.25–138.0) | 115.0 (92.0–140.0) |
| Diastolic blood pressure | 67.0 (53.0–82.0) | 64.0 (53.0–82.0) | 67.0 (54.0–82.0) |
| Heart rate | 94.5 (76.0–116.0) | 105.0 (76.0–122.0) | 90.0 (74.0–111.0) |
| Respiratory rate | 23.0 (18.0–30.0) | 26.0 (20.0–32.0) | 22.0 (18.0–28.0) |
| Oxygen saturation | 96.0 (91.0–98.0) | 94.0 (87–98) | 96.0 (93–98) |
| Body temperature | 36.9 (36.5–37.625) | 37.1 (36.5–38.125) | 36.8 (36.5–37.5) |
| Outcomes (%) | |||
| Death within 24 h | 216 (4.0) | – | – |
| Unexpected ICU transfer | 1377 (25.4) | – | – |
| Composite outcome | 1540 (28.4) | – | – |
Abbreviations: CPC, cerebral performance category; DNAR, do‐not‐attempt resuscitation; ICU, intensive care unit; IQR, interquartile range; LOMT, limitation of medical therapy; OB/GYN, obstetrics and gynaecology; OHCA, out of hospital cardiac arrest; RRS, rapid response system.
Event: death or ICU transfer within 24 h.
Multiple answer allowed.
n = 4224.
n = 4984.
n = 5404.
Post operation within 7 days.
n = 5340.
n = 3460.
n = 5108.
n = 4667 for systolic blood pressure; n = 4414 for diastolic blood pressure; n = 4712 for heart rate; n = 3827 for respiratory rate; n = 4307 for oxygen saturation; n = 3016 for body temperature.
The median age was 74 years, and 60.9% of the patients were male. Most of the patients were inpatients (78.0%). The most frequent primary disease in the patient population was non‐cardiac disease (36.4%), followed by non‐cardiac surgery (36.4%) and malignancy (15.7%). Of the patients, 71.8% had a cerebral performance category of 1. The percentage of cases that had post‐status of ICU discharge, anaesthesia, and operation before the outcome occurred was 4.8%, 6.0%, and 12.2%, respectively. In terms of code status, the full code and Do‐Not‐Resuscitate (DNR) order accounted for 84.2% and 12.3% of the cases, respectively. Compared to the non‐event group, the event group showed a lower rate of DNR order, a low rate of alert patients, lower systolic blood pressure, a higher heart rate, a higher respiratory rate, and lower oxygen saturation.
Model training and performance
All collected variables, including omitted variables and defaults, are shown in Table S1 with the univariable analysis, and sensitivity and positive predictive value of these variables are shown in Figure S1. The model performances are listed in Table 2. The XGBoost model showed the highest AUC of 0.798, compared to the RF (AUC, 0.796) and LR (AUC, 0.785) models. In terms of sensitivity and PPV at low alert rates, the XGB model showed better performance than RF and LR models. The sensitivity and PPV in the top 5% of low alert rates for the XGBoost, RF, and LR models were 0.140 and 0.782, 0.136 and 0.764, and 0.136 and 0.764, respectively (Table 3). The NEWS (AUC, 0.6963) and MEWS (AUC, 0.6597) models showed significantly lower AUCs than the XGB model.
TABLE 2.
Model comparison.
| Validation | Cross validation | Hold out | |
|---|---|---|---|
| XGB | 0.789 | 0.802 | 0.798 |
| Random forest | 0.791 | 0.795 | 0.796 |
| Logistic regression | 0.776 | 0.784 | 0.785 |
| NEWS | 0.696 | ||
| MEWS | 0.660 |
Abbreviations: MEWS, modified early warning score; NEWS, national early warning score; XGB, eXtreme Gradient Boosted Tree Classifier.
TABLE 3.
Sensitivity and positive predictive value at low alert rates.
| Population | Measure | XGB (%) | RF (%) | LR (%) |
|---|---|---|---|---|
| Top 5% | Positive cases | 4.0 | 3.9 | 3.6 |
| Sensitivity | 14.0 | 13.6 | 12.7 | |
| PPV | 78.2 | 76.4 | 70.9 | |
| Top 10% | Positive cases | 7.2 | 6.8 | 7.0 |
| Sensitivity | 25.3 | 24.0 | 24.7 | |
| PPV | 71.6 | 67.9 | 69.7 | |
| Top 30% | Positive cases | 10.1 | 9.8 | 9.4 |
| Sensitivity | 35.3 | 34.4 | 33.1 | |
| PPV | 66.5 | 64.6 | 62.2 |
Abbreviations: LR, logistic regression; PPV, positive predictive value; RF, random forest; XGB, eXtreme Gradient Boosted Tree Classifier.
Feature impact
The top 20 weighted features for developing the XGB model are shown in Figure 2. The most weighted feature was RRS calls by physicians, followed by hypotension as activation criteria, usage of oxygen, and desaturation as activation criteria. Outpatients, daytime calls, and oxygen saturation of 96%–98% were negative predictive weights for the outcome.
FIGURE 2.
Feature impact for the eXtreme Gradient Boosted Tree Classifier model, top 20 features. Feature impact shows which features are driving model decisions. The most important feature is always normalised to 1. AMS, altered mental status; BT, blood temperature; DBP, diastolic blood pressure; DNAR, do‐not‐attempt resuscitation; Dr, doctors; Pt., primary team; SBP, systolic blood pressure; SpO2, oxygen saturation. x axis: Feature impact. y axis: Features. 
, Positive impact for predicting outcome. 
, Negative impact for predicting outcome.
DISCUSSION
Usefulness of the model in short‐term prognosis after RRS activation
In this study, we developed a machine learning model for short‐term prognosis after RRS activation using Japanese hospital data. The XGBoost model, the best model, showed better prediction than NEWS and MEWS, which are the most widely used EWSs. Furthermore, we found that the features which were related to RRS activation had good predictive value, and these features were not included in the previous models. 7 , 8 , 24 , 27 , 28 The usefulness of EWS and machine learning models has been widely reported in predicting adverse events among all hospitalised patients. 7 , 8 , 24 , 27 , 28 However, these were used as activation triggers for RRS in all hospitalised patients. To the best of our knowledge, ours is the first study to develop a machine learning model for short‐term prognosis after RRS activation.
It has been reported that the RRTs/METs should not only decide whether to transfer a patient to the ICU but also accurately assess the severity and escalate care accordingly. 12 , 13 Additionally, it is necessary to discuss treatment goals with end‐of‐life patients. The use of prognostic models in support of assessments of RRTs/METs may contribute to medical safety through accurate provision and appropriate medical treatment. This approach is expected to prevent avoidable cardiac arrests and also improve the quality of death for end‐of‐life patients. This might be more useful for inexperienced physicians, APPs, and RNs.
Although our model is not ideal to detect all adverse events, it is worth using in clinical settings. The XGB model had the highest AUC; thus, using this model alone to identify patients who would have further adverse events might not be sufficient. However, compared to NEWS and MEWS, our machine learning model showed superior accuracy in predicting short‐term prognosis after RRS activation. Thus, as a decision‐support tool, this model may be useful in clinical practice.
Impact of predictive parameters
In this analysis, features different from those reported in previous studies were identified as important prognostic factors. Features that reflect the patient's medical condition, such as blood pressure, pulse rate, and respiratory rate, have been identified as highly influential factors in previous studies. 5 , 29 , 30 However, this analysis revealed that features related to RRS activation, such as activator occupation, reason for the call, and time of RRS call, were also important predictors. In previous studies, models have been built using data from all hospitalised patients. 6 , 8 , 12 , 24 , 25 However, in this study, we used data from patients who had RRS activation; thus, we could include features that were not used in previous studies.
Some might argue that similar features such as vital signs and activation criteria were included in the model. However, machine learning showed that information on RRS activation was more useful than vital signs in predicting adverse events. Although it might require some effort to set up the system, it is reasonable to use the information not only before RRT/MET activation, but also after RRT/MET arrival.
Strengths and limitations
The main strength of this study is that it is the first to develop a machine learning model for predicting short‐term prognosis after RRS activation using a multicentre database.
Despite the potential implications of its findings, this study has some limitations. Firstly, the study population did not include all patients who became sick in the hospital. The RRS activation rate in Japan is extremely low. 22 In this study, there was considerable variability in the number of cases registered by each facility, with many facilities reporting a low number of activations (Table S2). Therefore, if the activation rate of the RRS increases and the registry covers more patients in poor condition, different results may be obtained. Secondly, overfitting is a concern as machine learning and data from a Japanese multicentre registry were used for this analysis. Furthermore, not all Japanese hospitals participated, and the external validity needs to be carefully considered when applying the model clinically. Finally, we did not consider the variety of RRS and hospitalised patients in each facility. Outcomes might be influenced by the characteristics of the hospitals and circumstances surrounding RRT/MET in each hospital. Therefore, the potential heterogeneity of hospital characteristics and RRT/MET type may have affected this analysis.
Future research
The prediction model should be used for clinical assessment by medical staff. Thus, prospective studies using this model are needed. In particular, the workload of the data input into the model and its impact on actual RRT/MET decisions should be considered.
CONCLUSION
In this study, we developed a machine learning model to predict short‐term prognosis using the RRS multicentre registry. The model showed a higher accuracy than the existing EWS. The model incorporated features different from those of previously studied models used as RRS triggers. The new prediction model could prevent patients from inadequate evaluation and treatment during RRS activation.
FUNDING INFORMATION
This work was supported by JSPS KAKENHI (grant number JP18K16548). This funding source had no role in the design, practice, or analysis of this study.
CONFLICT OF INTEREST STATEMENT
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. SH reported receipt of grants from Nihon Kohden outside the submitted work. TN reported honoraria for lectures from Nihon Kohden, Sompo Risk Management Inc. outside the submitted work. No other disclosures were reported.
ETHICS STATEMENT
Approval of the research protocol: The study protocol was reviewed and approved by the ethics committee of the St. Marianna University School of Medicine, Japan (IRB No. 5552).
Informed consent: The need for written informed consent was waived because this study used an anonymous case registry and was conducted by a patient safety committee.
Registry and the registration no. of the study/trial: Not applicable.
Animal studies: Not applicable.
Supporting information
Figure S1.
Table S1.
ACKNOWLEDGEMENTS
We thank the In‐Hospital Emergency Committee in Japan, organised by the patient safety promotion committee of the Japanese Society of Emergency Medicine, Rapid Response System Committee in the Japanese Society of Intensive Care Medicine, Japan Resuscitation Council, Japanese Society of Emergency Paediatrics, Japanese Circulation Society, Japanese Society for Quality and Safety in Healthcare, and Japanese Coalition for Patient Safety for their contributions to this study. We also thank the In‐Hospital Emergency Study Group for helping with data collection. This work was supported by JSPS KAKENHI (grant number JP18K16548).
Naito T, Li M, Fujitani S. A machine learning model for predicting short‐term outcomes after rapid response system activation. Acute Med Surg. 2025;12:e70083. 10.1002/ams2.70083
DATA AVAILABILITY STATEMENT
The datasets used and analysed during the current study are available from the corresponding author upon reasonable request.
REFERENCES
- 1. Ludikhuize J, Brunsveld‐Reinders AH, Dijkgraaf MGW, Smorenburg SM, de Rooij SEJA, Adams R, et al. Outcomes associated with the nationwide introduction of rapid response systems in The Netherlands. Crit Care Med. 2015;43:2544–2551. [DOI] [PubMed] [Google Scholar]
- 2. Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review and meta‐analysis. Crit Care. 2015;19:254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Jones D, Bellomo R, Bates S, Warrillow S, Goldsmith D, Hart G, et al. Long term effect of a medical emergency team on cardiac arrests in a teaching hospital. Crit Care. 2005;9:R808–R815. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Chen J, Ou L, Hillman K, Flabouris A, Bellomo R, Hollis SJ, et al. The impact of implementing a rapid response system: a comparison of cardiopulmonary arrests and mortality among four teaching hospitals in Australia. Resuscitation. 2014;85:1275–1281. [DOI] [PubMed] [Google Scholar]
- 5. Churpek MM, Yuen TC, Winslow C, Meltzer DO, Kattan MW, Edelson DP. Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards. Crit Care Med. 2016;44:368–374. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Lee YJ, Cho K‐J, Kwon O, Park H, Lee Y, Kwon JM, et al. A multicentre validation study of the deep learning‐based early warning score for predicting in‐hospital cardiac arrest in patients admitted to general wards. Resuscitation. 2021;163:78–85. [DOI] [PubMed] [Google Scholar]
- 7. Cho K‐J, Kwon O, Kwon J‐M, Lee Y, Park H, Jeon KH, et al. Detecting patient deterioration using artificial intelligence in a rapid response system. Crit Care Med. 2020;48:e285–e289. [DOI] [PubMed] [Google Scholar]
- 8. Kia A, Timsina P, Joshi HN, Klang E, Gupta RR, Freeman RM, et al. MEWS++: enhancing the prediction of clinical deterioration in admitted patients through a machine learning model. J Clin Med Res. 2020;9:343. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Pimentel MAF, Redfern OC, Malycha J, Meredith P, Prytherch D, Briggs J, et al. Detecting deteriorating patients in the hospital: development and validation of a novel scoring system. Am J Respir Crit Care Med. 2021;204:44–52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified early warning score in medical admissions. QJM. 2001;94:521–526. [DOI] [PubMed] [Google Scholar]
- 11. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84:465–470. [DOI] [PubMed] [Google Scholar]
- 12. Batterbury A, Douglas C, Jones L, Coyer F. Illness severity characteristics and outcomes of patients remaining on an acute ward following medical emergency team review: a latent profile analysis. BMJ Qual Saf. 2023;32:404–413. [DOI] [PubMed] [Google Scholar]
- 13. Batterbury A, Douglas C, Coyer F. The illness severity of patients reviewed by the medical emergency team: a scoping review. Aust Crit Care. 2021;34:496–509. [DOI] [PubMed] [Google Scholar]
- 14. Subbe CP, Bannard‐Smith J, Bunch J, Champunot R, DeVita M, Durham L, et al. Quality metrics for the evaluation of Rapid Response Systems: proceedings from the third international consensus conference on Rapid Response Systems. Resuscitation. 2019;141:1–12. [DOI] [PubMed] [Google Scholar]
- 15. Føns‐Sønderskov MB, Subbe C, Kodal AM, Bunkenborg G, Bestle MH. Rapid response teams‐how and who? A protocol for a randomised clinical trial evaluating the composition of the efferent limb of the rapid response system. Acta Anaesthesiol Scand. 2021;66:401–407. [DOI] [PubMed] [Google Scholar]
- 16. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards. Crit Care Med. 2014;42:841–848. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Jenkins PF, Thompson CH, Barton LL. Clinical deterioration in the condition of patients with acute medical illness in Australian hospitals: improving detection and response. Med J Aust. 2011;194:596–598. [DOI] [PubMed] [Google Scholar]
- 18. Naito T, Hayashi K, Hsu H‐C, Aoki K, Nagata K, Arai M, et al. Validation of National Early Warning Score for predicting 30‐day mortality after rapid response system activation in Japan. Acute Med Surg. 2021;8:e666. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365:2091–2097. [DOI] [PubMed] [Google Scholar]
- 20. Santiano N, Young L, Hillman K, Parr M, Jayasinghe S, Baramy LS, et al. Analysis of medical emergency team calls comparing subjective to “objective” call criteria. Resuscitation. 2009;80:44–49. [DOI] [PubMed] [Google Scholar]
- 21. Jones D, Bellomo R, DeVita MA. Effectiveness of the medical emergency team: the importance of dose. Crit Care. 2009;13:313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Foraida MI, DeVita MA, Braithwaite RS, Stuart SA, Brooks MM, Simmons RL. Improving the utilization of medical crisis teams (condition C) at an urban tertiary care hospital. J Crit Care. 2003;18:87–94. [DOI] [PubMed] [Google Scholar]
- 23. Baxter AD, Cardinal P, Hooper J, Patel R. Medical emergency teams at the Ottawa Hospital: the first two years. Can J Anaesth. 2008;55:223–231. [DOI] [PubMed] [Google Scholar]
- 24. Akel MA, Carey KA, Winslow CJ, Churpek MM, Edelson DP. Less is more: detecting clinical deterioration in the hospital with machine learning using only age, heart rate, and respiratory rate. Resuscitation. 2021;168:6–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in‐hospital clinical deterioration. N Engl J Med. 2020;383:1951–1960. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Naito T, Fujiwara S, Kawasaki T, Sento Y, Nakada T‐a, Arai M, et al. First report based on the online registry of a Japanese multicenter rapid response system: a descriptive study of 35 institutions in Japan. Acute Med Surg. 2019;7:e454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Kipnis P, Turk BJ, Wulf DA, LaGuardia JC, Liu V, Churpek MM, et al. Development and validation of an electronic medical record‐based alert score for detection of inpatient deterioration outside the ICU. J Biomed Inform. 2016;64:10–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Shamout FE, Zhu T, Sharma P, Watkinson PJ, Clifton DA. Deep interpretable early warning system for the detection of clinical deterioration. IEEE J Biomed Health Inform. 2020;24:437–446. [DOI] [PubMed] [Google Scholar]
- 29. Kim SH, Choi HS, Jin ES, Choi H, Lee H, Lee SH, et al. Predicting severe outcomes using national early warning score (NEWS) in patients identified by a rapid response system: a retrospective cohort study. Sci Rep. 2021;11:18021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Adielsson A, Danielsson C, Forkman P, Karlsson T, Pettersson L, Herlitz J, et al. Outcome prediction for patients assessed by the medical emergency team: a retrospective cohort study. BMC Emerg Med. 2022;22:200. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Figure S1.
Table S1.
Data Availability Statement
The datasets used and analysed during the current study are available from the corresponding author upon reasonable request.