Abstract
BACKGROUND
Clinical deterioration of ward patients can result in intensive care unit (ICU) transfer, cardiac arrest (CA), and/or death. These different outcomes have been used to develop and test track and trigger systems, but the impact of outcome selection on the performance of prediction algorithms is unknown.
METHODS
Patients hospitalized on the wards between November 2008 and August 2011 at an academic hospital were included in the study. Ward vital signs and demographic characteristics were compared across outcomes. The dataset was then split into derivation and validation cohorts. Logistic regression was used to derive four models (one per outcome and a combined outcome) for predicting each event within 24 hours of a vital sign set. The models were compared in the validation cohort using the area under the receiver operating characteristic curve (AUC).
RESULTS
A total of 59,643 patients were included in the study (including 109 ward CAs, 291 deaths, and 2,638 ICU transfers). Most mean vital signs within 24 hours of the events differed statistically, with those before death being the most deranged. Validation model AUCs were highest for predicting mortality (range 0.73–0.82), followed by CA (range 0.74–0.76), and lowest for predicting ICU transfer (range 0.68–0.71).
CONCLUSIONS
Despite differences in vital signs before CA, ICU transfer, and death, the different models performed similarly for detecting each outcome. Mortality was the easiest outcome to predict and ICU transfer the most difficult. Studies should be interpreted with these differences in mind.
Introduction
Serious adverse events on the wards are often preceded by vital sign instability.1–6 Attempts to characterize this instability and derive or validate risk prediction algorithms such as physiology-based track and trigger systems have been marred by a lack of consensus on which outcome(s) to use. As such, the published literature on the topic utilizes various outcomes, including cardiac arrest,2,7,8 intensive care unit (ICU) transfer,9,10 mortality11–13 or a composite outcome.3,14,15 This variation makes comparison across studies challenging. In addition, it is not known how vital sign instability differs prior to these outcomes and if those differences would impact prediction model derivation for a chosen outcome and the accuracy for predicting the alternate outcomes.
Each of the routinely utilized outcomes has advantages and disadvantages associated with its use. Mortality is often the preferred outcome for clinical studies since it is the final endpoint of clinical deterioration and is reliably obtained from hospital administrative data. However, many hospital deaths are not unexpected, especially among medical patients, and identifying patients who are having medical care actively withdrawn by the primary team, in anticipation of death, would not result in a decrease in preventable in-hospital death. Cardiac arrest on the wards is unambiguously defined (loss of a pulse with attempted resuscitation) and generalizable, similar to mortality. Importantly, unlike mortality, cardiac arrest is generally unwanted (as evidenced by the lack of a Do-Not-Attempt-Resuscitation order) and therefore represents a failure of the hospital’s surveillance and response system. What limits its use is that these data are difficult to obtain and the outcome itself is less common than the other outcomes, limiting statistical power. The most frequently occurring individual outcome in most hospitals is ICU transfer,14,16 which explains its appeal. However, criteria for ICU admission vary among hospitals and providers as well as with external factors such as bed availability, making it the least generalizable outcome of the three. In addition, ICU transfer is by definition an event recognized by the hospital staff, so developing an algorithm for predicting this event may not be as meaningful clinically. Finally, the combined outcome provides the greatest statistical power for analysis but is driven largely by ICU transfer given the unequal distribution of the three individual outcomes.
The aims of this study were to describe the physiologic instability preceding each of the three individual outcomes and analyze the impact of outcome selection on the derivation and validation of risk prediction algorithms for ward patients. We hypothesized that vital signs prior to these events would differ significantly and that algorithms derived for the more frequent outcomes (ICU transfer and the composite outcome of any event) or most reliably obtained (mortality) would be less accurate for predicting cardiac arrest, the most clinically important outcome.
Methods
The study protocol, consent, and data collection mechanisms were approved by the University of Chicago Institutional Review Board. A waiver of consent was granted based on minimal harm and general impracticability. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.
Study setting and population
This observational cohort study was performed at an academic, tertiary care hospital with approximately 500 inpatient beds. All patients hospitalized between November 1, 2008 and August 31, 2011 who had vital signs documented on the general hospital wards were included in the study. A rapid response team (RRT), led by a critical care nurse and respiratory therapist with consultation from a hospitalist (i.e. an attending physician who specializes in the acute care of medical patients on the wards17) and/or pharmacist available upon request, has been in place in our hospital since 2008. This team is separate from the team that responds to a cardiac arrest. The RRT activation criteria include “tachypnea,” “tachycardia,” “hypotension,” and “staff worry,” but specific vital sign thresholds are not stated.
Data collection
Demographic information was obtained using hospital administrative databases for all study patients. Location and time stamped vital signs, including temperature, systolic and diastolic blood pressure, heart rate, respiratory rate, oxygen saturation, use of supplemental oxygen, and mental status were obtained from the hospital’s electronic health record (EHR) (EPIC, Verona, Wisconsin). Mental status was collapsed from four drop-down menu fields in the EHR (orientation, level of consciousness, motor response, and responsiveness) into one score (alert, responsive to voice, responsive to pain, and unresponsive (AVPU)) as previously described.2 Only vital signs collected on the wards were used in this study.
Outcomes
The outcomes investigated in this study were cardiac arrest on the wards, ward to ICU transfer, and death within 24 hours of a ward vital sign. Cardiac arrest was defined as the loss of a palpable pulse with attempted resuscitation and was identified using a prospectively collected and verified quality improvement database that has been previously described.2 Ward to ICU transfer and mortality were determined using hospital administrative databases, and the last vital sign on the ward (for ICU transfers) or last vital sign regardless of location (for mortality patients) were used to approximate the time of the event. Ward to ICU transfers that occurred immediately following surgery were not counted because they were not unexpected events. More than one outcome could occur for each patient and following each vital sign set in the study (e.g. a patient who dies following an unsuccessful resuscitation attempt would count as both a cardiac arrest and a death).
Statistical Analysis
Patient characteristics and vital signs measured within 24 hours of each event were compared using t-tests, Wilcoxon rank sums, and chi-squared tests where appropriate. For the purpose of significance testing between groups, data occurring in more than one outcome category were dropped from the analysis. Logistic regression was used in the full dataset to develop four models (one for each outcome independently and a composite outcome) for predicting the event of interest within 24 hours of a vital sign so that model coefficients could be compared across outcomes. All vital signs were entered linearly into the regression models except mental status, which was entered categorically, and all variables were kept in the model so that coefficient comparisons could be made. If any vital sign was missing at a time-point, the most recent value was imputed, similar to what would be done in clinical practice. If a patient had no previous values of the missing variable then a normal value was imputed. To determine the accuracy of vital signs for detecting each event, the dataset was then split in half based on admission date. The first half of the dataset was used to derive four logistic regression models, similar to above, and the second half of the dataset was used to validate each model to detect each of the four outcomes. Areas under the receiver operating characteristic curves (AUCs) were then calculated for each of the four models to detect each of the four outcomes, with vital signs within 24 hours of the outcome of interest denoted as positive for that outcome. All tests of significance used a 2-sided P<0.05. Statistical analyses were completed using Stata version 12.0 (StataCorp, College Station, Texas).
Results
A total of 59,643 patients were included in the study, and 109 ward cardiac arrests, 2,638 ICU transfers, and 291 deaths within 24 hours of a ward vital sign occurred during the study period (Figure 1). Of the 291 deaths, 158 (54%) did so on the wards without attempted resuscitation. Patient characteristics are shown in Table 1 and mean vital sign comparisons occurring within 24 hours of each event are shown in Table 2. Control patients differed significantly from all three outcome groups across all measured vital signs with the exception of oxygen saturation, which did not differ from the mean oxygen saturation 24 hours prior to cardiac arrest. In addition, of all the outcomes, vital signs prior to death demonstrated the greatest degree of derangement when compared to controls with respect to systolic blood pressure (110 ± 28 vs. 126 ± 22 mm Hg; P<0.001), diastolic blood pressure (63 ± 18 vs. 71 ± 13 mm Hg; P<0.001), heart rate (102 ± 23 vs. 84 ± 16 beats/min; P<0.001), oxygen saturation (median 96% (IQR 93%–98%) vs. 98% (IQR 96%–99%); P<0.001), and mental status (44% alert vs. 85% alert; P<0.001 for mental status distribution). Compared to cardiac arrest, vital signs prior to ICU transfer had a higher mean heart rate (97 ± 23 vs. 94 ± 19 beats/min; P<0.01), diastolic blood pressure (69 ± 17 vs. 67 ± 17 mm Hg; P<0.01), and temperature (36.6 ± 1.0 vs. 36.2 ± 0.8 °C; P<0.01).
Figure 1.
Venn diagram illustrating the overlap of the study outcome events (ICU transfer, cardiac arrest, and death).
Table 1.
Patient characteristics by outcome group.
| Controls (N=56,751) |
ICU Transfer (N=2,638) |
Cardiac Arrest (N=109) |
Mortality (N=291) |
|
|---|---|---|---|---|
| Age, mean (SD), years | 54 (18)b,c,d | 60 (16)a,c,d | 64 (17)a,b | 66 (13)a,b |
| Female sex | 32,189 (57%)b | 1,290 (50%)a | 64 (59%) | 126 (53%) |
| Race | b,c,d | a | a,d | a,c |
| Black | 24,639 (43%) | 1,148 (44%) | 68 (62%) | 114 (49%) |
| White | 20,250 (36%) | 1,053 (40%) | 24 (22%) | 104 (36%) |
| Other | 1,461 (3%) | 96 (4%) | 3 (3%) | 10 (3%) |
| Unknown | 10,401 (18%) | 641 (13%) | 14 (13%) | 33 (11%) |
| Surgical patiente | 19,127 (34%)b,d | 1,118 (43%)a,d | 36 (33%)d | 45 (19%)a,b,c |
| Length of stay, median (IQR), days | 3 (1–5)b,c,d | 12 (7–21)a,c,d | 11 (4–25)a,b,d | 5 (3–9)a,b,c |
| RRT call during study period | 154 (0%)b,c,d | 358 (14%)a | 19 (17%)a | 32 (13%)a |
| Survived to discharge | 56,751 (100%)b,c,d | 2,104 (81%)a,c,d | 30 (28%)a,b,d | 0 (0%)a,b,c |
Abbreviations: ICU, intensive care unit; SD, standard deviation; IQR, interquartile range. All results are shown as n (%) unless otherwise indicated. Controls are patients discharged without experiencing a cardiac arrest, ICU transfer, or death. N refers to number of patients in each group.
P-value <0.05 for comparison to controls.
P-value <0.05 for comparison to ICU transfer patients.
P-value<0.05 for comparison to cardiac arrest patients.
P-value<0.05 for comparison to mortality patients.
Surgical patients are patients who went to the operating room during their admission.
Table 2.
Comparisons of mean vital signs 24 hours before cardiac arrest, intensive care unit transfer, death, or during entire hospitalization (controls).
| Variable | Controls (N=56,751) |
ICU Transfer (N=2,638) |
Cardiac Arrest (N=109) |
Mortality (N=291) |
|---|---|---|---|---|
| Respiratory rate, breaths/min | 18 (2)b,c,d | 20 (5)a,d | 21 (4)a | 22 (7)a,b |
| Systolic blood pressure, mmHg | 126 (22)b,c,d | 124 (29)a,d | 123 (25)a,d | 110 (28)a,b,c |
| Diastolic blood pressure, mmHg | 71 (13)b,c,d | 69 (17)a,c,d | 67 (17)a,b,d | 63 (18)a,b,c |
| Heart rate, beats/min | 84 (16)b,c,d | 97 (23)a,c,d | 94 (19)a,b,d | 102 (23)a,b,c |
| Oxygen saturation, median (IQR), % | 98 (96–99)b,d | 97 (95–99)a,d | 98 (96–99)d | 96 (93–98)a,b,c |
| Temperature, °C | 36.4 (0.7)b,c,d | 36.6 (1.0)a,c,d | 36.2 (0.8)a,b,d | 36.5 (1.1)a,b,c |
| Mental status, total observations (%) | b,c,d | a,d | a,d | a,b,c |
| Alert | 377190 (85) | 4286 (67) | 167 (64) | 181 (44) |
| Responsive to voice | 66208 (15) | 1898 (30) | 83 (32) | 104 (25) |
| Responsive to pain | 1076 (0.2) | 126 (2) | 6 (2) | 43 (10) |
| Unresponsive | 258 (0.1) | 82 (1) | 3 (1) | 88 (21) |
Abbreviations: ICU, intensive care unit; IQR, interquartile range. Data shown are mean (standard deviation) unless otherwise noted. Controls are patients discharged without experiencing a cardiac arrest, ICU transfer, or death. N refers to number of patients in each group.
P-value <0.05 for comparison to controls.
P-value<0.05 for comparison to ICU transfers.
P-value<0.05 for comparison to cardiac arrest.
P-value<0.05 for comparison to mortality.
Vital sign coefficients in the four logistic regression models were similar in direction across the outcomes (Table 3). However, the model derived to predict mortality was more heavily weighted for systolic hypotension, supplemental oxygen administration, and decreased responsiveness compared to the other outcomes. Meanwhile, hypothermia had the greatest impact on the model for predicting cardiac arrest while hypoxia had a larger impact for predicting ICU transfer.
Table 3.
Logistic regression coefficients for models derived for cardiac arrest, intensive care unit transfer, death, or a composite of all three outcomes.
| Variable (units) | Combined Outcome |
ICU transfer | Cardiac Arrest | Mortality |
|---|---|---|---|---|
| Heart rate (beats/min) | 0.03 (0.03, 0.03) | 0.03 (0.03, 0.03) | 0.03 (0.02, 0.03) | 0.04 (0.04, 0.04) |
| Systolic blood pressure (mmHg) | 0.001 (0.001, 0.002) | 0.002 (0.002, 0.003) | 0.01 (0.01, 0.01) | −0.26 (−0.28, −0.23) |
| Diastolic blood pressure (mmHg) | −0.02 (−0.02, −0.02) | −0.02 (−0.02, −0.02) | −0.04 (−0.04, −0.03) | −0.02 (−0.03, −0.02) |
| Respiratory rate (breaths/min) | 0.15 (0.14, 0.15) | 0.15 (0.14, 0.15) | 0.16 (0.15, 0.17) | 0.14 (0.13, 0.15) |
| Oxygen saturation (%) | −0.06 (−0.06, −0.05) | −0.51 (−0.55, −0.48) | −0.01 (−0.03, 0.01) | −0.08 (−0.08, −0.07) |
| Temperature (°C) | −0.05 (−0.07, −0.04) | −0.02 (−0.04, −0.005) | −0.54 (−0.62, −0.45) | −0.35 (−0.40, −0.30) |
| Supplemental oxygen requirement | 0.50 (0.53, 0.47) | 0.47 (0.50, 0.44) | 0.65 (0.81, 0.48) | 0.85 (0.97, 0.73) |
| Mental Status | ||||
| Alert | Reference | Reference | Reference | Reference |
| Responsive to voice | 1.03 (0.99, 1.08) | 1.03 (0.98, 1.08) | 0.97 (0.74, 1.20) | 0.81 (0.62, 0.99) |
| Responsive to pain | 1.64 (1.46, 1.80) | 1.30 (1.09, 1.50) | 0.93 (0.10, 1.96) | 2.41 (2.07, 2.76) |
| Unresponsive | 3.09 (2.87, 3.30) | 1.95 (1.68, 2.22) | 1.16 (−0.03, 2.36) | 4.47 (4.19, 4.75) |
All data shown are regression coefficients (95% confidence interval).
The validation results for the four different outcome models are shown in Figure 2. In general, the four models were most accurate for predicting mortality (AUC range: 0.73–0.82) and least accurate for predicting ICU transfer (AUC range: 0.68–0.71) and the combined outcome (AUC range: 0.68–0.71). All four models performed similarly for detecting cardiac arrest within 24 hours (cardiac arrest model AUC 0.74 [95% CI 0.72–0.76]; mortality model AUC 0.75 [95% CI 0.73–0.77]; ICU model AUC 0.76 [95% CI 0.74–0.78]; combined model AUC 0.76 [95% CI 0.75–0.78]). Of note, the combined model was as accurate as the best individual outcome model for detecting all outcomes except mortality (mortality model AUC 0.82 [95% CI 0.81–0.83] vs. combined model AUC 0.78 [95% CI 0.76–0.79]).
Figure 2.
Areas under the receiver operating characteristic curves for each derived model to detect each outcome in the validation cohort.
Discussion
Our study of vital signs prior to ICU transfer, cardiac arrest, and death resulted in several important findings. First, we found that vital signs preceding death were generally the most deranged, those preceding ICU transfer were the least deranged, and vital signs prior to cardiac arrest were somewhere in the middle. This resulted in the highest AUCs for prediction of death and the lowest for prediction of ICU transfer, independent of how the model was derived. Second, we noted significant differences in the strength of association between specific vital signs and the various outcomes, such as the correlation between hypoxia and ICU transfer and systolic hypotension with death. However, despite these differences, each of the derived models detected the other events with similar accuracy.
The implications for these findings are twofold. First, it argues for deriving a vital sign model for predicting the combined outcome of cardiac arrest, ICU transfer or death given the increased statistical power and model stability associated with a higher frequency of adverse events and the accuracy of the model for detecting the most clinically important outcome, cardiac arrest. In addition, our study highlights the importance of reporting all three outcomes when validating any prediction model, as the accuracy will vary depending on the outcome reported.
Previous studies in ward patients have investigated vital sign accuracy and developed prediction models using different outcomes and methodologies. For example, Prytherch and colleagues compared the accuracy of the ViEWS to that of 33 other early warning scoring systems for predicting mortality in acute medical patients within 24 hours of a vital sign set and demonstrated AUCs ranging from 0.80 to 0.89.11 Cuthbertson et al. developed a prediction algorithm for detecting ICU transfer in a case-control study of surgical patients, which had an AUC of 0.88 compared to 0.83–0.86 for three other track and trigger systems.9 In addition, using a cohort of hospitalized medical and surgical patients, our group derived the cardiac arrest risk triage (CART) score for predicting cardiac arrest, which had an AUC of 0.84 for detecting ward arrest and an AUC of 0.71 for detecting ICU transfer.18 Finally, the combined outcome of cardiac arrest, ICU transfer, or death has been used in other studies, such as in the case-control study by Cretikos et al., which investigated the accuracy of the MERIT study Medical Emergency Team activation criteria within 24 hours of each outcome independently, as well as combined.3 Although AUC values were not reported, they did find that the accuracy varied depending on the outcome chosen. Several other investigations have utilized different study methodologies and outcomes and have reported various levels of accuracy.7,8,10,12,13,19–21 The wide range in the outcome of interest, patient population, time window prior to the event studied, definition of prediction model accuracy, and study design makes it difficult to compare results across studies. Our results provide some clarity by illustrating how accuracy can vary depending on the outcome chosen.
There were some interesting differences in the vital signs prior to the different adverse events and the regression model coefficients. For example, patients transferred to the ICU had a higher mean heart rate and lower oxygen saturation than those patients who remained on the wards and suffered a cardiac arrest. In addition, hypoxia was heavily weighted in the ICU transfer model, whereas hypothermia was heavily weighted for both cardiac arrest and mortality, but not ICU transfer. In fact, patients transferred to the ICU had a higher mean temperature than control patients, and the constellation of fever, hypoxia, tachycardia, and hypotension may reflect a sub-population of septic patients on the wards. These findings may represent a selection bias introduced by clinicians who recognize these signs as markers of clinical deterioration more readily than other signs such as hypothermia, resulting in such patients being transferred to the ICU. In addition, mental status was more heavily weighted in the mortality model than the other models. One explanation for this is that mental status may be more frequently and accurately documented in patients receiving comfort care, who are more likely to receive frequent monitoring for pain and comfort level and often receive medications that alter mentation. However, despite these vital sign differences, our finding that prediction models developed for cardiac arrest, ICU transfer, mortality, or the combined outcome all detected each outcome as well as the model developed specifically for that outcome suggests that these differences may not be clinically relevant when developing a risk score for ward patients.
Our study has several important strengths compared to previous vital sign investigations. First, it encompassed over 55,000 admissions and a heterogeneous patient population including surgical and medical patients. Second, our study investigated vital signs throughout the course of the hospitalization instead of just those measured on admission or during the first few days of hospitalization. In addition, our study was large enough to have a separate derivation and validation cohort when evaluating the accuracy of the different models. Finally, to our knowledge this is the first study to investigate the differences in vital signs before ICU transfer, cardiac arrest, and death and the implications the chosen outcome has on model accuracy.
There are several limitations to the current study. First, the investigation was a single-center study performed at a tertiary academic medical center and the results may not be applicable to other settings. In addition, our ICU transfer patients may not all be unexpected transfers. However, patients who were transferred to the ICU from the operating room were not counted as ICU transfers in an attempt to control for the largest likely source of expected ICU transfers. Also, there are a considerable number of physiologic variables available in the EHR, which adds to the complexity of modeling clinical outcomes. We only investigated vital signs in this study so it is unknown whether the addition of laboratory or demographic variables would create clinically meaningful differences in the accuracy of the models developed for the various outcomes.
Conclusions
We found that vital signs are different prior to cardiac arrest, ICU transfer and death. However, despite these differences, models developed for each of these outcomes and a combined outcome performed similarly for detecting the three outcomes, with mortality being the easiest to predict and ICU transfer the most difficult. Model accuracy is strongly associated with the chosen outcome of interest, and studies should be interpreted with these differences in mind.
Acknowledgments
Financial disclosures: Dr. Edelson is supported by a career development award from the National Heart, Lung, and Blood Institute (K23 HL097157-01) and has received research support from Philips Healthcare (Andover, MA). Dr. Churpek is supported by a National Institutes of Health grant [T32 HL 07605].
Funding sources
We would like to thank Donald Saner, MS and Contessa Hsu for assistance with data extraction and technical support and Nicole Babuskow for administrative support.
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
These data were presented as a poster session at the American Thoracic Society International 23 Conference on May 19, 2012 (San Francisco, CA).
Conflicts of interest statement
Dr. Edelson has received research support, speaking honoraria and consulting fees from Philips Healthcare (Andover, MA), and is on the advisory board of Sotera Wireless (San Diego, CA). Dr. Churpek has no conflicts of interest to disclose.
References
- 1.Berlot G, Pangher A, Petrucci L, et al. Anticipating events of in-hospital cardiac arrest. Eur J Emerg Med. 2004;11:24–28. doi: 10.1097/00063110-200402000-00005. [DOI] [PubMed] [Google Scholar]
- 2.Churpek MM, Yuen TC, Huber MT, et al. Predicting cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):1170–1176. doi: 10.1378/chest.11-1301. Epub 2011 Nov 3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cretikos M, Chen J, Hillman K, et al. The objective medical emergency team activation criteria: a case-control study. Resuscitation. 2007;73:62–72. doi: 10.1016/j.resuscitation.2006.08.020. [DOI] [PubMed] [Google Scholar]
- 4.Buist MD, Jarmolowski E, Burton PR, et al. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care. A pilot study in a tertiary-care hospital. Med J Aust. 1999;171:22–25. doi: 10.5694/j.1326-5377.1999.tb123492.x. [DOI] [PubMed] [Google Scholar]
- 5.Hillman KM, Bristow PJ, Chey T, et al. Antecedents to hospital deaths. Intern Med J. 2001;31:343–348. doi: 10.1046/j.1445-5994.2001.00077.x. [DOI] [PubMed] [Google Scholar]
- 6.Devita MA, Bellomo R, Hillman K, et al. Findings of the first consensus conference on medical emergency teams. Crit Care Med. 2006;34:2463–2478. doi: 10.1097/01.CCM.0000235743.38172.6E. [DOI] [PubMed] [Google Scholar]
- 7.Fieselmann JF, Hendryx MS, Helms CM, et al. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med. 1993;8:354–360. doi: 10.1007/BF02600071. [DOI] [PubMed] [Google Scholar]
- 8.Hodgetts TJ, Kenward G, Vlachonikolis IG, et al. The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team. Resuscitation. 2002;54:125–131. doi: 10.1016/s0300-9572(02)00100-4. [DOI] [PubMed] [Google Scholar]
- 9.Cuthbertson BH, Boroujerdi M, McKie L, et al. Can physiological variables and early warning scoring systems allow early recognition of the deteriorating surgical patient? Crit Care Med. 2007;35:402–409. doi: 10.1097/01.CCM.0000254826.10520.87. [DOI] [PubMed] [Google Scholar]
- 10.Cuthbertson BH, Boroujerdi M, Prescott G. The use of combined physiological parameters in the early recognition of the deteriorating acute medical patient. J R Coll Physicians Edinb. 40:19–25. doi: 10.4997/JRCPE.2010.105. [DOI] [PubMed] [Google Scholar]
- 11.Prytherch DR, Smith GB, Schmidt PE, et al. ViEWS--Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation. 81:932–937. doi: 10.1016/j.resuscitation.2010.04.014. [DOI] [PubMed] [Google Scholar]
- 12.Bleyer AJ, Vidya S, Russell GB, et al. Longitudinal analysis of one million vital signs in patients in an academic medical center. Resuscitation. 82:1387–1392. doi: 10.1016/j.resuscitation.2011.06.033. [DOI] [PubMed] [Google Scholar]
- 13.Smith GB, Prytherch DR, Schmidt PE, et al. Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation. 2008;77:170–179. doi: 10.1016/j.resuscitation.2007.12.004. [DOI] [PubMed] [Google Scholar]
- 14.Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster-randomised controlled trial. Lancet. 2005;365:2091–2097. doi: 10.1016/S0140-6736(05)66733-5. [DOI] [PubMed] [Google Scholar]
- 15.Subbe CP, Davies RG, Williams E, et al. Effect of introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia. 2003;58:797–802. doi: 10.1046/j.1365-2044.2003.03258.x. [DOI] [PubMed] [Google Scholar]
- 16.Gao H, McDonnell A, Harrison DA, et al. Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward. Intensive Care Med. 2007;33:667–679. doi: 10.1007/s00134-007-0532-3. [DOI] [PubMed] [Google Scholar]
- 17.Wachter RM, Goldman L. The hospitalist movement 5 years later. JAMA. 2002;287:487–494. doi: 10.1001/jama.287.4.487. [DOI] [PubMed] [Google Scholar]
- 18.Churpek MM, Yuen TC, Park SY, et al. Derivation of a cardiac arrest risk score using ward vital signs. Crit Care Med. 2012;40(7):2102–2108. doi: 10.1097/CCM.0b013e318250aa5a. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Lighthall GK, Markar S, Hsiung R. Abnormal vital signs are associated with an increased risk for critical events in US veteran inpatients. Resuscitation. 2009;80:1264–1269. doi: 10.1016/j.resuscitation.2009.08.012. [DOI] [PubMed] [Google Scholar]
- 20.Smith GB, Prytherch DR, Schmidt PE, et al. A review, and performance evaluation, of single-parameter "track and trigger" systems. Resuscitation. 2008;79:11–21. doi: 10.1016/j.resuscitation.2008.05.004. [DOI] [PubMed] [Google Scholar]
- 21.Subbe CP, Kruger M, Rutherford P, et al. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94:521–526. doi: 10.1093/qjmed/94.10.521. [DOI] [PubMed] [Google Scholar]