Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Jul 7.
Published in final edited form as: J Intensive Care Med. 2019 Oct 21;35(12):1529–1535. doi: 10.1177/0885066619882675

Performance of Critical Care Outcome Prediction Models in an Intermediate Care Unit

Rebeccah M Brusca 1, Catherine E Simpson 2, Sarina K Sahetya 2, Zeba Noorain 3, Varshitha Tanykonda 4, R Scott Stephens 2, Dale M Needham 2,5,6,7, David N Hager 2
PMCID: PMC8262077  NIHMSID: NIHMS1712774  PMID: 31635507

Abstract

Background:

Intermediate care units (IMCUs) are heterogeneous in design and operation, which makes comparative effectiveness studies challenging. A generalizable outcome prediction model could improve such comparisons. However, little is known about the performance of critical care outcome prediction models in the intermediate care setting. The purpose of this study is to evaluate the performance of the Acute Physiology and Chronic Health Evaluation version II (APACHE II), Simplified Acute Physiology Score version II (SAPS II) and version 3 (SAPS 3), and Mortality Probability Model version III (MPM0III) in patients admitted to a well-characterized IMCU.

Materials and Methods:

In the IMCU of an academic medical center (July to December 2012), the discrimination and calibration of each outcome prediction model were evaluated using the area under the receiver–operating characteristic and Hosmer-Lemeshow goodness-of-fit test, respectively. Standardized mortality ratios (SMRs) were also calculated.

Results:

The cohort included data from 628 unique IMCU admissions with an inpatient mortality rate of 8.3%. All models exhibited good discrimination, but only the SAPS II and MPM0III were well calibrated. While the APACHE II and SAPS 3 both markedly overestimated mortality, the SMR for the SAPS II and MPM0III were 0.91 and 0.91, respectively.

Conclusions:

The SAPS II and MPM0III exhibited good discrimination and calibration, with slight overestimation of mortality. Each model should be further evaluated in multicenter studies of patients in the intermediate care setting.

Keywords: intermediate care unit, stepdown care, progressive care, outcome prediction, mortality prediction

Introduction

Intermediate care units (IMCUs), also known as high-dependency units, stepdown units, and progressive care units, were created to accommodate patients whose needs do not require an intensive care unit (ICU) but surpass the care and monitoring that is feasible on a general medical or surgical unit.1 Although increasing in prevalence,2–4 the value of IMCUs for improving patient outcomes and reducing costs of care is unclear.5,6 A notable challenge in evaluating IMCUs is heterogeneity in the design and operation between (and within) different institutions and patient populations, which makes comparative effectiveness studies challenging.1 Such comparisons could be improved, in part, by a generalizable outcome prediction model.1 To date, only the Intermediate Care Unit Severity Score (ImCUSS) has been developed and validated for the IMCU population.7 However, generalizability of this model is uncertain.8

In the absence of a well-validated outcome prediction model for IMCU patients, many have borrowed models developed for ICU populations.9–17 However, the performance characteristics of these models among IMCU patients are rarely described. One exception is the Simplified Acute Physiology Score, version 2 (SAPS II), which has exhibited good performance characteristics in both a French and a Spanish IMCU.9,10,18 In another IMCU, the Acute Physiology and Chronic Health Evaluation version II (APACHE II) showed good discrimination, but calibration was not described.14,19 The Simplified Acute Physiology Score, version 3 (SAPS 3) had good discrimination but overestimated mortality.7,20

The objective of this study is to characterize the performance of the APACHE II, SAPS II, SAPS 3, and Mortality Probability Model version III (MPM0III) among patients admitted to a well-characterized IMCU in the United States.11,21

Methods

The study protocol was approved by the Institutional Review Board of Johns Hopkins University.

Study Population

This is a retrospective study of patients aged 18 years or older admitted to an 18-bed medical IMCU at an urban academic medical center between July and December 2012. Only data related to each patient’s first IMCU admission during this period were collected. Patients could be admitted from any source, including the emergency department (ED), general medical unit, ICU, and other hospitals.

Intermediate Care Units Setting

The organization of the IMCU, including the physical layout, admission guidelines with triggers for ICU consultation and transfer, as well as staffing, have been described previously.11 Briefly, the IMCU is an “open” unit intended for adult medical patients. It is located in close proximity to the general medical units in a different building than the ICUs. The nurse-to-patient ratio is 1:3. All patients receive continuous pulse oximetry and cardiac telemetry. A respiratory therapist is on site 24 hours per day to support patients receiving noninvasive ventilation, high-flow nasal oxygen, nebulizer treatments, and frequent airway suctioning.

Data Collection

Patient-level data were abstracted in duplicate by trained staff working independently. A third independent reviewer arbitrated any inconsistencies. Data abstracted included demographics, date and time of admission/discharge to/from other locations of care, hospital location prior to IMCU admission, IMCU admitting diagnosis, and vital status at hospital discharge. The APACHE II and SAPS II were calculated from the most aberrant data elements documented during the 24 hours following IMCU admission.18,19 The SAPS 3, including the version customized for North America (SAPS 3NA), and MPM0III were calculated from data available within 1 hour before or after admission to the IMCU.20 The Charlson Comorbidity Index was calculated using data available at the time of hospitalization.22 For all scores, missing data were assumed normal.

Outcome

Hospital mortality was the outcome of interest.

Statistical Analysis

Continuous variables are reported as means with standard deviations (SDs) or medians with interquartile ranges (IQRs) and categorical variables as counts and proportions. Comparisons between survivors and nonsurvivors are made using a 2-sample test of proportions, the Fisher exact test, and the Wilcoxon rank-sum test, as appropriate.

The performance of each model was assessed based on its ability to accurately predict mortality. Accuracy, in this case, is an assessment of how well predicted mortality matched observed mortality and is characterized by discrimination and calibration. A model with good discriminative properties consistently assigns a higher risk of death to patients who die and a lower risk to those who live. Discrimination is characterized by the area under the receiver–operating characteristic (AUROC), with an area of 0.5 indicating no discriminant value and an area of 1.0 indicating perfect discrimination. Comparisons of the AUROC use the method of Delong et al.23 Calibration is the ability of the model to accurately predict what proportion of patients will experience an outcome of interest within subgroups of risk. Patients are assigned to different subgroups (typically deciles) by ranking their predicted risk of experiencing an outcome of interest. Graphically, assessments of calibration demonstrate if, and over what range of risk, the model tends to over- or underestimate an outcome. Calibration in this study was assessed using the Hosmer-Lemeshow goodness-of- fit χ2 test and inspection of calibration curves.24 Finally, a standardized mortality ratio (SMR = observed deaths/predicted deaths) was calculated for all models as a global assessment of each model’s performance.

All statistical analyses were performed using Stata version 11.2 (Stata Corp, College Station, Texas).

Results

Study Population

As previously reported, between July and December 2012, there were 765 IMCU admissions representing 628 unique patients.8 Patient characteristics are summarized in Table 1. Approximately half of all patients were admitted directly from the ED (n = 319), with the medical floor being the second most common source (n = 213). Admissions to the IMCU from the ICU were infrequent (n = 45). With the exception of blood gas data, only 0.4% of data elements were missing, and these most often affected the SAPS 3 model due to the limited window of time for data collection. For the APACHE II and SAPS II, blood gas data were available in only 21% of patients, and for the SAPS 3, it was only available for 5% of patients. The only data element missing for calculation of the MPM0III was the GCS, which was not available in 6.5% of patients.

Table 1.

Characteristics of 628 Patients Admitted to the Intermediate Care Unit.

All Patients Survivors Deaths P Value
Number of patients 628 576 52
Age in years, median (IQR) 57 (45–67) 56 (44–66) 66 (55–76) <.001a
Female sex, % 53 53 46 .335b
IMCU source, % <.001c
 ED 50.8 52.8 28.9
 Floor 33.9 32.3 51.9
 ICU 7.2 7.6 1.9
 Procedure 3.7 3.8 1.9
 Other hospital 2.4 1.9 7.7
 Admitting 1.6 1.2 5.8
 IMCU 0.5 0.4 1.9
Primary diagnosis, % .042c
 Respiratory 30.7 30.4 34.6
 Cardiac 22.1 22.4 19.2
 Nonpulmonary sepsis 12.6 11.6 23.1
 Neurological 10.7 11.1 5.8
 Gastrointestinal 9.7 9.4 13.5
 Endocrine 7.5 7.2 0.0
 Metabolic/renal 6.7 6.9 3.9
Outcome prediction model
 Acute physiology score 15 (12–21) 15 (11–20) 22 (17–28) <.001a
 SAPS II Score 21 (13–30) 20 (13–29) 39 (27–51) <.001a
 SAPS 3 Score 46 (39–55) 45 (39–53) 59 (52–70) <.001a
Probability of in-hospital death
 APACHE II 0.18 (0.07–0.31) 0.16 (0.06–0.28) 0.38 (0.21–0.55) <.001a
 SAPS II 0.04 (0.02–0.11) 0.03 (0.02–0.09) 0.20 (0.08–0.47) <.001a
 SAPS 3 0.12 (0.06–0.26) 0.11 (0.06–0.22) 0.32 (0.20–0.55) <.001a
 SAPS 3NA 0.12 (0.07–0.23) 0.11 (0.17–0.21) 0.29 (0.19–0.46) <.001a
 MPM0III 0.06 (0.03–0.12) 0.06 (0.03–0.11) 0.15 (0.09–0.27) <.001a
Charlson Index, median (IQR) 2 (1–4) 2 (1–4) 4 (3–6) <.001a
Hospital mortality, % 8.3 0 100
DNI/DNR, % 5.7 4.3 21.2 <.001c
Hospital LOS, days, median (IQR) 6.9 (3.0–14.0) 6.5 (2.8–13.7) 12.2 (6.3–19.7) <.001a
Pre-IMCU LOS, days, median (IQR) 0 (0–1.5) 0 (0–1.3) 0.6 (0–4.3) .007a
IMCU LOS, days, median (IQR) 2.2 (1.3–3.8) 2.2 (1.4–3.7) 2.3 (1.0–4.6) .868a
Open in a new tab

Abbreviations: APS, Acute Physiology Score; DNI/DNR, Do Not Intubate/Do Not Resuscitate; ED, Emergency Department; GCS, Glasgow Coma Scale; ICU, Intensive Care Unit; IMCU, Intermediate Care Unit; IQR, Interquartile Range; LOS, Length of Stay; MPM0III, Mortality Probablility Model version 3; SAPS II, Simplified Acute Physiology Score version 2; SAPS 3, Simplified Acute Physiology Score version 3; NA: North American equation of SAPS 3.

a

Wilcoxon rank sum test.

b

Two sample test of proportions.

c

Fischer exact test.

In total, 52 IMCU patients died during their hospitalization. Of these deaths, 54% died after transfer to an ICU, 25% in the IMCU, and 19% after transfer to a general medical unit. Those who died were older, with higher comorbidity scores, and were more often admitted from a general medical unit. Patients who died had higher APACHE II, SAPS II, and SAPS 3 scores and a higher probability of death as predicted by all outcome prediction models, including the SAPS 3NA and MPM0III (Table 1).

Score Performance

Although all models showed acceptable and similar discrimination (P > .05), the AUROC for SAPS II was highest at 0.80 (95% confidence interval [95% CI]: 0.74–0.87; Table 2). The SAPS II and MPM0III exhibited good calibration, while calibration was poor for the APACHE II, SAPS 3, and SAPS 3NA (Table 2; Figure 1). Finally, the SMR for the SAPS II and MPM0III were similar, with modest overestimation of mortality. The APACHE II and both versions of the SAPS 3 markedly overestimated mortality (Table 2).

Table 2.

Performance of Outcome Prediction Models in the Intermediate Care Unit.

Score SMR (95% CI) GOF χ2 GOF P Value AUROC (95% CI)
APACHE II 0.38 (0.28–0.49) 97.7 <.001 0.76 (0.70–0.83)
SAPS II 0.91 (0.68–1.19) 7.8 .650 0.80 (0.74–0.87)
SAPS 3 0.47 (0.35–0.62) 47.6 <.001 0.78 (0.72–0.86)
SAPS 3NA 0.51 (0.38–0.66) 37.3 <.001 0.79 (0.72–0.86)
MPM0III 0.91 (0.68–1.19) 8.0 .630 0.78 (0.71–0.85)
Open in a new tab

Abbreviations: APACHE II, Acute Physiology and Chronic Health Evaluations version 2; AUROC, Area Under Receiving Operating Characteristic; CI, confidence interval; GOF, Goodness of Fit; MPM0III, Mortality Probablility Model version 3; NA, North American equation of SAPS 3; SAPS II, Simplified Acute Physiology Score version 2; SAPS 3, Simplified Acute Physiology Score version 3; SMR, Standardized Mortality Ratio.

Figure 1.

Figure 1.

Open in a new tab

A–D, Calibration of each model is shown. The total number of patients (left axis) in each decile of predicted risk of death (x-axis) is shown in gray bars. Calibration is characterized by a plot of observed (solid circles) and expected (open circles) in-hospital mortality (right axis) within each decile of risk. Less overlap (A and C) is consistent with poor calibration. By contrast, with good calibration (B and D), the 2 curves are close to being superimposed.

Discussion

The performance of 4 commonly used critical care outcome prediction models was characterized in a single-center study of IMCU patients. In this IMCU population, the SAPS II and MPM0III exhibited good discrimination, calibration, and only slightly overestimated mortality. By contrast, the APACHE II and both SAPS 3 models exhibited poor calibration and markedly overestimated mortality, although discrimination was comparable to the SAPS II and MPM0III.

The performance of the SAPS II in our IMCU population is notable in the context of its performance in 2 previously studied intermediate care settings.9,10 Although the patient populations in each prior study were predominantly medical, like our IMCU population, these assessments occurred in different countries at different times. In France, Auriant et al reported that the SAPS II performed well in a population of 433 patients admitted to a closed 4-bed IMCU with a nurse-to-patient ratio of 1:4, located within an ED, and in close proximity to an ICU.9 Triage into that unit and patient care were supervised by attending intensivists. Overall hospital mortality was 8%, the AUROC for the SAPS II was 0.85, calibration was good, and the SMR was 1.07. In Spain, Lucena et al also assessed the performance of the SAPS II in predicting hospital mortality among 607 patients admitted to a closed 9-bed IMCU, staffed by hospitalists, with a nurse-to-patient ratio of 1:3.10 The unit was adjacent to but independent from the ICU. Hospital mortality was 20%, the SAPS II exhibited an AUROC of 0.76, was well calibrated, and had an SMR of 0.87. If we include our experience, it is notable that the SAPS II has now been shown to perform well in 3 different models of intermediate care, in 3 different countries, over a period spanning nearly 15 years.

The MPM0III performed almost as well as the SAPS II in our population and is an appealing model for the IMCU setting for a number of reasons. It has a low burden of data collection with fewer variables than APACHE II, SAPS II, or SAPS 3.25,26 Like SAPS 3, all data collected are from within an hour of admission, which improves efficiency and reduces the complexity of data collection. Moreover, with the exception of age, all data elements are dichotomous and, thus, less subject to interpretation.21 In comparison to SAPS II, blood gas data and urine output are not needed for the MPM0III. Indeed, blood gasses are often not indicated for IMCU patients, and because shock syndromes are less often admitted to this level of care, many IMCU patients will not have urine output followed as closely as in an ICU setting. Notably, urine output is known to be a difficult variable to capture, even in the ICU.27 The MPM0III also considers established limitations to care, such as code status, which has been demonstrated to have an impact on hospital mortality.28,29 Finally, the MPM0III is the most recent model developed and validated and may therefore be more reflective of medical practice at the time the data for this study were collected.

Despite these appealing features and the performance of the MPM0III, we are aware of only 1 published use of MPM0III in the IMCU setting.16 In that single-center study (in North America), a formal assessment of MPM0III performance was not reported. However, using the reported mortality of 10.4% in that population, and the predicted mortality of 8.8%, the MPM0III appeared to underestimate mortality with an SMR of approximately 1.18. For comparison, the SMR in our IMCU was 0.91.

The APACHE II may be the most widely used outcome prediction model.30 Indeed, several studies addressing different aspects of intermediate care have used the APACHE II to characterize studied populations.11,12,14,15,31–34 However, only 1 study assessed performance of the model. In that study, the APACHE II exhibited excellent discrimination (AUROC = 0.85), but substantially overestimated mortality (SMR = 0.44).14 Calibration was not characterized.

The performance of the SAPS 3 in the IMCU setting was recently studied in another IMCU.7 Although the model demonstrated good discrimination (AUROC = 0.79), it also substantially overestimated mortality (SMR = 0.51). In our IMCU population, we observed a very similar performance of the SAPS 3 with an AUROC of 0.78, an SMR 0.47, and poor calibration. Further, use of the SAPS 3 equation for North America did not improve the performance.

Finally, we have previously assessed the performance of the ImCUSS, the only outcome prediction model to date developed and validated for use in the IMCU.7,8 This model has only 9 data elements that are dichotomous and assessed at the time of IMCU admission. Like the MPM0III, this low burden of data collection is appealing. In the IMCU in which it was developed, the performance of the ImCUSS was very similar to the performance of the SAPS II with good discrimination (AUROC 0.80; 95% CI: 0.73–0.87) and calibration (P = .33) and an SMR of 0.89. However, in our IMCU, the ImCUSS exhibited only acceptable discrimination (AUROC 0.72), poor calibration, and underestimated mortality (SMR 1.22).

It is not clear why the SAPS II and MPM0III performed relatively well in our population, while the APACHE II, SAPS 3, and ImCUSS performed less well. The decline in outcome prediction model performance over time has often been attributed to improvements in health care, especially when models predict a mortality that exceeds what is observed. This pattern has been demonstrated for the APACHE, SAPS, and MPM systems.21,35,36 However, the SAPS models have not always behaved this way. In fact, in some comparisons of SAPS II versus SAPS 3, the SAPS II model exhibited better performance.37,38

Other factors known to alter model performance may also explain some of the variability in the models we assessed. These factors include differences in case-mix, overfitting of the original models, and the presence of unmeasured context-sensitive features, such as socioeconomic conditions and differences in health-care delivery systems that may vary by geographical region.37,39 For example, the SAPS 3 was developed from a multinational cohort with little North American representation (only 5%, or 797 patients), which may have affected its performance in our IMCU population.20 In contrast, patients from North America composed 28% of the population used to develop the SAPS II and nearly all of those used to develop the MPM0III.18,21 Better representation of North American patients in the development of the SAPS II and MPM0III models may contribute to the better performance of these models in our IMCU.30 It should also be considered that the types of patients admitted to an IMCU in 2012 may fall along the spectrum of patients admitted to the many ICUs that contributed patient data to the development of the SAPS II and MPM0III models.

Inconsistent model performance may also be the result of different interpretations of definitions for different data elements.40 We mitigated this effect by using written definitions for the data components of each score to reflect the definitions used in the original articles.18–21

For descriptive studies of medical patients admitted to IMCUs, use and further assessment of the MPM0III and SAPS II are reasonable. If further validation were to demonstrate the MPM0III to perform well, it would be an appealing model to monitor care quality in the IMCU setting. Because the MPM0III is calculated at the time of admission, differences between observed outcomes and those predicted by the MPM0III are more likely to reflect differences in care quality and/or intermediate care organization. By contrast, models like the SAPS II, which are calculated from the most aberrant data available in the 24 hours following admission, have been criticized due to the potential that lower or higher scores could be the result of better or worse care. For example, consider 2 identical populations admitted to different IMCUs. One group may receive perfect care and as a result have less aberrant physiology and lower predicted and observed mortality. The other group may receive suboptimal care, progress to very aberrant physiology, and exhibit higher predicted and observed mortality. Despite significant differences in quality of care, the SMR (ie, observed/predicted mortality ratio) in each case may be identical. For these reasons, some have argued against using data acquired well after the time of admission for the purposes of comparing unit performance and care quality. This has been referred to as the Boyd and Grounds effect.41

The counterargument is that an expanded period of data collection, as with the SAPS II, will better detect significant derangements in physiology that have important predictive value, independent of care quality. Further, even if there are concerns regarding the appropriateness of comparing SMRs from data captured during the first 24 hours of IMCU admission, there is general agreement that higher scores correlate with a higher severity of illness and likelihood of death. The SAPS II is therefore also a useful parameter when describing a population under study and when adjusting outcomes other than care quality, such as mortality and length of stay.

There are limitations to our analysis. First, our results are from a single center and limited to predominantly noncardiac medical patients. Second, we assumed missing data elements were normal for each prediction model as is recommended.18–20,42 As a result, our calculations of predicted mortality could be low relative to predictions in the reference populations used to develop these models, and this could have the effect of overstating our measured SMRs. However, with the exception of blood gas data elements, which are often not indicated for IMCU patients, there were very few missing data (<1%). Third, our cohort was created in 2012. Because the performance of outcome prediction models may change over time, it is not certain the performance we observed would be true of a more recent study. However, our previous detailed characterization of our IMCU as of 2012 provides important context (ie, how our IMCU was staffed/organized) to the performance of these outcome prediction models at that time. Fourth, due to the labor-intensive nature of data collection, our cohort is relatively small and only represents 6 months of admissions to our IMCU. However, our manual collection of over 120 data elements (in duplicate for each patient with adjudication by a third independent reviewer as needed), gives us a very high level of confidence in the quality of our data. Although a larger scale, multicenter assessment of each model in the intermediate care setting would be of value, and necessary before any of these models could fairly be used for benchmarking, we are not aware of large IMCU data sets that reliably contain the data elements needed for each outcome prediction model.43,44 Finally, the mortality in the population was low relative to ICU mortality. It is possible that this affected the discrimination and calibration of the models assessed. However, as mentioned, it is likely that patients admitted to the IMCU fall along the spectrum of patients who historically would have been admitted to the ICU at the time the APACHE II, SAPS II and 3, and MPM0III were developed.

Conclusion

In a medical IMCU in an urban academic medical center in the United States, the SAPS II and MPM0III exhibited good discrimination and calibration for hospital mortality. Although the APACHE II and SAPS 3 exhibited good discrimination, neither was well calibrated and both markedly overestimated mortality. Further assessment of outcome predication models in the intermediate care setting is needed.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

  • 1.Prin M, Wunsch H. The role of stepdown beds in hospital care. Am J Respir Crit Care Med. 2014;190(11):1210–1216. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Sjoding MW, Valley TS, Prescott HC, Wunsch H, Iwashyna TJ, Cooke CR. Rising billing for intermediate intensive care among hospitalized Medicare beneficiaries between 1996 and 2010. Am J Respir Crit Care Med. 2016;193(2):163–170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Sakr Y, Moreira CL, Rhodes A, et al. The impact of hospital and ICU organizational factors on outcome in critically ill patients: results from the extended prevalence of infection in intensive care study. Crit Care Med. 2015;43(3):519–526. [DOI] [PubMed] [Google Scholar]
  • 4.Capuzzo M, Volta C, Tassinati T, et al. Hospital mortality of adults admitted to Intensive Care Units in hospitals with and without intermediate care units: a multicentre European cohort study. Crit Care. 2014;18(5):551. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Vincent JL, Burchardi H. Do we need intermediate care units? Intensive Care Med. 1999;25(12):1345–1349. [DOI] [PubMed] [Google Scholar]
  • 6.Vincent JL, Rubenfeld GD. Does intermediate care improve patient outcomes or reduce costs? Crit Care. 2015;19(1):89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Alegre F, Landecho MF, Huerta A, et al. Design and performance of a new severity score for intermediate care. PloS One. 2015; 10(6):e0130989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Hager DN, Tanykonda V, Noorain Z, et al. Hospital mortality prediction for intermediate care patients: assessing the generalizability of the intermediate care unit severity score (IMCUSS). J Crit Care. 2018;46:94–98. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Auriant I, Vinatier I, Thaler F, Tourneur M, Loirat P. Simplified acute physiology score II for measuring severity of illness in intermediate care units. Crit Care Med. 1998;26(8):1368–1371. [DOI] [PubMed] [Google Scholar]
  • 10.Lucena JF, Alegre F, Martinez-Urbistondo D, et al. Performance of SAPS II and SAPS 3 in intermediate care. PloS One. 2013; 8(10):e77229. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Simpson CE, Sahetya SK, Bradsher RW, et al. Outcomes of emergency medical patients admitted to an intermediate care unit with detailed admission guidelines. Am J Crit Care. 2017;26(1): e1–e10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Ip SP, Leung YF, Ip CY, Mak WP. Outcomes of critically ill elderly patients: is high-dependency care for geriatric patients worthwhile? Crit Care Med. 1999;27(11):2351–2357. [DOI] [PubMed] [Google Scholar]
  • 13.Meaudre E, Nguyen C, Contargyris C, et al. Management of septic shock in intermediate care unit. Anaesth Crit Care Pain Med. 2018;37(2):121–127. Epub 2017. [DOI] [PubMed] [Google Scholar]
  • 14.Hallengren M, Astrand P, Eksborg S, Barle H, Frostell C. Septic shock and the use of norepinephrine in an intermediate care unit: mortality and adverse events. PloS One. 2017;12(8):e0183073. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Torres OH, Francia E, Longobardi V, Gich I, Benito S, Ruiz D. Short- and long-term outcomes of older patients in intermediate care units. Intensive Care Med. 2006;32(7):1052–1059. [DOI] [PubMed] [Google Scholar]
  • 16.Yoo EJ, Damaghi N, Shakespeare WG, Sherman MS. The effect of physician staffing model on patient outcomes in a medical progressive care unit. J Crit Care. 2016;32:68–72. [DOI] [PubMed] [Google Scholar]
  • 17.Lucena JF, Alegre F, Rodil R, et al. Results of a retrospective observational study of intermediate care staffed by hospitalists: impact on mortality, co-management, and teaching. J Hosp Med. 2012;7(5):411–415. [DOI] [PubMed] [Google Scholar]
  • 18.Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–2963. [DOI] [PubMed] [Google Scholar]
  • 19.Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med. 1985;13(10):818–829. [PubMed] [Google Scholar]
  • 20.Moreno RP, Metnitz PG, Almeida E, et al. SAPS 3–From evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–1355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AA. Assessing contemporary intensive care unit outcome: an updated mortality probability admission model (MPM0-III). Crit Care Med. 2007;35(3):827–835. [DOI] [PubMed] [Google Scholar]
  • 22.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–383. [DOI] [PubMed] [Google Scholar]
  • 23.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3): 837–845. [PubMed] [Google Scholar]
  • 24.Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115(1):92–106. [DOI] [PubMed] [Google Scholar]
  • 25.Kuzniewicz MW, Vasilevskis EE, Lane R, et al. Variation in ICU risk-adjusted mortality: impact of methods of assessment and potential confounders. Chest. 2008;133(6):1319–1327. [DOI] [PubMed] [Google Scholar]
  • 26.Breslow MJ, Badawi O. Severity scoring in the critically ill: part 1–interpretation and accuracy of outcome prediction scoring systems. Chest. 2012;141(1):245–252. [DOI] [PubMed] [Google Scholar]
  • 27.Jin K, Murugan R, Sileanu FE, et al. Intensive monitoring of urine output is associated with increased detection of acute kidney injury and improved outcomes. Chest. 2017;152(5):972–979. [DOI] [PubMed] [Google Scholar]
  • 28.Bradford MA, Lindenauer PK, Wiener RS, Walkey AJ. Do-not- resuscitate status and observational comparative effectiveness research in patients with septic shock*. Crit Care Med. 2014; 42(9):2042–2047. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Walkey AJ, Weinberg J, Wiener RS, Cooke CR, Lindenauer PK. Association of do-not-resuscitate orders and hospital mortality rate among patients with pneumonia. JAMA Intern Med. 2016; 176(1):97–104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14(2):207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Ranzani OT, Zampieri FG, Taniguchi LU, Forte DN, Azevedo LC, Park M. The effects of discharge to an intermediate care unit after a critical illness: a 5-year cohort study. J Crit Care. 2014; 29(2):230–235. [DOI] [PubMed] [Google Scholar]
  • 32.Porath A, Reuveni H, Grinberg G, Lieberman D. The intermediate care unit as a cost-effective option for the treatment of medical patients in critical condition. Isr J Med Sci. 1995;31(11):674–680. [PubMed] [Google Scholar]
  • 33.Hager DN, Chandrashekar P, Bradsher RW, et al. Intermediate care to intensive care triage: A quality improvement project to reduce mortality. J Crit Care. 2017;42:282–288. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Fujii T, Uchino S, Takinami M. Life-threatening complications after postoperative intermediate care unit discharge: a retrospective, observational study. Eur J Anaesthesiol. 2016;33(1):22–27. [DOI] [PubMed] [Google Scholar]
  • 35.Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute physiology and chronic health evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–1310. [DOI] [PubMed] [Google Scholar]
  • 36.Glance LG, Osler TM, Dick A. Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med. 2002;30(9):1976–1982. [DOI] [PubMed] [Google Scholar]
  • 37.Poole D, Rossi C, Latronico N, Rossi G, Finazzi S, Bertolini G; GiViTi. Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better? Intensive Care Med. 2012;38(8):1280–1288. [DOI] [PubMed] [Google Scholar]
  • 38.Strand K, Soreide E, Aardal S, Flaatten H. A comparison of SAPS II and SAPS 3 in a Norwegian intensive care unit population. Acta Anaesthesiol Scand. 2009;53(5):595–600. [DOI] [PubMed] [Google Scholar]
  • 39.Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6): 515–524. [DOI] [PubMed] [Google Scholar]
  • 40.Vincent JL, Bruzzi de Carvalho F. Severity of illness. Semin Respir Crit Care Med. 2010;31(1):31–38. [DOI] [PubMed] [Google Scholar]
  • 41.Boyd O, Grounds M. Can standardized mortality ratio be used to compare quality of intensive care unit performance? Crit Care Med. 1994;22(10):1706–1709. [PubMed] [Google Scholar]
  • 42.Higgins TL, Kramer AA, Nathanson BH, Copes W, Stark M, Teres D. Prospective validation of the intensive care unit admission Mortality Probability Model (MPM0-III). Crit Care Med. 2009;37(5):1619–16123. [DOI] [PubMed] [Google Scholar]
  • 43.Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–2056. [DOI] [PubMed] [Google Scholar]
  • 44.Keenan SP, Doig GS, Martin CM, Inman KJ, Sibbald WJ. Assessing the efficiency of the admission process to a critical care unit: does the literature allow the use of benchmarking? Intensive Care Med. 1997;23(5):574–580. [DOI] [PubMed] [Google Scholar]

RESOURCES