Abstract
Background
The aim of this study was to determine which of the most commonly used scoring systems for evaluation of critically ill patients in the ICU is the best and simplest to use in our hospital.
Material/Methods
This prospective study included 60 critically ill patients. After admittance to the ICU, APACHE II, SAPS II, and MPM II0 were calculated. During further treatment in the ICU, SOFA and MPM II were calculated at 24 h, 48 h, and 72 h and 7 days after admittance using laboratory and radiological measures.
Results
In comparison with survivors, non-survivors were older (p<0.01) and spent significantly more days on mechanical ventilation (p<0.01). ARDS was significantly more common in patients who survived compared to those who did not (chi-square=7.02, p<0.01), which is not the case with sepsis (chi-square=0.388, p=0.53). AUROC SAPS II was 0.690, and is only slightly higher than the other 2 AUROC incipient scoring systems, MPM II and APACHE II (0.654 and 0.623). The APACHE II has the highest specificity (81.8%) and MPM II the highest sensitivity (85.2%). MPM II7day AUROC (1.0) shows the best discrimination between patients who survived and those who did not. MPM II48 (0.836), SOFA72 (0.821) and MPM II72 (0.817) also had good discrimination scores.
Conclusions
APACHE II and SAPS II measured on admission to the ICU were significant predictors of complications. MPM II7day has the best discriminatory power, followed by SOFA7day and MPM II48. MPM II7day has the best calibration followed by SOFA7day and APACHE II.
MeSH Keywords: Critical Illness, APACHE, Intensive Care Units, Organ Dysfunction Scores
Background
Critical care medicine is a complex, multidisciplinary specialty, designed to care for all sort of patients with critical illnesses. Critical illness is any disease process which causes physiological instability leading to disability or death within minutes, hours or days [1]. The main clinical indicators that point to a critical state of the patient are: hypotension, tachycardia, tachypnea, a reduced level of urine output, and altered consciousness. The sensitivity and specificity of these findings for critical illness are greatly improved if they are considered together.
In addition to required clinical observation of acutely or potentially critically ill patients, scoring systems are used at different stages of their in-hospital treatment. Scoring systems quantify the severity of critically ill and/or injured on the basis of anatomical, physiological, and biochemical variables and classify the patient in a specific risk group [2].
The most commonly used scoring systems in adult patients in the ICU are APACHE II and III, GCS, SAPS, MPM, SOFA, MODS, and LODS [3]. In the first 24 h of stay in the ICU, APACHE, SAPS, LODS, and MPM24 are calculated. Repetitive scores that are calculated in the further course of treatment in the ICU (SOFA and MODS) are primarily used for assessment of organ dysfunction of critically ill patients [4]. They are calculated daily during ICU stay.
In fact, scoring systems have become a necessary tool to describe ICU populations and to explain differences in mortality. However, it is also important to note that the choice of the severity score scale, index, or model should accurately match the event, setting, or application because misuse of such systems can lead to wasted time, increased cost, unwarranted extrapolations, and poor science [5].
APACHE (Acute Physiology And Chronic Health Evaluation) is an acute physiological and chronic health evaluation of critically ill patients [6]. The most commonly used is the second version of the score, which monitors 12 physiological parameters as well as patient age and competing comorbidities. It is a comprehensive system that integrates clinical parameters through a scoring system of 0 to 71. Then we estimate the prediction of mortality for the given condition, expressed as a percentage. APACHE III follows the 17 variables. The APACHE III scoring is done on a daily basis and calculates whether the patient’s condition improves or deteriorates daily. An increased score represents deterioration of the disease. APACHE score is declared the “gold standard” for the evaluation of intensive care and is one of the most commonly used scoring systems in intensive care unit around the world [7,8].
SAPS (Simplified Acute Physiology Score) was first described in 1984 as an alternative to the APACHE. The original score was assayed in the first 24 h of admission to the ICU and included 14 physiological variables, but did not include previous diseases. It is now replaced by SAPS II and SAPS III score, which include 12 physiological variables during the first 24 h of admission to the ICU and include the reason for admission (planned or emergency surgery or other reasons), the previous medical condition and years of age. It is not recommended for children under 18 years of age, patients with burns, or cardiac patients [8–10].
SOFA (Sepsis-related Organ Failure Assessment Score) was designed by a group of scientists from the European Society of Intensive Care Medicine, aimed to describe the degree of organ dysfunction in sepsis. However, since that time it is used for assessment of organ dysfunction of critically ill patients, regardless of the cause. It rates the 6 major organ systems: respiratory, cardiovascular, CNS, renal, liver, and coagulation (1 to 4 points), to give the final score of 6 to 24 points maximum [5,11].
MPM (Mortality Prediction Model) an assessment of in-hospital mortality based on chronic medical conditions and the diagnosis of acute and physiological variables. The MPM0, admission score includes 15 variables and MPM24 24-h score that contains 5 admission variables and 8 additional variables, and is designed for patients who remain in the ICU for more than 24 h. Scoring can be done every day during the stay of critically ill patients in the ICU (e.g., MPM24, MPM48, MPM72) [4,12].
MODS – Multiple Organ Dysfunction Score constructed by simply following the dysfunction of six organ systems, which are strongly correlated with mortality in ICU and the hospital at all. Each system is marked from 0 to 4. MODS is correlated with the SOFA [13].
LODS (Logistic Organ Dysfunction Score) is determined in the first 24 h. It is significantly simpler to calculate than the APACHE II and III scores. Calculating the LOD score can predict mortality rate in ICU, but its predictive value is less than the APACHE II and III [14]. The LODS lies somewhere between a mortality prediction score and an organ failure score because it combines a global score summarizing the total degree of organ dysfunction across the organ systems with and a logistic regression equation that can be used to convert the score into a probability of mortality [4].
The primary aim of this study was to assess the clinical application of the APACHE II, SAPS II, MPM II, and SOFA scales to evaluate the risk of death in critically ill patients in the Intensive Care Unit (ICU). The secondary aims were to investigate and demonstrate which of the most commonly used scoring systems for evaluation of critically ill patients in the ICU is the best and simplest to use in our hospital, and which is the most objective in assessing the current general condition and predicting length of stay and occurrence of complications.
Material and Methods
Our research was a prospective study conducted over a period from 01.02.2013 to 01.02.2014 at the Clinical Hospital Centre Bezanijska Kosa in Belgrade. Sixty critically ill patients older than 18 years who were hospitalized in the surgical ICU were randomly selected.
The following prognostic scales were used to assess the patients’ conditions: APACHE II, SAPS II, MPM II, and SOFA. After admittance to the ICU, APACHE II, SAPS II, and MPM II0 were calculated in accordance with the original methodology, using the worst physiological values on the first ICU day. During further treatment in the ICU, SOFA and MPM II were calculated at 24 h, 48 h, 72 h, and 7 days after admittance using certain laboratory and radiological variables. Of the existing ICU scoring systems, we included these systems because there are frequently used in the critical care literature and have performed well in comparison to other critical care scoring systems [15].
And the following parameters were analyzed in detail during treatment: demographic data (e.g., age and sex), admission surgical diagnosis, the type and number of surgical interventions and their extent and occurrence of complications, surgical and medical, number of days spent on mechanical ventilation, and length of stay in the ICU. Scores were calculated online using the SFAR (Société Française d’Anesthesie et de Reanimation) website calculator, where the numerical value of the score and the percentage of predicted mortality is given. The ethics approval was obtained from the Human Research Ethics Committee (HREC) of our hospital before the research began.
Descriptive statistical methods and methods for statistical hypothesis testing were used in the analysis of primary data. Data are presented for continuous variables as means, median, and standard deviation, and for categorical variables as absolute and relative frequencies. For testing hypothesis about difference of means between the 2 groups, continuous variables were compared using either the t-test (normal distribution) or the Mann-Whitney test (non-normal distribution). To compare the mean values between 3 or more groups, 1-way analysis of variance (ANOVA) applying the Tukey post hoc analysis was used to detect differences within the groups if they showed normal distribution. Data that did not show normal distribution were analyzed with the use of the Kruskal-Wallis test. To test hypothesis about the difference in frequency, the chi-squared test was used. Bivariate correlation was tested by the Spearman’s and Pearson’s tests, where appropriate.
The prognostic performance of the different scores was tested by assessing the calibration and discrimination. Calibration (i.e., the degree of agreement between the predicted mortality and actual mortality) was assessed by calculating Lemeshow-Hosmer goodness-of-fit test. A high p value (>0.05) indicated a good fit of the model. Discrimination (i.e., the ability to distinguish between survivors and non-survivors) was assessed using the Receiver Operator Characteristic (ROC) Curves and the Area under ROC curves (AUROC). A value of 0.5 indicated no discrimination, while 1 indicated perfect discrimination. Both discrimination and calibration are important to determine whether a score is appropriate to use in a given population [9].
P values of <0.05 were considered statistically significant for the analysis. The IBM SPSS Statistics 22 statistical software package was used in the statistical analysis.
Results
Demographic and general characteristics of the patients are shown in Table 1. The study included 60 patients of whom 27 (45%) did not survive. In comparison with survivors, non-survivors were older (t=3.227, p<0.01) and spent significantly more days on mechanical ventilation (U =183.0, p<0.01). The average number of days spent in the ICU (ICU LOS) did not differ significantly in regards to the final outcome (U=435.5, p=0.881). Arithmetic mean of MPM II0 and SAPS II in patients who survived was statistically significantly lower than in patients who did not survive (t=2.249, p<0.05) compared to the arithmetic mean of APACHE II and SOFA24, which was not statistically significant.
Table 1.
Demographic and general characteristics of the patients.
| Parameters | Total | Survivor | Non-survivor | p |
|---|---|---|---|---|
| Total number n (%) 60 (100.0) | 33 (55%) | 33 (55%) | 27 (45%) | |
| Sex | ||||
| Men n (%) | 34 (56.7) | 20 (60.6) | 14 (51.9) | 0.50 |
| Women n (%) | 26 (43.3) | 13 (39.4) | 13 (48.1) | |
| Age (yr. mean ±SD) | 59.0±15.8 | 53.5±14.8 | 65.8±14.6 | <0.01* |
| Duration of mechanical ventilation (days) (mean ±SD) | 3.9±2.8 | 2.6±1.7 | 5.6±3.0 | <0.01* |
| LOS ICU (days) (mean ±SD) | 7.5±3.9 | 7.5±3.8 | 7.4±4.2 | 0.992 |
| MPM II0 (mean ±SD) | 69.4±19.4 | 64.4±20.3 | 75.4±16.7 | <0.05* |
| SAPS II (mean ±SD) | 50.3±15.6 | 45.9±15.3 | 55.6±14.6 | <0.05* |
| APACHE II (mean ±SD) | 22.2±6.0 | 21.0±5.4 | 23.7±6.5 | 0.085 |
| SOFA24 (mean ±SD) | 11.8±2.9 | 11.5±2.6 | 12.1±3.3 | 0.388 |
| Complications | ||||
| ARDS n (%) | 11 (18.3) | 10 (30.3) | 1 (3.7) | <0.01* |
| Sepsis n (%) | 18 (30.0) | 11 (33.3) | 7 (25.9) | 0.53 |
| MODS and MOF n (%) | 18 (30.0) | 1 (3.0) | 17 (63.0) | <0.01* |
| Without complication n (%) | 13 (21.7) | 11 (33.3) | 2 (7.4) | <0.05* |
| Surgical treatment | ||||
| Unoperated | 10 (16.7) | 7 (21.2) | 3 (11.1) | 0.30 |
| Operated on admission | 25 (41.7) | 16 (48.5) | 9 (33.3) | 0.24 |
| Operated at a later time | 17 (28.3) | 5 (15.2) | 12 (44.4) | <0.05* |
| Operated more than once | 8 (13.3) | 5 (15.2) | 3 (11.1) | 0.65 |
| Admission surgical diagnosis | ||||
| GIT – bleeding | 10 (16.7) | 6 (18.2) | 4 (14.8) | 0.73 |
| Ileus | 8 (13.3) | 3 (9.1) | 5 (18.5) | 0.29 |
| Peritonitis acute | 17 (28.3) | 12 (36.4) | 5 (18.5) | 0.13 |
| Hernia inguinalis incarcerata | 3 (5.0) | 3 (9.1) | 0 (0.0) | 0.11 |
| Pancreatitis acuta | 10 (16.7) | 4 (12.1) | 6 (22.2) | 0.30 |
| St. post op during this hospitalization. – surg. complication | 12 (20.0) | 5 (15.2) | 7 (25.9) | 0.30 |
| Malignancy | ||||
| Yes | 21 (35.0) | 14 (42.4) | 7 (25.9) | 0.183 |
| No | 39 (65.0) | 19 (57.6) | 20 (74.1) | |
LOS ICU – Length of Stay in the Intensive Care Unit;MPM II – Mortality prediction model II; SAPS II – Simplified Acute Physiology Score II; APACHE II – Acute Physiology and Chronic Health Evaluation II; SOFA – Sepsis-related Organ Failure Assessment Score; ARDS – Acute respiratory distress syndrome; MODS – Multiple Organ Dysfunction Syndrome; MOF – Multiple Organ Failure; GIT – Gastrointestinal Tract.
ARDS was significantly more common in patients who survived compared to those who did not (chi-square=7.02, p<0.01), which was not the case with sepsis (chi-square=0.388, p=0.53). MODS and MOF were significantly more common in patients who did not survive than patients who survived (chi-square=25.4, p<0.01). The proportion of patients with no complications was significantly higher in patients who survived compared to those who did not survive (chi-square =5.88, p<0.05).
The incidence of patients operated on at a later period was statistically significantly more frequent in the group of non-surviving patients (44% vs. 15%) (chi-square=6.28, p<0.05). The incidence of other forms of operative treatment and the outcome were not statistically significantly associated. The type of admission, surgical diagnosis, and the outcome were not statistically significantly associated, nor was the presence of malignancy (Table 1).
In comparison of patients with ARDS, sepsis, MODS, MOF, and patients without complications, there was no statistically significant difference between the average age (F=2.614, p=0.06). Comparing both groups of patients, a statistically significant difference was found in median SAPS II and APACHE II (chi-square=14.164, p<0.01; chi-square=13.124, p<0.01), which was not the case with median MPM II0 and SOFA24 (Table 2).
Table 2.
The occurrence of complications according to age and scoring systems.
| Parameters | Complications | p | |||
|---|---|---|---|---|---|
| ARDS | Sepsis | MODS and MOF | Without complication | ||
| Number (%) | 11 (18.0) | 18 (30.0) | 18 (30.0) | 13 (22.0) | |
| Age (mean ±SD) | 54.3±15.6 | 60.0±14.9 | 66.1±14.3 | 51.9±16.4 | 0.06 |
| MPM II0 (mean ±SD, (Med)) | 67.6±21.5 (68.0) | 69.3±19.6 (79.0) | 78.6±12.8 (78.0) | 58.2±20.6 (57.0) | 0.062 |
| SAPS II (mean ±SD, (Med)) | 50.1±10.1 (49.0) | 47.6±18.0 (48.0) | 60.4±12.0 (60.0) | 40.2±13.1 (40.0) | <0.01* |
| APACHE II (mean ±SD) | 23.7.±4.1 (25.0) | 20.4±7.2(20.5) | 25.7±5.7(26.0) | 18.9±2.9 (18.0) | <0.01* |
| SOFA24 (mean ±SD) | 11.3±2.7 | 11.7±3.5 | 12.4±2.6 | 11.4±2.6 | 0.718 |
MPM II – Mortality prediction model II; SAPS II – Simplified Acute Physiology Score II; APACHE II – Acute Physiology and Chronic Health Evaluation II; SOFA – Sepsis-related Organ Failure Assessment Score.
There was a statistically significant correlation between length of stay on mechanical ventilation and the following scoring system for all patients: a weak positive correlation with the MPMII48 (p<0.05), a medium positive correlation (p<0.01) with MPMII72, a medium positive correlation (p<0.01) with MPMII7d, a weak positive correlation (p<0.05) with SOFA48, a medium positive correlation (p<0.01) with SOFA72, and a medium positive correlation (p<0.01) with SOFA7day. There was no statistically significant correlation between length of stay in the ICU and the scoring systems for all patients (Table 3).
Table 3.
Correlation LOS MV and LOS ICU with scoring systems.
| Scoring system | LOS MV | LOS ICU | ||||
|---|---|---|---|---|---|---|
| R | p | N | r | p | n | |
| MPMII0 | 0.191 | 0.143 | 60 | −0.119 | 0.366 | 60 |
| MPMII24 | 0.117 | 0.374 | 60 | −0.128 | 0.328 | 60 |
| MPMII48 | 0.263 | <0.05* | 60 | −0.087 | 0.510 | 60 |
| MPMII72 | 0.349 | <0.01* | 60 | −0.016 | 0.906 | 55 |
| MPMII7d | 0.617 | <0.01* | 60 | −0.050 | 0.773 | 36 |
| SOFA24 | 0.120 | 0.360 | 60 | −0.146 | 0.267 | 60 |
| SOFA48 | 0.268 | <0.05* | 60 | −0.031 | 0.813 | 60 |
| SOFA72 | 0.457 | <0.01* | 55 | 0.043 | 0.755 | 55 |
| SOFA7d | 0.619 | <0.01* | 36 | 0.002 | 0.991 | 36 |
| SAPS II | 0.192 | 0.141 | 60 | −0.016 | 0.902 | 60 |
| APACHE II | 0.044 | 0.738 | 60 | −0.137 | 0.298 | 60 |
LOS MV – Length of Stay on Mechanical ventilation; LOS ICU – Length of Stay in the Intensive Care Unit; MPM II – Mortality prediction model II; SOFA – Sepsis-related Organ Failure Assessment Score; SAPS II – Simplified Acute Physiology Score II; APACHE II – Acute Physiology and Chronic Health Evaluation II.
Calculating the Pearson correlation coefficient between the MPM II and SOFA scores each day, we proved that there is a strong positive correlation between MPM72 and SOFA72 and between MPM7day and SOFA7day (r>0.69, p<0.01). Among patients who survived, there was a statistically significant medium positive correlation between APACHE II, SOFA24 and SOFA48 (p<0.01). Among patients who did not survive, there was a statistically significant medium positive correlation between APACHE II, SOFA24 and SOFA7day (p<0.05) (Table 4).
Table 4.
Correlation SOFA each day and MPM and APACHE II by the outcome.
| Scoring system | Correlation | SOFA24 | SOFA48 | SOFA72 | SOFA7d |
|---|---|---|---|---|---|
| MPMII24 | r | 0.405 | 0.451 | 0.202 | 0.224 |
| p | <0.01* | <0.01* | 0.140 | 0.188 | |
| n | 60 | 60 | 55 | 36 | |
| MPMII48 | r | 0.233 | 0.483 | 0.390 | 0.320 |
| p | 0.073 | <0.01* | <0.01* | 0.057 | |
| n | 60 | 60 | 55 | 36 | |
| MPMII72 | r | 0.122 | 0.359 | 0.692 | 0.428 |
| p | 0.377 | <0.01* | <0.01* | <0.01* | |
| n | 55 | 55 | 55 | 36 | |
| MPMII7d | r | −0.002 | 0.192 | 0.483 | 0.855 |
| p | 0.989 | 0.261 | <0.01* | <0.01* | |
| n | 36 | 36 | 36 | 36 | |
| APACHE II survivor | r | 0.680 | 0.658 | 0.145 | −0.299 |
| p | <0.01* | <0.01* | 0.437 | 0.213 | |
| n | 33 | 33 | 31 | 19 | |
| APACHE II Not survivor | r | 0.419 | 0.312 | 0.278 | −0.570 |
| p | <0.05* | 0.113 | 0.188 | <0.05* | |
| n | 27 | 27 | 24 | 17 |
r – Pearson correlation coefficient; n – number of patients; * statistical significance. r>0 – positive correlation (increase of the value of one variable leads to an increase in the value of other variables); r<0 – negative correlation (increase of the value of one variable leads to a reduction in the value of other variables). MPM II – Mortality Prediction Model II; SAPS II – Simplified Acute Physiology Score II; APACHE II – Acute Physiology and Chronic Health Evaluation II; r – Pearson correlation coefficient; n – number of patients; * statistical significance; r >0 – positive correlation (increase the value of one variable lead to an increase in the value of other variables); r <0 – negative correlation (increase the value of one variable leads to a reduction in the value of other variables).
MPM II7day (0.0) with p=1.0 had the best calibration, followed by SOFA7day (0.85) with p=0.991 and APACHE II (3:05) with p=0.931. SOFA24 (13:35) with p=0.038 and MPM II24 (16.23) with p=0.039 had the worst calibration, suggesting that SOFA7day had the smallest statistically significant discrepancy between the observed and expected deaths (Table 5).
Table 5.
Lemeshow-Hosmer calibration test.
| χ2 | P | |
|---|---|---|
| MPM II | 6.90 | 0.548 |
| MPM II24 | 16.23 | 0.039 |
| MPM II48 | 11.37 | 0.181 |
| MPM II72 | 6.04 | 0.534 |
| MPM II7d | 0.00 | 1.000 |
| SOFA24 | 13.35 | 0.038 |
| SOFA48 | 9.12 | 0.332 |
| SOFA72 | 8.83 | 0.265 |
| SOFA7d | 0.85 | 0.991 |
| SAPS II | 4.41 | 0.732 |
| APACHE II | 3.05 | 0.931 |
MPM II – Mortality Prediction Model II; SOFA – Sepsis-related Organ Failure Assessment Score; SAPS II – Simplified Acute Physiology Score II; APACHE II – Acute Physiology and Chronic Health Evaluation II.
AUROC SAPS II was 0.690, and is only slightly higher than the other 2 AUROC incipient scoring systems, MPM II and APACHE II (0.654 and 0.623). However, the APACHE II has the highest specificity (81.8%) and MPM II the highest sensitivity (85.2%) (Figure 1).
Figure 1.
Area under curve (AUROC) for three scoring systems. APACHE II – Acute Physiology And Chronic Health Evaluation II; SAPS II – Simplified Acute Physiology Score II; MPM II – Mortality prediction model II.
MPM II7day AUROC (1.0) shows the best discrimination between patients who survived and those who did not, followed by SOFA7days AUROC (0.981). MPM II48 (0.836), SOFA72 (0.821) and MPM II72 (0.817) also had good discrimination scores. MPM II7day had the highest sensitivity and specificity (100.0%) and SOFA7day had specificity of 100.0% (Figure 2).
Figure 2.
Area under curve (AUROC) for two scoring systems each days. MPM II – Mortality prediction model II; SOFA – Sepsis-related Organ Failure Assessment Score.
Discussion
A critically ill patient is one at imminent risk of death and needs medical treatment in the ICU for more than 5 days [17]. Severe sepsis and septic shock are major reasons for intensive care unit (ICU) admission and are leading causes of mortality in non-coronary ICUs [18]. Apart from in the Western countries (including the USA, France, Italy, and the UK), little is known about the outcomes of patients admitted to the ICU with severe sepsis and septic shock, despite the seriousness of sepsis as a public health problem in developing countries such as Serbia [14].
According to the literature, over 50% of patients in most ICUs are older than 65 years. Age, although a main variable of almost all the scoring systems used in critically ill patients, may not be the main parameter for admission or discharge from the ICU. Comorbidity, currently evaluated by physiological state and the main cause that led to the development of a condition that requires continuous monitoring, are basic variables for assessing outcome of critically ill patients in the ICU within different scoring systems [7]. In our study, patients who did not survive were significantly older than patients who survived (SD – 65.8±14.6 years old).
The diagnosis must be documented within the first ICU day; should reflect the primary reason for ICU admission; and when multiple diagnoses are relevant, should be the diagnosis with the worst prognosis (e.g., sepsis rather than hyperglycemia) [19]. Peritonitis was the most common surgical diagnosis at admission in our study (17 patients, 28.3%). Secondary peritonitis related to a pathologic process in a visceral organ is by far the most common form of peritonitis encountered in clinical practice. Necrotizing pancreatitis can also be associated with peritonitis in case of an infection of the necrotic tissue. In our study, pancreatitis occurred in 10 patients (16.7%) and resulted in death in 6 (22.2%). The overall incidence of peritoneal infection and abscess is difficult to establish and varies with the underlying abdominal disease processes. Uncomplicated SP and simple abscesses carry a mortality rate of less than 5%, but this rate may increase to 30–50% in severe infections. In our study, 5 patients (18.5%) admitted to the ICU with a diagnosis of peritonitis died [20].
Breslowi published the results obtained from the US ICU Program 2008 database about the most common ICU admission diagnoses. GI bleeding was present in 1.7% and sepsis in 1.4% [19]. In our study, bleeding from the GI tract was the admission diagnosis in 16.7% and sepsis in 30% of cases.
Factors that independently predict a worse outcome in patients with peritonitis include advanced age, malnutrition, presence of cancer, preoperative organ dysfunction, and high values in scoring systems, in particular APACHE II score on presentation [16]. The concurrent development of sepsis, SIRS, and MOF can increase the mortality rate to greater than 70%, and in these patients more than 80% of deaths occur with an active infection present. In general, the mortality rate is less than 5% with an APACHE II score of less than 15 and rises to greater than 40% with scores above 15. In our patients, MODS and MOF was statistically more frequent in patients who did not survive. The proportion of patients with no complications was significantly higher in patients who survived.
Development of MOFA and/or consequential re-laparotomy delayed for more than 24 hours leads to a higher mortality in patients with postoperative intra-abdominal infections [21]. A delay in instituting either medical therapy or surgical therapy has been clearly associated with increased complication rates, the development of tertiary peritonitis, the need for reoperation, MODS and consequently increases and mortality. In many cases, emergency surgery is required, especially if peritonitis has been caused by conditions such as appendicitis, a perforated stomach ulcer, or diverticulitis [20]. And in our study the respondents operated on at a later period had a significantly higher mortality rate. Patients with intra-abdominal infections have a greater benefit from an aggressive and prompt surgical treatment and re-laparotomy if there is any indication, all in order to better control the development of MODS [22].
The existence of a large number of prognostic scoring systems suggests that the ideal model is still not found [1]. This is why different authors prefer different scoring systems in relation to others. APACHE and MPM were developed predominantly from patients cared for in the US ICUs, whereas SAPS included patients from 35 countries [8–10,12,23].
In a retrospective study of 11,300 patients from 35 hospitals in California, authors noted that only the APACHE scoring system shows good discrimination and calibration for predicting mortality in the ICU and length of stay in hospital. The same authors have shown that MPM showed better prediction for assessing the duration of mechanical ventilation and length of stay in the ICU. Considering that one of the main goals of the health system is the reduction in cost of treatment, assessment of the length of stay in the ICU and the hospital in general is of great importance [24].
In our study there was no statistically significant correlation between length of stay in the ICU and scoring systems for all patients. However, there is a statistically significant correlation between the length of stay on mechanical ventilation and individual scoring systems in all patients and in such a way that the most pronounced positive correlation was for MPM II72 and MPMII7day and SOFA72 and SOFA7day, which coincides with the data of Vasilevski and associates. There is a statistically significant mean negative correlation between length of stay on mechanical ventilation and MPMII24 in patients who did not survive (p=0.05).
Calculating the Pearson correlation coefficient between the MPM II and SOFA scores each day we have proved that there is a strong positive correlation between MPM72 and SOFA72 and between MPM7dan and SOFA7dan which means that increase of the value of one variable leads to an increase of the value of other variables.
In a study of 10,393 patients from Scottish ICUs, Livingston and colleagues compared the APACHE II, SAPS II, MPM0 and MPM24. These authors reported that all models showed good discrimination. SAPS II had the best performance overall, but APACHE II had better calibration. In our case, MPM II7day showed the best calibration, followed by APACHE II. MPM II7day showed the best discrimination although MPM II measured initially showed better discrimination than the APACHE II [25].
Several studies have directly compared APACHE and SOFA to predict hospital mortality in the ICU and the hospital in general [26]. And again, different authors publish different data, i.e. some give priority to the APACHE score on admission and others to the SOFA score. In a study from 2007, the APACHE II score was reported to have slightly better discrimination and calibration at admission than SOFA [25].
However, Janssens et al. reported that at admission SOFA has a higher AUC (0.82) than SAPS II (0.77). Finally, five studies that are listed in this Review article compared SOFA with other organ failure scores [13,25,27]. Generally, no clear differences were found in calibration or discrimination.
In our study SOFA7day showed better calibration compared to APACHE II, suggesting that SOFA7days had the small statistically significant discrepancy between the observed and expected deaths. By calculating the correlation between APACHE II and SOFA each day we have proved that there is a statistically significant mean positive correlation in the first two days of stay in the ICU.
We have also demonstrated by calculating AUROC that the MPM II7day score had the best discrimination between patients with lethal outcome and patients who survived – AUROC (1.0). Also with good discrimination scores were SOFA7day (0.981), MPM II48 (0836), MPM II72 (0817) and SOFA72 (0.821). In this way we proved that for our patient population MPM II and SOFA score is the most authoritative especially in the later period of treatment in the ICU and the initial measurements are more valid in MPM compared to APACHE II. However, calculating the correlation of certain scoring systems and the occurrence of complications, we concluded that among patients with ARDS, sepsis, MODS and MOF there is a statistically significant difference of the APACHE II median and the SAPS II median, which MPM II and SOFA scores did not show [28].
Although these scoring systems have certain advantages, limitations still exist in routine use. These scores were prospectively recorded by medical personnel, a bias due to differences in calculating scores and validating patient-derived parameters cannot be completely excluded. Second, this study was conducted at only one center. The results therefore, reflect the outcome of specific patients in a tertiary care center and may not be generally applicable to all hospitals in all cases. The question of where the patients came from and how long they were in the hospital prior to ICU admission may influence outcome [5]. The scoring systems are not adequate to make decisions for the management of individual patients due to the relatively high mortality rate predicted in survivors and the low one predicted in non-survivors [29]. The general conclusion is that the decision on the continuation of treatment should not be made solely on the basis of the value of these scoring systems, but they are to be used only to evaluate the predicted mortality [23,30].
Conclusions
APACHE II and SAPS II scores measured on admission to the ICU were significant predictors of complications. MPM II 7day has the best discriminatory power, followed by SOFA 7day and MPM II48. MPM II7day has the best calibration followed by SOFA7day and APACHE II. In patients on mechanical ventilation, MPM II and SOFA scores measured on the third and seventh days were significant for the prognosis of outcome.
Based on the results of this study, our institution, in order to predict the outcome of critically ill patients, began routine use of APACHE II and SAPS II at admission to the ICU, and during the hospital stay we use MPM II and SOFA scores.
Footnotes
Source of support: Departmental sources
References
- 1.Boniati MM, Friedman G, Castilho RK, et al. Characteristics of chronically critically ill patients: comparing two definitions. Clinics (Sao Paulo) 2011;66(4):701–4. doi: 10.1590/S1807-59322011000400027. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Harrison DA, Parry GJ, Carpenter JR, et al. A new risk prediction model for critical care: the Intensive Care National Audit & Research Centre (ICNARC) model. Crit Care Med. 2007;35(4):1091–98. doi: 10.1097/01.CCM.0000259468.24532.44. [DOI] [PubMed] [Google Scholar]
- 3.Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;52:467–78. doi: 10.1111/j.1399-6576.2008.01586.x. [DOI] [PubMed] [Google Scholar]
- 4.Vincent JL, Moreno RP. Clinical review: Scoring systems in the critically ill. Crit Care. 2010;14(2):142–311. doi: 10.1186/cc8204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Rapsang AG, Shyam DC. Scoring systems in the intensive care unit: A compendium. Indian J Crit Care Med. 2014;18(4):220–28. doi: 10.4103/0972-5229.130573. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Zimmerman JE, Kramer A, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310. doi: 10.1097/01.CCM.0000215112.84523.F0. [DOI] [PubMed] [Google Scholar]
- 7.Bouch C, Thompson J. Severity scoring systems in the critically ill. Continuing Education in Anaesthesia. Critical Care and Pain. 2008;8(5):181–85. [Google Scholar]
- 8.Salluh JI, Soares M. ICU severity of illness scores: APACHE, SAPS and MPM. Curr Opin Crit Care. 2014;20(5):557–65. doi: 10.1097/MCC.0000000000000135. [DOI] [PubMed] [Google Scholar]
- 9.Metnitz PGH, Moreno RP, Almeida E, et al. SAPS 3 Investigators. SAPS 3-from evaluation of the patient to evaluation of the intensive care unit. Part 1: objectives, methods and cohortdescription. Intensive Care Med. 2005;31(10):1336–44. doi: 10.1007/s00134-005-2762-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Moreno RP, Metnitz P, Almeida E, et al. SAPS 3 Investigators. SAPS 3-from evaluation of the patient to evaluation of the intensive care unit. Part 2: development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–55. doi: 10.1007/s00134-005-2763-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Sawicka W, Owczuk R, Wujtewicz MA, Wujtewicz M. The effectiveness of the APACHE II, SAPS II and SOFA prognostic scoring systems in patients with haematological malignancies in the intensive care unit. Anaesthesiol Intensive Ther. 2014;46(3):166–70. doi: 10.5603/AIT.2014.0030. [DOI] [PubMed] [Google Scholar]
- 12.Higgins TL, Teres D, Copes WS, et al. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III) Crit Care Med. 2007;35(3):827–35. doi: 10.1097/01.CCM.0000257337.63529.9F. [DOI] [PubMed] [Google Scholar]
- 13.Bota PD, Melot C, Ferreira LF, et al. The multipleorgan dysfunction score (MODS) versus the sequential organ failure assessment (SOFA) score in outcome prediction. Intensive Care Med. 2002;28:1619–24. doi: 10.1007/s00134-002-1491-3. [DOI] [PubMed] [Google Scholar]
- 14.Arabi Y, Shirawi NA, Memish Z, et al. Assessment of six mortality prediction models in patients admitted with severe sepsis and septic shock to the intensive care unit: a prospective cohort study. Crit Care. 2003;7(5):116–22. doi: 10.1186/cc2373. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Moseson EM, Zhuo H, Chu J, et al. Intensive care unit scoring systems outperform emergency department scoring systems for mortality prediction in critically ill patients: a prospective cohort study. J Intensive Care. 2014;2:40. doi: 10.1186/2052-0492-2-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Daley JB, Katz J, Anand BS, et al. Peritonitis and abdominal sepsis. Medscape. Updated: Feb 23, 2015. [Google Scholar]
- 17.Khwannimit B, Geater A. A comparison of APACHE II and SAPS II scoring systems in predicting hospital mortality in Thai adult intensive care units. J Med Assoc Thai. 2007;90(4):643–52. [PubMed] [Google Scholar]
- 18.Dellinger RP, Levy MM, Rhodes A. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580–637. doi: 10.1097/CCM.0b013e31827e83af. [DOI] [PubMed] [Google Scholar]
- 19.Breslow JM, Badawi O. Severity scoring in the critically ill (part 1 – interpretation and accuracy of outcome prediction scoring systems) Chest. 2012;141(1):245–52. doi: 10.1378/chest.11-0330. [DOI] [PubMed] [Google Scholar]
- 20.Bosscha K, Reijnders K, Hulstaert PF, et al. Prognostic scoring systems to predict outcome in peritonitis and intra-abdominal sepsis. Br J Surg. 1997;84(11):1532–34. [PubMed] [Google Scholar]
- 21.Torer N, Yorganci K, Elker D, Sayek I. Prognostic factors of the mortality of postoperative intraabdominal infections. Infection. 2010;38(4):255–60. doi: 10.1007/s15010-010-0021-4. [DOI] [PubMed] [Google Scholar]
- 22.Hutchins RR, Gunning MP, Lucas DN, et al. Relaparotomy for suspected Intraperitoneal sepsis after abdominal surgery. World J Surg. 2004;28(2):137–41. doi: 10.1007/s00268-003-7067-8. [DOI] [PubMed] [Google Scholar]
- 23.Park SK, Chun HJ, Kim DW, et al. Acute Physiology and Chronic Health Evaluation II and Simplified Acute Physiology Score II in predicting hospital mortality of neurosurgical intensive care unit patients. J Korean Med Sci. 2009;24(3):420–26. doi: 10.3346/jkms.2009.24.3.420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Vasilevskis EE, Kuzniewicz MW, Cason BA, et al. Mortality probability model III and simplified acute physiology score II: assessing their value in predicting length of stay and comparison to APACHE IV. Chest. 2009;136(1):89–101. doi: 10.1378/chest.08-2591. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Ho K, Lee K, Williams T, et al. Comparison of acute physiology and chronic health evaluation (APACHE) II score with organ failure scores to predict hospital mortality. Anaesthesia. 2007;62:466–73. doi: 10.1111/j.1365-2044.2007.04999.x. [DOI] [PubMed] [Google Scholar]
- 26.Minne L, Hanna AA, Jonge E. Evaluation of SOFA-based models for predicting mortality in theICU: A systematic review. Critical Care. 2008;12:R161. doi: 10.1186/cc7160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Janssens U, Graf J, Radke P, et al. Evaluation of the sofa score: A single-center experience of a medical intensive care unit 303 consecutive patients with predominantly cardiovascular disorders. Sequential Organ Failure Assessment. Intensive Care Med. 2000;26:1037–45. doi: 10.1007/s001340051316. [DOI] [PubMed] [Google Scholar]
- 28.Alizadeh AM, Hassanian-Moghaddam H, Shadnia S, et al. Simplified Acute Physiology Score II/Acute Physiology and Chronic Health Evaluation II and prediction of the mortality and later development of complications in poisoned patients admitted to intensive care unit. Basic Clin Pharmacol Toxicol. 2014;115(3):297–300. doi: 10.1111/bcpt.12210. [DOI] [PubMed] [Google Scholar]
- 29.Rincon T, Welcher B, Srikanth D, Seiver A. Economic implications of data collection from a remote center utilizing technological tools. Crit Care Med. 2007;35(12):A161. [Google Scholar]
- 30.Cowen JS, Kelley MA. Errors and bias in using predictive scoring systems. Crit Care Clin. 1994;10:53–72. [PubMed] [Google Scholar]