The predictive power of SAPS-3 and SOFA scores and their relations with patient outcomes in the Surgical Intensive Care Unit

Abstract

Objectives

Individual risk of surgical patients is more often underestimated and there is not an absolute criterion demonstrating which patient deserves intensive care. Since a nominative assessment of these patients to quantify the intensity of critical illness is not appropriate, prognostic scores are used to assess the mortality rate and prognosis for critical patients including surgical ones. This study aimed to test the calibration power of SAPS-3 score and SOFA score of surgical patients undergoing gastrointestinal surgery, and identify any relation with patient outcomes in the department of surgical ICU.

Material and Methods

This retrospective observational study was conducted during the period between August 2017 and December 2017. It was performed at a Gastroenterological Surgical ICU, a tertiary care hospital in Ankara, Türkiye. To calculate SAPS-3 and SOFA score, physiological data and laboratory analysis on the day of ICU admission were used. Records were reviewed from hospitalization to medical discharge or hospital mortality. Statistical analysis included Mann Whitney U-test and ROC-curves to predict 30-day mortality.

Results

A total of 233 patients admitted to the Gastroenterological Surgical ICU were included into the study and the main reason for ICU admission was surgical problems. Mortality rate was 2.6 % (6 patients). Average SAPS -3 score was 32.5 and SOFA score was 30.1. A significant correlation was observed with the SAPS-3 score, but not with the SOFA score statistically in mortality as a dependent factor. The discriminative power, assessed using the AUC and the probability of death estimation, was satisfactory with the SAPS-3 scores (AUC 0.754) while it was lower with the SOFA score (AUC 0.631)

Conclusion

We found that SAPS-3 score was significantly correlated not only with mortality rate, but also with LOS in the ICU. Nonetheless, SOFA score was not related to mortality, but related to LOS in the ICU. Prognostic score systems are used to estimate mortality but they may be used to identify LOS in the ICU and postoperative complications. It can be concluded that SAPS-3 and SOFA scores may be used to prognosticate postoperative ICU requirement.

Introduction

Surgical operations are classified as high-risk procedures and a significant proportion of intensive care unit (ICU) is composed of postoperative patients. These do not shadow the fact that the individual risk of surgical patients is more often underestimated, and less than 15% of patients who undergo those procedures have been admitted to an intensive care unit (1,2). There is not an absolute criterion or study demonstrating which patient deserves intensive care while debate is still continuing. The Society of Critical Care Medicine has published a guideline for ICU admission intending a better use of limited resources, but it has not gained popularity especially in surgical patients (3). So, risk adjustment via mortality prediction methods for this group of patients is a must.

Since a nominative assessment of these patients to quantify the intensity of critical illness is not appropriate, prognostic scores are used to assess the mortality rate and prognosis for critical patients including the surgical ones (4). Even though contradictious data are present in the literature, prognostic scoring systems (PSS) are used widely to make judgements in an objective manner (5). Among the PSSs, the Simplified Acute Physiology Score (SAPS) was investigated thoroughly and revised due to the improvements in health care (6,7). The revised version of this score system - SAPS-3 - uses 20 variables to predict hospital mortality from admission data (recorded within ± 1 h) (8). The Sequential Organ Failure Assessment (SOFA) score has primarily been proposed to assess the severity of organ dysfunction in septic patients in a sequential manner (9). Six organ systems (respiratory, cardiovascular, renal, hepatic, central nervous, coagulation) are taken into account, and the function of each is scored from 0 (normal function) to 4 (most abnormal). Serial changes in SOFA score over time are useful in predicting outcome. Its usage for mortality prediction based upon the fact that multiorgan dysfunction is commonly seen in critical situations like sepsis, chronic or acute liver failure, cancer or cardiac surgery (5,9).

Calibration and the discrimination are the two objective measures that evaluate the performance of prognostic models. Calibration -the relation of the estimated mortality with the observed mortality - is claimed to be as important as the discrimination - differentiation between survivors and nonsurvivors- in the means of the ICU eligibility of the patients and comparison of care between ICUs (7,10). Nonetheless, the calibration and discrimination power of the PSS can vary in time with the improvement in healthcare and it can vary across centers (11). As we searched in the literature, SAPS-3 and SOFA scores have not been evaluated in terms of sensitivity and specificity to predict mortality in postoperative patient population in a comparison manner. This study aimed to test the calibration power of SAPS-3 score and SOFA scores of surgical patients undergoing gastrointestinal surgery, and identify any relation with the patient outcomes in the department of surgical ICU.

Patients and Methods

This retrospective observational study was conducted during the period between August 2017 and December 2017. It was performed at a Gastroenterological Surgical ICU, a tertiary care hospital in Ankara, Türkiye. The patients included in the study were those admitted to the surgical ICU for any reason after surgery. Patients with incomplete records and length of stay of less than 24 hours were excluded from the study. 

An extra formal consent other than the patients had given prior to hospitalization was not required for the current study because it was a case-control medical record review. Since our study was in the category of non-interventional clinical research with its retrospective structure, no ethics committee approval was applied. 

Data Acquisition

All clinical variables of the patients were retrospectively collected from our institutional database. Patients were analyzed on the following variables: diagnosis on admission, surgery type, medical history of the patients, reason for ICU admission, laboratory findings, mortality, length of stay (LOS) in ICU and in hospital at all. 

Surgery types were classified by localization of the pathology and divided mainly into four groups; biliopancreatic system (BPS), upper gastrointestinal system (GIS), lower GIS, and liver pathology. Reasons for ICU admission were classified mainly in four groups; cardiac problems like arrhythmia, respiratory problems like medical history of Chronic Obstructive Pulmonary Disease, hemodynamic problems like hypotension, and surgical problems like bleeding (bleeding from surgical site or any suspicion of bleeding, and observational admissions were classified in this category). Mortality as a variable in this study was described as death from any cause occurring within 30 days after surgery.

To calculate the SAPS-3 and SOFA score, physiological data and laboratory analysis on the day of ICU admission were used (postoperative 0). Records were reviewed from hospitalization to medical discharge or hospital mortality. Data were imported into a spreadsheet (Microsoft Excel 2013, Microsoft Corporation) for the calculation of the scores and their derived probabilities of death using the published equations and coefficients. To forestall the variability in the data collection, all values were reviewed by the authors of the study.

Statistical Analysis

Statistical analysis was performed using SPSS version 20.0 for Windows (SPSS Inc.; Chicago, IL , USA). Data were analyzed, and the continuous variables were reported as mean ± standard deviation (SD), and nominal variables were reported as total number and percentages. 

Variables were first evaluated by One-Sample Kolmogorov-Smirnov test as a normality test to choose the type of statistical tests -parametric or non-parametric test-, and the results showed asymp. Sig. (2-tailed) levels ≤ 0.05, so we decided to use non-parametric tests. For statistical analysis, correlations between variables were evaluated for significance by using the Spearman’s rho test. Categorical variables were evaluated by the Mann-Whitney U test of contingency. In all analyses, a ‘p’ value less than 0.05 was considered statistically significant and comparisons were 2-tailed.

Hosmer-Lemeshow test was used to calculate the calibration of SAPS-3 and SOFA test which express the ability of the test to determine the probability of death in accordance with the observed mortality. Discrimination was assessed using receiver operating characteristic (ROC) curves. The ROC curves were established as discrimination measurements with distributions per 10%, according to the predicted mortality and the obtained curve was appraised using the calculated area under the curve (AUC). AUC values > 0.75 was appraised as satisfactory, AUC values > 0.8 was appraised as well, and AUC values > 0.9 was appraised as very good.

Results

A total of 233 patients admitted to the Gastroenterological Surgical ICU were included in the study between August 2017 and December 2017. Patients older than 18 years of age, who stayed 24 h or more in the ICU, were included. Forty-nine patients’ records were excluded due to incomplete or unavailable data. Eight patients were younger than 18 years of age and excluded from the study. Patients’ demographic data and operational details of the groups (survivors versus non-survivors) are presented in Table 1.

Table 1. Demographic data and operational details of the groups.

	All (233)	survivors (227)	non-survivors (n= 6)	p+
Age (Years)	58.52 (± 13.94)	58.38 (± 14.09)	63.50 (± 3.08)	0.459
Gender (Male)	132 (56.7%)	129 (56.8%)	3 (50%)	0.74
Operation type
BPS	92 (39.5%)	89 (39.2%)	3 (50%)	0.930
Upper GIS	83 (35.6%)	80 (35.2%)	3 (50%)
Lower GIS	47 (20.2%)	47 (20.7%)	0
Liver pathology	11 (4.7%)	11 (4.8%)	0
Malignancy	126 (54.1%)	120 (52.9%)		0.022
Duration of surgery (minutes)	204.47 (± 130.4)	201.5 (± 136.5)	328 (± 182)	0.083
Medical history
COPD	17 (7.3%)	15 (6.6%)	2 (33.33%)	0.023
Renal disease	1 (0.4%)	1 (0.4%)	0
Hypertension	77 (33%)	77 (33.9%)	0
CHF	20 (8.6%)	18 (7.9%)	2 (33.33%)
DM	11 (4.7%)	11 (4.8%)	5 (3%)
Urgent operation	16 (6.9%)	16 (7%)	0	0.501
Reason for ICU admission
Hemodynamic problems	23 (9.9%)	22 (9.7%)	1 (16.7%)	0.043
Surgical problems	154 (66.1%)	153 (67.4%)	1 (16.7%)
Cardiac problems	36 (15.5%)	35 (15.4%)	1 (16.7%)
Respiratory problems	20 (8.6%)	17 (7.5%)	3 (50%)
MVP	7 (3%)	3 (%1.3)	4 (66.7%)	< 0.001
* Values are either expressed as mean ± standard deviation or n (%). +p-values calculated for comparison of survivors versus non-survivors group by Mann- Whitney U test. BPS: Biliopancreatic system; GIS: Gastrointestinal system; COPD: Chronic obstructive pulmonary disease; CHF: Congestive heart failure; DM: Diabetes mellitus; ICU: Intensive care unit; MVP: Mechanical ventilatory support.

Mean age of the patients was 58.52 years, with a standard deviation of 13.94 years and the representatives of both sexes were relatively proportional (56.7% males versus 43.3% females). Main reason for ICU admission was surgical problems (66.1%), and as it was stated previously, bleeding from surgical site or any suspicion of bleeding, and observational admissions were classified in this category. Patients were operated for malignancy in 54.1% of the cases and mortality rate was 2.6 % (6 patients). Statistical analysis showed that surgery for malignancy, presence of comorbidities and reason for ICU admission were related to mortality (p <0.05). The duration of surgery and being urgent had no relation with mortality. Mechanical ventilatory support was required in 7 patients and the difference between survivors and non-survivors was significant statistically (p <0.001).

In Table 2, patients’ postoperative outcomes with prognostic scores related with mortality are evaluated. The average length of stay (LOS) in hospital was 14.35 days and LOS in ICU was 5.46 days. The average SAPS-3 score was 32.5 and SOFA score was 30.1. A significant correlation was observed with the SAPS-3 score, but not with the SOFA score statistically in mortality as a dependent factor.

Table 2. Postoperative outcomes with prognostic scores related with mortality*.

	All (233)	survivors (n= 227)	non-survivors (n= 6)	p+
LOS in-hospital (days)	14.35 (± 15.37)	14.03 (± 15.23)	26.67 (± 17.01)	0.035
LOS in- ICU (days)	5.46 (± 8.1)	5.11 (± 7.3)	18.83 (± 19.24)	0.011
LOS in- ICU > 3 days	118 (50.6%)	113 (49.8%)	5 (83.3%)	0.106
SAPS-3 score	32.5 (± 8.95)	32.29 (± 8.87)	40.83 (± 8.7)	0.033
SOFA score	3.01 (± 1.38)	2.98 (± 1.35)	4.17 (± 2.22)	0.249
30 day mortality	6 (2.6%)
* Values are expressed as either mean ± standard deviation or n (%). +p-values calculated for comparison of survivors versus nonsurvivors group by statistical analysis. LOS: Length of stay; ICU: Intensive care unit; SAPS-3: Simplified acute physiology score; SOFA: Sequential organ failure assessment.

It is observed that 50.6 % of the patients were treated in the ICU for more than 3 days. So, further evaluation of the patients was decided on according to the length of stay in the ICU for statistical analysis in this study. In Table 3, groups were formed according to LOS in the ICU; group A represents the patients with LOS in ICU > 3 days and group B represents the patients with LOS in ICU ≤ 3 days.

Table 3. Postoperative outcomes with prognostic scores and relation with LOS in the ICU*.

	All (233)	Group A (n= 118)	Group B (n= 115)	p+
Age	58.52 (± 13.94)	60.66 (12.76)	56.31 (14.79)	0.026
LOS in-hospital (days)	14.35 (± 15.37)	20.46 (± 18.37)	8.09 (± 7.48)	< 0.001
LOS in- ICU (days)	5.46 (± 8.1)	9.18 (± 10.1)	1.64 (± 0.74)	< 0.001
SAPS-3 score	32.5 (± 8.95)	36.13 (± 8.52)	28.79 (± 7.82)	< 0.001
30 day mortality	6 (2.6%)	5 (4.2%)	1 (0.9%)	0.105
SOFA score	3.01 (± 1.38)	3.39 (± 1.53)	2.62 (± 1.09)	< 0.001
Malign pathology	126 (54.1%)	86 (72.9%)	40 (34.8%)	< 0.001
* Values are expressed as either mean ± standard deviation or n (%). +p-values calculated for comparison of survivors versus nonsurvivors group by statistical analysis. LOS: Length of stay; ICU: Intensive care unit; SAPS-3: Simplified acute physiology score; SOFA: Sequential organ failure assessment.

In this statistical analysis, it was clearly seen that both SOFA and SAPS-3 scores were related to LOS in ICU > 3 days (Asymp. Sig. (2-tailed) <0.001). Age, as a variable, was also found to be related to LOS in the ICU statistically, but this was expected- as comorbidity rate increases with age- and Asymp. Sig. (2-tailed) level was higher (p= 0.026)

 The discriminative power assessed using the AUC and the probability of death estimation was satisfactory with SAPS-3 scores (AUC 0.754) while it was lower with SOFA score (AUC 0.631) (Figure 1 and Table 4). The Hosmer-Lemeshow goodness-of-fit test also revealed this with the discordance of these two scores with a good calibration for the SAPS-3 global model as shown in Table 4 with sig.level 0.757 versus 0.613.

Figure 1. ROC curve for SAPS-3 and SOFA scores.

Table 4. Hosmer and Lemeshow Test and Area under curve for SAPS 3 and SOFA score.

	Chi-square	df	sig	Area under curve
SAPS-3	5.009	8	0.757	0.754
SOFA	1.807	3	0.613	0.631
SAPS-3: Simplified acute physiology score-3; SOFA: Sequential organ failure assessment.

Discussion

Due to the fact that surgical patients have different physiological and functional characteristics than other patients, estimation of prognosis and optimizing postsurgical care with lower misuse of ICUs are hard and complicated tasks (4). Prognostic scoring systems (PSS) like SAPS-3 and SOFA are utilized for this purpose in critical care medicine by disease severity assessment, comparing ICU performance and optimizing resource allocation. These systems use particular medical data to prognosticate an outcome’s possibility of occurrence (10,12,13 ).

SAPS-3 and SOFA scoring systems are simple and require non-sophisticated data, unlike other prognostic score systems. SOFA is mainly used to evaluate the clinical course sequentially and maintain a semiquantitative estimation of mortality. Therefore, it can be concluded that SOFA has a disadvantage in failing to show actual clinic state comparing to SAPS-3 (4,13). SAPS-3 utilization is claimed to be valid for surgical patients with good discrimination and calibration power, but the SOFA score has not been evaluated in surgical patients in our country. We compare these two commonly used PSS in mortality rate and in LOS in ICU mainly.

Mortality rates in the ICU differ between centers and patient admission criteria plays a very important role in this issue. Knaus et al. have described the lowest mortality rate as 6.4% in 42 ICUs, while in our study the mortality rate was 2.6 (14). This is probably due to the fact that in our study population, the major reason for the ICU admission was basic and observational reasons- surgical reasons- and we consider mortality as death from any cause occurring within 30 days after surgery. 

In our study, age factor as a variable did not affect mortality rate, but it was linked to prolonged LOS in the ICU, which was  in line with the finding in the study of Leong et al.  (15). 

Originally, SOFA score is computed 1 day after ICU admission and every 2 days subsequently. The highest scores are claimed to be most predictive of mortality (16). In our study, SOFA and SAPS-3 scores were calculated using data derived in the ICU admission day 0 retrospectively and the average scores were significantly higher in the non-surviving group. 

Discriminative power of the SAPS-3 was satisfactory and calibration was appropriate. Moreover, this model showed the relation of SAPS- 3 scores with mortality and length of stay in the ICU. AUC with SOFA score was found as 0.63,1 whereas it was found as 0.754 with SAPS-3. It is parallel to other studies that have found AUC 0.748 versus 0.810 (SOFA score versus SAPS-3 score) (16,17). Besides that, according to our findings, in this study SAPS-3 and SOFA are closely related to LOS in  the ICU for more than 3 days.

In a recent study, it has been demonstrated that approximately one fourth of surgical patients in the ICU were late admission- not admitted promptly postoperatively- and developed postoperative complications. In their study, Silva et al. have claimed that clinical evaluation underestimates the risk and has low predictive value, whereas PSS usage may be helpful in ICU referral and admission as an objective criterion (4). In another study, Nassar et al. have verified the feasibility of SAPS-3 in surgical patients and recommended its use as a distinction tool in ICU eligibility (18). PSS could be customized to advance their performance in specific patient populations (13). Our findings suggest that SAPS-3 or SOFA could be used as an admission criterion after proper customization.

Some studies have compared the SOFA score with another PSS-Acute Physiology and Chronic Health Disease Classification System (APACHE) score- as a mortality prediction model to predict hospital mortality and concluded that the combination of both scores enhances prognostic value (19,20). In another study, initial SOFA score has been found comparable to APACHE 3 in the ICU mortality prediction (16). 

The present study has some importance with the evaluation of different prognostic score systems in predicting mortality and relation to length of stay in ICU but it also has some limitations. First of all, one can claim that for comparison of different PSS, describing the sample size before the study is mandatory. As our study was designed in a retrospective manner this condition could not be achieved. Relatively small sample size limits the power of the analysis of goodness-of-fit Hosmer and Lemeshow Test which is poor to assess. Another potential limitation is being a single-center study with a different patients’ case mix as compared to the original SAPS-3 hospital outcome cohort. Lastly, the data collector reliability could be criticized in this study. Even though this is an important topic, we are quite reliant that, in this study, bias related to inadequate data collection was limited, since collection was done by the ICU doctors.

Conclusion

We found that SAPS-3 score was significantly correlated not only with mortality rate, but also with LOS in ICU. Nonetheless SOFA score was not related to mortality, but related to LOS in the ICU. PSS are used to estimate mortality but they may be used to identify LOS in the ICU and postoperative complications. It can be concluded that SAPS-3 and SOFA scores may be used to prognosticate postoperative ICU requirement.