Application of a modified sequential organ failure assessment score to critically ill patients

Abstract

The purpose of the present study was to explore the usefulness of the Mexican sequential organ failure assessment (MEXSOFA) score for assessing the risk of mortality for critically ill patients in the ICU. A total of 232 consecutive patients admitted to an ICU were included in the study. The MEXSOFA was calculated using the original SOFA scoring system with two modifications: the PaO₂/FiO₂ ratio was replaced with the SpO₂/FiO₂ ratio, and the evaluation of neurologic dysfunction was excluded. The ICU mortality rate was 20.2%. Patients with an initial MEXSOFA score of 9 points or less calculated during the first 24 h after admission to the ICU had a mortality rate of 14.8%, while those with an initial MEXSOFA score of 10 points or more had a mortality rate of 40%. The MEXSOFA score at 48 h was also associated with mortality: patients with a score of 9 points or less had a mortality rate of 14.1%, while those with a score of 10 points or more had a mortality rate of 50%. In a multivariate analysis, only the MEXSOFA score at 48 h was an independent predictor for in-ICU death with an OR = 1.35 (95%CI = 1.14-1.59, P < 0.001). The SOFA and MEXSOFA scores calculated 24 h after admission to the ICU demonstrated a good level of discrimination for predicting the in-ICU mortality risk in critically ill patients. The MEXSOFA score at 48 h was an independent predictor of death; with each 1-point increase, the odds of death increased by 35%.

Introduction

Originally, the sequential organ failure assessment (SOFA) score was introduced during a consensus conference in 1994 to describe the degree of organ dysfunction over time in septic patients 1, but it can also be applied to non-septic patients 2. The SOFA score is composed of scores for six organ systems (respiratory, cardiovascular, neurological, hepatic, renal, and coagulation), with a possible total score ranging from 0 to 24 points 1. The main purpose of the organ failure scores is to describe the sequence of complications, not to predict mortality. Nevertheless, there is a relationship between organ failure and death 2. The performance of SOFA-based models seems to be comparable to that of other organ failure scores and good to excellent discrimination between survivors and non-survivors has been found in these models 3. Studies on different critically ill patient groups have demonstrated that multiple organ failures and high SOFA scores for any individual organ are associated with higher mortality 2,4-6.

A limitation of the use of the SOFA score on a daily basis is the need to obtain the pertinent laboratory data each day. Although this is not necessarily a problem with daily venous blood sampling that is common in most intensive care units (ICU), measurements of arterial oxygenation to calculate the PaO₂/FiO₂ (PF) ratio are not always available. In addition, this test can result in patient discomfort as well as complications 7. An alternative approach is to use pulse oximetry to measure oxyhemoglobin saturation (SpO₂) as a substitute for PaO₂ to calculate the SpO₂/FiO₂ (SF) ratio 8. Rice et al. 9 have reported SF ratio cut-off points for defining acute lung injury and acute respiratory distress syndrome. The validity of using the SOFA score based on the SF ratio in predicting mortality has not been assessed. However, Pandharipande et al. 10 have determined SF ratio correlations with the PF ratio to calculate the respiratory parameters of the SOFA score and to validate the respiratory SOFA obtained using SF ratios to predict clinical outcomes (ICU length of stay and ventilator-free days). The authors found a good correlation between the respiratory SOFA scores calculated with the SF ratio and the score calculated with the PF ratio. The SF ratios provide an alternate method for calculating the respiratory component of the SOFA score when the PF ratios are unavailable due to a lack of arterial blood gas data 10.

The neurological evaluation of critically ill patients is complicated by the frequent use of sedative agents 1. Although the neurological score was found to be the second most important predictor of death 2, recent studies have demonstrated moderate accuracy and reliability of the Glasgow Coma Scale (GCS) 11-13. Excluding the neurological component from the SOFA score still allowed for the prediction of mortality in medical and surgical cancer patients 14.

The aim of this study was to explore the usefulness of our simplified SOFA score: the Mexican SOFA (MEXSOFA) score for assessing mortality risk in critically ill patients. The calculation of the MEXSOFA score was performed using the original SOFA scoring system with two modifications: we used SF ratios to assess the respiratory parameters of the SOFA score 10, and we excluded the evaluation of neurologic dysfunction.

Material and Methods

Design and setting

An observational study was performed at the Instituto Nacional de Ciencias Médicas y Nutrición Salvador Zubirán (INCMNSZ), a tertiary care and teaching hospital in Mexico City, from April through December 2009. The multidisciplinary ICU has 14 beds. The present study was observational and descriptive. The Institutional Ethics Committee of Biomedical Research in Humans of the INCMNSZ approved this study, and the need for informed consent was waived.

Selection of participants, data collection, and definitions

During the study period, all patients over 16 years of age requiring ICU admission were evaluated. Patients with an ICU stay of less than 24 h, and readmissions were not considered. Data were collected in paper format by 3 of the investigators, who reviewed the patients' charts. The following variables were recorded: age, gender, need for mechanical ventilation, duration of mechanical ventilation, positive end-expiratory pressure (PEEP), length of stay in the ICU, reasons for admission to the ICU, and in-ICU death. The length of stay in the ICU was measured as the number of days from admission to the ICU until discharge from the ICU.

The SOFA score was calculated during the first 24 h after admission to the ICU and the MEXSOFA score was computed during the first 24 and 48 h of the ICU stay. In this study, calculation of the MEXSOFA score was performed using the original SOFA scoring system with two modifications. The respiratory component of the SOFA score was measured in patients without PEEP using the SF ratios proposed by Pandharipande et al. 10. In patients who required PEEP, the SF ratios were used according to the level of PEEP (Table 1). In our MEXSOFA score, the evaluation of neurologic dysfunction was excluded, resulting in a total score of 20 points. Our ICU has a sedation and analgesia protocol, which includes the use of targeted sedation, measured by sedation scales, and daily interruption of continuous infusion sedation with awakening and retitration 15.

Table 1. The Mexican sequential organ failure assessment (MEXSOFA) score.

Doses for dopamine, dobutamine and norepinephrine are given in µg·kg^-1·min^-1. PEEP = positive end-expiratory pressure (cmH₂O).

We defined the delta MEXSOFA (ΔMEXSOFA) score as the change in the MEXSOFA score over 48 h (the difference between the 48-h MEXSOFA score and the initial MEXSOFA score) as proposed by Ferreira et al. 16 for the SOFA score. If multiple laboratory values were available for a given patient, the worst value for each organ system in each 24-h period was considered. The MEXSOFA score was arranged into categories according to a previous report 16.

Data presentation and statistical analyses

Continuous variables are reported as means ± SD, or as medians and interquartile ranges if the distribution was not normal. Categorical variables are reported as percentage. The Student t-test or the Mann-Whitney U-test was used to compare continuous variables depending on whether they had a normal or non-normal data distribution, respectively, determined by the Kolmogorov-Smirnov test, and the chi-square or the Fisher exact test was used to compare categorical variables. Discrimination of SOFA and MEXSOFA scores was quantified by the area under the receiver operating characteristic curve (AUROCC) 17. Comparison of the AUROCC was performed using the methodology suggested by Hanley and McNeil 18. ICU mortality was the end point. Univariate and multivariate logistic regressions were used to identify factors associated with ICU mortality. For variables with multiple levels, the reference level was set as the one with the lowest probability of ICU mortality. Variables with a P < 0.2 in univariate analysis were entered into the model using a forward stepwise procedure. Results were summarized as odds-ratios (OR) and their respective 95% confidence interval (CI). The AUROCC was used to evaluate the ability of the model to discriminate between patients who lived and those who died (discrimination) 18. Goodness-of-fit (Hosmer-Lemeshow) was calculated to assess the relevance of the logistic regression model 19. A two-sided P value <0.05 was used to determine statistical significance. Data were entered into a database using the Statistical Package for the Social Sciences software (version 19.0; SPSS, USA) and the MedCalc statistical software (version 11.6.1; Belgium) for statistical analysis.

Results

During the study period, there were 303 admissions to the ICU. We excluded 71 (23.4%) patients who stayed less than 24 h. A total of 232 patients were included, with an overall ICU mortality rate of 20.25% (47/232). Their mean age was 48 ± 20 years, and 133 patients (57.3%) were women. Table 2 summarizes the demographics and clinical characteristics of the entire study population, and Table 3 summarizes reasons for admission to the ICU and the mortality of critically ill patients. Nine (3.9%) patients had neurological disorders, including viral encephalitis (N = 3), acute cerebrovascular accident (N = 3), status epilepticus (N = 1), subarachnoid hemorrhage (N = 1), and bacterial meningitis (N = 1; Table 3).

Table 2. Demographic and clinical characteristics of the study population.

ICU = intensive care unit; IQR = interquartile range; PEEP = positive end-expiratory pressure; SOFA = sequential organ failure assessment; MEXSOFA = Mexican sequential organ failure assessment.

Table 3. Reasons for admission to the intensive care unit and mortality of critically ill patients.

The incidence of organ dysfunction was noted most frequently for the respiratory (100%), cardiovascular (61.2%), and coagulation systems (47.8%). The median initial MEXSOFA score was 6 5-9, and at 48 h, the median score was 5 3-8 for the entire population. Nonsurvivors had significantly higher MEXSOFA scores both initially and at 48 h of admission when compared to survivors (Tables 4 and 5).

Table 4. Variables of the Mexican sequential organ failure assessment (MEXSOFA) on the day of admission to the intensive care unit (initial) according to outcome.

IQR = interquartile range. *P < 0.05 compared to survivors (chi-square test or Mann-Whitney U-test).

Table 5. Variables of the Mexican sequential organ failure assessment (MEXSOFA) at 48 h of admission to the intensive care unit according to outcome.

IQR = interquartile range. *P < 0.05 compared to survivors (chi-square test, Student t-test or Mann-Whitney U-test).

Patients with an initial MEXSOFA score of 9 points or less had a mortality rate of 14.8%, while patients with an initial MEXSOFA score of 10 points or greater had a mortality rate of 40% (Figure 1A). The MEXSOFA score at 48 h was also associated with mortality: patients with a score of 9 points or less had a mortality rate of 14.1%, while patients with a score of 10 points or greater had a mortality rate of 50% (Figure 1B). Figure 1C shows the relationship between ΔMEXSOFA score and ICU mortality. A trend toward higher mortality was noted for subjects with a ΔMEXSOFA score ≥2 points (36.6%) than for the entire cohort (20.25%; P = 0.712), and patients with a ΔMEXSOFA score ≤-2 points had a lower mortality rate (16%) than the entire cohort (P = 0.506). The mortality rate was 27.8% when the MEXSOFA score increased or did not change and 13.7% when it decreased (P = 0.008).

Figure 1. Mortality rate in relation to the changes in the Mexican sequential organ failure assessment (MEXSOFA) score during the first 48 h in the intensive care unit. Numbers above the bars indicate number of deaths. ΔMEXSOFA score = difference between the 48-h score and the initial score.

There was no difference between the initial SOFA [AUROCC = 0.69 (95%CI = 0.62-0.74)] and the initial MEXSOFA [AUROCC = 0.73 (95%CI = 0.66-0.78)] scores (P = 0.102) for predicting ICU mortality (Figure 2).

Figure 2. Comparisons of the areas under the receiver operating characteristic curves (AUROCC) for the prediction of mortality of the initial sequential organ failure assessment (SOFA) score and the Mexican sequential organ failure assessment (MEXSOFA). There was no difference between the initial SOFA [AUROCC = 0.69 (95%CI = 0.62-0.74)] and the initial MEXSOFA [AUROCC = 0.73 (95%CI = 0.66-0.78)] scores.

Table 6 lists the results of the univariate and multivariate analyses, with ICU death as the outcome variable of interest. In multivariate analysis, only the MEXSOFA score at 48 h was an independent predictor for in-ICU death, with an OR = 1.35 (95%CI = 1.14-1.59, P < 0.001), Hosmer-Lemeshow χ² = 3.54 (P = 0.896), and AUROCC = 0.79 (95%CI = 0.72-0.86, P <0.001).

Table 6. Univariate and multivariate logistic regression analysis for indentifying independent risk factors for mortality in the intensive care unit (ICU).

OR = odds ratio; CI = confidence interval; MEXSOFA = Mexican sequential organ failure assessment. Goodness of fit (Hosmer-Lemeshow) χ² = 3.54, P = 0.896, AUROCC (area under the receiver operating characteristic curve) = 0.79 (95%CI = 0.72-0.86), P < 0.001.

Discussion

The main findings of this study are that there was no difference between the initial SOFA and initial MEXSOFA scores for predicting mortality. The MEXSOFA score demonstrated a good level of discrimination for in-ICU mortality prediction, and the MEXSOFA score at 48 h was an independent predictor of in-ICU death.

Models with sequential SOFA scores seem to have a performance comparable to that of other organ failure scores 3. Previous studies have reported that the discrimination of the SOFA score on day 1 for predicting ICU mortality, quantified by the AUROCC, ranged between 0.67 and 0.82 16,20,21. In our study, outcome prediction of the initial SOFA score was similar to that of the MEXSOFA score, with respective AUROCC values of 0.69 and 0.73.

Vincent et al. 2 evaluated the use of the SOFA score in assessing the incidence and severity of organ dysfunction in 1449 critically ill patients. In their study 2, the patients' median ICU length of stay was approximately half of that reported in the present study (5 vs 9 days), while patients' mortality was almost the same (22 vs 20.2%). The trend toward a lower mortality in our study may be explained by our greater ICU length of stay.

The SOFA score can be used in several ways to predict ICU mortality. Variables derived from the SOFA score, such as max SOFA 22,23, mean SOFA 16,20 and delta SOFA 6,16,23 have been evaluated in critically ill patients admitted to the ICU. A recent systematic review 3 of the performance of the original SOFA score as a predictor of mortality reported that ΔSOFA had the lowest AUROCC, indicating poor fit. The ΔSOFA score could be used to reflect the patient's response to therapeutic strategies and allow the health care professionals to monitor daily progress. In addition, a decreasing SOFA score over the first 48 h of an ICU stay has been associated with a decrease in mortality 16. Ferreira et al. 16 reported by univariate logistic analysis that ΔSOFA score, initial SOFA score and SOFA score at 48 h correlated with mortality: with each 1-point increase of the initial SOFA score and 48-h SOFA score, the odds of death increased by 45%. By multivariate logistic analysis, our results indicate that the MEXSOFA score calculated at 48 h after ICU admission can be considered to estimate the outcome of critically ill patients admitted to the ICU. The MEXSOFA score at 48 h was an independent predictor of death: with each 1-point increase, the odds of death increased by 35%.

The severity of organ failure may change over time and a scoring system needs to account for this time factor. Previous investigations have reported the usefulness of assessing the changes in the SOFA score during ICU care to evaluate outcome 6,16,24. Levy et al. 24 reported that, although organ dysfunction at baseline was predictive of outcome, dynamic assessment of organ dysfunction using the SOFA score (improved versus unchanged or worsened) was superior in predicting 28-day mortality. In another study, an increase in the SOFA score during the first 48 h of ICU admission was associated with a higher mortality rate, independent of the initial score 25. A decrease in the SOFA score over the first 48 h of ICU admission has been associated with a lower mortality rate 16. Similarly, we found that patients with a decreased MEXSOFA score at 48 h had a lower mortality.

Several investigators have examined the predictive performance of various modifications to the SOFA score 13,26,27. Junger et al. 26 reported good discrimination (AUROCC = 0.799; 95%CI = 0.739-0.858) by a modified version of the SOFA score (registering unavailable data as normal and using a surrogate of the GCS) at the time of admission to the ICU of 524 patients (509 surgical and 15 medical patients). A recent study by Grissom et al. 27 determined a serial SOFA score and a modified SOFA score and compared their ability to predict the need for mechanical ventilation and mortality in critically ill patients admitted to an ICU. The modified SOFA score proposed by Grissom et al. 27 eliminates the platelet count, replaces PaO₂ with SpO₂, and replaces serum bilirubin with the clinical assessment of scleral icterus or jaundice. Their modified SOFA score on day 1 predicted the need for mechanical ventilation and mortality with AUROCCs = 0.83 (95%CI = 0.81-0.84) and 0.84 (95%CI = 0.81-0.85), respectively.

Pandharipande et al. 10 reported the validation of substituting SF for PF ratios in the SOFA score in patients undergoing general anesthesia and those with acute respiratory distress syndrome. In our study, we used the derived SF ratio cutoffs 10. This substitution makes the MEXSOFA score more generalizable to the ICU setting when daily arterial blood gas data are not readily available.

Jones et al. 28 reported the utility of the SOFA score for assessing the outcome of patients with severe sepsis and evidence of hypoperfusion at the time of presentation to the emergency department. They used the PF ratio when arterial blood gases were obtained. In cases in which the PaO₂ was not available, the SF ratio was substituted. The SOFA score at the time of emergency department recognition and resuscitation demonstrated an AUROCC = 0.75 (95%CI = 0.68-0.83), suggesting fair accuracy for mortality prediction. In our cohort of critically ill patients, the MEXSOFA score demonstrated a good level of discrimination for mortality prediction.

Since 1974, the GCS has been the standard method for assessing the consciousness level of head trauma patients 29,30. The neurological component of the SOFA score is based on the GCS 1; however, Vincent et al. 1 did not define whether the actual or the assumed GCS should be used in patients without sedation. The neurological score was found to be the second most important predictor of ICU mortality 2, nevertheless, in their study, Vincent et al. 2 did not describe the number of patients who were intubated and sedated, nor how the GCS was determined in these patients. Khwannimit 31 reported that the neurological component of the SOFA score had the best AUROCC (0.84; 95%CI = 0.81-0.86) for predicting ICU mortality in critically ill patients. However, it should be considered that for patients who were sedated, GCS was determined either from their medical records before sedation or by interviewing the physician who ordered the sedation. If a variable could not be measured, the GCS was assumed to be normal 31.

Nates et al. 14 modified the original SOFA score, excluding the neurological component, and redefining the cardiovascular component (blood pressure was not included) by including the total number of vasopressor drugs administered to the patient during each day of the ICU stay. The AUROCC for medical and surgical patients was 0.72 (95%CI = 0.69-0.74) and 0.78 (95%CI = 0.76-0.79), respectively 14.

Despite its drawbacks, the GCS represents an essential component of severity of illness scoring systems, including APACHE II score 32 and SAPS II 33. Also, the GCS-based neurological subcomponent of the SOFA score remains of major prognostic importance in mixed patient populations 2,31. However, the variability of application of the GCS to intubated and sedated patients could reduce its ability to predict outcome in this group of patients. Furthermore, previous studies have also demonstrated low accuracy and reliability of GCS scoring 11,12. In addition, without the verbal response, the total GCS could be biased. Considering these limitations, and the variation in GCS scoring in several studies 1,2,16,33, neurologic dysfunction was not included in the MEXSOFA score.

In our cohort, primary neurological diagnoses were present in only a minority of our patients (3.9%), and their impact may not have significantly affected the overall predictive performance of the MEXSOFA score, although its performance in those patients with neurological states remains uncertain. It is noteworthy that the prognosis of patients with severe cerebral dysfunction depends on its specific cause. In populations with high numbers of patients with neurologic dysfunction we suggest using scales to assess the prognosis according to type of neurological insult or condition (ischemic stroke, hemorrhagic stroke, traumatic brain injury). The MEXSOFA score could be used to determine the incidence of non-neurological organ dysfunction and its association with outcome in neurocritical care patients. We recommend the use of the original SOFA score when arterial blood gases can be obtained. In cases where the PaO₂ is not available, we suggest that the SF be included in the calculation of the MEXSOFA score. The MEXSOFA score, like the SOFA score, has several potential applications: assessment of new therapies in clinical trials, characterization of patients for inclusion in clinical trials or in epidemiological analyses and the two scores are useful additional tools for assessing and monitoring organ dysfunction 34.

The parameters used in the MEXSOFA score make repeated sequential assessment an easier task and ensure that the method is readily accessible to all health care professionals working in ICUs or emergency units for the initial evaluation of critically ill patients. However, the present study has some limitations. First, we are aware that the sample was not large enough to reach a definitive conclusion. Second, we did not include the neurological component of the original SOFA scoring system, third we included a low proportion of patients with neurological disorders, and finally the study was performed at a single institution. Future studies will be necessary for assessing the external validity of the MEXSOFA score outside our center.

SOFA and MEXSOFA scores calculated at 24 h after admission to the ICU demonstrated a good level of discrimination for predicting the in-ICU mortality risk of critically ill patients. The MEXSOFA score at 48 h was an independent predictor of death; with each 1-point increase, the odds of death increased by 35%. The simplicity of the MEXSOFA score with the use of the SF ratios, in addition to the exclusion of the neurological component, resulted in a more flexible score and may provide an alternate method for assessing mortality in critically ill adults. However, the conclusions of our study merit future external validation in larger datasets.