Glasgow Coma Score dominates the association between admission Sequential Organ Failure Assessment score and 30-day mortality in a mixed ICU population

Abstract

OBJECTIVE

The Sequential Organ Failure Assessment (SOFA) score, a measure of multiple organ dysfunction syndrome (MODS), is used to predict mortality in critically ill patients by assigning equally weighted scores across six different organ systems. We hypothesized that specific organ systems would have a greater association with mortality than others.

DESIGN

We retrospectively studied patients admitted over a period of 4.2 years to a mixedprofile intensive care unit (ICU). We recorded age and comorbidities, and calculated SOFA organ scores. The primary outcome was 30-day all-cause mortality. We determined which organ sub-scores of the SOFA score were most associated with mortality using multiple analytic methods: random forests, conditional inference trees, distanced-based clustering techniques, and logistic regression.

SETTING

A 24-bed mixed-profile adult ICU that cares for medical, surgical, and trauma (Level 1) patients at an academic referral center.

PATIENTS

All patients’ first admission to the study ICU during the study period.

MEASUREMENTS AND MAIN RESULTS

We identified 9,120 first admissions during the study period. Overall 30-day mortality was 12%. Multiple analytical methods all demonstrated that the best initial prediction variables were age and the central nervous system (CNS) SOFA subscore, which is determined solely by Glasgow Coma Scale score.

CONCLUSIONS

In a mixed population of critically ill patients, the Glasgow Coma Scale score dominates the association between admission SOFA score and 30-day mortality. Future research into outcomes from multiple organ dysfunction may benefit from new models for measuring organ dysfunction with special attention to neurologic dysfunction.

INTRODUCTION

Multiple organ dysfunction syndrome (MODS) occurs in a severely ill patient in whom organismal homeostasis cannot be maintained without external therapeutic support.¹ MODS typically occurs in shock states in which arterial blood pressure is too low to adequately perfuse organs, and can occur even after successful initial resuscitation of a shock state.¹ Over half of intensive care unit (ICU) patients have at least one organ system dysfunction, and 20% have multiple organ dysfunction.² As much as half of the mortality observed in modern ICUs is attributable to MODS.^3,4

The Sequential Organ Failure Assessment (SOFA) score was developed to assess the extent of MODS.⁵ As opposed to more complex, sometimes proprietary predictive models, the SOFA score is easily calculated from routinely collected data and provides categorizations directly relevant to MODS. While SOFA was originally developed as a tool for evaluating and comparing organ failure in patients with sepsis, it has been used widely to predict mortality in general ICU populations.^6–11 The SOFA score sums dysfunction across six organ systems, assigning equal ranges of sub-scores, 0 to 4, to each organ system (Table 1).¹² The equal weighting of organ failure allows for easier calculation, but fails to recognize that organ systems may contribute unequally to the sequelae of MODS, including death. Recognition of unequal contribution of organ systems towards mortality could allow for a more refined scoring system or for targeted research or more patient-tailored treatments, based on the pattern of organ dysfunction. We selected the SOFA score for study because of its widespread clinical use, ease of calculation from routine clinical data, and its direct applicability to MODS.

Table 1.

SOFA Score

SOFA Sub-score	0	1	2	3	4
Respiratory   PaO2/FiO2 (torr)a	>340	≤ 340	≤ 255	≤ 170b	≤ 85b
Central Nervous System   Glasgow Coma Score	15	13–14	10–12	6–9	<6
Cardiovascular   MAP (mmHg)   Vasopressor ratec (µg/kg/min)	≥ 70	< 70	Dopamine ≤ 5 OR Dobutamine > 0	Dopamine > 5 OR Epinephrine ≤ 0.1 OR Norepinephrine ≤ 0.1 OR Phenylephrine ≤ 0.22	Dopamine > 15 OR Epinephrine > 0.1 OR Norepinephrine > 0.1 OR Phenylephrine > 0.22
Liver   Bilirubin (mg/dl)	<1.2	≥1.2	≥2.0	≥6.0	≥ 12.0
Coagulation   Platelet (×103/µL)	>150	≤ 150	≤ 100	≤ 50	≤ 20
Renal   Creatinine (mg/dl)   Urine output (ml/day)	<1.2	≥1.2	≥2.0	≥3.5	≥5.0 OR ≥3.5 AND < 200 mL/day

^aPaO2/FiO2 ratio was adjusted for altitude (1500m) per ARDS-Network methodology of adjusting the cutoffs by a factor of (Hospital P_atm/Sea level P_atm)⁵²

^bRequires the use of positive pressure ventilation

^cPer original description, vasopressor dose must have been present for ≥ 60 minutes to qualify⁵

^cWe added phenylephrine to the standard list of vasopressors according to standard equivalency⁵³

Various groups have attempted to refine the SOFA score,^6,13–19 but there is no consensus about which elements of the SOFA score are most relevant to mortality. In this study, we evaluated the relative importance of the organ systems measured with the SOFA score in a mixed ICU population with regard to 30-day mortality. Secondarily, we assessed the relevance of age and comorbidities to the association between admission SOFA score and 30-day mortality and investigated whether clustering and classification techniques could identify relevant patterns of organ dysfunction within MODS as measured by the SOFA score.

METHODS

We performed a retrospective study of all patients admitted from November 2007 through January 2012 to the Shock Trauma ICU, a 24-bed ICU that cares for adult medical, surgical, and trauma patients, at Intermountain Medical Center, a university-affiliated 452-bed Level 1 Trauma Center and referral hospital in Murray, Utah, USA. The Intermountain Medical Center Institutional Review Board (IRB #1014318) approved this study with waiver of consent.

We included all patients admitted to the ICU from the emergency department (ED), hospital floor, or other facilities. We only included the first ICU admission for a given patient during the study period. We excluded no other ICU admissions. For purposes of analysis, we assigned patients into one of five admitting diagnostic categories: medical, surgical, neurological, cardiac, or acute trauma. Cardiac diagnoses included primarily heart failure, endocarditis, and acute coronary syndrome admissions; the study ICU does not routinely admit cardiac surgery patients. We measured comorbidities using the Elixhauser score, according to standard methodology.^20,21 Our primary outcome was 30-day all-cause mortality. We determined mortality from the Intermountain Master Death Record, which incorporates results from Utah state vital statistics. We queried the electronic medical record (EMR) for SOFA data, admission diagnosis, comorbidities from discharge diagnostic codes, gender, age and ethnicity.

Calculating SOFA scores

For the admission SOFA score, we included data obtained up to 6 hours preceding ICU admission and 24 hours after ICU admission, in order to capture the first ICU day. We included data up to 6 hours prior to ICU admission in order to avoid excluding relevant values obtained in the ED (at the study hospital, most ED stays are < 5 hours). We calculated SOFA scores for each patient.^2,5,22 We used the standard method with the exception of our treatment of urine output in the renal sub-score which we solely used to reclassify patients with a renal sub-score of 3 to a renal sub-score of 4 due to less reliable charting of hourly urine flow rates in our system (Table 1). We calculated the SOFA by using the highest score for each organ system measured during the 30-hour scoring period, and then summed the organ system sub-scores to calculate the final SOFA score.

The SOFA central nervous system sub-score is based on the Glasgow Coma Scale (GCS) score., which is routinely measured in the study ICU according to standard methods.²³ All patients undergoing mechanical ventilation through an endotracheal tube were assigned a GCS verbal score of 1, irrespective of the clinician’s estimate of the patient’s underlying verbal capacity due to the fact that estimated verbal capacity was not available in all study patients.

Calculation of the PaO₂/FiO₂ was performed from arterial blood gas, when available. When arterial blood gases were not available, PaO₂/FiO₂ was estimated from SpO₂/FiO₂, using the Severinghaus equation²⁴ to convert SpO₂ to PaO₂. This method has previously been applied to estimate oxygenation in clinical studies on similar populations.^25–29

Statistical Methods

Summary statistics are reported as: 1) mean with standard deviation, 2) median with interquartile range, or 3) count and percentage as dictated by the data. We compared between- or among-group central tendencies with Chi-square or Kruskall-Wallis statistic where appropriate.

For our primary analysis, we utilized random forests^30–32 and conditional inference trees³³ to identify the SOFA sub-scores most predictive of 30 day-mortality. The random forest method uses bootstrapped samples with a random number of variables to create a large number of regression trees and assess the most influential variables in terms of classification. Conditional inference trees are a supervised classification method for analyzing data that select covariates by permutation-based significance tests, thereby avoiding potential bias of the more traditional decision tree algorithms.³³ We also employed a multivariate naïve Bayesian classification model of mortality.³⁴ For the Bayesian classification model, we utilized a feature selection strategy that employed repeated bootstrap sampling to maximize the area under the receiver operating characteristic curve (AUC) in the “out of bag” sample (those sample observations not used to build the given model), a technique used to avoid over-fitting of the model.^35,36

To evaluate the presence of clusters of organ dysfunction, we utilized three standard clustering techniques: k-means partitioning, partitioning around k-medoids and hierarchical agglomerative clustering.^37–39 The k-means and k-medoids algorithms partition observations into clusters wherein each observation belongs to the cluster with the nearest central data point, while the agglomerative clustering technique is a “bottom-up” approach wherein each observation starts in its own cluster and pairs of clusters are merged as one moves up the hierarchy. We generated silhouette plots and gap statistics to evaluate the optimal number of clusters.^40,41

By convention GCS assigns a verbal sub-score of 1 to all intubated patients, which could lead to over-weighting of respiratory failure in the overall SOFA score because it is also represented in the pulmonary sub-score. Some investigators have modified the GCS by imputing a verbal score to intubated patients.^42–44 To determine whether an imputed verbal sub-score changed our main findings we performed a sensitivity analysis in which we imputed the verbal sub-score for both intubated patients and, separately, for all patients, using the methodology of Meredith et al.⁴³

While our primary analysis was of the entire patient population, given the statistically significant differences between patients in different admission diagnostic groups, we performed a sensitivity analysis to confirm overall findings in each diagnostic group.⁴⁵ To report the results of our regression analysis we generated effect plots (rather than display coefficients from the regression equation), which display the association between a given predictor and outcome, with other predictors held constant, over the range of values observed in the predictor.⁴⁶ We performed all analyses in the R Statistical Package, version 2.14.2 (www.r-project.org) using packages randomForests, party, gbm, effects, cluster, fpc and MASS.⁴⁷

Missing Data

Missing data were imputed using data from the 24-hour period preceding ICU admission. Where missing data persisted after the initial imputation, we employed a mixed approach to evaluate the robustness of our findings because these data were likely to be Missing Not at Random. Conditional inference trees and our bootstrap logistic regression technique accommodate missing data natively. For random forests and the clustering analyses, we performed several sensitivity analyses: incomplete case (also known as list-wise) deletion, built-in imputation techniques for the clustering analysis, and imputation of normal to missing values i.e. if data were missing, the associated SOFA sub-score was considered to be 0.¹⁸

RESULTS

During the study period, 9,558 patients were admitted to the study ICU: 9,120 admissions were the patients’ first admission. SOFA sub-scores varied by admitting diagnosis with trauma and neurological patients being younger and having fewer comorbidities, albeit with slightly higher CNS sub-scores (i.e., lower GCS score), than patients admitted with other diagnoses (Table 2). Overall mortality was 12%. Histogram and line charts displaying the distribution and mortality of the SOFA sub-scores are displayed in Figure 1.

Table 2.

Descriptive statistics.

	Total	Medical	Surgical	Trauma	Cardiac	Neurologic
Patients	9120	4507	1289	2733	386	205
Total SOFA Score	5 (3–8)	6 (3–9)	5 (4–8)	4 (3–7)	7 (5–11)	6 (3–8)
Liver	0 (0–1)	0 (0–1)	0 (0–2)	0 (0–0)	0 (0–1)	0 (0–0)
Renal	0 (0–1)	1 (0–2)	0 (0–1)	0 (0–1)	1 (0–2)	0 (0–1)
Coagulation	0 (0–1)	0 (0–1)	0 (0–1)	0 (0–1)	0 (0–1)	0 (0–1)
CNS	1 (0–3)	1 (0–2)	1 (0–2)	1 (0–3)	1 (0–4)	2 (1–3.25)
Cardiovascular	1 (1–1)	1 (1–1)	1 (1–1)	1 (0–1)	1 (1–2)	1 (0–1)
Respiratory	2 (1–2)	2 (1–2)	2 (1–2)	2 (1–2)	2 (2–3)	2 (1–2)
30-day Mortality	1086 (12%)	559 (12%)	82 (6%)	295 (11%)	115 (30%)	35 (17%)
Age (years)	54 (±20)	56 (±19)	59 (±17)	49 (±22)	61 (±18)	54 (±18)
Elixhauser Score	3.3 (2.3)	4.1 (2.2)	3.4 (2.3)	1.9 (1.7)	4.6 (2.2)	3.6 (2.3)
% Receiving Vasopressors for >1 hr	1328 (15%)	846 (19%)	119 (15%)	233 (14%)	113 (16%)	17 (19%)
% Mechanically Ventilated	2098 (23%)	876 (23%)	306 (25%)	688 (22%)	164 (25%)	64 (31%)
% on Hemodialysis	329 (4%)	237 (5%)	33 (3%)	22 (1%)	32 (8%)	5 (2%)
Female sex	4115 (45%)	2265 (50%)	675 (52%)	885 (32%)	196 (51%)	94 (46%)
Race:
American Indian/ Alaskan Native	55 (1%)	22 (0%)	7 (1%)	22 (1%)	3 (1%)	1 (0%)
Asian/Pacific Island	291 (3%)	146 (3%)	32 (2%)	92 (3%)	10 (3%)	11 (5%)
Black	140 (2%)	57 (1%)	11 (1%)	57 (2%)	9 (2%)	6 (3%)
Hispanic	627 (7%)	282 (6%)	66 (5%)	231 (8%)	29 (8%)	19 (9%)
Other	65 (1%)	40 (1%)	5 (0%)	19 (1%)	1 (0%)	0 (0%)
Unknown	409 (4%)	126 (3%)	13 (1%)	251 (9%)	12 (3%)	7 (3%)
White	7533 (83%)	3834 (85%)	1155 (90%)	2061 (75%)	322 (83%)	161 (79%)

Legend: Descriptive statistics organized by diagnostic group. Data reported as mean (standard deviation), median (25%–75% interquartile range) or count (percentage of total). Each group was statistically different from one another in all variables with a p<0.01.

Figure 1.

Histogram and line charts demonstrating the distribution of the SOFA organ subscores, age and Elixhauser comorbidities.

Random forests analysis produces a variable importance plot, which measures how important to the predictive model each variable is. We display the variable importance plot from the study cohort in Figure 2‥ We tested for interactions between the variables and found interactions between the Elixhauser comorbidity score and both CNS and Cardiovascular sub-scores (h-statistic > 0.15).⁴⁸ The SOFA CNS sub-score and age were the most relevant variables for prediction of death, while Renal and Coagulation sub-scores were the least relevant. A representative consensus classification tree is depicted in Figure 3, suggesting that a CNS SOFA sub-score of 3 (actual GCS threshold of 6) is the best initial selection variable. This tree is made without manual direction, i.e., all division points are selected by the algorithm automatically. The naïve Bayes classification technique demonstrated age and CNS sub-score were the most important for predicting death, followed by cardiovascular and liver sub-scores. The AUC for the prediction of mortality increased to 0.757, 0.802, 0.809 and 0.817 respectively with each additional variable (age, CNS sub-score, cardiovascular sub-score, liver sub-score) as shown in eTable 1. Effect plots of the prediction model are displayed in eFigure 1.

Figure 2. Plot of Variable Importance Scores from Random Forest Analysis.

Legend: The variable importance plot shows how important to the model each variable is.

Figure 3. Consensus classification tree.

Legend: This is a computer-generated “flowchart” to predict 30-day mortality. A p<0.01 is achieved at each division and each final grouping contains at least 800 patients. The bar charts at the base of the figure depict 30-day mortality for that patient population.

Although each model generated slightly different rankings of variable importance, all models included age and CNS sub-scores as most predictive of death and Renal and Coagulation sub-scores as least predictive of mortality. The sensitivity analysis in which we imputed the GCS verbal sub-score did not change the main findings of the study.

The secondary clustering analyses revealed no meaningful clusters within the SOFA sub-scores (gap statistic confirmed the ideal number of clusters = 1). Silhouette plots with varying cluster sizes and techniques demonstrated that no meaningful clusters had a silhouette width > 0.25 (maximum average silhouette width = 0.24 with two clusters), as shown in eTable 2.³⁸

Our burden of missing data after our standard imputation of using values in the prior 24 hour period was 9% for bilirubin and <1% for all other organ systems. Sensitivity analyses employing various approaches to missing data as well as analyses utilizing diagnostic subgroups versus the entire population revealed minimal differences and did not influence the main findings of the study.

DISCUSSION

In a large, unselected cohort of patients admitted to a mixed-profile ICU, the SOFA CNS sub-score dominated the association between admission SOFA score and 30-day mortality. Age, Cardiovascular sub-score, Liver sub-score, Respiratory sub-score and Elixhauser comorbidity index were all moderately predictive of 30-day mortality. Renal and Coagulation were the least predictive SOFA sub-scores. It may be relevant to the practicing intensivist that not all SOFA sub-scores are equally predictive of mortality. Similarly, this finding may be relevant to applications of the SOFA score to questions of ICU triage.^19,49–51

This study highlights several important issues with the SOFA scoring system. While we included trauma patients in this study, the domination of the SOFA mortality prediction by the CNS sub-score was present in patients with all other admitting diagnoses. Our sensitivity analysis imputing the GCS verbal sub-score for the SOFA CNS sub-score confirms the importance of the CNS component of the SOFA score. By utilizing three different methods (imputed verbal component in all patients, imputed verbal component in only intubated patients, and assigning all intubated patients a verbal sub-score of 1 as per the standard GCS methodology) we show that the association between the SOFA CNS sub-score and mortality is independent of intubation status.

Second, we found the Cardiovascular sub-score to be poorly discriminatory. Employing non-standard cutoffs/conversion factors for different vasopressors, and specifying single vasopressors rather than the sum of simultaneous vasopressor infusion rates may have contributed to the poor discrimination associated with the Cardiovascular sub-score. There appears to be room for improvement in the SOFA cardiovascular sub-score.

Third, the addition of age to the SOFA score appears to improve mortality prediction substantially. Age was the second most predictive variable after the CNS sub-score in our patient population. It is likely that age is a combined marker of multiple factors including comorbidities, physiologic reserve, desire for aggressive medical care, and ability to recover after injury.

We hypothesized the existence of meaningful sub-groups, “clusters” or “phenotypes”, of MODS measured by SOFA in a general ICU population. However, standard clustering techniques did not demonstrate the presence of meaningful sub-groups. In a general ICU population, SOFA appears unable to identify meaningful MODS phenotypes. We speculate that within specific disease groupings, such as sepsis, the SOFA score or a modification thereof might still be able to identify meaningful MODS phenotypes.

This study has several limitations. The data were obtained retrospectively and are subject to the limitations inherent in all retrospective studies. We were unable to capture urine output in a reliable manner to use it as described in the original SOFA publications, which may have contributed to the poor predictive ability of the renal component. The study population is from a single ICU staffed mainly by one group of intensivists and may not generalize to other healthcare environments. This study also has several strengths. The broad patient composition increases the generalizability of this study. The EMR at the study hospital is highly detailed, with all study data entered prospectively, allowing for considerably less missing data than in most other retrospective studies.¹⁸ Furthermore, we tested our hypothesis with multiple statistical methods, all of which arrived at the same conclusion.

CONCLUSION

In a mixed population of critically ill patients, the Glasgow Coma Scale score dominates the association between admission SOFA score and 30-day mortality. The addition of age to the SOFA score may be beneficial in prediction of 30-day mortality in a mixed ICU population. Future research into outcomes from multiple organ dysfunction may benefit from new models of meaningful organ dysfunction, or perhaps developing or validating a differently weighted SOFA scoring system.

Supplementary Material

01. eTable 1 – Logistic regression model results.

Legend: Stepwise bootstrap feature selection for multivariate naïve Bayesian classification model of 30-day mortality with 95% confidence intervals. Each additional variable is added to the prior variable(s) to increases the predictive nature of the model.

02. eTable 2 – Average Silhouette Width using 3 techniques.

Legend: A cluster with a silhouette width > 0.25 suggests the clustering may be significant. The lack of a solution with a silhouette width > 0.25 suggests all the clustering solutions are artificial categorizations and not meaningful.

03. eFigure 1 – Effect Plots.

Legend: Effect plots detailing the association, with other predictors held constant, between a given predictor and 30-day mortality. Dotted lines represent 95% confidence intervals for the association.