Performance of SAPS II AND SAPS III Scores in Post-Cardiac Arrest

Abstract

Background

Cardiac arrest is a major public health issue affecting an estimated 300,000 patients in the United States each year. The American Heart Association has recommended the Simplified Acute Physiology Score II and 3 (SAPS) to assess severity of illness and to predict outcomes in the post-cardiac arrest population. Our objective was to determine if SAPS II and SAPS III scores predict outcomes in post-cardiac arrest patients.

Methods

We performed an observational study of patients suffering cardiac arrest with return of spontaneous circulation. Data were collected prospectively and recorded in the Utstein style. SAPS II and SAPS III scores were calculated for each subject. Logistic regression was used to assess the relationship between the calculated severity of illness score and in-hospital mortality and poor neurologic outcome.

Results

A total of 274 subjects were identified for analysis. SAPS II was a significant predictor of in-hospital mortality (OR: 1.05, 95%CI: 1.03 – 1.07) and poor-neurologic outcome (OR: 1.06, 95%CI: 1.04 – 1.08). SAPS III was a significant predictor of in-hospital mortality (OR: 1.04, 95%CI: 1.02 – 1.06) and poor-neurologic outcome (OR: 1.04, 95%CI: 1.02 – 1.05). Both scores had moderate ability to discriminate survivors from non-survivors (SAPS II AUC: 0.70; SAPS III AUC: 0.66), and good neurologic outcome from poor neurologic outcome (SAPS II AUC: 0.71; SAPS III AUC: 0.65).

Conclusions

SAPS II and SAPS III scores have only moderate discrimination and are not clinically relevant tools to predict outcome in post-cardiac arrest patients. Further study is needed to identify a more reliable severity of illness score in the post-arrest population.

Introduction

Cardiac arrest is a major public health issue affecting an estimated 300,000 patients in the United States each year.[1] The survival rate from cardiac arrest remains discouraging with recent studies estimating survival between 6% and 18% in out-of-hospital cardiac arrest (OHCA) and in-hospital cardiac arrest (IHCA), respectively.[2, 3] Recent advances in post-arrest management including therapeutic hypothermia have helped improve outcomes.[4, 5] However, we have yet to establish a valid prediction tool for outcomes in the post-arrest population. The development of a validated scoring system is, therefore, a major focus of current research in cardiac arrest.

Several scoring systems have been developed to assist with predicting outcomes in critically ill patients.[6–8] The Simplified Acute Physiology Score II (SAPS II) is a general measure of severity of disease and is used commonly as a severity of illness score in the ICU and is calculated from data collected over a 24-hour observation period.[9] SAPS III was developed recently as an alternative severity of illness score in a global cohort of critically ill patients and is computed from data available immediately at the time of ICU admission.[10] In light of the dynamic nature of the immediate post-arrest period a recent consensus statement from the American Heart Association (AHA) suggested that the SAPS III Admission Score might be a more relevant predictor of post-arrest mortality given the immediacy of risk-stratification.[11] However, neither SAPS II nor SAPS III has been tested specifically in the post-arrest population.

The aim of the current investigation was to determine whether SAPS II and SAPS III scores predict mortality in post-arrest patients and to identify individual components of the score that are most useful in predicting both mortality and neurologic outcome in post-cardiac arrest populations. We hypothesized that these previously validated severity of illness scores would offer good discrimination for outcomes in the post-arrest population.

Methods

Study Population and Setting

This is an observational study of patients who presented to the emergency department (ED) after out-of-hospital cardiac arrest or suffered a cardiac arrest while inpatient at Beth Israel Deaconess Medical Center (BIDMC) between December 2007 and December 2010. BIDMC, located in Boston, MA is an urban tertiary medical center and a major teaching hospital of Harvard Medical School, with an annual ED census of 56,000 patients and 60 intensive care unit beds. BIDMC has served as a cardiac resuscitation center since 2008 and, as such, has an active therapeutic hypothermia protocol, which is standard of care at our institution. Eligible subjects were identified prospectively in the ED or intensive care units of our facility. Inclusion criteria consisted of: 1. age ≥ 18 years; and 2. cardiac arrest with subsequent return of spontaneous circulation (ROSC). Patients were excluded from the study if the etiology of the cardiac arrest was primary traumatic etiology, or if the patient was pregnant. The institutional review board approved the study and a waiver of informed consent was provided.

Data Collection

Data was collected prospectively by reviewing medical records of eligible patients and recorded in the Utstein style, which includes a glossary of terms and a template of features to describe when reporting on cardiopulmonary resuscitation.[12] The worst measurements in the first 24 hours post-hospital arrival were recorded. Patient demographics, co-morbid conditions, vital signs and laboratory values, data necessary for computing SAPS II, SAPS III scores were collected prospectively. Neurologic outcomes were determined by Cerebral Performance Category (CPC) scores at the time of hospital discharge. In-hospital mortality was recorded.

Severity Scores

CPC

CPC scoring is the standard neurologic assessment in post-cardiac arrest patients and is graded on a scale of 1 – 5 (one being no neurologic injury and 5 being death). For the purposes of the current investigation, subjects were categorized into good (CPC 1, 2) or poor (CPC 3,4,5) neurologic outcome for analysis. This method of dichotomization has been used previously in cardiac arrest investigations.[4, 5]

SAPS II

SAPS II scores were calculated according to previous literature and were calculated based on the worst values present in the first 24 hours following ROSC.[9] Variables included in this score are type of admission (medical or surgical), chronic or co-morbid disease prior to admission, Glasgow Coma Scale (GCS), age, systolic blood pressure (SBP), heart rate (HR), temperature (degrees Fahrenheit), PaO₂/FiO₂ if the patient is receiving invasive or mechanical ventilation, total urine output over the previous 24 hours, white blood cell count (WBC), potassium (K), sodium (Na), sodium bicarbonate (HCO₃), blood urea nitrogen (BUN), and total bilirubin. All OHCA subjects were automatically assigned ‘Medical’ for ‘type of admission’. Subjects receiving vasopressor therapy were assigned the worst score for the systolic blood pressure variable. The cumulative score was calculated for each study subject. Scoring ranges from 0–163.

SAPS III

SAPS III scores were calculated based on guidelines for use provided by the SAPS III outcomes research group.[10] Scores are generated using variables separated into three categories, all of which are available at time of admission. Category I variables include age, co-morbidities, use of vasoactive drugs before admission, intra-hospital location prior to ICU admission, and length of stay in the hospital prior to ICU admission. Category II variables include indication(s) for ICU admission, planned/unplanned admission, surgical status and anatomical site of surgery, as well as presence of infection at ICU admission. All OHCA admissions were listed as “cardiovascular” for reason for admission and all were listed as “unplanned admission” per recommendation from the original article. Category III variables include hemodynamic and physiologic measurements. These include GCS, heart rate, systolic blood pressure, total bilirubin, core body temperature, creatinine, leukocytes and platelet counts, hydrogen ion concentration (pH), and ventilatory support and oxygenation. For the purpose of our study these values were taken as the first available after ROSC. If a patient was on a vasoactive medication at the time the SBP was measured they were automatically given the worst SBP score. SAPS III scores range from 0–217 with the highest score being the worst.

Statistical Analysis

Simple descriptive statistics including means or medians for continuous data, or frequencies with percentages for discrete data, were used to describe the study population. Logistic regression was used to predict probability of mortality and poor neurologic outcome based on the calculated severity scores as a continuous variables. The odds ratio (OR) and 95% confidence interval (95% CI) were calculated for each score, with in-hospital mortality and poor-neurologic outcome acting as the dependent variables. An additional analysis was performed to identify individual variables associated with outcome. Individual SAPS II and SAPS III variables associated with outcomes were identified with a forward stepwise selection procedure with an entrance criterion of p = 0.2 and a maintenance criterion of p = 0.05. Units of measure for predictor variables are the numeric risk-scores assigned to abnormal values in each of the SAPS scores. As pre-planned sub-group analysis of OHCA and IHCA was performed. OR and 95% CI were calculated for the variables selected. Discrimination was assessed using area under the curve (AUC) of the receiver operating characteristic (ROC) curve.[13] Calibration was assessed with the Hosmer-Lemeshow (H-L) Chi-square goodness-of-fit test. Statistical significance was set at p = 0.05 and all tests of the data were performed in SAS v9.2 (SAS Institute, Cary, NC, USA).

Results

Baseline Characteristics

A total of 274 subjects who suffered cardiac arrest with ROSC were included in this analysis. The mean age of the cohort was 66 (+/− 17), and 34% (92/274) of the patients were female. In total, there were 139 (51%) OHCA and 135 (49%) IHCA. The median downtime was 15 minutes (IQR: 6 – 27) and the median initial lactate was 6.0 mmol/L (IQR: 3.6 – 8.6). In-hospital mortality was 54% (143/274). Baseline characteristics and cardiac arrest event data, as well as a comparison of data between OHCA and IHCA cohorts, are in Table 1.

Table 1.

Baseline characteristics of cardiac arrest subjects.

Characteristic	All CA	OHCA	IHCA
Total number (N)	274	139	135
Age – yr. (+/− SD)	66 (17)	65 (18)	68 (16)
Female – no. (%)	92 (34)	42 (30)	50 (37)
Race – no. (%)
White	223 (83)	118 (85)	105 (78)
Black	29 (11)	7 (5)	22 (16)
Asian	11 (4)	5 (4)	6 (4)
Other/Unknown	5 (2)	5 (4)	0 (0)
Co-Morbid Conditions – no. (%)
Cancer	30 (11)	9 (6)	21 (16)
Coronary artery disease	89 (32)	41 (29)	48 (36)
Congestive heart failure	56 (20)	20 (14)	36 (27)
Chronic obstructive pulmonary disease	25 (9)	10 (7)	15 (11)
Diabetes	47 (17)	22 (16)	25 (19)
Hypertension	148 (54)	70 (50)	78 (58)
Witnessed Arrest – no. (%)	225 (84)	99 (71)	126 (93)
Bystander CPR – no. (%)	207 (76)	86 (62)	121 (90)
Initial arrest rhythm – no. (%)
Ventricular fibrillation/tachycardia	103 (38)	63 (45)	40 (30)
Pulseless Electrical Activity	116 (44)	42 (30)	74 (55)
Asystole	41 (16)	25 (18)	16 (12)
Unknown	14 (5)	11 (8)	3 (2)
Intravenous Medication Administered – no. (IQR)
Epinephrine	2.0 (1.0 – 3.0)	2.0 (1.0 – 4.0)	1.0 (1.0 – 4.0)
Atropine	1.0 (1.0 – 2.0)	2.0 (1.0 – 3.0)	1.0 (1.0 – 3.0)
Downtime – min. (IQR)	15 (6 – 27)	19 (9 – 33)	10 (5 – 20)
Initial lactate – mmol/L (IQR)	6.0 (3.6 – 8.6)	6.0 (3.9 – 8.6)	5.8 (3.2 – 8.5)
Vasopressor use – no. (%)	164 (60)	77 (55)	87 (64)
Therapeutic hypothermia – no. (%)	103 (38)	69 (50)	34 (25)
In-Hospital Mortality – no. (%)	143 (54)	80 (58)	63 (47)

Cumulative SAPS Scores

The mean SAPS II score was 70 (95% CI: 68 – 72) and the mean SAPS III was 66 (95% CI: 64 – 68). SAPS II was found to be a significant predictor of in-hospital mortality (OR: 1.05, 95%CI: 1.03 – 1.07) and poor-neurologic outcome (OR: 1.06, 95%CI: 1.04 – 1.08). SAPS III was a significant predictor of in-hospital mortality (OR: 1.04, 95%CI: 1.02 – 1.06) and poor-neurologic outcome (OR: 1.04, 95%CI: 1.02 – 1.05). Both scores had moderate ability to discriminate survivors from non-survivors (SAPS II AUC: 0.70; SAPS III AUC: 0.66), and good neurologic outcome from poor neurologic outcome (SAPS II AUC: 0.71; SAPS III AUC: 0.65). See Table 2 for complete data. Although significant predictors of outcome, the moderate discrimination suggest that neither score is a clinically relevant tool in this population.

Table 2.

Performance of cumulative SAPS II and SAPS 3 scores to predict outcomes in cardiac arrest.

	SAPS II		SAPS 3
	In-Hospital Mortality	Poor Neurologic Outcome	In-Hospital Mortality	Poor Neurologic Outcome
OR (95% CI)	1.05 (1.03 – 1.07)	1.06 (1.04 – 1.08)	1.04 (1.02 – 1.06)	1.04 (1.02 – 1.05)
AUC
All Subjects	0.70	0.71	0.66	0.65
OHCA	0.70	0.74	0.69	0.71
IHCA	0.71	0.70	0.71	0.67

Identification of Individual Predictor Variables

Stepwise selection identified individual SAPS II and SAPS III variables as significant predictors of outcomes (Table 3). SAPS II variables age, BUN, HC0₃ and GCS were found to be most sensitive to predict poor outcomes. Together these variables have good discrimination for in-hospital mortality and poor-neurologic outcome (both AUC = 0.74) in the total population. From SAPS III, age, SBP, blood pH, and GCS had good discrimination for in-hospital mortality and poor-neurologic outcome (AUC = 0.77 and 0.76, respectively) in the total population. Figure 1 shows the AUC results with the variables re-fitted to predict in-hospital mortality. Each of these models were well calibrated for the data (all H-L statistic p > 0.05).

Table 3.

Selection of variables to predict outcomes in SAPS II and SAPS 3 scores in postcardiac arrest patients.

Variable	In-Hospital Mortality OR (95% CI)	Poor Neurologic Outcome OR (95% CI)
SAPS II
HCO3	1.24 (1.09 – 1.41)	1.17 (1.03 – 1.33)
GCS	1.11 (1.06 – 1.16)	1.12 (1.07 – 1.17)
BUN	1.10 (1.01 – 1.20)	1.12 (1.03 – 1.22)
Age	1.07 (1.01 – 1.12)	1.06 (1.00 – 1.12)
AUC	0.74	0.74
H-L Statistic	0.74	0.49
SAPS 3
pH	1.52 (1.18 – 1.97)	1.27 (1.19 – 2.00)
Age	1.15 (1.07 – 1.23)	1.14 (1.06 – 1.22)
SBP	1.14 (1.02 – 1.26)	1.11 (1.01 – 1.22)
GCS	1.06 (1.01 – 1.13)	1.05 (1.00 – 1.12)
AUC	0.77	0.76
H-L Statistic	0.10	0.50

Figure 1.

Figure 1a and 1b: Discrimination of SAPS II variables (age, BUN, HC03 and GCS; left) and SAPS III variables (age, GCS, pH, and SBP; right) to predict in-hospital mortality.

Discussion

In this investigation, we found that cumulative SAPS II and SAPS III scores had moderate ability to discriminate outcomes following cardiac arrest. Examining the individual component variables in these scores, we were able to identify select variables that were strongly associated with neurologic outcome and in-hospital mortality in our post-cardiac arrest population. Together these variables offer an improvement from the original SAPS tools to discriminate survivors from non-survivors and good neurologic outcome from poor neurologic outcome. The same three variables from the SAPS II and SAPS III scores were identified as significant predictors of outcome: age, neurologic status (GCS), and measure of acidosis (blood bicarbonate or blood pH).

The SAPS II and SAPS III scores were first developed for risk-stratification and outcome prediction in all critical illness, as well as a general measure of disease severity.[9, 10] The major difference between the two scores are that the SAPS II is computed using 24 hours of observation in the ICU, while the SAPS III is computed at the time of ICU admission. In the original studies, these scores had good discrimination (SAPS II AUC = 0.88; SAPS III AUC = 0.88), as well as good calibration, in their populations. The use of these scores in post-cardiac arrest populations has not been tested previously, although calibration of these scores in the post-arrest population was recommended by the American Heart Association in a recent position statement.[11] We computed the admission scores (SAPS III) as well as the 24-hour follow-up scores (SAPS II) in a population of post-cardiac arrest patients. We found that neither initial nor follow-up severity of illness scores performed adequately to discriminate good from poor outcomes in this population.

The use of severity of illness scores may assist providers in informing family members of a very poor overall prognosis. With recent advances in post-cardiac arrest care, such as the use of therapeutic hypothermia, a valid severity of illness score may also be useful in assessing the effectiveness of new therapies or in helping to standardize the evaluation of hospital quality of care. The results of the current investigation suggest that current critical illness severity scores such as SAPS II and SAPS III are unable to accurately characterize the severity of the post-arrest syndrome.

An alternative severity of illness score in this population might include cardiac arrest event characteristics, such as initial arrest rhythm, presence or absence of bystander CPR, and total downtime, all of which may contribute significantly to the post-arrest syndrome. Cocchi et al. have previously demonstrated that it is possible to risk-stratify OHCA patients on the basis of initial blood lactic acid level and presence or absence of hypotension immediately following the cardiac arrest.[14] However, because SAPS scores do not include lactic acid level, we did not evaluate the influence of this parameter on outcomes in the current investigation. In addition, we have recently externally validated the OHCA Score [15], a severity of illness score that was derived specifically for OHCA patients.[16] This score also includes variables that are available immediately following cardiac arrest (e.g., downtime, initial cardiac rhythm, lactic acid level, and serum creatinine) and do not require an observation period for data collection. The results of these investigations may help to inform future efforts to identify a reliable severity of illness score for cardiac arrest patients.

Potential limitations should be considered when interpreting the results of the current investigation. First, data was collected at a single center which may limit generalizability of the results due to the relatively small sample size. Second, data were not collected specifically for computation of SAPS II and SAPS III scores but were adapted from a standardized post-cardiac arrest database at our institution. Third, we assessed all cause mortality as an outcome variable in a population that frequently suffers from a neurologic cause of death. While we did not assess the ultimate causes for death our finding that GCS was one of the key predictor variables in this population supports this notion. Finally, we have used a single data set to both identify predictor variables and to assess model fit. This may result in an over-estimate of the strength of our proposed models (age, acidosis and neurologic status). However, this was a secondary aim of our investigation and future investigations should use larger, multi-center datasets with cross-validation to assess the strength of new prediction models.

Conclusion

SAPS II and SAPS III scores have moderate ability to discriminate outcomes in post-cardiac arrest patients. Future research in post-arrest care should identify a more reliable severity of illness score. Such a score will provide proper assessment the effectiveness of new interventions as well as for appropriate risk-adjustments in the evaluation of the quality of post-arrest care.

Key Messages.

• Early severity of illness scoring is important to assist physicians in informing family members about treatment options and to risk-stratify post-cardiac arrest patients

• SAPS II and SAPS III scores are previously established severity of illness scores that were recommended for outcome prediction in the post-arrest patients but have not been tested in this population

• SAPS II and SAPS III scores have only moderate discrimination for in-hospital mortality and poor neurologic outcome

• Individual parameters including age, neurologic status and measures of blood acidosis together offer improved discrimination of outcomes over cumulative severity of illness scores in this population

Abbreviations

AUC

Area Under the Curve (of the Receiver Operator Characteristic Curve)

BUN

Blood Urea Nitrogen

CPC

Cerebral Performance Category

ED

Emergency Department

GCS

Glasgow Coma Scale

ICU

Intensive Care Unit

IHCA

In-hospital cardiac arrest

OHCA

Out of-hospital cardiac arrest

ROSC

Return of Spontaneous Circulation

SAPS II

Simplified Acute Physiology Score II

SAPS III

Simplified Acute Physiology Score 3

SBP

Systolic Blood Pressure