External validation of GO-FAR 2 calculator for outcomes after in-hospital cardiac arrest with comparison to GO-FAR and trial of expanded applications

Abstract

Aim

Externally validate the GO-FAR 2 tool for predicting survival with good neurologic function after in-hospital cardiac arrest with comparison to the original GO-FAR tool. Additionally, we collected qualitative descriptors and performed exploratory analyses with various levels of neurologic function and discharge destination.

Methods

Retrospective chart review of all patients who underwent in-hospital resuscitation after cardiac arrest during the calendar years 2016–2019 in our institution (n = 397). GO-FAR and GO-FAR 2 scores were calculated based on information available in the medical record at the time of hospital admission. Cerebral performance category (CPC) scores at the time of admission and discharge were assessed by chart review.

Results

The GO-FAR 2 score accurately predicted outcomes in our study population with a c-statistic of 0.625. The original GO-FAR score also had accurate calibration with a stronger c-statistic of 0.726. The GO-FAR score had decreased predictive value for lesser levels of neurologic function (c-statistic 0.56 for alive at discharge) and discharge destination (0.69). Descriptors of functional status by CPC score were collected.

Conclusion

Our findings support the validity of the GO-FAR and GO-FAR 2 tools as published, but the c-statistics suggest modest predictive discrimination. We include functional descriptors of CPC outcomes to aid clinicians in using these tools. We propose that information about expected outcomes could be valuable in shared decision-making conversations.

Introduction

Background

In United States, a patient’s code status must be documented at the time of hospital admission. It is widely accepted that the code status order should reflect the patient’s (and/or their surrogate decision-maker’s) expressed wishes. A decision about code status is ideally reached through a process of shared decision-making, in which patients and clinicians consider patient preferences in the context of available treatments and likely outcomes. However, medical professionals often have inaccurate perceptions of prognosis for patients with advanced illness, which can complicate these critical conversations.1, 2, 3, 4

Worse outcomes after cardiac arrest are associated with a variety of patient characteristics including increasing age (over age 70 years), pre-existing medical comorbidities, worse baseline functional status, malignancy, sepsis, pneumonia, renal dysfunction, liver dysfunction, hypotension, non-ventricular fibrillation/tachycardia arrest, and longer duration of resuscitation.⁵ Several of these pre-arrest variables were incorporated as prognostic indicators into the Good Outcome Following Attempted Resuscitation (GO-FAR) calculator.⁶ This validated tool was developed to help risk stratify patients for outcomes after in-hospital attempted resuscitation.7, 8, 9, 10, 11 The GO-FAR score groups patients into one of five risk pools – very low (<1%), low (1–3%), average (3–15%), or above average (>15%) – that reflect their probability of “survival with good neurologic outcome,” defined as having a cerebral performance category (CPC) equal to 1 at the time of hospital discharge. Patients with a CPC of 1 are alert and deemed able to work, with a risk of only mild neurologic or psychological deficit. Table 1 provides definitions for CPC categories.¹²

Table 1.

Definitions of cerebral performance categories.

Cerebral Performance Category	Standard Definition
CPC 1	Good cerebral performance: conscious, alert, able to work, might have mild neurologic or psychological deficit.
CPC 2	Moderate cerebral disability: conscious, sufficient cerebral function for independent activities of daily life. Able to work in a sheltered environment.
CPC 3	Severe cerebral disability: conscious, dependent on others for daily support because of impaired brain function. Ranges from ambulatory state to severe dementia or paralysis.
CPC 4	Coma or vegetative state: any degree of coma without the presence of all brain death criteria. Unawareness, even if appears awake (vegetative state) without interaction with environment; may have spontaneous eye opening and sleep/awake cycles. Cerebral unresponsiveness.
CPC 5	Brain death: apnea, areflexia, EEG silence, etc.

The GO-FAR score’s narrow definition of “good neurologic outcome” may limit its utility for shared decision-making. For some patients and their surrogate decision-makers, alternative neurologic outcomes might be deemed favorable enough to pursue resuscitation. With this in mind, the GO-FAR 2 calculator was developed as a tool to predict patients’ probability of survival with a CPC score of 1 or 2 at the time of hospital discharge.¹³ GO-FAR 2 stratifies a patient’s prognosis for this clinical outcome into very poor (0–5%), poor (6–10%), average (11–30%), or above average (>30%). Many of the GO-FAR 2 predictors overlap with those used in the original GO-FAR calculator. Differences in GO-FAR 2 include the addition of surgical cardiac admission; omission of acute stroke, major trauma, and pneumonia; and substitution of admission with a CPC ≤ 2 for admission from skilled nursing facility.

Objective

We sought to externally validate the GO-FAR 2 tool, since the premise of gaining information about the broader neurologic outcome has promising clinical utility. In addition to this primary objective, we compared the performance of GO-FAR 2 against the well-established GO-FAR calculator. Lastly, with a goal of increasing the clinical utility of both tools, we described the functional attributes of patients in specific CPC categories as derived from our chart review and explored whether the GO-FAR and/or GO-FAR 2 calculators might be applied to other levels of neurologic function and discharge destination- outcomes that could be of interest to patients and clinicians.

Methods

Study design

Our study protocol was reviewed and deemed exempt (#2021-0811) by the Institutional Review Board at Western Michigan University Homer Stryker MD School of Medicine (WMed). We conducted a retrospective chart review for the cohort of all cases of attempted resuscitation after in-hospital cardiac arrest for calendar years 2016–2019 at our institution. Medical record number (MRN) and date of birth were used to identify the electronic medical records (EMR) of patients who had in-hospital cardiac arrest. Data necessary to determine the GO-FAR and GO-FAR 2 scores were automatically extracted from the EMR into a confidential REDCap system on the WMed server for data privacy. Because the extracted data did not include CPC and sometimes had incomplete problem lists, four members of our study team manually reviewed patients’ charts (divided up among reviewers) to identify missing information. Review of clinical notes to determine CPC scores has been identified as a reliable alternative when that information is not documented at the time of treatment.¹⁴ Unfortunately our group did not have the time to complete duplicate reviews. All analyses were conducted without patient identifiers to maintain privacy.

Setting

This study was conducted at an approximately 400-bed community teaching hospital in Southwest Michigan. A wide variety of surgical and medical services are performed, but certain very specialized procedures and sub-specialties are lacking (transplant for example).

Participants

Cases of cardiac arrest were identified from our local hospital’s manually documented “code blue” log. This log is documented manually by the rapid response team and is supposed to include all patients for whom a “code blue” is called and the code team (including critical care RN, respiratory therapist, pharmacist, and resident physician shortly joined by the intensivist) finds it medically indicated to attempt resuscitation with chest compressions and/or defibrillation.

Variables

We calculated GO-FAR and GO-FAR 2 scores by applying the definitions and weights of their respective predictor variables as previously described.6, 13 We used time of hospital admission for assessment of predictor variable values rather than the 2 hours prior to arrest that was used in the original GO-FAR derivation. Time of admission has greater practical utility and has been used in a prior successful external validation study of the GO-FAR score.⁹ In the process of reviewing notes to determine a patient’s CPC, we recorded qualitative comments about functional status. A second reviewer (DB), who was blinded to the original CPC rating, used these comments to independently assign a CPC. A Spearman rank order correlation coefficient was calculated to evaluate the concordance of the two reviewer’s CPC ratings.

Data sources

All data for this study was obtained retrospectively from the electronic medical record. Numerical data (lab values, vital signs) from the time of admission were exported directly from the EMR to RedCap and categorized based on cut-offs defined by the GO-FAR and GO-FAR 2 scores. Admission and discharge CPC scores were assessed by manual review of notes in the electronic medical record including history and physical, discharge summary, physical and occupational therapy notes, and care management notes.

Bias

Reviewers were not aware of participants’ race, ethnicity, or socioeconomic status during chart review unless it was stated in the narrative portion of the note. Our retrospective study was unable to examine effects of bias in the original documentation. Inclusion of all cases over the study time period attempted to reduce selection bias.

Study size

Ad-hoc evaluation of the precision of the 95% simultaneous confidence interval for the proportion with a CPC of 1 within each of the GO-FAR risk category groups was evaluated. Given a sample size of n = 400, we expected 9.3% of patients in the average risk group to have a CPC score of 1 with precision of the estimate as 4.7% margin of error.

Statistical methods

To externally validate the GO-FAR and GO-FAR 2 scores with our study sample, we used the formula published by each calculator to determine which “risk group” defined a patient’s likelihood of survival to discharge with “good neurologic outcome,” which is defined as CPC = 1 for GO-FAR and CPC = 1 or 2 for GO-FAR 2. The frequency and percentage of “good outcome” were reported for each risk group. Stratum specific likelihood ratios were used to evaluate the accuracy of each of the GO-FAR and GO-FAR2 risk categories' correspondence with poor CPC outcome. Area under the ROC curve (AUC) was calculated using the generalized U statistic as a measure of each score's ability to accurately predict survival to discharge at the designated CPC level.

For the exploratory analysis of applying GO-FAR and GO-FAR 2 tools to alternative levels of neurologic outcome and discharge destination, we used a logistic regression model to predict each additional target outcome. The corresponding c-statistic was reported for the following outcomes: CPC of 1; CPC of 1 or 2; CPC of 1, 2, or 3; alive at discharge; and discharged to home. SAS v9.4 was used for analysis.

Results

Participants

The code blue logs identified 401 cases of cardiac arrest over the 4-year study period, but four cases were found to be erroneous entries (code team was called but no cardiac arrest). Therefore, our final sample for analysis was 397 cases of in-hospital cardiac arrest which we believe to be all observed cases in the four-year period.

Descriptive data

Our review found 99.25 cases of in-hospital cardiac arrests with attempted resuscitation per year. During this same timeframe, the hospital had an average of 23,181 adult hospital admissions (16,949 unique patients) per year so the annual rate of in-hospital cardiac arrests was 4.28 per 1,000 hospital admissions.

Patient demographics for our sample, overall and by discharge CPC score, are provided in Table 2. Median age was 66 years. The majority were male (58%), White (78%) and non-Hispanic (97%). In comparison, the median age of adult patients over this same time span in our hospital was 60 years and 38% were male.

Table 2.

Demographic information of patient sample, overall and by discharge CPC score.

Patient Characteristic	Overall (n = 397)	CPC = 1 (n = 36)	CPC = 2 (n = 25)	CPC = 3 (n = 34)	CPC = 4 (n = 17)	Deceased (n = 285)
Age in years – median (IQR)	66 (57, 74)	62.5 (48.5, 70.0)	66.0 (58.0, 73.0)	66.0 (61.0, 72.0)	62.0 (58.0, 73.0)	66.0 (57.0, 75.0)
Sex, N (%)
Female	166 (41.8%)	19 (52.7%)	12 (48%)	18 (52.9%)	3 (17.6%)	114 (40%)
Male	231 (58.2%)	17 (47.2%)	13 (52%)	16 (47.1%)	14 (82.4%)	171 (60%)
Race, N (%)
Black or African American	62 (15.6%)	5 (13.9%)	6 (24%)	5 (14.7%)	5 (29.4)	41 (14.4%)
White or Caucasian	308 (77.6%)	28 (77.8%)	18 (72%)	29 (85.3%)	12 (70.6%)	221 (77.5%)
Other or Unknown	27 (6.8%)	3 (8.3%)	1 (4%)	0	0	23 (8.1%)
Ethnicity, N (%)
Hispanic	10 (2.5%)	1 (2.8%)	1 (4%)	0 (0%)	0 (0%)	8 (2.8%)
Non-Hispanic	387 (97.5%)	35 (97.2%)	24 (96%)	34 (100%)	17 (100%)	277 (97.2%)

In total, 112 of the 397 (28%) patients with in-hospital cardiac arrest survived to the time of hospital discharge. Of these surviving patients, 42 (38%) were able to discharge home. Thirty-eight patients were discharged to a nursing or rehabilitation facility, 15 to a long-term acute care hospital, 8 to a higher level of acute care, 7 to hospice, and 2 to psychiatric hospitals. Survival and discharge destination by GO-FAR risk categories are presented in the supplementary material.

Main results

Our external validation of the GO-FAR 2 tool confirmed the ranges predicted by the tool with a c-statistic of 0.625. The original study reported a c-statistic of 0.6975 using the Get with the Guidelines Database from which the tool was derived. All observed outcomes fell within the ranges predicted by the tool (Table 3). We report the actual observed outcomes of interest with 95% confidence intervals and likelihood ratio compared to the reference group of average survival.

Table 3.

Performance of GO-FAR 2 Tool.

		GO-FAR2 Risk Category
	Very Poor Survival (n = 21)	Poor Survival (n = 77)	Average Survival (n = 273)	Above Average Survival (n = 26)
Predicted Survival Rate Range per GO-FAR2 Tool	0–5%	6–10%	11–30%	>30%
Expected Count	3.2	11.8	41.9	4.0
Observed Frequency*	n = 1	n = 6	n = 43	n = 11
Rate	4.8%	7.8%	15.7%	42.3%
95% Confidence Interval	(2.2%, 7.4%)	(4.6%, 11.0%)	(11.3%, 20.1%)	(36.4%, 48.2%)
Likelihood Ratio	3.63	2.15	0.97	0.25

^*Observed Frequency of subjects with a CPC of 1 or 2.

We ran a similar analysis for the original GO-FAR tool to test its applicability to our sample. For our data set, the GO-FAR tool demonstrated good discrimination between risk pools with a c-statistic of 0.726 as well as good calibration with the anticipated outcomes for predicting hospital discharge with a CPC score of 1 (Table 4).

Table 4.

Performance of GO-FAR Tool.

	GO-FAR Risk Category
	Very Low Survival (n = 24)	Low Survival (n = 93)	Average Survival (n = 194)	Above Average Survival (n = 86)
Predicted Survival Rate Range per GO-FAR Tool	<1%	1–3%	>3–15%	>15%
Expected Count	2.2	8.4	17.6	7.8
Observed Frequency*	n = 0	n = 2	n = 15	n = 19
Rate	0%	2.15%	7.73%	22.09%
95% Confidence Interval	–	(0.41%, 3.89%)	(4.52%, 10.93%)	(17.11%, 27.06%)
Likelihood Ratio	–	4.54	1.19	0.35

^*Observed frequency of subjects with a CPC of 1.

Other analyses

To explore whether GO-FAR and GO-FAR 2 could be used to predict other outcomes of interest- including survival with different neurologic status at discharge, overall survival at discharge, and ability to discharge home- we analyzed the c-statistic for each score and outcome and compared these to the predictive ability of simple age groups (Table 5). The original GO-FAR score had a stronger c-statistic than the GO-FAR 2 for predicting discharge with CPC of 1 or 2. Both scores had decreasing usefulness for predicting survival with worse CPC score at discharge. For survival to discharge without regard to neurologic function, age alone had a stronger c-statistic than either the GO-FAR or GO-FAR 2 score with older age being associated with decreased likelihood of survival. The scores retained their superior predictive value to age alone for ability to discharge home. Numbers and percentages of patients who survived to discharge and went to each discharge destination by GO-FAR and GO-FAR 2 risk categories are available in the supplementary material.

Table 5.

Ability of GO-FAR, GO-FAR 2, and age group to predict various outcomes, values presented as c-statistics.

	CPC of 1	CPC of 1 or 2	CPC of 1, 2, or 3	Alive at discharge (CPC of 1, 2, 3, or 4)	Discharge Home
GO-FAR	0.726*	0.630	0.564	0.558	0.690
GO-FAR2	0.635	0.625*	0.581	0.561	0.605
Age Groupings by year (<70, 70–74, 75–79, 80–84, 85+)	0.584	0.564	0.602	0.596	0.573

^*Outcome the score was designed to predict.

In our chart review, we recorded qualitative comments about functional status to aid in clinical interpretation of CPC scores. Example comments for each CPC category can be seen in Table 6. Eighty-two of these comments were assessed by a second reviewer (DB) blinded to the original rating and assigned a numerical rating. The ratings were then compared and had a Spearman rank-order correlation coefficient of r = 0.869, indicating acceptable correlation.

Table 6.

Descriptive characteristics of functional status by CPC score.

Cerebral Performance Category	Examples of Functional Status Derived from Chart Review
CPC 1	Lives at home, able to complete activities of daily living and at least most independent activities of daily living. Able to participate in volunteer and/or work activities outside the home. May help care for another family member.
CPC 2	Uses assistive device (walker, cane, shower chair), needs help with some personal activities of daily living and some independent activities of daily living (help from family, chore provider, or outside caregiver). Memory is adequate for daily function. Usually living in the community but might be in nursing facility either short- or long-term.
CPC 3	Memory deficits AND/OR physical deficits significantly impair function. Requires 24-hour supervision, direct assistance with basic toileting, dressing, and mobility. May require Hoyer lift or 2-person assist. Most often living in skilled nursing environment but might be at home with significant support.
CPC 4	Impaired responsiveness on neurologic exam. Included those who were intubated and sedated due to critical illness. We had some subjects transfer to outside facilities in critical condition.
CPC 5	We did not have any individuals in our study discharged from the hospital in a brain-dead state.

Discussion

Key results

Our data suggest that the GO-FAR and GO-FAR 2 tools are valid as reported when used at the time of hospital admission, but the GO-FAR 2 tool has less predictive ability. All observed outcomes and confidence intervals in our sample fell within the ranges predicted by each tool. The c-statistic for GO-FAR (0.726) is stronger than the c-statistic for the GO-FAR 2 (0.625), suggesting that the GO-FAR 2 has lesser ability to differentiate risk between groups. The poorer performance could be related to differences in the patient characteristics. Compared to the original GO-FAR 2 publication, our sample was less likely to have a CPC of 1 or 2 on admission (69% vs 81%), less likely to be over 75 years of age (13.5% vs 30.1%), less likely to have septicemia (10.6% vs 18.9%), and more likely to have a non-cardiac admission (80% vs 66.1%). Generally, a c-statistic of at least 0.7 is preferred for predictive models. However, prior work shows that clinician’s guesses of outcomes after cardiac arrest are no better than random chance (c-statistic 0.5).¹⁵ An external validation of the GO-FAR 2 in South Korea had a much stronger c-statistic of 0.807. The accurate predictive ranges in our sample still support clinical utility of the GO-FAR 2 tool. It must always be understood that individuals in any group might have a favorable or unfavorable outcome.

Our exploratory analysis suggests that it might be feasible to expand the reporting on a variety of outcomes within each GO-FAR and GO-FAR 2 risk category. This would require the acknowledgement that the differentiation between groups is strongest for the best neurologic status. Different patients may value different outcomes as they make decisions, and clinicians ideally should be able to provide factual information that is relevant to each patient’s values.

In order for GO-FAR and GO-FAR 2 to be used clinically, clinicians must accurately understand and explain the meanings of the different levels of neurologic function described in each tool. Our inclusion of qualitative descriptors for each CPC category may help clinicians accurately counsel patients about these scores which remain predominantly research rather than bedside tools.

Limitations

Limitations of our study include the limited sample size at a single institution. The assessment of CPC and GO-FAR and GO-FAR 2 score by a single reviewer is a limitation in terms of the objective validity of the score but was necessary given time and personnel constraints. However, the scores’ ability to perform in this setting support the applicability to real-world use by a clinician. Because our cases were identified retrospectively by a manually documented log, we were unable to assess the accuracy and completeness of this data source.

Interpretation

There are many actively evolving discussions about how to best make decisions regarding aggressive medical interventions in advanced illness. Clinical prediction tools like GO-FAR and GO-FAR 2 offer an option for clinicians to provide better prognostication based on substantiated data. The ideal role of predictive scoring tools as part of shared decision-making conversations warrants additional future investigation.

Our study confirms the validity of GO-FAR and GO-FAR 2 risk stratification tools for predicting outcomes after in-hospital cardiac arrest as published in the original studies, namely for discharge CPC score 1 and 1 or 2 respectively. The c-statistic for the GO-FAR2 in our sample was 0.625, which suggests that it is not particularly good at differentiating the risk in the different groups but still better than chance and the ranges predicted by the tool proved accurate. We note that in order for these to be used correctly, the clinician must be able to explain the neurologic outcomes by CPC score to the patient and family and acknowledge the possibility of any outcome in any risk group. Our exploratory analysis suggests that it could be reasonable to report additional outcomes for risk categories of the GO-FAR score, but the tool is best at predicting good neurologic outcome as it was designed for. We propose that the clinical utility of the GO-FAR score could be improved by reporting the percentage likelihood of various neurologic as well as functional outcomes for each risk pool so that discussion could be tailored to the outcome most important to each individual.

CRediT authorship contribution statement

Rheanne Maravelas: Conceptualization, Methodology, Investigation, Writing – original draft, Writing – review & editing. Baturay Aydemir: Methodology, Investigation, Writing – review & editing. Duncan Vos: Methodology, Software, Validation, Formal analysis, Data curation, Writing – original draft, Writing – review & editing. Daniel Brauner: Conceptualization, Methodology, Investigation, Writing – review & editing, Supervision. Rachel Zamihovsky: Investigation. Kelly O'Sullivan: Investigation. Anita F. Bell: Software, Resources.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary material

The following are the Supplementary material to this article: