Utility of the “Surprise” Question in Predicting Survival among Older Patients with Acute Surgical Conditions

Abstract

Background: The surprise question is a validated tool for identifying patients with increased risk of death within one year who could, therefore, benefit from palliative care. However, its utility in surgery is unknown.

Objective: We sought to determine whether the surprise question predicted 12-month mortality in older emergency general surgery patients.

Design: This was a prospective cohort study.

Setting/Subjects: Emergency general surgery attendings and surgical residents in or beyond their third year of training at a single tertiary care academic hospital from January to July 2014.

Measurements: Surgeons responded to the surprise question within 72 hours of evaluating patients, ≥65 years, hospitalized with an acute surgical condition. Patient data, including demographic and clinical characteristics, were extracted from the medical record. Mortality within 12 months of initial evaluation was determined by using Social Security death data.

Results: Ten attending surgeons and 18 surgical residents provided 163 responses to the surprise question for 119 patients: 60% of responses were “No, I would not be surprised” and 40% were “Yes, I would be surprised.” A “No” response was associated with increased odds of death within 12 months in binary logistic regression (OR 4.8 [95% CI 2.1–11.1]).

Conclusions: The surprise question is a valuable tool for identifying older patients with higher risk of death, and it may be a useful screening criterion for older emergency general surgery patients who would benefit from palliative care evaluation.

Introduction

Recent trends toward increased healthcare intensity preceding death have expanded the surgeon's role as a crucial provider of end-of-life care. Up to one-third of Medicare beneficiaries receive surgery in the last year of life, which is associated with increased hospitalization and intensive care unit utilization.¹ Older emergency surgery patients experience particularly high morbidity and mortality rates due to their high burden of serious underlying illnesses, including cancer, dementia, and frailty.^2–5

Previous work suggests that a dearth of prognostic tools hinders clinical decisions and leads to nonbeneficial emergency surgery in older, seriously ill patients.⁶ A better understanding of patients' overall prognoses can improve the appropriateness of surgical care. This may reduce the intensity of healthcare utilization preceding death, leading to earlier hospice referrals and an improved quality of life.^7–9 Therefore, it is critically important to arm surgeons with tools to recognize and address patients' palliative needs. Nonetheless, few studies have examined screening tools for palliative care needs in the surgical patient population.¹⁰

A consensus report from the Center to Advance Palliative Care endorsed criteria for identifying hospitalized patients who would benefit from inpatient palliative care.¹¹ One of the recommended criteria is the surprise question, a validated tool that asks physicians whether they would be surprised if a patient died within one year.^12–16 However, studies have not investigated the utility of the surprise question for acutely ill surgical patients. We conducted a study to determine the prognostic utility of a “No” response to the surprise question for older patients hospitalized with acute surgical conditions.

Materials and Methods

This prospective cohort study was performed at a single academic medical center. The Partners Human Research Committee approved this study.

Participants

The emergency general surgery service (ESS) at this hospital is staffed by surgical attendings and residents who provide 24-hour, in-house coverage for patients admitted to the ESS, as well as inpatient and emergency department consultations. Surgical residents in or beyond their third year of training and ESS attendings were eligible to participate.

Procedures

Surgical residents and attendings were given a fact sheet detailing the study, but they received no special training for using the surprise question. During the six-month enrollment period, attendings and residents were sent an e-mail survey containing the surprise question within 36 hours of evaluating a patient, ≥65 years, for an acute surgical condition. The e-mail survey contained the question: “Would you be surprised if the patient did not survive in the next 12 months, even with surgery?” with possible responses of “Yes, I would be surprised” or “No, I wouldn't be surprised.” Surveys were completed within 72 hours of the initial patient evaluation. Patient data were collected from the electronic medical records, and 12-month vital status was determined by using Social Security death data.

Variables

The primary outcome was mortality within 12 months of ESS evaluation. The primary independent variable was response to the surprise question. Patient characteristics included age, sex, race, surgical diagnosis, presence of a do-not-resuscitate (DNR) code status, and the following clinical indicators of advanced illness: poor baseline performance (limited self-care or >50% in bed or chair), progressive weight loss (>10% of total body weight in prior 6 months), cancer diagnosis, metastatic cancer, end-stage renal disease, severe heart failure, oxygen-dependent lung disease, and serum albumin <2.5 g/dL.¹⁷ These clinical indicators were chosen based on recommended screening criteria to identify patients potentially meeting eligibility for palliative care consultation or hospice.^6,11,17–19 Receipt of surgical treatment and palliative care consultation during the hospital stay were reported as secondary descriptive outcomes.

Analyses

The unit of analysis was response to the surprise question. Patients were divided into two groups according to surprise question classification. Responses were obtained from each surgical resident and attending independently, without knowledge of the other's assessment. Therefore, each response to the surprise question was treated as a discrete data point. Of 163 total responses, 44 (27%) were about the same patient. In cases where both the resident and attending responded to the survey regarding the same patient, patient characteristics and outcomes were counted multiple times. Groups were compared by using T-tests, chi-square, or Fisher's exact test, as directed by the data. The association between surprise question classification and 12-month mortality was assessed by using binary logistic regression, clustering standard errors by respondent. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and area under the receiver operating characteristic curve (AUROC) were calculated. Analyses were performed by using Stata v.14.0 (StataCorp, College Station, TX) with α = 0.05.

Results

From January to July 2014, 10 attendings and 18 residents participated in this study. Responses from residents (n = 90) and attendings (n = 73) were combined for a total of 163 responses for 119 patients. The overall survey response rate was 70.6%. Patient characteristics associated with a “No” response to the surprise question (n = 93, 60%) included older age, non-Hispanic white race/ethnicity, DNR code status, and metastatic cancer (Table 1). Fewer patients from the “No” group underwent surgical procedures (23.5% vs. 44.6%, p < 0.01). Six patients received inpatient palliative care consultation, all of whom were from the “No” group.

Table 1.

Patient Characteristics Grouped by Response to the “Surprise” Question (N = 163)

	“Yes”	“No”	p
Age mean (SD), years	73.5 (7.0)	79.3 (7.9)	<0.01
Female, n (%)	53.8	51.0	0.72
Caucasian/White race	80.0	90.8	0.05
DNR code status	0	13.3	<0.01
Poor baseline performance	12.3	19.4	0.23
End-stage renal disease	3.1	10.2	0.09
Severe heart failure	1.5	1.0	0.77
O2 dependent	3.1	9.2	0.13
Current cancer diagnosis	26.2	38.8	0.10
Metastatic cancer	4.6	19.4	0.01
Progressive weight loss	10.8	13.3	0.63
Serum albumin <2.5 g/dL	6.2	13.3	0.15
Surgical diagnosis			0.17
Small bowel obstruction	20.0	28.6
Biliary disease	18.5	10.2
Non-ischemic colitis	12.3	8.2
Ischemic bowel	3.1	15.3
Perforated viscus	9.2	10.2
Gastrointestinal bleed	7.7	7.1
Hernia	4.6	4.1
Failure to thrive/feeding tube	4.6	3.1
Soft tissue infection	6.2	5.1
Pancreatitis	3.1	5.1
Other	10.8	3.1

DNR, do-not-resuscitate.

Overall, 39% of patients died within 12 months of the initial ESS evaluation. A “No” response was associated with increased odds of death within 12 months (OR 4.8 [95% CI 2.1–11.1], p < 0.001).

Discrimination and predictive value statistics are shown in Table 2 alongside previously published results.^12–16,21 As a screening tool for 12-month mortality, a “No” response had 81% [95% CI 71–91%] sensitivity, 51% [41–61%] specificity, 52% [42–61%] PPV, 82% [72–91%] NPV, and 64% accuracy. AUROC was 0.67 [0.59–0.75], indicating poor predictive performance.

Table 2.

Published Data on the Prognostic Value of the Surprise Question

Article	Respondent	Patients	SENS (%)	SPEC (%)	PPV (%)	NPV (%)	ACC (%)	Mortality
This study	Emergency general surgeons	Older patients with acute surgical conditions (N = 119)	81	51	52	82	64	39% at 12 months
Barnes et al.20	General practitioners	Heart failure patients (N = 542)	79	61	12	98	62	6% at 12 months
Cohen et al.12	Nephrologists	Adult ESRD patients on HD (N = 450)	38	91	55	83	78	23% at 24 months
Moroni et al.13	General practitioners	Patients with Stage IV cancer (N = 231)	84	69	69	94	76	45% at 12 months
Moss et al.14	Nephrology APN	ESRD patients on HD (N = 147)	46	81	29	89	76	15% at 12 months
Moss et al.15	Oncologists	Oncology patients with breast, lung, or colon cancer (N = 826)	75	90	41	98	87	9% at 12 months
Pang et al.16	Nephrologists	ESRD patients on PD (N = 367)	61	75	25	93	73	12% at 12 months

ACC, accuracy; APN, advanced practice nurse; ESRD, end-stage renal disease; HD, hemodialysis; PD, peritoneal dialysis; PPV, positive predictive value; NPV, negative predictive value; SENS, sensitivity; SPEC, specificity.

Discussion

Among older patients hospitalized with acute surgical conditions, this study found that a “No” response to the surprise question was associated with increased 12-month mortality. Our findings suggest that the surprise question may be a valuable screening tool for identifying older patients hospitalized with acute surgical conditions who would benefit from inpatient palliative care.

These data corroborate previous work demonstrating an association between surprise question responses and 12-month mortality, and they expand the literature by establishing these findings in older patients with acute surgical conditions. Previous studies have tested the surprise question with general practitioners, nephrologists, and oncologists who often have established relationships and longitudinal follow-up with their patients.^12–16,22 These aforementioned physician-patient relationships differ from hospital-based surgeons, who typically meet patients for the first time during an acute decline in health and are rarely involved in their care far beyond the acute hospitalization, especially when surgery is not performed. This may be why 60% were “No” responses, of whom only half eventually died. Despite basing their assessments on a limited glimpse of patients' health trajectory, a “No” response from a surgeon was associated with nearly five times higher odds of death and was more sensitive than what has been reported in most similar studies.^{12,14–16,21}

Much has been written about surgeons' tendency toward overly optimistic prognostic assessments.^23–25 Interestingly, surgeons in this study erred in the opposite direction: Responses were the weakest in terms of specificity, with surgeons indicating that they would not be surprised if 60% died within one year. The emergency surgeon's scope of practice may generate this overestimation of mortality. Patients with acute surgical conditions are often older and have more baseline comorbidities than patients undergoing elective surgery.⁵ Furthermore, they have excess morbidity and mortality associated with the emergent nature of the surgery itself.⁴ The sudden, unexpected nature of critical illness may impact emergency surgeons' perceptions of risk and raise their threshold for surprise.

Responses to the surprise question demonstrated low accuracy for predicting 12-month mortality; however, responses to the surprise question are not intended to predict death, dictate patient management, or limit appropriate surgical care. Rather, it is a primary screening tool for identifying patients at risk of unmet palliative needs at the time of admission and during their hospitalization.¹¹ The utility in using the surprise question in the acute setting is that it prompts clinicians to consider the patient's prognosis more broadly than the typical, in-the-moment decision making, which is characteristic of the emergency setting. Asking and answering the surprise question is an extremely cost-effective (free) way to trigger this assessment of the patient's overall prognosis. In our study, the surgeons received no special training in how to answer the question—they were simply asked to respond with their own personal judgment after initial evaluation of the patient. Moreover, screening patients by asking and answering the surprise question is remarkably time-efficient—it requires only a few moments, but introduces a pause that can prompt a consideration of the patients' possible palliative care needs. In spite of the large number of “No” responses, few patients received palliative care consultations, suggesting that these patients may have unmet palliative care needs. Multicenter studies are needed to validate the surprise question as a prognostic tool in older emergency surgery patients, and then to determine whether a “No” response to the surprise question can be used to deliver targeted palliative care interventions.

Findings from this study should be interpreted in light of its limitations. First, the generalizability of this study is limited by its small size, and that it was conducted at a single tertiary institution. However, the purpose of this study was to demonstrate proof-of-concept that the surprise question is relevant in an acute surgical population. Second, because of the random nature of the on-call schedule, each individual surgeon provided responses for a small number of patients over the course of the study, which prohibited analysis of surgeon-level factors associated with prognostic accuracy. Third, data on healthcare utilization beyond the hospital admission were not available; therefore, end-of-life outcomes, such as enrollment in the hospice benefit before death, which could further illustrate gaps in palliative care, were not available for analysis. Finally, the delay in response may have introduced recall bias, and early management during this time may have altered surgeons' initial impressions.

Conclusion

Given national efforts to improve access to palliative care and hospice, there is an urgent need for validated screening tools to trigger palliative care assessments in surgical patients. The surprise question may be a useful screening criterion in initiating goals of care conversations and considering palliative care consultation. Larger, multisite investigations are needed to determine whether the surprise question can be used to improve recognition of unmet palliative care needs and to promote timely access to palliative care for hospitalized older emergency general surgery patients.

Acknowledgments

The authors would like to thank Christopher Calahan and Justin Wang for help with data acquisition. This study was funded in part by the American Geriatrics Society Jahnigen Career Development Award (Z.C.) and NIA GEMSSTAR 1R03AG042361 (Z.C).

Author Disclosure Statement

No competing financial interests exist.