Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 1.
Published in final edited form as: Am J Kidney Dis. 2017 Feb 15;70(1):93–101. doi: 10.1053/j.ajkd.2016.11.025

Reliability and Utility of the Surprise Question in CKD Stages 4–5

Andrei D Javier 1,2,*, Rocio Figueroa 3,*, Edward D Siew 1,2, Huzaifah Salat 1,2, Jennifer Morse 4, Thomas G Stewart 4, Rakesh Malhotra 5, Manisha Jhamb 6, Jane O Schell 6,7, Cesar Y Cardona 8, Cathy A Maxwell 9, T Alp Ikizler 1,2, Khaled Abdel-Kader 1,2
PMCID: PMC5771496  NIHMSID: NIHMS841878  PMID: 28215946

Abstract

Background

Prognostic uncertainty is one barrier to engaging in goals-of-care discussions in chronic kidney disease (CKD). The surprise question (“Would you be surprised if this patient died in the next 12 months?”) is a tool to assist in prognostication. However, it has not been studied in non–dialysis-dependent CKD and its reliability is unknown.

Study Design

Observational study

Setting & Participants

388 patients at least 60 years of age, with non–dialysis-dependent CKD stages 4–5, who were seen at an outpatient nephrology clinic.

Predictor

Trinary (i.e., ‘Yes’, ‘Neutral’, ‘No’) and binary (‘Yes’, ‘No’) surprise question response.

Outcomes

Mortality, test-retest reliability, and blinded inter-rater reliability

Measurements

Baseline comorbidities, Charlson comorbidity index, cause of CKD, and baseline laboratory values (i.e., serum creatinine/estimated glomerular filtration rate, serum albumin, and hemoglobin).

Results

The median patient age was 71 years with median follow-up of 1.4 years, during which time 52 (13%) patients died. Using the trinary surprise question, providers responded ‘Yes’, ‘Neutral’, and ‘No’ for 202 (52%), 80 (21%), 106 (27%) patients, respectively. About 5%, 15%, and 27% of ‘Yes’, ‘Neutral’, and ‘No’ patients died, respectively (p<0.001). The trinary surprise question inter-rater reliability was 0.58 (95% CI, 0.42–0.72) and the test-retest reliability was 0.63 (95% CI, 0.54–0.72). The trinary surprise question ‘No’ response had a sensitivity and specificity of 55% and 76%, respectively (95% CIs, 38%-71% and 71%-80%, respectively). The binary surprise question had a sensitivity of 66% (95% CI, 49%-80%; p=0.3 vs trinary) but a lower specificity of 68% (95% CI, 63%-73%; p=0.02 vs trinary).

Limitations

Single center, small number of deaths.

Conclusions

The surprise question associates with mortality in CKD stages 4–5 and demonstrates moderate to good reliability. Future studies should examine how best to deploy the surprise question to facilitate advance care planning in advanced non–dialysis-dependent CKD.

Keywords: Chronic kidney disease (CKD), non-dialysis-dependent CKD, advanced CKD, subjective health measure, clinical prediction, survival, mortality, mortality risk prediction, reliability, clinical trajectory, prognostication, end-of-life preferences, Advance planning, Goals-of-care discussions, geriatric, nephrology provider

Millions of older patients with chronic kidney disease (CKD)1,2 have multiple comorbidities, significant symptoms, and decrements in quality of life, independence, cognition, and survival.2–10 A significant number of these patients may not gain meaningful survival or quality-of-life benefit with dialysis initiation. Nephrologists spend considerable time preparing patients for progression to end-stage renal disease (ESRD); however, relatively few engage in discussions of prognosis, goals of care, or advance care planning.11–15 These deficiencies contribute to poor patient awareness of their potential clinical trajectories and the range of available resources and treatment options near the end of life.11–16 These gaps may also result in the delivery of aggressive life-prolonging care that frequently misaligns with patients’ values and preferences.13,15,17–19

Prognostic uncertainty is one barrier to initiating conversations regarding goals of care in older patients with advanced CKD.11,20 One pragmatic provider-based tool to assist in prognostication is the surprise question: “Would you be surprised if this patient died in the next 12 months?”.21,22 The surprise question is a subjective health measure that can identify patients at increased risk of death, similar to patient-based subjective health measures.21,23–26 A key benefit of the surprise question is that it efficiently captures providers’ global assessments of functional capacity and severity of comorbidities, and has been shown to help identify general population, cancer, and ESRD patients who could benefit from shared decision-making and additional support or palliative care services.22,26–29 Despite a clear need for improved mortality risk prediction in advanced CKD, tools like the surprise question have not been studied in this population. Furthermore, some providers have expressed skepticism regarding the scant characterization of its reliability.30,31

We hypothesized that the surprise question would be an effective predictor of mortality in older patients with advanced non–dialysis-dependent CKD and demonstrate acceptable inter-rater and test-retest reliability. To test these hypotheses, we examined the reliability of the surprise question and its association with 12-month mortality in patients aged 60 years or older with non–dialysis-dependent CKD stages 4–5. We also examined whether provider characteristics modified the association between the surprise question and mortality.

Methods

Study Setting and Participants

As part of a larger observational, prospective multicenter study examining mortality in older (age 60 years or older) adults with advanced non–dialysis-dependent CKD stages 4–5, a total of 388 patients were enrolled at an academic, hospital-based ambulatory nephrology clinic from June 2014 through January 2015. Patient eligibility criteria were age 60 years or older and CKD stages 4–5, defined as a most recent estimated glomerular filtration rate (eGFR) < 30 ml/min/1.73m2 and a median eGFR by the MDRD (Modification of Diet in Renal Disease) Study equation of < 30 ml/min/1.73m2 for values within 180 days of the office visit. Exclusion criteria were dialysis dependence, history of kidney transplantation, new patient appointment/initial visit, or eGFR ≥60 ml/min/1.73m2 within the prior 12 months or eGFR ≥40 ml/min/1.73m2 within the prior 4 months (i.e., possible acute kidney injury). Because the study included a provider-based subjective health measure, we excluded visits where providers evaluated a patient for the first time in order to allow them to gain additional familiarity with their patients. Eligible patients were independently identified through daily reviews of the nephrology clinic schedule by members of the research team.

We approached all nephrology providers with an outpatient nephrology continuity clinic to participate in the study. Providers included nephrology faculty, advanced practitioners (who see CKD outpatients independently), and nephrology fellows who provided outpatient CKD care. Faculty or fellows who were leaving the department within 6 months were excluded. All providers gave written informed consent and completed a questionnaire of provider characteristics including age, level of training, medical school country, years since completion of specialty training, proportion of time spent in clinical care, and number of half-day outpatient continuity clinics per week. The Vanderbilt University Institutional Review Board approved this study including a waiver of patient informed consent (#140402).

Surprise Question Assessment

We conducted a brief didactic session to orient providers to the surprise question. For all patients, we used the previously described binary response options (i.e., ‘Surprised’, ‘Not surprised’) as well as a 5-point Likert scale to capture the degree of provider surprise (i.e., How surprised would you be if this patient died in the next 12-months?; Figure S1, available as online supplementary material). Due to provider feedback regarding redundancy of the 5-point scale early in the study, it was collapsed to a trinary scale after responses were collected. The 5-point scale was transformed in the following manner: responses of ‘4’ or ‘5’ were categorized as ‘Surprised’, a response of ‘3’ was categorized as ‘Neutral’, and responses of ‘1’ or ‘2’ were categorized as ‘Not Surprised’. Providers were discretely asked the surprise question (i.e., the 5-point Likert scale and binary versions simultaneously) immediately following an eligible patient’s office visit and responses were recorded on a paper form (Figure S1).

To assess inter-rater reliability, we captured separate assessments from nephrology fellows and faculty who were jointly caring for outpatients. When a fellow and faculty pair saw a patient at the same office visit, research staff ensured that respondents were blinded to the other’s responses. In addition, fellows’ responses were obtained immediately after they saw patients (prior to discussing the case with faculty) to minimize potential contamination. To assess test-retest reliability, we repeated the surprise question with the same provider-patient pairing at the next follow-up visit (generally, 2–4 months later). For test-retest reliability assessments, we excluded patients with documented intervening events that could have influenced the provider’s updated responses (e.g., emergency room visit, hospitalization, new diagnosis or worsening disease status, initiation of dialysis, or severe illness/death of a close family member). The vast majority of test-retest exclusions were for intervening emergency room visits and hospitalizations.

Covariates

We performed manual chart reviews to obtain baseline variables including sociodemographics, comorbidities, common clinical measurements including laboratory values, imaging tests, and non-elective hospitalizations within the prior 12 months (Table S1). We reviewed clinical notes; problem lists; past medical, surgical, and social history; and scanned documents for information. The local electronic health record (EHR) also allows for key word searches, which we used to augment our findings using a published list of key word search terms32 that were supplemented with additional terms for disease states.

To ensure standardized chart review, abstracters were trained using a structured data abstraction form that included search terms and common abbreviations and acronyms. All charts were abstracted by at least 2 non-nephrologist physicians (A.D.J., R.F., or H.S.) and 1 nephrologist (R.M. or K.A.). Discrepancies were adjudicated through consensus. All data were double entered into a REDCap database.33

The data were generally complete for those variables included in the analysis. A total of 23 patients (6% of patients) were missing baseline serum albumin, 2 patients were missing race, and another 2 patients were missing baseline hemoglobin. Overall, 361 patients (93%) had complete data, and 386 patients (99%) had at most one missing variable.

Outcome

Vital status and date of death were determined through linkage with the Social Security Death Master File. To ensure all deaths were captured, EHR chart review and obituary searches were also performed for all patients. Death status and date of death are updated in the EHR based on deaths at affiliated health facilities within the health network. Bereaved family communications are also captured. In the rare cases when survival status and/or date of death remained uncertain, the Tennessee Department of Health Vital Records Office was queried to provide verification on vital status (www.tn.gov/health/article/vr-genealogy). Vital status abstractors were blinded to surprise question responses.

Statistical Analysis

Sociodemographic characteristics, comorbidities, clinical measurements, and laboratory values were described using standard descriptive statistics. The association of first surprise question response (from a faculty or advanced practitioner) and death was examined using time-to-event analyses. A Cox proportional hazards (CPH) regression analysis was used to obtain hazards adjusted for age, gender, and race. To allow for a potentially non-linear effect of age, the variable was included in the model as a restricted cubic spline with three knots. To describe the utility of the surprise question in modeling mortality, we compared the gain in model fit from adding an a priori variable of interest to the age-, gender-, and race-adjusted CPH model. Specifically, we measured improved model fit by calculating the improvement in a measure of predictive discrimination known as the g-index or the Gini mean difference of hazards.34 We calculated 95% confidence intervals (CIs) by implementing the bootstrap (1,000 draws) with the percentile method. The variables of interest were identified from the CKD literature.9,21,24,25,35–47 Because the baseline acuity of patients may not be uniform among providers, we also tested the provider effect in the mortality model. For the few covariates with missing values, we estimated the CPH model using multiple imputation by chained equations (10 imputations).48

To assess inter-rater reliability, we calculated Krippendorff’s alpha and calculated its CI with bootstrapping (M=500 re-samples). To aid in interpretation of the agreement measure, we performed a simulation study by generating data similar in size (number of patients and number of providers) to the data collected for the binary and trinary surprise questions in this study. By varying the probability of agreement, we were able to show how alpha relates to the probability of agreement. To assess test-retest reliability we fit a mixed effects model in which second visit scores were predicted from first visit scores. A random effect for individual providers was included to control for individual provider optimism/pessimism. The scores were scaled so that the resulting regression estimate was a correlation parameter. As a sensitivity analysis, a weighted average of the provider-specific Pearson correlation was used to assess test-retest reliability. We subjectively defined moderate inter-rater and test-retest reliability as values between 0.4–0.6 and good as values between 0.6–0.8.49–51 All analyses used two-sided P values <0.05 for significance. Analyses were performed using R statistical software, version 3.3.0.52

Results

Patient Characteristics

For the 388 patients, the median age was 71 years, 50% were female, 15% were African-American, and 51% had diabetes with microvascular complications (Table 1). Providers responded ‘Yes’, ‘Neutral’, and ‘No’ to the trinary version of the surprise question for 202 (52%), 80 (21%), and 106 (27%) participants, respectively (Table S2). Patients who received a ‘No’ response were older and had more vascular disease and comorbidities as well as lower eGFR, serum albumin, and serum hemoglobin than their ‘Yes’ counterparts (Table 1). Using the binary surprise question, providers responded ‘No’ for 137 (35%) patients. These participants were similarly older and had more vascular disease and comorbidities than their ‘Yes’ counterparts (Table S3).

Table 1.

Baseline characteristics stratified by trinary surprise question response

Total Cohort
(N=388)		 ‘Yes’
(n=202)		 ‘Neutral’
(n=80)		 ‘No’
(n=106)		 P-value
Age, y 71 (65–77) 69 (64–74) 74 (65–80) 74 (68–81) <0.001
Female sex 195 (50) 94 (47) 51 (64) 50 (47) 0.03
Race* 0.8
    White 321 (83) 168 (84) 66 (82) 87 (82)
    Black 58 (15) 30 (15) 12 (15) 16 (15)
    Other 7 (2) 2 (1) 2 (3) 3 (3)
Marital Status* 0.1
    Married 238 (62) 132 (66) 41 (51) 65 (62)
    Divorced / Separated 42 (11) 25 (12) 11 (14) 6 (6)
    Widow/Widower 89 (23) 35 (17) 25 (31) 29 (28)
    Single 14 (4) 7 (3) 3 (4) 4 (4)
Insurance* 0.05
    Medicare 312 (80) 150 (74) 71 (89) 91 (86)
    Private 72 (19) 50 (25) 8 (10) 14 (13)
    Medicaid 3 (1) 1 (0) 1 (1) 1 (1)
CCI 5 (4–6) 5 (3–6) 5 (4–7) 6 (5–8) <0.001
Time since first visit with
provider (mo)		 26.3 (8.6–64.9) 28.6 (12.1–62.0) 30.6 (9.8–82.9) 17.2 (6.2–61.1) 0.07
Comorbidities
    HTN 381 (98) 200 (99) 76 (95) 105 (99) 0.06
    DM with microvascular 196 (51) 91 (45) 44 (55) 61 (58) 0.2
    DM without microvascular 20 (5) 13 (6) 3 (4) 4 (4)
    Dyslipidemia 312 (80) 166 (82) 64 (80) 82 (77) 0.6
    CAD 151 (39) 62 (31) 38 (48) 51 (48) 0.002
    CHF 119 (31) 42 (21) 32 (40) 45 (42) <0.001
    Arrhythmia 97 (25) 34 (17) 25 (31) 38 (36) <0.001
    Cerebrovascular disease 88 (23) 32 (16) 26 (32) 30 (28) 0.003
    PVD 78 (20) 31 (15) 22 (28) 25 (24) 0.04
    Chronic lung disease 87 (22) 38 (19) 18 (22) 31 (29) 0.1
    ESLD 17 (4) 2 (1) 6 (8) 9 (8) 0.02
    Dementia 23 (6) 4 (2) 5 (6) 14 (13) <0.001
    Active cancer 67 (17) 35 (17) 6 (8) 26 (25) 0.01
Cause of CKD 0.01
    Diabetes 183 (47) 85 (42) 42 (52) 56 (53)
    Hypertension 278 (72) 140 (69) 57 (71) 81 (76)
    NSAID nephropathy 31 (8) 19 (9) 6 (8) 6 (6)
    Cardiorenal 30 (8) 7 (3) 6 (8) 17 (16)
    IRD/RAS 30 (8) 13 (6) 7 (9) 10 (9)
    AKI 59 (15) 22 (11) 18 (22) 19 (18)
    Glomerular disease 32 (8) 20 (10) 8 (10) 4 (4)
    ADPKD 9 (2) 8 (4) 1 (1) 0 (0)
Laboratory tests
    eGFRCKD-EPI 21.3 (15.4–26.5) 22.8 (16.9–27.3) 20.4 (16.0–26.4) 19.5 (14.2–24.8) 0.01
    Serum creatinine (mg/dl) 2.5 (2.1–3.3) 2.5 (2.1–3.2) 2.5 (2.1–3.1) 2.8 (2.3–3.7) 0.07
    Serum albumin* (g/dl) 4.0 (3.7–4.2) 4.1 (3.9–4.2) 3.9 (3.7–4.1) 3.8 (3.5–4.1) <0.001
    Hemoglobin* (g/dl) 11.4 (10.3–12.6) 11.9 (10.9–12.9) 10.8 (10.0–11.9) 11.1 (9.80–12.4) <0.001
Open in a new tab

Note: Values for categorical variables are given as count (percentage); for continuous variables, as median [interquartile range]. Conversion factor for serum creatinine in mg/dL to µmol/L, ×88.4.

CCI Charlson Comorbidity Index, CKD-EPI, Chronic Kidney Disease Epidemiology Collaboration; HTN hypertension, DM diabetes, microvascular, microvascular complications, CAD coronary artery disease, CHF congestive heart failure, PVD peripheral vascular disease, ESLD end-stage liver disease, CKD chronic kidney disease, NSAID non-steroidal anti-inflammatory drug, IRD ischemic renal disease, RAS renal artery stenosis, AKI acute kidney injury, ADPKD autosomal dominant polycystic kidney disease, eGFR estimated glomerular filtration rate,

*

N = 386 for race and hemoglobin, N=383 for marital status, N=387 for insurance status, N=365 for serum albumin.

Provider Characteristics

All 28 eligible providers enrolled in the study. A total of 11 nephrology attending physicians, 1 nurse practitioner, 10 second and third year nephrology fellows, and 6 first year nephrology fellows (Table 2) assessed the 388 patients.

Table 2.

Baseline provider characteristics

Provider Characteristics Attending/NP
(n = 12)		 Fellow
(n = 16)
Age, y 48 [39–60] 32 [30–34]
Male sex 7 (58) 10 (62)
Race
    White 10 (83) 9 (56)
    Asian 2 (17) 5 (31)
    Black/African-American 0 (0) 2 (12)
Years since fellowship 11 [5–29] NA
Medical school location
    United States 11 (92) 9 (56)
    Foreign (Graduate) 1 (8) 7 (44)
Percent clinical effort 75 [51 – 80] 93 [78 – 100]
Weekly half-day clinics
    0.5–1.5 3 (25) 16 (100)
    2–4 6 (50) 0 (0)
    >4 3 (25) 0 (0)
Open in a new tab

Note: Values for categorical variables are given as count (percentage); for continuous variables, as median [interquartile range].

Attending, attending physician; NA, not applicable; NP, nurse practitioner

Surprise Question Performance

During a median follow-up of 1.5 (interquartile range [IQR], 1.2–1.6) years, 52 (13.4%) patients died. Using the trinary surprise question, 5%, 15%, and 27% of the surprise question ‘Yes’, ‘Neutral’, and ‘No’ groups died, respectively (p<0.001). The surprise question response was associated with survival (Figure 1), and this association was not significantly modified by provider characteristics (p=0.2) nor the length of the patient-provider relationship (p=0.2). At 1 year, the trinary surprise question ‘No’ response had a sensitivity of 55% (95% CI, 38%-71%) and a specificity of 76% (95% CI, 71%-80%).

Figure 1.

Figure 1

Figure 1

Open in a new tab

a. Survival by (a) trinary surprise question response (p<0.001) and (b) binary surprise question response (p<0.001).

The binary surprise question performed similarly, with 8% and 24% of ‘Yes’ and ‘No’ respondents dying, respectively (p<0.001). At 1 year, the binary surprise question ‘No’ response had a sensitivity of 66% (95% CI, 49%-80%; p=0.3 versus trinary) and a specificity of 68% (95% CI, 63%-73%; p = 0.02 versus trinary).

In CPH models adjusted for age, gender, and race, the trinary surprise question response remained significantly associated with mortality (hazard ratios of 5.3 [95% CI, 2.5–11.1] for ‘No’ versus ‘Yes’ and 2.6 [95% CI, 1.1–6.2] for ‘Neutral’ versus ‘Yes’). Using the Gini mean difference to determine gain in model fit for the age-, gender-, and race-adjusted CPH mortality model, the trinary surprise question performed at least as well as any traditional predictor of mortality (Table S4). The binary surprise question response performed similarly (hazard ratio, 3.3; 95% CI, 1.8-–6.0; Table S4). There was no detectable provider effect (p = 0.2).

Reliability

There were 185 patients who were seen by two nephrology providers (attending and fellow) at the same visit. The inter-rater reliability was 0.58 (95% CI, 0.42–0.72) for the trinary surprise question and 0.53 (95% CI, 0.37–0.67) for the binary surprise question (an agreement probability of approximately 0.75–0.77). There were 306 patients who had test-retest assessments performed. The median time between assessments was 3.5 (IQR, 2.3–4.2) months. The mixed effect model test-retest correlation was 0.63 (95% CI, 0.54–0.72) for the trinary surprise question and 0.66 (95% CI, 0.58–0.75) for the binary surprise question. Weighted average estimates of test-retest correlation were largely unchanged for the trinary and binary surprise question (0.60 [95% CI, 0.51–0.70] and 0.65 [95% CI, 0.58–0.75], respectively). Provider characteristics (i.e., age, years since fellowship, percent clinical effort, half-day clinics per week) did not significantly modify the test-retest reliability (p = 0.2) and there was no significant interaction between time since last follow-up visit and test-retest reliability (p = 0.3).

Discussion

The surprise question associates with mortality in older adults with CKD stages 4–5; patients who received a ‘No’ response died at over 5 times the rate as their ‘Yes’ counterparts (trinary surprise question). Both versions (i.e., trinary and binary) of the surprise question are helpful predictors of death in patients with CKD stages 4–5, comparable to other well-accepted markers of an elevated risk of mortality (e.g., heart failure, serum albumin, and the Charlson Comorbidity Index). Both versions of the surprise question also demonstrated reasonable inter-rater reliability.49–51 In a sample of 28 providers including nephrology fellows, physicians’ ratings were identical in approximately 3 of every 4 patients using either version of the surprise question. In addition, the trinary version of the instrument appropriately decreases the number of patients identified as highest-risk (i.e., better specificity) while still capturing a similar number of deaths as the binary instrument (i.e., similar sensitivity).

Subjective health measures associate with mortality in a variety of settings.21–23,53 When placed in rank order, provider-based predictions of survival highly correlate with observed rank order patient survival.54 However, provider estimates of specific length of survival are often overly optimistic.55,54 The surprise question was designed to overcome the optimistic prognostic assessments of most providers54,55 by modifying the framing of the survival question.21,22 Few studies have been performed in patients with kidney disease, and they have been limited to patients with ESRD. In one study of 822 incident Canadian maintenance dialysis patients, nephrologists were accurate when assigning up to a 50% risk of death within 6 months.56 Moss and colleagues first described the utility of the binary surprise question in patients with ESRD and noted an approximately 30% mortality rate at 1 year for patients in whom the provider would not be surprised if the patient died.21 Similarly, in a cohort of about 370 prevalent Hong Kong Chinese peritoneal dialysis patients, 25% of patients with a ‘No’ response died at 1 year.24

Our findings in an older, prevalent non–dialysis-dependent CKD stages 4–5 cohort extend the potential utility of the surprise question to a much broader patient population,1 where discussions regarding prognosis may be timelier. A surprise question ‘No’ response may represent an opportune occasion for renal providers to engage older patients in prognostic discussions to guide treatment decisions for advanced kidney disease and prepare patients for future setbacks. Older patients with CKD stages 4–5 have substantial symptoms, functional impairments, and frequent hospitalizations that contribute to decrements in quality of life and survival.8,39,57,58 For patients with limited life expectancy who prioritize the quality of their days, conservative management with a focus on symptom management is a reasonable alternative to dialysis.44,59

For several reasons, the surprise question may also be a useful mechanism to initiate advance care preparatory conversations in older CKD patients whose goals are consistent with dialysis. First, these conversations can be revisited following major health setbacks.60 Second, older US patients with CKD stages 4–5 have markedly elevated annual mortality ranging from 12% to 50% depending on age and eGFR.2,61 Third, cognitive decline is less likely to represent a hindrance to meaningful goals of care discussions in patients with non–dialysis-dependent CKD compared to ESRD,62,63 thus increasing the number of patients who can communicate their values and preferences. Fourth, unlike hemodialysis clinic visits, patient-provider interactions routinely occur in a private setting suitable to discussing sensitive topics.16 Fifth, CKD stages 4–5 represent a disease transition point with impending, life-altering decisions that should require informed discussions and shared decision making, even more so in high-risk patients.13,64,65 Sixth, patients desire conversations regarding end-of-life preferences.13,14 Given the high early mortality following dialysis initiation, we believe these conversations should occur earlier. Our study indicates the surprise question can be used to quickly identify patients at significant risk of dying, which may be useful for efficiently prioritizing and targeting those who could benefit from goals-of-care discussions.

Current guidelines on shared decision making and the initiation of dialysis recommend that providers engage in prognostic discussions in patients with CKD stages 4–5 and facilitate advance care planning.66,67 Patients with advanced kidney disease receive more aggressive care near the end of life than patients with other chronic illnesses, including death in the hospital and intensive care unit.17–19 These disparities may negatively affect patient and bereaved caregiver symptoms and mental health at the end of life.68–70 One approach to identify patients and caregivers who could benefit from advance care planning and supportive care would be the use of validated mortality risk prediction models for non–dialysis-dependent CKD.71 While several potentially helpful models for patients with ESRD have been developed,25,72,73 most non–dialysis-dependent CKD models perform suboptimally or are not clinically useful, hindering wide-spread adoption.47 Further research is needed to determine whether the addition of geriatric assessments,74 non–disease-specific measures,75 and subjective health measures, like the surprise question, can improve identification of patients at high risk of dying in the near future.

Structured variables used to represent comorbid diagnoses in risk prediction models do not adequately capture disease severity, frailty, impact on patient function, or the interaction between psychosocial factors, patient adherence, and clinical course. The surprise question may capture some of these elements. Notably, our provider assessments were completed rapidly and may be helpful in practice settings where patient care is compressed and where the feasibility of implementing real-time multivariable risk prediction is limited. The trinary surprise question also demonstrated a step-wise increase in risk, suggesting providers were effective in ordering patients based on estimated prognosis. This added granularity could be helpful when tailoring pathways for additional risk assessment or interventions according to individualized, estimated baseline risk.71 Since most CKD patients prefer that providers initiate prognostic discussions and advance care planning,13,14 pragmatic, reliable, validated tools are needed to facilitate this process.

Both versions of the surprise question demonstrated moderate to good inter-rater reliability despite the inclusion of fellows and attendings with disparate nephrology experience. Given the inherent subjectivity of the measure and the lack of structured training in prognostication, our findings suggest that providers’ clinical intuition is grounded in similar domains. Notably, we asked providers the surprise question immediately upon exiting the clinical encounter to ensure capture of an immediate clinical gestalt. We are not aware of other publications characterizing the reliability of the surprise question.

Our study has several strengths. We excluded patients with acute kidney injury who may exhibit higher mortality, we obtained surprise question assessments immediately after office visits, we ensured providers were blinded to other providers’ surprise question assessments, and we rigorously abstracted health information from the medical record to minimize misclassification bias when comparing accepted mortality predictors to the surprise question. Our study also has several limitations. First, this was a single center study; however, our findings were remarkably similar to prior findings in maintenance hemodialysis and peritoneal dialysis patients. Second, although we enrolled nearly 400 patients, the number of observed deaths precluded us from examining the surprise question association with death in a fully adjusted mortality model. Similarly, the number of deaths may have limited our power to detect meaningful interactions (e.g., between provider characteristics and surprise question performance) or to more precisely compare the surprise question to other predictors of mortality. Fourth, our test-retest reliability was conducted over a median of 3.5 months. Subtle changes in patient status that were not documented in the EHR may have occurred during this time. Hence, the test-retest reliability of the surprise question may be higher if repeated responses are collected over substantially shorter periods than in our study. Finally, we attempted to collect surprise question responses in a blinded fashion to minimize contamination while assessing inter-rater reliability; however, nephrology fellows staffing their patient encounters may have influenced subsequent attending responses.

In conclusion, the surprise question demonstrates acceptable reliability and associates with mortality in older adults with non–dialysis-dependent CKD stages 4–5. The trinary surprise question appears to improve specificity while demonstrating similar sensitivity. Future studies are needed to determine the responsiveness of the instrument and to understand whether incorporating the surprise question into clinical practice can improve patient-centered outcomes in those with advanced CKD.

Supplementary Material

1
2
3
4
5

Acknowledgments

Support: This work was supported by the Satellite Health Norman Coplon Extramural Grant Program and the National Institutes of Health (K23DK090304 to Dr Abdel-Kader), the National Institute of Diabetes and Digestive and Kidney Diseases (K24 DK62849 to Dr Ikizler), the American Heart Association (grant 11FTF7520014 to Dr Jhamb), the Vanderbilt Center for Kidney Disease, and Clinical and Translational Science Award No. UL1TR000445 from the National Center for Advancing Translational Sciences. The funders of this study had no role in study design, data collection, analysis, interpretation, manuscript preparation, or the decision to report the findings.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

An abstract based on these findings was presented at the American Society of Nephrology’s Kidney Week 2016 meeting in Chicago.

Contributions: Research idea and study design: ADJ, RF, KA; data acquisition: ADJ, RF, HS, KA; data analysis/interpretation: ADJ, RF, EDS, HS, JM, TGS, RM, MJ, JOS, CYC, CAM, TAI, KA; statistical analysis: JM, TGS; funding: TAI, KA. Each author contributed important intellectual content during manuscript drafting or revision and accepts accountability for the overall work by ensuring that questions pertaining to the accuracy or integrity of any portion of the work are appropriately investigated and resolved. KA takes responsibility that this study has been reported honestly, accurately, and transparently; that no important aspects of the study have been omitted, and that any discrepancies from the study as planned have been explained.

Financial Disclosure: The authors declare that they have no other relevant financial interests.

Peer Review: Evaluated by two external peer reviewers, a Statistical Editor, a Co-Editor, and Editor-in-Chief Levey.

Supplementary Material

Table S1: Variables of interest.

Table S2: Frequencies of trinary and binary surprise question responses.

Table S3: Baseline characteristics stratified by binary surprise question response.

Table S4: Increase in model fit gained from each respective variable in Cox model after adjustments.

Figure S1: Surprise question collection instrument.

Note: The supplementary material accompanying this article (doi:_______) is available at www.ajkd.org

Supplementary Material Descriptive Text for Online Delivery

Supplementary Table S1 (PDF). Variables of interest.

Supplementary Table S2 (PDF). Frequencies of trinary and binary surprise question responses.

Supplementary Table S3 (PDF). Baseline characteristics stratified by binary surprise question response.

Supplementary Table S4 (PDF). Increase in model fit gained from each respective variable in Cox model after adjustments.

Supplementary Figure S1 (PDF). Surprise question collection instrument.

References

  • 1.Coresh J, Selvin E, Stevens LA, et al. Prevalence of chronic kidney disease in the United States. JAMA. 2007 Nov 7;298(17):2038–2047. doi: 10.1001/jama.298.17.2038. [DOI] [PubMed] [Google Scholar]
  • 2.O’Hare AM, Choi AI, Bertenthal D, et al. Age affects outcomes in chronic kidney disease. J Am Soc Nephrol. 2007 Oct;18(10):2758–2765. doi: 10.1681/ASN.2007040422. [DOI] [PubMed] [Google Scholar]
  • 3.Stevens LA, Li S, Wang C, et al. Prevalence of CKD and comorbid illness in elderly patients in the United States: results from the Kidney Early Evaluation Program (KEEP) Am J Kidney Dis. 2010 Mar;55(3 Suppl 2):S23–S33. doi: 10.1053/j.ajkd.2009.09.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Kurella M, Chertow GM, Fried LF, et al. Chronic kidney disease and cognitive impairment in the elderly: the health, aging, and body composition study. J Am Soc Nephrol. 2005 Jul;16(7):2127–2133. doi: 10.1681/ASN.2005010005. [DOI] [PubMed] [Google Scholar]
  • 5.Kurella M, Chertow GM, Luan J, Yaffe K. Cognitive impairment in chronic kidney disease. J Am Geriatr Soc. 2004 Nov;52(11):1863–1869. doi: 10.1111/j.1532-5415.2004.52508.x. [DOI] [PubMed] [Google Scholar]
  • 6.Kurella M, Covinsky KE, Collins AJ, Chertow GM. Octogenarians and nonagenarians starting dialysis in the United States. Ann Intern Med. 2007 Feb 6;146(3):177–183. doi: 10.7326/0003-4819-146-3-200702060-00006. [DOI] [PubMed] [Google Scholar]
  • 7.Abdel-Kader K, Jhamb M, Mandich LA, et al. Ecological momentary assessment of fatigue, sleepiness, and exhaustion in ESKD. BMC Nephrol. 2014;15:29. doi: 10.1186/1471-2369-15-29. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Abdel-Kader K, Unruh ML, Weisbord SD. Symptom burden, depression, and quality of life in chronic and end-stage kidney disease. Clin J Am Soc Nephrol. 2009 Jun;4(6):1057–1064. doi: 10.2215/CJN.00430109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kurella Tamura M, Covinsky KE, Chertow GM, Yaffe K, Landefeld CS, McCulloch CE. Functional status of elderly adults before and after initiation of dialysis. N Engl J Med. 2009 Oct 15;361(16):1539–1547. doi: 10.1056/NEJMoa0904655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jassal SV, Chiu E, Hladunewich M. Loss of independence in patients starting dialysis at 80 years of age or older. N Engl J Med. 2009 Oct 15;361(16):1612–1613. doi: 10.1056/NEJMc0905289. [DOI] [PubMed] [Google Scholar]
  • 11.Schell JO, Patel UD, Steinhauser KE, Ammarell N, Tulsky JA. Discussions of the kidney disease trajectory by elderly patients and nephrologists: a qualitative study. Am J Kidney Dis. 2012 Apr;59(4):495–503. doi: 10.1053/j.ajkd.2011.11.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Davison SN. Facilitating advance care planning for patients with end-stage renal disease: the patient perspective. Clin J Am Soc Nephrol. 2006 Sep;1(5):1023–1028. doi: 10.2215/CJN.01050306. [DOI] [PubMed] [Google Scholar]
  • 13.Davison SN. End-of-life care preferences and needs: perceptions of patients with chronic kidney disease. Clin J Am Soc Nephrol. 2010 Feb;5(2):195–204. doi: 10.2215/CJN.05960809. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Fine A, Fontaine B, Kraushar MM, Rich BR. Nephrologists should voluntarily divulge survival data to potential dialysis patients: a questionnaire study. Peritoneal dialysis international : journal of the International Society for Peritoneal Dialysis. 2005 May-Jun;25(3):269–273. [PubMed] [Google Scholar]
  • 15.Wachterman MW, Marcantonio ER, Davis RB, et al. Relationship between the prognostic expectations of seriously ill patients undergoing hemodialysis and their nephrologists. JAMA internal medicine. 2013 Jul 8;173(13):1206–1214. doi: 10.1001/jamainternmed.2013.6036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Goff SL, Eneanya ND, Feinberg R, et al. Advance care planning: a qualitative study of dialysis patients and families. Clin J Am Soc Nephrol. 2015 Mar 6;10(3):390–400. doi: 10.2215/CJN.07490714. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.O’Hare AM, Rodriguez RA, Hailpern SM, Larson EB, Kurella Tamura M. Regional variation in health care intensity and treatment practices for end-stage renal disease in older adults. JAMA. 2010 Jul 14;304(2):180–186. doi: 10.1001/jama.2010.924. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Wong SP, Kreuter W, O’Hare AM. Treatment intensity at the end of life in older adults receiving long-term dialysis. Arch Intern Med. 2012 Apr 23;172(8):661–663. doi: 10.1001/archinternmed.2012.268. discussion 663–664. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Wachterman MW, Pilver C, Smith D, Ersek M, Lipsitz SR, Keating NL. QUality of end-of-life care provided to patients with different serious illnesses. JAMA internal medicine. 2016 Aug 1;176(8):1095–102. doi: 10.1001/jamainternmed.2016.1200. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Parvez S, Abdel-Kader K, Song MK, Unruh M. Conveying uncertainty in prognosis to patients with ESRD. Blood Purif. 2015;39(1–3):58–64. doi: 10.1159/000368954. [DOI] [PubMed] [Google Scholar]
  • 21.Moss AH, Ganjoo J, Sharma S, et al. Utility of the “surprise” question to identify dialysis patients with high mortality. Clin J Am Soc Nephrol. 2008 Sep;3(5):1379–1384. doi: 10.2215/CJN.00940208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Moss AH, Lunney JR, Culp S, et al. Prognostic significance of the “surprise” question in cancer patients. Journal of palliative medicine. 2010 Jul;13(7):837–840. doi: 10.1089/jpm.2010.0018. [DOI] [PubMed] [Google Scholar]
  • 23.Robinson-Cohen C, Hall YN, Katz R, et al. Self-rated health and adverse events in CKD. Clin J Am Soc Nephrol. 2014 Dec 5;9(12):2044–2051. doi: 10.2215/CJN.03140314. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Pang WF, Kwan BC, Chow KM, Leung CB, Li PK, Szeto CC. Predicting 12-month mortality for peritoneal dialysis patients using the “surprise” question. Peritoneal dialysis international : journal of the International Society for Peritoneal Dialysis. 2013 Jan-Feb;33(1):60–66. doi: 10.3747/pdi.2011.00204. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Cohen LM, Ruthazer R, Moss AH, Germain MJ. Predicting six-month mortality for patients who are on maintenance hemodialysis. Clin J Am Soc Nephrol. 2010 Jan;5(1):72–79. doi: 10.2215/CJN.03860609. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Amro OW, Ramasamy M, Strom JA, Weiner DE, Jaber BL. Nephrologist-Facilitated Advance Care Planning for Hemodialysis Patients: A Quality Improvement Project. American Journal of Kidney Diseases. 2016 Jul;68(1):103–109. doi: 10.1053/j.ajkd.2015.11.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Pattison M, Romer AL. Improving Care Through the End of Life: launching a primary care clinic-based program. Journal of palliative medicine. 2001 Summer;4(2):249–254. doi: 10.1089/109662101750290335. [DOI] [PubMed] [Google Scholar]
  • 28.Gomez-Batiste X, Martinez-Munoz M, Blay C, et al. Identifying patients with chronic conditions in need of palliative care in the general population: development of the NECPAL tool and preliminary prevalence rates in Catalonia. BMJ Support Palliat Care. 2013 Sep;3(3):300–308. doi: 10.1136/bmjspcare-2012-000211. [DOI] [PubMed] [Google Scholar]
  • 29.Moroni M, Zocchi D, Bolognesi D, et al. The ‘surprise’ question in advanced cancer patients: A prospective study among general practitioners. Palliative medicine. 2014 Mar 24;28(7):959–964. doi: 10.1177/0269216314526273. [DOI] [PubMed] [Google Scholar]
  • 30.Elliott M, Nicholson C. A qualitative study exploring use of the surprise question in the care of older people: perceptions of general practitioners and challenges for practice. BMJ Supportive & Palliative Care. 2014 Aug 28; doi: 10.1136/bmjspcare-2014-000679. 2014. [DOI] [PubMed] [Google Scholar]
  • 31.Small N, Gardiner C, Barnes S, et al. Using a prediction of death in the next 12 months as a prompt for referral to palliative care acts to the. Palliative medicine. 2010;24(7):740–741. doi: 10.1177/0269216310375861. [DOI] [PubMed] [Google Scholar]
  • 32.Singh B, Singh A, Ahmed A, et al. Derivation and Validation of Automated Electronic Search Strategies to Extract Charlson Comorbidities From Electronic Medical Records. Mayo Clinic Proceedings. 2012;87(9):817–824. doi: 10.1016/j.mayocp.2012.04.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research Electronic Data Capture (REDCap) - A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of biomedical informatics. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. 09/30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Harrell FE. Regression modeling strategies : with applications to linear models, logistic regression, and survival analysis. New York: Springer; 2001. [Google Scholar]
  • 35.Berthoux F, Mohey H, Laurent B, Mariat C, Afiani A, Thibaudin L. Predicting the risk for dialysis or death in IgA nephropathy. J Am Soc Nephrol. 2011 Apr;22(4):752–761. doi: 10.1681/ASN.2010040355. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Bowling CB, O’Hare AM. Managing older adults with CKD: individualized versus disease-based approaches. Am J Kidney Dis. 2012 Feb;59(2):293–302. doi: 10.1053/j.ajkd.2011.08.039. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Dalrymple LS, Katz R, Kestenbaum B, et al. Chronic kidney disease and the risk of end-stage renal disease versus death. Journal of general internal medicine. 2011 Apr;26(4):379–385. doi: 10.1007/s11606-010-1511-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Desai AS, Toto R, Jarolim P, et al. Association between cardiac biomarkers and the development of ESRD in patients with type 2 diabetes mellitus, anemia, and CKD. Am J Kidney Dis. 2011 Nov;58(5):717–728. doi: 10.1053/j.ajkd.2011.05.020. [DOI] [PubMed] [Google Scholar]
  • 39.Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY. Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med. 2004 Sep 23;351(13):1296–1305. doi: 10.1056/NEJMoa041031. [DOI] [PubMed] [Google Scholar]
  • 40.Johnson ES, Thorp ML, Yang X, Charansonney OL, Smith DH. Predicting renal replacement therapy and mortality in CKD. Am J Kidney Dis. 2007 Oct;50(4):559–565. doi: 10.1053/j.ajkd.2007.07.006. [DOI] [PubMed] [Google Scholar]
  • 41.Keith DS, Nichols GA, Gullion CM, Brown JB, Smith DH. Longitudinal follow-up and outcomes among a population with chronic kidney disease in a large managed care organization. Arch Intern Med. 2004 Mar 22;164(6):659–663. doi: 10.1001/archinte.164.6.659. [DOI] [PubMed] [Google Scholar]
  • 42.Kurella Tamura M. Incidence, management, and outcomes of end-stage renal disease in the elderly. Current opinion in nephrology and hypertension. 2009 May;18(3):252–257. doi: 10.1097/MNH.0b013e328326f3ac. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Landray MJ, Emberson JR, Blackwell L, et al. Prediction of ESRD and death among people with CKD: the Chronic Renal Impairment in Birmingham (CRIB) prospective cohort study. Am J Kidney Dis. 2010 Dec;56(6):1082–1094. doi: 10.1053/j.ajkd.2010.07.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Murtagh FE, Marsh JE, Donohoe P, Ekbal NJ, Sheerin NS, Harris FE. Dialysis or not? A comparative survival study of patients over 75 years with chronic kidney disease stage 5. Nephrol Dial Transplant. 2007 Jul;22(7):1955–1962. doi: 10.1093/ndt/gfm153. [DOI] [PubMed] [Google Scholar]
  • 45.O’Hare AM, Bertenthal D, Covinsky KE, et al. Mortality risk stratification in chronic kidney disease: one size for all ages? J Am Soc Nephrol. 2006 Mar;17(3):846–853. doi: 10.1681/ASN.2005090986. [DOI] [PubMed] [Google Scholar]
  • 46.Tamura MK, Tan JC, O’Hare AM. Optimizing renal replacement therapy in older adults: a framework for making individualized decisions. Kidney Int. 2012 Aug;82(3):261–269. doi: 10.1038/ki.2011.384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Tangri N, Kitsios GD, Inker LA, et al. Risk prediction models for patients with chronic kidney disease: a systematic review. Ann Intern Med. 2013 Apr 16;158(8):596–603. doi: 10.7326/0003-4819-158-8-201304160-00004. [DOI] [PubMed] [Google Scholar]
  • 48.van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate Imputation by Chained Equations in R. 2011;45(3):67. 2011 2011-12-12. [Google Scholar]
  • 49.Hinkle DE, Wiersma W, Jurs SG. Applied statistics for the behavioral sciences. 5th. Boston: Houghton Mifflin; 2003. [Google Scholar]
  • 50.Mukaka MM. A guide to appropriate use of Correlation coefficient in medical research. Malawi Medical Journal : The Journal of Medical Association of Malawi. 2012;24(3):69–71. [PMC free article] [PubMed] [Google Scholar]
  • 51.Viera AJ, Garrett JM. Understanding interobserver agreement: the kappa statistic. Family medicine. 2005 May;37(5):360–363. [PubMed] [Google Scholar]
  • 52.R: A language and environment for statistical computing. R Foundation for Statistical Computing; 2013. [computer program] [Google Scholar]
  • 53.DeSalvo KB, Bloser N, Reynolds K, He J, Muntner P. prediction with a single genera. A meta-analysis. Journal of general internal medicine. 2006 Mar;21(3):267–275. doi: 10.1111/j.1525-1497.2005.00291.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 54.Glare P, Virik K, Jones M, et al. A systematic review of physicians’ survival predictions in terminally ill cancer patients. BMJ : British Medical Journal. 2003;327(7408):195–195. doi: 10.1136/bmj.327.7408.195. 06/12/accepted. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 55.Christakis NA, Lamont EB. Extent and determinants of error in physicians’ prognoses in terminally ill patients: prospective cohort study. The Western journal of medicine. 2000 May;172(5):310–313. doi: 10.1136/ewjm.172.5.310. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Barrett BJ, Parfrey PS, Morgan J, et al. Prediction of early death in end-stage renal disease patients starting dialysis. Am J Kidney Dis. 1997 Feb;29(2):214–222. doi: 10.1016/s0272-6386(97)90032-9. [DOI] [PubMed] [Google Scholar]
  • 57.Jassal SV, Watson D. Dialysis in late life: benefit or burden. Clin J Am Soc Nephrol. 2009 Dec;4(12):2008–2012. doi: 10.2215/CJN.04610709. [DOI] [PubMed] [Google Scholar]
  • 58.Abdel-Kader K, Myaskovsky L, Karpov I, et al. Individual quality of life in chronic kidney disease: influence of age and dialysis modality. Clin J Am Soc Nephrol. 2009 Apr;4(4):711–718. doi: 10.2215/CJN.05191008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59.Verberne WR, Geers ABMT, Jellema WT, Vincent HH, van Delden JJM, Bos WJW. Comparative Survival among Older Adults with Advanced Kidney Disease Managed Conservatively Versus with Dialysis. Clinical Journal of the American Society of Nephrology. 2016 Mar 17; doi: 10.2215/CJN.07510715. 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Schell JO, Arnold RM. What are the Ill Effects of Chronic Dialysis?: The Benefits of a Second Conversation: How Experience Influences Decision-Making. Semin Dial. 2013 Nov 22; doi: 10.1111/sdi.12166. [DOI] [PubMed] [Google Scholar]
  • 61.United States Renal Data System. 2015 USRDS annual data report: Epidemiology of kidney disease in the United States. Bethesda, MD: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2015. 2015. [Google Scholar]
  • 62.Davey A, Elias MF, Robbins MA, Seliger SL, Dore GA. Decline in renal functioning is associated with longitudinal decline in global cognitive functioning, abstract reasoning and verbal memory. Nephrology Dialysis Transplantation. 2013 Jul 1;28(7):1810–1819. doi: 10.1093/ndt/gfs470. 2013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Etgen T, Chonchol M, Forstl H, Sander D. Chronic kidney disease and cognitive impairment: a systematic review and meta-analysis. Am J Nephrol. 2012;35(5):474–482. doi: 10.1159/000338135. [DOI] [PubMed] [Google Scholar]
  • 64.Renal Physicians Association. Shared decision-making in the appropriate initiation of and withdrawal from dialysis. Rockville, MD: Renal Physicians Association; 2010. [Google Scholar]
  • 65.Golper TA, Schreiber MJ. The course of therapy: Changing the paradigm. In: Fadem S, editor. Issues in Dialysis. New York: Nova Science Publishers, Inc; 2013. pp. 35–47. [Google Scholar]
  • 66.Davison SN, Levin A, Moss AH, et al. Executive summary of the KDIGO Controversies Conference on Supportive Care in Chronic Kidney Disease: developing a roadmap to improving quality care. Kidney Int. 2015 Apr 29; doi: 10.1038/ki.2015.110. 2015 Sep;88(3):447-59. [DOI] [PubMed] [Google Scholar]
  • 67.Moss AH. Revised Dialysis Clinical Practice Guideline Promotes More Informed Decision-Making. Clinical Journal of the American Society of Nephrology. 2010 Dec 1;5(12):2380–2383. doi: 10.2215/CJN.07170810. 2010. [DOI] [PubMed] [Google Scholar]
  • 68.Bakitas M, Lyons KD, Hegel MT, et al. Effects of a palliative care intervention on clinical outcomes in patients with advanced cancer: the Project ENABLE II randomized controlled trial. JAMA. 2009 Aug 19;302(7):741–749. doi: 10.1001/jama.2009.1198. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Temel JS, Greer JA, Muzikansky A, et al. Early Palliative Care for Patients with Metastatic Non-Small-Cell Lung Cancer. New England Journal of Medicine. 2010;363(8):733–742. doi: 10.1056/NEJMoa1000678. [DOI] [PubMed] [Google Scholar]
  • 70.Wright AA, Zhang B, Ray A, et al. ASsociations between end-of-life discussions, patient mental health, medical care near death, and caregiver bereavement adjustment. JAMA. 2008;300(14):1665–1673. doi: 10.1001/jama.300.14.1665. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Billings JA, Bernacki R. Strategic Targeting of Advance Care Planning Interventions: The Goldilocks Phenomenon. JAMA internal medicine. 2014 Feb 3; doi: 10.1001/jamainternmed.2013.14384. [DOI] [PubMed] [Google Scholar]
  • 72.Couchoud C, Labeeuw M, Moranne O, et al. A clinical score to predict 6-month prognosis in elderly patients starting dialysis for end-stage renal disease. Nephrology Dialysis Transplantation. 2009;24(5):1553–1561. doi: 10.1093/ndt/gfn698. 12/18. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Thamer M, Kaufman JS, Zhang Y, Zhang Q, Cotter DJ, Bang H. Predicting Early Death Among Elderly Dialysis Patients: Development and Validation of a Risk Score to Assist Shared Decision Making for Dialysis Initiation. American Journal of Kidney Diseases. 66(6):1024–1032. doi: 10.1053/j.ajkd.2015.05.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.van Loon IN, Wouters TR, Boereboom FTJ, Bots ML, Verhaar MC, Hamaker ME. The Relevance of Geriatric Impairments in Patients Starting Dialysis: A Systematic Review. Clinical Journal of the American Society of Nephrology. 2016 Apr 26; doi: 10.2215/CJN.06660615. 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 75.Bowling CB, Booth JN, Safford M, et al. Nondisease-Specific Problems and All-Cause Mortality in the REasons for Geographic and Racial Differences in Stroke (REGARDS) Study. Journal of the American Geriatrics Society. 2013;61(5):739–746. doi: 10.1111/jgs.12214. 04/25. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS841878-supplement-1.pdf (136.5KB, pdf)
2
NIHMS841878-supplement-2.pdf (129.8KB, pdf)
3
NIHMS841878-supplement-3.pdf (119.8KB, pdf)
4
NIHMS841878-supplement-4.pdf (181.9KB, pdf)
5
NIHMS841878-supplement-5.pdf (121.8KB, pdf)

RESOURCES