iChoose Kidney: a Clinical Decision Aid for Kidney Transplantation vs. Dialysis Treatment

Abstract

Background

Despite a significant survival advantage of kidney transplantation compared to dialysis, nearly one-third of end stage renal disease (ESRD) patients are not educated about kidney transplantation as a treatment option at the time of ESRD diagnosis. Access to individualized, evidence-based prognostic information is needed to facilitate and encourage shared decision making about the clinical implications of whether to pursue transplantation or long-term dialysis.

Study Design

We utilized a national cohort of incident ESRD patients in the U.S. Renal Data System surveillance registry from 2005 to 2011 to develop and validate prediction models for risk of 1- and 3-year mortality among dialysis vs. kidney transplantation. Using these data, we developed a mobile clinical decision aid that provides estimates of risks of death and survival on dialysis compared to kidney transplantation.

Results

Factors included in the mortality risk prediction models for dialysis and transplantation included age, race/ethnicity; dialysis vintage, and comorbidities, including diabetes, hypertension, cardiovascular disease, and low albumin. Among the validation cohorts, the discriminatory ability of the model for 3-year mortality was moderate (c-statistic = 0.7047 [95% CI: 0.7029-0.7065] for dialysis and 0.7015 [95% CI: 0.6875-0.7155] for transplant). We used these risk prediction models to develop an electronic, user-friendly, mobile (iPad, iPhone, and website) clinical decision aid called iChoose Kidney.

Conclusions

The use of a mobile clinical decision aid comparing individualized mortality risk estimates for dialysis vs. transplantation could enhance communication between ESRD patients and their clinicians when making decisions about treatment options.

Introduction

Patients with advanced chronic kidney disease (CKD) or end stage renal disease (ESRD) have two primary treatment options: long-term dialysis or kidney transplantation [1]. For most ESRD patients, transplantation is the treatment of choice because of longer survival, better quality of life, and lower hospitalization rates compared to dialysis [2-4]. However, the relative survival advantage associated with transplantation varies substantially depending on individual characteristics, including age, race, or comorbidities [4]. Despite overwhelming evidence to support transplantation for certain ESRD patients [5], information is rarely used by nephrologists to calculate a patient's specific, individualized prognosis [6]. The current standard of care is to communicate average, population-based, non-tailored prognosis estimates to individual ESRD patients, if these estimates are communicated at all [7]. The majority of patients want information about life expectancy, and yet more than 90% of ESRD patients have never discussed with their physicians how much time they have left to live [8]. Among CKD stage 3-5 patients surveyed in a multicenter cohort study, 35% of patients reported no knowledge of any treatment modality for ESRD, and 56% had no knowledge about transplantation.

The Centers for Medicare & Medicaid Services (CMS) Conditions for Coverage for dialysis units mandate that ESRD patients must be educated about all treatment modalities, including transplantation, referred for transplantation as appropriate, and subsequently followed up at least annually [9]. Despite these regulations, many dialysis patients in the US lack the necessary information about how to get a transplant [10-17]. Nearly one-third of ESRD patients are reportedly not informed of transplantation as a treatment option at the start of ESRD, and uninformed patients have a 53% lower rate of transplant [18]. Even when patients choose transplantation, it is unclear whether the decision is a shared decision between both patient and provider [19]; further, the timing and comprehensiveness of the information about ESRD treatment modalities is unclear [20].

Providing access to evidence-based prognostic information to the provider and patient is essential to ensuring patient understanding prior to the decision to pursue transplantation or initiate dialysis and may be a means of encouraging a higher proportion of ESRD patients to consider transplantation as a therapeutic option. According to a Cochrane review of 55 randomized trials, clinical decision aids can increase shared decision-making by improving patients’ knowledge of treatment options and increasing involvement in informed, value-based decisions [21]. Providing access to evidence-based prognostic information to both the provider and patient that is communicated through a dual-process medical decision making theory [22] is essential to ensuring patient understanding prior to the decision to pursue transplantation or initiate dialysis and may be a means of increasing the proportion of incident ESRD patients who are informed of transplantation.

Prior studies have described predictive models of mortality for ESRD patients and transplant recipients. However, to our knowledge, a simple model utilizing nationally-representative ESRD surveillance data at the time of ESRD start to compare mortality estimates for dialysis vs. kidney transplantation has not yet been translated into a simple, usable clinical decision aid that providers could use to calculate individualized survival by treatment modality. In this paper, we describe the development and validation of several risk prediction models for mortality of patients on dialysis and for patients with a kidney transplant among a nationally representative cohort of U.S. ESRD patients and the translation of these models into a clinical decision aid called iChoose Kidney.

Materials and Methods

Development and Validation of iChoose Kidney Models

The United States Renal Data System (USRDS) is a national data system that collects, analyzes, and distributes information about ESRD in the United States. The USRDS collaborates with members of Centers for Medicare & Medicaid Services (CMS), the United Network for Organ Sharing (UNOS), and the ESRD networks, sharing datasets and actively working to improve the accuracy of ESRD patient information. Among 740,933 incident, adult ESRD patients in the national cohort of United States Renal Data System (USRDS) surveillance data from January 1, 2005 through September 30, 2010, with follow-up through September 30, 2011, we excluded patients less than 18 years (n=6,688) or older than 100 (n=43), patients with multiple or previous organ transplants (n=1,804), and patients who resided outside of the 50 US states or District of Columbia were excluded (n=10,779). Finally, patients missing race (n=77) or sex (n=12), and transplant recipients with unknown donor type (n=191) were also excluded. Thus, there were a total of N=721,339 patients considered in the analysis, including n=663,860 dialysis patients and n=57,479 transplant patients. The transplant cohort included both preemptive (no dialysis) patients and patients who were on dialysis and then received a transplant, since the majority (76.1%) of transplanted patients receive dialysis as their first treatment modality. Of the patients who received a transplant, 47.4% received a living donor (LD) transplant and 52.6% received a deceased donor (DD) transplant (Figure 1).

Figure 1.

Study Population and Exclusion Criteria for Predictive Models for Mortality at ESRD Start

Study Variables

The study outcome was death due to any cause (yes/no) within the relevant time period (one or three years following ESRD diagnosis). Covariates for predictive modeling included characteristics available at the time of ESRD start from the CMS 2728 Medical Evidence Report. We examined demographic information (sex, age at incident ESRD, race, ethnicity, ESRD etiology, body mass index (BMI), pre-ESRD nephrology care), socioeconomic status (health insurance at time of ESRD, clinical lab values as proxy for access to care – erythropoietin use prior to ESRD, serum albumin, and serum hemoglobin), time on dialysis (for transplant models), region of the country (West, South, Midwest, East) and comorbidities from the CMS 2728 form. Comorbidities included cardiovascular disease (history of congestive heart failure, atherosclerotic heart disease, cerebrovascular disease, peripheral vascular disease, or other cardiac disease), hypertension, diabetes, tobacco use, drug abuse, and cancer. Nephrologists and transplant surgeons helped determine categories and functional forms of variables, based on a combination of model fit but also on practical and convenience aspects of collecting data in real-time for use in a clinical decision aid. We examined age as a continuous variable and considered model fit based on various forms of age, including linear, polynomial, and log-transformed. Race was categorized as white, black, or other race (76.6% Asian, 20.2% American Indian, 3.1% Multi-racial). Ethnicity was categorized as Hispanic or non-Hispanic.

Statistical Analysis

The surveillance data from USRDS used in this study were virtually 100% complete with no loss to follow-up. Logistic regression models were used to predict 3-year risk of death for dialysis patients vs. transplant patients. Data from the cohort of dialysis (N=663,860) and transplant (N=57,479) patients were each randomly divided into 50% training and 50% validation datasets. In the training cohorts, we examined the associations between select patient characteristics and mortality. Univariate analysis was performed on our initial list of predictor variables that were known to be associated with mortality based on prior literature and available at the time of ESRD start. All variables associated with the outcome in univariate analyses (p<0.1) were then included in a multivariable logistic regression model, and backwards elimination was performed based on statistical significance of variables (p<0.1) and model discrimination (concordance or c-statistics). Variables that were statistically significant in the model, but did not increase the c-statistic (area under the curve), were not included to create a more parsimonious model.

Predictive accuracy of the model was assessed using the validation data sets by the use of the c-statistic of the associated receiver operating characteristic (ROC) curve, which estimates the probability of concordance between the observed number of deaths and the predicted number of deaths based on the model. Model calibration was assessed by comparing the observed and expected number of deaths for each model. Statistical significance for predictive model calibration is typically assessed with the use of Hosmer-Lemeshow statistic. However, this test statistic is not recommended for sample sizes larger than 25,000 patients because the test is over-powered and likely to produce statistically significant results [23]. Thus, we examined the absolute concordance (% agreement) between observed and expected number of deaths.

Modeling Sensitivity Analyses

We conducted a subanalysis where ‘time on dialysis’ was excluded from transplant models, since this covariate may not be relevant to patients at the start of ESRD but is an important influence on the absolute risk of mortality following transplantation. To maintain consistency with how mortality rates are reported in USRDS, we conducted subanalyses examining models where patients who died in the first 90 days were excluded from models. Because there is inherent selection bias between dialysis patients and transplant patients, we conducted sensitivity analyses on patients who were waitlisted only. We also examined models by year of entry into the cohort to ensure that the model was relevant to patients across time. While main analyses were conducted as “intention to treat,” we also conducted sensitivity analyses excluding patients in the transplant cohort that had graft loss (10.5% of patients in transplant cohort). Finally, while the follow-up time was short (one or three years) for main analyses, we also conducted time to event analyses instead of logistic models to determine if main effect estimates or c-statistics were similar.

Development of the Shared Clinical Decision Aid

We used the following equation to translate model coefficients for 1-year and 3-year mortality estimates into an individualized, estimated risk of mortality $\widehat{Risk}=\frac{e^{{\widehat{\beta }}_0+{\widehat{\beta }}_1{X}_1+\cdots +{\widehat{\beta }}_i{X}_i}}{1+e^{{\widehat{\beta }}_0+{\widehat{\beta }}_1{X}_1+\cdots +{\widehat{\beta }}_i{X}_i}}$ Where ${\widehat{\beta }}_1{X}_1+\dots {\widehat{\beta }}_{\mathrm{i}}{X}_i$ represents the sum of the individual's risk factors (risk score) and ${\widehat{\beta }}_0$ is the estimated Y-intercept (or baseline risk). An example, including baseline risks (intercepts) is included in the Supplementary Appendix (Figure S1).

The iChoose Kidney clinical decision aid is a mobile [24] and web [25] application that calculates an ESRD patient's individualized risk of mortality of staying on dialysis compared to getting a kidney transplant (Figure 2). The purpose of the iChoose Kidney clinical decision aid is to enhance the communication between providers and patients about their individualized treatment outcomes when making decisions between dialysis or a kidney transplant. Best practices for the development of a clinical decision aid were used in the development of the tool [22, 26, 27], such as presenting absolute and relative probabilities and the use of visual aids and plain language.

Figure 2.

Screenshots of iChoose Kidney Decision Aid (iPad application), Version 3.0

We also followed standard protocols for designing and evaluating the clinical decision aid in clinical practice, adapted from Coulter et al [28]. We first created a multidisciplinary steering committee consisting of clinical experts (e.g. nephrologist, transplant surgeon), a behavioral scientist, an epidemiologist, ESRD patients and patient family members, and a medical illustrator to define the scope, purpose, and target audience of the clinical decision aid. An initial prototype was designed and developed into the alpha version of iChoose Kidney (version 1.0). Several phases of usability testing were conducted, including among ESRD patients in a dialysis facility (phase 1 – version 1.0), and among patients and transplant nephrologist and surgeon providers at a large urban transplant center (phase 2-version 2.0).

Satisfaction with the tool was generally high during usability testing, where patients described the tool as “interesting and informative” and “very user friendly,” while providers believed the tool facilitated conversations about LD vs. DD transplantation. However, we incorporated several physician and patient suggestions into version 3.0 of iChoose Kidney decision aid, including adding survival estimates (in addition to mortality), allowing for the emailing of patient results, creating a website version of the tool, and enlarging and lightening text for patient viewing (Figure 2). A multicenter randomized control trial is currently in progress to determine the impact of iChoose Kidney on knowledge of treatment outcomes and on decisional conflict during patient evaluation for kidney transplantation.

Ethical Considerations

This study was approved by the Emory University Institutional Review Board (IRB protocol #00059703).

Results

Patient Population

Patient demographic and clinical characteristics of dialysis and transplant recipients from the USRDS national surveillance data (2005-2011) are presented in Table 1. Patients in the dialysis training (N=331,930) and validation (N=331,930) cohorts and patients in the transplant training (N=28,740) and validation (N=28,739) cohorts were similar. Patients in the dialysis cohort were more likely than patients in the transplant cohort to be older, African American vs. white, female vs. male, from the southern region, have higher BMIs, and have Medicaid or Medicare vs. employer-based insurance. Patients in the dialysis cohort were also more likely to have comorbidities (diabetes, hypertension, cardiovascular disease, cancer), smoke, and have lower albumin and hemoglobin values. Patients in the transplant cohort were more likely to have increased access to a nephrologist prior to ESRD and receive pre-dialysis erythropoiesis-stimulating agents than patients in the dialysis cohort.

Table 1.

Select Baseline Patient Characteristics of Dialysis and Transplant Cohorts among Derivation and Validation Cohorts of the United States Renal Data System: 2005-2011

	Dialysis Cohort		Transplant Cohort
	Training Cohort N=331,930	Validation Cohort N=331,930	Training Cohort N=28,740	Validation Cohort N=28,739
Patient Demographics
Age, Mean (SD), years	64.1 (14.9)	64.0 (14.9)	49.3 (13.6)	49.4 (13.5)
Age Category, N (%), years
20-39	22,136 (6.7)	22,371 (6.7)	7,055 (24.5)	6,954 (24.2)
40-49	33,373 (10.1)	33,445 (10.1)	6,409 (22.3)	6,510 (22.7)
50-59	63,296 (19.1)	63,318 (19.1)	7,941 (27.6)	7,953 (27.7)
60-69	79,974 (24.1)	80,288 (24.2)	5,897 (20.5)	5,796 (20.2)
70-85	133,151 (40.1)	132,508 (39.9)	1,438 (5.0)	1,526 (5.3)
Sex
Male	185,341 (55.8)	185,640 (55.9)	17,621 (31.3)	17,442 (60.7)
Female	146,589 (44.2)	146,290 (44.1)	11,119 (39.7)	11,297 (39.3)
Race
White	216,664 (65.3)	216,379 (65.2)	20,363 (70.9)	28,488 (71.3)
Black	97,080 (29.3)	97,231 (29.3)	5,714 (19.9)	5,580 (19.4)
Other	18,186 (5.5)	18,320 (5.5)	2,663 (9.3)	2,671 (9.3)
Ethnicity
Hispanic	42,763 (12.9)	42,933 (12.9)	3,331 (11.6)	3,236 (11.3)
Non-Hispanic	289,167 (87.1)	288,997 (87.1)	25,409 (88.4)	25,503 (88.7)
Residential Region
West	70,985 (21.4)	71,059 (21.4)	5,699 (19.8)	5,749 (20.0)
Midwest	170,952 (21.4)	70,059 (21.1)	7,308 (25.4)	7,211 (25.1)
South	112,107 (33.7)	112,479 (33.9)	8,034 (28.0)	7,932 (27.6)
East	77,886 (23.5)	78,333 (23.6)	7,699 (26.8)	7,847 (27.3)
Health Insurance Coverage*
Medicaid	85,772 (25.8)	85,776 (25.8)	3,411 (11.9)	3,488 (12.1)
Medicare	119,900 (36.1)	119,392 (40.0)	3,600 (12.5)	3,637 (12.7)
Employer Group	74,373 (22.4)	74,595 (22.5)	16,810 (58.5)	16,798 (58.4)
Other coverage	22,989 (6.9)	23,094 (7.0)	2,933 (10.2)	2,867 (9.98)
No coverage	25,627 (7.7)	25,878 (7.8)	1,878 (6.5)	1,854 (6.45)
Missing	3,269 (0.98)	3,195 (0.96)	108 (0.38)	95 (0.33)
Patient Clinical Characteristics
Cause of ESRD, N (%)
Diabetes	152,039 (45.8)	152,868 (46.1)	9,168 (31.9)	9,111 (31.7)
Hypertension	97,716 (29.4)	97,291 (29.3)	5,128 (17.8)	5,237 (18.2)
Glomerulonephritis	18,626 (5.6)	18,628 (5.6)	5,862 (20.4)	5,787 (20.1)
Other	63,549 (19.2)	63,143 (19.0)	8,582 (29.9)	8,604 (29.9)
Time on Dialysis
0-6 months	------	------	3,449 (15.8)	3,499 (16.0)
6-12 months	------	------	3,901 (17.9)	3,739 (17.1)
>12 months	------	------	14,497 (66.4)	14652 (66.9)
BMI > 35 kg/m2	62,155 (18.7)	62,544 (18.8)	3,415 (11.9)	3,441 (12.0)
Tobacco use	21,512 (6.5)	21,441 (6.5)	1,033 (3.6)	1,047 (3.6)
Cardiovascular Disease	191,118 (57.6)	190,679 (57.5)	6,365 (22.2)	6,441 (22.4)
Congestive Heart Failure	113,837 (34.3)	113337 (34.1)	2,404 (8.4)	2,439 (8.5)
History of Cancer	26,122 (7.9)	26,036 (7.8)	714 (2.5)	684 (2.4)
Serum Albumin < 3.5 g/dL	168,649 (50.8)	169,111 (50.9)	9,115 (31.7)	9,007 (31.3)
Hemoglobin < 10 g/dL	227,192 (68.5)	227,304 (68.5)	15,252 (53.1)	15,424 (53.7)
Pre-dialysis ESA	85,054 (25.6)	85,256 (25.7)	10,150 (35.3)	10,233 (35.6)
Access to Nephrologist Prior to ESRD
Yes	117,518 (53.5)	177,086 (53.3)	20,924 (72.8)	20,799 (72.4)
No	98,642 (29.7)	98,581 (29.7)	3,712 (12.9)	3,769 (13.1)
Unknown	55,770 (16.89)	56,263 (16.9)	4,104 (14.3)	4,171 (14.5)

BMI, body mass index; ESA, erythropoietin-stimulating agent.

The proportion of deaths in the dialysis training cohort was similar to the proportion of deaths in the validation cohort (40.0% in both cohorts). Similarly, the proportion of deaths in the transplant training cohort was the same as the proportion number of deaths in the validation cohort (4.5% in both cohorts) over the three year period.

Predictive Models

We used the dialysis and transplantation training cohorts from USRDS to construct multivariable logistic regression models for three-year mortality. The Odds Ratios and 95% CIs for the variables included in the final models for three year mortality on dialysis and transplantation are presented in Table 2. Our final model included age, race, ethnicity, sex, time on dialysis, and comorbidities, including diabetes, hypertension, cardiovascular disease, congestive heart failure, and low albumin (< 3.5 g/dL) (Table 2).

Table 2.

Multivariable Logistic Regression Analysis Results for 3-Year Mortality among the Dialysis Training Cohort (N=331,930) and Transplant Training Cohort (N=28,740)

	Dialysis N=331,930			Transplant N=28,740
Variable	Estimate (beta)	Standard Error	Odds Ratio (95% CI)	Estimate (beta)	Standard Error	Odds Ratio (95% CI)
Intercept	−2.8457	0.0227	-	−5.4292	0.1646	-
Sex (Female vs. Male)	0.0067	0.0076	1.01 (0.99-1.02)	−0.0475	0.0603	0.95 (0.85-1.07)
Age, years	0.0388	0.0003	1.04 (1.04-1.04)	0.0382	0.0025	1.04 (1.03-1.04)
Race
White	Reference	Reference	Reference	Reference	Reference	Reference
Black	−0.2990	0.0090	0.74 (0.73-0.76)	−0.0261	0.0733	0.97 (0.84-1.13)
Other	−0.6111	0.0179	0.54 (0.52-0.56)	−0.5080	0.1296	0.60 (0.47-0.78)
Hispanic Ethnicity (vs. non-Hispanic)	−0.5253	0.0124	0.59 (0.58-0.61)	−0.4034	0.1071	0.67 (0.54-0.82)
Cardiovascular Disease	0.4737	0.0081	1.61 (1.58-1.63)	0.3369	0.0633	1.47 (1.30-1.66)
Hypertension	−0.4696	0.0105	0.63 (0.61-0.64)	−0.2000	0.0799	0.84 (0.72-0.98)
Diabetes	0.0169	0.0078	1.02 (1.00-1.03)	0.4013	0.0600	1.54 (1.37-1.73)
Albumin < 3.5 g/dL	0.3619	0.0076	1.44 (1.42-1.46)	0.2102	0.0605	1.23 (1.15-1.46)
Time on Dialysis
0-6 months	-	-	-	Reference	Reference	Reference
6-12 months	-	-	-	0.1360	0.1015	1.15 (0.94-1.40)
>12 months	-	-	-	0.4906	0.0700	1.63 (1.42-1.87)

¹ Cardiovascular disease was defined as a history of congestive heart failure, atherosclerotic heart disease, other cardiac disease, cerebrovascular disease, or peripheral vascular disease as designated on the CMS-2728 form (item 17a, 17b, 17c, 17d, or 17e).

² Hypertension was defined as ‘history of hypertension’ at the time of ESRD start, as designated on the CMS-2728 form (item 17f)

³ Diabetes was defined as a history of diabetes, either with or without medications, as designated on the CMS-2728 form (item 17h, 17i, or 17j)

Prediction Model Discrimination and Performance

We performed internal validation of the risk prediction models for dialysis and transplantation at 3-years using the validation cohorts from USRDS. The discriminatory ability of the model for 3-year mortality was moderate (c-statistic = 0.7047 [95% CI: 0.7029-0.7065] for dialysis and for transplant (c-statistic = 0.7015 [95% CI: 0.6875-0.7155]) . The c-statistic was 0.6640 (95% CI: 0.6458-0.6822) for DD transplant and 0.7209 (95% CI: 0.6954-0.7463) for LD transplant. The c-statistics were similar among the training and validation datasets, indicating that the predictive models were generalizable among the national surveillance data of ESRD patients and kidney transplant recipients (Table 3).

Table 3.

Model Performance Measures for 3-Year Development and Validation Data Sets.

Models	C-Statistic (Area Under Curve) 95% CI
Dialysis Patients (N=663,860)
Development	0.7034 (0.7016-0.7052)
Validation	0.7047 (0.7029-0.7065)
Transplantation (n=57,479)
Development	0.6948 (0.6806-0.7090)
Validation	0.7015 (0.6875-0.7155)
Deceased Donor Transplant (n=30,228)
Development	0.6705 (0.6526-0.6884)
Validation	0.6640 (0.6458-0.6822)
Living Donor Transplant (n=27,251)
Development	0.7146 (0.6899-0.7392)
Validation	0.7209 (0.6954-0.7463)

Predictive Model Calibration

Figure 3a and Figure 3b show the observed vs. predicted probability of death in three years within the dialysis and transplantation validation cohort. The predicted and observed number of deaths was similar among the percentiles of predicted probability for dialysis patients (90% concordance or higher). For example, in the dialysis cohort, there were 3,402 observed deaths and 3,070 expected deaths among patients with the lowest risk of death (first decile), resulting in 90.3% concordance, and 28,534 observed deaths and 27,969 expected deaths among patients with the highest risk of death (10^th decile), resulting in 98.0% concordance (Figure 3a). Among the transplant cohort, the proportion of predicted vs. observed deaths was also similar (ranging from 87% concordance in decile 8 and 99% concordance in decile 10) (Figure 3b).

Figure 3a.

Observed and Expected Number of Deaths over 3 Years for Dialysis (n=663,860) Validation Cohort in Multivariable Predictive Model

Figure 3b.

Observed and Expected Number of Deaths over 3 Years for Transplant (n=57,479) Validation Cohort in Multivariable Predictive Model

Sensitivity Analyses

In time to event analyses, hazard ratios were similar to main models and c-statistics were unchanged. For dialysis models the c-statistic for the 3-year model was 0.6882 (95% CI: 0.6862-0.6901). For transplant models the c-statistic for the 3-year model was 0.7089 (95% CI: 0.6948-0.7230). Absolute and relative risks of mortality on transplant were lower when time on dialysis was excluded from models. We also calculated risk prediction models for 3-year mortality outcomes for DD vs. LD transplantation (Supplementary Appendix, Tables S1, S2) and for 1-year mortality outcomes by treatment modality (Tables S3-S6).

Relative risk estimates in the sensitivity analyses among the dialysis cohort of patients who were waitlisted only were very similar to those estimates including all patients (Table S7), and the c-statistic was similar. The absolute risks of mortality tended to be substantially lower among waitlisted patients compared to all ESRD patients (Table S7).

Further sensitivity analyses examining time trends among transplant and dialysis cohorts show similar c-statistics and effect estimates for model covariates across time periods (from 2000 to 2011). In the sensitivity analyses in the transplant cohort excluding patients with graft loss (10.5% of transplant cohort), effect estimates were very similar to those including patients with graft loss (Table S8).

Discussion

In this paper, we described the development and validation of models incorporated into a novel clinical decision aid – iChoose Kidney – to assist providers in discussing treatment options with patients at the start of ESRD. For ESRD patients, validated risk prediction models for mortality in ESRD [6, 29-36] have not yet been translated into simple, nationally representative ESRD patient data on dialysis vs. transplantation mortality risks in the clinical setting in the form of a decision aid to use at the time of ESRD start, or earlier. To our knowledge, this study is the first to describe simple, nationally representative risk prediction estimates for prognosis for ESRD patients at the time of diagnosis on dialysis vs. living or deceased donor kidney transplantation and translate these estimates into a novel decision tool to help patients and their providers objectively appreciate the implications of treatment choices.

The models utilized in iChoose Kidney are based upon more than 700,000 ESRD patients from a nationally representative surveillance dataset, and rely on patient and clinical data readily available to providers. Conversion of risk estimates into a simple, but novel, decision aid has several important clinical utilities, both for patients with kidney disease and clinicians who treat this patient population. A decision aid that provides absolute and relative risk of mortality on dialysis vs. kidney transplantation and allows comparisons of those risk estimates across patient factors provides physicians the ability to identify patients at the bedside who are at high absolute risk for mortality and contribute to better informed treatment choices for patients. The accessibility of the technology and the clinical data required, as well as patient-friendly graphical representation of results, provides an opportunity for shared-decision making for late-stage CKD patients or new ESRD patients in deciding whether to pursue long-term dialysis or kidney transplantation. The tool is specifically designed for the clinician provider (e.g. family physicians, nephrologists, transplant surgeons, nurses, social workers, and/or patient educators) to use with patients they believe may be reasonable candidates for transplantation. While the aid will be freely available and accessible via a website for patients to access themselves, we intend the use of this tool as a shared decision aid.

Compared to other widely-used risk prediction models in the field of transplantation, such as the Kidney Donor Risk Index to assess donor quality (c-statistic of 0.62), the risk scores we calculated for mortality on dialysis and kidney transplantation had slightly higher discrimination (c-statistic ranging from 0.66-0.72). While there are likely additional predictors that may improve the ability to discriminate mortality outcomes, there is a tradeoff between both easy availability of the data in a real-time setting, generalizability of the data to the national ESRD population, and simplicity of a decision aid in clinical practice. In a Cochrane review of decision aids, simpler vs. more detailed decision aids led to greater improvements in knowledge and more realistic expectations of the tool [37]. We chose to translate risk prediction estimates into the simplest model possible, while maintaining model discrimination and performance. Our risk prediction models only include age, race, ethnicity, sex, time on dialysis, three comorbid conditions and an indicator for low serum albumin. We expect that providers will have easy if not immediate access to each of the covariates included, which may increase the usability of the tool. Patient accessibility to these risk estimates is enhanced with the use of a decision aid that employs best practices for the communication of risks to patients, including simple language, visual aids, and presentation of both absolute and relative risks of mortality, which may aid patients and their healthcare providers in better understanding these risks[27].

The need to provide evidence-based information to patients regarding the relative benefits of different treatment options is important to ensure equal access to information about optimal treatment for patients. According to the United States Agency for Healthcare Research and Quality, disparities in health outcomes are due in part to differences in access to health care, provider biases, poor patient-provider communication, and poor health literacy [38]. Medical decision making theories suggest that patients comprehend risk through two constructs: verbatim and gist [22]. By providing patients numeric risk estimates (verbatim), and pictograph representation of risks (gist), iChoose Kidney provides the patients literal and bottom-line interpretations of their risk comparison. In addition, effective risk communication strategies must consider patients with varying degrees of health literacy, numeracy, and education levels to ensure the information provided is both relevant and understandable to patients from diverse backgrounds. Our usability testing among a lower socioeconomic status, lower health literacy population suggests that iChoose Kidney could be a valuable tool to communicate treatment risks to this patient population. We utilized best practices for developing a clinical decision aid by including a multidisciplinary stakeholder group of advisors from the start, and by obtaining direct user feedback to improve the decision aid. The acceptability and usability testing we conducted among patients and physicians helped to improve the iChoose Kidney decision aid. Feedback from both physicians and patients led to three iOS versions of the mobile decision aid [24], and the development of a website [25].

Shared decision making about the treatment options for ESRD should occur in the pre-ESRD CKD clinic, at the time of initiation of chronic dialysis and/or during a transplant evaluation for new ESRD patients. While preemptive transplantation offers optimal patient survival [39], few (<5%) patients are transplanted prior to needing dialysis. For patients who do chose transplantation as a treatment option, limiting the time on dialysis is an important potentially modifiable risk factor for post-transplant outcomes [40]. The failure of health care providers to discuss treatment options with late-stage CKD patients may be due to a variety of causes. One reason may be that physicians may not feel adequately trained to discuss transplant as an ESRD treatment option [41]. Disparities in who is educated about transplantation as a treatment option may also be related to lack of sufficient training to provide comprehensive information about transplantation to patients [42]. We expect that the use of evidence-based, patient-specific risk estimates may help improve physicians’ comfort level in discussing the benefits of transplantation or dialysis with patients, although this hypothesis should be tested in clinical practice. A multicenter, randomized study of the clinical effectiveness of the iChoose Kidney clinical decision aid to improve shared decision making about treatment options is ongoing (NCT02235571).

There are many other important decisions that patients with kidney disease must make with the help of their provider, including the decision to choose peritoneal dialysis vs. hemodialysis, whether to initiate and complete transplant evaluation workup, or whether to seek a transplant at one center vs. another [19]. In addition, mortality is not the only important outcome for patients. Quality of life may be more important than survival for some patients [43], and some patients may be content with their current life expectancy on dialysis [44].

There are a number of limitations to the iChoose Kidney decision aid. First, there is an inherent selection bias in the model comparison between dialysis vs. kidney transplantation, and it is not possible to examine who is ineligible for transplantation in these data. Patients in the USRDS database who are at the extremes of age and/or comorbidity burden but were transplanted likely represent a healthier, selected population compared to an ESRD patient in the USRDS database who did not get a transplant. Our observation that diabetes has a protective effect on mortality in the first year of dialysis (both in logistic and time to event models) may be one example of this potential survival bias, and although this observation has been reported in prior literature [33, 45, 46], it is counterintuitive. Thus, the use of these risk prediction models alone is not warranted to attempt to identify patients who may not be good transplant candidates. While the iChoose Kidney tool could compare cohorts of waitlisted dialysis patients to transplanted patients to reduce this selection bias, doing so would limit generalizability of dialysis mortality risk estimates for all ESRD patients, especially to those patients who ought to pursue transplantation but have not done so currently. Further, since evidence suggests that many of the patients who remain on dialysis are not educated or referred for transplantation, examining mortality of waitlisted dialysis patients only selects a population that may not be inclusive of all ESRD patients eligible for transplantation. Analyses examining waitlisted patients only showed similar risk estimates of covariates in the model, but substantial differences in absolute risks of mortality were observed when calculated among a cohort of all ESRD patients vs. only those who were waitlisted, where not surprisingly, waitlisted patients had lower absolute mortality compared to all ESRD patients. The risk prediction estimates utilized in iChoose Kidney are best applied among a cohort of patients not yet waitlisted for transplantation, since absolute estimates of mortality on dialysis may be overestimated among a population of highly selected patients who are already waitlisted for transplant. However, separate modeling estimates are available in the supplementary appendix for use in other populations, including among those waitlisted for transplantation. Second, there are likely a number of unmeasured predictors that influence mortality and are not captured in the risk prediction models utilized in the iChoose Kidney decision aid. We used USRDS data, but there are many other factors that are not captured in surveillance data that influence mortality for ESRD patients and transplant recipients. However, there is a tradeoff between the inclusion of many variables and the availability of obtaining this information in real-time to provide evidence-based risks for patients. We chose not to include transplant factors in transplant mortality models, since this tool is designed for use when these factors are unknown. Third, risk estimates are based on patient data captured around the time of ESRD start, with the exception of ‘time on dialysis.’ We intend the risk estimates from this tool to be used to guide patients and providers in understanding/discussing the patient's theoretical risk of dying on dialysis vs. transplantation assuming that both options were immediately available to the patient. We considered ‘time on dialysis’ as a covariate for transplantation models to allow providers and patients to develop an appreciation of the adverse influence of longer time on dialysis on mortality post-transplantation and since most patients must wait several years before receiving a transplant. Finally, although the risk prediction estimates were derived and validated in the U.S. population and thus are generalizable to all ESRD patients in the U.S., there may be geographic differences that are not reflected in these risk prediction estimates. Region-specific validation studies may be needed to capture some of the unmeasured regional factors unaccounted for in the risk prediction estimates. Risk estimates obtained from the models are population-based estimates only; caution must be used when applying risk prediction models to some patients, including those with substantial comorbidities (not captured in the risk prediction models), as well as patients at the extremes of age. Because demographics and outcomes change over time, it will be important to update risk prediction models that are reflected in the iChoose Kidney decision aid; in addition, longer-term predictions and graft survival estimates are planned for future iterations of the decision aid.

To our knowledge, this is the first study to use a nationally representative cohort of ESRD patients to generate risk prediction models of mortality on dialysis vs. kidney transplantation and translate these risk estimates into a novel shared decision making tool. The current standard of care is to communicate average, population-based, non-tailored prognosis estimates to individual ESRD patients, despite evidence that patients want information about treatment prognosis. The patient information needed to use these models is easily obtainable, and risk estimates are generalizable to ESRD patients across the nation. Future research should test the effectiveness of using individualized mortality risk estimates for patients with kidney disease in improving provider/patient shared decision making.

Abbreviations

BMI

Body Mass Index

CVD

Cardiovascular Disease

CKD

Chronic Kidney Disease

DD

Deceased Donor

ESRD

End Stage Renal Disease

LD

Living Donor

ROC

Receiver Operator Characteristic

USRDS

United States Renal Data System