Prediction of Risk of Death for Patients Starting Dialysis

Visual Abstract

Abstract

Background and objectives

Dialysis is a preference-sensitive decision where prognosis may play an important role. Although patients desire risk prediction, nephrologists are wary of sharing this information. We reviewed the performance of prognostic indices for patients starting dialysis to facilitate bedside translation.

Design, setting, participants, & measurements

Systematic review and meta-analysis following the PRISMA guidelines. We searched Ovid MEDLINE, Ovid Embase, Ovid Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, and Scopus for eligible studies of patients starting dialysis published from inception to December 31, 2018. Selection Criteria: Articles describing validated prognostic indices predicting mortality at the start of dialysis. We excluded studies limited to prevalent dialysis patients, AKI and studies excluding mortality in the first 1–3 months. Two reviewers independently screened abstracts, performed full text assessment of inclusion criteria and extracted: study design, setting, population demographics, index performance and risk of bias. Pre-planned random effects meta-analysis was performed stratified by index and predictive window to reduce heterogeneity.

Results

Of 12,132 articles screened and 214 reviewed in full text, 36 studies were included describing 32 prognostic indices. Predictive windows ranged from 3 months to 10 years, cohort sizes from 46 to 52,796. Meta-analysis showed discrimination area under the curve (AUC) of 0.71 (95% confidence interval, 0.69 to 073) with high heterogeneity (I²=99.12). Meta-analysis by index showed highest AUC for The Obi, Ivory, and Charlson comorbidity index (CCI)=0.74, also CCI was the most commonly used (ten studies). Other commonly used indices were Kahn-Wright index (eight studies, AUC 0.68), Hemmelgarn modification of the CCI (six studies, AUC 0.66) and REIN index (five studies, AUC 0.69). Of the indices, ten have been validated externally, 16 internally and nine were pre-existing validated indices. Limitations include heterogeneity and exclusion of large cohort studies in prevalent patients.

Conclusions

Several well validated indices with good discrimination are available for predicting survival at dialysis start.

Introduction

Dialysis is a lifesaving therapy for patients with ESKD, but its utility is increasingly questioned for frail and multimorbid patients (1,2), who experience rapid functional decline (3) and high symptom burden (4). Qualitative studies and surveys report suffering and decisional regret (5,6), yet patients are reluctant to discontinue dialysis treatments (7). Ideally, clinicians counseling patients with ESKD would present patients with all treatment options and likely outcomes before dialysis initiation (8) and use shared decision making focused on patients’ goals and preferences (8).

Prognostic information is an essential component of shared decision making (9), and dialysis patients desire this information (5,10). Nephrologists feel unprepared for these discussions (10,11) and avoid giving prognostic information even when asked (10). Consequently, attempts to help patients choose a treatment usually do not include prognostic information (12,13). As a result, patients may assent to dialysis without understanding the risks and benefits, a failure of both shared decision making and respect for patient autonomy (7,14). Both existing guidelines and the American Society of Nephrology Choosing Wisely campaign emphasize that individualized prognostic information should be included in the decision to initiate dialysis (15,16).

The objective of our study was to perform a systematic review and meta-analysis to identify validated prognostic indices that predict mortality for patients with ESKD starting dialysis. At this crucial decision point, there may be value in identifying patients at higher mortality risk for goals of care discussion and patients with better prognosis for kidney transplantation. A scoping review was done as part of the Kidney Disease Improving Global Outcomes (KDIGO) work on kidney palliative care but, to our knowledge, there has been no systematic review of the availability and performance of prognostic indices applied at dialysis start (17).

Materials and Methods

Protocol

We developed a protocol before the review was initiated, and incorporated feedback from content experts. We adhered to the recommendations made in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (18), except for registration of our protocol.

Eligibility Criteria

We included original research articles and observational studies that described validated prognostic indices for patients with incident ESKD, with the outcomes of patient death and at least 3 months of prognostication.

We excluded studies not specific to CKD or ESKD, studies that were limited to patients with AKI in hospital or intensive care unit, studies limited to prevalent dialysis patients, studies that excluded early death (usually first 3 months), and studies where an index was just used as a variable in a regression model. Letters, editorials, narrative reviews, commentaries, and case reports were excluded but used as sources for references.

Data Sources and Searches

A medical librarian (P.J.E.) searched Ovid MEDLINE, Ovid Embase, Ovid Central Register of Controlled Trials, Ovid Cochrane Database of Systematic Reviews, and Scopus from inception to December 31, 2018. No language restrictions were applied to the search strategy, which used controlled vocabulary and was supplemented with keywords (Supplemental Appendix 1). We also screened reference lists of included articles and included the list of prognostic articles identified by the KDIGO expert review (17).

Study Selection

All titles and abstracts were independently screened by two college and medical students (R.T.A., H.C., A.M., and A.M.M). Each full text was distributed to two coauthors (R.T.A., H.C., B.T., A.H., R.M.G.C., M.A.F., and M.E.W.) for independent determination of inclusion and data abstraction with at least one physician in each pair. The senior author (B.T.) adjudicated for disagreement. Articles in other languages were reviewed by at least one physician at our institution fluent in the language. Abstract authors were noted and monitored intermittently for subsequent publication. Reviewers were not masked to the authors, journals, or results of studies. At this step, the unweighted Cohen κ statistic was used for quantifying the chance-corrected agreement between reviewers.

Data Extraction

Pertinent data were extracted and assessed for quality with the use of standardized predefined forms (Supplemental Appendices 2 and 3), including study design, study setting and population, demographics of the development and validation cohorts.

Metrics of Model Performance

The primary outcome of interest for meta-analysis was discrimination as measured by the c-statistic or area under receiver operating characteristics curve (AUC). Data on calibration was also collected positive predictive values (PPVs), negative predictive values, calibration plots, Hosmer–Lemeshow goodness-of-fit test results, and chi-squared test results. Discrimination refers to the ability to distinguish between persons with a particular characteristic and those without. Calibration refers to the accuracy of a model’s predictions (19,20).

Quality Assessment, Risk of Bias, and Clinical Usefulness

Quality and risk of bias in each study were assessed with the checklist developed by Hayden et al. (21). Clinical usefulness was defined as a combination of utility and usability as described by Tangri et al. (22).

Missing Data

If studies were missing data or the outcomes were unclear, we contacted the authors by email. We received and incorporated an answer from one (23). Missing confidence intervals for the reported c-statistic were calculated where possible and otherwise imputed on the basis of similar studies, as per the Cochrane handbook (24).

Data Synthesis and Statistical Analyses

Weighted averages were calculated for demographic information. Clinical heterogeneity was assessed by determining whether the characteristics of participants, interventions, comparison group, and outcome measures were similar across studies. When data for more than three cohorts was available random effects meta-analysis and Forrest plots were conducted with pooled estimates and associated 95% confidence intervals. Meta-analysis was performed using OpenMeta Analyst software for meta‐analyses following a random‐effects model as described by DerSimonian‐Laird, to account for clinical and methodologic heterogeneity between studies (25). To quantify the degree of statistical heterogeneity, I² was used.

A preplanned subgroup analysis by prediction index and length of predictive window was performed. We also performed a subgroup analysis by whether the data were from development, internal validation or external validation cohorts. We performed the following additional sensitivity analyses, excluding studies that were not externally validated and excluding the three smallest studies (n<100) and the two outliers in terms of length of prediction (8 and 10 years).

Results

Summary Measures

After screening 12,132 titles, we retrieved 214 references for full-text review. We selected 36 articles describing 32 unique indices predicting mortality for patients with incident ESKD (Figure 1). For inclusion of the articles reviewed in full text, the κ score was 0.86. Table 1 shows the included articles, indices used, and performance metrics. Supplemental Table 1 shows a list of articles excluded after full text review.

Figure 1.

Flow diagram of the literature search and selection.

Table 1.

Included studies showing cohort size and characteristics, indices developed, and validated predictive window and performance

Study, Year	Index	Length	Discrimination	Calibration	Validation Type	Source of Cohort	Cohorta		Mortality
							Number (Female)	Age (SD)
Barrett et al., 1997 (55)	Foley	6 mo	NR	NR	N/A	Canadian academic medical centers HD (62%) and PD patients	822 (40.9%)	58.3 (15.4)	13.7%
Beddhu et al., 2002 (26)	CCI (Beddhu)	2 yr	NR	NR	Internal	US academic medical center PD patients	97 (58%)	53 (14)	21.6%
Chandna et al., 1999 (61)	Kahn-Wright	1 yr	0.70 (NR)	NR	N/A	UK dialysis unit HD (NR) and PD patients	292 (34%)	61.3 (15.8)	44%
Cheung et al., 2014 (23)	nCI-Liu	6 mo	0.62/0.65b	NR	N/A	USRDS HD patients	44,109 (46%)	77.2 (6.5)	23.3%
	REIN		0.63/0.66b
	HEC		0.65/0.68b
Cho et al., 2017 (27)	CCI	Avg f/u 47.1 mo	0.82	NR	Internal	Korean health insurance database PD patients	7606 (43.6%)a	54.5 (13.8)	39.5%
	mCCI-IPD		0.82				664 (40.1%)	48.4 (14.1)	12.3%
	mCCI-IHD		0.82				24,738 (40.5%)	57.9 (NR)
Couchoud et al., 2009 (33)	REIN	6 mo	0.70	Observed versus predicted H-L P=93	Internal and external	French ERSD all-comers Renal Epidemiology and Information Network HD (NR) and PD patients	2500 (39.6%)a	80.9 (4.1)	19%
	REIN	3 mo	0.74				1642 (40.5%)	81.1 (4.1)
	REIN	12 mo	0.68
Couchoud et al., 2015 (37)	Updated REIN	3 mo	0.76	NR	Internal	French dialysis centers REIN HD (NR) and PD patients	12,500 (39.6%)a	NR	10.5%
		3 mo	0.75				11,848 (NR)	NR
		3 mo	0.79				2490 (NR)	NR
Davies et al., 2002 (34)	Davies	2 yr	NR	NR	External	UK dialysis center Stoke PD study PD patients	303 (NR)	58.8 (NR)	35%
	Kahn-Wright	5 yr
Di Iorio et al., 2004 (28)	CCI (Di Iorio)	15 mo	NR	NR	Internal	Italian dialysis centers HD patients	515 (38.6%)	63.2 (15.4)	14.6%
Doi et al., 2015 (43)	Doi	1 yr	0.83	Observed versus predicted	Internal	Japanese dialysis centers HD patients	688 (33.4%)	69 (NR)	9%
Fernandez Lucas et al., 2007 (38)	ACPI	3 yr	0.75	NR	External	Spanish dialysis center HD (80%) and PD patients	304 (37%)	64 (15)	31%
	CCI		0.76
	CCI-H		0.71
Floege et al., 2015 (45)	AROii	3 mo	0.71	Observed versus predicted 1-yr R2= 0.94-yr R2= 0.98	Internal	Development: AROii study population Validation: DOPPS III study population HD patients	9277 (40.2%)a	64.4 (14.7)	17.6%
	AROii	1 yr	Range 0.75–0.76				4247 (NR)	NR
	AROii	2 yr	Range 0.73–0.75
	Kahn-Wright	2 yr	0.66
	nCI-Liu	2 yr	0.60
Foley et al., 1994 (39)	Foley	6 mo	0.90	Observed versus predicted histogram	External	Canadian hospital HD patients	325a(35%)	55 (NR)	22.4%
Fried et al., 2001 (35)	CCI	3 yr	NR	NR	External existing index	US dialysis centers PD patients	268 (35.1%)	56.0 (14.6)	20.5%
Fried et al., 2003 (36)	CCI	N/A	NR	NR	External existing indices	US dialysis centers PD patients	415 (37.8%)	55.9 (14.8)	NR
	Davies
	SF-36 MCS
	Karnofsky scale
Geddes et al., 2006 (44)	RBM	1 yr	NR	Observed versus expected bar graph	Internal	European dialysis centers ERA-EDTA Registry HD (86%) and PD patients	1139a (42.3%)	63.4 (13.5)	12.8%
	CCI-H	5 yr					1171 (39.8%)	60.4 (15.5)	27.3%
Gomez et al., 2015 (62)	CCI	6.5 yr	0.61	NR	Internal and external existing indices	Canadian academic medical center HD (78%) and PD patients	771 (38%)	62.6 (15.1)	40.3%
	CCI-H		0.62
Hemmelgarn et al., 2003 (29)	CCI-H	2 yr	0.73	NR	Internal and external modification of a validated scale subsequently validated in several studies	Canadian dialysis centers HD (77%) and HD patients	237 (36.7%)a	62.5 (15.2)	23.6%
	CCI		0.72
Inaguma et al., 2017 (46)	Barthel	NR	NR	NR	External existing Index	Japanese dialysis centers Aichi Cohort Study of the Prognosis in Patients Newly Initiated into Dialysis cohort HD patients	1496 (32.3%)	67.4 (13.0)	17.5%
Ivory et al., 2017 (63)	Ivory	6 mo	0.75	H-L P=21.9 calibration curve	Internal and in separate cohort	Australian and New Zealand dialysis centers Australia and New Zealand Dialysis and Transplant Registry; modality not reported	23,658 (40.1%)a	60.6 (14.9)	6.1%
	Ivory		0.71				32,664 (NR)
	Ivory		0.76				5518 (NR)
	REIN		0.70	H-L P=15.6			5518 (NR)
	Wagner		0.69	H-L P=6.1			5518 (NR)
	Wagner		0.73	H-L P=29.5			32,664(NR)
Kan et al., 2013 (42)	nCI-Liu	10 yr	0.91	NR	Internal and external	Taiwan National Health Insurance Research Database HD (96%) and HD patients	21,043 (55.6%)	NR	55.3%
	CCI		0.90
Khan et al., 1993 (31)	Kahn-Wright	2 yr	NR	NR	External modification of a validated scale subsequently validated in several studies	UK dialysis centers HD (NR) and PD patients	375 (42.9%)	53.9 (18)	35%
Khan et al., 1998 (64)	Kahn-Wright	2 yr	NR	NR	External existing index	UK dialysis centers HD (NR) and PD patients	1407 (40%)	55.6 (15.7)	NR
Lee et al., 2017 (65)	Multidimensional frailty core	Avg f/u 18 mo	NR	NR	External existing index	Korean dialysis center HD patients	46 (37%)	71.5 (NR)
Lopez Revuelta et al., 2004 (66)	SF-36 PCS	2 yr	NR	NR	External existing indices	Spanish hospitals HD (80%) and PD patients	318 (39.6%)	60.2 (14.6)	25%
	SF– 36 MCS
	Karnofsky scale
Ma et al., 2018 (67)	CCI	3 yr	NR	NR	External existing indices	Chinese dialysis unit PD patients	461 (47%)	57.7 (13.7)	49.9%
	CCI-H
	nCI-Liu
Marinovich et al., 2010 (68)	New index	1 yr	0.74	NR	New index not validated, external validation existing indices	Argentinian dialysis centers National Registry of the Instituto Centro Unico Cordinator de Ablación e Implante de Organos HD patients	5360 (33.8%)	58.9 (15.5)	20.4%
	CCI		0.70
	ACPI		0.68
	Kahn-Wright		0.67
	CCI-H		0.63
Mauri et al., 2008 (40)	RMRC	1 yr	0.78	Observed versus predicted H-L P=49	Internal and external	Spanish dialysis centers Catalan Renal Registry HD patients	3445 (37.8%)a	64.6 (14.4)	16.5%
							2283 (NR)
Nicolucci et al., 1992 (69)	ICED	1,2 and 5 yr	0.594		External existing index	Italian dialysis center HD (61%) and PD patients	255 (43.5%)	54 (NR)c	16%, 27% and 49% at 1, 2 and 5y
Obi et al., 2018 (49)	Obi low GFR	3,6,9 and 12 mo	0.71	Observed versus predicted	Internal and in separate cohorts	Development: US Veterans Affairs dialysis cohort; Validation: US health system; Data from USRDS HD (94%) and PD patients	15,142 (2%)a	69.4 (11.1)	20.0%
							7463 (1.9%)	68 (11)
	Obi high GFR		0.66				8888 (1%)a	72 (10)	39.8%
							4385 (1%)	72 (10)
	Obi low GFR men only		0.77				1872 (0%)	64 (13)
	Obi low GFR women only		0.74				1491 (100%)	65 (15)
	Obi high GFR men only		0.71				570 (0%)	66 (13)
	Obi high GFR women only		0.67				351 (100%)	65 (14)
Otero-López et al., 2012 (70)	RMRC score	1 yr	0.59	Observed versus predicted	External existing indices	Spanish hospital HD (89%) and PD patients	63 (40%)	80.4 (3.9)	20.6%
	REIN	6 mo	0. 68	NR					27%
Peeters et al., 2016 (30)	Abbreviated REIN	3 mo	0.74	NR	Internal modified existing index	Belgian dialysis centers Neederlandstalige Vereniging voor Nefrologie HD (87%) and PD patients	2679 (43.3%)	67.6 (14.3)	19.6%
		6 mo	0.74
		12 mo	0.74
Postorino et al., 2007 (50)	NYHA	Avg f/u 41 mo	0.74	NR	Existing indices	Italian dialysis centers Calabrian Registry of Uremia, Dialysis, and Transplantation HD (82%) and PD patients	1322 (41.6%)	60.8 (NR)	41.7%
	ESRD-SI (RDSS)		0.70
	Kahn-Wright		0.69
Thamer et al., 2015 (47)	Thamer, Simple	6 mo	0.72	NR	Internal	USRDS HD (96%) and PD patients	52,796 (46.2%)a	76.9 (6.5)	20.3%
	Comprehensive
	REIN		0.68				16,645 (46.9%)	76.8 (NR)
	Wagner		0.73
Van Manen et al., 2002 (71)	New Index	2 yr	0.72	NR	Internal	Dutch dialysis centers HD (61%) and PD patients	616 (39%)	59.3 (15.1)	25%
	Kahn-Wright		0.73
	Davis		0.74				589 (40%)	59.1 (15.8)
	CCI		0.72
Wick et al., 2017 (48)	Wick	6 mo	NR	H-L P=2	Internal	Canadian dialysis centers Northern and Southern Alberta Renal Program registries HD (85%) and PD patients	2199 (39.2%)	75.2 (6.5)	17.1%

NR, not reported; N/A, Not applicable; HD, Hemodialysis; PD, peritoneal dialysis; CCI, Charlson comorbidity index; nCI, new comorbidity index; USRDS, US Renal Data System; REIN, Renal Epidemiology and Information Network; HEC, hospice eligibility criteria; Avg f/u, Average follow up; mCCI-IPD, Modified Charlson comorbidity index - Intermittent Peritoneal Dialysis; mCCI-IHD, Modified Charlson comorbidity index - Intermittent Hemodialysis; H-L, Hosmer–Lemeshow; ACPI, age-comorbidity prognostic index; CCI-H, Charlson comorbidity index Hemmelgarn modification; AROii, Analyzing data, Recognizing excellence, and Optimizing outcomes; DOPPS, Dialysis Outcomes and Practice Patterns Study; SF-36, Short Form 36; MCS, mental component score; RBM, Rule-Based Model; ERA-EDTA, European Renal Association - European Dialysis and Transplant Association; PCS, physical component score; RMRC, Registre de Malalts Renals de Catalunya; ICED, Index of Coexisting Disease; NYHA, New York Heart Association; ESRD-SI, End-Stage Renal Disease Severity Index; RDSS, Renal Disease Severity Score.

^aTraining/development cohort.

^bAdjusted for age.

^cMedian age.

Synthesis of Results

The 36 studies included a total of 299,373 patients. Sixteen papers developed 18 new indices or modified an existing index in 140,108 patients with an average cohort size of 3.106 (range 97–52,796) and a weighted average age of 69.7 years. Of the 18 new indices, 11 had internal validation and nine compared the performance of their new index to existing ones (Table 1). Five studies modified existing indices to better fit their population or available data (26–30). Thirteen studies were solely validation studies existing indices, often more than one (Table 1). Predicted survival time ranged from 3 months to 10 years. The most commonly used indices were the Charlson comorbidity index (CCI) (ten studies) and its modifications (eight studies), the Renal Epidemiology and Information Network (REIN) score (five studies) and modifications (two studies), and the Kahn-Wright Index (eight studies) (31–33). Five studies were limited to patients on peritoneal dialysis (26,27,34–36). Of the 32 indices reported, nine were preexisting indices from other disease states (Barthel, CCI, Hospice criteria, Karnofsky, New York Heart Association Heart failure score, Frailty score, Short Form 36 physical and mental component scores), nine have been validated externally (age-comorbidity prognostic index, CCI Hemmelgarn, Davies, Foley, Kahn-Wright Index, new comorbidity index (NCI)-Liu, REIN Renalts, Registre Malalts Renals de Catalunya (RMRC), Wagner) (29,31,32,34,37–42), 13 only have internal validation (CCI-Beddhu, CCI-Di Iorio, Doi, mCCI-IPD, mCCI-IHD, REIN updated, Analyzing Data, Recognizing Excellence, and Optimizing Outcomes (AROii), REIN abbreviated, Rule Based Model, Thamer Simple and Comprehensive, Van Manen, Wick), and one was validated in a separate population cohort in the original development study (Ivory). Only five indices in four papers had a c-statistic >0.8, indicating excellent discrimination (27,39,42,43) and most indices had c-statistics between 0.70 and 0.79, suggesting good discrimination; other studies had worse discrimination or did not report it. Eleven studies reported calibration (most by reporting a Hosmer–Lemeshow P value >0.05), but two studies chose a cutoff and calculated sensitivity and specificity of 63%–84% or PPV of 80%–93% for a certain cutoff and age group (23,44), and one study published a nomogram to guide the choice of a clinically meaningful cutoff (45).

Meta-Analysis

Preplanned meta-analysis of discrimination by index showed that the c-statistic or AUC was highest for the Obi low GFR (0.74; 95% CI, 0.70 to 0.78) and Ivory indices (0.74; 95% CI, 0.71 to 0.77), followed by the CCI (0.74; 95% CI, 0.65 to 0.83) and new comorbidity index (nCI) (0.72; 95% CI, 0.57 to 0.87) (Figure 2). Heterogeneity was high (I²=98.89; P<0.001) for the overall meta-analysis with a c-statistic of 0.71 (0.69 to 0.73). Figure 3 shows comparative meta-analyses of the existing predictors by predictive window showing the best discrimination in the >5 year (AUC, 0.79; CI, 0.72, 0.86) and 3-month windows (AUC, 0.76; 95% CI, 0.74, 0.77). Meta-analysis not surprisingly showed the highest AUC for development cohorts (0.75; 95% CI, 0.77 to 0.78) followed by internal validation (0.74; 95% CI, 0.72 to 0.76), and indices performed worst in external validation (0.7; 95% CI, 0.68 to 0.71) (Supplemental Figure 1). When analyzed by cohort size the largest studies had the best discrimination (Supplemental Figure 2). A separate meta-analysis was done for the five papers that had cohorts with average age >75 years showing worse discrimination (0.69; 0.67 to 0.70), with I²=91.57 (Supplemental Figure 3). Heterogeneity did not decrease when we limited this analysis to only the four papers reporting the performance of the REIN index.

Figure 2.

Meta-analysis of discrimination score of mortality prediction by index for those where three or more results were available. Weights are from random-effects analysis. 95% CI, 95% confidence interval; K-W, Kahn-Wright.

Figure 3.

Meta-analysis by predictive window, categorized to within 3 months, within 6 months, 1–5 years, or >5 years. Weights are from random-effects analysis. 95% CI, 95% confidence interval; ACPI, age-comorbidity prognostic index; AROii, Analyzing Data, Recognizing Excellence, and Optimizing Outcomes; ESRD-SI, End stage Renal Disease Severity Index; HEC, hospice eligibility criteria; K-W, Kahn-Wright; mCCI-IHD, Modified Charlson comorbidity index (CCI) - Intermittent Hemodialysis; mCCI-IPD, Modified CCI Intermittent Peritoneal Dialysis; NYHA, New York Heart Association; RMRC, Registre de Malalts Renals de Catalunya.

Heterogeneity was high for all meta-analyses (Figures 2 and 3, Supplemental Figures 1 and 2). Sources of clinical heterogeneity included variable sample size, demographics, distribution of dialysis type, predictive window, predictive grouping, and predictors used in the different indices (Supplemental Tables 2 and 3, Table 1). Sensitivity analyses omitting the three studies with sample sizes of <100 patients and the two studies with the longest predictive window (8 and 10 years) did not alter results or decrease heterogeneity.

We attempted a meta-analysis of calibration, however only one paper gave the slope for the calibration curve, most of the papers only presented the observed versus expected data in graphs and none gave confidence intervals.

Variables in Death Prediction Indices

The variables or risk predictors used by the prediction indices varied between the indices (Supplemental Tables 2 and 3). Most included age, only two included sex and one included race (41). There was variability in included comorbidities, some including severity weighting. Functional status was used in five studies (33,40,43,45,46). Socioeconomic variables were not used as predictors in any of the studies. Clinical context was included in some indices, i.e., unplanned dialysis initiation (33), temporary vascular access (45), complicated uremia (39), number of hospital days (28), and ventilator dependency (39). Only one index used the type of primary kidney disease as a variable (40).

Risk of Bias Across Studies

Most studies had at least a moderate risk of bias, limiting the level of certainty in the estimates (Table 2). Most indices omitted important risk factors, such as race and socioeconomic status, raising concerns about confounding in the majority of the studies.

Table 2.

Risk of bias of the included studies

Indices Evaluated	Study Participationa	Study Attritionb	Prognostic Factor Measurementc	Outcome Measurementd	Confounding Measurement and Accounte	Statistical Analysesf
Foley	g	g	h	h	i	h
CCI (Beddhu)	g	h	h	g	–	–
Kahn-Wright	g	–	g	g	h	g
nCI-Liu	g	g	g	g	h	g
REIN
HEC
CCI mCCI-IPD mCCI-IHD	g	g	g	g	h	g
REIN score	g	h	g	h	–	g
REIN	g	g	g	g	h	g
Kahn-Wright	g	–	h	g	?	g
Davies
CCI-H
CCI (Di Iorio)	g	h	h	g	–	–
Doi	–	–	g	h	h	g
CCI	g	h	h	g	–	g
ACPI
AROii	g	–	g	g	h	g
Foley	g	g	h	h	–	g
CCI	g	–	h	h	–	g
CCI Davies	g	?	—	g	h	g
CCI-H Rule-based model	g	h	h	h	g	g
CCI	–	–	h	–	g	g
CCI-H
CCI-H	g	g	h	h	–	g
CCI
Barthel	g	g	g	g	–	g
Ivory	g	g	g	g	h	g
REIN
Wagner
CCI	g	–	g	g	–	g
Kahn-Wright	g	?	h	g	?	g
Kahn-Wright	g	?	h	g	?	g
Multifactorial Frailty Score	g	g	g	g	—	g
SF-36 PCS	—	h	g	g	—	g
SF-36 MCS
Karnofsky scale
CCI	_	?	h	h	h	g
CCI-H
nCI-Liu
CCI	g	g	h	h	–	g
ACPI
Kahn-Wright
CCI-H
RMRC score	g	h	h	h	g	g
ICED	h	?	g	g	h	g
Obi low GFR	g	g	g	g	h	g
Obi high GFR
REIN score	g	g	h	g	–	g
RMRC score
REIN	g	?	h	g	?	g
NYHA class	g	h	–	g	g	g
ESRD-SI (RDSS)
Kahn-Wright
Simple	g	h	g	g	g	g
Comprehensive
REIN
Wagner
Kahn-Wright	g	–	h	g	h	g
Davies
CCI
Wick	g	g	g	g	h	g

CCI, Charlson comorbidity index; –, does not meet low risk of bias criteria; nCI, new comorbidity index; REIN, Renal Epidemiology and Information Network; HEC, hospice eligibility criteria; mCCI-IPD, Modified CCI Intermittent Peritoneal Dialysis; mCCI-IHD, Modified CCI Intermittent Hemodialysis; ?, unknown; CCI-H, Charlson comorbidity index (Hemmelgarn);ACPI, age-comorbidity prognostic index; AROii, Analyzing data, Recognizing excellence, and Optimizing outcomes; SF-36, Short Form 36; PCS, physical component score; MCS, mental component score; RMRC, Registre de Malalts Renals de Catalunya; ICED, Index of Coexisting Disease, NYHA, New York Heart Association; ESRD-SI, End-Stage Renal Disease Severity Index; RDSS, Renal Disease Severity Score.

^aThe study sample represents the population of interest on key characteristics sufficient to limit potential bias to the results.

^bLoss to follow-up (from sample to study population) is not associated with key characteristics sufficient to limit potential bias (i.e., the study data adequately represent the sample).

^cThe prognostic factor of interest is adequately measured in study participants to sufficiently limit potential bias.

^dThe outcomes of interest are adequately measured in study participants to sufficiently limit potential bias.

^eImportant potential confounders are appropriately accounted for, limiting potential bias with respect to the prognostic factor of interest.

^fThe statistical analysis is appropriate for the design of the study, limiting potential for presentation of invalid results.

^gMeets low risk of bias criteria.

^hUnclear; ? unknown/NR.

ⁱDoes not meet low risk of bias criteria.

Clinical Utility and Usability of the Included Studies

Clinical utility and usability was limited for many of the indices. The age-comorbidity prognostic index; Analyzing Data, Recognizing Excellence, and Optimizing Outcomes, Doi, Foley, Kahn-Wright, REIN, Updated REIN, Thamer, and Wick indices provided risk categories and/or mortality percentiles for easy translation (31,33,37,39,43,45,47,48). Geddes et al. (44) reported calibration for the CCI predicting 1- and 5-year mortality (PPV, 78.7% and 79.4%; negative predictive value, 40.8% and 70.4%; and likelihood ratio, 1.1 and 7.0, respectively) and defined survival-based cutoffs for the Hemmelgarn index. Other studies have provided cutoffs for the nCI (42); and for the nCI, REIN, and Hospice Eligibility Criteria indices (23). In the study by Cheung et al. on elderly patients on dialysis, a REIN score of ≥9 had a specificity of 80%–92% for predicting death (23). The 3-, 6-, and 12-month mortality rates associated with different REIN index scores have also been reported (30). Others reported less useful data.

One of the studies referred to an online calculators (www.DialysisScore.com) and one gave a nomogram to help with bedside translation (30,49). The CCI can be calculated with several online calculators, the updated REIN (https://qxmd.com/calculate/calculator_286/3-month-mortality-in-incident-elderly-esrd-patients), Thamer (http://www.pmidcalc.org/?sid=26123861&newtest=Y), and Obi (www.DialysisScore.com) are available online (47).

Discussion

Our systematic review identified several well validated indices for predicting survival of incident ESKD that show promise for bedside translation. The CCI was most commonly used and delivered the best and most consistent discrimination performance. Calibration was rarely reported and not consistently enough to allow for meta-analysis. There was a lack of information on predictive performance of specific score cutoffs and risk thresholds that could inform clinical practice. A third of the tools lacked external validation and statistical heterogeneity was very high, suggesting lack of reproducibility and/or transportability as well as case-mix effects. Clinical heterogeneity was also substantial in the chosen prognostic variables, prognostic window, and risk cutoffs and there was significant risk of bias. Few scores were easy to calculate (31,33,38,39,50). The indices were developed and validated in cohorts that underrepresented patients of races and ethnicity other than white and Asian, affecting generalizability.

Implications for Practice

Dialysis is a prime example of a preference-sensitive decision with a strong technological and moral imperative to treat despite clinical equipoise regarding the balance of risks and benefits. Treatment burden is high and strong arguments have been made for a more palliative approach to dialysis treatment of patients who have a limited life expectancy (51,52). However, patients with good prognosis stand to benefit from proactive vascular access placement and consideration of preemptive kidney transplantation (8).

Dialysis patients also receive more aggressive end-of-life care than most other patient groups, and hospice is underused (53,54). A major barrier to earlier hospice transition is that most hospices require patients to discontinue dialysis because it is not covered under the Medicare hospice benefit. We found eight indices that could predict 6-month mortality (23,28,33,39,42,47,48,55) and five that could predict 3-month mortality (30,37,44,45,47) with dialysis treatment. For the oldest patients, a REIN index score of ≥9 predicted over 70% mortality at 6 months (specificity, 80%–92%) (23). Currently, Medicare requires >50% expected 6-month mortality, thus a strong argument could be made for a policy change allowing for concurrent Medicare reimbursement for dialysis and hospice for this high-risk subgroup of patients to promote earlier hospice enrollment and avoid costly and burdensome hospitalizations and interventions at the end of life.

Implications for Research

We did not find any prospective studies testing the performance or utility of these indices in clinical practice. We also did not identify any studies testing the effect of mortality prediction on improving shared decision making. Patients, surrogates, and clinicians tend to be overly optimistic in the face of a poor prognosis (10,56,57). We do not know if providing patients with prognostic information at dialysis start will help patients develop more insight into their prognosis. Clinicians may hesitate to use prognostic indices with discrimination of only 0.7–0.75 and feel that those are not accurate enough. Furthermore, even a near-perfect model for mortality, validated externally in numerous populations and settings, would provide no evidence for its utility in improving shared decision making at the bedside. Indeed, qualitative research suggests that a major barrier for clinicians to discussing prognosis with patients is fear of taking away hope and giving patients a wrong estimate (58). In addition these conversations are emotionally charged and can be uncomfortable for clinicians who often lack communication skills training to facilitate these conversations (58). More research is needed to determine the value of risk prediction to patients and their family members and clinicians, the utility and usability of existing tools at the bedside, and their effect on shared decision making and other patient important outcomes.

Strengths and Limitations

Our study has limitations. Most importantly, our decision to limit the review to patients on incident dialysis and exclude studies that did not report early mortality resulted in the exclusion of many large studies utilizing the prevalent population in the US Renal Data System and other large population databases. This was deliberate, as we feel strongly that excluding early mortality presents a biased survival estimate to starting dialysis and that there is a need to present a more “realistic” estimate for patients trying to decide whether to start dialysis. We did not use the recommended screening filters suggested by the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies (CHARMS) checklist as our work started before this publication (59). However, we worked closely with a librarian with decades of experience in systematic reviews (P.J.E.) who developed a very sensitive search strategy, yielding 12,132 titles for screening and followed the PRISMA statement. We included all prognostic indices that we could confirm had been validated, either internally during index development or subsequent to the index development, but we cannot be certain that none were missed. Several of the studies did not report key performance data, prohibiting direct comparison. Our review did not evaluate the use of the surprise question, which has been shown to perform well in prevalent dialysis patients. (60) In fact, clinician intuition outperformed the index in one of the included studies (55). Strengths include a comprehensive and sensitive search strategy that covered several databases without language restrictions. We used a validated tool to assess bias and a previously defined assessment of clinical usability. We also focused our review on the time of starting dialysis, when these decisions are most likely to add value to therapeutic recommendations and shared decision making.

In conclusion, prognosis is a key element of shared decision making (9), yet poorly integrated into dialysis decision making. Several validated indices with good prognostic ability are available to predict mortality at the start of dialysis. Little attention has been paid to bedside translation of these indices to determine the value of sharing prognostic information with patients. Further research is needed to understand how best to utilize these indices in patient care to support shared decision making.

Disclosures

Dr. Tangri reports grants and personal fees from Astra Zeneca Inc. related to SGLT2 inhibitors and hyperkalemia treatments, personal fees from Otsuka Inc. related to tolvaptan and ADPKD, personal fees from Janssen related to SGLT2 inhibitors, personal fees from Boehringer Ingelheim and Eli Lilly related to SGLT2 inhibitors, and grants and personal and other fees, including consulting fees and stock options related to veverimer, from Tricida Inc., outside the submitted work. Dr. Anderson, Dr. Bellolio, Dr. Cleek, Dr. Erwin, Dr. Feely, Dr. Giddings Connolly, Dr. Hart, Dr. Hickson, Dr. Majzoub, Dr. Mayukha, Dr. Pajouhi, Dr. Giddings Connolly, Dr. Thorsteinsdottir, and Dr. Wilson have nothing to disclose.

Funding

This project was supported by a Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery award (to Thorsteinsdottir and Hickson); the Norman S. Coplon Extramural Grant Program of Satellite Healthcare, a not-for-profit renal care provider (to Thorsteinsdottir and Hickson); National Institute on Aging grant K23AG051679 (to Thorsteinsdottir); and National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases grant K23 DK109134 (to Hickson). Additional support was provided by the National Center for Advancing Translational Sciences grant UL1 TR000135.

Supplemental Material

This article contains the following supplemental material online at http://cjasn.asnjournals.org/lookup/suppl/doi:10.2215/CJN.00050119/-/DCSupplemental.

Supplemental Appendix 1. Search strategy.

Supplemental Appendix 2. Data abstraction form.

Supplemental Appendix 3. Data Abstraction form: study quality.

Supplemental Table 1. Papers excluded after full-text review.

Supplemental Table 2. Overview of predictive variables by index.

Supplemental Table 3. Predictive variables and scoring of included indices.

Supplemental Figure 1. Meta-analysis by validation status.

Supplemental Figure 2. Meta-analysis by cohort size.

Supplemental Figure 3. Meta-analysis for cohorts with >75 years mean age.