Prospective Validation of Prediction Model for Kidney Discard

Abstract

Background

Many kidneys are discarded every year, with 3631 kidneys discarded in 2016 alone. Identifying kidneys at high risk of discard could facilitate “rescue” allocation to centers more likely to transplant them. The Probability of Delay or Discard (PODD) model was developed to identify marginal kidneys at risk of discard or delayed allocation beyond 36 hours of cold ischemia time. However, PODD has not been prospectively validated, and patterns of discard may have changed following policy changes such as the introduction of Kidney Donor Profile Index and implementation of the Kidney Allocation System (KAS).

Methods

We prospectively validated the PODD model using SRTR data in the KAS era (1/1/15-3/1/18). C statistic was calculated to assess accuracy in predicting kidney discard. We assessed clustering in center’s utilization of kidneys with PODD>0.6 (“high-PODD”) using Gini coefficients. Using match run data 1/1/15-12/31/16, we examined distribution of these high-PODD kidneys offered to centers that never accepted a high-PODD kidney.

Results

PODD predicted discard accurately under KAS (C-statistic=0.87). Compared to utilization of low-PODD kidneys (Gini coefficient = 0.41), utilization of high-PODD kidneys was clustered more tightly among a few centers (Gini coefficient = 0.84 with >60% of centers never transplanted a high-PODD kidneys). In total 11,684 offers (35.0% of all high-PODD offers) were made to centers that never accepted a high-PODD kidney.

Conclusions

Prioritizing allocation of high-PODD kidneys to centers that are more likely to transplant them might help reduce kidney discard.

Introduction

Increasing number of new patients were added to the kidney waiting list every year; between 2015 and 2016, this increase was 1.9%.¹ Based on Organ Procurement and Transplantation Network data as of April 2018, over 95 000 candidates were on the kidney waiting list. The increasing demand for organs far exceeded the supply of available kidneys, with only 19 849 waitlist candidates received a kidney transplant in 2017. Because of this organ shortage, over 16% candidates waited over 5 years for a transplant, and the number of patients who were removed for death or deteriorating medical conditions reached above 9000 in 2016.² Moreover, the number of living donor transplants has declined in the past decade, exacerbating the shortage.^2,3

Despite the severe organ shortage, and numerous studies showing a survival benefit from transplantation even with marginal kidneys^4–7, the discard rate rose from 2127 (14.9%) in 2006 to 3631 (20%) in 2016, and remained high following the introduction of Kidney Donor Profile Index (KDPI) and the implementation of the Kidney Allocation System (KAS).^1–3,7–12 The increase in discard was mostly explained by the underlying changes in donor characteristics.⁷ In Europe and the United Kingdom, allocation schemes such as a “rescue” allocation policy and Kidney Fast-Track Scheme (KFTS) have been enacted to reduce unnecessary kidney discard.^13–15 In the United States, although there has been discussion of policies that might reduce discard, none currently exist.¹⁶ Identifying kidneys at high risk of discard could help facilitate development of a “rescue” allocation policy.

In an effort to help operationalize prioritized allocation of kidneys more likely to be discarded, in 2010 we created a model to predict the probability that a kidney will be discarded or transplanted with cold ischemia time (CIT) exceeding 36 hours (Probability of Discard/Delay, PODD).¹⁷ This model seemed to function well, but has never been validated, and seems more needed than ever in light of markedly increased discard rates. Additionally, the mechanisms of kidney discard might have changed under KAS, further emphasizing the need for validation. Finally, the benefit of a “rescue” allocation policy requires that there be variation in center-level willingness to transplant an organ labeled likely to be discarded by the specified index. That is, there must be variation in rates of acceptance of high-PODD kidneys which can be used to direct such kidneys to a center that might be more willing to transplant them.^17,18

We conducted a national registry study in the KAS era to prospectively validate the accuracy of PODD in predicting kidney discard. Additionally, we compared PODD with other scores including a reduced PODD score (PODD with terms for biopsy, glomerular sclerosis, and use of machine perfusion removed); KDPI; and a model developed by Marrero et al for predicting risk of discard.¹⁹ Finally, we characterized the variation in utilization of hard-to-place kidneys predicted by PODD among centers.

Materials and Methods

Data Source

This study used data from the Scientific Registry of Transplant Recipients (SRTR). The SRTR data system includes data on all donors, wait-listed candidates, and transplant recipients in the US, submitted by the members of the Organ Procurement and Transplantation Network (OPTN), and has been described elsewhere.²⁰ The Health Resources and Services Administration (HRSA), U.S. Department of Health and Human Services provides oversight to the activities of the OPTN and SRTR contractors. Our study was classified by the Johns Hopkins Institutional Review Board as “exempt” – not human subjects research (NA_00042871).

PODD

We included 50 207 adult deceased donor kidneys recovered for transplantation between January 1, 2015 and March 1, 2018. Informed by the previous study, double/en-bloc kidneys were considered as 1 observation.¹⁷ For each recovered kidney, we calculated the PODD score as described in the previous study.¹⁷ Predictors of PODD included female, age spline at 40 years, blood type AB, donor Body Mass Index (BMI, kg/m²), BMI spline at 23, cancer, smoking, hypertension, history of myocardial infarction, diabetes, diabetes over 10 years, diabetes requiring insulin, donation after cardiac death (DCD), death by cerebrovascular accident (CVA), death by head trauma, human T lymphocyte virus (HTLV) positive, cytomegalovirus (CMV) positive, Hepatitis B core antibody (HBcAb), Hepatitis B core antibody and surface antigen (HBsAg), hepatitis C antibody (HCV), CDC high-risk donor, machine perfusion, glomerulosclerosis over 20%, and serum creatinine (mg/dL). Because this study is an external (temporal) validation of the previous study, we did not refit the PODD model, rather, we kept the coefficients of the original PODD score and calculated the probabilities of discard/delay.²¹ Candidate comorbidities were assumed to absent in the case of missing or unknown data. There was <0.1% missing data for continuous predictors for BMI and serum creatinine.

Comparing PODD to other Indices of Discard

For comparison, we tested the validity of several other indices in predicting discard. The Marrero score was calculated based on the odds ratios of the predictors and intercept described elsewhere.^19,21 Predictors included age over 50, biopsy of either kidney, CMV status, DCD, CVA, donor height (cm), donor weight (kg), tattoos, either kidney pumped, HBcAb, HBsAg, HCV, history of cigarette use, history of diabetes, history of drug use, history of hypertension, terminal lab creatinine over 1.5 (mg/L), blood type, and donor race. KDPI was calculated using 2017 as the reference year.

Both PODD and Marrero score might have an advantage in predicting discard since they include factors that are not typically known at time of initial allocation (glomerulosclerosis, use of machine perfusion, and whether biopsy was done). We calculated reduced scores for PODD (r-PODD) and Marrero score (r-Marrero). For r-PODD, we fit the model with glomerulosclerosis and machine perfusion removed. In order to conduct a strict validation, we fit this model on the dataset used to fit PODD by Massie et al in 2010.¹⁷ We calculated r-PODD for our current study population using the coefficients from the model. For Marrero score, because of the lack of access to the original data, we calculated the reduced score (r-Marrero) with the coefficients for machine perfusion and biopsy removed. We examined the correlation between all the indices.

Accuracy in Predicting Discard

Area Under the Curve (AUC)

Because PODD was developed to predict discard/delay while Marrero score was developed to predict discard only, we examined the validation on the same outcome – kidney discard. The fit of each model in predicting discard was evaluated and compared using C statistics and the area under the receiver operating characteristic curve (AUC).^22–24

We calculated positive predictive value (PPV), negative predictive value (NPV), sensitivity, and specificity for each of the indices. Cutoffs were informed by our previous study: a stringent cutoff (0.6) was close to 90^th percentile of PODD.¹⁷ This was applied to PODD, r-PODD, Marrero score, and r-Marrero. While we aim to validate the discrimination of PODD, the calibration power of PODD might change over time. In other words, the proportion of kidneys that were more likely to be discarded for a certain cutoff might change. We also calculated the proportion of kidneys with PODD above other cutoffs (0.5 and 0.7), and the discard rate for kidneys with PODD above these cutoffs. We used 85% as the cutoff for high KDPI.^11,25 Because KDPI was not designed to predict discard, we also explored the observed probabilities of discard of kidneys with KDPI over 90% and at 100%.

Net Reclassification Improvement (NRI) Index

We quantified the value of each model using the net reclassification improvement (NRI) index.^26–28 In a perfect model, all kidneys discarded would be classified as high PODD and all kidneys transplanted would be classified as low PODD. The NRI index offers an intuitive way to measure improvement offered by one model over the other by describing how well the new model reclassifies subjects with events into high or low likelihood categories.²⁹ In this setting, kidney discard may be considered an event, and cutoffs for high versus low likelihood of discard must be selected for each model. To make KDPI comparable to the other indices, we converted KDPI to predicted probability of discard using logistic regression and included KDPI as the only term as a linear predictor. Thus, each KDPI score corresponded to a probability of kidney discard. We used both a stringent cutoff (0.6) and a permissive cutoff (0.5), which was close to the 80^th percentile of PODD, to evaluate the improvement from one index to the next.

Missing Data

For PODD and r-PODD, we first performed complete case analysis excluding the <0.1% missing data for BMI and serum creatinine. We then performed a sensitivity analysis using multiple imputation to handle the missing data. We conducted multiple imputation by chained equations (MICE) for 10 cycles.^30,31 We checked the normality of BMI, serum creatinine, and their log transformation. We specified linear regression model for the log transformation of BMI, and predictive mean matching (PMM) for serum creatinine.³² PMM is a linear regression approach appropriate for nonnormally distributed continuous variables. This approach imputes missing values by randomly selecting a nonmissing value from 1 of the 5 nearest neighbors after linear regression. In other words, it calculates the regression-predicted values for each observation; then fills in a value randomly chosen from the observed values from the 5 observations with the closest regression-predicted values. For all other indices, we performed complete case analysis.

Clustering of Kidney Utilization among Centers

We assessed offer acceptance using match run data that were available to us through December 2016. Using match run data for each kidney recovered between January 1, 2015 and December 31, 2016, we determined the offer acceptance of kidneys in each transplant center. We defined “offer acceptance” as acceptance and subsequent transplantation of an offered kidney for any candidate on the center’s match run list. That is to say, if a center declined the offer for 1 or more candidates but eventually accepted it for another candidates, we consider this an acceptance.³³ Kidneys that were accepted but eventually not transplanted were treated equivalent as being declined. Centers that received fewer than 100 kidney offers during the study period were excluded for statistical stability (38 out of 241 were excluded).

We defined a high-PODD kidney as one with a PODD >0.6 and a low-PODD kidney as one with a PODD ≤0.6. We examined the clustering among transplant centers in high-PODD versus low-PODD kidney utilization using Lorenz curves and the Gini coefficient.³⁴ Lorenz curves showed cumulative distribution of kidneys by center, sorted from those who transplanted the fewest kidneys to those who transplanted the most kidneys. The diagonal line represents perfectly equal utilization among the centers, the curves represent the actual distribution of utilization. Curves that are further away from the diagonal line represent more clustering. Gini coefficient is the area between the diagonal line and the curve. The Gini coefficient is a dimensionless measure of the degree of clustering of kidney utilization. It ranges from 0 to 1. A Gini coefficient of 0 means that all centers transplanted the same number of kidneys, while a Gini coefficient of 1 means that 1 center transplanted all kidneys.

In addition, we examined correlation between center practices over time. We identified the top 15 centers that transplanted the most high-PODD kidneys in 2015 and compared with the top 15 centers that transplanted the most of high-PODD kidneys in 2016. We assessed the correlation between these top 15 centers in 2015 and 2016. We repeatedly assessed the correlation between the top 20, 30, and 40 centers in 2015 and 2016.

Sensitivity Analysis

As a sensitivity analysis, we calculated Gini coefficients for high- and low-PODD kidneys for all centers (N=241).

Surplus Offers

Surplus offers were defined as offers (kidneys being offered to 1 or more recipients at a single transplant center) of high-PODD kidneys to centers that have never accepted a high-PODD kidney between January 1, 2015 and December 31, 2016.¹⁸ We plotted the distribution of surplus offers among centers that received at least 100 total offers during the study period.¹⁸

Statistical analysis

We used Wilcoxon rank-sum test to compare differences in continuous variables. We used Pearson’s chi-squared test to compare differences in categorical variables. We calculated Pearson’s correlation coefficients between different indices. The DeLong test was used to compare differences in AUCs.²² The Gini coefficient were calculated based on the INEQUAL7 module for Stata by Philippe Van Kerm. Lorenz curves were created based on sg30 module for Stata by Edward Whitehouse. We used Q-Q plot to check the normality of the distributions. We used Kappa coefficient to assess the correlation between the top 15, 20, 30, and 40 centers in different years.³⁵ Confidence intervals were reported as per the method of Louis and Zeger.³⁶ All analyses were performed using Stata/SE 14.1 for Windows (College Station, Texas).

Results

Characteristics of Donors

Of the 50 207 kidneys recovered during the study period, 10 381 (20.7%) were discarded (Table 1). Compared to kidneys that were transplanted, kidneys that were discarded came from donors with a higher median age (54 vs. 38 years, p<0.001) and body mass index (BMI, 28 vs. 27, p<0.001). Discarded kidneys were more likely to be from donors who were African American (17.5% vs. 14.1%, p<0.001), female (46.0% vs. 38.2%, p<0.001), DCD (20.2% vs. 18.8%, p<0.001), Extended Criteria Donor (ECD, 49.5% vs. 13.2%, p<0.001), died from CVA (45.3% vs. 25.7%, p<0.001), had a history of smoking (32.5% vs. 19.0%, p<0.001), hypertension (60.7% vs. 27.4%, p<0.001), diabetes (23.8% vs. 6.7%, p<0.001), myocardial infarction (7.7% vs. 2.4%, p<0.001), and a history of infectious disease (Table 1). Discarded kidney were less likely to be from donors who died of head trauma (14.8% vs. 32.6%) and be labeled CDC increased risk donors (21.5% vs. 25.1%, p<0.001). Discarded kidneys were less likely to be machine perfused (27.3% vs. 36.6%, p<0.001), and more likely to have sclerosis >20% (23.2% vs. 1.4%, p<0.001, Table 1).

Table 1.

Characteristics of donors of all kidneys recovered for transplantation in the United States 1/1/15-3/1/18.

	Transplanted	Discarded	p value
	N= 39 826	N=10 381
Age, median (IQR)	38 (27, 50)	54 (44, 62)	<0.001
Race/ethnicity			<0.001
White	68.18%	67.67%
African American	14.07%	17.46%
Hispanic	14.08%	11.25%
Asian	2.45%	2.75%
Other	1.22%	0.87%
Female	38.20%	46.00%	<0.001
BMI, median (IQR)	27.02 (23.55, 31.50)	28.47 (24.61, 33.41)	<0.001
Blood type			0.06
A	35.14%	34.06%
B	11.92%	12.26%
AB	3.33%	3.73%
O	49.61%	49.95%
Medical History
smoking	19.04%	32.46%	<0.001
hypertension	27.36%	60.69%	<0.001
diabetes	6.71%	23.81%	<0.001
diabetes >10 yrs	1.52%	8.30%	<0.001
Insulin-dependent diabetes	2.28%	9.42%	<0.001
myocardial infarction	2.44%	7.69%	<0.001
Donation Characteristics
DCD	18.76%	20.20%	<0.001
ECD	13.17%	49.49%	<0.001
Cause of death: CVA	25.73%	45.29%	<0.001
Cause of death: head trauma	32.56%	14.83%	<0.001
History of Infection
HTLV positive	0.03%	0.05%	0.3
CMV positive	60.72%	65.07%	<0.001
HBcAb positive	3.56%	7.42%	<0.001
HBsAg positive	0.04%	0.23%	<0.001
HCV positive	4.31%	11.21%	<0.001
IRD	25.09%	21.47%	<0.001
Machine Perfusion	36.62%	27.29%	<0.001
Biopsy done	49.79%	87.13%	<0.001
Sclerosis >20%	1.40%	23.22%	<0.001
Terminal serum creatinine, median (IQR)	0.93 (0.70, 1.32)	1.30 (0.90, 2.20)	<0.001

^*BMI: Body Mass Index; DCD: Donation after Cardiac Death; ECD: Extended Criteria Donor; CVA: Cerebrovascular accident; HTLV: Human T lymphotropic virus; CMV: cytomegalovirus; HBcAb: Hepatitis B Core Antibody; HBsAg: Hepatitis B Antigen; HCV: Hepatitis C Virus; IRD: Infectious Risk Donor

Correlation between indices

Predictors of each index were listed in Table 2. Correlation was good between PODD, r-PODD, Marrero score, and r-Marrero, with all the correlation coefficients between 0.8 and 1.0 (Table 3). KDPI was correlated less well with the other indices, with all correlation coefficients between 0.7 and 0.8 (Table 3).

Table 2. Predictors of indices.

PODD and Marrero indices were similar in donor demographic predictors including age, blood type, and BMI, though different functional forms were used. PODD had more predictors of comorbidities including cancer and myocardial infarction, and infection history including IRD and HTLV. Marrero score had additional predictors of drug use and tattoos. PODD included donor sex while Marrero score included donor race. Both PODD and Marrero score had predictors that are not known at the time of allocation including machine perfusion, biopsy, and glomerulosclerosis. These were excluded in the reduced indices.

Index	PODD	r-PODD	Marrero	r-Marrero
Original Outcome	Delay or discard	Delay or discard	Discard	Discard
Predictors	Female	Female
			Race category	Race category
	Age spline at 40	Age spline at 40	Age > 50	Age > 50
	Blood type AB	Blood type AB	Blood type category	Blood type category
	BMI	BMI	Height	Height
	BMI spline at 23	BMI spline at 23	Weight	Weight
	Smoking	Smoking	Smoking	Smoking
			Drug use	Drug use
			Tattoos	Tattoos
	Hypertension	Hypertension	Hypertension	Hypertension
	Diabetes	Diabetes	Diabetes	Diabetes
	Diabetes>10yrs	Diabetes>10yrs
	Insulin-dependent diabetes	Insulin-dependent diabetes
	Cancer	Cancer
	Myocardial infarction	Myocardial infarction
	Death by head trauma	Death by head trauma
	DCD	DCD	DCD	DCD
	CVA	CVA	CVA	CVA
	CMV	CMV	CMV	CMV
	HCV	HCV	HCV	HCV
	HBcAb	HBcAb	HBcAb	HBcAb
	HBcAg+HBsAg	HBcAg+HBsAg	HBsAg	HBsAg
	HTLV	HTLV
	IRD	IRD
	Machine perfusion		Machine perfusion
	Creatinine	Creatinine	Creatinine > 1.5	Creatinine > 1.5
	Sclerosis > 20%
			Biopsy

^*BMI: Body Mass Index; DCD: Donation after Cardiac Death; ECD: Extended Criteria Donor; CVA: Cerebrovascular accident; HTLV: Human T lymphotropic virus; CMV: cytomegalovirus; HBcAb: Hepatitis B Core Antibody; HBsAg: Hepatitis B Antigen; HCV: Hepatitis C Virus; IRD: Infectious Risk Donor

Table 3. Correlation between indices.

Correlation coefficients that are closer to 1 indicate good correlation while correlation coefficients that are closer to 0 indicate no correlation. Correlations between PODD, Marrero score, and the reduced scores were good; while KDPI was moderately correlated with the other indices.

	PODD	r-PODD	Marrero score	r-Marrero	KDPI
PODD	1.00
r-PODD	0.94	1.00
Marrero score	0.84	0.85	1.00
r-Marrero	0.81	0.85	0.95	1.00
KDPI	0.71	0.76	0.78	0.76	1.00

Accuracy in Predicting Discard

Of kidneys studied, 9.5% had PODD over 0.6, 13.0% had Marrero score over 0.6, 9.3% had r-PODD over 0.6, 8.2% had r-Marrero over 0.6, 12.7% had KDPI over 85% (Table 4). The observed probability of discard increased from 63.6% for kidneys with KDPI ≥85% to 70.5% for kidneys with KDPI ≥90%, and 92.0% for kidneys with KDPI of 100%. Among all the indices, PODD had the highest positive predictive value of 78.5%. This means if a kidney has a PODD greater than 0.6, then there is a 78.5% chance that this kidney will be discarded. The simpler r-PODD still had a higher positive predictive value than the Marrero score, r-Marrero, or KDPI (Table 4). When looking at other PODD cutoffs, 13.7% had PODD over 0.5, 71.7% of these kidneys were discarded; 6.2% had PODD over 0.7, 85.2% of these kidneys were discarded. AUC was the largest for PODD (0.87), followed by r-PODD (0.85), Marrero (0.84), r-Marrero (0.82), and KDPI (0.82) (all p<.001) (Figure 1). Using 0.6 as the cutoff, Net Reclassification Improvement (NRI) was 15.6% from KDPI to PODD, and 4.1% from Marrero to PODD. NRI was 11.4% from KDPI to r-PODD, and 11.9% from r-Marrero to r-PODD. Using 0.5 as the cutoff, NRI was 9.3% from KDPI to PODD, and 4.2% from Marrero to PODD. NRI was 7.0% from KDPI to r-PODD, and 14.7% from r-Marrero to r-PODD. In summary, by all 3 metrics – AUC, NRI, and positive predictive value – PODD predicted discard most accurately. After multiple imputation, the conclusions stayed the same.

Table 4. Evaluation of Indices.

Positive predictive value (PPV) is the probability of discard for a kidney with a score that is above the threshold. For example, 78.5% of kidneys with PODD>0.6 were discarded. Negative predictive value (NPV) is the probability that a kidney with a score that is below the threshold is being transplanted. Sensitivity is the probability that a discarded kidney has a score that is above the threshold. Specificity is the probability that a transplanted kidney has a score that is below the threshold. PODD and r-PODD had the highest PPV; over 70% of kidneys flagged as high-PODD or high r-PODD were discarded.

Indice	% of all kidneys	PPV (%)	NPV (%)	Sensitivity (%)	Specificity (%)
PODD>0.6	9.46	78.45	85.36	35.90	97.43
Marrero>0.6	13.04	55.51	84.55	35.02	92.69
r-PODD>0.6	9.33	71.98	84.60	32.46	96.71
r-Marrero>0.6	8.23	52.19	82.15	20.77	95.04
KDPI≥85%	12.71	63.60	85.57	39.08	94.17

Figure 1. ROC curves showing accuracy in predicting discard.

Curves closer to the diagonal line represent worse accuracy while curves closer to the upper left represent better accuracy in predicting discard. PODD have greater accuracy in predicting discard than all other scores (p<0.001).

Clustering of Kidney Utilization among Centers

There were 203 centers that received at least 100 kidney offers over the study period. The median number of low-PODD kidneys transplanted by centers was 66 (range=2-375, IQR=36-119). However, the median number of high-PODD kidneys transplanted by centers during the study period was 0 (IQR=0-1, range=0-54). There was markedly greater clustering among centers in use of high-PODD kidneys as compared to use of low-PODD kidneys (Gini=0.84 vs 0.41, Figure 2). In total 122 (60.1%) centers did not use a single high-PODD kidney, and the top 15 centers (ranked according to number of high-PODD kidneys accepted) used the majority (63.0%) of high-PODD kidneys (Figure 2). When including all 241 centers, Gini coefficients remained the same (0.84 vs 0.41). In addition, among the top 15 centers that used the most high-PODD kidneys in 2015, 13 remained to use the most high-PODD kidneys in 2016 (kappa coefficient=0.77). The consistency between 2015 and 2016 was less well among the top 20, 30, and 40 centers (kappa coefficient=0.61, 0.52, 0.38, respectively).

Figure 2. Clustering in kidney utilization among centers.

Curves closer to the diagonal line represent more equity and those closer to the lower right indicate more clustering and less equity of distribution. The Gini coefficient denotes the degree of clustering. There was less equitably distributed utilization of high-PODD kidneys than for low-PODD kidneys (Gini=0.84 vs 0.41). In total 122 (60.1%) centers did not use a single high-PODD kidney, 15 centers have used the majority (63.0%) of high-PODD kidneys.

Surplus Offers

Among the 203 centers offered (“offer” was defined as a donor kidney being offered to 1 or more recipients at a single transplant center) at least 100 kidneys during the study period, 122 (60.1%) never accepted a high-PODD kidney. There were 11 684 of surplus offers (35.0% of all 33 386 high-PODD offers) to these centers. Distribution of these surplus offers were shown in Figure 3. The median (IQR) number of surplus offers per center was 78 (27-125). Surplus offers per center ranged from 0 to 629.

Figure 3. Distribution of Surplus Offers.

Surplus offers were defined as offers (kidneys being offered to 1 or more recipients at a single transplant center) of high-PODD kidneys to centers that never accepted a high-PODD kidney. This figure shows the distribution of surplus offers among 122 centers that never accepted a high-PODD kidney during the study period. In total, 11684 surplus offers were made to these centers; a median (IQR) of 78 (27-125) surplus offers were made per center. At maximum 629 surplus offers were made to a center.

Discussion

In this national study of kidney discard in the KAS era, we prospectively validated PODD in predicting discard of deceased donor kidneys. The accuracy of PODD was excellent, and better than the other indices using several evaluation metrics including C-statistic, PPV, and NRI. We also found substantial clustering in center’s use of high-PODD kidneys: while 122 centers did not use a single high-PODD kidneys, 15 centers used the majority of these kidneys. The centers that used the most high-PODD kidneys remained consistent over time. Of all high-PODD kidney offers, 35.0% were arguably unnecessarily made to centers that never accepted such kidneys.

The high rate of kidney discard has been a concern for decades and has continued despite recent allocation changes in KAS.^7,37–40 A “rescue” strategy has been proposed to reduce discard in which kidneys at high risk of discard are identified prospectively and preferentially offered to centers more likely to transplant such kidneys. Such a strategy requires a reliable method for identified kidneys at high risk of discard; the number of kidneys thus identified should be small enough to ensure minimal disruption to the overall allocation system. In this study, we have validated that PODD would be an appropriate tool for such a system. Only 9% of kidneys had PODD exceeding 0.6; over 79% of these kidneys were discarded. Over 13% of kidneys have PODD exceeding 0.5; 71.7% of these kidneys were discarded. Only 6.2% of kidneys had PODD exceeding 0.7; 85.2% of these kidneys were discarded. The OPTN could implement rescue allocation by allocating kidneys differently if PODD exceeded a certain threshold. The higher the threshold, the higher the chance that a “rescued” kidney would otherwise be discarded – but the fewer kidneys would be “rescued”. In addition, we have also validated the usefulness of reduced PODD, which does not depend on factors that may not be known at time of initial allocation, thus would be a more practical tool. Concerns have been raised that such a system might direct all “rescue” offers to a few centers, denying other centers the opportunity to show that they are willing to utilize these marginal kidneys. Applying “rescue” allocation only to kidneys refused at the local OPO level, or allowing centers to petition for a temporary opportunity to prove themselves, could partially alleviate such concerns.

Our findings are consistent with previous reports that KDPI inadequately identifies kidneys at high risk of being discarded.^17,19 While KDPI was designed to describe organ quality, it was not designed to predict kidney discard.⁴¹ The current findings were consistent with our initial report of PODD, showing that PODD was substantially better at predicting discard than KDPI with an improvement of 15.6%.¹⁷ The ability of PODD to discriminate between acceptance and discard has been consistently high, with an AUC of 0.83 in our prior study and 0.87 in our current validation study. ¹⁷ After excluding machine perfusion and glomerulosclerosis as predictor variables which are often not available at the time of initial organ offer, our reduced PODD (r-PODD) model remained more accurate in predicting discard than KDPI with an improvement of 11.4%. Compared to the Marrero score, for kidney discard prediction, PODD still performed significantly better with an improvement of 4.1%. Reduced PODD also performed significantly better than reduced Marrero with an improvement of 11.9%.

In considering center-level behavior, we found that the use of high-PODD kidneys varied widely in the US. This confirms several other reports of center-level variation in utilization of kidneys. Wey et al found that lower offer acceptance in a donation service area (DSA) was associated both with higher discard rates in that DSA and with higher rates of export of organs from that DSA.⁴² Our group previously demonstrated that there was clustering of the utilization of suboptimal grafts among transplant centers.⁴³ We also found the clustering of use of high-PODD kidneys over time were the most consistent among the top 15 centers. In addition, we found that 35.0% high-PODD kidney offers went to centers that have never accepted a single high-PODD kidney. This is also consistent with previous findings that marginal kidneys offers were concentrated among a small amount of centers.¹⁸ Expedited offer of high-PODD kidneys to centers that are willing to accept them could be an effective way to reduce discard and improve access to transplantation.

Internationally, there have been efforts to target distribution of kidneys that were more likely to be discarded to those centers more likely to transplant them. In Europe, a policy was introduced in 2006 that transitioned organ offers to a center-oriented “rescue allocation” following a series of 5 rejections in a patient-oriented offer scheme.¹³ This was effective at decreasing discard rate¹³, and following changes to the policy in 2013, the cold ischemia time of transplanted kidneys decreased as well.¹⁵ In another example, a policy was implemented in the UK in 2006 to allocate kidneys to centers agreeing to participate in the “Declined Kidney Scheme” which then have the discretion to use the offered kidney locally for one of their patients, following a series of 5 rejections in the national allocation policy.⁴⁴ Later in 2012, the “Kidney Fast-Track Scheme” further expanded upon this by allowing these kidneys to be offered simultaneously to multiple centers.^14,45 Studies have not only shown decreased kidney discard following both policies, but perhaps more importantly, have demonstrated posttransplant outcomes that are comparable to those of organs transplanted through standard allocation schemes.^{13,14,45–47}

Our study validated the effectiveness of PODD to identify kidneys that are more likely to be discarded, and demonstrated existing variation in utilization of kidneys with high PODD among centers. Use of such a validated index to improve the distribution of kidneys that are at higher risk of being discarded could decrease kidney discard rates and potentially improve candidate survival. In the setting of an ongoing dialogue about the high rates of kidney discard, we recommend developing a policy using a validated metric to improve distribution of kidneys likely to be discarded.

Abbreviations

AUC

area under the curve

BMI

body mass index

CIT

cold ischemia time

CMV

cytomegalovirus

CVA

cerebrovascular accident

DCD

donation after cardiac death

ECD

extended criteria donor

HBcAb

hepatitis B core antibody

HBsAg

hepatitis B surface antigen

HCV

hepatitis C virus

HRSA

health resources and services administration

HTLV

human T lymphotropic virus

IRD

infectious risk donor

KAS

kidney allocation system

KDPI

kidney donor profile index

KFTS

kidney fast-track scheme

MICE

multiple imputation by chained equation

NPV

negative predictive value

NRI

net reclassification improvement

OPTN

organ procurement and transplantation network

PMM

predictive mean matching

PODD

probability of discard/delay

PPV

positive predictive value

ROC

receiver operating curve

r-PODD

reduced probability of discard/delay

SRTR

scientific registry of transplant recipients