Machine Learning Predictions for Assessing Hard-to-Place Deceased Donor Kidneys

Abstract

Rationale & Objective

Nearly 20% of deceased donor kidneys in the United States are placed “out-of-sequence” (ie, outside of standard allocation rules). The rationale for out-of-sequence placements is to expedite placement of kidneys at risk of nonuse. We aimed to (1) develop machine learning (ML) models to predict the risk of kidney nonuse over time during the allocation process and (2) use the ML predictions to assess current out-of-sequence placements.

Study Design

Retrospective cohort study using Organ Procurement and Transplantation Network data.

Setting & Participants

Deceased donors with at least one kidney recovered for transplant between January 1, 2022, and December 31, 2023 (25,785 donors; 51,320 kidneys).

Predictor

Clinical information available at distinct time points throughout the allocation process (donor medical history, biopsy, and center refusal patterns).

Outcome

Probability of kidney nonuse.

Analytical Approach

We trained ML models, evaluating area under the receiver operating characteristic curve, accuracy, and other metrics. Feature importance was assessed using Gini impurity. We compared predicted nonuse probabilities across kidneys by outcome (in-sequence, out-of-sequence, not used), conditioned on the Kidney Donor Profile Index (KDPI).

Results

Adding refusal information up to clamp time performs better than a model that uses biopsy but no refusal information (area under the receiver operating characteristic curve 0.90 vs 0.88). Center refusal information by time of prediction was among the most important predictors. Donors with out-of-sequence placements had intermediate predicted nonuse probabilities between donors with in-sequence placements and donors with unused kidneys. ML models were able to discriminate hard-to-place kidneys within each KDPI strata.

Limitations

Incomplete data on out-of-sequence placements.

Conclusions

ML can identify kidneys at high risk of nonuse before biopsy data become available and better than the KDPI. Overall, ML can provide real-time, data-driven tools to identify hard-to-place kidneys, offer a standardized and transparent way to guide accelerated placement and evaluate current practices, and ultimately reduce organ wastage.

Plain-Language Summary

In the United States, deceased donor kidneys are allocated via a sequential offering process. Presently, nearly 20% are placed discretionarily “out-of-sequence,” outside of standard allocation rules. The rationale for this practice is to avoid organ loss. In this study, we used machine learning (ML) to predict whether a kidney would go unused, using donor medical history, biopsy results, and early refusal data from transplant centers. Real-time data on other centers’ assessment of the kidney was an extremely powerful predictor, even outperforming biopsy results. According to the ML predictions, kidneys currently placed out-of-sequence were generally harder-to-place. Overall, ML can provide real-time, data-driven tools to identify hard-to-place kidneys. It also offers a standardized way to guide accelerated placement and evaluate current practices.

Introduction

In the United States, a national system exists whereby deceased donor kidneys are allocated to waitlisted candidates based on medical and geographic criteria.¹ Organ procurement organizations (OPOs) can exercise discretion to offer kidneys to any transplant program ‘out-of-sequence’ (ie, outside usual allocation criteria), if they feel that the allocation system will not place the kidney in time, as excessive cold ischemia time is associated with poorer posttransplant survival outcomes.2, 3, 4, 5 Since 2021, out-of-sequence placements of deceased donor kidneys have risen significantly, to nearly 20% of placements by the end of 2023.⁶ This shift in allocation practice coincides with the adoption of the 250 nautical mile-based Kidney Allocation System, which increased both the volume of offers and the complexity of the OPO-transplant center network.7, 8, 9, 10, 11, 12

The allocation of nearly 20% of placed kidneys via a process outside the formal allocation system leads to natural concerns about inequity and disparities.^6,13,14 OPOs develop their own criteria for out-of-sequence offers, but the process lacks standards and oversight, and there is no meaningful way for public engagement. This opacity can make out-of-sequence allocation feel unfair to both patients and transplant surgeons.

In August 2023, the Organ Procurement and Transplantation Network formed an “Expeditious Task Force” to develop data-driven accelerated placement pathways.¹⁵ Current proposed policies consider pathways for donors with a Kidney Donor Profile Index (KDPI) of 75% or higher.¹⁶ The KDPI, a composite of 10 donor attributes available pre-offer (eg, age, weight, and creatinine), excludes the wealth of data that becomes available during the offer process.¹⁷ A prior experiment to expedite offers of adult kidney donors based on KDPI failed to increase utilization.¹⁸ Thus, a potential problem with the Organ Procurement and Transplantation Network’s current approach is that the KDPI-based classification does not reliably identify kidneys at risk of nonuse.

Patients awaiting transplantation urgently need a more effective and responsive allocation system that prevents the loss of viable deceased donor kidneys. A critical first step is to establish rigorous, data-driven criteria to identify kidneys likely to be hard to place under the standard allocation system, early in the offer process. Prioritizing these kidneys through accelerated allocation pathways would reduce the number that unnecessarily go unused. This more flexible and transparent system would in turn shorten wait times for all patients and foster greater trust among both patients and surgeons.

This study has 2 objectives. First, we developed machine learning (ML) models to estimate the likelihood of a deceased donor kidney not being used (ie, recovered for the purposes of transplantation but not transplanted), incorporating information that arrives at distinct time points during the allocation process. Throughout the allocation process, new information arrives, such as refusals by transplant programs, and biopsy results and pump values after the kidney has been procured and moved into cold storage. This study expands upon previous work19, 20, 21 by incorporating refusal data into ML models. Second, we utilized these ML predictions to assess whether current out-of-sequence placements are appropriately focused on organs at risk of nonuse. Ultimately, patients awaiting transplantation and transplant surgeons urgently need greater transparency into current out-of-sequence allocation practices. ML provides an objective way to identify viable but hard-to-place kidneys to ensure they are not lost, and refusal data incorporates a human-in-the-loop element to capture contextual factors that may not be fully represented in clinical data alone.

Methods

Data

Approval for this study was obtained from the Stanford University Institutional Review Board (Protocol 68925). Using datasets from the Organ Procurement and Transplantation Network, our study cohort was donors with match runs (ie, whose kidneys were allocated) between January 1, 2022, and December 31, 2023. We excluded donors that followed nonstandard allocation pathways and donors missing key data (see Item S1 for full details).

Predicting Risk of Nonuse Over Time

Similar to previous studies,19, 20, 21 we used logistic regression, decision trees, and random forests to identify whether a donor would be at risk of nonuse. To assess how informative data arriving during allocation is, we explicitly made predictions at different time points during the allocation process. We assumed that during the initial hours of allocation, an OPO would not treat kidneys from the same donor differently, so observations were at the donor level.

Features/Predictors

We used donor features known at different points during the allocation process (Table 1). Prior work has made predictions after biopsy and machine perfusion variables became available,²⁰ whereas we added additional information from centers’ refusals that may be available earlier. Based on discussions with OPO personnel and transplant surgeons, we selected 3 hours after clamp as the time point at which biopsy results became available. We trained models with and without refusal data to assess how much refusals add on top of clinical information. In total, we trained on 6 different feature sets: (1) before offer (ie, using donor medical history data available at the time of offering); (2) clamp time, without refusals; (3) clamp time, with refusals; (4) 3 hours after clamp, without refusals; (5) 3 hours after clamp, with refusals; and (6) 6 hours after clamp, with refusals. Further sensitivity analysis on these features is given in Item S1.

Table 1.

Features Used in the Machine Learning Models to Predict Risk of Nonuse at Different Times During the Offering Process

Time	Information Available to Use As Features
Before offer	Donor characteristics known before match run: •Kidney Donor Risk Index, age, height, weight, sex, blood type •Admission, peak, and terminal creatinine •Donation after cardiac death •Cause of death •Donor health history (diabetes, insulin dependence, protein in urine, high risk for HIV, history of cancer, cigarette usage, cocaine usage, hypertension, IV drug usage, other drug usage, arginine) •Whether the donor is homozygous for A, B, and DR Human Leukocyte Antigens •Organ procurement organization •Number of centers within 250 nautical miles of donor hospital
Clamp time	Additional information gained: •Urine output lower bound •Whether the donor was offered pre-clamp If including refusal data: •Number of different centers that sent a multiple-patient simultaneous refusal by clamp for >1 and >5 patients •Number of different centers that sent a single-patient refusal by clamp
Clamp + 3 ha	Additional information gained: •Whether ≥1 kidney was biopsied •“Good biopsy”: 0%-10% glomerulosclerosis AND absent or minimal interstitial fibrosis in all biopsied kidneysb •“Bad Biopsy”: >20% glomerulosclerosis OR mild-moderate or severe interstitial fibrosis in ≥1 kidneyb If including refusal data: •The 3 features relating to the number of different centers sending refusals were updated to include refusals sent within 3 h of clamp.
Clamp + 6 h	Additional information gained: •The 3 features relating to the number of different centers sending refusals were updated to include refusals sent within 6 h of clamp.

Abbreviation: IV, intravenous.

^aWe excluded pump values as most were null even 3 hours after clamp.

^bWe grouped biopsy results in this manner because some categories (eg, severe interstitial fibrosis) had small sample sizes, which were further reduced when stratified by kidney side (left or right). Combining these categories increased sample sizes, and this approach was informed by personal communication with transplant surgeons. In Item S2 and Figure S4, we present results using the original biopsy categories.

Centers often decline offers for multiple patients at the same time, rather than only for the patient who was offered the kidney, as observed in the refusal timestamps in the dataset.²² We constructed features representing the number of distinct centers that have sent single- and multiple-patient refusals (>1 or >5 patients simultaneously refused) by certain time points after clamp, based on the hypothesis that multipatient refusals are linked to intrinsic organ issues and single-patient refusals result from a mismatch between the patient and the organ.²² We only captured refusals up to the specified time point; if no offers were made, there were consequently no refusals.

Label/Outcome

We predicted donors for whom all recovered kidneys were not used.

Model Training and Evaluation

We trained the models on donors whose match runs were between January 1, 2022, and June 30, 2023 (n = 19,695) and evaluated the models on donors whose match runs were between July 1, 2023, and December 31, 2023 (n = 6,090). We used cross-validation with randomized search to tune hyperparameters on the training set, maximizing the area under the receiver operating characteristic curve (AUC). The hyperparameters and their values are given in Table S1. We then assessed model performance on the held-out test set, using the evaluation metrics of AUC, accuracy, balanced accuracy, F1 score, false positive rate, and false negative rate. For robustness, we also report model performance on the training set. For all metrics, we computed 95% confidence intervals by bootstrapping the evaluation set with 1,000 iterations.

The ML models estimated the probability of nonuse. We chose the threshold to compute accuracy, balanced accuracy, false positive rate, and false negative rate by taking the threshold with the highest F1 score on the training set. We quantified the 8 most important features of the random forest models via Gini impurity-based feature importance to assess if and how much the additional clinical and refusal information were important predictors compared to the donor medical history.

Characterizing Donors With Kidneys Placed Out-of-Sequence

As in previous work,⁶ we identified kidneys placed through out-of-sequence allocation by looking at the refusal codes entered during the match run. The data does not record whether out-of-sequence allocation was attempted for kidneys that were not used. A kidney was defined as being placed out-of-sequence if there was at least one bypass offer with a refusal code “Operational OPO” (861), “Donor medical urgency” (862), or “Offer not made due to expedited placement attempt” (863) at a lower sequence number than that of the accepted offer. In a scenario in which a donor has one kidney placed prior and one kidney placed after bypasses, the first kidney would not be but the second kidney would be considered out-of-sequence. We created a histogram to visualize deceased donor kidney outcomes stratified by KDPI under the current Kidney Allocation System-250, marking kidneys placed in-sequence versus out-of-sequence (see Item S1 for full details).

We used the predicted nonuse probabilities from our random forest models as a score reflecting the complexity of placing organs. As we developed donor-level prediction models, we stratified our analysis by donors whose kidneys were all placed in-sequence via the standard allocation system (“placed in-sequence”) (n = 16,804), donors who had at least one kidney placed out-of-sequence (“placed out-of-sequence”) (n = 3,216), and donors whose recovered kidneys were not used (“not used”) (n = 5,765). We analyzed the key characteristics of these donors with descriptive statistics. We used a density plot to analyze the distribution of these predicted nonuse probabilities from the pre-offer random forest, the clamp time random forest with refusals, and the 3-hour postclamp random forest with refusals. To assess how predicted probabilities vary based on prerecovery organ quality and postrecovery observed refusal patterns, we investigated the relative contributions of (1) KDPI and (2) the number of centers issuing multipatient refusals for >5 patients to the probabilities predicted by the clamp time random forest with refusals. Lastly, to understand if OPOs were applying out-of-sequence allocation to harder-to-place organs and to see if the ML model can differentiate hard-to-place organs conditional to the KDPI, we created a density plot of the probabilities from the clamp time random forest with refusals, stratified by KDPI categories. We also further explored the differences between donors with 1 and 2 kidneys placed out-of-sequence.

Results

Of the 51,320 kidneys from 25,785 donors in our analysis, 37,215 kidneys (72.5%) were transplanted, and 14,105 kidneys (27.5%) were not used (ie, recovered for the purposes of transplantation but not transplanted).

Predicting Risk of Nonuse Over Time

At all times, the random forests performed better than both logistic regression and the decision trees (Fig 1; Table S2). Incorporating additional clinical information as features enhanced model performance, and adding offer refusal data further augmented model performance (Fig 1; Table S2). For example, on the test set, the pre-offer random forest had an AUC of 0.87 and an accuracy of 0.79. Including refusal data by clamp time increased the AUC to 0.90 and the accuracy to 82%; including both biopsy results and refusal data by 3 hours after clamp further increased the AUC to 0.91 and accuracy to 84%; and including refusal data by 6 hours after clamp further increased the AUC to 0.92 and accuracy to 85%. The clamp time random forest with refusal data outperformed the 3 hours after clamp random forest without refusal data (AUC, 0.900 [95% confidence interval, 0.892-0.908] vs 0.883 [95% confidence interval, 0.874-0.892]), even though the latter incorporated biopsy results and the former did not. For robustness, Table S3 reports model performance on the training set, demonstrating consistent results.

Figure 1.

Area under the receiver operating characteristic curve (AUC) and corresponding 95% confidence interval of machine learning models trained with features derived from data up to different time points on held-out test set donors (n = 6,090). The x-axis represents the time points at which the features were determined: before offer, at clamp time, 3 hours after clamp, and 6 hours after clamp. The y-axis displays the AUC values. The colors and line styles indicate the type and features of the machine learning model.

The Kidney Donor Risk Index, donor age, peak and terminal creatinine, and diabetes status were the most important features of the pre-offer random forest model (Fig 2A). The second most important feature of the clamp time and 3-hour postclamp random forests with refusal data was the number of unique centers sending a refusal for >5 patients simultaneously (Fig 2C and E), and this feature was relatively more important at 3 hours after clamp than at clamp time. In the 6-hour postclamp random forest with refusal data, this was the most important feature, surpassing both the Kidney Donor Risk Index and biopsy results (Fig 2F).

Figure 2.

Feature importances of random forest models trained with features derived from data up to different time points. The row represents the time points at which the features were determined: before offer (A), clamp (B-C), 3 hours after clamp (D-E), and 6 hours after clamp (F). The colors indicate whether the model was trained with or without refusal information. Abbreviations: DCD, donation after cardiac death; KDRI, Kidney Donor Risk Index.

Characterizing Donors With Kidneys Placed Out-of-Sequence

In our cohort, 32,830 kidneys were placed in-sequence, and 4,385 kidneys were placed out-of-sequence (11.8%) with 162, 6, and 4,217 following refusal codes 861, 862, and 863, respectively. Kidneys across all KDPIs underwent out-of-sequence allocation (Fig S1).

There were 3,216 donors who had at least one kidney placed out-of-sequence (Table 2). These donors were more likely to have higher KDPI, Kidney Donor Risk Index, age, and creatinine compared with donors placed in-sequence, but lower values compared to not used donors. Generally, donors placed out-of-sequence had characteristics that fell in between those of donors placed in-sequence and not used donors. More centers issued multipatient refusals prior to clamp time for donors placed out-of-sequence and not used donors compared with donors placed in-sequence. For example, by clamp time, an additional 2.32 (1.56 vs 3.88) centers had sent multipatient refusals for donors placed out-of-sequence versus donors placed in-sequence (Table 2).

Table 2.

Selected Donor Characteristics

Characteristic	Donors With All Kidneys Placed In-Sequence (n = 16,804)	Donors With ≥1 Kidney Placed Out-of-Sequence (n = 3,216)	Donors With All Recovered Kidneys Not Used (n = 5,765)
KDPI	43 (27)	54 (25)	79 (20)
KDRI	1.29 (0.40)	1.45 (0.41)	1.97 (0.52)
Donor age (y)	39.3 (15.2)	43.7 (14.9)	54.7 (13.8)
Creatinine (mg/dL)	1.28 (1.28)	1.68 (1.67)	2.12 (1.95)
Donation after cardiac death	5,373 (32.0%)	1,389 (43.2%)	2,570 (44.6%)
History of hypertension	461 (2.7%)	111 (3.5%)	402 (7.0%)
History of diabetes	1,468 (8.7%)	369 (11.5%)	1,713 (29.7%)
At least 1 kidney biopsied	8,330 (49.6%)	2,219 (69.0%)	5,317 (92.2%)
Glomerulosclerosis >20% or mild-moderate or severe interstitial fibrosis in ≥1 kidney	504 (3.0%)	126 (3.9%)	1,817 (31.5%)
Blood type
O	7,997 (47.6%)	1,624 (50.5%)	2,758 (47.8%)
A	6,154 (36.6%)	1,132 (35.2%)	2,146 (37.2%)
B	2,021 (12.0%)	408 (12.7%)	620 (10.8%)
AB	632 (3.8%)	52 (1.6%)	241 (4.2%)
Number of unique centers sending a single-patient refusal
Clamp	3.13 (3.05)	4.21 (3.77)	3.44 (3.15)
Clamp + 3 h	3.49 (3.21)	4.62 (3.91)	3.71 (3.27)
Number of unique centers sending a multipatient refusal for >1 patients
Clamp	1.56 (3.24)	3.88 (5.77)	7.13 (8.83)
Clamp + 3 h	1.83 (3.53)	4.53 (6.15)	8.20 (9.53)
Number of unique centers sending a multipatient refusal for >5 patients
Clamp	0.66 (2.33)	2.53 (4.94)	5.63 (8.00)
Clamp + 3 h	0.81 (2.59)	3.02 (5.31)	6.60 (8.71)
Average number of centers within 250 NM of donor hospital	26.61 (17.84)	32.43 (18.99)	28.34 (17.48)
Accepted by 1 h after clamp	208 (1.2%)	25 (0.8%)
Accepted by 3 h after clamp	4,560 (27.1%)	270 (8.4%)

Note: Continuous variables are represented as mean (SD). Categorical variables are represented as n (%).

Abbreviation: KDPI, Kidney Donor Profile Index; KDRI, Kidney Donor Risk Index; NM, nautical mile.

In our test set, there were 3,734 donors placed in-sequence, 960 donors placed out-of-sequence, and 1,396 not used donors. The pre-offer random forest separated these donors, with average predicted nonuse probabilities of 0.28, 0.40, and 0.70, respectively (Fig 3A). The clamp time random forest with refusals was further able to separate these donors in terms of the predicted probabilities, with average probabilities of 0.22, 0.40, and 0.72, respectively (Fig 3B).

Figure 3.

Stacked distribution of the predicted nonuse probabilities of the random forest models. The x-axis displays predicted nonuse probabilities from (A) the pre-offer random forest, (B) the clamp time random forest with refusals, and (C) the 3-hour postclamp random forest with refusals. The y-axis shows the number of donors with each predicted nonuse probability (stacked). The colors represent donors whose kidneys were placed in-sequence (green), donors with at least one kidney placed out-of-sequence (yellow), and donors whose kidneys were not used (purple).

Across all time points, as the predicted probabilities increased, the share of donors placed in-sequence decreased, and the share of not used donors increased (Fig 3). The share of donors placed out-of-sequence peaked around the probability of 0.6. In the mid-range of probabilities (0.5-0.7), all 3 outcomes were represented.

Low KDPI donors with high numbers of multipatient refusals (Fig 4, top right of each panel) had similar predicted probabilities as high KDPI donors with low numbers of multipatient refusals (Fig 4, bottom left of each panel). When the number of multipatient refusals was large, the predicted nonuse probability was slightly lower for donors placed out-of-sequence than for donors placed in-sequence.

Figure 4.

Average predicted nonuse probability of (A) donors whose kidneys were placed in-sequence and (B) donors with at least one kidney placed out-of-sequence based on the Kidney Donor Profile Index (KDPI) and the number of unique centers sending multipatient refusals for >5 patients by clamp time. In each heatmap, the x-axis shows the number of unique centers sending multipatient refusals for more than 5 patients by clamp time, and the y-axis shows the KDPI bin. Each cell displays the average predicted nonuse probability from the clamp time random forest with refusals.

There was clear separation in the distributions of predicted nonuse probabilities of placed in-sequence, placed out-of-sequence, and not used donors within each KDPI bin (Fig 5). Across all KDPI groups, the probabilities predicted by the clamp time random forest with refusals were higher for donors placed out-of-sequence and not used donors than for donors placed in-sequence. Slightly enhanced separation was seen in the 3-hour postclamp random forest with refusals (Fig S2). We observed a separation in predicted probabilities between donors with 1 kidney placed out-of-sequence and those with both kidneys placed out-of-sequence, with the latter generally having higher predicted nonuse probabilities except in the highest KDPI group (Fig S3).

Figure 5.

Density plot of predicted nonuse probability from the clamp time random forest with refusals, by Kidney Donor Profile Index (KDPI) and donor outcome. Each subfigure illustrates a density of the predicted nonuse probability from the clamp time random forest with refusals for donors with KDPI (A) 0%-34%, (B) 35%-60%, (C) 61%-85%, and (D) 86%-100%. The colors represent donors whose kidneys were placed in-sequence (green), donors with at least one kidney placed out-of-sequence (yellow), and donors whose kidneys were not used (purple).

Most OPOs appear to expedite harder-to-place donors compared to the donors that they place in-sequence (Item S2; Fig S5).

Discussion

Since 2022, the number of deceased donor kidneys placed out-of-sequence has grown rapidly.⁶ This shift may partly be due to the increasing complexity of OPO and transplant program interactions, engendered by the change to the 250 nautical mile-based Kidney Allocation System, which increased the number of offers considered local.⁷ Given the ongoing efforts to transition all solid organ allocation in the United States to a continuous distribution system, which will only magnify the complexity of OPO-center relations, the rise of discretionary offering is especially problematic. The current use of out-of-sequence allocation can appear unfair to waitlisted candidates and lacks sufficient transparency for both patients and transplant surgeons. Hence, a high policy priority is to identify organs at risk of nonuse, early during the offer process and using a standardized set of criteria, and selecting these organs for accelerated placement.¹⁶

Because kidney allocation occurs over a period of time, metrics of organ quality that are updated throughout the allocation process (eg, a “real-time” risk index) would be particularly advantageous over static measures like the KDPI. Time-updating ML predictions provide an objective way to identify viable but hard-to-place kidneys. Beyond clinical data about the donor, refusal information from offers that have already been made can further improve identification of kidneys at risk of nonuse. Although refusal data reflect subjective clinical judgment and may embed human biases, clinical data alone also contain biases that would influence any accelerated placement selection criteria, including currently used approaches. Moreover, refusal patterns often capture valid clinical concerns not reflected in KDPI or other registry-based measures, such as infection or malignancy risk, anatomical or surgical anomalies, procurement issues, or other medical factors.²² Incorporating refusal data thus introduces a human-in-the-loop element that captures these contextual factors (eg, anatomical data) not fully represented by clinical registry data. Motivated by this, the goals of this study were to use ML to predict, over time, the likelihood of a kidney being hard to place and to characterize current out-of-sequence placements based on these predictions. Leveraging such predictions to guide accelerated placement could lead to higher utilization.

We find that organ refusal information, even prior to clamp, is highly informative. Adding refusal information up to the time of clamp improves random forest model accuracy by 4% (83% vs 79%) and AUC by 0.03 (0.90 vs 0.87) compared with the pre-offer model (Fig 1; Table S2). Additionally, the number of unique centers that have sent simultaneous refusals for >5 patients is the second most important feature at clamp time and 3 hours after clamp and the most important feature at 6 hours after clamp (Fig 2). The random forest with refusal data up until clamp outperforms the random forest without refusal data, even when biopsy information is incorporated (Fig 1). As biopsy results arrive hours into the allocation process, when it may already be too late to begin to begin out-of-sequence allocation, refusals can be a valuable early signal for organs that are hard to place. Overall, refusal data generated during the allocation process improves the identification of hard-to-place kidneys and may serve as an objective criterion in creating pathways for accelerated placement.

Consistent with previous work,⁶ we found that when measured by the predicted nonuse probabilities, out-of-sequence placements result in transplanting higher-KDPI, harder-to-place kidneys compared with in-sequence allocations. The predicted nonuse probabilities from the ML models are higher for donors with at least one organ placed out-of-sequence than for donors whose recovered kidneys were placed in sequence (Fig 3). The ML model can identify which organs are hard to place better than KDPI, as there is separation of probabilities by donor outcomes even within KDPI buckets (Fig 5). Although this finding does not imply that kidneys placed out-of-sequence would have been placed had they been offered in-sequence, it is encouraging that ML predictions identify them as harder-to-place.

We characterized donors with out-of-sequence kidney placements, but the existing data are insufficient to conclusively answer whether out-of-sequence placements improve utilization. Our analysis cannot predict counterfactual placement outcomes. Whether organs with low predicted probabilities that were placed out-of-sequence would have been placed in-sequence is an important open question. Similarly, whether organs with high predicted probabilities would have gone unplaced without the out-of-sequence allocation deserves further study. However, many kidneys that are not used in the United States are successfully transplanted in France,²³ suggesting that some organs with high predicted probabilities of nonuse could have resulted in successful transplants as well. Nonetheless, since all placed organs share the same label in training, the ML model generates a “score” quantifying each organ’s risk of nonuse. This score can be used to objectively prioritize the organs based on their likelihood of being used and guide allocation decisions. As time progresses during the allocation process, the greater amount of information available allows for more refined criteria for identifying hard-to-place donors. Still, despite limiting the input information to that which is accumulated up to clamp time, the clamp time random forest with refusals is quite accurate and can identify hard-to-place donors better than the KDPI.

One limitation of this analysis is that out-of-sequence allocation is only recorded for kidneys that were ultimately transplanted. Calls by OPOs to place organs outside the standard allocation system are not recorded for kidneys that go unused, resulting in a biased dataset that does not include failed out-of-sequence placement attempts. If some of the kidneys that were not used had previously undergone unsuccessful out-of-sequence placement attempts, organs that are placed out-of-sequence are likely to be even harder-to-place than our analysis shows. Additionally, we were unable to count certain out-of-sequence placement attempts due to miscoding.²⁴ Another limitation with regard to the dataset is that it is missing important information, such as the time OPOs began calling centers to initiate the expediting process, the time of biopsy information arrival, and donor anatomic data and surgical damage. Further, the dataset does not fully capture logistical and transportation factors, such as missed flights, which can limit allocation to geographically nearby centers. Better data collection to capture the full picture of the allocation process will be helpful in improving the accuracy of our models and understanding out-of-sequence allocation.

Overall, our analysis offers valuable insights into the current state of out-of-sequence placements and provides a method to improve identification of hard-to-place kidneys early on during the offer process (ie, time of clamping). Current out-of-sequence allocation is discretionary at the OPO level, creating a lack of transparency for both patients and transplant surgeons and potentially leading to perceptions of unfairness. ML provides a systematic, data-driven approach to support pathways that improve kidney utilization, rather than promoting indiscriminate out-of-sequence allocation. ML predictions themselves should also adapt over time to reflect policy changes and shifts in donor supply. After identifying a kidney as hard to place based on standardized, objective, and easily obtainable information within the offering process, the priority order should be adapted in a transparent manner to facilitate placement. These kidneys could still lead to successful transplants but might otherwise go unused under the current sequential offering system. Increasing utilization can shorten waiting times for all waitlisted patients. This data-driven approach can enhance transparency and trust in the allocation process, improve the efficiency of accelerated placement, and ultimately achieve the objective of better outcomes for all patients.

Supplementary Material

Supplementary File (PDF)

Figure S1-S5; Item S1 and S2; Table S1-S3.