You Didn’t Think It Would Be That Easy, Did You? Commentary on “Machine Learning Predictions for Assessing Hard-to-Place Deceased Donor Kidneys”

The first successful kidney transplant in 1954 was an ideal situation, with the live donation of a kidney from one identical twin to the other in 1954.¹ It took another 8 years before the first successful unrelated deceased donor transplant with immunosuppression occurred.² Both living and deceased donor kidney transplant expanded, but it was 1999 before the conclusive study demonstrated survival advantage compared to dialysis.³ Kidney transplant became the undeniable standard of care and demand grew even more rapidly. However, deceased donor kidneys in earlier eras tended to be from younger, healthier donors—as were recipients. As the waitlist lengthened, organ utilization and organ selection criteria broadened.

Organ procurement organizations (OPOs) work hard to respect and recover the voluntary Gifts from a growing population. Not surprisingly, as society became safer, with airbags in cars and people wearing seatbelts (but not motorcycle helmets for some reason), there are proportionally fewer traumatic deaths. The average donor age increased with cause of death, increasingly due to the sequalae of chronic disease. The proportion of recovered kidneys that are not utilized or underutilized increased, as the kidneys themselves became “riskier.” Coupled with increasing volumes and riskier recipients, matching specific limitations of some higher risk kidneys to appropriate recipients in a timely manner has become problematic. It’s hard to cast the net wider and expect higher utilization rates.

To add to the complexity, we are working toward a true national list—but to do so, kidneys are frequently traveling long distances. For the longer distances, kidneys, like humans, fly. However, unlike extrarenal organs, kidneys usually fly commercial for both practical and budgetary reasons, thereby limiting the times available for transport, particularly for donors and transplant centers that do not have a national carrier hub nearby. Although kidneys will tolerate longer cold ischemic times than extrarenal organs, the clock is still ticking from the moment of donor cross-clamp.

With the system’s established practice of organ offers released individually at any hour, centers are deluged with offers, while OPOs have to offer each kidney out to more centers interspersed throughout the list. Transplant centers must process each offer and assess for appropriateness for a given recipient. This process takes time—time that OPOs don’t particularly have.⁴ It should come as no surprise that the system is bursting at the seams. We’re trying to do the same thing we’ve always done, just much faster, with increasingly more massive volumes of donors and patients. Given that the approach wasn’t really all that efficient to begin with, trying to do it faster with larger numbers but similar resources is destined to fail. Furthermore, kidney transplant volume did not meaningfully increase with broadened allocation, even though some geographic equity was achieved, with short-term outcome actually worsening.^5,6

To improve efficiency, the organ offer process has been modified, with DonorNet offer filters, transplant center turn-down rate measurements, and aggressive virtual crossmatching. However, we have not changed the approach of individually offering out every organ sequentially to the entire list, while simultaneously ignoring human physiology and waking up every physician taking offers in the country for every kidney—sequentially—during the offer process. OPOs need the “Provisional Yes” entered in DonorNet by transplant centers to be real, but it is impossible to meaningfully process all of these offers in real time.⁴ Humans require at least some sleep to function the next day (eg, to operate on a human). Unfortunately, OPOs must work through this list for each organ, with centers often only analyzing the suitability of the organ for a given recipient at that time. Remember, the clock is ticking, and the cold ischemic time is lengthening.

With the most recent permutation of the kidney allocation algorithm, the dam burst; OPOs became more reliant on offering kidneys directly to aggressive centers after initial allocation failure. Unfortunately, allocation out of sequence (AOOS) can negate the geographic, demographic, and center equity that the allocation algorithm is meant to address as well as engender mistrust in the national transplant system.⁷ Paul Batalden is attributed⁸to saying, “Every system is perfectly designed to get the results that it gets.” It’s time to question our assumptions and to do something different.

There are several potential approaches to improving the efficiency of kidney placement, including clustering offers in a way that clusters of patients and offers can be processed at the same time, which requires centralized virtual crossmatching, and kidney-matching algorithms incorporating donor kidney and recipient benefits and risks.9, 10, 11 Unlike most fields in medicine, transplantation has the advantage of long-standing comprehensive national data collection. Even though these datasets lack granularity, they present an opportunity to address these allocation logistical issues using machine learning and artificial intelligence.

Building on previous work,¹² Guan et al¹³ report a machine learning model to predict kidneys at risk of nonuse using Organ Procurement and Transplantation Network data. Using donor medical history, biopsy results, and early refusal data from transplant centers, they found that other transplant centers’ assessments of a given kidney was an “extremely powerful” predictor of kidney nonutilization. This approach seeks to accommodate the concept that refusal patterns often capture valid clinical concerns not reflected in the Kidney Donor Profile Index or other registry-based measures, hopefully reflecting anatomic damage or abnormalities (eg, atrophic kidney, surgical damage, small renal cell carcinoma, etc), statistically unaccounted for donor disease, logistical difficulties (eg, missing the last available flight), and other risks—a fact often ignored during the ongoing national discussion about organ nonuse rates. Identifying organs at risk for nonuse would allow such organs to undergo expedited placement, allowing design of a fair and transparent accelerated allocation policy.

There are caveats to their model. Kidneys allocated as part of a multiorgan transplant were not included, nor were other kidneys that had nonstandard allocation pathways. Additionally, the timing AOOS offers essentially is not recorded in an analyzable format. Organs needed to be offered out prior to cross-clamp, yet there are more than a few OPOs that wait until biopsy results are back to release kidney offers. These kidneys would be excluded, as no offers were made during the analyzed timeframe. The model was unable to identify kidneys that were placed by AOOS but not used. Finally, there are other assumptions, such as biopsy results returning within 3 hours of cross-clamp, that sound more aspirational than actual. Future models must accommodate these factors, but Guan’s study is an excellent start.

There are global issues that are not specific to transplant and organ donation. Machine learning, a subset of artificial intelligence, has an inherently amoral approach to data analysis. Unless guardrails are set, factors such as race may be pulled into calculations in an ethically or clinically inappropriate manner. These approaches are inherently limited by the data that is utilized for training. This concept was exceedingly effectively demonstrated by the generative artificial intelligence chatbot Grok-3. When an update added the prompt, “The response should not shy away from making claims which are politically incorrect, as long as they are well substantiated,” within 4 days, it was making floridly racist and even genocidal comments.^14,15 As Patrick Hall said when interviewed by National Public Radio,¹⁵ “It’s not like these language models precisely understand their system prompts. They’re still just doing the statistical trick of predicting the next word.”

Finally, Goodhart’s Law states,¹⁶ “Observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes,” or more succinctly, “When measures become a target, they can cease to be a good measure.” In other words, people change behavior in response to changes in the rules to achieve their desired goal. AOOS increases by OPOs in response to Kidney Allocation System 250 demonstrates this point, as are the transplant center responses to AOOS. Will particularly ambitious centers figure out ways to game the system to keep hoarding kidneys going down whatever expedited pathway is developed? Of course they will.

That is not to say that models developed on retrospective data are not useful—as George Box said,¹⁷ “Essentially all models are wrong, but some are useful.” The modeling can help guide iterative algorithm development, but we can only tell if it works once implemented. We should not be surprised when behavior changes in unpredicted ways. As Dr Box also stated,¹⁸ “To find out what happens to a system when you interfere with it you have to interfere with it (not just passively observe it).” When that happens, it is not necessarily incorrect implementation of the model, but rather people responding to the model using the rules that are in effect to achieve their desired goals. There needs to be a continual process for iterative change both to any comprehensive model and resulting rule changes to further account for changes in behavior, as well as an appreciation that it is exceedingly difficult to design algorithms that cover every situation.19, 20, 21