Modernizing the Design Process for US Organ Allocation Policy: Toward a Continuous Distribution Policy for Kidneys

Abstract

Background.

The allocation of a limited supply of donor organs remains a critical challenge for organ transplantation. The analytical tools that policymakers rely upon for improving allocation policy have seen little advancement since the introduction of computer simulation in 1995. In recent years, simulation has increasingly become a bottleneck in the policy design process. Partnering with the Organ Procurement and Transplantation Network Kidney Transplantation Committee, our team introduced new analytical techniques into the policy design process.

Methods.

A new simulation algorithm was developed that reduces the time required to simulate 1 y of allocation from >6 h down to about 15 s while using the same simulation model as the preexisting simulator used by the Organ Procurement and Transplantation Network. This improvement enabled the simulation of thousands of allocation policies, allowing the introduction of multiobjective optimization as a primary method for policy design. An interactive website was created for committee members to analyze results and perform policy optimization.

Results.

These techniques were applied to the development of new continuous distribution allocation policies for kidneys. We detail the policy design process, present graphical results from 50 000 policy simulations, and highlight 4 policies optimized to balance between multiple objectives differently.

Conclusions.

Advances in analytical tools offer a path to improving organ transplantation through more effective and equitable organ allocation policies.

The allocation of a limited supply of donor organs is a central problem facing organ transplantation. In the United States, the Organ Procurement and Transplantation Network (OPTN) organ allocation policies determine transplant candidate priority. With the aim of saving as many lives as possible, as equitably as possible, as efficiently as possible, and with the best possible medical outcomes, developing a policy that simultaneously meets these goals is challenging. Although the traditional OPTN policy development process has made progress,^1,2 it has also been criticized as slow and inefficient.³ During the past few years, our team at the Massachusetts Institute of Technology and the OPTN partnered to introduce a new approach to policy design that leverages high-speed simulation and optimization to design policies that better meet these multiple policy goals.

The traditional OPTN policy development process begins with policymakers creating specific allocation policy options. These proposed policies have then been evaluated through computer simulation since 1995.⁴ Simulations reprioritize historical candidates for organs donated during a particular period according to the allocation policy under study. Models predict the resulting transplants and medical outcomes. OPTN committees review these simulation results to evaluate and compare the policies and then either accept 1 or create more to analyze.

There are several challenges with this approach. Only a handful of policy options are considered at once, so although simulation results allow policymakers to evaluate the effects of these policies, they provide little evidence that these are the best options possible. In addition, it is not uncommon for simulations to identify flaws across all proposed policies, necessitating new proposals and simulations. The limited available information makes it difficult for policymakers to determine how to improve their proposals. The turnaround to simulate 1 group of policies is typically several months, which, when combined with time for the public to provide feedback, can stretch the development timeline into years. These challenges can be addressed by evaluating more policies at once. However, with >6 h of run time per simulation and 10 runs per policy, this quickly becomes computationally intractable.

The main contributions of our work are as follows: (1) a simulation algorithm that reduces the time required to simulate 1 y of allocation from hours down to seconds without impacting accuracy. (2) The introduction of an optimization-based policy design process, enabled by the substantial improvement in computational capabilities. This allows policymakers to iterate through policy options much more rapidly, to better understand tradeoffs between competing goals, and to find policies that optimize (rather than simply improve) key metrics.

Aspects of these techniques played a limited role in the development of an allocation policy adopted for deceased-donor lungs.^5–7 Beginning in October 2022, our team worked with the OPTN Kidney Transplantation Committee to fully use the new approach for the development of a continuous distribution allocation policy for deceased-donor kidneys. In this article, we detail the new methods and policy development process used, and present findings on kidney allocation that suggest a pathway for improving the efficiency and equity of deceased-donor organ allocation.

MATERIALS AND METHODS

In this section, we describe the policy design process and detail the infrastructure that enabled it. First, we briefly describe the continuous distribution framework for the allocation policy.

Under the continuous distribution framework, transplant candidates are prioritized for a donated organ by a weighted average of several subscores. Each subscore captures 1 prioritization reason called an attribute. For example, the proximity efficiency attribute assigns a subscore based on the distance between candidate and donor hospitals. The subscore of an attribute is calculated using a formula called a rating scale and weighted by an attribute weight. See Table 1 for proposed attributes. For more about the framework and reasoning behind its introduction, see Stewart.⁷

TABLE 1.

Definition of attribute rating scales and weights for optimized policies

Attribute	Policy attribute weights				Attribute rating scale
	Policy A2	Policy B2	Policy C2	Policy D2
Medical urgency	0.096	0.096	0.096	0.096	$\{\begin{array}{l}1,\mathrm{i}\mathrm{f}\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{i}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{y}\mathrm{u}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{i}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{e}0,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$
DR mismatch	0.010	0.010	0.016	0.030	$\{\begin{array}{l}1,\mathrm{i}\mathrm{f}0\mathrm{H}\mathrm{L}\mathrm{A}\mathrm{D}\mathrm{R}\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{s}0.7,\mathrm{i}\mathrm{f}1\mathrm{H}\mathrm{L}\mathrm{A}\mathrm{D}\mathrm{R}\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}0,\mathrm{i}\mathrm{f}2\mathrm{H}\mathrm{L}\mathrm{A}\mathrm{D}\mathrm{R}\mathrm{m}\mathrm{i}\mathrm{s}\mathrm{m}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{s}\end{array}$
Longevity matching	0.055	0.064	0.043	0.042	$\{\begin{array}{l}1,\mathrm{i}\mathrm{f}\mathrm{K}\mathrm{D}\mathrm{P}\mathrm{I}\le 20\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{E}\mathrm{P}\mathrm{T}\mathrm{S}\le 200,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$
Blood type	0.093	0.093	0.093	0.093	$\{\begin{array}{l}1,\mathrm{i}\mathrm{f}\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}\mathrm{t}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{B}0.772,\mathrm{i}\mathrm{f}\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}\mathrm{t}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{O}0.648,\mathrm{i}\mathrm{f}\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}\mathrm{t}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{A}0,\mathrm{i}\mathrm{f}\mathrm{b}\mathrm{l}\mathrm{o}\mathrm{o}\mathrm{d}\mathrm{t}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{A}\mathrm{B}\end{array}$
cPRA	0.400	0.400	0.400	0.400	$\frac{0.075}{1+e^{-9+7.2x}}+\frac{0.773}{1+e^{21.9+69.6x}}+\frac{0.152}{1+e^{1.47+2.87x}},$ $\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}x\mathrm{i}\mathrm{s}{\log }_{10}(100-\mathrm{c}\mathrm{P}\mathrm{R}\mathrm{A})$
Prior living donor	0.096	0.096	0.096	0.096	$\{\begin{array}{l}1,\mathrm{i}\mathrm{f}\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{d}\mathrm{o}\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{K}\mathrm{D}\mathrm{P}\mathrm{I}\le 850,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$
Pediatric	0.100	0.100	0.100	0.100	$\{\begin{array}{l}1,\mathrm{l}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{a}\mathrm{g}\mathrm{e}<18\mathrm{a}\mathrm{n}\mathrm{d}{\mathrm{a}\mathrm{d}\mathrm{d}}^{\prime }\mathrm{t}\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{d}\mathrm{s}\mathrm{m}\mathrm{e}\mathrm{t}0,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$ Additional conditions are either $\mathrm{K}\mathrm{D}\mathrm{P}\mathrm{I}\le 35$ or $35<\mathrm{K}\mathrm{D}\mathrm{P}\mathrm{I}<85$ and donor age < 18.
Safety net	0.032	0.032	0.032	0.032	$\{\begin{array}{l}1,\mathrm{l}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{s}\mathrm{a}\mathrm{f}\mathrm{e}\mathrm{t}\mathrm{y}\mathrm{n}\mathrm{e}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{K}\mathrm{D}\mathrm{P}\mathrm{I}\ge 200,\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}\end{array}$
Waiting time	0.039	0.041	0.049	0.051	$0.1t,\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}t\mathrm{i}\mathrm{s}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{f}\mathrm{y}\mathrm{i}\mathrm{n}\mathrm{g}\mathrm{w}\mathrm{a}\mathrm{i}\mathrm{t}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{i}\mathrm{n}\mathrm{y}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{s}$
Proximity	0.079	0.069	0.075	0.059	$\{\begin{array}{l}1,d\le 501-\frac{0.15}{200}(d-50),50<d\le 2500.85-\frac{0.6}{250}(d-250),250<d\le 5000.25-\frac{0.25}{4681}(d-500),500<d\le 51810,5181\end{array}$ where d is the distance between donor hospital and transplant center in NM. Multiply by 3 if KDPI >85

This table defines the final rating scales considered during the project and the attribute weights of 4 optimized continuous distribution policies. Continuous distribution policies prioritize transplant candidates for a particular donated organ according to a weighted average of rating scale subscores. The attribute weights are the weights used in the weighted average.

cPRA, calculated panel-reactive antibody; EPTS, expected posttransplant survival; KDPI, Kidney Donor Profile Index; NM, nautical miles.

The Allocation Policy Design Process

Traditionally, policy development begins with creating 5–10 policy options for prioritizing candidates. These policies, along with the current policy, are compared by simulating the transplant system under each one. Policymakers review fairness metrics, such as the number of transplants received by various groups of candidates in simulations of each policy, clinical metrics, such as the number of grafts that fail within 1 y, and many others. If a policy appears to improve over the current policy, it may be implemented; otherwise, policymakers may create additional policies to simulate.

The new policy design process consists of criteria specification, rating scale analysis, rating scale design, and weight optimization. During criteria specification, policymakers determine the criteria used to evaluate policies. This includes requirements, such as maintaining current pediatric transplant access, as well as the relative importance of competing goals, such as that between travel distance and geographic disparity.

Next, policymakers select initial attributes and rating scales. These determine the subscores that, when combined by weighted average, prioritize candidates. To explore the outcomes achievable with a set of rating scales, thousands of policies are simulated that vary the weight placed on each subscore. This step, referred to as rating scale analysis, provides a wealth of information about policy design choices.

If key criteria cannot be met by any policy simulated during rating scale analysis, rating scales are adjusted in a rating scale design phase. For instance, as discussed in further paragraphs, the initial blood type rating scale was redesigned when simulation demonstrated that it was incapable of maintaining the historical level of access to blood type B candidates. Adjusting rating scales and evaluating them with simulation may be repeated until satisfactory rating scales are identified.

Once rating scales are selected, weight optimization is performed. Here, multiobjective optimization algorithms are used to select attribute weights that best meet OPTN goals as defined by the evaluation criteria discussed previously. For a diagram comparing the 2 processes, see Figure 1.

FIGURE 1.

Comparison of processes for policy development. Traditionally, policy development begins with the design of about 10 allocation policy options. The transplant system is simulated under these policies and the results are presented to the Organ Procurement and Transplantation Network (OPTN) committee by written report. The policy options are either revised for additional rounds of simulation or recommended for implementation. Policymakers have limited information when designing policies, generally leading to multiple rounds of simulation, each of which can take several months. Rather than starting by fully specifying policies, the new process begins with specifying policy selection criteria and a set of attributes and rating scales. Rating scales are analyzed through the simulation of thousands of policies that use the proposed rating scales but vary the attribute weights. These results inform changes to policy selection criteria and to attribute rating scales. Finally, attribute weights are optimized according to the committee’s policy selection criteria.

Analytical Tools

High-speed Simulation

For decades, computer simulation has been the main tool used for predicting the effects of an allocation policy on the US transplant system. Simulation uses a representation of the transplant system called a simulation model (see Figure 2). Starting from the waitlist on a particular date, a simulation proceeds through a sequence of donated organs. For each organ, patients are screened and prioritized for transplant offers according to the allocation policy under study. If a patient receives and accepts an offer, they receive the transplant. Because an allocation policy affects the likelihood of certain candidates receiving an offer, it can be evaluated by studying the population of patients who received a transplant during simulation.

FIGURE 2.

The KPSAM 2019 and MITSAM transplant system model. The simulation model implemented in the KPSAM 2019 and MITSAM simulators begins with the waitlist of patients on a given start date (shown as January 1, 2020). The model then progresses through a series of donated kidneys or pancreata, allocating organs to patients on the waitlist. The allocation process involves first screening and prioritizing waitlisted patients for transplant offers according to the rules of the allocation policy under study. Then, patients sequentially accept or refuse transplant offers with a probability determined by an acceptance model. The allocation policy determines the order in which patients receive offers, thereby influencing the likelihood that a certain population of patients receives a transplant. A transplant outcome model predicts the days until graft failure, death, or relisting. MITSAM uses the same offer acceptance model, transplant outcome model, and structure as KPSAM 2019. The key difference between MITSAM and KPSAM 2019 is the high-speed simulation algorithm used by MITSAM to implement this model. More details on the relationship between MITSAM and KPSAM 2019 are available in the Supplemental Materials (SDC, https://links.lww.com/TXD/A752).

Traditionally, designing an allocation policy involved around 10 simulations at a time. Performing rating scale analysis in the new approach involves thousands. Simulation is carried out by software known as a simulator. Such extensive simulation was made possible by a novel high-speed algorithm used in a new simulator, MITSAM, and the use of a supercomputing cluster.⁸ The algorithm accelerates simulation by performing the calculations involved in simulating the transplant system more efficiently. It is important to distinguish between the algorithm that executes a simulation and the simulation model that represents the transplant system.

To match the accuracy of simulators used historically to support OPTN policymaking and to provide for direct performance comparison, the same model used by the Scientific Registry of Transplant Recipient’s (SRTR) KPSAM 2019 was implemented in MITSAM. KPSAM 2019 was the most recent simulator made available to us and was used by the OPTN in designing the current kidney allocation policy. MITSAM includes pancreas allocation as KPSAM does.

The performance of the MITSAM algorithm is not limited to the simulation model used by KPSAM 2019 or the continuous distribution policy framework. By making computationally intensive operations across practical transplant system models more efficient, the algorithm is capable of accelerating any simulator.

This study used data from the SRTR. The SRTR data system includes data on all donor, waitlisted candidates, and transplant recipients in the United States, submitted by the members of the OPTN. The Health Resources and Services Administration, US Department of Health and Human Services, provides oversight to the activities of the OPTN and SRTR contractors.

Multiobjective Optimization

During the weight optimization phase of the new policy design process, multiobjective optimization methods are used to find the best organ allocation policy from a set of policy options given evaluation criteria. The main elements of this process are (1) the set of policy options, (2) how evaluation criteria were set based on OPTN committee goals, and (3) the algorithm used to find optimized attribute weights.

In the continuous distribution framework, a natural set of policy options arises for a particular set of attributes and rating scales, namely the set of policies with different attribute weights. Optimization identifies values of attribute weights, and hence a specific continuous distribution policy, that perform best according to evaluation criteria.

The evaluation criteria are specified in collaboration with OPTN committees and United Network for Organ Sharing (UNOS) analysts. This entails the selection of transplant system metrics, used to quantify outcomes of interest, and constraints and objectives that represent the goals of the OPTN committee during optimization. A constraint is a requirement that is either met or not met and an objective aims to maximize or minimize the value of a metric. For example, consider the goal of maintaining high access for pediatric transplant candidates. The metric used to quantify pediatric access was the number of transplants received by pediatric candidates within the 1-y simulation period. Represented as a constraint, this could mean requiring the number of pediatric transplants to be at least the number under the current policy. As an objective, this could mean maximizing the number of pediatric transplants.

Weight optimization was performed as follows. Starting with simulation results for 50 000 policies, the algorithm filtered out policies that failed to meet constraints. Then it computed the improvement of the remaining policies on each objective relative to the greatest observed in the data set. It removed policies that failed to improve upon all objectives by at least some specified minimum. The policy with the largest total relative improvement across objectives was used to create variants to simulate and add to the initial policies.

Policy Analysis and Design Website

A policy analysis and design website was created to support the OPTN Kidney Transplantation Committee and UNOS analysts. The results of each round of rating scale analysis were uploaded to this site. The website allowed simulation results to be shared quickly and for committee members and policy analysts to have greater analysis capabilities than the traditional written simulation reports. The tool is available at transplants.mit.edu. A screenshot is in Figure 3.

FIGURE 3.

Transplant allocation policy design tool. A screenshot of the Tx allocation policy design tool, publicly available at transplants.mit.edu. cPRA, calculated panel-reactive antibody; Tx, transplant.

The website has several capabilities. Users can analyze simulation results of previously proposed policies, current policy, and any of the most recent 50 000 policies simulated during rating scale analysis. Users can inspect >20 metrics across nearly 100 candidate subgroups. Results can be displayed in bar graphs, scatter plots, or histograms similar to those in the results section. Users can also design allocation policies and view predicted simulation results. Finally, an optimization tool allows users to perform real-time optimization, identifying policies that meet customizable objectives and constraints.

RESULTS

Simulation Algorithm

The novel simulation algorithm was implemented in MITSAM. To allow for direct speed comparison and to match the accuracy of results provided by SRTR, MITSAM implements the transplant system model of KPSAM 2019, although it could implement any future model. Detailed validation between MITSAM and KPSAM 2019 was performed and confirms that their results align across a wide array of metrics. The greatest difference observed was in the number of transplants across calculated panel-reactive antibody (cPRA)⁹ subgroups (see Supplemental Materials and Tables S1 and S2, SDC, https://links.lww.com/TXD/A752). The time required to simulate the transplant system for 1 y under the current US allocation policy was measured in KPSAM 2019 and MITSAM. Both were run on the same desktop computer with an Intel Core i7-6700 CPU, 24GB of RAM, running Windows 10. KPSAM 2019 required 6 h and 42 min. MITSAM took 14.8 s, >1600 times faster.

Rating Scale Analysis

In this section, we present results of a rating scale analysis conducted for the OPTN Kidney Transplantation Committee. Simulations were performed in MITSAM for 50 000 continuous distribution policies using the attributes and rating scales in Table 1. For definitions of transplant system metrics, see Table 2.

TABLE 2.

Definition of transplant system metrics

Metric	Definition
Waitlist mortality	The number of candidates who died within 14 d of being listed on the waitlist during simulation
Transplants	The number of transplants that occurred during simulation
cPRA 99.9–100 transplants	The number of transplants that occurred during the simulation for candidates with a cPRA between 99.9 and 100 at the time of transplant. These are candidates estimated to be incompatible with 99.9%–100% of the donor population due to immunosensitivity
EPTS 0–20 transplants	The number of transplants that occurred during the simulation for candidates with an EPTS value between 0 and 20, inclusive, at the time of transplant. These are candidates who have the longest EPTS
Pediatric transplants	The number of simulated transplants that occurred for candidates <18 y of age at the time of listing
Transplant rate	The number of transplants that occurred during simulation for candidates within a certain group (eg, blood type A candidates) is divided by the sum of qualifying time accrued by all candidates within that group
Average waiting time at transplant	The average waiting time of transplant recipients at the transplant throughout the simulation
≥5 y wait transplant percentage	The percentage of candidates who accrued >5 y of qualifying wait time during the simulation that received a transplant during the simulation
Median travel distance	The median distance between the hospitals of transplant recipients and organ donors during the simulation
1-y graft failures	The number of simulated transplants that failed within 1 y of transplant
Blood type transplant rate disparity	The difference between the highest and lowest transplant rate among A, B, AB, and O blood type candidates during the simulation
cPRA transplant rate disparity	The difference between the highest and lowest transplant rate during the simulation among the following cPRA groups: 0%, 0%–20%, 20%–80%, 80%–98%, 98%–99.5%, 99.5%–99.9%, and 99.9%–100%
Geographic transplant rate disparity	Twice the SD in simulated transplant rates between DSAs
Racial transplant rate disparity	The difference between the highest and lowest simulated transplant rate among candidates grouped according to race in the following categories: Black, Asian, White, and Other Race
Sex transplant rate disparity	The difference between the simulated transplant rate of male and female candidates
Latino/non-Latino transplant rate disparity	The difference between the transplant rates of Latino candidates and the transplant rate of non-Latino candidates during the simulation
Group wait time	The total qualifying time accrued by all candidates within a certain group over the simulation is divided by the number of transplants that group received
Geographic wait time disparity	Twice the SD in simulated group wait times between DSAs

Definitions of kidney transplant system metrics used in the article.

cPRA, calculated panel-reactive antibody; DSA, Donation Service Area; EPTS, expected posttransplant survival.

First, we report how several rating scales were found to affect system metrics. Each point in the plots in Figure 4 corresponds to 1 of 50 000 simulated policies. Figure 4A shows how increasing the proximity efficiency attribute weight caused the median travel distance to decrease. Figure 4B shows how increasing the waiting time attribute weight increased the percentage of candidates waiting ≥5 y who received a transplant. Figure 4C shows the manner in which the pediatric attribute weight increased the number of pediatric transplants. Finally, Figure 4D shows how the DR mismatch attribute weight reduced the number of 1-y graft failures.

FIGURE 4.

The effects of several attributes on important Tx system metrics. Each point corresponds to simulation results for 1 of 50 000 randomly sampled continuous distribution policies that use the rating scales defined in Table 1. A point’s horizontal coordinate represents the attribute weight for the policy, and its vertical coordinate is the average value of a Tx system metric >50, 1-y simulations. A, Increasing the proximity efficiency attribute weight decreased the median travel distance. B, Increasing the waiting time attribute weight increased the percentage of candidates waiting for ≥5 y who received a Tx. C, Increasing the pediatric attribute weight increased the number of pediatric Tx. D, Increasing the DR mismatch attribute weight reduced the number of 1-y graft failures. Tx, transplant.

In certain cases, proposed rating scales did not have the intended effect. For instance, the blood type rating scale was meant to compensate for imbalanced donor pools of different blood types. The first version proposes O, A, B, and AB candidates receive scores of 0.00446, 0.00005, 0.00111, and 0. Figure 5A shows that this rating scale failed to maintain the number of blood type B transplants that occurred in simulations of the current policy. Blood type B candidates already have reduced access, so a decrease was unacceptable. Results of a redesigned rating scale (in Table 1) are shown in Figure 5B. Using the previous policy design process, it would have been difficult to determine that the initial rating scale was unable to maintain blood type B access. Similar issues were identified regarding the longevity matching and cPRA attributes.

FIGURE 5.

The effects of different blood type rating scales on access for blood type B candidates. The blood type attribute is intended to compensate for unequal availability of donated organs for candidates with different blood types. A, The initial rating scale failed to maintain the number of blood type B Tx observed in simulations of the current policy (dashed line). Because blood type B candidates already have reduced access, this decrease was deemed unacceptable. B, The results of a redesigned blood type rating scale (defined in Table 1) that addressed this issue. Using the previous policy design process, it would have been difficult to detect that the initial rating scale reduced blood type B access. Tx, transplant.

The previously mentioned results show how attributes affect the metrics they were intended to target. However, spillover effects on other metrics were also found. Figure 6 shows a weight-metric correlation matrix. Each correlation captures the direction in which a metric tended to change in response to increasing the attribute weight. The color coding shows whether increasing the attribute weight tended to improve (blue) or worsen (red) the value of each metric. As an example of a spillover effect, consider the waiting time attribute designed to prioritize candidates who have been waiting the longest. The strong correlation between the waiting time attribute weight and the average waiting time at transplant (r = +0.87) indicates that the attribute performs as intended. However, increasing the waiting time attribute weight was also strongly correlated (r = –0.81) with decreasing transplants for patients with the greatest expected posttransplant survival (EPTS¹⁰ 0–20), an unintended spillover effect.

FIGURE 6.

Weight-metric correlation matrix. The correlation between attribute weights and important 1-y simulation metrics for 50 000 simulated policies using rating scales in Table 1. Metric definitions are in Table 2. A positive correlation between an attribute and a particular metric means that increasing the attribute’s weight tends to increase the value of that metric. The color coding reflects whether increasing the attribute weight corresponds to an improvement in metrics. Blue represents improvement and red represents worsening. cPRA, calculated panel-reactive antibody; EPTS, expected posttransplant survival.

When balancing competing goals, it is important to quantify how they interact. Several tradeoff curves in Figure 7 show relationships between system metrics. Figure 7A quantifies the relationship between the number of transplants received by candidates with high EPTS (0–20) and the percentage of candidates waiting ≥10 y that received transplants. Figure 7B quantifies the relationship between travel distance and geographic disparity. Figure 7C quantifies the relationship between transplant access for highly immunosensitized candidates (cPRA, 99.9%–100%) and overall disparity among the different immunosensitivity levels. In Figure 8, we repeat the disparity-distance tradeoff but using 2 metrics for disparity, discussed in further paragraphs.

FIGURE 7.

Three tradeoffs between Tx system metrics. Each point corresponds to simulation results for 1 continuous distribution policy. The results are from 1-y simulations of 50 000 randomly sampled continuous distribution policies using the rating scales defined in Table 1. The simulated current policy (“Ref”) and the 4 optimized policies defined in Table 1 are also shown. The optimized policies simultaneously meet many constraints and objectives, so they do not necessarily appear in the efficient frontier of these 2-dimensional plots. A, Quantified relationship between the number of Tx received by candidates with high EPTS (0–20) and the percentage of candidates waiting ≥10 years who received Tx. B, Quantified relationship between travel distance and geographic disparity. C, Quantified relationship between Tx access for highly immunosensitized candidates (cPRA 99.9%–100%) and overall disparity among the different immunosensitivity levels. For metric definitions, see Table 2. cPRA, calculated panel-reactive antibody; EPTS, expected posttransplant survival; Tx, transplant.

FIGURE 8.

Metric definition. A, The same disparity-distance tradeoff shown as in Figure 7A but with policies on the efficient frontier highlighted. These are “best-in-class” policies in the sense that no simulated policy simultaneously reduced geographic Tx rate disparity and travel distance. B, The same policies are highlighted, which show the same tradeoff but with geographic disparity measured with geographic wait time disparity. When using this alternative metric, these policies are no longer “best-in-class.” See Table 2 for definitions of these 2 metrics. Tx, transplant.

Criteria Specification and Policy Optimization

During criteria specification, the goals of the Kidney Transplantation Committee were translated into objectives and constraints for a number of transplant system metrics. Four alternative sets of evaluation criteria were selected and these are listed in Table 3. Each required maintaining the number of transplants for pediatric, cPRA 99.9%–100%, and blood type B candidates found in simulations of current policy, as well as maintaining average waiting time at transplant (to ensure candidates with longer wait times are prioritized). All 4 sets of criteria also sought to minimize waitlist mortality, 1-y posttransplant graft failures, transplant rate disparities across blood types, Donation Service Areas, and racial groups. They differ in the constraints placed on median travel distance and on the number of EPTS 0–20 transplants.

TABLE 3.

Definition of objectives and constraints for optimized policies

Objectives
Transplant system metric	Direction of improvement
Waitlist mortality	Minimize
1-y graft failures	Minimize
Blood type transplant rate disparity	Minimize
Geographic transplant rate disparity	Minimize
Racial transplant rate disparity	Minimize
Constraints
Transplant system metric	Direction	Threshold
Pediatric transplants	No less than	Current policy (1036 [Tx])
cPRA 99.9–100 transplants	No less than	Current policy (452 [Tx])
Average waiting time at transplant	No less than	Current policy (5.85 [y])
Blood type B transplants	No less than	Current policy (1806 [Tx])
Median travel distance	No greater than	Policy A2	current policy (134 [NM])
		Policy B2	110% current policy (147 [NM])
		Policy C2	110% current policy (147 [NM])
		Policy D2	125% current policy (168 [NM])
EPTS 0–20 transplants	No less than	Policy A2	current policy (3888 [Tx])
		Policy B2	current policy (3888 [Tx])
		Policy C2	97% current policy (3771 [Tx])
		Policy D2	97% current policy (3771 [Tx])

In this table, we present how the OPTN committee goals were represented in the optimization algorithms used to create the 4 optimized policies A2, B2, C2, and D2.

EPTS, expected posttransplant survival; NM, nautical miles; OPTN, Organ Procurement and Transplantation Network; Tx, transplant.

Weight optimization was performed using these 4 sets of criteria, resulting in 4 policies (A2, B2, C2, and D2) with attribute weights found in Table 1. MITSAM simulation results for these policies are shown in Figure 9. Each policy met the constraints set by the OPTN committee and achieved notable improvements in access disparity metrics. Policies A2, B2, C2, and D2 reduced geographic transplant rate disparity by 6%, 11%, 14%, and 26%, blood type transplant rate disparity by >50% and racial transplant rate disparity by as much as 10%. Full results are available at transplants.mit.edu.

FIGURE 9.

Optimized policy simulation results. One-year MITSAM simulation results for 5 allocation policies: simulated current policy (“Ref”) and the 4 proposed policies defined in Table 1. Results are averaged over 50 simulations. For metric definitions, see Table 2. cPRA, calculated panel-reactive antibody; Tx, transplant.

DISCUSSION

The purpose of this research was to develop new methodologies and analytical tools to support the development of organ allocation policy. A new high-speed, generalizable simulation algorithm was created that, when implementing the models of the preexisting simulator used by the OPTN, KPSAM 2019, increased simulation speed >1600 times. This greatly enhances the capabilities of computation in the design of organ allocation policy.

In particular, it enabled a new process for policy development based on multiobjective optimization. This was applied in practice by the OPTN Kidney Transplantation Committee. The committee was able to identify and converge on key objectives, and policies were successfully designed that met the committee’s evaluation criteria. In addition, by exploring hundreds of thousands of policies rather than a handful, policymakers had much greater insight into the effects of their design choices. For instance, they could identify that certain rating scales were incapable of achieving their objectives, as with the initial blood type rating scale. Furthermore, the availability of extensive simulation data provided policymakers with stronger evidence to support the selection of any particular policy.

Policy Objectives

Optimization methods use metrics to quantify the goals of policymakers for automated policy design. Selecting these metrics is critical for the optimization process—they must accurately reflect the concerns of policymakers. In some cases, this is easily achieved. The goal of increasing pediatric transplants, for instance, can be clearly defined using the number of pediatric transplants per year. However, a metric for equitable transplant access could be defined in many ways. Figure 8 compares 2 metrics for geographic disparity. The highlighted policies may be considered “best-in-class” with respect to the first disparity metric (for each highlighted policy, no policy reduces both travel distance and disparity simultaneously); however, they are not according to the second metric. There is progress to be made in selecting intuitive metrics that closely capture underlying concerns.

Once metrics are defined, policymakers determine target values for the metrics through constraints and objectives. It was not always possible to meet constraints using certain rating scales, prompting rating scale redesigns as with the blood type rating scale. Meeting objectives is less black-and-white than constraints. Maximizing improvement in 1 metric tends to prevent maximal improvement of another, even requiring worsening of another. Figure 7 depicts objectives that came into conflict. Figure 7A shows that increasing transplant access for candidates with relatively high EPTS reduced the percentage of candidates with >5 y of waiting time that received a transplant. Figure 7B shows that decreasing travel distance increased geographic disparity. Figure 7C shows that increasing the transplant rate for highly immunosensitive candidates also increased the overall transplant rate disparity across all immunosensitivity levels.

The tradeoffs considered by the OPTN in the 4 optimized policies are illustrated in Figure 7. Policies B2, C2, and D2 allow increased travel distance in exchange for reduced geographic disparity, small decreases in transplants for candidates with high EPTS (0–20), for increased transplants for candidates with long wait times, and finally, they maintain historical levels of cPRA access disparity to maintain high access for the most immunosensitive group (cPRA 99.9%–100%). Policy A2 does not increase travel distance or reduce transplants for EPTS 0–20 candidates, but, to a lesser degree, still reduces geographic disparity and increases transplants for candidates with long wait times.

The Role of Optimization in Allocation Policy Design

To an observer, using optimization methods to design an allocation policy may be troubling; the attribute weights selected through optimization may seem counterintuitive. It should be noted that allocation policies operate at the candidate level (by assigning ranks to individuals for a particular donor organ), whereas the goals of the committee operate at the system level. The relationship between the 2 is complex. The effects of individual attributes are intricate, as evidenced in Figure 4, and interrelated, as shown in the heatmaps of Figure 5. An attribute meant to target 1 metric can, in fact, impact many others. For example, increasing the proximity efficiency weight not only decreases travel distance but it also increases geographic disparities and decreases transplant access for immunosensitive populations. Conversely, 1 metric may be influenced by many attributes. Pediatric transplant access depends on the pediatric, longevity matching, and proximity efficiency weights.

Due to the complexity of the relationships between system-level goals and candidate-level allocation policies, it is difficult to manually find a policy that best meets the goals of the committee. Hence, selecting (or interpreting) attribute weights in isolation as a proxy for ethical judgments is cautioned. One may believe they have made an appropriate decision at the attribute level, but in practice, the system-level behavior may be quite different. Thus, it may be more effective to use simulation results and well-selected metrics to evaluate the implications of a proposed policy and optimization techniques to design it.

Limitations

Although these advances are promising in their ability to aid in the design of improved organ allocation policy, challenges remain. The optimization of organ allocation policies is performed with respect to an underlying simulation model of the transplant system. These models encompass complex phenomena such as graft survival and clinical decision making. As noted previously, the simulator used in this work, MITSAM, replicates SRTR’s KPSAM 2019 simulator. Although KPSAM 2019 has been used by the OPTN for policy development, detailed validation of its models against historical data has not been released by SRTR. Hence, it is difficult to assess its accuracy. As such, it is appropriate to consider the simulation results reported here as illustrative and informative, but with caution. SRTR has released some validation of other models.^11,12

Furthermore, it is imperative that the underlying simulation model can capture the key variables of interest to policymakers. KPSAM 2019 was designed such that the number of transplants performed per year does not change, regardless of the allocation policy under simulation. Capturing the relationship between allocation policy and kidney utilization is important to avoid optimization blind to this factor, especially considering substantial rates of nonuse of deceased-donor kidneys in recent years. Using simulators that ignore these critical effects may result in policies that exacerbate nonuse upon implementation.

Research to develop validated models that capture the underlying mechanisms of nonuse is ongoing. In fact, the computational advances in simulation described here are also significant for the development of more accurate simulation models. As this work progresses, high-speed simulation and multiobjective optimization will remain valuable tools for policymakers to improve organ allocation policy and the US organ transplant system.

ACKNOWLEDGMENTS

The authors thank Dr Martha Pavlakis and Dr Jim Kim, chair and vice-chair, as well as the entire Kidney Transplantation Committee, for their insightful advice and feedback. They also thank Lauren Milechin of Massachusetts Institute of Technology’s Supercloud for her technical support and Ted Papalexopolous for offering his experience and expertise.

Supplementary Material

txd-11-e1780-s001.pdf ^{(672.8KB, pdf)}