Decision Support for Organ Offers in Liver Transplantation

Abstract

Organ offers in liver transplantation are high-risk medical decisions with a low certainty of whether a better liver offer will come along before death. We hypothesized that decision support could improve the decision to accept or decline. With data from the Scientific Registry of Transplant Recipients, survival models were constructed for 42,857 waiting-list patients and 28,653 posttransplant patients from 2002 to 2008. Daily covariate-adjusted survival probabilities from these 2 models were combined into a 5-year area under the curve to create an individualized prediction of whether an organ offer should be accepted for a given patient. Among 650,832 organ offers from 2008 to 2013, patient survival was compared by whether the clinical decision was concordant or discordant with model predictions. The acceptance benefit (AB)—the predicted gain or loss of life by accepting a given organ versus waiting for the next organ—ranged from 3 to −2 years (harm) and varied geographically; for example, the average benefit of accepting a donation after cardiac death organ ranged from 0.47 to −0.71 years by donation service area. Among organ offers, even when AB was >1 year, the offer was only accepted 10% of the time. Patient survival from the time of the organ offer was better if the model recommendations and the clinical decision were concordant: for offers with AB > 0, the 3-year survival was 80% if the offer was accepted and 66% if it was declined (P < 0.001). In conclusion, augmenting clinical judgment with decision support may improve patient survival in liver transplantation.

Liver transplantation is lifesaving for patients with end-stage liver disease, but it remains limited by the shortage of high-quality organs. Organ quality can be classified according to 2 types of donor-specific risks: (1) the risk of disease transmission, such as malignancy or infection, and (2) the risk of graft failure, which can vary from 19% to 40% by 3 years according to the organ received.¹

When an organ is offered, the transplant center and the potential recipient must decide whether to accept that offer or wait in hopes that a better organ will come along. These decisions are high-risk ones; a recent study revealed that 84% of the patients who die on the waiting list with Model for End-Stage Liver Disease (MELD) scores ≥ 15 had previously declined at least 1 organ offer.² These decisions are also complex ones. Physicians must incorporate multiple donor factors, recipient factors, and donor-recipient interactions as well as the local magnitude of the organ shortage and various technical and logistical concerns. Thus, it is perhaps not surprising that decisions about organ quality vary widely by transplant center, suffer from misprediction and cognitive bias, and are susceptible to external forces such as policy changes, regulatory scrutiny, and competition between centers.^3–7

Despite the modern era, physicians still evaluate the tens to hundreds of pieces of data in an organ offer with mental math and gestalt opinion. We hypothesize that the availability of a point-of-care decision aid could improve the consistency and accuracy of organ acceptance decisions. Such a tool would be intended not to replace clinical judgment but rather to augment it. In fact, the literature on physician decision support suggests that in many situations, it is the expert physician whose judgment is aided the most.⁸ This article describes the development and validation of a tool to predict acceptance benefit (AB)—the increase or decrease in predicted survival associated with accepting a given offer for a given patient versus waiting for the next available organ.

PATIENTS AND METHODS

Brief Summary

Using data from the Scientific Registry of Transplant Recipients (SRTR), survival models were constructed for 42,857 waiting-list patients and 28,653 posttransplant patients from 2002 to 2008. Daily covariate-adjusted survival probabilities from these 2 models were combined into a 5-year area under the curve to calculate AB. Importantly, patients were not censored at the time of receiving a liver transplant, and this quantified potential benefits from waiting for a better organ in some cases.

Model Development

This study used data from the SRTR. The SRTR data system includes data on all donors, wait-listed candidates, and transplant recipients in the United States submitted by the members of the Organ Procurement and Transplantation Network (OPTN), and it has been described elsewhere. The Health Resources and Services Administration of the US Department of Health and Human Services provides oversight for the activities of the OPTN and SRTR contractors.

The methodology was adapted from the survival benefit techniques of Schaubel et al.⁹ Patients from the SRTR were included if they were ≥18 years old and had received a liver transplant or were on the waiting list from 2002 to 2008. This time period was chosen to allow at least 3 years of follow-up for each patient, with a buffer of at least 1 year from the end of follow-up in order to maximize the completeness of the data. Patients listed as status 1 or with MELD exceptions other than hepatocellular carcinoma (HCC) were excluded in order to minimize heterogeneity in the subject population. Notably, a sensitivity analysis including these patients was also performed, and the results were not significantly changed, as reported later.

A Cox regression was used to create 2 separate models, (1) the waiting-list model and (2) the post-transplant model, with the Breslow method to estimate the baseline hazard function. The waiting-list model was created with 10 cross-sections of the waiting list at 8-month intervals during the study period. The outcome was death, and patients were censored upon the end of follow-up or the receipt of a living donor transplant. However, patients were not censored for any other reason, such as removal from the waiting list or receipt of a deceased donor transplant. This key difference from a traditional survival-benefit analysis permitted the model to account for what would happen if a given offer was declined and the patient was then later transplanted with a different organ. The posttransplant model began at the time of transplant and ended with death or the end of the follow-up period. In both models, a time horizon of 5 years was used.

Recipient covariates included the following: most recently assigned MELD score (match MELD score), recipient age, serum creatinine, serum bilirubin, serum international normalized ratio (INR), serum sodium, serum albumin, body mass index (BMI), dialysis, diabetes, chronic obstructive pulmonary disease (COPD), portal vein thrombosis, HCC, hepatitis C virus (HCV), prior abdominal surgery, prior liver transplant, prior nonliver transplant, prior malignancy, blood type, and percentage of time inactive on the waiting list (waiting-list model only). Donor covariates (posttransplant model only) included donor age, donation after cardiac death (DCD), donor cause of death [anoxia, cerebrovascular accident (CVA), and other], donor height, donor weight, history of malignancy, hepatitis C, split liver, regional or national share, and cold ischemia time (CIT). Notably, because CIT is not available at the time of the organ offer and must be estimated, we performed a sensitivity analysis by comparing the results with CIT excluded. Other covariates included the donation service area (DSA) and the interaction between donor age and recipient HCV status. Other interactions, such as the interaction between donor and recipient height and weight, were not significant in a bivariate analysis and were thus excluded from the final model. Candidates and recipients who were missing data on creatinine, bilirubin, INR, or albumin at the time of the cross-section or transplant were excluded from the analysis [919 candidates (2%) and 8 recipients (<0.01%)]. Data for creatinine, bilirubin, INR, and albumin were log-transformed and centered; values for age and CIT were centered before entry in the models. Indicator variables were created and included in the models for missing values for each of the following covariates: BMI (<1.5% missing), sodium (38%), and CIT (9%). The percentage of missing serum sodium values was high because collection of this data field did not begin until 2004. All other variables had a very low rate of missing data and/or a low proportion of positive values; any misclassification bias caused by this should have biased the parameter estimates toward the null.

Subject-specific daily survival probabilities from the covariate-adjusted Cox models were combined into a 5-year area under the survival curve. The benefit (in years) of accepting a given organ versus staying on the waiting list with the possibility of a future transplant was calculated as follows: the posttransplant area minus the waiting-list area equals AB. This can be described in mathematical terms as follows.

The survival probability at a given time (t) for a given patient (i) after transplantation with a given organ (x) can be expressed as

$$S_{TX,i,q}(t)=S_{TX,0,o}{(t)}^{e^{\mathrm{\beta }1X1+\mathrm{\beta }2X2+\dots +\mathrm{\beta }\mathrm{\rho }X\mathrm{\rho }}}$$

where S_TX,i,q(t) is the posttransplant survival probability at time t for patient i and organ q. S_TX,0,o(t) is the baseline posttransplant survival probability at time t (for the average patient and organ). β₁, β₂, … β_p are the coefficients for variables in the posttransplant model described. The cumulative survival probability TX_i,q can be calculated for that same patient and reflects the sum of the daily survival probabilities, S_TX,i,q(t), for the study period (5-year area under the survival curve):

$${TX}_{i,q}=\sum _{t=1}^{1095}{S}_{TX,iq}(t)$$

Similarly, the survival measure WL_i can be calculated for that same patient if he or she were to turn down the organ offer with the daily survival probability, S_WL,i(t), based on the waiting-list model.

Finally, the survival benefit of a given patient accepting a given organ rather than waiting for another to come along is calculated as

$${\text{AB}}_{i,q}={TX}_{i,q}-{WL}_i$$

This number, which can range from −5 to 5, reflects the additional number of extra (or fewer) years a given patient would be expected to live with that particular organ versus remaining on the waiting list with the possibility of future transplantation. A positive number suggests that the organ offer should be accepted, whereas a negative number suggests that it should be declined.

Postestimation was then used to calculate predicted AB values for various recipient-donor combinations. Determining the degree of uncertainty in model predictions proved difficult because no established methods exist to express confidence intervals in a survival-benefit analysis. We, therefore, estimated the best-case scenario for AB by calculating the 25th percentile of estimated survival for the waiting-list model and the 75th percentile of estimated survival for the posttransplant model (and vice versa for the worst-case scenario). Recipient-donor combinations with nondiscrepant best-case/worst-case combinations (AB, both ≥0 and ≤0) were determined to be high-certainty predictions. We also chose 0.5 years as an a priori cutoff for significant benefit or harm because this is a clinically meaningful interval during which some patients may lose their window for transplantation. Finally, the independent contribution of each of the recipient and donor characteristics was assessed with linear regression with predicted AB as the dependent variable.

Model Validation

Several types of model validity, including concurrent, construct, criterion, and predictive validity, were assessed. Concurrent validity was assessed through the comparison of model predictions with those calculated by another statistical method—in this case, sequential stratification. An offshoot of Cox regression, sequential stratification can be used to compare the relative risk of patient mortality associated with transplant while accounting for time-dependent patient characteristics.¹⁰ With the same waiting-list cohort cited previously, patients who received a transplant were matched to subjects on the waiting list that day who did not receive a transplant but who had the same laboratory MELD score, were in the same DSA, and had been listed for the same amount of time. The cohort was then divided by MELD and donor risk index (DRI) quartiles, and this yielded 16 subgroups that were fitted by separate Cox regressions. The hazard ratio in each subgroup represents the relative risk associated with transplantation for a candidate in that MELD group receiving an organ in that DRI group. The hazard ratios from the sequential stratification for each subgroup were then compared with the AB scores from this cohort with linear regression. Notably, sequential stratification was not used for the primary analysis because the ultimate goal of this project is to create a point-of-care decision support tool, and the output from sequential stratification (hazard ratio) is not easily interpreted by patients or physicians.

Construct validity was assessed by the calculation of AB at various MELD score and DRI levels; we anticipated that AB would increase with an increasing MELD score and would decrease with an increasing DRI. We also analyzed AB by DSA and hypothesized that AB would vary on the basis of the local severity of the organ shortage. The criterion validity was assessed with match-run data from 2008 to 2013; we excluded offers labeled “B” for bypass and offers of organs that were not eventually used for transplantation. These offers were restricted to the top patient in each match run and thus did not include provisional offers or candidates lower on the list than the one who eventually accepted the organ. Among the 26,792 accepted offers, 26,045 patients (97.2%) underwent transplantation with that organ. We expected that with increasing AB, a higher proportion of offers would be accepted; thus, the criterion was the expert opinion of the transplant surgeon in each case. Predictive validity was assessed by an analysis of the patient survival after the organ offer according to whether the model predicted a positive or negative AB and whether or not the offer was accepted.

This study was approved by our institutional review board.

RESULTS

The waiting-list model included 117,741 observations among 42,857 individual patients, and it resulted in a C-statistic of 0.67 for predicting survival. The post-transplant model included 28,653 patients, and it resulted in a C-statistic of 0.63 for predicting survival. Among the patients in the waiting-list model who received a deceased donor liver transplant, there was a strong correlation between AB scores and hazard ratios from sequential stratification (r² = 0.90), and this indicated good concurrent validity of the model. A sensitivity analysis, which was performed by the inclusion of patients with non-HCC exceptions and status 1 patients and by the exclusion of CIT, demonstrated a strong correlation with the original results (Pearson correlation coefficients of 0.997 and 0.992, respectively).

As expected, median AB scores were higher among patients with higher MELD scores and were lower with organs having higher DRI scores, as shown in Fig. 1. The predicted benefit also varied by geographic location. For example, as shown in Fig. 2, for an average 54-year-old man with alcoholic cirrhosis and a MELD score of 22 being offered a DCD liver, the AB ranged from −0.71 to 0.47 years according to the DSA.

Figure 1.

AB by laboratory MELD score and DRI. The box indicates the interquartile range, with the horizontal line at the median, the diamond at the mean, and the whiskers indicating values that are 1.5 times the interquartile range.

Figure 2.

Predicted benefit of accepting a DCD liver for a 54-year-old man with a match MELD score of 22 by DSA.

The independent contributions of various donor and recipient characteristics to AB are shown in Table 1 with adjustments for DSA. A positive coefficient means having that characteristic is associated with an increased likelihood of benefit from accepting a given organ offer rather than waiting for another offer to come along. For example, after adjustments for laboratory measures of the severity of liver disease (creatinine, bilirubin, and INR), a higher match MELD score was associated with lower AB because those patients are more likely to receive additional offers in the near future. Conversely, older age was associated with higher AB, possibly because older patients tend to have a more rapid decline in their functional status. Several case examples are provided in Table 2.

TABLE 1.

Multivariate Model Demonstrating the Impact of Recipient and Donor Factors on AB

Variable	Parameter Estimate	Pr >| t|
Match MELD
11–15 (versus <11)	−0.0194	0.0005
16–20 (versus <11)	−0.0988	<0.0001
21–25 (versus <11)	−0.1603	<0.0001
26–30 (versus <11)	−0.1578	<0.0001
31–40 (versus <11)	−0.3368	<0.0001
Age at transplant (per year)	0.0196	<0.001
loge creatinine	0.1325	<0.0001
loge albumin	−1.1687	<0.0001
loge INR	0.0198	<0.0001
loge bilirubin	0.1948	<0.0001
Sodium, mEq/L
Sodium missing	−0.2015	<0.0001
Sodium < 130 (versus 134 ≤ sodium < 139)	0.3762	<0.0001
130 ≤ sodium < 134 (versus 134 ≤ sodium < 139)	0.1310	<0.0001
139 ≤ sodium < 131 (versus 134 ≤ sodium < 139)	−0.1315	<0.0001
141 ≤ sodium (versus 134 ≤ sodium < 139)	−0.1276	<.0001
BMI, kg/m2
BMI missing	−0.0046	0.67
BMI < 20 (versus 20 ≤ BMI < 25)	0.0832	<0.0001
25 ≤ BMI < 30 (versus 20 ≤ BMI < 25)	0.0096	<0.0001
30 ≤ BMI < 35 (versus 20 ≤ BMI < 25)	0.0669	<0.0001
35 ≤ BMI (versus 20 ≤ BMI < 25)	0.1971	<0.0001
Dialysis	−0.0023	0.52
Diabetes	0.0978	<0.0001
0% < Inactive % ≤ 10% (versus 0%)	0.1519	<0.0001
10% < Inactive % ≤ 40% (versus 0%)	0.1094	<0.0001
40% < Inactive % (versus 0%)	0.0234	<0.0001
COPD	0.2201	<0.0001
Portal vein thrombosis	−0.2090	<0.0001
HCC	−0.0323	<0.0001
HCV	−0.0920	<0.0001
Previous abdominal surgery	−0.0152	<0.0001
Previous liver transplant	−0.2690	<0.0001
Previous nonliver transplant	−0.5363	<0.0001
Previous malignancy	−0.0249	<0.0001
Blood type
A (versus O)	−0.0121	<0.0001
B (versus O)	−0.1578	<0.0001
AB (versus O)	−0.2320	<0.0001
Donor age (per year)	−0.0046	<0.0001
DCD donor	−0.29312	<0.0001
Donor cause of death
Anoxia (versus trauma)	−0.0030	0.27
CVA (versus trauma)	−0.0509	<0.0001
Donor height (cm)	0.0026	<0.0001
Donor weight (kg)	0.0002	0.0003
Donor race
African American (versus white)	0.0045	0.07
Hispanic (versus white)	−0.1260	<0.0001
Other (versus white)	−0.1985	<0.0001
Split liver transplant	−0.2322	<0.0001
Regional share transplant (versus local)	−0.06454	<0.0001
National share transplant (versus local)	−0.1152	<0.0001
Donor anti-HCV positive	−0.0840	<0.0001
Donor history of cancer	−0.1046	<0.0001
CIT centered at 8 hours	−0.0107	<0.0001
Missing CIT	−0.2149	<0.0001
Interaction: HCV × donor age	−0.0106	<0.0001

NOTE: A positive coefficient means that the variable is associated with an increased likelihood of benefit from accepting a given organ (vice versa for a negative coefficient).

TABLE 2.

Examples of Donor-Recipient Combinations and the Associated AB

Recipient	Donor	AB (Years)
55 years old, laboratory MELD 9, HCC exception, otherwise good candidate	50-year-old brain-dead donor, expected CIT 6 hours	0.11
55 years old, laboratory MELD 22 (bilirubin 7, INR 2, creatinine 1), otherwise good candidate	50-year-old brain-dead donor, expected CIT 6 hours	0.35
55 years old, laboratory MELD 9, HCC exception, otherwise good candidate	50-year-old cardiac death donor, expected CIT 6 hours	−0.15 (harm)
55 years old, laboratory MELD 22 (bilirubin 7, INR 2, creatinine 1), otherwise good candidate	50-year-old cardiac death donor, expected CIT 6 hours	0.14

NOTE: All examples are in the Michigan organ procurement organization.

Among organ offers from 2008 to 2013 that were eventually used for transplantation, the distribution of AB scores is shown in Fig. 3. A large proportion of offers had a predicted AB clustered near zero, and this highlights the difficulty in making these clinical decisions. However, 23% and 8% had an AB > 0.5 and an AB <–0.5, respectively, and this indicated a significant predicted benefit or harm from accepting that offer for that patient. Furthermore, in 44% of the offers, the best-case and worst-case AB probabilities were both either ≥0 or ≤0, and this indicated a high degree of certainty for the model prediction among those cases.

Figure 3.

Distribution of AB scores among liver offers from 2008 to 2013 (restricted to offers of organs that were eventually used for transplantation).

The proportion of offers that were actually accepted is shown in Table 3. There was a clear increase in the acceptance rate with increasing AB, and this supported the criterion validity of the model; however, it is interesting that even among organs eventually used for transplant, 90% of the offers were declined despite a predicted AB > 1 year. As shown in Fig. 4, if the predicted AB was <0 (the model predicted that the patient would be better off waiting), the actual patient survival from the time of the organ offer was marginally better if the offer was declined rather than accepted. Conversely, if the predicted AB was >0, the actual patient survival was substantially better if the offer was accepted rather than declined (3-year survival, 80% versus 66%; P < 0.001).

TABLE 3.

Proportion of Liver Offers Accepted by AB Score (Restricted to Offers of Organs That Were Eventually Used for Transplantation)

	Number	Accepted
AB (years)	of Offers	(%)
Overall	650,832	4.1
<–0.5 (harm)	54,466	1.5
−0.5 to −0.25 (harm)	79,096	2.1
−0.25 to 0 (harm)	136,157	2.5
0 to 0.25	132,387	3.6
0.25 to 0.5	98,858	4.7
0.5 to 1	103,596	6.5
>1	46,272	10.2

Figure 4.

Patient survival from the time of organ offer by predicted AB (>0 versus <0) and by whether or not the liver was actually accepted.

DISCUSSION

This study aimed to develop and validate the statistical methodology for a decision support tool for organ offers in liver transplantation. Our model demonstrated strong concurrent, construct, criterion, and predictive validity. Most importantly, patient survival was better if the model suggestion to accept or decline was followed in actual practice. This finding suggests that the use of this decision support tool at the time of the organ offer might improve overall outcomes in the liver transplant population. The next step will be to test this concept in real-world clinical practice.

It is striking that among organ offers with a predicted AB > 1 year, 90% were declined; it is particularly striking because patients with a positive AB had an improved 3-year survival of 80% if the organ was accepted versus 66% if the organ was declined. This finding might be explained by unmeasured covariates such as sarcopenia and infection¹¹; in other words, perhaps the clinical decision was the correct decision, and the patient would in reality have done poorly even with transplantation. However, this possibility seems unlikely to be the sole explanation; in our clinical experience, fewer than 1 in 5 patients on the waiting list are frail and marginal.¹² Unmeasured donor characteristics are also unlikely to account for all declined offers because each of these organs was eventually used for transplantation. Most importantly, the converse was also true: patients with a negative AB had better survival if the organ was declined rather than accepted. An alternate explanation is that these offers represent missed opportunities for successful transplantation because of well-intended errors in decision making.¹³ The decision to decline may also be related to regulatory or financial considerations; recent evidence has pointed to risk aversion as an unintended consequence of regulatory oversight.⁶ Ultimately, a prospective study will be the only way to know for sure which explanation is the correct one and whether the decision tool would improve decision making in actual practice.

Although a strength of this study lies in the use of the SRTR database to allow for the analysis of the entire national experience with liver transplantation, a limitation is the database’s lack of granularity, particularly with respect to additional information such as donor liver biopsy results, donor laboratory trends, donor hemodynamic stability, and other factors that surgeons may use to make decisions about the suitability of a given organ for a given recipient. This lack of granularity makes it difficult to understand the context in which the organ offer decisions were made. For example, in many situations, an offer may be declined for one patient because of plans to later accept it for another patient; it is difficult to piece together such scenarios with large databases. Another limitation of the study was the relatively modest predictive ability of the statistical model. Although it is difficult with this methodology to generate confidence intervals around each predicted AB, one can assume reasonable confidence in the direction of the model prediction in 44% of organ offers, as evidenced by concordance between best-case and worst-case estimates. Furthermore, it is reassuring that the results appeared very similar when a different statistical method (sequential stratification) was used. A third important limitation with respect to future implementation in clinical practice is that organ allocation policy changes over time. For example, in the past 10 years, there have been changes in priority for patients with HCC as well as regional sharing rules for patients with MELD scores > 15 and > 35. This means that the model would need to be updated periodically in order to reflect current reality. However, despite these policy changes, the model (which was developed from 2002–2008 data) seemed to outperform clinical judgment in the time period from 2008 to 2013. Finally, the model endpoint is survival, which has the advantage of being objective, but it does not account for important subjective endpoints such as quality of life.

In summary, our findings support the proof of concept that decision support may be a useful adjunct to clinical judgment when organ offers are being considered. We emphasize that this tool is never intended to replace clinical judgment. Furthermore, this was a retrospective study, and this tool is not yet ready for routine clinical practice. In addition to further model improvement, the next step will be to study this support tool prospectively in order to determine its effectiveness and feasibility. To this end, we have converted the tool into a web-based interface, which can be seen at https://dev.ltorganoffer.org/demo/index/ (the username and the password are user and demo, respectively). We welcome feedback from the transplant community about improvements for the next iteration, which will include information about potential regulatory impacts from the decision as well as other considerations.

Abbreviations

AB

acceptance benefit

BMI

body mass index

CIT

cold ischemia time

COPD

chronic obstructive pulmonary disease

CVA

cerebrovascular accident

DCD

donation after cardiac death

DRI

donor risk index

DSA

donation service area

HCC

hepatocellular carcinoma

HCV

hepatitis C virus

INR

international normalized ratio

MELD

Model for End-Stage Liver Disease

OPTN

Organ Procurement and Transplantation Network

SRTR

Scientific Registry of Transplant Recipients