Acute Graft‐Versus‐Host Disease After Orthotopic Liver Transplantation: Predicting This Rare Complication Using Machine Learning

Abstract

Acute graft‐versus‐host disease (GVHD) is a rare complication after orthotopic liver transplantation (OLT) that carries high mortality. We hypothesized that machine‐learning algorithms to predict rare events would identify patients at high risk for developing GVHD. To develop a predictive model, we retrospectively evaluated the clinical features of 1938 donor‐recipient pairs at the time they underwent OLT at our center; 19 (1.0%) of these recipients developed GVHD. This population was divided into training (70%) and test (30%) sets. A total of 7 machine‐learning classification algorithms were built based on the training data set to identify patients at high risk for GVHD. The C5.0, heterogeneous ensemble, and generalized gradient boosting machine (GGBM) algorithms predicted that 21% to 28% of the recipients in the test data set were at high risk for developing GVHD, with an area under the receiver operating characteristic curve (AUROC) of 0.83 to 0.86. The 7 algorithms were then evaluated in a validation data set of 75 more recent donor‐recipient pairs who underwent OLT at our center; 2 of these recipients developed GVHD. The logistic regression, heterogeneous ensemble, and GGBM algorithms predicted that 9% to 11% of the validation recipients were at high risk for developing GVHD, with an AUROC of 0.93 to 0.96 that included the 2 recipients who developed GVHD. In conclusion, we present a practical model that can identify patients at high risk for GVHD who may warrant additional monitoring with peripheral blood chimerism testing.

Abbreviations

AHN

acute hepatic necrosis

AIH

autoimmune hepatitis

ALD

alcohol‐related liver disease

ATG

antithymocyte globulin

AUROC

area under the receiver operating characteristic curve

BMI

body mass index

CIT

cold ischemia time

CMV

cytomegalovirus

CVA

cerebrovascular accident

DM

diabetes mellitus

EBV

Epstein‐Barr virus

EGBT

extreme gradient boosting tree

EHR

electronic health record

GGBM

generalized gradient boosting machine

GI

gastrointestinal

GVHD

graft‐versus‐host disease

HBV

hepatitis B virus

HCC

hepatocellular carcinoma

HCV

hepatitis C virus

HLA

human leukocyte antigen

HLA‐A

human leukocyte antigen A

HLA‐B

human leukocyte antigen B

HLA‐C

human leukocyte antigen C

HLA‐DR

human leukocyte antigen DR

HLA‐DR51/52/53

human leukocyte antigen DR51/52/53

HLA‐DQ

human leukocyte antigen DQ

MMF

mycophenolate mofetil

NASH

nonalcoholic steatohepatitis

NK

natural killer

NPV

negative predictive value

OLT

orthotopic liver transplantation

PPV

positive predictive value

SLKT

simultaneous liver‐kidney transplantation

UWMC

University of Washington Medical Center

Acute graft‐versus‐host disease (GVHD) is a rare and serious complication after orthotopic liver transplantation (OLT) and is caused when donor lymphocytes contained within the transplanted liver encounter alloantigens become activated and mount an immune response to tissues of the recipient.^{( 1} , ^{2 )} GVHD can affect the recipient skin, gastrointestinal (GI) tract, and bone marrow.^{( 3} , ⁴ , ^{5 )} The incidence of GVHD after OLT is estimated at 0.1% to 2%, and the observed mortality rate is >75%.^{( 6} , ⁷ , ^{8 )} GVHD usually appears within 2 to 8 weeks after OLT, and the diagnosis is often delayed.^{( 5} , ^{8 )} The signs and symptoms of GVHD are nonspecific and include a skin rash, nausea and emesis, loss of appetite, diarrhea, weight loss, and cytopenias. Marrow involvement by GVHD and the resulting cytopenias drive the most common causes of death—sepsis and hemorrhage.^{( 1} , ⁸ , ⁹ , ^{10 )}

Previously reported risk factors for the development of GVHD are based on small numbers of recipients who developed GVHD, which constrain statistical power and have made it difficult to identify recipients who are at increased risk for GVHD.^{( 6} , ⁸ , ¹¹ , ¹² , ¹³ , ¹⁴ , ^{15 )} We hypothesized that using a statistical method designed to predict rare events would identify which patients will be at high risk for GVHD using data available at the time of OLT. To this end, we retrospectively evaluated the clinical features of 1938 recipients who underwent OLT at our center, 19 of whom developed GVHD. We report the results of our analysis using a machine‐learning approach for fraud detection, which yielded models to predict which recipients were at high risk of developing GVHD using recipient‐specific, donor‐specific, and transplant‐specific characteristics available at the time of OLT.

Patients and Methods

Donors and Recipients

This study was approved by the Institutional Review Board of the Human Subjects Division at the University of Washington. From January 17, 1996, to April 13, 2019, 1992 consecutive OLTs were performed at the University of Washington Medical Center (UWMC). Of these, 33 recipients and 21 donors did not have human leukocyte antigen A (HLA‐A), human leukocyte antigen B (HLA‐B), or human leukocyte antigen DR (HLA‐DR) loci results available, and those donor‐recipient pairs were excluded. The remaining 1938 donor‐recipient pairs were included in our analysis as the study data set to construct our machine‐learning algorithms using a fraud detection technique.

From May 1, 2019, to May 1, 2020, an additional 75 consecutive recipients underwent OLT at the UWMC, and these 75 donor‐recipient pairs were collected as a validation data set.

Clinical Outcome Measures

The primary clinical outcome for this study was whether a patient was diagnosed with GVHD after OLT, and the secondary clinical outcome was survival after OLT. A diagnosis of GVHD required histologic evidence of GVHD on biopsy and biopsy sites included skin, oropharyngeal mucosa, upper and/or lower GI tract mucosa, and bone marrow. Chimerism testing of leukocytes in peripheral blood to determine proportions that were of liver donor origin was performed using polymerase chain reaction amplification of single tandem repeat loci after lineage‐specific fractionation via flow cytometry. Although chimerism testing showing that a significant proportion of leukocytes were of liver donor origin was supportive of a diagnosis of GVHD, it was not required for a diagnosis of GVHD.

Machine‐Learning Analysis

All variables were categorical variables after the continuous variables were converted into categories after binning into groups. Categorical variables were given as number and percentage. Few variables had missing data, but for those variables, “unknown” was recorded as the value. No variables had imputation of values due to lack of adequate imputation methods for those variables. A random pattern was found for the missing variables. The data labeled with the most unknowns were the human leukocyte antigen C (HLA‐C) and human leukocyte antigen DR51/52/53 (HLA‐DR51/52/53) loci and lymphocyte cross‐matching between the recipient and donor.

Our study population of adult OLT recipients included those undergoing whole‐liver transplantation, simultaneous liver‐kidney transplantation (SLKT), and liver retransplantation. Recipients <18 years of age as well as those undergoing multivisceral transplantation were excluded from our analysis. Variables selected for analysis were based on prior published studies on the development of GVHD and survival after OLT.^{( 1} , ⁶ , ⁷ , ⁸ , ⁹ , ¹⁰ , ¹⁵ , ¹⁶ , ^{17 )} Many variables were created by combining donor to recipient data included blood type ABO matching (identical, compatible, incompatible, and unknown), cytomegalovirus (CMV) and Epstein‐Barr virus (EBV) serostatus matching, age differences, and donor race/ethnicity to recipient race/ethnicity matching. The human leukocyte antigen (HLA) combination for each HLA loci evaluated both for the recipient and the donor mismatches at that loci. A recipient could have 0, 1, or 2 mismatches to the donor and a donor could also have 0, 1, or 2 mismatches to the recipient. For example, the HLA‐A loci could be coded as HLA‐A loci 1‐2, meaning the HLA‐A loci had a mismatch of the recipient to the donor of 1 and the donor had a mismatch to the recipient of 2.

All statistical analyses were performed using R version 3.5.1 (R Foundation for Statistical Computing, Vienna, Austria) as well as the Boruta 7.0.0, Classification and Regression Training (caret) 6.0‐84, and caret Ensemble 2.0.0 packages.^{( 18 )} Descriptive statistics to assess differences between the recipients with or without GVHD were analyzed using the chi‐square test. Kaplan‐Meier unadjusted survival curves were used to compare patient survival between patients who developed GVHD after OLT and those who did not. All P values were 2‐sided, and statistical significance was defined as P < 0.05.

Because of the unbalanced data set with relative few recipients developing GVHD, we used a machine‐learning anomaly or fraud detection technique to classify patients at high risk for developing GVHD. This technique mainly calls for balancing the training set and standardizing the values. In preparing the data set for predictive machine‐learning algorithms, we split the study data set into a training set and a test set. We performed a random 70/30 split to preserve the proportion of GVHD in the training and test sets. For our classifier algorithms to learn better, we balance the training set by oversampling from recipients with GVHD and undersampling from recipients without GVHD. The test set was not balanced.

All collected variables were entered into the Boruta algorithm to find the most important variables for predicting the outcome. We found that including these important variables and those variables from the descriptive statistics with a P value ≤0.20 resulted in the best model performance in our machine‐learning algorithms. These variables in our training data set were used to train a total of 6 classification algorithms (logistic regression, neural network, generalized gradient boosting machine [GGBM], extreme gradient boosting tree [EGBT], adaptive boosting, and C5.0; see the Predictive Classifier Algorithms section) and an heterogenous ensemble (combining predictions of 2 or more algorithms) to predict the recipients at high risk for developing GVHD. The caret (Classification and Regression Training 6.0‐84) package was used to build the classification algorithms. Each algorithm had a specific method in the caret package: logistic regression used the glm method, neural network used nnet, GGBM used gbm, regularized gradient boosting tree used xgbTree, adaptive boosting used Adaboost, and C5.0 used the C5.0 method. Hyperparameter tuning was performed using a grid search and 10‐fold cross‐validation to determine the best selection of hyperparameters to train each algorithm on the training data set.

Each trained algorithm was then evaluated on the test data set. Our goal was to label the fewest recipients at high risk and to capture at least 80% of those recipients who develop GVHD, so we tuned our model to have a sensitivity of 0.80 and the lowest detection prevalence. Detection prevalence measures the proportion of recipients labeled high risk by the predictive algorithm. The area under the receiver operating characteristic curve (AUROC), specificity, positive predictive value (PPV), and negative predictive value (NPV) were also recorded for each model. The better performing algorithms became our production model in our health care database and can be used on our Clinical and Bio‐analytic Transplant Laboratory Web site (https://cbatl.shinyapps.io/GVHD).

The 75 donor‐recipient pairs in our validation data set were collected after the algorithms were developed on the training set and scored on the test set. Data from the validation set were collected at a later time than the training and test set to determine how the models would work on data never seen by the algorithms.

Predictive Classifier Algorithms

Generalized linear model with a binomial distribution and a logit link function performs logical regression that allows tuning of parameters. A neural network or deep‐learning is a classifier algorithm that clusters raw input into different layers to recognize complex patterns in data.^{( 19 )} Neural network reviews the raw data repeatedly to find the best output and has been used for several years in predicting various medical conditions.^{( 20 )} Both the GGBM and EGBT repeatably build new model decision trees from the errors of prior models and then add models together using a gradient descent algorithm to minimize loss in making the final prediction. One difference is that the GGBM focuses on the difference in variance of the models, whereas the EGBT focuses on the regularization factor.^{( 21 )} Adaptive boosting develops weighted sums of weak learner decision trees. The weak learners are modified over subsequent stages, and the growing algorithm in later trees focuses on improving the model’s performance.^{( 22 )} This algorithm can handle many variables and has been used to predict renal cell carcinoma from gene expression.^{( 23 )} C5.0 algorithm builds either a decision tree or a rule set.^{( 24 )} The data are split into smaller samples that provide information gain. The subsamples are split again and again until no more splits can be performed. Any split that does not provide value to the predicting model is removed. This technique has become widely used in medical research.^{( 25 )}

Results

Study Data Set Characteristics Associated With Development of GVHD

From January 17, 1996, to April 13, 2019, 1938 recipients underwent OLT at UWMC and were included in the model creation portion of our study. Of these recipients, 19 (1.0%) developed GVHD (Table 1). All recipients in our study data set were followed for at least 1 year after OLT (median follow‐up 12.8 years; interquartile range, 10.7 years).

TABLE 1.

Data Set Recipient, Donor, and Transplant Characteristics

Recipient, Donor, and Transplant Characteristics	GVHD (n = 19), n (%)*	No GVHD (n = 1919), n (%)*	P Value
Recipient characteristics
Age ≥50 years	17 (89.5)	1321 (68.8)	0.08
Female sex	3 (15.8)	605 (31.5)	0.21
Retransplantation	—	96 (5.0)	0.99
Race/ethnicity
Asian	2 (10.5)	113 (5.9)	0.31
Black	1 (5.3)	55 (2.9)	0.42
Hispanic	—	44 (2.2)	0.99
White	15 (78.9)	1471 (76.7)	0.99
Other race/ethnicity ◊	1 (5.3)	238 (12.3)	0.72
Diagnosis
AHN	—	39 (2.0)	0.99
AIH	1 (5.3)	68 (3.5)	0.50
Cholestatic †	2 (10.5)	204 (10.6)	0.99
Cryptogenic/NASH	1 (5.3)	233 (12.2)	0.63
HBV	2 (10.5)	90 (4.7)	0.23
HCV	11 (57.9)	934 (48.7)	0.49
Other	—	124 (6.5)	0.63
ALD ± other diagnosis ‡	8 (42.1)	664 (34.6)	0.48
HCC ± other diagnosis §	12 (63.2)	574 (29.9)	0.004
DM at time of OLT
No	10 (52.6)	1024 (53.4)	0.99
Type I	—	46 (2.4)	0.99
Type II	5 (26.3)	264 (13.8)	0.17
Unspecified type	4 (21.1)	585 (30.5)	0.46
BMI, kg/m2
<18.5	1 (5.3)	13 (0.7)	0.13
18.5‐30.0	13 (68.4)	1149 (59.9)	0.49
30.1‐35.0	2 (10.5)	505 (26.3)	0.19
>35.0	3 (15.8)	252 (13.1)	0.73
ABO blood group
A	8 (42.1)	774 (40.3)	0.99
B	—	241 (12.6)	0.16
AB	2 (10.5)	69 (3.6)	0.15
O	9 (47.4)	835 (43.5)	0.82
CMV serostatus
Negative	6 (31.6)	619 (32.3)	0.99
Positive	13 (68.4)	1273 (66.3)	0.99
Unknown	—	27 (1.4)	0.99
EBV serostatus
Negative	1 (5.3)	50 (2.6)	0.40
Positive	17 (89.4)	1733 (90.3)	0.71
Unknown	1 (5.3)	136 (7.1)	0.99
Donor characteristics
Age, years
0‐17	1 (5.3)	185 (9.6)	0.99
18‐40	9 (47.4)	953 (49.7)	0.99
41‐55	7 (36.8)	531 (27.7)	0.44
>55	2 (10.5)	250 (13.0)	0.99
Sex
Female	5 (26.3)	725 (37.8)	0.35
Male	14 (73.7)	1121 (58.4)	0.24
Unknown	—	73 (3.8)	0.99
Race/ethnicity
Asian	1 (5.3)	96 (5.0)	0.99
Black	—	71 (3.7)	0.99
Hispanic	—	84 (4.4)	0.99
White	14 (73.7)	1422 (74.1)	0.99
Other race/ethnicity ◊	4 (21.1)	95 (5.0)	0.01
Unknown	—	151 (7.9)	0.39
Cause of death
Anoxia	2 (10.5)	288(15.0)	0.99
CVA	6 (31.6)	255 (13.3)	0.03
Trauma	3 (15.8)	382 (19.9)	0.99
Other	—	36 (1.9)	0.99
Unknown	8 (42.1)	958 (49.9)	0.65
Death confirmation before donation
Circulatory	2 (10.5)	149 (7.8)	0.66
Neurologic	17 (89.5)	1430 (77.1)	0.28
Unknown	—	290 (15.1)	0.10
BMI, kg/m2
<18.5	1 (5.3)	80 (4.2)	0.56
18.5‐30.0	10 (52.6)	1286 (67.0)	0.22
30.1‐35.0	4 (21.1)	237 (12.4)	0.28
>35.0	3 (15.7)	144 (7.5)	0.17
Unknown	1 (5.3)	172 (9.0)	0.99
CMV serostatus
Negative	9 (47.4)	720 (37.5)	0.48
Positive	10 (52.6)	1157 (60.3)	0.49
Unknown	—	42 (2.2)	0.99
EBV serostatus
Negative	1 (5.2)	113 (5.9)	0.99
Positive	9 (47.4)	794 (41.4)	0.64
Unknown	9 (47.4)	1012 (52.7)	0.65
HCV serostatus
Negative	12 (63.2)	1094 (57.0)	0.65
Positive	—	25 (1.3)	0.99
Unknown	7 (36.8)	800 (41.7)	0.82
Liver fat content at retrieval, % ||
0	1 (5.3)	155 (8.1)	0.99
1‐5	2 (10.5)	242 (12.6)	0.99
6‐19	4 (21.1)	142 (7.4)	0.049
≥20	—	117 (6.1)	0.62
Unknown	12 (63.2)	1263 (65.8)	0.81
Transplantation characteristics
CIT, minutes ¶
0‐333	7 (36.8)	457 (23.8)	0.18
334‐426	3 (15.8)	460 (24.0)	0.59
427‐547	6 (31.6)	449 (23.4)	0.42
548‐2200	3 (15.8)	450 (23.4)	0.59
Unknown	—	103 (5.4)	0.62
Whole‐liver transplantation #	19 (100.0)	1897 (98.9)	0.99
Combined liver/kidney transplant**	—	82 (4.3)	0.99
≥1 rejection episode §§	5 (26.3)	521 (27.2)	0.99
Induction immunosuppression ††
None	4 (21.1)	995 (51.9)	0.01
Basiliximab	5 (26.3)	385 (20.1)	0.56
ATG	10(52.6)	539 (28.1)	0.04
Maintenance immunosuppression ‡‡
Azathioprine	2 (10.5)	141 (7.4)	0.65
Cyclosporine	1 (5.3)	75 (3.9)	0.53
MMF	5 (26.3)	569 (29.7)	0.99
Prednisone	4 (21.1)	411 (21.4)	0.99
Tacrolimus	12 (63.2)	1107 (57.9)	0.82

^*From January 17, 1996, to April 13, 2019, 1992 patients underwent OLT at the UWMC. A total of 33 recipients and 21 donors did not have HLA‐A, HLA‐B, or HLA‐DR loci results available, and those recipient‐donor pairs were excluded. Our analysis included 19 patients who developed GVHD after OLT (GVHD) and 1919 patients who did not develop GVHD after OLT (no GVHD)

^† Includes primary biliary cirrhosis and primary sclerosing cholangitis.

^‡ Diagnosis was labeled ALD if the primary liver disease was from alcohol or alcohol was associated with any other diagnosis.

^§ Diagnosis was labeled HCC if the recipient had primary HCC or HCC associated with any other disease.

^|| Donor liver fat content was assessed by histology at the time of organ procurement.

^¶ CIT was measured as the time in minutes from when the donor liver was placed on ice at procurement to the time it was removed from ice for transplantation.

^# Includes patients who underwent transplantation of a whole‐liver graft. The remaining patients underwent transplantation of a partial liver graft.

^**Includes patients who also received a kidney graft at the time of liver transplantation.

^†† Includes all agents administered as intensive perioperative prophylactic immunosuppression used to prevent acute cellular rejection in the first months after transplantation.

^‡‡ Includes all agents administered after induction as maintenance immunosuppression within the first 30 days after transplantation.

^§§ Requiring histological evidence and medication treatment to confirm rejection episode.

^◊ Includes American Indian, Alaska Native, Native Hawaiian/Pacific Islander, or multiracial recipients and/or donors.

For recipient characteristics, the presence of hepatocellular carcinoma (HCC) either as the primary diagnosis or associated with other diagnosis of liver disease occurred significantly (P = 0.004) more often in those recipients developing GVHD (n = 12, 63.2%) than those recipients who did not develop GVHD (n = 574, 29.9%). Recipients were more often aged ≥50 years (P = 0.08) in those developing GVHD (n = 17, 89.5%) than those not developing GVHD (n = 1321, 68.8%).

For donor characteristics, other race (which included American Indian, Alaska Native, Native Hawaiian/Pacific Islander, or multiracial) was significantly (P = 0.01) associated with those who developed GVHD (n = 4, 21.1%) compared with those who did not develop GVHD (n = 95, 5.0%). Cause of donor death from a cerebrovascular accident (CVA) occurred significantly more often (P = 0.03) in those who developed GVHD (n = 6, 31.6%) than in those who did not develop GVHD (n = 255, 13.3%). A donor liver having 6% to 19% fat on histology occurred significantly more often (P = 0.049) in those who developed GVHD (n = 4, 21.4%) than in those who did not (n = 142, 7.4%).

Regarding transplant characteristics, a GVHD occurred more often in recipients who had induction therapy compared with those who did not have induction therapy (51.9% versus 21.1%; P = 0.01). When specific induction therapy agents were evaluated, the use antithymocyte globulin (ATG) was significantly associated with the development of GVHD (P = 0.04). There was no significant difference in the proportions of recipients who developed GVHD after basiliximab induction therapy compared with those recipients who received basiliximab induction therapy and did not develop GVHD.

Comparison of donor and recipient combination variables in recipients who developed GVHD to recipients who did not develop GVHD are shown in Table 2. A White recipient transplanted with a liver graft from an “other” donor race, which included American Indian, Alaska Native, Native Hawaiian/Pacific Islander, or multiracial recipients, occurred at a different frequency (P = 0.005) in recipients who developed GVHD (n = 4, 21.1%) as compared to those who did not develop GVHD (n = 69, 3.6%). No HLA‐B loci mismatch (HLA‐B loci 0‐0) either between the donor to recipient or recipient to donor occurred more frequently (P = 0.007) in recipients who developed GVHD (n = 2, 10.5%) than in recipients who did not develop GVHD (n = 11, 0.6%). A donor to recipient or recipient to donor with an HLA‐DR mismatch of 2‐1 also occurred at a different frequency (P = 0.05) in recipients who developed GVHD (n = 5, 26.3%) than in recipients who did not develop GVHD (n = 212, 11.1%).

TABLE 2.

Donor‐Recipient Combination Variables

Donor‐Recipient Combination	GVHD (n = 19), n (%)*	No GVHD (n = 1919), n (%)*	P Value
Blood group matching †
Identical	15 (78.9)	1666 (86.7)	0.30
Compatible	3 (15.8)	159 (8.3)	0.21
Incompatible	—	11 (0.7)	0.99
Unknown	1 (5.3)	83 (4.3)	0.57
CMV serostatus matching, donor‐recipient
Negative‐negative	4 (21.1)	228 (11.9)	0.27
Negative‐positive	5 (26.3)	483 (25.2)	0.99
Positive‐negative	2 (10.5)	376 (19.6)	0.56
Positive‐positive	8 (42.1)	766 (39.9)	0.82
Unknown	—	66 (3.4)	0.99
EBV serostatus matching, donor‐recipient
Negative‐negative	1 (5.3)	112 (5.8)	0.99
Positive‐negative	1 (5.3)	24 (1.3)	0.22
Positive‐positive	8 (42.1)	748 (39.0)	0.82
Unknown	9 (47.4)	1035 (53.9)	0.65
Race/ethnicity matching, donor‐recipient
Same race/ethnicity ‡	10 (52.6)	1149 (58.3)	0.50
Asian‐White	1 (5.3)	66 (3.4)	0.50
Hispanic‐White	1 (5.3)	65 (3.3)	0.49
Other‐White	4 (21.1)	69 (3.6)	0.005
White‐Asian	2 (10.5)	80 (4.1)	0.19
White‐Black	1 (5.3)	39 (2.0)	0.33
White‐White	10 (52.6)	1127 (57.2)	0.65
Lymphocyte cross‐matching ‡
Negative	7 (36.8)	653 (34.0)	0.81
Positive	—	118 (6.2)	0.63
Unknown	12 (63.2)	1148 (59.8)	0.82
HLA loci mismatches §
HLA‐A
0‐0	2 (10.5)	67 (3.5)	0.15
0‐1	1 (5.3)	84 (4.4)	0.56
1‐0	—	79 (4.1)	0.99
1‐1	5 (26.3)	583 (30.4)	0.81
1‐2	—	189 (9.9)	0.25
2‐1	2 (10.5)	191 (10.0)	0.99
2‐2	9 (47.4)	726 (37.8)	0.48
HLA‐B
0‐0	2 (10.5)	11 (0.6)	0.007
0‐1	—	30 (1.6)	0.99
1‐0	—	38 (2.0)	0.99
1‐1	3 (15.8)	298 (15.5)	0.99
1‐2	2 (10.5)	142 (7.4)	0.65
2‐1	1 (5.3)	192 (10.0)	0.99
2‐2	11 (57.9)	1208 (63.0)	0.64
HLA‐C
0‐0	1 (5.3)	67 (3.5)	0.49
0‐1	1 (5.3)	82 (4.3)	0.57
1‐0	—	92 (4.8)	0.99
1‐1	3 (15.8)	412 (21.5)	0.78
1‐2	1 (5.3)	165 (8.6)	0.99
2‐1	1 (5.3)	151 (7.9)	0.99
2‐2	8 (42.1)	654 (34.1)	0.47
Unknown	4 (21.1)	296 (15.4)	0.80
HLA‐DR
0‐0	1 (5.3)	43 (2.2)	0.35
0‐1	—	55 (2.9)	0.99
1‐0	—	60 (3.1)	0.99
1‐1	6 (31.6)	535 (27.9)	0.80
1‐2	—	154 (8.0)	0.39
2‐1	5 (26.3)	212 (11.1)	0.05
2‐2	7 (36.8)	860 (44.8)	0.64
HLA‐DR51/52/53
0‐0	1 (5.3)	333 (17.4)	0.23
0‐1	4 (21.1)	326 (17.0)	0.55
1‐0	2 (10.5)	292 (15.2)	0.76
1‐1	8 (42.1)	520 (27.1)	0.19
1‐2	1 (5.3)	153 (8.0)	0.99
2‐1	—	149 (7.8)	0.39
2‐2	—	26 (1.4)	0.99
Unknown	3 (15.8)	120 (6.3)	0.12
HLA‐DQ
0‐0	—	118 (6.2)	0.63
0‐1	—	118 (6.2)	0.63
1‐0	2 (10.5)	94 (4.9)	0.24
1‐1	8 (42.1)	687 (35.8)	0.63
1‐2	1 (5.3)	198 (10.3)	0.71
2‐1	3 (15.8)	286 (14.9)	0.76
2‐2	5 (26.3)	418 (21.8)	0.58

^*From January 17, 1996, to April 13, 2019, 1992 patients underwent OLT at the UWMC. A total of 33 recipients and 21 donors did not have HLA‐A, HLA‐B, or HLA‐DR loci results available, and those recipient‐donor pairs were excluded. Our analysis included 19 patients who developed GVHD after OLT (GVHD) and 1919 patients who did not develop GVHD after OLT (no GVHD).

^† Compatible and incompatible blood group matching is defined as previously described.

^‡ Lymphocyte cross‐matching performed by combining recipient serum with donor lymphocytes to detect preformed donor‐specific HLA antibodies.

^§ HLA combination for each HLA loci was evaluated both for recipient and donor mismatches at that loci. A recipient could have 0, 1, or 2 mismatches to the donor and a donor could also have 0, 1, or 2 mismatches to the recipient. For example, the HLA‐A loci could be coded as 1‐2, meaning the HLA‐A loci had a mismatch of the recipient to the donor of 1 and the donor had a mismatch to the recipient of 2.

Survival After OLT

For the recipients in our study data set, we compared survival after OLT in recipients who developed GVHD (n = 19) with those who did not (n = 1919). Recipients who developed GVHD after OLT had markedly different survival rates, with 40% surviving at 1 year compared with 91% in those who did not develop GVHD (Fig. 1, P < 0.001).

FIG. 1.

Unadjusted survival after OLT for patients with (solid line, n = 19) and without (dashed line, n = 1919) GVHD.

Characteristics of Recipients Who Developed GVHD

Characteristics of the 19 recipients from our study data set who developed histologically proven GVHD are shown in Table 3. The diagnosis of GVHD was made at a median of 31 days (interquartile range, 44.5 days) after OLT. Skin was the most commonly involved organ (100%), and 11 (57.9%) recipients had marrow involvement. Peripheral blood chimerism testing was performed in 14 (73.7%) recipients, and 12 of the 14 recipients had evidence of macrochimerism, which is defined as >1% lymphocytes of liver donor origin.^{( 5 )} The most common treatments were calcineurin inhibitors, systemic corticosteroids, and hematopoietic growth factors. A total of 12 (63.2%) recipients died during follow‐up, with 10 attributed to complications of GVHD. Of the 12 recipients who died, 9 (75.0%) had GVHD involvement of the marrow. In contrast, of the 7 patients who survived having GVHD, only 2 (28.6%) had GVHD involving the marrow. For the 10 patients who died from complications of GVHD, infection was the most common cause of death (n = 7, 58.3% of total deaths). All deaths attributed to GVHD occurred within 1 year after OLT.

TABLE 3.

Characteristics of 19 OLT Recipients Who Developed GVHD^*

Organs involved by GVHD, n (%) †
Skin	19 (100)
Marrow	11 (57.9)
Upper or lower GI mucosa	11 (57.9)
Patients with chimerism testing, n (%)	14 (73.7)
Median chimerism at GVHD diagnosis, % ‡	24 donor (range, 0‐100)
Treatments for GVHD, n (%)
Calcineurin inhibitor §	19 (100)
Systemic corticosteroids	19 (100)
Growth factor support	11 (57.9)
ATG	8 (42.1)
Alefacept	3 (15.8)
Topical GI steroids	3 (15.8)
Alemtuzumab	2 (10.5)
Allogeneic hematopoietic cell transplantation	2 (10.5)
Ruxolitinib	1 (5.3)
Etanercept	1 (5.3)
Vedolizumab	1 (5.3)
Extracorporeal photopheresis	1 (5.3)
Cyclophosphamide	1 (5.3)
Total deaths, n (%)	12 (63.2)
Deaths due to GVHD, n (% of total deaths) ||	10 (83.3)

^*For OLT performed at the UWMC from August 7, 1996, to May 1, 2019.

^† Organ involvement verified by histology.

^‡ Chimerism testing performed using polymerase chain reaction amplification of single tandem repeat loci after leukocyte fractionation via flow cytometry and quantifies the proportion of cells of liver donor origin.

^§ For 2 patients, the intensity of calcineurin inhibitor therapy was decreased as treatment for GVHD.

^|| Cause of death for 2 recipients was unknown, for the remaining 10 recipients who died, the cause of death was attributed to complications of GVHD.

Development of Machine‐Learning Models Using Training Data Set

To train the machine‐learning algorithms, we used a subset of the total variables (see Tables 1 and 2 for the total variables). This subset was determined through a dimension‐reduction strategy that selected important variables confirmed by the Boruta algorithm (data not shown) and variables from our chi‐square analysis with a P value ≤0.20 (see Table 4 for selected variables used in the machine‐learning algorithms). Our study data set was unbalanced because of the rare occurrence of GVHD: of 1938 total recipients, only 19 (1.0%) developed GVHD. After performing a 70/30 split in the study data set, the training set had 1358 recipient observations with 14 (1.0%) having GVHD. The test set had 580 observations with 5 (0.7%) recipients with GVHD. After balancing the training set for the final training of our algorithms, the set consisted of 1362 (50.1%) observations without GVHD and 1354 (49.9%) observations with GVHD. Each classification algorithm was trained on the balanced training set after determining the best hyperparameters for each algorithm.

TABLE 4.

Variables Used in the Machine‐Learning Models^*

Recipient characteristics

Age

BMI

HCC ± other diagnosis

Race/ethnicity

DM at time of OLT

ABO blood type

Donor characteristics

BMI

Cause of death

Liver fat content at retrieval

OLT characteristics

CIT

Induction immunosuppression

Donor‐recipient combination

HLA loci mismatches

Race/ethnicity matching

^*This subset of variables was determined through a dimension‐reduction strategy that selected important variables confirmed by the Boruta algorithm and significant variables from descriptive statistics (see Tables 1 and 2 for the full list of descriptive variables).

The performance of each algorithm for identifying recipients with GVHD on the unbalanced test data set is provided in Table 5. The models were ranked by AUROC value. C5.0 had the best AUROC of 0.86 with a sensitivity of 0.80 and a detection prevalence of 0.21. This means that this model would identify 21% of recipients at high risk for developing GVHD. The second‐best model ranked by an AUROC of 0.84 was the heterogenous ensemble developed by averaging the prediction of the other 6 algorithms. This model had a detection prevalence of 0.28. The GGBM had a slightly lower AUROC of 0.83 and a detection prevalence of 0.27. The adaptive boosting (Adaboost) algorithm could not be tuned to a sensitivity of 0.80, and the AUROC was 0.72.

TABLE 5.

Comparison of Statistical Algorithm Performance on the Test Data Set

Algorithm	AUROC	Sensitivity	Detection Prevalence	Specificity	PPV	NPV
C5.0	0.86	0.8	0.21	0.79	0.03	1.0
Heterogenous ensemble	0.84	0.8	0.28	0.73	0.03	1.0
GGBM	0.83	0.8	0.27	0.73	0.03	1.0
Logistic regression	0.79	0.8	0.37	0.64	0.02	1.0
EGBT	0.78	0.8	0.23	0.78	0.03	1.0
Neural network	0.77	0.8	0.35	0.65	0.02	1.0
Adaptive boosting	0.72	0.6	0.15	0.85	0.03	1.0

Application of Machine‐Learning Models to Validation Data Set

A validation data set was collected from May 1, 2019, to May 1, 2020, immediately after the training and test data set collection dates. The validation data set included 75 consecutive donor‐recipient pairs who underwent OLT at UWMC. During this time, 2 (2.7%) recipients developed GVHD after OLT. All recipients in the validation data set were followed for at least 3 months after OLT (median follow‐up, 8.2 months; interquartile range, 4.1 months).

Model performance on the validation data set is shown in Table 6. The logistic regression algorithm performed best with an AUROC of 0.96 and a detection prevalence of 0.09. The next‐best models were heterogenous ensemble and GGBM, both with AUROCs of 0.93 and detection prevalences of 0.11. Based on donor‐specific, recipient‐specific, and transplant‐specific characteristics at the time of OLT, both recipients in the validation data set who developed GVHD were predicted to be at high risk by our models.

TABLE 6.

Comparison of Statistical Algorithm Performance for Our Validation Data Set

Algorithm	AUROC	Sensitivity	Detection Prevalence	Specificity	PPV	NPV
Logistical regression	0.96	1.0	0.09	0.93	0.29	1.0
Heterogenous ensemble	0.93	1.0	0.11	0.92	0.25	1.0
GGBM	0.93	1.0	0.11	0.92	0.25	1.0
Neural network	0.91	1.0	0.19	0.84	0.14	1.0
EGBT	0.90	1.0	0.13	0.89	0.20	1.0
Adaptive boosting	0.90	1.0	0.13	0.89	0.20	1.0
C5.0	0.84	1.0	0.27	0.75	0.10	1.0

Discussion

Using a cohort of 1938 recipients who underwent OLT at our center from 1996 to 2019, 19 (1.0%) of whom developed GVHD, we developed 7 machine‐learning models to predict which recipients were at high risk for developing GVHD after OLT using donor‐specific, recipient‐specific, and transplant‐specific characteristics available at the time of OLT. We then applied these machine‐learning models to a more recent cohort of 75 recipients at our center, which included 2 who developed GVHD. Of our best models, 3 predicted that 9% to 11% of these validation recipients were at high risk for developing GVHD with AUROCs of ≥0.93, which included the 2 recipients who developed GVHD.

A summary of the donor‐specific, recipient‐specific, and transplant‐specific characteristics we identified as significantly associated with the development of GVHD is shown in Table 7. Of these characteristics, both HLA matching at the HLA‐B locus and recipient age ≥50 years have been previously reported as risk factors for the development of GVHD.^{( 8} , ^{14 )} Shared HLA antigens are the most important risk factors for the induction of GVHD and have been shown to be associated with GVHD after OLT.^{( 7} , ¹² , ^{13 )} Evasion of the recipient immune system by passenger T lymphocytes in the donor liver is the presumed mechanism by which shared HLA antigens contribute to the development of GVHD.^{( 26} , ²⁷ , ^{28 )}

TABLE 7.

Factors Associated With the Development of GVHD After OLT

Recipient characteristics

Presence of HCC

Age ≥50 years*

Donor characteristics

Other race †

CVA as cause of death

Liver with 6%‐19% fat on histology

OLT procedural logistics

Receiving any form of induction immunosuppression, particularly with ATG

Donor‐recipient pair characteristics

White recipient receiving liver from donor of a different race/ethnicity †

Absence of HLA‐B loci mismatch either between donor to recipient or recipient to donor

Recipient HLA‐DR mismatch of 2‐1*

^*Trended toward significance in our analysis.

^† Other race category included American Indian, Alaska Native, Native Hawaiian/Pacific Islander, or multiracial.

Several of the factors we identified as significantly associated with the development of GVHD are either new or contrary to prior reports. Kitajima et al. recently evaluated 121 patients with fatal GVHD reported in the United Network for Organ Sharing database and found that induction therapy with basiliximab was an independent risk factor for fatal GVHD.^{( 16 )} Our analysis did not confirm this association and found that induction therapy with ATG and not basiliximab was significantly associated with the development of GVHD. The potential reasons for the discrepancy include our smaller number of patients with GVHD and our inclusion of all patients with GVHD, as opposed to only patients who had fatal GVHD.

Factors we identified as being associated with the development of GVHD, that have not previously been described, include CVA as the cause of donor death, donor race/ethnicity, and donor liver fat content at the time of retrieval. We can only speculate as to the potential reasons for these associations. Inflammation is a known trigger of acute GVHD after allogeneic hematopoietic cell transplantation,^{( 29 )} and both CVA and hepatic steatosis would be sources of inflammation in the donor that could activate or expand lymphocyte populations within the liver prior to retrieval. Recently, hepatic steatosis prior to allogeneic hematopoietic cell transplantation was shown to be associated with the development of chronic GVHD.^{( 30 )} Although an association between hepatic steatosis and the development of acute GVHD has not been described, we surmise that inflammation within a steatotic donor liver could contribute to donor lymphocyte activation and an increased risk of developing GVHD. Based on this rationale, it is interesting that only lower levels of hepatic steatosis (6%‐19%) were associated with the development of GVHD, and this suggests a more complex interaction between donor liver inflammation and metabolic dysfunction on the activation of donor lymphocytes and recognition of recipient HLA antigens. Patient ethnicity has been reported to influence the risk of acute GVHD after allogeneic hematopoietic cell transplantation.^{( 31} , ³² , ^{33 )} Although the genetic basis for this remains undefined, it is presumed to be attributed to differences between conserved HLA haplotype‐linked polymorphisms unique to each ethnic group.

Although immunological differences have been proposed for SLKT, compared with kidney transplantation or OLT alone, no recipient in our population who underwent SLKT subsequently developed GVHD.^{( 34 )} We included SLKT recipients in our training and test sets, and in the development of our machine‐learning algorithms, but SLKT was not found to be an important variable for predicting GVHD. We had no recipients who underwent SLKT during the time period of our validation data set.

Our machine‐learning models developed on the training set performed well on both the test and validation data sets. Having an AUROC >0.80 for each of the top 3 algorithms (C5.0, heterogenous ensemble, and GGBM) on the test data set indicated that these were clinically useful for GVHD prediction. These algorithms labeled 21% to 28% of the OLT recipients as high risk for GVHD. On the validation data set, logistic regression was the top model, followed closely by heterogenous ensemble and GGBM. These algorithms had an AUROC ≥0.93 and labeled 9% to 11% of the OLT recipients as high risk for GVHD. It is understandable that each model’s performance could change with data from a new era. The training set had data from 1996 to 2019, and during this 23‐year period many different protocols and other clinical changes occurred. The validation set had the latest 75 OLTs performed, and we will continue to follow all 7 algorithms on new data with a special focus on logistic regression, heterogenous ensemble, and GGBM. For a wider audience, our machine‐learning algorithms are available for use at our Clinical and Bio‐analytic Transplant Laboratory Web site (https://cbatl.shinyapps.io/GVHD). The variables included in Table 4 and on our Web site are all important for predicting GVHD and must be used in all analyses. Other collected variables were not found to have influence in predicting the risk of developing GVHD. Each of the machine‐learning algorithms has a different set of variables that are important for that algorithm’s predictions. Based on the different variable sets, the most important variables by relative weight for predicting risk were recipient age, body mass index (BMI), presence of HCC, and type of induction immunosuppression.

From a practical standpoint, a transplant program could use our machine‐learning algorithms in 3 different ways depending on their institution’s electronic health record (EHR). Without an EHR, the transplant program can use our GVHD Web site in the posttransplant period if a recipient develops a fever with an unknown source or a skin rash. If the algorithms predict that the recipient is at high risk, a GVHD workup could be initiated before further symptoms develop. If the transplant program has the algorithms deployed in their EHR, once all recipient, donor, and transplant variable data are entered, the program could identify those at high risk early to start surveillance for GVHD. Finally, if all data were collected at the time of donor allocation, with the cold ischemia time (CIT) and proposed induction therapy selected in either the institution’s EHR or a national allocation EHR, then those predicted to be at high risk for GVHD could either have their transplant variables modified to see if the risk level was lowered or allocation the donor liver to another recipient predicted to be at lower risk. If the EHR of the transplant program or national allocation organization had our algorithms deployed (only 1 algorithm would be sufficient) and if the important variables were entered in the EHR, then our algorithms could be used to potentially select donors or modify the few transplant variables that could be modified. For instance, if a program routinely used ATG induction and including ATG led to being at high risk, the program could select another induction method or no induction. Likewise, if a long CIT led to high risk for GVHD and modifying the time could be accomplished, then the transplant might proceed to limit the CIT.

One potential limitation of fraud analysis is inflating the importance of some variables that are not significant predictors. Also, by inflating these variables there is concern for overfitting the model on data collected during the same period of study. To overcome these limitations, we had 7 algorithms with good performance on data from a new era (the validation data set). This lowers the concern of an insignificant predicting variable dominating model performance and lowering model usefulness. Medical practice is always changing, and predictive algorithms will need to change as well. Our machine‐learning algorithms will be designed to improve predictions as more recipients receive transplants. When machine‐learning algorithms are deployed in an EHR, data from ongoing transplant recipients can be added to the machine‐learning training and test data sets to continually improve the models’ predictions. Only the best model needs to be deployed in the EHR. There are hundreds of machine‐learning classification algorithms or modifications of these algorithms. Each algorithm’s predictions might work better on 1 type of data and not as well on other types of data. We chose 7 basic types of machine‐learning algorithms to study to determine what algorithm worked best on our liver transplant data. On the validation set, the generalized logistical regression, the GGBM model, and combining all of the models to be used in the heterogenous ensemble were the better predictors. Only 1 algorithm could be chosen to be put into production. If the heterogenous model was the far superior model, then all of the algorithms would be developed to be combined into the final heterogenous model. With modern algorithm deployment, this would not be a burden on the EHR.

For recipients predicted to be at high risk for GVHD by our machine‐learning algorithms, we have developed an approach to monitor these recipients after OLT. Regular evaluation for signs and symptoms and obtaining an urgent biopsy if there is clinical suspicion of GVHD are routine practices for all recipients after OLT. However, for recipients predicted to be at high risk, we regularly hold multidisciplinary meetings to review their post‐OLT clinical course, including chimerism testing of the peripheral blood every 1 to 2 weeks for 2 months after OLT. Our multidisciplinary meetings involve transplant surgeons, hepatologists, hematologists, infectious disease specialists, and transplant pharmacists to ensure that a wide range of potential complications after OLT are considered, including infections and drug reactions, which are more common than GVHD. Performing chimerism testing of the peripheral blood at regular intervals is minimally invasive, and the presence of macrochimerism has been reported to be associated with the development of GVHD.^{( 5 )} Our chimerism testing includes evaluation for the proportions of CD3+ T cells, CD8+ T cells, and CD56+ natural killer (NK) cells of donor origin, as increased donor levels of these cell types are associated with both the diagnosis and severity of GVHD.^{( 35} , ^{36 )} For this current retrospective analysis, we only had chimerism data for recipients who developed GVHD. We are now beginning to collect chimerism data for recipients who are predicted to be at high risk for developing GVHD, and we look forward to reporting those results in the future.

GVHD involving the bone marrow can lead to marrow failure and life‐threatening cytopenias. In our cohort of 19 patients who developed GVHD, involvement of the marrow was more common in patients who died, which is consistent with prior reports.^{( 1} , ⁸ , ⁹ , ^{10 )} A diagnosis of GVHD involving the marrow is suggested based on findings from a bone marrow biopsy, including histologically hypocellular marrow, abnormally low myeloid blast count on flow cytometry, presence of an abnormal lymphocytic infiltrate via histology or flow cytometry, and/or chimerism testing of the marrow showing leukocytes of liver donor origin. We perform a marrow biopsy in any recipient who has unexplained peripheral blood cytopenias in the setting of suspected or confirmed GVHD. The evaluation of a marrow sample includes chimerism testing for T cells and NK cells of liver donor origin.

In summary, we show that a machine‐learning approach can predict which recipients are at high risk for developing GVHD after OLT based on factors known or measurable at the time of transplantation. For recipients predicted to be at high risk, we monitor closely for signs and symptoms of GVHD, including peripheral blood chimerism testing after OLT. Knowing which recipients are at high risk for developing GVHD may allow for changes in immunosuppression or immunomodulation at the time of OLT to reduce this risk.