Abstract
Introduction
There are significant sex, racial, ethnic, and socioeconomic disparities in kidney transplantation in the United States, but upstream factors driving these disparities are understudied because prewaitlisting data are not yet routinely collected by national data surveillance systems.
Methods
To address this gap, we developed a rule-based semiautomated natural language processing (NLP) pipeline to extract prewaitlist milestones from nephrologist and social worker clinical notes at a nonprofit dialysis organization in New York City. Our outcomes were (i) transplant discussed with the patient; (ii) patient interest in transplant; (iii) transplant referral; and (iv) receipt of a kidney transplant. In a retrospective observational cohort study of patients with incident end-stage kidney disease (ESKD) receiving dialysis, we evaluated sex-based, racial and ethnic, and socioeconomic variation (on the basis of primary insurance payer and census block group-level Area Deprivation Index [ADI]) in the outcomes of interest using multivariable logistic regression and Cox models.
Results
Our NLP pipeline showed excellent precision, recall, and F1 scores > 0.8. Among 2624 patients, documentation of transplant discussion (adjusted subhazard ratio [sHR]: 1.30; 95% confidence interval [CI]: 1.01–1.67) and patient interest (sHR: 2.11; 95% CI: 1.65–2.69) were associated with a greater likelihood of receiving a kidney transplant. Patients with Medicaid (vs. non-Medicaid, adjusted odds ratio [OR]: 0.73; 95% CI:, 0.59–0.90) were less likely to have interest in transplant documented. Sex, race and ethnicity, and ADI were not associated with time to transplant referral in adjusted analyses. Non-Hispanic Black race (adjusted sHR: 0.40; 95% CI: 0.23–0.69), Medicaid insurance (adjusted sHR: 0.73; 95% CI: 0.56–0.94), and below-median ADI (adjusted sHR: 0.77; 95% CI: 0.60–0.98) were associated with lower receipt of a kidney transplant.
Conclusions
Using a novel NLP pipeline, we identified socioeconomic disparities in the documentation of patient interest in transplant, but not referrals. Patient-level interventions to increase interest in transplantation may be needed to address disparities in kidney transplantation.
Keywords: dialysis, end-stage kidney disease, end-stage renal disease, kidney transplant, natural language processing, transplant referral
Graphical abstract
There are significant sex, racial, ethnic, and socioeconomic disparities in kidney transplantation in the United States, with Black and Hispanic individuals being 27%–31% less likely to be waitlisted and 31%–34% less likely to receive a deceased donor kidney transplant.1, 2, 3 Although modifications to the Kidney Allocation System in 2014 decreased some disparities in transplantation rates after waitlisting, disparities in the likelihood of waitlisting persist.4 Factors driving disparities in waitlisting are understudied, in part because prewaitlist data on referrals and evaluations are not yet routinely collected by national data surveillance systems.5 Regional collaboratives collecting prewaitlist data have found sex-based, racial, and socioeconomic disparities in initiation of the transplant evaluation, but these studies have limited geographic scope and require manual data collection.6,7
The evaluation process begins with education about transplantation, assessment of patient interest, and a referral by the local nephrologist. There is significant variation in patient education about transplantation as a treatment option and referral for evaluation at both the clinician- and dialysis facility levels.8 Once referred, patients are evaluated by a multidisciplinary transplant team, must complete multi-modal testing, and meet center-specific medical, surgical, and psychosocial metrics for waitlisting. Not all patients who are referred start an evaluation, and not all evaluations are completed. Only 10.9% of patients receiving dialysis are waitlisted, but the actual number of eligible transplant candidates on dialysis is unknown.9
Patient interest in transplantation is a relatively unexplored factor in transplant disparities. Although the medical community recognizes kidney transplantation as having better survival and improved quality of life than dialysis, survey and interview studies reveal that some patients are hesitant to pursue transplantation because of potential adverse outcomes after transplantation, long waiting times, financial burden, and acceptance of dialysis.10,11 Although the CMS-2728 Form collects some information on transplant education and interest at the time of incident ESKD, large-scale longitudinal data on patient interest in transplantation are lacking. It is unknown whether variation in patient interest contributes to referral and waitlisting disparities.
In the absence of national registry data, NLP offers a novel method to collect information about the prewaitlisting process from unstructured clinical notes. NLP is a subfield of artificial intelligence and computational linguistics focused on making computers understand human language.12 NLP can be used to convert the vast amounts of information in free-text clinical notes into structured data. NLP has been used in nephrology to identify patient symptoms and predict acute kidney injury, ESKD, transplant waitlisting, and post-transplant complications, but has not been used to understand disparities in prewaitlist kidney transplant milestones.13, 14, 15, 16, 17, 18, 19 In this study, we leveraged NLP to extract prewaitlist information from nephrologist and social worker notes about transplant discussions, patient interest in transplantation, and referrals to transplant evaluation among patients receiving dialysis at a nonprofit dialysis organization. We examined the association of sex, race, ethnicity, and socioeconomic status with these prewaitlist transplant milestones and assessed whether differences in receipt of a kidney transplant were explained by variation in transplant discussions or patient interest.
Methods
Study Population and Data Source
We performed a retrospective cohort study of adult (age ≥ 18 years) patients with ESKD at 8 dialysis units within a nonprofit nephrology organization in New York City. We included patients with incident ESKD from January 2015 to May 2022 who received in-center hemodialysis, peritoneal dialysis, or home hemodialysis. We extracted structured and unstructured data from the dialysis electronic health record (EHR). Structured data included demographic and clinical variables, whereas unstructured data included free-text notes from social workers and nephrologists. We excluded patients with acute kidney injury requiring dialysis (n = 136), who were transient patients (n = 375), had missing insurance status (n = 378) or an incident ESKD date (n = 90), died within 90 days of dialysis start (n = 61), or had no clinical notes available for extraction (n = 77).
Exposures
Our primary exposures were sex, race and ethnicity, primary insurance payer, and census block group-level ADI. Sex was considered a biological variable and was administratively recorded in the EHR. Race and ethnicity were classified as (i) non-Hispanic White; (ii2) non-Hispanic Black; (iii) Hispanic; (iv) Asian, Native Hawaiian, or Pacific Islander; or (v) other, unknown, or declined and were self-reported at dialysis intake.20 Primary payer was classified as Medicaid (as primary or secondary payer) versus non-Medicaid (all other payers). ADI was calculated by geocoding patient addresses to census block groups using the US Census Bureau website and linking to the 2020 New York State ADI percentiles, derived from the 2016–2020 American Community Survey.21, 22, 23 ADI was modeled as above median (high socioeconomic status) versus below median (low socioeconomic status) on the basis of the state-level ADI decile. Exposures and covariates were obtained at the time of incident end-stage renal disease.
Outcomes
Our 4 outcomes of interest were (i) transplant discussed with the patient (yes vs. missing); (ii) patient interest in transplant (yes vs. no); (iii) transplant referral (yes vs. no); and (iv) receipt of a kidney transplant (yes vs. no). Prewaitlist milestones of transplant discussion, interest, and referral were ascertained using NLP of unstructured clinical notes (Supplementary Methods). Receipt of a kidney transplant was entered in the dialysis EHR by dialysis facility staff in the “reason for discharge” field.
NLP Pipeline Development
We developed a rule-based semiautomated NLP pipeline. First, 3 investigators (nephrologists SLT, JL, and DS) manually annotated 800 clinical notes from a random sample of 200 patients in the cohort, comprising 2 social workers and 2 nephrology notes (initial and follow-up) for each patient. Free-text indicating whether the transplant was discussed with the patient, whether the patient was interested or not interested in a transplant, or whether the patient was referred or not referred was independently tagged by each investigator using the brat rapid annotation tool, an online environment for collaborative text annotation (Supplementary Figure S1).24 Interannotator agreement was calculated using Cohen’s κ scores (Supplementary Table S1).25 The investigators (SLT, JL, and DS) then met as a group to rereview the 800 notes, resolve disagreements, and develop a gold standard corpus.
Clinical notes were preprocessed to tokenize the text into individual words, and tags were assigned to each token as part of an annotation. From this gold standard, we created sets of rules for capturing each tag, consisting of token-level regular-expression patterns (case-insensitive). These rules were manually created from 50% of the annotated gold standard notes, tested on the remaining 50%, and then applied to the entire corpus of 151,512 clinical notes. Some rules also included “fuzzy” or partial matching, as well as context-based requirements. Sequences of tokens that exactly matched one of these rules were taken as an entity. From that entity, we took a window of 5 to 8 adjacent tokens as our final pipeline output to ensure we fully captured the concept and its context. We used Python 3.10 and the spaCy and medspaCy NLP python packages to develop and run the pipeline.26
Statistical Analysis
We compared the demographic and clinical characteristics of patients in the entire study cohort at incident end-stage renal disease, and among those who had annotations of transplant discussion, patient interest or disinterest, and referral status. We examined the association of transplant discussion with receipt of a kidney transplant using Fine and Gray models accounting for the competing risk of death. We adjusted all models, including Model 1, for age because of the strong association between age and receipt of a kidney transplant. Model 2 was adjusted for potential demographic and clinical confounders—age, sex, race and ethnicity, marital status, employment, Medicaid status, above versus below-median ADI, dialysis modality, cause of ESKD, and number of Elixhauser comorbidities as recorded in the EHR. Separately, we examined the association of patient interest with receipt of a kidney transplant using the same approach. ADI was missing in 182 (6.9%) patients, who were modeled using a missing indicator as a separate category; the remaining variables did not have missing values.
We evaluated sex-based, racial and ethnic, and socioeconomic variation in the outcomes of interest. Logistic regression was used to examine the association of sociodemographic factors (sex, race and ethnicity, Medicaid status, and ADI) with transplant discussion in models adjusted for age (Model 1) and multivariable models (Model 2), accounting for sex and race interactions. Because patients could have “conflicting” clinical notes indicating both interest and lack of interest in transplant within the same note or in different notes, we used multinomial logistic regression models with the outcome as a categorical variable of (i) patient interested in transplant; (ii) patient not interested in transplant; (iii) annotations of both interested and not interested in transplant; or (iv) interest not documented.
To examine the associations of sociodemographic factors with time-to-referral, we used Cox models adjusted for age and multivariable models adjusted for the factors listed above, excluding patients with missing referral status. Patients were censored at death, transplant, transfer to another dialysis organization, hospice or withdrawal of dialysis, or end of study (June 11, 2022). To examine the associations between sociodemographic factors and time-to-receipt of a kidney transplant, we used Fine and Gray models that accounted for the competing risk of death. Patients were censored at transfer to another dialysis organization, hospice or withdrawal of dialysis, or end of study because they do not preclude the outcome but instead result in loss of observation and were therefore treated as censoring events.
In sensitivity analyses, we excluded individuals aged ≥ 80 years because many transplant centers have age cutoffs for transplant evaluation. We also modeled ADI as state-level deciles rather than above- vs. below-median ADI. Because participants followed for longer have more opportunities for discussions about transplantation, we conducted a sensitivity analysis that adjusted for follow-up time. Our study was approved by the Weill Cornell Medicine Institutional Review Board (Protocol # 20-07022363). Patients and the general public were not involved in the design, recruitment, or dissemination of study findings. Analyses were performed using Stata/IC, version 15.1 (StataCorp) and R version 4.5.0 statistical software, considering an α of < 0.05 to be statistically significant.
Results
Patient Cohort
Among 5059 patients within the dialysis organization EHR, we included 2624 in our final analysis (Figure 1). Patients were a median age of 64 years, 57% male, 32% non-Hispanic Black, 15% Hispanic, and 14% Asian or Pacific Islander (Table 1, Supplementary Table S2A–C). A total of 1125 (43%) had Medicaid as a primary or secondary payer, and the majority lived in census block groups that were more advantaged than other neighborhoods in the state. Over a median follow-up of 1.8 years (interquartile range 0.8–3.4 years), 336 (13%) patients received a kidney transplant, 507 (19%) died, 658 (25%) were transferred, 27 (1%) were admitted to hospice or stopped dialysis, and 1096 (42%) remained on dialysis through the end of the study period.
Figure 1.
CONSORT flow diagram. ESKD, end-stage kidney disease.
Table 1.
Demographic and clinical characteristics of patients in the study cohort
| Characteristics | Entire cohort (n = 2624) | Annotation of transplant discussed (n = 1835) | Annotation of interested/not interested in transplant (n = 910) | Annotation of referred/not referred for transplant (n = 887) |
|---|---|---|---|---|
| Sociodemographics | ||||
| Median Age at ESKD [IQR] | 64.0 [51.9–73.6] | 63.7 [51.7–73.3] | 59.8 [47.0–69.1] | 64.1 [51.3–73.1] |
| Male | 1505 (57.4%) | 1048 (57.1%) | 547 (60.1%) | 528 (59.5%) |
| Race and ethnicity | ||||
| Non-Hispanic White | 665 (25.3%) | 471 (25.7%) | 208 (22.9%) | 244 (27.5%) |
| Non-Hispanic Black | 850 (32.4%) | 612 (33.4%) | 286 (31.4%) | 280 (31.6%) |
| Hispanic | 383 (14.6%) | 240 (13.1%) | 154 (16.9%) | 138 (15.6%) |
| Asian, Native Hawaiian or Other Pacific Islander | 364 (13.9%) | 260 (14.2%) | 163 (17.9%) | 145 (16.4%) |
| Other/unknown/declined | 362 (13.8%) | 252 (13.7%) | 99 (10.9%) | 80 (9.0%) |
| Marital status | ||||
| Married | 930 (35.4%) | 650 (35.4%) | 335 (36.8%) | 331 (37.3%) |
| Divorced or Separated | 201 (7.7%) | 140 (7.6%) | 67 (7.4%) | 61 (6.9%) |
| Widowed | 170 (6.5%) | 118 (6.4%) | 42 (4.6%) | 41 (4.6%) |
| Single | 650 (24.8%) | 474 (25.8%) | 248 (27.3%) | 242 (27.3%) |
| Missing | 673 (25.7%) | 453 (24.7%) | 218 (24.0%) | 212 (23.9%) |
| Employment | ||||
| Full- or part-time | 303 (11.6%) | 222 (12.1%) | 140 (15.4%) | 108 (12.2%) |
| Retired due to age | 605 (23.1%) | 425 (23.2%) | 154 (16.9%) | 172 (19.4%) |
| Disability | 383 (14.6%) | 289 (15.8%) | 156 (17.1%) | 123 (13.9%) |
| Unemployed | 229 (8.7%) | 151 (8.2%) | 88 (9.7%) | 65 (7.3%) |
| Othera | 108 (4.1%) | 88 (4.8%) | 38 (4.2%) | 35 (4.0%) |
| Missing | 996 (38.0%) | 660 (36.0%) | 334 (36.7%) | 384 (43.3%) |
| Primary insurance | ||||
| Medicare FFS | 395 (15.1%) | 256 (14.0%) | 126 (13.9%) | 119 (13.4%) |
| Medicare advantage | 438 (16.7%) | 287 (15.6%) | 124 (13.6%) | 157 (17.7%) |
| Medicaid FFS | 761 (29.0%) | 545 (29.7%) | 254 (27.9%) | 248 (28.0%) |
| Managed Medicaid | 214 (8.2%) | 154 (8.4%) | 83 (9.1%) | 76 (8.6%) |
| Commercial, employer-sponsored, marketplace, or essential plan | 385 (14.7%) | 269 (14.7%) | 141 (15.5%) | 138 (15.6%) |
| Self-Pay or other | 431 (16.4%) | 324 (17.7%) | 182 (20.0%) | 149 (16.8%) |
| State-level ADI decileb | ||||
| Deciles 1 and 2 | 995 (40.8%) | 748 (43.7%) | 354 (41.5%) | 402 (48.0%) |
| Deciles 3 and 4 | 789 (32.3%) | 530 (31.0%) | 261 (30.6%) | 219 (26.2%) |
| Deciles 5 and 6 | 502 (20.6%) | 322 (18.8%) | 179 (21.0%) | 171 (20.4%) |
| Deciles 7 and 8 | 140 (5.7%) | 99 (5.8%) | 52 (6.1%) | 41 (4.9%) |
| Deciles 9 and 10 | 16 (0.7%) | 12 (0.7%) | 8 (0.9%) | 4 (0.5%) |
| Clinical characteristics | ||||
| Modality | ||||
| In-center hemodialysis | 2,184 (83.2%) | 1,497 (81.6%) | 750 (82.4%) | 706 (79.6%) |
| Peritoneal dialysis | 334 (12.7%) | 272 (14.8%) | 134 (14.7%) | 149 (16.8%) |
| Home hemodialysis | 106 (4.0%) | 66 (3.6%) | 26 (2.9%) | 32 (3.6%) |
| Cause of ESKD | ||||
| Diabetes | 1,043 (39.8%) | 716 (39.0%) | 355 (39.0%) | 336 (37.9%) |
| Hypertension | 694 (26.5%) | 501 (27.3%) | 225 (24.7%) | 233 (26.3%) |
| Glomerulonephritis | 234 (8.9%) | 162 (8.8%) | 105 (11.5%) | 89 (10.0%) |
| Cystic disease | 74 (2.8%) | 47 (2.6%) | 26 (2.9%) | 24 (2.7%) |
| HIV | 36 (1.4%) | 29 (1.6%) | 10 (1.1%) | 7 (0.8%) |
| Cancer | 32 (1.2%) | 19 (1.0%) | 7 (0.8%) | 9 (1.0%) |
| Failed transplant | 161 (6.1%) | 114 (6.2%) | 75 (8.2%) | 69 (7.8%) |
| Other/unknown | 350 (13.3%) | 247 (13.5%) | 107 (11.8%) | 120 (13.5%) |
| Median number of Elixhauser comorbidities | 4 (2–5) | 4 (2–5) | 4 (2–5) | 4 (2–5) |
ADI, area deprivation index; ESKD, end-stage kidney disease; FFS, fee-for-service; IQR, interquartile range.
ADI decile 1 is the lowest ADI (least disadvantaged) and 10 is the highest ADI (most disadvantaged).
Homemaker, medical leave, or student.
Among patients with non-missing ADI. ADI was missing in 182 (6.9%) participants.
NLP Pipeline Performance for Prewaitlist Transplant Milestones
Our rule-based NLP pipeline developed from the gold standard annotations showed excellent precision, recall, and F1 scores in the training set and F1 scores > 0.8 in the test set of clinical notes (Table 2). Performance was comparable across the 5 NLP concepts extracted.
Table 2.
Precision, recall, and F1 scores of the rule-based NLP pipeline
| Training set | Precision | Recall | F1 score |
|---|---|---|---|
| Transplant discussed with patient: Yes | 1.00 | 0.96 | 0.98 |
| Patient interest in transplant: Yes | 0.91 | 0.89 | 0.90 |
| Patient interest in transplant: No | 1.00 | 1.00 | 1.00 |
| Transplant referral: Yes | 1.00 | 0.79 | 0.88 |
| Transplant referral: No | 1.00 | 0.93 | 0.97 |
| Transplant discussed with patient: Yes | 1.00 | 0.78 | 0.88 |
| Patient interest in transplant: Yes | 1.00 | 0.84 | 0.91 |
| Patient interest in transplant: No | 0.90 | 0.75 | 0.82 |
| Transplant referral: Yes | 1.00 | 0.73 | 0.84 |
| Transplant referral: No | 0.88 | 0.88 | 0.88 |
The precision, recall, and F1 score for each concept tag, applied to the training and testing data sets.
Transplant Discussion
A total of 1835 (70%) patients had clinical text indicating that transplant was discussed (examples, Supplementary Table S3). Documentation of having a transplant discussion was associated with a greater likelihood of receiving a kidney transplant (adjusted sHRL 1.30; 95% CI: 1.01–1.67), accounting for the competing risk of death and adjusting for sociodemographic and clinical factors (Table 3, Supplementary Table S4).
Table 3.
Association of transplant discussions and patient interest with receipt of a kidney transplant (n = 2,624)
| sHRa (95% CI) | P-value | Adjusted sHRb (95% CI) | P-value | |
|---|---|---|---|---|
| Transplant discussion | 1.37 (1.07–1.75) | 0.011 | 1.30 (1.0–1.67) | 0.045 |
| Interest in transplant | ||||
| Patient interested | 2.22 (1.76–2.81) | < 0.001 | 2.11 (1.65–2.69) | < 0.001 |
| Not interested | 0.60 (0.31–1.16) | 0.126 | 0.73 (0.37–1.47) | 0.382 |
| Annotations of both interested and not interested | 0.37 (0.15–0.90) | 0.028 | 0.44 (0.18–1.08) | 0.074 |
| Interest not documented | Ref. | Ref. | -- | |
CI, confidence interval; sHR, subhazard ratio.
Fine and Gray models were used accounting for the competing risk of death.
Adjusted for age.
Multivariable models adjusted for age, sex, race and ethnicity, marital status, employment, Medicaid status, above vs. below median Area Deprivation Index, dialysis modality, cause of ESKD, and number of Elixhauser comorbidities.
In multivariable-adjusted logistic regression, sex, race and ethnicity, and Medicaid status were not significantly associated with transplant discussions (Table 4). Patients living in more disadvantaged neighborhoods, as measured by the ADI, were less likely to have a transplant discussion documented (adjusted OR: 0.81; 95% CI: 0.68–0.98).
Table 4.
Association of sociodemographic variables with transplant discussion, adjusted for age and multivariable adjusted (n = 2624)
| Characteristics | ORa (95% CI) | P-value | aORb (95% CI) | P-value |
|---|---|---|---|---|
| Male | 0.97 (0.82–1.14) | 0.689 | 1.03 (0.72–1.49) | 0.867 |
| Race and ethnicity | ||||
| Non-Hispanic White | Ref. | -- | Ref. | -- |
| Non-Hispanic Black | 1.03 (0.81–1.29) | 0.827 | 1.12 (0.77–1.63) | 0.556 |
| Hispanic | 0.67 (0.51–0.88) | 0.004 | 0.72 (0.46–1.13) | 0.150 |
| Asian, Native Hawaiian or Other Pacific Islander | 1.01 (0.76–1.34) | 0.949 | 1.24 (0.77–2.02) | 0.375 |
| Other/unknown/declined | 0.92 (0.69–1.22) | 0.555 | 0.90 (0.57–1.43) | 0.659 |
| Medicaid as primary or secondary payer (vs. non-Medicaid) | 1.01 (0.85–1.20) | 0.929 | 1.03 (0.85–1.24) | 0.756 |
| ADIc below median (vs. at or above median) | 0.82 (0.69–0.98) | 0.027 | 0.81 (0.68–0.98) | 0.027 |
ADI, area deprivation index; aOR, adjusted odds ratio; CI, confidence interval; OR, odds ratio.
Results are from logistic regression models.
Adjusted for age.
Multivariable models adjusted for age, sex, race and ethnicity, marital status, employment, Medicaid status, above vs. below median ADI, dialysis modality, cause of ESKD, and number of Elixhauser comorbidities, while accounting for sex and race interactions.
Below median ADI represents more neighborhood-level disadvantage.
Patient Interest in Transplant
The NLP pipeline identified 665 (25%) patients who were interested in transplant, 149 (6%) who were not interested, 96 (4%) with annotations indicating both interest and lack of interest, and 1174 (65%) with interest not documented (examples, Supplementary Table S3). Patient interest in transplant was associated with a greater likelihood of receiving a kidney transplant (sHR: 2.11; 95% CI: 1.65–2.69), compared with patients whose interest or lack of interest was not documented (Table 3, Supplementary Table S4).
Non-Hispanic Black race was associated with a lower odds of having interest in transplant documented (OR: 0.71; 95% CI: 0.55–0.92), but this finding was no longer statistically significant in multivariable multinomial logistic regression models (adjusted OR: 0.68; 95% CI: 0.44–1.05; Table 5). Asian and Pacific Islanders had a higher unadjusted odds of transplant interest (OR: 1.57; 95% CI: 1.17–2.12), but this was not statistically significant in multivariable models (adjusted OR: 1.58; 95% CI: 0.94–2.65). Patients with Medicaid were less likely to have interest in transplant documented (adjusted OR: 0.73; 95% CI: 0.59–0.90). Sex and ADI were not significantly associated with patient interest in transplant.
Table 5.
Association of sociodemographic variables with patient interest in transplant, adjusted for age and multivariable adjusted (n = 2,624)
| Characteristics | ORa (95% CI) | P-value | aORb (95% CI) | P-value |
|---|---|---|---|---|
| Male | 1.35 (1.11–1.63) | 0.002 | 1.05 (0.69–1.59) | 0.821 |
| Race and ethnicity | ||||
| Non-Hispanic White | Ref. | -- | Ref. | -- |
| Non-Hispanic Black | 0.71 (0.55–0.92) | 0.010 | 0.68 (0.44–1.05) | 0.079 |
| Hispanic | 1.00 (0.73–1.35) | 0.983 | 0.856 | |
| Asian, Native Hawaiian or Other Pacific Islander | 1.57 (1.17–2.12) | 0.003 | 0.95 (0.57–1.59) | 0.084 |
| Other/unknown/declined | 0.58 (0.42–0.81) | 0.002 | 1.58 (0.94–2.65) 0.50 (0.28–0.88) |
0.017 |
| Medicaid as primary or secondary payer (vs. non-Medicaid) | 0.70 (0.58–0.85) | < 0.001 | 0.73 (0.59–0.90) | 0.003 |
| ADIc below median (vs. at or above median) | 1.01 (0.83–1.22) | 0.959 | 1.03 (0.84–1.26) | 0.784 |
ADI, area deprivation index; aOR, adjusted odds ratio; CI, confidence interval; OR, odds ratio.
Results are from multinomial logistic regression models with the outcome as a categorical variable of (1) patient interested in transplant, (2) patient not interested in transplant, (3) annotations of both interested and not interested in transplant, (4) interest not documented.
Adjusted for age.
Multivariable models adjusted for age, sex, race and ethnicity, marital status, employment, Medicaid status, above vs. below median ADI, dialysis modality, cause of ESKD, and number of Elixhauser comorbidities, while accounting for sex and race interactions.
Below median ADI represents more neighborhood-level disadvantage.
Referral to Transplant
The NLP pipeline identified 673 (26%) patients who were referred for transplant, 214 (8%) who were not referred, and 1737 (66%) with referral status not documented. Medicaid insurance was associated with shorter time-to-referral (adjusted HR: 1.24; 95% CI: 1.04–1.49) (Table 6). Sex, race and ethnicity, and ADI were not associated with time to transplant referral in adjusted analyses. Sensitivity analyses excluding patients aged ≥ 80 years yielded similar results (Supplementary Table S5A–C).
Table 6.
Association of sociodemographic variables with referral to transplant and transplantation, adjusted for age and multivariable adjusted
| Characteristics | HRa (95% CI) | P-value | aHRb (95% CI) | P-value |
|---|---|---|---|---|
| Referral to transplant (n = 887) | ||||
| Male | 1.05 (0.89–1.23) | 0.566 | 1.09 (0.75–1.58) | 0.649 |
| Race and ethnicity | ||||
| Non-Hispanic White | Ref. | -- | Ref. | -- |
| Non-Hispanic Black | 0.99 (0.79–1.23) | 0.906 | 0.92 (0.63–1.33) | 0.641 |
| Hispanic | 1.19 (0.92–1.55) | 0.181 | 1.06 (0.68–1.65) | 0.784 |
| Asian, Native Hawaiian or Other Pacific Islander | 1.81 (1.41–2.34) | < 0.001 | 1.38 (0.88–2.15) | 0.163 |
| Other/unknown/declined | 1.21 (0.90–1.64) | 0.210 | 1.25 (0.78–2.01) | 0.358 |
| Medicaid as primary or secondary payer (vs. non-Medicaid) | 1.26 (1.07–1.47) | 0.005 | 1.24 (1.04–1.49) | 0.015 |
| Area Deprivation Indexc below median (vs. at or above median) | 0.95 (0.81–1.12) | 0.544 | 0.95 (0.80–1.13) | 0.544 |
| Transplant (n = 2,624)d | sHRa (95% CI) | P-value | Adjusted sHRb (95% CI) | P-value |
|---|---|---|---|---|
| Male | 1.16 (0.93–1.45) | 0.179 | 1.10 (0.69–1.75) | 0.690 |
| Race and ethnicity | ||||
| Non-Hispanic White | Ref. | -- | Ref. | -- |
| Non-Hispanic Black | 0.32 (0.23–0.44) | < 0.001 | 0.40 (0.23–0.69) | 0.001 |
| Hispanic | 0.47 (0.33–0.67) | < 0.001 | 0.89 (0.53–1.48) | 0.657 |
| Asian, Native Hawaiian or Other Pacific Islander | 0.92 (0.67–1.27) | 0.605 | 1.26 (0.69–2.30) | 0.446 |
| Other/unknown/declined | 0.88 (0.62–1.25) | 0.487 | 0.78 (0.44–1.37) | 0.382 |
| Medicaid as primary or secondary payer (vs. non-Medicaid) | 0.51 (0.41–0.65) | < 0.001 | 0.73 (0.56–0.94) | 0.016 |
| ADIc below median (vs. at or above median) | 0.68 (0.54–0.86) | 0.001 | 0.77 (0.60–0.98) | 0.036 |
ADI, area deprivation index; aHR, adjusted hazard ratio; CI, confidence interval; HR, hazard ratio; sHR; subhazard ratio.
Adjusted for age.
Multivariable models adjusted for age, sex, race and ethnicity, marital status, employment, primary payer, above vs. below median ADI, dialysis modality, cause of ESKD, and number of Elixhauser comorbidities, while accounting for sex and race interactions.
Fine and Gray models accounting for the competing risk of death.
Below median ADI represents more neighborhood-level disadvantage.
Receipt of Kidney Transplant
In adjusted analyses, non-Hispanic Black race (adjusted sHR: 0.40; 95% CI:, 0.23–0.69), Medicaid status (adjusted sHR: 0.73; 95% CI: 0.56–0.94), and below-median ADI (adjusted sHR: 0.77; 95% CI, 0.60–0.98) were associated with lower receipt of a kidney transplant (Table 6). Sex was not significantly associated with kidney transplantation. Sensitivity analyses that modeled ADI as state-level deciles showed similar results (data not shown). Analyses of transplant discussions that adjusted for follow-up time also yielded similar results to the main analysis (Supplementary Table S6A).
Discussion
In this longitudinal analysis of incident patients with ESKD receiving dialysis, we used novel NLP methods to extract prewaitlist milestones from clinical notes. Our analysis found racial and socioeconomic disparities in the receipt of a kidney transplant, but not disparities in transplant discussions or referral. We identified racial and socioeconomic disparities in documentation of patient interest in transplant as a previously unrecognized factor that may contribute to the lower rate of kidney transplantation among non-Hispanic Black and Medicaid-insured patients.
A novel contribution of our findings is that downstream waitlisting and transplant disparities may stem not only from nephrologists’ and social workers’ conversations about transplantation or referrals, but also from patient interest in transplant as an important determinant. On the basis of our study, we suggest that enhanced transplant education and shared decision-making to support patient interest in the process may be a key intervention to implement in attempts to ameliorate disparities in kidney transplantation. We acknowledge that many patients in our analysis did not have clinical documentation of interest in transplant, which represents a large and heterogenous group including patients who were never asked about their interest, as well as those who were asked but interest was not documented.
The racial and socioeconomic disparities in the kidney transplantation rate we observed are generally consistent with prior studies.27 Unlike other single-institution analyses, we did not detect a lower rate of kidney transplantation among Hispanic individuals in adjusted analyses, likely because of the fact that they composed a smaller proportion of our cohort.28 Importantly, we did not observe racial differences in transplant referral; Patzer et al.29 found that non-Hispanic Black patients had higher rates of referral but a lower likelihood of evaluation start, compared with non-Hispanic White patients. This may be related to geographic differences in our cohort compared with previous publications or clinician and system-level factors.29,30
Our results inform ongoing efforts to collect pretransplant waitlisting data from dialysis organizations and transplant centers. The Organ Procurement and Transplantation Network Data Advisory Committee’s Pre-Waitlist Data Collection Workgroup is currently developing a proposal to collect data about transplant referrals, evaluations, and selection committee outcomes.31,32 Obtaining this data nationwide will be an important step in understanding patient access and barriers to transplantation; however, our study suggests that recording events even further upstream, such as transplant discussions and interest, may be required to truly understand and eliminate disparities in access. Furthermore, interventions that solely target the dialysis facility or the transplant center may not yield the desired outcomes; engaging with patients to improve transplant education and increase patient interest will be necessary.
Our study adds to the growing body of literature establishing the feasibility and use of using NLP to extract unstructured information from dialysis clinical notes. Dialysis organizations could deploy similar NLP pipelines for internal quality improvement purposes or external quality reporting. Importantly, social determinants of health data are also poorly collected in structured dialysis EHR data, national transplant registries, and other state- and federal-level data bases.33 This contributes to the overall challenges of discerning the true drivers of disparities in waitlisting and highlights unstructured EHR data as an important data source to be explored.34,35 For example, Zaribafzadeh et al.36 used NLP to extract social determinants of health data from clinical notes, finding that income and employment were associated with kidney transplant waitlisting. Using NLP, Robitschek et al.37 found that social support gaps, recent substance use, and mental health challenges were associated with lower likelihood of liver transplant waitlisting. We are currently developing additional NLP pipelines to elucidate how social determinants of health data, patient perceptions of transplant, social support, adherence, health-related social needs, and language and cultural barriers influence access to kidney transplantation.
A fundamental constraint of NLP is that it can only capture information documented in clinical notes during routine care. Many clinical notes did not have transplant discussions, interest, or referral clearly documented, which may be systems- or workflow-related (e.g., clinicians lacking sufficient time to document or not thinking that these concepts were important enough to document). Clinician-level biases, structural barriers, and patient-level challenges may all influence the completeness and accuracy of documentation. This makes interpretation of missing data challenging and limits the sample size for our analyses of patient interest and transplant referrals. Furthermore, documentation of our outcomes may vary by our exposures of interest (sex, race and ethnicity, measures of socioeconomic status), which could potentially bias our results. Characteristics of patients with interest and referral documented versus those not documented were statistically significantly different, but overall, the magnitude of absolute differences was small and did not show a clear pattern that would systematically bias our results (Supplementary Table S2A–C).
Strengths of our analysis include the development of a novel NLP pipeline to extract transplant-related concepts from over 150,000 dialysis clinical notes, which enabled us to use novel data that were previously not easily analyzable at scale. Using our NLP pipeline, we were able to analyze clinical events that are not currently captured in any registry data set. Our work represents a first step toward training large language models to extract and analyze vast quantities of unstructured data for use in clinical research, generative artificial intelligence, and other applications.
Our work has limitations. First, we used data from a dialysis organization serving a single urban geography, which may limit generalizability to the broader US population. In particular, New York City-based dialysis facilities have relative proximity to several major transplant centers, contributing to a higher rate of waitlisting than other areas of the US. Our NLP pipeline has not yet been externally validated using another dialysis organization data set, and this is an important next step that we will undertake. Second, we did not link to transplant registry data to ascertain the outcome of kidney transplantation but relied on dialysis EHR data. Third, use of neighborhood-level ADI masks heterogeneity within census block groups, particularly in New York City, where there may be substantial socioeconomic variation within neighborhoods; however, we also investigated individual-level socioeconomic measures, such as Medicaid status. Fourth, our results are limited by potential residual confounding and relatively modest sample size, but this study does provide unique insights into the prewaitlisting process.
In summary, using a novel data source and methodology, we were able to show that patient interest rather than transplant discussion or transplant referrals alone is a key determinant in kidney transplant disparities. Interventions to increase patient education and interest in transplantation may be beneficial.
Disclosure
SLT reports research funding from SAIGroup and Novartis Foundation (funds to institution) and personal fees from the National Kidney Foundation as Associate Editor of Kidney Medicine and Peterson Health Technology Institute, unrelated to the submitted work. JS reports consulting fees from Alkahest, Bayer AG, and Kaneka unrelated to this work and cochairs the American Society of Nephrology COVID-19 Response Team and Chairs the American Society of Nephrology Emergency Partnership Initiative. DS reports consulting fees from Vertex Pharmaceuticals, Eurofins, and Maze Therapeutics unrelated to this work. All other authors report no conflicts of interests.
Acknowledgments
We acknowledge the thoughtful comments of anonymous reviewers that have served to strengthen this manuscript.
Funding
This research was supported by the Ritu Banga Healthcare Disparities Research Award (Dr. Sawinski) and the Agency for Healthcare Research and Quality (K08HS028684, Dr. Tummalapalli).
Footnotes
Supplementary Methods. Rule-based NLP pipeline development.
Figure S1. Examples of note annotations within BRAT.
Table S1. Interannotator agreement.
Table S2. (A) Comparison of demographic and clinical characteristics by annotation content for transplant discussion. (B) Comparison of demographic and clinical characteristics by annotation content for patient interest in transplant. (C) Comparison of demographic and clinical characteristics by annotation content for referral to transplant.
Table S3. Examples of rule-based NLP output.
Table S4. Outcomes of transplant and death, stratified by transplant discussion and interest in transplantation (n = 2,624).
Table S5. (A) Association of sociodemographic variables with transplant discussion, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (B) Association of sociodemographic variables with patient interest in transplant, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (C). Association of sociodemographic variables with referral to transplant and transplantation, sensitivity analysis excluding patients aged ≥ 80.
Table S6A. Association of sociodemographic variables with transplant discussion, adjusted for age and multivariable adjusted (n = 2,624), sensitivity analysis adjusting for follow-up time.
STROBE Checklist.
Supplementary Material
Supplementary Methods. Rule-based NLP pipeline development. Figure S1. Examples of note annotations within BRAT. Table S1. Interannotator agreement. Table S2. (A) Comparison of demographic and clinical characteristics by annotation content for transplant discussion. (B) Comparison of demographic and clinical characteristics by annotation content for patient interest in transplant. (C) Comparison of demographic and clinical characteristics by annotation content for referral to transplant. Table S3. Examples of rule-based NLP output. Table S4. Outcomes of transplant and death, stratified by transplant discussion and interest in transplantation (n = 2,624). Table S5. (A) Association of sociodemographic variables with transplant discussion, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (B) Association of sociodemographic variables with patient interest in transplant, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (C). Association of sociodemographic variables with referral to transplant and transplantation, sensitivity analysis excluding patients aged ≥ 80. Table S6A. Association of sociodemographic variables with transplant discussion, adjusted for age and multivariable adjusted (n = 2,624), sensitivity analysis adjusting for follow-up time. STROBE Checklist.
References
- 1.Harding J.L., Hu C., Pastan S.O., Rossi A., Patzer R.E. Trends in sex disparities in access to kidney transplantation: a nationwide US cohort study. Am J Kidney Dis. 2025;86:18–31.e1. doi: 10.1053/j.ajkd.2024.12.008. [DOI] [PubMed] [Google Scholar]
- 2.Cron D.C., Tsai T.C., Patzer R.E., Husain S.A., Xiang L., Adler J.T. The association of dialysis facility payer mix with access to kidney transplantation. JAMA Netw Open. 2023;6 doi: 10.1001/jamanetworkopen.2023.22803. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Wesselman H., Ford C.G., Leyva Y., et al. Social determinants of health and race disparities in kidney transplant. Clin J Am Soc Nephrol. 2021;16:262–274. doi: 10.2215/CJN.04860420. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Melanson T.A., Hockenberry J.M., Plantinga L., et al. New kidney allocation system associated with increased rates of transplants among black and Hispanic patients. Health Aff (Millwood) 2017;36:1078–1085. doi: 10.1377/hlthaff.2016.1625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Patzer R.E., Adler J.T., Harding J.L., et al. A population health approach to transplant access: challenging the status quo. Am J Kidney Dis. 2022;80:406–415. doi: 10.1053/j.ajkd.2022.01.422. [DOI] [PubMed] [Google Scholar]
- 6.Harding J.L., Di M., Pastan S.O., et al. Sex/gender-based disparities in early transplant access by attributed cause of kidney disease–evidence from a multiregional cohort in the Southeast United States. Kidney Int Rep. 2023;8:2580–2591. doi: 10.1016/j.ekir.2023.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kelty C.E., Buford J., Drewry K.M., et al. Sociodemographic barriers to starting the kidney transplantation evaluation process and waitlisting in the Ohio River Valley. Am J Nephrol. 2025;56:675–687. doi: 10.1159/000546108. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Buford J., Di M., Harding J.L., et al. Regional variation in early kidney transplant access across dialysis facilities in four US regions. Kidney Int Rep. 2025;11 doi: 10.1016/j.ekir.2025.103721. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.United States Renal Data System . National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2025. 2025 USRDS Annual Data Report. Epidemiology of kidney disease in the United States.https://usrds-adr.niddk.nih.gov/2025 [Google Scholar]
- 10.Holley J.L., McCauley C., Doherty B., Stackiewicz L., Johnson J.P. Patients’ views in the choice of renal transplant. Kidney Int. 1996;49:494–498. doi: 10.1038/ki.1996.70. [DOI] [PubMed] [Google Scholar]
- 11.Biabani F., Rahmani A., MahmudiRad G., Hassankhani H., Azadi A. Reasons for kidney transplant refusal among patients receiving peritoneal dialysis: a qualitative study. Perit Dial Int. 2023;43:395–401. doi: 10.1177/08968608221146865. [DOI] [PubMed] [Google Scholar]
- 12.Khurana D., Koli A., Khatter K., Singh S. Natural language processing: state of the art, current trends and challenges. Multimedia Tools Appl. 2023;82:3713–3744. doi: 10.1007/s11042-022-13428-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Chan L., Beers K., Yau A.A., et al. Natural language processing of electronic health records is superior to billing codes to identify symptom burden in hemodialysis patients. Kidney Int. 2020;97:383–392. doi: 10.1016/j.kint.2019.10.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Van Vleck T.T., Farrell D., Chan L. Natural language processing in nephrology. Adv Chronic Kidney Dis. 2022;29:465–471. doi: 10.1053/j.ackd.2022.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gatsinga R., Lau R.S.-E., Lim B.J.H., et al. Current applications and developments of natural language processing in kidney transplantation: a scoping review. Transplant Proc. 2025;57:558–568. doi: 10.1016/j.transproceed.2025.02.027. [DOI] [PubMed] [Google Scholar]
- 16.Arenson M., Hogan J., Xu L., et al. Predicting kidney transplant recipient cohorts’ 30-day rehospitalization using clinical notes and electronic health care record data. Kidney Int Rep. 2023;8:489–498. doi: 10.1016/j.ekir.2022.12.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Gotlieb N., Azhie A., Sharma D., et al. The promise of machine learning applications in solid organ transplantation. npj Digit Med. 2022;5:89. doi: 10.1038/s41746-022-00637-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Garcia-Lopez A., Cuervo-Rojas J., Garcia-Lopez J., Giron-Luque F. Using Natural Language Processing and Machine Learning to classify the status of kidney allograft in Electronic Medical Records written in Spanish. PLoS One. 2025;20 doi: 10.1371/journal.pone.0322587. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Dickerson L.K., Rouhizadeh M., Korotkaya Y., et al. Language impairment in adults with end-stage liver disease: application of natural language processing towards patient-generated health records. NPJ Digit Med. 2019;2:106. doi: 10.1038/s41746-019-0179-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Flanagin A., Frey T., Christiansen S.L., AMA Manual of Style Committee. Committee AMoS Updated guidance on the reporting of race and ethnicity in medical and science journals. JAMA. 2021;326:621–627. doi: 10.1001/jama.2021.13304. [DOI] [PubMed] [Google Scholar]
- 21.United States Census Bureau Find Batch Address Geographies. https://geocoding.geo.census.gov/geocoder/geographies/addressbatch?form
- 22.Kind A.J., Buckingham W.R. Making neighborhood-disadvantage metrics accessible—the neighborhood atlas. N Engl J Med. 2018;378:2456–2458. doi: 10.1056/NEJMp1802313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.University of Wisconsin School of Medicine and Public Health Area Deprivation Index v4. 2020. https://www.neighborhoodatlas.medicine.wisc.edu/
- 24.Brat Rapid Annotation Tool: Online Environment for Collaborative Text Annotation. https://brat.nlplab.org/
- 25.Landis J.R., Koch G.G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–174. doi: 10.2307/2529310. [DOI] [PubMed] [Google Scholar]
- 26.Honnibal M., Montani I., Van Landeghem S., Boyd A. spaCy: Industrial-Strength Natural Language Processing in Python. 2020. https://spacy.io
- 27.Hall Y.N., Choi A.I., Xu P., O’Hare A.M., Chertow G.M. Racial ethnic differences in rates and determinants of deceased donor kidney transplantation. J Am Soc Nephrol. 2011;22:743–751. doi: 10.1681/ASN.2010080819. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Gordon E.J., Lee J., Kang R., Uriarte J., Caicedo J.C. Disparities persist among Hispanic patients: completing evaluation, waitlisting, and receiving a kidney transplant. Transplant Direct. 2024;10 doi: 10.1097/TXD.0000000000001595. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Patzer R.E., McPherson L., Wang Z., et al. Dialysis facility referral and start of evaluation for kidney transplantation among patients treated with dialysis in the Southeastern United States. Am J Transplant. 2020;20:2113–2125. doi: 10.1111/ajt.15791. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Ashby V.B., Kalbfleisch J.D., Wolfe R.A., Lin M., Port F.K., Leichtman A.B. Geographic variability in access to primary kidney transplantation in the United States, 1996–2005. Am J Transplant. 2007;7:1412–1423. doi: 10.1111/j.1600-6143.2007.01785.x. [DOI] [PubMed] [Google Scholar]
- 31.Patzer R.E., Schold J.D., Hirose R., et al. Transforming transplantation access: a federal directive for comprehensive pre-waitlisting data collection. Am J Transplant. 2025;25:900–907. doi: 10.1016/j.ajt.2025.01.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.OPTN Data Advisory Committee Pre-Waitlist Data Collection Workgroup Meeting Summary. https://optn.transplant.hrsa.gov/media/wf0jwzrj/20240116_optn_dac_pre-waitlist-data-collection-workgroup-meeting_summary-final.pdf
- 33.Chan N.W., Moya-Mendez M., Henson J.B., et al. Social determinants of health data in solid organ transplantation: national data sources and future directions. Am J Transplant. 2022;22:2293–2301. doi: 10.1111/ajt.17096. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Mupfudze T.G., Martinez A., Noreen S.M., Stewart D.E., Schold J.D., Cartwright L. Individual-level social determinants of health and disparities in access to kidney transplant and waitlist mortality. PLoS One. 2024;19 doi: 10.1371/journal.pone.0308407. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Mupfudze T.G., Handarova D., Noreen S.M., Stewart D.E., Mohan S., Schold J.D. Influence of individual- and area-level social determinants of health on likelihood of living versus deceased donor kidney transplantation. Kidney Int Rep. 2025;10:791–802. doi: 10.1016/j.ekir.2024.12.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Zaribafzadeh H., Henson J.B., Chan N.W., et al. Development of a natural language processing algorithm to extract social determinants of health from clinician notes. Am J Transplant. 2025;25:1306–1318. doi: 10.1016/j.ajt.2025.02.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Robitschek E., Bastani A., Horwath K., et al. A large language model-based approach to quantifying the effects of social determinants in liver transplant decisions. npj Digit Med. 2025;8:665. doi: 10.1038/s41746-025-02025-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary Methods. Rule-based NLP pipeline development. Figure S1. Examples of note annotations within BRAT. Table S1. Interannotator agreement. Table S2. (A) Comparison of demographic and clinical characteristics by annotation content for transplant discussion. (B) Comparison of demographic and clinical characteristics by annotation content for patient interest in transplant. (C) Comparison of demographic and clinical characteristics by annotation content for referral to transplant. Table S3. Examples of rule-based NLP output. Table S4. Outcomes of transplant and death, stratified by transplant discussion and interest in transplantation (n = 2,624). Table S5. (A) Association of sociodemographic variables with transplant discussion, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (B) Association of sociodemographic variables with patient interest in transplant, sensitivity analysis excluding patients aged ≥ 80 (n = 2,295). (C). Association of sociodemographic variables with referral to transplant and transplantation, sensitivity analysis excluding patients aged ≥ 80. Table S6A. Association of sociodemographic variables with transplant discussion, adjusted for age and multivariable adjusted (n = 2,624), sensitivity analysis adjusting for follow-up time. STROBE Checklist.