Discrepant Outcomes between National Kidney Transplant Data Registries in the United States

Visual Abstract

Abstract

Significance Statement

Effects of reduced access to external data by transplant registries to improve accuracy and completeness of the collected data are compounded by different data management processes at three US organizations that maintain kidney transplant-related datasets. This analysis suggests that the datasets have large differences in reported outcomes that vary across different subsets of patients. These differences, along with recent disclosure of previously missing outcomes data, raise important questions about completeness of the outcome measures. Differences in recorded deaths seem to be increasing in recent years, reflecting the adverse effects of restricted access to external data sources. Although these registries are invaluable sources for the transplant community, discrepancies and incomplete reporting risk undermining their value for future analyses, particularly when used for developing national transplant policy or regulatory measures.

Background

Central to a transplant registry's quality are accuracy and completeness of the clinical information being captured, especially for important outcomes, such as graft failure or death. Effects of more limited access to external sources of death data for transplant registries are compounded by different data management processes at the United Network for Organ Sharing (UNOS), the Scientific Registry of Transplant Recipients (SRTR), and the United States Renal Data System (USRDS).

Methods

This cross-sectional registry study examined differences in reported deaths among kidney transplant candidates and recipients of kidneys from deceased and living donors in 2000 through 2019 in three transplant datasets on the basis of data current as of 2020. We assessed annual death rates and survival estimates to visualize trends in reported deaths between sources.

Results

The UNOS dataset included 77,605 deaths among 315,346 recipients and 61,249 deaths among 275,000 nonpreemptively waitlisted candidates who were never transplanted. The SRTR dataset included 87,149 deaths among 315,152 recipients and 60,042 deaths among 259,584 waitlisted candidates. The USRDS dataset included 89,515 deaths among 311,955 candidates and 63,577 deaths among 238,167 waitlisted candidates. Annual death rates among the prevalent transplant population show accumulating differences across datasets—2.31%, 4.00%, and 4.03% by 2019 from UNOS, SRTR, and USRDS, respectively. Long-term survival outcomes were similar among nonpreemptively waitlisted candidates but showed more than 10% discordance between USRDS and UNOS among transplanted patients.

Conclusions

Large differences in reported patient outcomes across datasets seem to be increasing, raising questions about their completeness. Understanding the differences between these datasets is essential for accurate, reliable interpretation of analyses that use these data for policy development, regulatory oversight, and research.

Podcast

This article contains a podcast at https://dts.podtrac.com/redirect.mp3/www.asn-online.org/media/podcast/JASN/2023_10_24_JASN0000000000000194.mp3

Introduction

The Organ Procurement and Transplantation Network (OPTN) data registry serves many important roles in the field of transplantation. With mandatory reporting at prespecified time points for waitlisted candidates, donors, and transplant recipients, it has facilitated important analyses that have informed innovation and improvements in clinical practice and outcomes. The registry's mandatory reporting structure has also allowed it to serve the needs of regulators, clinical and health service researchers, and inform the development of policies related to allocation and value-based care.^1–5 However, central to the quality of any registry is the extent to which the relevant clinical information being captured is accurate and complete, especially for critically important outcomes of interest, such as graft failure or death.⁶ The OPTN data are the primary data source for the research files provided by the United Network for Organ Sharing (UNOS), the Scientific Registry of Transplant Recipients (SRTR) database, and associated reporting, including its research files, and provide all the transplant-related data to the United States Renal Data System (USRDS) and its research files.

Although graft failures and deaths after transplant are reported by the transplant centers, patients in the United States (US) who have been stable for long periods of time are often not followed closely by their centers but instead followed by clinicians outside the transplant centers (e.g., primary care providers or community nephrologists). Similarly, candidates on the kidney transplant waitlist almost invariably receive their care outside the transplant centers through their general nephrologist either in a nephrology clinic or in the dialysis unit. This creates potential delays or failures on the part of transplant centers to identify graft failures and deaths among both candidates and recipients in a timely manner.⁷ Transplant center-level differences in clinical follow-up and tracking of patients over the long term create potential variations in the data availability and subsequent reporting of outcomes to the OPTN. As a result, the challenges of nonrandom, incomplete reporting of these outcomes have traditionally been overcome by supplementation from other sources of data, such as the Social Security Death Master File (SSDMF) and/or Center for Medicare & Medicaid Services (CMS) administrative data, which can provide insights into graft failure with the subsequent return to dialysis and death. Changes over the past decade in access to these external sources and choices of which data are supplemented have been further compounded by differences in practice between the OPTN, SRTR, and USRDS.⁸ In addition, all seem to have different processes and different policies for what they share in the analytic files that they provide to researchers.⁹ We aimed to describe the extent of the differences between the registries, including changes over time and the potential implications for researchers and policy makers that depend on the completeness of these data.

Methods

Study Design

We conducted a registry study examining differences in reported death data among transplant candidates and recipients from three sources: UNOS, SRTR, and USRDS (Figure 1).¹⁰ The files used in this analysis were based on data current as of March 20, 2020, September 2, 2020, and March 13, 2020, respectively. This study used data from the SRTR. The SRTR data system includes data on all donor, waitlisted candidates, and transplant recipients in the United States, submitted by the members of the OPTN. The Health Resources and Services Administration (HRSA), US Department of Health and Human Services, provides oversight to the activities of the OPTN and SRTR contractors.

Figure 1.

Flow diagram of data sources and registries to research data files produced by each organization, available to researchers as of 2022. Dotted arrows represent flows of information that is available to the OPTN for internal use but are not shared externally with researchers. Processes that have not been described or clearly disclosed are represented in black boxes with white text. Recent changes in what data have been included in publicly available datasets are represented in red. *No external verification for graft failure for return to dialysis. Limited internal/policy development use. CMS, Center for Medicare & Medicaid Services; EQRS, End-Stage Renal Disease Quality Reporting System; FFS, fee for service; HRSA, Health Resources and Services Administration; NIH, National Institutes of Health; OPO, organ procurement organization; OPTN, Organ Procurement and Transplantation Network; SAF, Standard Analysis File; SRTR, Scientific Registry of Transplant Recipient; STAR, standard transplant analysis and research; TIEDI, transplant information electronic data interchange; UNOS, United Network for Organ Sharing; USRDS, United States Renal Data System.

Data Sources

The OPTN registry includes information directly reported from the transplant centers at the time of waitlisting, transplantation, at 6 and 12 months post-transplant, and annually, thereafter, for the duration of graft survival (or until the patient is lost to follow-up) for all transplant candidates of all solid organ transplants. These data are reported by centers to the OPTN via web-based forms that are together referred to as Transplant Information Electronic Data Interchange forms and part of the UNOS contract with the HRSA. In addition, there are also mandatory data reporting from organ procurement organizations and HLA laboratory results. The OPTN registry linkage to external death data is a more recent, evolving process that has access to some CMS data, such as the date of dialysis initiation for candidates' waitlisted for a kidney transplant. The OPTN contractor, UNOS, maintains the OPTN registry and releases the standard transplant analysis and research (STAR) files to researchers.

For many years, the HRSA contract for the SRTR was held by the same entity that held the OPTN contract (i.e., UNOS). Although it was awarded to a different contractor more than a decade ago, the SRTR remains largely reliant on the OPTN registry data as its primary data source, using these data to create program-specific reports and, more recently, a patient-facing website for all solid organ transplants. The SRTR also uses other external data sources and procedures to verify death data for waitlisted candidates and transplant recipients beyond what is provided by the UNOS/OPTN registry.

The USRDS is funded by the National Institute of Diabetes and Digestive and Kidney Diseases and is a registry that is unique to patients with ESKD. It collects information from the 2728 (ESKD Medical Evidence) form, the 2746 (Death Notification) form, and claims information from fee-for-service Medicare, in addition to including patient-level linkage to candidates and recipients in the UNOS/OPTN registry. As a result, the USRDS does not include information on candidates or recipients of other solid organ transplants and has no information on preemptively waitlisted candidates until they require a form for renal replacement therapy. The USRDS also has additional data that are not relevant to this discussion, including data from the CKD 5% general Medicare payment data and other special studies. However, its complete capture of the ESKD population in the United States has been critical for discussions and analyses around access to transplantation.

Inclusion/Exclusion Criteria

For transplant recipient deaths, we identified all single-organ kidney transplants that occurred in the United States from January 01, 2000, to December 31, 2019. In the event of multiple transplants in this time frame for an individual patient, only the most recent transplant was considered when analyzing death date by the patient.

For candidate deaths, we identified all waitlisted kidney candidates who remained on the waitlist at time of death (i.e., did not receive transplant) from January 1, 2000, to December 31, 2019. To compare differences for waitlisted candidates between data sources, preemptively waitlisted candidates were excluded because USRDS is not expected to capture these data. Follow-up was censored at waitlist removal, and deaths after waitlist removal were not taken into account.

Variables

All variable names from the UNOS, SRTR, and USRDS files and their descriptions are detailed in the Supplement (Supplemental Table 2). If more than one separate variable described a death date (e.g., the SRTR Standard Analysis File [SAF] contains four separate death date variables obtained from the OPTN, Social Security Administration database, restricted records, and follow-up records), the first occurring date was used in the analysis. Deaths after graft failure were identified if the reported date of graft failure preceded the date of death. To compare death reporting with the OPTN after patients are no longer followed by their transplant centers, deaths after lost to follow-up status were identified using the patient status variable(s). Lost to follow-up status was not reported by USRDS.

Statistical Analysis

For each source, unadjusted patient survival was assessed using the Kaplan–Meier method. The total prevalent transplant population at the start of each year (i.e., January 1) and the total number of deaths that occurred by the end of year (i.e., December 31) were counted to assess the rates of death among transplanted patients. Year of transplant was also tabulated against year of death and formatted as heat maps to visualize trends in reported deaths and allow for informed comparisons across the datasets. Thirteen data discrepancies in the USRDS dataset where death dates preceded transplant date were identified and excluded. Growing discrepancies within each dataset were assessed for trend using Cochran–Armitage tests. Because data use agreements precluded individual patient-level linkage between the different data files, pairwise comparisons (SRTR-UNOS, USRDS-SRTR, and USRDS-UNOS) were made for the absolute number of deaths. This process was repeated with the total number of deaths occurring after graft failure, the total number of deaths occurring after lost to follow-up status, and the total number of deaths of nonpreemptively waitlisted candidates who did not receive a kidney transplant.

The total number of transplant deaths in these files was further compared with the total number of transplant deaths in recent UNOS and SRTR files, current as of June 30, 2022, and June 2, 2022, respectively. Owing to a reporting lag in USRDS data, we were unable to use these more recent files for our main comparison. This study was approved by the Institutional Review Board of Columbia University. All research activities were consistent with the principles of the Declaration of Istanbul.

Results

Data Capture of Deaths after Transplantation

For our analysis of transplant deaths, inclusion criteria of US kidney-only transplants from 2000 to 2019 resulted in a total of 332,338 (UNOS), 332,206 (SRTR), and 329,109 (USRDS) recorded transplants from 315,346 (UNOS), 315,152 (SRTR), and 311,955 unique patients (Table 1). The 2020 UNOS STAR dataset showed a total of 77,605 deaths (54,992 deceased donor, 22,611 living donor, two foreign transplants) between 2000 and 2019. Of this total, 21,519 (28%) were deaths after graft failure, and 6734 (9%) were deaths among recipients who were reported as lost to follow-up by their transplant centers. The SRTR SAF showed a total of 87,149 deaths (61,594 deceased donor and 25,555 living donor transplants), of which 6782 were obtained from the SSDMF alone. Of the total, 24,821 (28%) were deaths after graft failure, and 10,055 (12%) were deaths after lost to follow-up status. The USRDS dataset showed a total of 89,515 deaths (63,434 deceased donor and 26,081 living donor transplants), with 25,590 (29%) deaths after graft failure.

Table 1.

Total transplant and waitlist deaths pre-KAS (2000–2014) and post-KAS (2015–2019), by data source

STUDY POPULATION	UNOS		SRTR		USRDS
	2000–2014	2015–2019	2000–2014	2015–2019	2000–2014	2015–2019
Total unique patients	218,949	96,397	218,761	96,391	219,666	92,289
Total transplants	235,438	96,900	235,311	96,895	236,306	92,803
Total deaths	47,848	29,757	48,343	38,806	49,905	39,610
Donor type
Deceased	33,985	21,007	34,355	27,239	35,532	27,902
Living	13,862	8749	13,988	11,567	14,373	11,708
Foreign	1	1	—	—	—	—
Deaths after graft failure	14,127	7392	14,260	10,561	14,780	10,810
Deaths after lost to follow-up	4867	1867	5211	4844	—	—
Total waitlist candidatesa	169,170	105,830	161,584	98,000	152,882	85,285
Total waitlist registrations	239,299	124,523	225,936	115,539	226,753	121,781
Deaths on waitlist	55,302	21,867	53,734	21,838	50,280	20,036
Deaths on waitlist—excluding preemptively waitlisted	44,650	16,599	43,475	16,567	46,385	17,192

KAS, kidney allocation system; UNOS, United Network for Organ Sharing; SRTR, Scientific Registry of Transplant Recipient; USRDS. United States Renal Data System.

^aNever transplanted.

Although direct comparisons on individual patients were not feasible due to data use restrictions, we found differences in the number of deaths recorded between all data sources, with the largest absolute differences were evident approximately 10–20 years after the patients' year of transplant (Figure 2). Analysis of long-term survival outcomes varied across the three datasets, with more than 10% discordance between USRDS and UNOS (Table 2). Shorter-term outcomes remained relatively similar; however, 5-year survival estimates varied approximately 1% more among recently transplanted patients (Supplemental Table 3). The rates of death among the prevalent transplant population within our cohort show accumulating differences across datasets—for example, 2.31%, 4.00%, and 4.03% by 2019 from UNOS, SRTR, and USRDS, respectively (Table 3, Supplemental Figure 1).

Figure 2.

Unadjusted Kaplan-Meier curves of transplant patient survival 2000-2019, by data source. (A) Overall transplant patient survival. (B) Transplant patient survival with follow-up censored at graft failure. SRTR, Scientific Registry of Transplant Recipient; UNOS, United Network for Organ Sharing; USRDS, United States Renal Data System.

Table 2.

Unadjusted Kaplan–Meier survival estimates of all patients transplanted (2000–2019)

Time (yr)	Survival Probability			Survival Probability, with Follow-up Censored at Graft Failure
	UNOS	SRTR	USRDS	UNOS	SRTR	USRDS
1	0.968	0.968	0.967	0.973	0.973	0.972
3	0.925	0.924	0.922	0.938	0.938	0.937
5	0.870	0.868	0.866	0.893	0.892	0.891
10	0.713	0.690	0.685	0.760	0.742	0.741
15	0.582	0.519	0.512	0.643	0.589	0.587
20	0.519	0.390	0.392	0.580	0.465	0.474

UNOS, United Network for Organ Sharing; SRTR, Scientific Registry of Transplant Recipient; USRDS. United States Renal Data System.

Table 3.

Rates of death, by data source, among the total prevalent transplant population within the study cohort

	UNOS			SRTR			USRDS
Year	Transplants (n)	Deaths (N)	Rate (%)	Transplants (n)	Deaths (N)	Rate (%)	Transplants (n)	Deaths (N)	Rate (%)
2001	11,218	777	6.93	11,186	781	6.98	11,218	804	7.17
2002	22,808	1084	4.75	22,748	1088	4.78	22,790	1108	4.86
2003	34,563	1469	4.25	34,476	1463	4.24	34,544	1493	4.32
2004	46,313	1926	4.16	46,213	1945	4.21	46,317	1975	4.26
2005	58,512	2407	4.11	58,374	2431	4.16	58,505	2486	4.25
2006	70,685	2812	3.98	70,520	2807	3.98	70,664	2875	4.07
2007	83,174	3185	3.83	83,002	3196	3.85	83,143	3263	3.92
2008	94,960	3495	3.68	94,766	3520	3.71	94,923	3591	3.78
2009	106,577	4090	3.84	106,338	4132	3.89	106,485	4203	3.95
2010	118,012	4470	3.79	117,725	4548	3.86	117,840	4598	3.9
2011	129,353	4854	3.75	128,984	4948	3.84	129,102	5189	4.02
2012	140,313	5296	3.77	139,845	5345	3.82	139,777	5638	4.03
2013	150,545	5503	3.66	150,026	5535	3.69	149,724	5817	3.89
2014	161,021	6087	3.78	160,466	6215	3.87	159,953	6480	4.05
2015	171,139	6671	3.9	170,455	6947	4.08	169,767	7057	4.16
2016	181,376	6853	3.78	180,414	7395	4.10	179,735	7508	4.18
2017	192,557	5498	2.86	191,050	7735	4.05	190,425	7996	4.20
2018	205,895	5634	2.74	202,147	8164	4.04	201,447	8455	4.20
2019	220,466	5103	2.31	214,188	8565	4.00	213,385	8594	4.03
Total		77,214a			86,760a			89,130a

UNOS, United Network for Organ Sharing; SRTR, Scientific Registry of Transplant Recipient; USRDS. United States Renal Data System.

^aTotal number of deaths occurring 2001-2019.

USRDS recorded the greatest number of deaths—more than UNOS or SRTR—and the discrepancy has widened in recent years (Supplemental Figure 3). Until recently, the SRTR SAF files have included more deaths than the UNOS STAR files (Figure 3). For deaths occurring in 2017 from any year of transplant, SRTR identified 2237 more deaths (28.9% difference) and USRDS identified 2498 more deaths (31.2% difference) compared with UNOS. Furthermore, the number of deaths that were discordant increased, and the degree of discordance increased quickly. In 2019, differences in deaths from any year of transplant amounted to 3491 additional deaths between USRDS and UNOS, resulting in a 40.6% difference. In all cases, recent discordance seemed to be driven by a gap in death data for patients who received transplants before 2010. For instance, among candidates transplanted in 2003, 362 deaths were reported by USRDS in 2019 versus only 150 deaths by UNOS—a 58.6% difference.

Figure 3.

Annual number of transplant deaths recorded by UNOS, SRTR, and USRDS, by year of death in 2000–2019, showing the extent of differences between data captured by each source. Solid lines indicate transplant deaths recorded in 2020 analysis files. Dashed lines indicate transplant deaths recorded in June 2022 analysis files. Vertical line marks the date of change in SSDMF availability. SRTR, Scientific Registry of Transplant Recipient; SSDMF, Social Security Death Master File; UNOS, United Network for Organ Sharing; USRDS, United States Renal Data System.

Deaths after graft failure also follow this trend (Supplemental Figure 4). Although differences still existed between USRDS and SRTR, the largest variation was between these sources and UNOS, where many fewer deaths after graft failure (i.e., a return to dialysis) were identified after 2015. There was a 5.3% difference between USRDS and UNOS in 2016 from deaths occurring from any year of transplant, increasing drastically to 44.8% in 2017 and climbing to as high as 51.7% by 2019. Similarly, there was a 4.2% difference between SRTR and UNOS in 2016, increasing from 42.6% in the next year to 51.3% by 2019.

Deaths after lost to follow-up status varied between sources—although SRTR and UNOS initially displayed similar numbers of deaths, the differences also increased over time from 2014 onward (Supplemental Figure 5). In 2018, SRTR reported a total of 850 more deaths occurring from any year of transplant, with the largest absolute differences occurring approximately 5–10 years after transplant.

Data Capture of Deaths on the Kidney Waiting List

Death discrepancies between data sources were also evident among waitlisted candidates (Figures 4 and 5). A total of 363,822 (UNOS), 341,475 (SRTR), and 348,534 (USRDS) waitlist registrations among 275,000 (UNOS), 259,584 (SRTR), and 238,167 (USRDS) candidates were identified from 2000 to 2019, of which 61,249 (UNOS), 60,042 (SRTR), and 63,577 (USRDS) were deaths among nonpreemptively waitlisted patients who did not receive a kidney transplant. Long-term survival outcomes were similar across datasets, provided that preemptively waitlisted candidates were removed from analysis (Tables 4 and 5). Differences between USRDS varied, and UNOS consistently captured more deaths than SRTR within approximately 5 years of waitlisting (Supplemental Figure 6). However, data capture by UNOS seemed to decline from a total of 79 more deaths in 2015 to 84 fewer deaths in 2019 compared with SRTR and from a total of 33 fewer deaths in 2015 to 191 fewer deaths in 2019 compared with USRDS. By 2019, accumulated differences in total deaths on the waitlist amounted to a 3.0% difference between SRTR and UNOS and a 6.5% difference between USRDS and UNOS.

Figure 4.

Unadjusted Kaplan-Meier curves of waitlist patient survival 2000-2019, by data source. (A) Overall waitlist patient survival. (B) Waitlist patient survival, excluding preemptively waitlisted candidates. SRTR, Scientific Registry of Transplant Recipient; UNOS, United Network for Organ Sharing; USRDS, United States Renal Data System.

Figure 5.

Annual number of waitlist deaths from 2000 to 2019, identified in in UNOS STAR (dated March 20, 2020), SRTR SAF (dated September 2, 2020), and USRDS (dated March 13, 2020), showing the differences and incomplete data capture of each source. SAF, Standard Analysis File; SRTR, Scientific Registry of Transplant Recipient; STAR, standard transplant analysis and research; UNOS, United Network for Organ Sharing; USRDS; United States Renal Data System.

Table 4.

Unadjusted Kaplan–Meier survival estimates of patients waitlisted (2000–2019)

Time (yr)	Survival probability All Waitlisted			Survival Probability Excluding Preemptively Waitlisted
	UNOS	SRTR	USRDS	UNOS	SRTR	USRDS
1	0.921	0.920	0.927	0.916	0.915	0.919
3	0.752	0.753	0.754	0.731	0.733	0.734
5	0.593	0.595	0.590	0.566	0.567	0.566
10	0.328	0.327	0.316	0.301	0.300	0.300
15	0.206	0.204	0.191	0.187	0.185	0.183
20	0.157	0.140	0.137	0.132	0.126	0.132

UNOS, United Network for Organ Sharing; SRTR, Scientific Registry of Transplant Recipient; USRDS. United States Renal Data System.

Table 5.

Potential interventions for US kidney transplant registries

Potential Interventions for US Kidney Transplant Registries
1. Data sharing agreements between all three registries and regular audits to ensure consistency of data being provided to researchers and regulatory agencies
2. Data sharing agreements between all three registries and regular audits to ensure consistency of data being provided to researchers and regulatory agencies
3. Development of a clear algorithm for determining whether there is a failed allograft or patient death, i.e., consistent across all three registries when there are discordant data being reported by different sources
4. Triangulation of data on outcomes for external sources to eliminate the risk of underreporting (inadvertent or otherwise) of adverse events. For example, deaths after graft failure are likely to be absent from the OPTN and SRTR registries but not from USRDS as a function of the data collection methods
5. Development of a researchers guide that outlines the limitations of the outcomes data (i.e., graft failure and deaths)
6. Transparency in the methods being used to ascertain outcomes so that any potential systemic biases that researchers and regulators can be considered as they develop their analytic plans

OPTN, Organ Procurement and Transplantation Network; SRTR, Scientific Registry of Transplant Recipient; USRDS. United States Renal Data System.

Discussion

The survival advantage provided by kidney transplantation has long been an important measure of success in the field—and by all measures, this has increased steadily.¹¹ Recent analyses suggesting improvements in short-term outcomes are reflected in improved long-term outcomes due to transplantation.^12,13 Post-transplant patient and allograft outcomes are also key elements of the regulatory framework for transplant centers in the United States, with focus on short-term outcomes such as 1-year and 3-year graft and patient survival.^14–17 As a result, the accurate assessment and recording of patient and graft status in transplant registries is a critical step in the accurate ascertainment of transplant center regulatory and performance metrics.¹⁸ These registry data are also the backbone of a robust body of transplant outcomes literature that has informed clinical practice. In this analysis, we found that the UNOS, SRTR, and USRDS datasets contain different numbers of deaths reported among the same set of transplant recipients and waitlisted candidates, with apparent missing events in these datasets. The differences reported here are likely an underestimate of the true discrepancies between registries because the deaths reported are unlikely completely concordant. For example, even if the absolute number of deaths in some years is similar across the datasets, they may capture the deaths of different patients. These findings raise important questions about whether historical regulatory actions may have been based on inaccurate or incomplete data and whether analyses using patient vital statuses reported from these databases may have inaccurate results.⁹

The loss of access to the complete SSDMF has resulted in the inability of the OPTN registry to directly validate deaths reported by transplant centers or, more importantly, to identify the latest deaths among candidates and recipients.⁸ Subsequently, UNOS has to develop labor and time-intensive alternative death verification processes.¹⁸ As a result, the patients in the program-specific reports subject to outcomes monitoring and regulatory oversight were prioritized. UNOS then regained access to a potentially more complete version of death records but was restricted from publicly sharing these records without external verification of the deaths. Thus, the UNOS STAR file included only the “verified deaths” that were identified by their manual process. Recent improvements in March 2022 in the processes and automation have led to identification of approximately 35,000 more events,⁹ including those affecting long-term outcomes,¹⁹ which were not part of the initial effort. However, the number of deaths that remain “unverified” and excluded from the STAR file is unclear because these data are not available to the researchers or to the public.

Incomplete data are likely missing in a nonrandom manner, given that the missingness is influenced to a great extent by transplant center practices.^18,19 Although prior analyses from Wilk et al. examined the potential effect of missing deaths from the dataset, these analyses had an exclusive focus on short-term outcomes and did not comment on the effect on any subgroups, long-term outcomes, or different censoring strategies. Differences in the completeness of the capture of outcomes by age, race, and geography were identified by an internal SRTR analysis, raising important questions about the effect of incomplete data capture on analyses that focus on subsets of transplant patients or examine long-term outcomes.⁹ These differences across subgroups are particularly concerning because they would limit the validity of metrics that attempt to measure the quality of transplant centers. Improvement in data capture is especially critical as the OPTN moves toward using death on the waitlist as an additional quality measure to evaluate transplant center performance.²⁰

Our data also demonstrate that although more recent changes have reduced the discrepancies between dataset because of the verification of 35,000 additional deaths by the OPTN, there remain significant discrepancies in the attribution of mortality even after 2020, with differences varying with recipient age, race, and state of residence. These ongoing discrepancies and their heterogeneous nature suggest an ongoing need to understand the challenges in accurate capture of vital status.

Because of the widespread use of these registry data by transplant regulators and investigators, the implications of inaccurate death data are broad. From a regulatory perspective, inaccurate capture of deaths after transplant may lead to inappropriate “flagging” of centers as underperforming if deaths are overreported or missed identification of underperforming centers if deaths are missed. Health services analyses relying on registry data for outcomes analyses risk producing discrepant results depending on which registry is used. The significance of these differences will also depend on the questions that registry data are attempting to answer.² In studies involving smaller cohorts⁹—such as single-center analyses or those studying uncommon exposure variables—a small number of inaccurate events may be more consequential than in large analyses of the whole transplant system, although the nonrandom nature of the missing data may still affect the latter. Identifying methods of improving the quality of these data is therefore urgently needed. In the short term, policy-making or regulatory bodies should consider conducting analyses on each registry to confirm concordant results with each before any policy changes or regulatory actions. Going forward, it is incumbent on the federal government to commission efforts to assess and improve the validity of outcomes data in each of these registries. These efforts may include restoring access to the SSDMF as a cross-reference for death outcomes, requiring registries to work together to find and reconcile cases with discrepant death data, funding investigators to identify novel methods to triangulate accurate outcomes capture, or perhaps having only a single comprehensive data repository.

Additional opportunities for data sharing between registries need to be pursued. For example, the identification of kidney allograft failures is potentially incomplete and not currently verified by the OPTN. Many, if not all, of these graft failures could be identified by patients' return to dialysis through a reappearance in the USRDS dataset via claims data, End-Stage Kidney Disease Quality Reporting System data, or a new 2728 form. Although HRSA has negotiated access to dialysis initiation data from CMS, the OPTN registry does not cross-reference reported dialysis start dates using additional data sources or create meaningful audits and these data are critical for accurate calculations of organ allocation time.⁵ Similarly, transplant centers are not required to report deaths after allograft failure when patients return to dialysis, creating another source of incomplete data capture. In comparison, deaths for patients on dialysis are captured by the USRDS dataset and represent another opportunity for the OPTN and SRTR datasets to be enriched through cross-collaboration and an agreement to share. Although we recognize that these datasets are governed by contracts with different government entities, at least two of these datasets (USRDS and SRTR) are held by the same contractor, which should facilitate opportunities for data sharing.

The disparate sources and identified gaps in both the clinical data and the outcomes data across these datasets underscore the need for these registries to identify and develop strategies to share patient-level data. Although the degree of overlap between research datasets from UNOS and SRTR is substantial, these datasets have fundamental differences. Researchers and other groups that use these datasets must be alerted to the limitations of the data and which patients are included to inform analytic plans. For example, although there is a detailed researcher's guide that accompanies the USRDS dataset,²¹ there is no such document with either the SRTR or UNOS research files.

It is possible that differences between reported deaths in a given period may decrease with time if centers have delayed reporting of changes in vital status several years after a death. However, the large discrepancies we observed even 3–5 years after the end of the study period suggest that delayed reporting is not the primary etiology of our findings. In addition, the delayed reporting of deaths by several years would nevertheless be consequential for its effect on the accuracy of center regulatory metrics. If this was the case, it would warrant at least a note of caution to researchers who are using these datasets to consider censoring patient follow-up earlier than they might choose otherwise.

In summary, our analyses suggest that there are large differences in the reported deaths and graft failures in three different national datasets that provide information on the transplant system. These differences seem to be growing rapidly, especially in more recent years, raising important questions about which datasets are the most complete and valid, as well as the sources of the differences. Understanding the observed differences is essential for the appropriate and accurate use of the data for regulatory oversight, policy development, and research purposes. It also highlights the need for detailed researcher's guides to lower the possibility of analytic errors. The kidney community in the United States has been fortunate to have robust data to support epidemiological research for advanced forms of kidney disease, but incomplete longitudinal data and the expanding loss of administrative data as more ESKD Medicare patients move to Medicare Advantage plans are concerning. A greater emphasis on data sharing between UNOS, SRTR, and USRDS would contribute immensely to the maintenance of a robust dataset, while additional changes in the OPTN contract would raise the potential for significant new investments in data collection, curation, and management.

Supplementary Material

jasn-34-1863-s001.pdf ^{(373.4KB, pdf)}

Disclosures

A.M. Huml reports Advisory or Leadership Role: Cleveland Minority Organ and Tissue Transplant Education Program (MOTTEP) Advisory Board Co-Chair, Chairperson for IPRO ESKD Network of the Ohio River Valley Medical Review Board, Member of the Medical Director Advisory Council for The National Forum of ESKD Networks. S.A. Husain reports Research Funding: Nelson family foundation; and Honoraria: Fresenius. S. Mohan receives grant funding from Kidney Transplant Collaborative and the NIH, and personal fees from Kidney International Reports and Health Services Advisory Group outside of the submitted work. S. Mohan also reports Consultancy: eGenesis and HSAG; Patents or Royalties: Columbia University; Advisory or Leadership Role: Chair of UNOS Data advisory committee, Deputy Editor of Kidney International Reports (ISN), Member of ASN Quality committee, Member of SRTR Review Committee, and National Faculty Chair of ETCLC. R.E. Patzer reports Employer: Vital (spouse); Ownership Interest: Vital Software (spouse has ownership); Advisory or Leadership Role: Editorial board of American Journal of Transplantation, CJASN editorial board, Chair of United Network for Organ Sharing Data Advisory Board; and Other Interests or Relationships: Vital ER—husband is a chief medical officer. J.D. Schold reports Consultancy: eGenesis and Sanofi Corporation; Research Funding: One Legacy Foundation; Honoraria: eGenesis and Sanofi Inc.; Advisory or Leadership Role: Data Safety Monitoring Board Member—Bristol Myers Squibb and Vice Chair UNOS Data Advisory Committee; and Speakers Bureau: Sanofi. The remaining author has nothing to disclose.

Funding

S. Mohan was supported by NIH grants DK114893, DK116066, DK126739, DK130058, and MD014161 and a Nelson Family Faculty Development Award. S.A. Husain was supported by a Nelson Family Faculty Development Award and NIDDK grant K23DK133729. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Author Contributions

Conceptualization: Sumit Mohan.

Data curation: S. Ali Husain, Kristen L. King, Sumit Mohan, Miko Yu.

Formal analysis: Kristen L. King, Miko Yu.

Funding acquisition: Sumit Mohan.

Methodology: S. Ali Husain, Kristen L. King, Sumit Mohan, Miko Yu.

Resources: Sumit Mohan.

Supervision: Sumit Mohan.

Visualization: Kristen L. King, Miko Yu.

Writing – original draft: S. Ali Husain, Kristen L. King, Sumit Mohan, Miko Yu.

Writing – review & editing: Anne M. Huml, S. Ali Husain, Kristen L. King, Sumit Mohan, Rachel E. Patzer, Jesse D. Schold, Miko Yu.

Data Sharing Statement

The data reported here have been supplied by the United Network for Organ Sharing as the contractor for the Organ Procurement and Transplantation Network and the United States Renal Data System (USRDS). The interpretation and reporting of these data are the responsibility of the authors and in no way should be seen as an official policy of or interpretation by the OPTN or the US Government. The data reported here have been supplied by the Hennepin Healthcare Research Institute (HHRI) as the contractor for the Scientific Registry of Transplant Recipients (SRTR). The interpretation and reporting of these data are the responsibility of the author(s) and in no way should be seen as an official policy of or interpretation by the SRTR or the US Government.

Supplemental Material

This article contains the following supplemental material online at http://links.lww.com/JSN/E490.

Supplemental Table 1. Description of acronyms.

Supplemental Table 2. Description of all variables used, by data source.

Supplemental Table 3. Unadjusted Kaplan–Meier survival estimates of patients transplanted pre-KAS (2000–2014) and post-KAS (2015–2019), by data source.

Supplemental Table 4. Unadjusted Kaplan–Meier survival estimates of patients waitlisted pre-KAS (2000–2014) and post-KAS (2015–2019), by data source.

Supplemental Figure 1. Annual rates of death, by data source, among the total prevalent transplant population within the cohort.

Supplemental Figure 2. Close-up view of heat map showing transplant deaths in UNOS STAR. Year of transplant and year of death are ordered from 2000 to 2019.

Supplemental Figure 3. Heat maps of single-organ kidney transplant deaths identified in UNOS STAR (dated March 20, 2020), SRTR SAF (dated September 2, 2020), and USRDS (dated March 13, 2020).

Supplemental Figure 4. Heat maps of single-organ kidney transplant deaths after graft failure status identified in UNOS STAR (dated March 20, 2020), SRTR SAF (dated September 2, 2020), and USRDS (dated March 13, 2020).

Supplemental Figure 5. Heat maps of single-organ kidney transplant deaths after lost to follow-up status identified in in UNOS STAR (dated March 20, 2020) and SRTR SAF (dated September 2, 2020).

Supplemental Figure 6. Heat maps of nonpreemptive kidney waitlist deaths identified in UNOS STAR (dated March 20, 2020), SRTR SAF (dated September 2, 2020), and USRDS (dated March 13, 2020).