Abstract
Potential regional variations in effects of COVID-19 on federally mandated, program-specific evaluations by the Scientific Registry of Transplant Recipients (SRTR) have been controversial. SRTR January 2022 program evaluations ended transplant follow-up on March 12, 2020, and excluded transplants performed from March 13, 2020 to June 12, 2020 (the “carve-out”). This study examined the carve-out’s impact, and the effect of additionally censoring COVID-19 deaths, on first-year posttransplant outcomes for transplants from July 2018 through December 2020. Program-specific hazard ratios (HRs) for graft failure and death estimated under two alternative scenarios were compared with published HRs: (1) the carve-out was removed; (2) the carve-out was retained, but deaths due to COVID-19 were additionally censored. The HRs estimated by censoring COVID-19 deaths were highly correlated with those estimated with the carve-out alone (r2 = .96). Removal of the carve-out resulted in greater variation in HRs while remaining highly correlated (r2 = .82); however, little geographic impact of the carve-out was observed. The carve-out increased average HR in the Northwest by 0.049; carve-out plus censoring reduced average HR in the Midwest by 0.009. Other regions of the country were not significantly affected. Thus, the current COVID-19 carve-out does not appear to impart substantial bias based on the region of the country.
KEYWORDS: COVID-19, health services and outcomes research, organ procurement and transplantation network (OPTN), organ transplantation, scientific registry of transplant recipients (SRTR)
Abbreviations: HR, hazard ratio; HRSA, Health Resources and Services Administration; OPTN, Organ Procurement and Transplantation Network; SRTR, Scientific Registry of Transplant Recipients
1. INTRODUCTION
Prior to March 2020, transplant centers were performing solid organ transplants generally unaware of the coming global pandemic. On January 31, 2020, a national public health emergency was declared in the United States by the US Secretary of Health and Human Services,1 and on March 13, 2020, a national emergency was declared by the president of the United States.2 Numbers of transplants performed, particularly living donor transplants, dropped substantially in the first months of the pandemic.3 , 4 However, the numbers of deceased donor transplants performed in the United States largely recovered by June 20204 and even outpaced previous years.5
The Scientific Registry of Transplant Recipients (SRTR), charged with publishing evaluations of transplant programs semiannually,6 had the challenge of whether and how to address the unprecedented disruption due to the pandemic in the program evaluations. The SRTR worked with its SRTR Review Committee and its subcommittees to evaluate how best to address these challenges. For the program evaluations published in January 2021, the decision was made to end follow-up of all transplants in the cohort on March 12, 2020.7 This decision was made because it was apparent that, in the early days of the pandemic, transplant operations were greatly affected and transplant programs may have been differently affected based on a variety of factors.
By the July 2021 program evaluations, the cohort of transplant recipients used in the evaluations—patients who underwent transplant between January 1, 2018, and June 30, 2020, and followed up for death or graft failure until December 31, 2020—had experienced several waves of COVID-19 affecting all US regions.8 Operations of the transplant system in the United States had rebounded from the initial dip in donors and transplants.4 Starting with the July 2021 program evaluations, the current COVID-19 “carve-out,” defined in the following paragraph, was applied to the program evaluation cohorts.9
The cohorts for SRTR 1-year posttransplant evaluations include transplants that occur during a 2.5-year period with an additional 6 months of follow-up, with the cohort rolling forward by 6 months at each round of program evaluations. For example, the July 2021 program evaluations included transplants that occurred between January 1, 2018, and June 30, 2020, and were followed up for death or graft failure until December 31, 2020. The COVID-19 carve-out for the program evaluations ends follow-up for all transplants performed before March 13, 2020, on March 12, 2020, and excludes all transplants performed from March 13, 2020, through June 12, 2020. Transplants performed after June 12, 2020 are followed up as usual.
It is important to emphasize that, for transplants performed before March 13, 2020, those patients do not return to the risk set for the cohort after June 12, 2020, even if they are still alive with a functioning graft. Follow-up for these transplant patients is censored on March 12, 2020. With cohorts moving forward 6 months at each round of program evaluations, the carve-out continues to affect the first-year posttransplant program evaluations until January 2024, which will be the first round of evaluations with all transplants in the cohort performed after June 12, 2020.
Because COVID-19 has continued to affect different regions of the United States at different times, we studied whether the COVID-19 carve-out propagates any geographic bias in the program evaluations and if there are methods to address COVID-19 deaths that might further reduce geographic bias beyond the current carve-out. Therefore, this analysis aims to (1) quantify the impact of the carve-out on first-year posttransplant metrics in the January 2022 SRTR program evaluations, particularly if there is a regional bias imparted by the carve-out and (2) quantify the impact of censoring COVID-19 deaths on first-year posttransplant metrics in the January 2022 SRTR program evaluations.
2. METHODS
2.1. Study population
This study used data from SRTR. The SRTR data system includes data on all donors, waitlisted candidates, and transplant recipients in the United States, submitted by the members of the Organ Procurement and Transplantation Network (OPTN), and has been described elsewhere.10 The Health Resources and Services Administration (HRSA), US Department of Health and Human Services, provides oversight of the activities of the OPTN and SRTR contractors. SRTR data are updated monthly.
The January 2022 cohorts for heart, kidney, liver, and lung transplants, the four most common organs transplanted, were used. Per standard SRTR methods, these evaluation cohorts included recipients who underwent single-organ transplant from July 1, 2018, through December 31, 2020, with follow-up for death or graft failure until the earlier of 1 year after transplant or June 30, 2021. For the January 2022 published program evaluations, the COVID-19 carve-out was applied. That is, follow-up for all transplants performed before March 13, 2020, was censored on March 12, 2020; all transplants performed from March 13, 2020, through June 12, 2020, were excluded; transplants performed after June 12, 2020, were followed as usual. For this study, models and analyses were stratified by pediatric (younger than 18 years) and adult (18 years and older) candidates. Data were from the November 2021 version of the SRTR data.
This research conforms to US Federal Policy for the Protection of Human Subjects. The study was conducted as secondary research on data collected on behalf of the US Federal Government, and as such is not considered human subjects research.
2.2. Variables
The outcomes of interest for this analysis are graft failure—defined as reported graft failure (including return to dialysis for kidney recipients), retransplant, or death—and patient death. Program-specific performance is presented as observed to expected event hazards ratios (HRs) for each of the possible evaluations for each outcome. Each evaluation metric is characterized by a unique combination of transplant center, organ transplanted, and age group (adult or pediatric). For example, a center that transplants heart, kidney, and lung in adult and pediatric patients would have six evaluations for graft failure and six evaluations for patient death.
The SRTR data are based on data collected by the OPTN. For patients who die, there are fields on the OPTN Transplant Recipient Registration form and Transplant Recipient Follow-up form for primary cause of death and up to two contributory causes of death, and text fields to enter additional details. In April 2020, a code was added to these cause-of-death fields for “Infection: Viral-COVID-19.” Centers were allowed to retroactively apply this code to deaths due to COVID-19 that occurred before April 2020. Any patient with the code for COVID-19 cause of death in the primary or contributory cause fields, or with the term “COVID” in one of the cause-of-death text fields, was considered to have had a COVID-19 death for the purpose of censoring at COVID-19 deaths.
Regions of the country were defined broadly as the Northeast (OPTN Regions 1, 2, and 9), Southeast (OPTN Regions 3, 4, and 11), Midwest (OPTN Regions 7, 8, and 10), Northwest (OPTN Region 6), and Southwest (OPTN Region 5). Region groupings were chosen to reflect geographic contiguity and roughly similar timing of waves of the COVID-19 pandemic.
2.3. Statistical methods
To show waves of the pandemic, the January 2022 program evaluation cohort window with the carve-out was plotted against stacked daily COVID-19 incidence rate plots for the general population by OPTN region. Daily incidence counts by county from the New York Times dataset (https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv) and American Community Survey population by county (https://data.census.gov/cedsci/) were aggregated by OPTN region level to show daily incidence rates.
For each evaluation metric, HRs for posttransplant graft failure and posttransplant death were estimated as previously described.11 , 12 In brief, expected counts of graft failures and patient deaths are estimated from risk-adjusted Cox proportional hazard models.13 The expected values for graft failure and patient death at the level of the individual patient are summed to get expected numbers of graft failures or patient deaths at each center given the risk of the patients and donors transplanted by the center.
The SRTR January 2022 program evaluations applied the aforementioned COVID-19 carve-out. In this study, the HRs were re-estimated under two alternative scenarios to compare with the published HRs. In the first scenario, the COVID-19 carve-out was removed. In the second scenario, the COVID-19 carve-out was retained but deaths caused by COVID-19 that were not already carved out were censored, meaning they were not treated as a graft failure or patient death for the purpose of calculating either observed or expected number of events. All risk-adjustment models used to derive the expected event counts were rebuilt under each alternative scenario.
3. RESULTS
In the January 2022 program evaluation cohort, most OPTN regions had waves of COVID-19 from June 13, 2020, through December 31, 2020, when follow-up had resumed as normal. However, deaths during the waves from June 13, 2020, through December 31, 2020, were only counted as events for the program evaluations if the deaths occurred in patients who underwent transplant on or after June 13, 2020 ( Figure 1). The January 2022 surge in incidence rates, which is not a part of the evaluation cohort but appears at the far right of Figure 1, represented the beginning of the surge in COVID-19 incidence from the Omicron variant in early 2022, and will warrant future study.
FIGURE 1.
SRTR January 2022 program evaluation cohort with COVID-19 carve-out overlaid on daily COVID-19 incidence rate by OPTN region. OPTN, Organ Procurement and Transplantation Network
The January 2022 posttransplant program evaluation cohort, before applying the COVID-19 carve-out, included 89 017 heart, kidney, liver, and lung patients who underwent transplant at 257 unique centers from July 1, 2018, through December 31, 2020. When evaluating patient survival, retransplants were excluded, yielding a cohort of 82 086 transplant recipients. Patients were followed for a year after transplant or until June 30, 2021. Programs are evaluated separately for graft failure and patient death, and separately for adult and pediatric patients. Each center, therefore, may have up to four evaluations for each type of organ transplant they perform. In the January 2022 evaluation of first-year patient and graft survival, there were 1608 unique HRs on which heart, kidney, liver, and lung programs were evaluated ( Table 1).
TABLE 1.
Characteristics of transplant recipients and January 2022 program evaluations without the COVID-19 carve-out
| Total | Heart | Kidney | Liver | Lung | |
|---|---|---|---|---|---|
| Recipients | |||||
| Total recipientsa | 89 017 (100%) | 8204 (9.2%) | 54 533 (61.3%) | 19 813 (22.3%) | 6467 (7.3%) |
| Sex | |||||
| Female | 33 700 (37.9%) | 2462 (30%) | 21 293 (39%) | 7319 (36.9%) | 2626 (40.6%) |
| Male | 55 317 (62.1%) | 5742 (70%) | 33 240 (61%) | 12 494 (63.1%) | 3841 (59.4%) |
| Age group | |||||
| Adult | 84 407 (94.8%) | 6982 (85.1%) | 52 477 (96.2%) | 18 575 (93.8%) | 6373 (98.5%) |
| Pediatric | 4610 (5.2%) | 1222 (14.9%) | 2056 (3.8%) | 1238 (6.2%) | 94 (1.5%) |
| Race | |||||
| Asian | 5571 (6.3%) | 310 (3.8%) | 4187 (7.7%) | 902 (4.6%) | 172 (2.7%) |
| Black | 18 937 (21.3%) | 1845 (22.5%) | 14 987 (27.5%) | 1479 (7.5%) | 626 (9.7%) |
| Hispanic | 14 984 (16.8%) | 882 (10.8%) | 10 147 (18.6%) | 3284 (16.6%) | 671 (10.4%) |
| White | 48 112 (54%) | 5083 (62%) | 24 247 (44.5%) | 13 832 (69.8%) | 4950 (76.5%) |
| Mixed or other raceb | 1413 (1.6%) | 84 (1%) | 965 (1.8%) | 316 (1.6%) | 48 (0.7%) |
| Metrics | |||||
| Total evaluationsa | 1608 (100%) | 385 (23.9%) | 672 (41.8%) | 379 (23.6%) | 172 (10.7%) |
| Unique centersa,c | 257 (100%) | 144 (56%) | 239 (93%) | 145 (56.4%) | 71 (27.6%) |
| Age group | |||||
| Adult | 1077 (67%) | 256 (66.5%) | 435 (64.7%) | 254 (67%) | 132 (76.7%) |
| Pediatric | 531 (33%) | 129 (33.5%) | 237 (35.3%) | 125 (33%) | 40 (23.3%) |
| Outcome | |||||
| Graft failure | 808 (50.2%) | 193 (50.1%) | 339 (50.4%) | 190 (50.1%) | 86 (50%) |
| Patient death | 800 (49.8%) | 192 (49.9%) | 333 (49.6%) | 189 (49.9%) | 86 (50%) |
Values are given as row percentages.
Other races reported, though with numbers too small to report individually, were American Indian or Alaska Native, Native Hawaiian, or Pacific Islander.
Sums of organ values for unique centers exceed 257 (100%) because some centers do multiple types of transplant.
3.1. COVID-19 deaths
Among transplant recipients in the January 2022 posttransplant evaluation graft-survival cohort, prior to applying the COVID-19 carve-out, there were 5419 total graft failures. Applying the COVID-19 carve-out to the graft-survival cohort removed 1337 (24.7%) graft failures from the evaluation. COVID-19 was identified as the cause of death in 350 of the 5419 graft-survival cohort graft failures. Of the 350 COVID-19 graft failures, 181 (51.7%) were removed due to applying the COVID-19 carve-out ( Table 2).
TABLE 2.
Graft failure events and patient deaths in January 2022 program evaluations
| Total | Heart | Kidney | Liver | Lung | |
|---|---|---|---|---|---|
| Graft failure events | |||||
| Total graft failures | 5419 (100%) | 673 (100%) | 2492 (100%) | 1571 (100%) | 683 (100%) |
| Carved graft failures | 1337 (24.7%) | 124 (18.4%) | 641 (25.7%) | 383 (24.4%) | 189 (27.7%) |
| Total COVID-19 graft failures | 350 (100%) | 23 (100%) | 242 (100%) | 53 (100%) | 32 (100%) |
| Carved COVID-19 graft failures | 181 (51.7%) | 9 (39.1%) | 126 (52.1%) | 26 (49.1%) | 20 (62.5%) |
| Patient deaths | |||||
| Total patient deaths | 3742 (100%) | 626 (100%) | 1348 (100%) | 1135 (100%) | 633 (100%) |
| Carved patient deaths | 1044 (27.9%) | 118 (18.8%) | 442 (32.8%) | 307 (27%) | 177 (28%) |
| Total COVID-19 patient deaths | 349 (100%) | 22 (100%) | 244 (100%) | 52 (100%) | 31 (100%) |
| Carved COVID-19 patient deaths | 188 (53.9%) | 8 (36.4%) | 135 (55.3%) | 26 (50%) | 19 (61.3%) |
Note: Carved refers to the COVID-19 carve-out being applied.
Compared with the graft-survival cohort, which includes retransplants, the January 2022 posttransplant patient-survival cohort includes only patients after their first transplant for the organ of interest. Among recipients in the patient-survival cohort, there were 3742 total patient deaths. Applying the COVID-19 carve-out to the patient-survival cohort removed 1044 patient deaths from the evaluation. COVID-19 was identified as the cause of death in 349 of the 3742 patient-survival cohort patient deaths. Of the 349 COVID-19 patient deaths, 188 (53.9%) were removed due to applying the COVID-19 carve-out (Table 2).
Patients carved out (i.e., removed) from the graft-survival evaluations were significantly more likely to be men and adults (18+ years) ( Table 3). Patients not carved out but with a COVID-19 death indicated were significantly more likely to be adults (Table 3).
TABLE 3.
Demographic characteristics of program evaluation cohort patients carved out and not carved out
| Carved out | Not carved out | |
|---|---|---|
| Among all patients | ||
| Sex* | ||
| Female | 466 (34.85%) | 33 234 (37.9%) |
| Male | 871 (65.15%) | 54 446 (62.1%) |
| Age group* | ||
| Adult (≥18 years) | 1294 (96.78%) | 83 113 (94.79%) |
| Pediatric (<18 years) | 43 (3.22%) | 4567 (5.21%) |
| Ethnicity/race | ||
| Asian | 70 (5.24%) | 5501 (6.27%) |
| Black | 293 (21.91%) | 18 644 (21.26%) |
| Hispanic | 249 (18.62%) | 14 735 (16.81%) |
| White | 705 (52.73%) | 47 407 (54.07%) |
| Other | 20 (1.5%) | 1393 (1.59%) |
| COVID-19 death | No COVID-19 death | |
|---|---|---|
| Among patients not carved out | ||
| Sex | ||
| Female | 67 (39.64%) | 33 167 (37.9%) |
| Male | 102 (60.36%) | 54 344 (62.1%) |
| Age group* | ||
| Adult (≥18 years) | 169 (100%) | 82 944 (94.78%) |
| Pediatric (<18 years) | 0 (0%) | 4567 (5.22%) |
| Ethnicity/race | ||
| Asian | 9 (5.33%) | 5492 (6.28%) |
| Black | 37 (21.89%) | 18 607 (21.26%) |
| Hispanic | 38 (22.49%) | 14 697 (16.79%) |
| White | 82 (48.52%) | 47 325 (54.08%) |
| Other | 3 (1.78%) | 1390 (1.59%) |
Statistically significant at p < .05.
3.2. Scenario 1: Removing the carve-out
Compared with the HRs from the January 2022 program evaluations as published with the COVID-19 carve-out, removing the COVID-19 carve-out yielded HRs that were highly, but not perfectly, correlated with the published HRs (r 2 = .82) ( Figure 2). Compared with the January 2022 program evaluations with the COVID-19 carve-out, removing the carve-out did not cause statistically significant changes in HRs by organ type or age group, or between graft failure and patient death outcomes ( Table 4). By geography, the Northwest region (OPTN Region 6) saw a slight, but statistically significant, average decline in the HRs of 0.049 when the carve-out was removed, indicating slightly better program evaluations on average without the carve-out ( Figure 3).
FIGURE 2.
Correlation plots for SRTR January 2022 program evaluation HRs compared with alternate scenarios. HR, hazard ratio; PSR, program-specific report
TABLE 4.
Average hazard ratio (HR) differences for January 2022 program evaluations relative to alternative scenarios
| Scenario 1: No carve-out minus published HR | Scenario 2: Carve-out and COVID-19 censoring minus published HR | |
| Organ | ||
| Heart | 0.003 (−0.012 to 0.017) | −0.003 (−0.01 to 0.003) |
| Kidney | −0.007 (−0.018 to 0.004) | −0.001 (−0.006 to 0.004) |
| Liver | 0.013 (−0.001 to 0.028) | 0 (−0.006 to 0.007) |
| Lung | −0.001 (−0.022 to 0.021) | 0.001 (−0.009 to 0.01) |
| Age group | ||
| Adult | −0.001 (−0.01 to 0.007) | −0.002 (−0.005 to 0.002) |
| Pediatric | 0.004 (−0.008 to 0.016) | 0.000 (−0.005 to 0.005) |
| Outcome | ||
| Graft failure | 0.001 (−0.009 to 0.011) | −0.001 (−0.005 to 0.004) |
| Patient death | 0.000 (−0.009 to 0.01) | −0.001 (−0.006 to 0.003) |
Note: Values are given as HR difference (95% confidence interval). HR difference is HR from the scenario minus the published HR (i.e., HR with no carve-out minus HR from the published program evaluations).
FIGURE 3.
Correlation plots for SRTR January 2022 program evaluation HRs compared with alternate scenario removing the carve-out by geographic region. HR, hazard ratio; PSR, program-specific report
3.3. Scenario 2: Additionally censoring for COVID-19 deaths
Compared with the HRs from the January 2022 program evaluations as published with the COVID-19 carve-out, adding censoring for the COVID-19 cause-of-death events that were not already removed by the carve-out resulted in near-perfect correlation with the HRs as published (slope = 1, r 2 = .96) (Figure 2). Compared with the January 2022 program evaluations with the COVID-19 carve-out, additionally censoring for COVID-19 deaths did not cause statistically significant changes in HRs by organ type or age group, or between graft failure and patient death outcomes (Table 4). By geography, the Midwest (OPTN Regions 7, 8, and 10) saw a slight, but statistically significant, average decline in the HRs of 0.009 when COVID-19 censoring was added, indicating slightly better program evaluations on average with the addition of censoring for COVID-19 ( Figure 4).
FIGURE 4.
Correlation plots for SRTR January 2022 program evaluation HRs compared with alternate scenario additionally censoring COVID-19 deaths by geographic region. HR, hazard ratio; PSR, program-specific report
3.4. Scenario impacts on professional review flagging or preferred provider status
Beyond the average differences in HRs, the consequences of changes in performance metrics of individual transplant programs include being flagged for professional review or maintaining “preferred provider” or “center of excellence” status with insurance providers. If the COVID-19 carve-out were removed from the January 2022 program evaluations, out of 599 unique transplant programs (unique combinations of transplant center and organ transplanted), 18 programs would no longer be flagged, but 23 programs that were not flagged would have a flag added (Table S1), a net of five more programs flagged for professional review if the carve-out were removed ( Table 5). If censoring for COVID-19 deaths were added to the carve-out, six programs would no longer be flagged, but nine programs that were not flagged would have a flag added (Table S1), a net of three more programs flagged for professional review. Even though the average HR for Midwest centers would be lower under COVID-19 censoring, this scenario would lead to a net of three more programs in the Midwest flagged for professional review (Table 5). By organ, removing the carve-out would also cause bigger changes in which programs are flagged compared with censoring. For example, if the carve-out were removed, nine liver programs would have a flag added and two liver programs would have a flag removed while six kidney programs would have a flag added and 10 kidney programs would have a flag removed; by contrast, if COVID-19 censoring were added, no liver programs would have a flag added and one liver program would have a flag removed while five kidney programs would have a flag added and three kidney programs would have a flag removed (Table S2).
TABLE 5.
Programs with at least one MPSC flag under different evaluation scenarios
| Region | January 2022 program evaluation | Scenario 1: No COVID-19 carve-out | Scenario 2: Carve-out plus COVID-19 censoring |
|---|---|---|---|
| Midwest (n = 155 programs) | 33 | 36 | 36 |
| Northeast (n = 143 programs) | 29 | 27 | 28 |
| Northwest (n = 21 programs) | 2 | 1 | 2 |
| Southeast (n = 197 programs) | 30 | 37 | 32 |
| Southwest (n = 83 programs) | 9 | 7 | 8 |
Abbreviation: MPSC, Membership and Professional Standards Committee.
Preferred-provider status algorithms vary across insurance providers, but a potential proxy is an SRTR 5-tier evaluation for first-year graft survival being within tier 4 or tier 5. SRTR’s tier evaluation is presented publicly only for the graft failure outcome, and it is unique and important for each organ transplant program at a given transplant hospital or medical center. If the COVID-19 carve-out were removed, 35 programs would move into top-tier status, but 42 programs would move out of top-tier status (Tables S1 and S3), a net of seven fewer transplant programs with a tier-4 or tier-5 rating. If COVID-19 deaths were censored in addition to the original carve-out, 13 programs would move into top-tier status, but 10 programs would move out of top-tier status (Tables S1 and S3), a net of three more transplant programs with a tier-4 or tier-5 rating ( Table 6).
TABLE 6.
Transplant programs with a tier-4 or tier-5 SRTR graft survival evaluation
| Region | January 2022 program evaluation | Scenario 1: No COVID-19 carve-out | Scenario 2: Carve-out plus COVID-19 censoring |
|---|---|---|---|
| Midwest (n = 155 programs) | 74 | 77 | 78 |
| Northeast (n = 143 programs) | 69 | 62 | 72 |
| Northwest (n = 21 programs) | 14 | 14 | 12 |
| Southeast (n = 197 programs) | 103 | 97 | 99 |
| Southwest (n = 83 programs) | 49 | 52 | 51 |
Abbreviation: SRTR, Scientific Registry of Transplant Recipients.
4. DISCUSSION
This study showed that removing the COVID-19 carve-out from SRTR program evaluations would cause bigger changes in posttransplant evaluations than adding censoring for COVID-19 deaths. While there were regions of the United States that had slightly, but statistically significantly, lower average HRs under either scenario, the impact at the program level in terms of flagging for professional review or high-tier ratings does not seem to systematically or predictably benefit or detriment some regions of the country over others. Some authors have speculated that because the pandemic surges occurred at different time points in the country, that the existing SRTR carve-out introduces biases that specifically disadvantage certain areas of the country (e.g., the Midwest and the Southeast) and favor others (e.g., the East Coast and West Coast).13 Our analysis of the effects of the carve-out refutes this notion. Although there could be ways the Northwest and Midwest, which had slightly lower HRs when removing the carve-out and when adding censoring, respectively, were affected by the pandemic that were not fully accounted for by the method we used, it is difficult to speculate why this is. But we can note that the effect was rather small, with HRs moving by an average of 0.049 for the Northwest when removing the carve-out and 0.009 for the Midwest when adding censoring for COVID-19 cause of death.
While differential impacts of COVID-19 by region was one reason cited for the initial carve-out in the January 2021 SRTR program evaluations,7 by the January 2022 program evaluations, all regions of the country had experienced waves of COVID-19, thus smoothing out geographic variation over the cohort. However, the smallest unit available for incidence data is the county level, so comparisons, for example, between individual centers or between urban and rural areas, are not possible. The decision to continue the COVID-19 carve-out, endorsed by the SRTR Review Committee,9 was made to avoid penalizing transplant programs for transplants performed before, or during the first 2 months of, a global pandemic that they did not anticipate, which brought unprecedented disruption to the US transplant system.5 Although the general population incidence rates for COVID-19 during the carved-out months was highest in the Northeastern United States, this does not mean that only the Northeast benefits from the carve-out. For example, while the Southeast began to have waves later in 2020 and in early 2021, centers in the Southeast would not have COVID-19 deaths counted as program evaluation events for transplants performed before June 13, 2020.
We explored additionally censoring COVID-19 deaths among recipients who underwent transplant on or after June 13, 2020. Relative to removing the carve-out, adding censoring has a small impact on the program evaluation HRs, because many COVID-19 deaths are already effectively censored with the carve-out. The SRTR has not previously adjudicated “transplant-related” and “nonrelated” outcomes, as censoring for certain causes of death can hide information that is of interest to patients, whose main interest is often this type of question: What are my chances at this center of surviving 1 year, 5 years, 10 years, and so on? SRTR’s Review Committee continues to review the pandemic at its quarterly meetings and to date has voted not to remove the carve-out, although SRTR has heard similar requests to report the data as observed without the carve-out. As the pandemic has progressed, centers have developed strategies and tools and vaccines have become available to improve patient outcomes. Further censoring for COVID-19 death later in the pandemic may obscure the effect of these improving practices and differences in their implementation across programs. However, we are currently unaware of any data collected at the national level on specific COVID-19 management practices at individual centers or organ procurement organizations that could quantify these best practices; we acknowledge this as an opportunity for future study.
Additionally, even adding censoring for COVID-19 cause of death is not universally beneficial to transplant program evaluations and could have a net detrimental effect for programs on professional review flagging. This is because program evaluation HRs are risk-adjusted, meaning that, in addition to removing observed events, censoring changes the calculation of expected events.11 , 12 Conversely, because COVID-19 has had differential impacts across demographics14, 15, 16 that are already included in the SRTR risk-adjustment models, expected graft failure or patient mortality are likely to be higher at programs where patients are at higher risk of infection or death due to COVID-19; this may mitigate the effect of COVID-19 on evaluations of programs serving vulnerable populations, even without censoring for COVID-19 deaths because higher expected graft failures will correspond to lower HRs given the same number of observed events. Risk adjustment is a much more versatile and precise method than censoring (risk-adjustment model coefficients by organ and scenario: Tables S4–S8).
COVID-19 has been a possible cause of death in OPTN reporting since April 2020. It is not yet clear how completely centers are reporting COVID-19 deaths. However, beyond concerns about underreporting of direct COVID-19 deaths, COVID-19 can also be an indirect factor even among people who did not have a COVID-19 infection (eg, delaying medical care). Therefore, excess mortality due to COVID-19 may be greater than the direct COVID-19 deaths reported in OPTN and SRTR data.17 Uncertainty in attributing deaths to COVID-19, due to both possible underreporting and inability to account for indirect effects of COVID-19, is an additional reason not to pursue censoring COVID-19 deaths.
With the continued support of its Review Committee, SRTR will continue publishing program-specific reports with the carve-out, which will pass out of the cohort in January 2024, and will not add any cause-of-death censoring. We will continue to monitor the COVID-19 pandemic and explore, as needed, ways to address the effects of COVID-19 on monitoring and reporting, particularly for new evaluation metrics recently introduced for professional review—transplant acceptance rate, 90-day post-transplant mortality, and 1-year survival conditional on 90-day survival—to which these results may not fully extrapolate. Pretransplant (waitlist) mortality rates will additionally be introduced as a new metric for program review, although not until after the COVID-19 carve-out is no longer affecting the cohorts.
ACKNOWLEDGMENT
This work was conducted under the auspices of the Hennepin Healthcare Research Institute (HHRI), contractor for the SRTR, as a deliverable under contract no. 75R60220C00011 (US Department of Health and Human Services, Health Resources and Services Administration, Healthcare Systems Bureau, Division of Transplantation). The US Government (and others acting on its behalf) retains a paid-up, nonexclusive, irrevocable, worldwide license for all works produced under the SRTR contract, and to reproduce them, prepare derivative works, distribute copies to the public, and perform publicly and display publicly, by or on behalf of the Government. The data reported here have been supplied by HHRI as the contractor for SRTR. The interpretation and reporting of these data are the responsibility of the author(s) and in no way should be seen as an official policy of or interpretation by SRTR or the US Government. The authors thank SRTR colleague Anna Gillette for manuscript editing.
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available from the corresponding.
author upon reasonable request.
DISCLOSURE
The authors of this manuscript have conflicts of interest to disclose as described by the American Journal of Transplantation. Dr. Israni has received research funds awarded to Hennepin Healthcare Research Institute from Gilead for investigator-initiated research related to Hepatitis C. Dr. Israni is supported by grants and contracts awarded to Hennepin Healthcare from Health Resources and Services Administration, National Institute of Allergy and Immunology, National Institute of Diabetes and Digestive and Kidney Diseases and the Agency of Healthcare Research and Quality. He has served on an Advisory Board for CSL Behring. Dr. Israni receives consulting fees from Medical Review Institute of America. Dr. Israni has family members that are employees of Astra Zeneca and GSK pharmaceutical companies. The other authors have no conflicts of interest to declare.
SUPPORTING INFORMATION
Additional supporting information may be found in the online version of the article at the publisher’s website.
Table S1
Table S2
Table S3
Table S4
Table S5
Table S6
Table S7
Table S8
REFERENCES
- 1.US Department of Health and Human Services. Determination that a Public Health Emergency Exists. 2020. Accessed February 11, 2022. https://www.phe.gov/emergency/news/healthactions/phe/Pages/2019-nCoV.aspx
- 2.President of the United States. Proclamation on Declaring a National Emergency Concerning the Novel Coronavirus Disease (COVID-19) Outbreak. Published 2020. Accessed February 11, 2022. https://trumpwhitehouse.archives.gov/presidential-actions/proclamation-declaring-national-emergency-concerning-novel-coronavirus-disease-covid-19-outbreak/
- 3.Scientific Registry of Transplant Recipients. Effect of COVID-19 on the Transplant System. 2021. Accessed February 11, 2022. https://www.srtr.org/tools/covid-19-evaluation/
- 4.United Network for Organ Sharing. COVID-19 and solid organ transplant. Published 2022. Accessed February 11, 2022. https://unos.org/covid/
- 5.Miller J, Wey A, Ahn YS, et al. OPTN/SRTR 2020 Annual Data Report: COVID. Am J Transplant. 2022;22(S2):587–622. doi: 10.1111/ajt.16981. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.42 CFR 121.11. Organ Procurement and Transplantation Network: Record Maintenance and Reporting Requirements. Accessed June 27, 2022. https://www.law.cornell.edu/cfr/text/42/121.11
- 7.Scientific Registry of Transplant Recipients. COVID-19 Changes: Upcoming Adjustments to Transplant Program and OPO Evaluation Metrics. 2021. Accessed February 11, 2022. https://www.srtr.org/news-media/news/news-items/news/#covid19psrosrchanges
- 8.Centers for Disease Control. Compare Trends in COVID-19 Cases and Deaths in the US. 2022. Accessed February 11, 2022. https://covid.cdc.gov/covid-data-tracker/#compare-trends_newcasessubmission
- 9.Scientific Registry of Transplant Recipients. COVID-19 Changes for July 2021 Reports: upcoming adjustments to transplant program and OPO evaluation metrics. 2021. Accessed February 11, 2022. https://www.srtr.org/news-media/news/news-items/news/#COVID19ChangesJuly2021
- 10.Salkowski N, Snyder JJ, Zaun DA, Leighton T, Israni AK, Kasiske BL. Bayesian methods for assessing transplant program performance. Am J Transplant. 2014;14(6):1271–1276. doi: 10.1111/ajt.12707. doi: [DOI] [PubMed] [Google Scholar]
- 11.Scientific Registry of Transplant Recipients. Technical Methods for the Program-Specific Reports. 2022. Accessed February 11, 2022. https://www.srtr.org/about-the-data/technical-methods-for-the-program-specific-reports/
- 12.Scientific Registry of Transplant Recipients. SRTR Risk Adjustment Model Documentation: Posttransplant Outcomes. 2022. Accessed February 11, 2022. https://www.srtr.org/tools/posttransplant-outcomes/
- 13.Subramanian V, Anderson C, Karp S, et al. COVID-19 and transplantation – data censoring. Am J Transplant. 2022:1–5. doi: 10.1111/ajt.17065. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Romano SD, Blackstock AJ, Taylor EV, et al. Trends in racial and ethnic disparities in COVID-19 hospitalizations, by region — United States, march–December 2020. MMWR Morb Mortal Wkly Rep. 2021;70(15):560–565. doi: 10.15585/mmwr.mm7015e2. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Yancy CW. COVID-19 and African Americans. J Am Med Assoc. 2020;323(19):1891–1892. doi: 10.1001/jama.2020.6548. doi: [DOI] [PubMed] [Google Scholar]
- 16.Dorjee K, Kim H, Bonomo E, Dolma R. Prevalence and predictors of death and severe disease in patients hospitalized due to COVID-19: a comprehensive systematic review and meta-analysis of 77 studies and 38,000 patients. PLoS One. 2020;15(12):1. doi: 10.1371/journal.pone.0243191. e0243191-27, December, doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Massie AB, Werbel WA, Avery RK, Chiang TPY, Snyder JJ, Segev DL. Quantifying excess deaths among solid organ transplant recipients in the COVID-19 era. Am J Transplant. 2022 doi: 10.1111/ajt.17036. Forthcomin. doi: [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Table S1
Table S2
Table S3
Table S4
Table S5
Table S6
Table S7
Table S8
Data Availability Statement
The data that support the findings of this study are available from the corresponding.
author upon reasonable request.