Improving 1-year liver allograft survival hazard ratios

Abstract

Background

The Scientific Registry for Transplant Recipients (SRTR) publishes outcomes of all transplant centres in the USA two times a year. The outcomes are publicly available and used by insurance payers and patients to assess the performance of a programme. Poor performance can result in temporary suspension or termination of a transplant programme. The estimated 1-year survival hazard ratio (EHR) is an important metric publicly reported by the SRTR.

Problem

The EHR at our institution was 1.13, indicating a graft loss rate that was 13% higher than the national average.

Methods/INTERVENTION

We defined an improvement in this metric as achieving an EHR of <1.0. Our balance measure was maintaining similar liver transplant volumes and avoiding limiting access to transplant. Using a causality tree, we identified there was no ‘real time’ assessment of programme risk or objective metric to assess this. An affinity diagram was used to determine high and intermediate risk factors for mortality and graft loss and, using a REDCap form (a web application used to manage our database) to track actual and potential complications, we calculated a weekly ‘risk metric’ that was introduced at multidisciplinary selection conference meetings.

Results

We remeasured our EHR at each interval release of the SRTR outcomes and found it to be 0.98 and 0.65 after implementing the ‘risk metric.’ During the intervention period, annual liver transplant volume remained above the baseline measure.

Conclusion

By implementing a ‘risk metric’ to prospectively assess the risk of a low EHR at transplant selection committee meetings, we were able to reduce the EHR well below the national average without limiting access to liver transplants.

WHAT IS ALREADY KNOWN ON THIS TOPIC

• The published Scientific Registry for Transplant Recipients (SRTR) outcomes reflect the quality of care provided at transplant centres and are determined by the patient and graft survival. The patient and graft survival are impacted by the patients selected for transplant and the health of the existing cohort of transplanted patients.

WHAT THIS STUDY ADDS

• We developed a program-level ‘risk metric’ to provide an objective assessment used in multidisciplinary selection conferences. This assesses the health of the transplanted patient cohort and a programme’s risk of poor SRTR outcomes.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• This ‘risk metric’ adds objective and real-time data to selection committee decisions, rather than prior experience or a ‘last case bias.’ This will be important when considering transplant surgery for a patient with multiple risk factors for morbidity and mortality.

Introduction

The Department of Health and Human Services established the Organ Procurement and Transplantation Network (OPTN) under the National Organ Transplantation Act of 1984 to maintain a national system for matching organs to patients.¹ The Scientific Registry for Transplant Recipients (SRTR) is an organisation that calculates and publishes patient and transplanted organ graft outcomes for all transplant centres in the USA based on data from the OPTN. The SRTR outcomes report allows patients and insurance companies to compare unbiased performance metrics of any transplant programme in the USA.² Outcomes from the SRTR are released semiannually, in the Spring and Fall, for the prior 18-month period per transplant centre.³

The ‘expected graft survival’ is the fraction of organ grafts that would be expected to be functioning at a reported time point. This metric is based on the national experience of patients who are similar to those transplanted at any centre.³ The expected graft survival for each organ is adjusted for patient, donor and transplant factors based on risk-adjustment models that are available on the SRTR website.⁴ SRTR then reports the ratio of the actual graft failure rate to the expected graft failure rate as a metric of the transplant centre’s outcomes. A ratio >1.0 indicates that more graft failures occurred than expected compared with the national experience, while a ratio <1.0 indicates fewer graft failures occurred than the national experience.⁴

The SRTR outcomes are used by insurance providers (payers) and governing bodies to evaluate a transplant programme’s performance and highlighted in media reports.² Payers can award increasing medical coverage and contracts, and their designation of ‘Centre of Excellence’ (COE) is based on the published SRTR outcomes. Transplant programmes that experience poor outcomes, regardless of statistical significance, may have their COE designation or payer contracts revoked.⁵ Governing bodies, such as Centres for Medicare and Medicaid Services and OPTN, frequently monitor transplant programme outcomes across the USA. Transplant programmes that perform below the national average of graft failures are subject to an extensive review and intervention, which can include programme suspension or termination.⁴ To avoid such unwanted outcomes, transplant programmes can evaluate the above-mentioned patient, donor and transplant factors and change their practices. However, other factors that may not

be known at the time of transplant listing impact post-transplant outcomes as well. Thus, to have a more accurate program-level risk adjustment model that incorporates more data points into a transplant graft survival predictor, we designed a quality initiative to incorporate more data into transplant graft survival prediction and provide a more accurate programme-level risk adjustment model. The project’s main goal was to improve patient care, experience and overall quality of life after liver transplant.

While extending graft survival and reducing patient mortality is important for better use of scarce donor organs, we also wanted to avoid limiting access to liver transplant. Thus, our balance measure was the 5-year average transplant volumes.

Methods

The baseline measure of the estimated 1-year graft survival hazard ratio (EHR) was 1.13 based on a population of 254 liver transplants performed at Mayo Clinic Hospital Minnesota (MNMC) as published by the SRTR as shown in figure 1 (7 and 6), indicating that the actual graft loss within the first year was 13% higher than expected by the SRTR metrics. The EHR for MNMC in the two SRTR cycles prior to the baseline was 1.29 and 1.23. This high EHR for a sustained period prompted an intervention by the department leadership. The baseline annual liver transplant volume at MNMC was 120 patients. This was calculated based on 598 liver transplants performed at MNMC over the preceding 5 years, 2015–2019.⁶ We also calculated the baseline monthly liver transplant volumes for closer monitoring of this metric which was 10 liver transplants per month (figure 2).

Figure 1. The baseline improvement measure. The estimated 1-year survival hazard ratio (EHR) for MNMC liver transplant programme is 1.13 in August of 2020. The historic EHR from the past 5 years is also shown and notably peaked at 1.29 in July 2019, as shown in the figure. MNMC, Mayo Clinic Hospital Minnesota.

Figure 2. The baseline balance measure. The top panel (A) demonstrates the yearly transplant volumes at MNMC. The bold dotted line demonstrates the yearly average transplant volumes from 2015 to 2019, which was 120 transplants. The bottom panel (B) shows the average monthly transplant volumes at MNMC from 2015 to 2019. The bold dotted line shows a monthly average of 10 transplants. The soft dotted line in the top and bottom panels represents the upper and lower control limits, which were three SDs of the mean.

The MNMC multidisciplinary transplant selection committee is made up of physicians, surgeons, nurses, social workers, registered dietitians and pharmacists. Committee members brainstormed potential causes of poor transplant outcomes and the high EHR. They used a Causality Tree to identify crucial factors that would contribute to an EHR >1.0 (figure 3). Controllable variables included (1) some graft losses may be directly related to the potential recipient’s medical risk factors and comorbidities, (2) the selection committee assesses the transplant programme’s risk retrospectively and not in ‘real-time’, therefore, (3) the selection committee does not incorporate ‘real-time’ and objective data but previous data that can be subject to recall bias, and (4) the programme risk is not being assessed during selection committee meetings.

Figure 3. Causality tree. This was created by liver transplant selection committee members pursuing the root causes of the high estimated 1-year survival HR.

The causality tree was used to pursue the root causes of our transplant programme’s high EHR. Although baseline selection criteria included known, preoperative risk factors (ie, coronary artery disease, medical comorbidities and history of malignancy), we also needed to include risk factors identified during and after liver transplant (ie, intraoperative or perioperative cardiac events, explant pathology findings, recurrent malignancy or graft vs host disease) which impact the risk for a higher EHR for graft loss and not accounted for in the SRTR metrics. The committee members also identified that there was no objective metric to incorporate post-transplant risk into the selection committee’s decisions.

High and intermediate patient risk factors for graft loss were identified and a risk calculation was developed to estimate the transplant programme’s risk for an increasing EHR based on post-transplant patients. This was done by reviewing 24 cases of graft loss over a 5-year period and an affinity diagram was created for the strongest predictors for future graft loss (figure 4). A REDCap form was used to create a programme risk calculator based on the clinical factors for graft loss from the same group of patients who underwent a liver transplant. This REDCap tool produced a ‘risk metric’ that was shared at the start of each weekly selection committee meeting. The following is a list of weighted risk factors assigned to post-transplant patients who were within 1 year of transplant:

Figure 4. Affinity diagram. Patient factors for high and intermediate risk of 1-year graft loss used in the calculation of the ‘risk metric’. LOS: Length of Stay.

1. High risk (>50% likelihood of 1-year graft loss based on a review of the historical graft losses; these were given a multiplier of 1, meaning they are expected to have a 1-year graft loss):

   1. Recurrent cancer

   2. Advanced stage de novo cancer (including post-transplant lymphoproliferative disorder)

   3. Graft versus host disease

   4. Severe, untreatable neurologic deficit

2. Intermediate risk (25%–50% likelihood of 1-year graft loss based on a review of historical graft losses; these were initially given a multiplier of 0.5 and revised to 0.33 after the first interim analysis. Multiplier of 0.33 means that every third case is expected to fail):

   1. Severe cardiac complication

   2. Initial post-transplant hospital stays >4 weeks AND age >60 years

   3. Nursing home placement post-transplant AND age >60 years

   4. Severe frailty

   5. Disseminated infection

   6. Increased risk due to surgical/anatomical complexity

The weighted numbers were added together and interpreted as a numerical representation of cumulative risk or the programme ‘risk metric’. A cumulative risk total >9 was associated with high programmatic risk, 5–9 was moderate risk and<5 was low risk.

We used the Plan-Do-Study-Act (PDSA) methods to evaluate the ‘risk metric’ implementation in practice in cycles; PDSA 1 was from 2 September 2020, through 12 October 2021, and PDSA 2 was from 13 October 2021, through 6 January 2022. During PDSA 1 and PDSA 2, we used the EHR released by the SRTR as quantitative measures of improvement. The monthly and annual liver transplant volumes were also monitored during the PDSA 1 and PDSA 2. We aim to have an EHR of <1.0 and have the transplant volumes stay with three SD of the yearly and monthly mean, which were 120 and 10 liver transplants, respectively.

There were ethical considerations for implementing the risk metric into the transplant selection conference. Presenting this metric at the beginning of the selection conference can influence the decisions of the selection committee, potentially denying patients for transplant after being presented with metrics that indicate poor outcomes. At the time of implementation, the risk metric was unvalidated and piloted at our centre. Although the SRTR outcomes at our centre improved, we did consider the ethics of limiting access to liver transplants with an unvalidated metric. However, the number of transplant volumes at our centre did not change and access to transplant remained the same. It should also be noted that there were no other interventions or different approaches to patient evaluation prior to the implementation of the risk metric.

Results

The intervention for quality improvement was introducing the ‘risk metric’ at the start of every selection conference meeting starting in September 2020. We then assessed our programme’s EHR at two intervals during PDSA 1; the EHR was 1.12 in January 2021 (above the target) followed by 0.98 in July 2021 (meeting the target). The EHR by the end of PDSA 2 was 0.65 in January 2022⁷ (meeting the target) as shown in figure 5. The project was subsequently closed after the goal of EHR was achieved in these two consecutive SRTR outcome releases and the intervention was implemented into practice. We continued to monitor our results to validate that the direct actions from our PDSA cycles did indeed impact the EHRs. PDSA 1 was from 2 September 2020 to 12 October 2021. Results from this PDSA cycle will be measured in the SRTR releases beginning in January 2022 through January 2025. PDSA 2 was from 13 October 2021 to 6 January 2022. The results from this PDSA will be observed in the SRTR releases beginning in July 2023 through July 2025.

Figure 5. The remeasured improvement measure. The EHR after the quality intervention is demonstrated in the figure. The baseline is demonstrated as the first data point in August 2020. After implementation, the EHR decreases and eventually reaches the estimated 1-year survival HR of <1.0. EHR, estimated 1-year survival hazard ratio; PDSA, Plan-Do-Study-Act; SRTR, Scientific Registry for Transplant Recipients.

The most recent intervals were based on 240 patients (PDSA 1) and 234 patients (PDSA 2), respectively. Notably, we included all the transplants performed in 2020 as PDSA 1 began in September 2020. Our balance measure was reassessed, comparing our annual and monthly liver transplant volumes while the intervention was implemented to the baseline liver transplant volumes (figure 6).

Figure 6. The remeasured balance measure. The top panel (A)demonstrates the yearly transplant volumes during the intervention period. The bottom panel (B)shows the monthly average transplant volumes during the intervention period. The bold dotted line in both panels represents the baseline average transplant volume. The soft dotted line in the top and bottom panel represents the upper and lower control limits, which are three SD of the mean.

Discussion

The SRTR outcomes are publicly available for review and, if unfavourable, can negatively impact a transplant programme. Poor performance can result in a loss of payer contracts from insurance companies, removal of designations and even programme suspension or termination.³ At baseline, our EHR was 1.13 and had peaked at 1.29 previously.⁶ We were able to reduce the EHR to 0.65 by the end of PDSA 2 by implementing a ‘risk metric’ for an overall risk assessment of the transplant programme for poor performance regarding allograft survival. This is a prospective way of measuring the programme’s risk of poor SRTR outcomes based on existing transplanted patients who have risk factors for increased mortality and graft loss. Selection committee decisions are then made based on objective and ‘real-time’ data rather than previous data or experience that is subject to recall or ‘last case’ bias. Other strengths of this project were that we sampled a similar number of patients reviewed at selection committee meetings compared with our baseline years measured. The sample period was large, spanning over 18 months.

Implementing this ‘risk metric’ into practice carries the risk of promoting a behaviour of excessive risk aversion during selection committee meetings and potentially limiting access to liver transplants for patients. We maintained our existing access to transplant by monitoring yearly and monthly transplant volumes as a balance measure to the intervention. Over the course of the PDSA cycles, the liver transplant volumes at our centre remained the same compared with the baseline volumes before the intervention. We noted the monthly liver transplant volumes approached and/or exceeded the third SD of the mean; however, it remained within these limits based on yearly transplant volumes. The approval rate percentage after the implementation was not different from the approval rate before, suggesting that we did not deny more patients during the study period.

The methods used to improve the primary outcomes are reproducible and can be expanded for use in our organ transplant disciplines and other transplant centres. However, the high and intermediate risk factors shown in the affinity diagram (figure 4) may be different at other centres because of different transplant volumes, geography or patient complexity. Our ‘risk metric’ is now a sustained implementation into our practice, and the next steps would be to validate its use at an external transplant centre or even in other departments such as nephrology or cardiology.

Illustrative case and application of the risk metric

A 71-year-old man presented for evaluation of a third liver transplant for decompensated cirrhosis. He had a history of a liver transplant 5 years ago and 15 years ago for primary sclerosing cholangitis (PSC) and now has recurrent PSC and cirrhosis of his second allograft. He was recently hospitalised for cholangitis and septic shock with a 2-month hospital stay. His hospital course was complicated by an ischaemic stroke with mild neurologic deficits, non-ST elevated myocardial infarction and hepatic encephalopathy. He is sarcopenic and his Body Mass Index (BMI) is 19. His Model for End Stafe Liver Disease (MELD) score is 26 at this clinic visit.

The patient’s history and work up are reviewed by the transplant selection committee who recognises the high perioperative mortality risk. This case generates a very lively discussion about the benefits this patient would have from a liver transplant but the risk the programme would undertake in deciding to approve him for a liver transplant. The transplant programme has had an EHR >1.0 for the past three SRTR cycles. The programmatic risk metric indicates a high risk of further increases in the transplant programme EHR.

After careful consideration and acknowledging the programmatic risk, the decision was made to close this patient’s evaluation and not proceed with a liver transplant.

Conclusion

Implementing a global programme risk assessment into multidisciplinary transplant selection committee meetings is an effective method for improving the 1-year graft survival hazard ratio (EHR), a metric that is monitored and published by the SRTR. By implementing the programme ‘risk metric’, we achieved our projected goal of an EHR <1.0 during the study period without compromising access to a liver transplant.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.