Influence of Reporting Methods of Outcomes across Transplant Centers

Transplant “center effect” is a phenomenon that describes the differences in the clinical outcomes of transplantation (graft and/or patient survival) which exist between transplant centers after accounting for the between-center variability in risk factors known to be associated with transplant outcomes. Transplant center effect was first described in 1975 by Opelz et al., who reported substantial variability across 95 kidney transplant centers in the 1-year graft survival of nearly 5000 deceased and living-donor kidney transplant recipients (1). Subsequently, several multivariate analyses from large databases consistently found that 25% to 30% of assignable variation in 1-year kidney graft survival was due to the phenomenon of center effect (2–6). Transplant center effect captures between-center differences in outcomes due to reasons that could not be established with available data. Center effect means that the probability of a successful kidney transplant is independently influenced by the transplant center where the transplant procedure occurred. In the last decade, the stakes consequent to marked differences in transplant outcomes between centers have dramatically risen because of several developments: (1) the number of kidney transplant centers in the United States has risen more than 3-fold since transplant center effect was first reported; (2) the standardized rates of patient and graft survival for each center is now published annually; (3) medical insurance providers and transplant candidates may select transplant centers on the basis of the published transplant outcomes; and (4) transplant centers are subject to probationary measures and risk decertification by the Centers for Medicare and Medicaid Services if the center-specific reports (CSRs) show consistent underperformance relative to expected outcomes.

The manuscript by Zenios et al. in this issue of CJASN is the first rigorous comparison of the statistical methods used to estimate patient and graft survival in the CSRs jointly published annually by the Organ Procurement and Transplantation Network and the Scientific Registry of Transplant Recipients (7). Zenios et al. demonstrated convincingly that the 3-year patient and graft survival estimated by their generalized mixed effect (ME) method is more accurate and less likely to lead to misclassification of a transplant center when compared with the observed-to-expected ratios (OE) method. With this finding, Zenios and colleagues have rendered a valuable service to the transplant community by showing that the current method utilized in the CSRs can be improved. As correctly pointed out by Zenios and colleagues, the ME and OE have significant limitations. Sophisticated statistical methods such as the cumulative summation technique and high dimensional instrumental variable selection may be more accurate and informative than the ME and OE methods and, hopefully, the study by Zenios and colleagues will stimulate additional methodologic research on the estimation of center-specific graft and patient survival rates (8–10). The importance attached to differences between centers in transplant outcomes is too great to continue to rely on statistical methods prone to significant imprecision. Although Zenios et al. noted that ME and OE methods are complicated and not intuitively straightforward enough to explain to a large fraction of the consuming audience, striving for simplification of the methodologic approach should yield to the higher priority of precision because of the high cost and the consequences implicit in biased survival estimates or the misclassification of a transplant center.

According to Zenios and colleagues, 33% of all deaths and 29% of all graft failures during the first 3 years after transplantation were attributable to transplant center effect. This magnitude of “unexplained” variation in results between transplant centers was similar to that previously reported in several studies that were conducted in the United States in the 1980s and 1990s (3,11–13). Zenios and colleagues also found that despite the publication of CSRs since 2001, the center effect has remained essentially unchanged. The expectation that the publication of CSRs would incentivize the underperforming transplant centers to improve their results has not materialized. This raises the question “Why is transplant center effect so intractable?” An obvious answer is that we do not know what “transplant center effect” is and, therefore, cannot measure it, let alone rectify it. First, the definition of transplant center effect embodies differences in outcomes due to unknown or unmeasured factors. Second, transplant center effect has remained elusive to mitigate, in part, because the tool used to measure it is not optimal. Third, the large databases used in the analysis of transplant outcomes and center differences do not contain the detailed clinical variables that are known to have an impact on graft and patient survival. By not accounting for these important clinical determinants of outcomes, one may miss the underlying differences among transplant recipients that drive graft and patient survival and, yet, are differentially distributed among transplant centers. For example, the database would typically have a variable to assess the presence of peripheral vascular disease as a dichotomous yes/no response; however, a transplant candidate with a single toe amputation due to peripheral vascular disease would likely have a very different transplant prognosis compared with a transplant candidate who presents with bilateral above the knee amputations due to peripheral vascular disease. In the era of electronic medical records, it should soon be possible to evaluate center-specific transplant results with clinical data of such fine granularity so as to capture most of the between-patient differences that are currently subsumed in between-center differences. Lastly, transplant center and provider characteristics along with practice pattern information are not captured in any meaningful detail in the database used to estimate center-specific transplant outcomes. If center effect is due to differences in these organizational- and provider-based variables, the current measurement would remain blind to them and hence, lend a false appearance of intractability. The findings by Zenios et al. that 29% to 33% of graft failures and deaths are due to center differences is a clarion call for multicenter studies of practice pattern in transplantation similar to the Dialysis Practice Pattern and Outcomes Studies that has been ongoing in the dialysis population for more than a decade (14).

The study by Zenios et al. and other studies of transplant center effect did not address the role of transplant recipient follow-up, which is an important correlate of transplant center effect that may lead to misguided actions by the Centers for Medicare and Medicaid Services and poorly targeted remedial steps by individual transplant centers. It is well established that the 1-year graft and patient survival is largely determined by early posttransplant events (15) that occur when the patient is under the care of the transplant center. Is it tenable to assign responsibility for 3- and 5-year transplant outcomes to the transplant center in an environment where most of the transplant recipient medical management occurs remotely from the transplant center in time and space? In the United States, only 26% of transplant recipients remain in the primary care of the transplanting center for more than 1 year after transplant, and 31% of recipients had no outpatient visit to any transplant center 24 to 36 months after transplantation. For many kidney transplant recipients, the only connection to the clinical center where the transplant was performed is limited to yearly visits, except when a transplant-related complication develops, although published guidelines recommend outpatient visits every 6 months. Unlike in dialysis patients for which the nephrologist is reimbursed on a capitated basis for professional services for patients under their care, long-term follow-up of kidney transplant patients' care is reimbursed under a scheme that invariably leads to a financial shortfall for the practice. The leading causes of late graft loss and death are cardiovascular disease and chronic allograft nephropathy, which are medical conditions that may not be detected or are managed remotely from the transplant center. In concert, several studies have found that the transplant center effect was most pronounced in the first year after transplantation and most of the differences in the results across transplant centers dissipate by the fifth year after transplantation (11–13).

Lastly, patient-based factors such as income, compliance with immunosuppressive medications, and insurance coverage for medications are important variables that have been associated with the risk of death and graft failure. These variables often vary across transplant centers but are not taken into account in the estimation of center-specific results except sometimes when their proxies are used (e.g., median income for the zip code). Transplant centers with large fractions of low-income transplant recipients are thus likely to have inferior results in the CSRs not because of any center-based modifiable factors, but because of unmeasured covariates in the patients.

Finally, the manuscript by Zenios and colleagues is an important contribution to the research of the methodology used in the assessment of outcome-based performance of kidney transplant centers. The improved accuracy of their ME method of estimating the 3-year patient and graft survival should be considered along with other sophisticated tools to refine the analysis used in the preparation of CSRs. The underlying database used in generating the CSRs is deficient because it lacks information on patient- and center-based critical determinants of transplant outcomes. The danger with the current use of CSRs is the potential for a chilling effect on transplant centers serving high-risk populations because these centers may be erroneously penalized. Furthermore, transplant centers misclassified as underperforming may divert substantial resources to remedial actions that are unfounded. Zenios et al. have taken an important first step toward improving the approach to the performance evaluation of transplant centers on the basis of the transplant recipient outcomes.

Disclosures

None.