Inclusion of Dynamic Clinical Data Improves the Predictive Performance of a 30-Day Readmission Risk Model in Kidney Transplantation

Abstract

Background

30-day readmissions (30DRA) are a highly scrutinized measure of healthcare quality and relatively frequent among kidney transplants (KTX). Development of predictive risk models are critical to reducing 30DRA and improving outcomes. Current approaches rely on fixed variables derived from administrative data. These models may not capture clinical evolution that is critical to predicting outcomes.

Methods

We directed a retrospective analysis towards: 1) developing parsimonious risk models for 30DRA and 2) comparing efficiency of models based on the use of immutable versus dynamic data. Baseline and in-hospital clinical and outcomes data were collected from adult KTX recipients between 2005 – 12. Risk models were developed using backward logistic regression and compared for predictive efficacy using ROC Curves.

Results

Of 1,147 KTX patients, 123 had 30DRA. Risk factors for 30DRA included recipient comorbidities, transplant factors, and index hospitalization patient level clinical data. The initial fixed variable model included 9 risk factors and was modestly predictive (AUC 0.64, 95% CI 0.58–0.69). The model was parsimoniously reduced to 6 risks, which remained modestly predictive (AUC 0.63, 95% CI 0.58–0.69). The initial predictive model using 13 fixed and dynamic variables was significantly predictive (AUC 0.73, 95% CI 0.67–0.80), with parsimonious reduction to 9 variables maintaining predictive efficacy (AUC 0.73, 95% CI 0.67–0.79). The final model using dynamically evolving clinical data outperformed the model using static variables (p=0.009). Internal validation demonstrated the final model was stable with minimal bias.

Conclusion

We demonstrate that modeling dynamic clinical data outperformed models utilizing immutable data in predicting 30DRA.

INTRODUCTION

The 30-day readmission (30DRA) rate is widely utilized by payers and regulators as a surrogate metric of hospital quality and a strong correlate of mortality that is viewed as potentially modifiable with more efficient systems of care.^1–4 Reducing the frequency and costs associated with preventable 30DRA is thought to be essential to improving the quality of the health care system, as readmissions substantially increase the patient’s risk of transition of care errors while also contributing to higher Medicare costs of approximately $17.4 billion in 2004 alone.⁵ Thus, rates of readmissions have attracted high levels of interest from policymakers as a method to both track and improve quality of care while also reducing costs. As part of the Centers for Medicare and Medicaid Services (CMS) Reporting Hospital and Quality Data Annual Payment Update Program, hospitals are required to publicly report readmission rates. Hospitals with higher than expected risk-adjusted readmission rates for certain admission types are now penalized via decreased reimbursement payments through the CMS payment system.^{6, 7}

Despite the high-risk and high-cost nature of transplant surgery, studies analyzing risks and outcomes associated with early readmission following kidney transplant are nominal. A study of national longitudinal Medicare claims data by McAdams-DeMarco et al reported that 31% of kidney transplant recipients were readmitted within 30-days of discharge. The authors also identified a number of important risk factors for readmission, including advanced age, African-American race and having chronic conditions such as diabetes, ischemic heart disease and COPD. Additional risk factors for 30DRA were obesity, ECD status, length of stay for the index hospitalization and lack of induction therapy. The reported 30DRA ranged between 18% and 47% but this large variation was not well-predicted by the data. A major strength in transplantation is the availability of administrative data used to populate SRTR risk prediction models. These models are widely used despite a modest c-statistic. By the same token, a major limitation of studies using national registry data is the poverty of patient-level data pertaining to in-hospital clinical variables that are likely to significantly contribute to readmission risk and may be modifiable.⁸ Thus, despite the popularity of the metric of 30DRA with payers and regulators, risk prediction modelling for 30DRA following kidney transplantation is not well-studied studied and modeling approaches using currently available data structures may be inadequate. However, it is clear that 30DRA following kidney transplantation is a major risk factor for poor outcomes. In a recent follow-up study, McAdams-DeMarco et al. report that 30DRA is a strong predictor of adverse post-transplant outcomes, including late hospital readmission (within 1 year after 30DRA) and mortality.⁹ Thus, in order to reduce 30DRA and potentially improve outcomes within this high-risk surgical procedure, it is imperative to better understand patient-level risk factors through predictive modeling in improving the transplant clinician’s ability to identify high-risk recipients.

We hypothesized that including patient-level in-hospital dynamic clinical data (blood pressure, laboratory values) a correlative predictive risk model for 30DRA for kidney transplant recipients will out-perform a model merely including static factors (age, race, comorbidities, etc). Our approach in this study was to use iterative regression analyses to develop risk models for 30DRA in kidney transplant recipients using baseline risks and in-hospital clinical variables and compare the relative performance of these models on predicting readmission.

RESULTS

Baseline Patient Characteristics

Between Jan 2005 and Dec 2012, a total of 1,500 kidney transplants were performed at the study institution; of which 79 (5%) were excluded for age (<18 yo), 184 (12%) were excluded for receiving non-renal transplants, 26 (2%) were excluded for graft loss within one month of transplant and 64 (4%) were excluded for being lost to follow-up, leaving a total of 1,147 (76%) that were included in the analysis. Of those in the study, 123 (11%) were readmitted within 30-days of discharge for the index hospitalization associated with the kidney transplant event.

Table 1 displays the baseline fixed characteristics compared across 30DRA status. Statistically significant risk factors for 30DRA included level of education, a history of heart disease, stroke, CABG and delayed graft function. The in-hospital dynamic clinical variables compared across groups are displayed in Table 2. Percent of DBP >100 mmHg and event costs were significantly associated with 30DRA. SBP trajectory and % glucose >250 mg/dL were also included in the dynamic risk model (p≤0.2).

Table 1.

Univariate comparison of fixed characteristics based on 30DRA

Characteristic	Control (n=1,024)	30DRA (n=123)	P-Value
Age (years)	51±14	52±14	0.880
Female Gender	40%	43%	0.609
African-American Recipient	53%	57%	0.359
BMI (kg/m2)	29±6	28±7	0.243
Insurance
Medicare Only	14%	13%	0.862
Medicaid Only	3%	2%	0.803
Did Not Complete High School	2%	7%	0.001*
Disability Income	43%	42%	0.734
Comorbidities
Hypertension	92%	90%	0.386
Diabetes	34%	36%	0.608
Heart Disease	18%	27%	0.015*
CVA	7%	12%	0.036*
Cardiac Catheterization	11%	12%	0.577
CABG	5%	9%	0.050*
Acute MI	5%	4%	0.792
CHF	5%	5%	0.818
Dialysis History			0.594
Preemptive	23%	18%	0.203*
Peritoneal Dialysis	15%	15%	0.883
Hemodialysis	62%	67%	0.355
Previous Renal Transplant	9%	14%	0.063*
HLA Mismatches	4.2±1.5	4.4±1.4	0.179*
PRA	17±30	17±29	0.959
Cold Ischemic Time (hours)	16±10	16±10	0.813
Donor ECD	13%	13%	0.947
Estimated KDRI	1.12±0.39	1.14±0.42	0.625
Cytolytic Induction	43%	49%	0.183*
Delayed Graft Function	12%	22%	0.002*

^*Included in the multivariate static and dynamic modeling (p≤0.2)

CVA – Cerebral Vascular Accident, CABG - Coronary Artery Bypass Grafting, MI – Myocardial Infarction, CHF – Congestive Heart Failure, HLA – Human Leukocyte Antigen, PRA – Panel Reactive Antibody, CIT – Cold Ischemia Time, ECD – Expanded Criteria Donor, KDRI – Kidney Donor Risk Index, SBP – Systolic Blood Pressure, PP – Pulse Pressure, SrCr – Serum Creatinine

Table 2.

Univariate comparison of in-hospital dynamic variables based on 30DRA

Variable	Control	30DRA	P-Value
SYSTOLIC BLOOD PRESSURE
Mean SBP (mmHg)	146±17	148±20	0.338
Median SBP (mmHg)	145±18	147±21	0.373
% SBPs >180 mmHg	7.9%	10.8%	0.198
% SBPs <90 mmHg	0.4%	0.8%	0.444
SBP Trajectory (mmHg change per day)	−3.3	−0.1	0.135*
DIASTOLIC BLOOD PRESSURE
Mean DBP (mmHg)	78±11	77±10	0.404
Median DBP (mmHg)	78±12	77±10	0.365
% DBP >100 mmHg	6.1%	3.3%	0.020*
% DBP <60 mmHg	10.1%	8.0%	0.280
DBP Trajectory (mmHg change per day)	−2.1	−2.7	0.672
ADDITIONAL HEMODYNAMICS
Mean MAP (mmHg)	101±11	101±11	0.923
MAP Trajectory (mmHg change per day)	−2.4	−1.8	0.588
LABORATORY VALUES AND COST DATA
SrCr Trajectory (mg/dL change per day)	−0.9	−0.9	0.988
Mean Glucose (mg/dL)	143±36	147±42	0.358
% Glucose >250 mg/dL	5.1%	7.0%	0.092*
% Glucose <70 mg/dL	1.6%	1.7%	0.913
Mean Tacrolimus Trough Level (ng/mL)
Post-Transplant Day 3	5.1±3.3	5.3±3.0	0.506
Post-Transplant Day 7	8.4±3.5	8.4±3.4	0.888
Transplant Event Total Costs	$72,446±$17,444	$65,706±$14,283	0.001*
Length of Stay (days)	3.5±2.0	3.6±1.2	0.826

^*Included in the multivariate dynamic modeling (p≤0.2)

Risk Model for 30DRA Using Fixed Variables

The initial risk model for 30DRA included 9 fixed variables that were associated with 30DRA (p≤0.2) in univariate analysis, which is displayed in supplemental Table 1. This model demonstrated a statistically significant correlation (Omnibus test: p<0.001, Hosmer and Lemeshow test: p=0.061) and was modestly predictive of 30DRA (AUC = 0.64, 95% CI 0.58–0.69; see Figure 1A). Following backward elimination (p>0.2), 6 variables remained in the final model, including below high school education (AOR 3.84, 95% CI 1.68–8.74), history of CVA (AOR 1.61, 95% CI 0.87–2.98), history of heart disease (AOR 1.62, 95% CI 1.04–2.53), retransplant (AOR 2.00, 95% CI 1.12–3.55), delayed graft function (AOR 1.93, 95% CI 1.20–3.10) and HLA mismatches (AOR 1.11, 95% CI 0.97–1.28). This model demonstrated a statistically significant correlation with 30DRA (Omnibus test: p<0.001, Hosmer and Lemeshow test: p=0.089) and remained modestly predictive (AUC = 0.63, 95% CI 0.58–0.69; see Figure 1B). The final static model performance demonstrated a NPV of 63.8% (95% CI 61 to 67%) and PPV 57.7% (95% CI 56 to 59%), with a probability cut point of 0.1.

Figure 1. Comparison of predictive model accuracy based on the input of fixed and dynamic variables.

These are the ROC curves for the four predictive models. Figures 1A and 1B display the initial and final ROC curves for the models using only fixed variables listed in Table 1, respectively. Figures 1C and 1D display the initial and final ROC curves that utilize both the fixed and dynamic variables listed in Tables 1 and 2, respectively.

Risk Model for 30DRA Using Fixed and Dynamic Variables

The preliminary risk model using both immutable and dynamic data included 13 variables and is displayed in supplemental Table 2. This model was comprised of the 9 fixed variables included in the above-mentioned model, plus 4 dynamic clinical data elements. The model demonstrated a statistically significant correlation (Omnibus test: p<0.001, Hosmer and Lemeshow test: p=0.120) and was well predictive for 30DRA (AUC = 0.732, 95% CI 0.67–0.80; see Figure 1C). After backward elimination (p>0.2), the final model consisted of 9 variables and is displayed in Table 3. The final model had a statistically significant correlation (Omnibus test: p<0.001, Hosmer and Lemeshow test: p=0.603) and was also well predictive of 30DRA (AUC = 0.731, 95% CI 0.67–0.79; see Figure 1D). The final model performance demonstrated a NPV of 73.3% (95% CI 70 to 76%) and PPV 62.8% (95% CI 61 to 65%), with a probability cut point of 0.1.

Table 3.

Final multivariate risk model for 30DRA using 6 fixed and 3 dynamic variables

Variable	Reference	Adjusted Odds-Ratio	95% Confidence Interval	P-Value
Did Not Complete High School	Completed High School	5.04	1.86 – 13.7	0.001
History of CVA	No CVA History	2.25	1.04 – 4.87	0.039
Previous Transplant	Primary Transplant	2.08	1.07 – 4.07	0.032
Delayed Graft Function	Graft Function	4.37	2.36 – 8.11	<0.001
History of CABG	No CABG History	1.85	0.45 – 4.56	0.183
Cytolytic Induction	IL2RA Induction	2.14	1.30 – 3.54	0.003
Trajectory of SBP (mmHg per day)	0 mmHg change per day	1.03	1.00 – 1.07	0.067
% In-Hospital DBPs >100 (mmHg)	0%	0.97	0.95 – 1.00	0.025
Txp Event Costs per $10,000	$20–30,000	0.62	0.53 – 0.73	<0.001

Comparison of Model Predictive Accuracy

Within the static models, the predictive accuracy of the initial model (AUC = 0.636) was similar to that of the final model (AUC = 0.634, p=0.960). Within the initial and final dynamic models, there was also no difference in predictive accuracy (AUC = 0.732 vs. 0.731, p=0.983, respectively). When comparing the initial immutable to dynamic models, the latter model had statistically significant improved predictive efficacy, as compared to the static model (AUC = 0.732 vs. 0.636, p=0.027 respectively); the final dynamic model demonstrated statistically significant improved predictive accuracy as compared to the final fixed model (AUC = 0.731 vs. 0.634, p=0.009).

Model Validation

Internal validation was conducted for the final dynamic model displayed in Table 3. Bootstrapping, using 1,000 iterations, demonstrated this model to be stable, with minimal bias or deviations within standard error. The beta of the model was −2.095, with bootstrapping producing a 95% CI of −2.44 to −1.81. The standard errors were comparable as well (0.162 for bootstrapping vs. 0.153) and model bias was minimal at −0.007. Individual bias for each of the nine variables included in the model was nominal, ranging from −0.471 to 0.007. Figure 2 displays the additional internal validation that was performed, which consisted of conducting 50 ROC analyses using a random 50% sampling. The AUCs for each of the models (mean 0.749, range 0.65 to 0.81) were statistically indifferent from the original AUC (0.731, p>0.06). Missing data assessment, using both logistic regression and multiple imputations methods demonstrated that missing variable data occurred at random and did not appreciably influence the model.

Figure 2. Internal model validation using 50 random iterations of 50% sampling.

This bar graph depicts the AUCs for the 50 models that were simulated as part of the internal validation analysis, using a 50% random sampling for each simulation model. Each AUC within these simulations was statistically similar to the AUC produced in the original model (AUC 0.731, p>0.06).

DISCUSSION

The results of this study demonstrate that incorporating dynamic clinical data into a parsimonious 30DRA risk model improves its predictive accuracy, as compared to modelling using only immutable data. The final model produced through these iterations included 9 variables and was able to discern which kidney transplant recipients were likely to be readmitted in a clinically relevant manner. These results suggest that with large datasets that include detailed fixed and dynamic clinical variables, potentially available through electronic health records (EHR) and health information exchanges (HIEx), it is likely that clinically relevant predictive models can be developed for 30DRA.

Healthcare reform within the U.S. has put improving the value of care in the national spotlight and 30DRA is a well-recognized surrogate utilized as a hospital quality metric.^{12, 13} The Affordable Care Act (ACA) has provided both the motivation to reduce 30DRA and the potential tools to do so (EHR). CMS now penalizes healthcare systems with high rates of 30DRA for hospitalizations due to CHF, acute MI and pneumonia; 2015 will see COPD and knee/hip replacement added to this list. By 2017, CMS is expected to dramatically expand this list.^14–17 Although it is uncertain if transplantation will be included in this list, given its high rates of 30DRA, it would not be surprising if this occurs.

It is incumbent on the transplant community to focus efforts on understanding risks associated with 30DRA and implement initiatives aimed at reducing these events. The ACA has provided tools to facilitate this endeavor, by incentivizing transition to EHRs and HIEx.^18–20 Because of this, the U.S. healthcare system has recently seen a revolution of technology implementation. Preliminary evidence from outside of transplantation suggests that leveraging this massive amount of data to produce clinical relevant, actionable information is a promising mechanism to improve healthcare value.^21–23 The results of this analysis provide evidence supporting this within the transplantation, as it is clear that risk models including dynamic clinical variables significantly outperform those containing only immutable data.²⁴ Future efforts should work towards using both the EHR and HIEx to compile much larger datasets that include transplant recipients from multiple centers while also leveraging larger amounts of dynamic clinical data.^{25, 26}

Previous studies assessing risk factors for early hospital readmission after kidney transplant have demonstrated similar findings to this analysis. The aforementioned analysis by McAdams-DeMarco demonstrated that baseline recipient demographics (age, BMI, comorbidities), donor characteristics (age, DCD, ECD) and transplant factors (cold time, induction therapy, HLA mismatches and DGF) were all statistically associated with readmission.⁸ Although the McAdams-DeMarco study did find a number of risk factors that were not present in our analysis, this is likely related to the use of national registry data. Thus, a number of risk factors in our study (CV related, induction therapy, in-hospital blood pressures) may be center-specific and not externally applicable to other programs. It is important to note the DGF was a predominant risk factor in both studies and likely represents one of the most important factors that, if modifiable, may lead to reduced readmissions. Several analyses have also identified additional risk factors, including frailty, waiting list time, weekend discharges and initial LOS.^27–31 Our study did not have data on frailty, waiting list time or discharge day of the week, nor was LOS a risk factor in our study. This may be due to the short LOS in our study coupled with our program’s enduring focus on structured care models within the outpatient setting to reduce both LOS and readmissions.^33–34 The results presented here also identify a number of new immutable risk factors for readmission, including below HS education. This may represent patients that have a difficult time following complex post-discharge care plans, including new medication regimens, vital sign and glucose monitoring, urine output and fluid intake. Conversely, this may be a surrogate socioeconomic marker for reduced support from caregivers within the patient’s home. In either case, this was an important risk factor in both the fixed and dynamic models and may identify a patient population to target with future interventions to reduce 30DRA.

Reducing early hospital readmissions is likely going to be a difficult task in kidney transplantation, as studies have demonstrated reasons to be readmitted are complex and often not preventable.²⁷ Therefore, developing parsimonious predictive risk models are an important step in this process, as they will allow clinicians the ability to identify patients at high-risk for this event, while also understanding the specific factors that are driving this risk; patient-individualized care plans can then be developed that may mitigate these risks, leading to reduced early hospital readmissions and, hopefully, reduced deleterious long-term sequelae.^{24, 32–34}

Transplantation has rich data available at the national level that provides a veritable census representation of the national population through the SRTR. However the SRTR data, while rich in baseline data and hard outcomes, lacks patient level clinical data that may represent key intervening time points.³⁵ This contribution of patient level data on the causal pathway to outcomes could explain the better predictive accuracy of the dynamic models. These patient level data points could also represent targets for early recognition and timely intervention to prevent loss of quality that the 30DRA constitutes. Blood pressure changes are likely such an example, as in-hospital SBP slope and the percent of elevated DBP were significantly associated with 30DRA. Such data may uncover disease processes among patients that are heading in the wrong direction prior to discharge, which may be amenable to modification, thus reducing the potential for early hospital readmission. Changing blood pressure may be a surrogate for intravascular volume status, which could potentially lead to graft dysfunction and subsequent readmission. Additionally, elevated SBPs, if within a critical threshold (hypertensive urgency or emergency), may actually be the sole indication for readmission. It is no fully clear why high in-hospital costs were protective for 30DRA, which was independent of LOS or transplant year. This may reflect patients that had biopsies or extensive work-up prior to discharge, thus not requiring readmission for continued work-up when an issue doesn’t resolve or a new clinical event develops (i.e. graft function doesn’t improve; rejection has already been ruled-out). Without further detail on costs, it is not possible to determine the etiology of this association. Future analysis of this association is clearly warranted. Cytolytic induction therapy may represent an important variable, as it is known to increase the risk of infection and cytopenias; which are likely common reasons for readmission post-transplant.

There are a number of limitations to this study. First, it was a retrospective single-center analysis. Missing and misclassification of data, as with all retrospective analyses, was a possibility. Detailed review of both electronic and paper records with predefined definitions was used to minimize these limitations. Data analysis determined absent values to be missing at random (MAR) and unlikely to influence model performance. The single-center design does limit the external validity of this study, as it has been demonstrated that 30DRA vary significantly between transplant centers and center-specific care models likely influence this outcome; additionally, our center had low 30DRA rates compared to national rates.⁸ However, the primary objective of this analysis was not to build a parsimonious 30DRA risk model that can be applied across all transplant programs; rather it was to demonstrate that dynamic clinical data improves the predictive performance of risk modeling. The 30DRA within the study institution was low (about one-third the national average) compared to other studies, likely related to our centers focus of improving perioperative care.^33,34 In fact, in this circumstance, one could expect that model performance would only improve in environments with higher event rates. Nevertheless, the results demonstrate that clinical evolving data should be considered for use to further develop and refine clinical risk models for each transplant program.

In summary, the results of this analysis demonstrate a novel approach to improve predictive risk modelling for 30DRA in kidney transplantation by incorporating dynamic clinical data in models, as compared to modeling with immutable data.

METHODS

Study Design and Patients

This was an IRB-approved cross-sectional analysis of kidney recipients that underwent transplant at a single institution between Jan 2005 and Dec 2012. Patients were grouped based on 30DRA and compared accordingly. Patients were included if they were recipients of a solitary kidney transplant and received follow-up care at the study institution during the prespecified time period. Pediatric recipients (<18 yo at the time of transplant), non-renal transplant recipients, those with grafts lost within one month of transplant or lost to follow-up were excluded from this study.

Study Objectives

The primary objective of this study was to develop and test the predictive accuracy of risk models that adequately discern patients likely to experience a 30DRA following the index hospitalization for kidney transplantation; with the predominant goal of determining if the use of dynamic clinical data captured during the index hospitalization significantly improved the predictive accuracy of the risk model.

Data Variables and Definitions

All data were collected in a retrospective manner from electronic and paper medical records. Initially, comprehensive baseline recipient and donor sociodemographics and transplant characteristics were recorded from the past medical history. Subsequently, clinical variables were collected during the entire index hospitalization and included hemodynamics and laboratory values. All documented clinical post-transplant events were captured, including hospitalizations. Delayed graft function was defined as the need for dialysis within 7 days following transplantation.

Multiple dynamic clinical data variables were captured and incorporated in the modelling. These included SBP and DBP, costs and laboratory values. Total in-hospital aggregates of these variables were calculated and those that demonstrated correlation to 30DRA (p≤0.2) were included in modelling, removing collinear variables. Calculations included in-hospital means, medians, peaks, nadirs, percentages above and below critical thresholds and trajectories (slope change per day). Hospitalization costs was also captured and tested in the modeling. Additional variables that were tested and not included in the model due to lack of correlation included serum chemistries, tacrolimus trough concentrations and other hemodynamic parameters (pulse pressures, mean arterial pressure).

Statistical Analysis

For the initial univariate analysis, patients readmitted to the hospital within 30-days following discharge were compared to the non-30DRA patients for all baseline and dynamic clinical variables. All variables demonstrating statistical association (p≤0.2) were entered into a multivariable model using binary logistic regression, with the dependent variable set as 30DRA. A stepwise backward elimination process was utilized to develop parsimonious models and remove covariates not associated with the outcome (p>0.2), leading to the final model. Iterative modelling was conducted, first using only fixed variables, followed by modelling using both fixed and dynamic clinical variables. We tested the modelling assumption that there are linear relationships between each of the continuous covariates included in the model and the model predictability (log[p/1−p]) and these assumptions were valid; meaning these relationships are linear. Model performance was determined using the Omnibus test, the Hosmer-Lemeshow goodness-of-fit test and patient-specific probability estimates to develop receiver operating characteristic (ROC) curves. The ROC curves predictive accuracy was assessed using the area under the ROC, 95% confidence intervals (CIs) and p-values. Negative and positive predictive values (NPV, PPV) were also calculated for the final risk models. Statistical comparison between model performances was conducted using the process described by Hanley et al.¹⁰ Our analysis assumes missing at random (MAR). Based on our knowledge of the field, we believe that missing 30DRA does not depend on the unobserved 30DRA values and a logistic regression model for missing indicator as a function of covariates shows that missingness is correlated with previous transplant, induction, SBP trajectory and costs, supporting our assumption of MAR against missing completely at random (MCAR). Thus, we did the analyses based on multiple imputation and complete case-analysis (CCA) which did not show significant difference. Therefore, our results follow CCA.

Internal validation of the modelling was accomplished using bootstrapping (1,000 iterations) to produce estimates of standard errors for the beta coefficients and to calculate model bias. A second internal validation analysis was accomplished by producing probability estimates for random samples of 50% of the original research group in 50 iterations to develop ROC curves. The performance of these curves was compared to the ROC curve produced with the original model.¹¹ All statistical analyses performed using SPSS [version 22, IBM Corporation, Armonk, NY].

Abbreviations

30DRA

30-day readmissions

KTX

Kidney transplant

AUC

Area under the curve

CI

Confidence Interval

CMS

Centers for Medicare and Medicaid Services

COPD

Chronic obstructive pulmonary disease

ECD

Expanded criteria donor

IRB

Institutional Review Board

SBP

Systolic blood pressure

DBP

Diastolic blood pressure

SrCr

Serum creatinine

ROC

Receiver operating characteristic

OR

Odds-ratio

AOR

Adjusted odds-ratio

EHR

Electronic health records

HIEx

Health information exchanges

ACA

Affordable care act

MAR

Missing at random

MCAR

Missing completely at random