Predicting Health Care Utilization and Charges Using a Risk Score for Poor Adherence in Pediatric Kidney Transplant Recipients

Abstract

Pediatric kidney transplant recipients must follow a complicated regimen of timely adherence to immunosuppressant medication, routine blood work, and medical follow-up visits. Failure to adhere to the recommended regimen can result in medical complications and costly treatment. We developed a novel risk score to identify patients at risk for poor adherence behaviors and evaluated whether it would predict future health care utilization and charges. Our risk stratification score combined three simple pass/fail metrics of adherence derived directly from the electronic health record including standard deviation of immunosuppression drug levels, timely laboratory monitoring, and timely clinic visits as indicated by our clinical protocol. Risk for poor adherence was assessed over a three-month period. Linear regression was used to predict subsequent health care charges and utilization. Greater than 75% of patients had some degree of nonadherence risk during the study period, but there were no significant differences found on any outcomes for the overall score. However, when the individual components of the overall risk score were evaluated independently, patients with tacrolimus drug level standard deviation ≥2 (e.g., a marker of poor adherence) had greater health care utilization (e.g., hospitalizations) and increased total charges. Additionally, patients who did not follow up in clinic at least every 4 months had more ED visits and ED-related charges, but fewer hospitalizations. Regular clinic visits and minimizing drug level variation may deter future costly ED visits and hospitalizations.

For children and adolescents with end-stage kidney disease, transplantation offers improved survival (United States Renal Data System [USRDS], 2016) and health-related quality of life (HRQOL) compared with dialysis (Riano-Galan et al., 2009). However, caring for a transplanted organ requires lifelong adherence to immunosuppressant medication, as well as increased medical monitoring, including frequent clinic visits and blood work (Griffin & Elkin, 2001). Unfortunately, organ survival rates are remarkably low for adolescent kidney transplant recipients compared with other age groups (Foster et al., 2011; Magee & Pascual, 2004; Van Arendonk et al., 2014), with only 68.7% of deceased donor and 79.1% of living donor grafts still functioning five years after transplant (USRDS, 2016). This high rate of kidney allograft failure among adolescents has been, in part, attributed to pervasive problems with medication adherence which are prevalent in up to 43% of adolescents (Chisholm-Burns et al., 2009; Dobbels et al., 2010; Morrissey et al., 2005).

Adherence is defined as “the extent to which a person’s behavior—taking medication, following a diet, and/or executing lifestyle changes— corresponds with agreed recommendations from a health care provider” (Sabaté, 2003). Poor adherence has been strongly associated with serious health consequences in this population, including acute and chronic rejection, hospitalization, graft loss, and death (Bunzel & Laed-erach-Hofmann, 2000; Evans et al., 2010; Jar-zembowski et al., 2004; Shaw, Palmer, Blasey, & Sarwal, 2003). Further, there is a significant economic burden associated with complications secondary to poor adherence (Pinsky et al., 2009). The 2016 report from the USRDS revealed that the cost of treating patients on dialysis was more than twice that of caring for patients with a functioning renal transplant (USRDS, 2016). These costs extend beyond payors to the individual patient and family. Estimates show that transplant patients with poor adherence spend an additional $12,840 on medical care over a three-year period compared with individuals with higher adherence (Pinsky et al., 2009). Despite the known negative impact of poor adherence on health outcomes, there are few methods to systematically identify patients at the point of care who are exhibiting poor adherence behaviors and who are thus at risk for negative clinical outcomes (e.g., rejection) or increased health care charges. Longitudinal monitoring of adherence in this population is particularly important since adherence declines as patients move further away from transplantation (Chisholm, Lance, & Mulloy, 2005; Dew et al., 2009).

Kidney transplant care is complex and treatment demands are high. Specifically, key elements of adherence common to all kidney transplant recipients include taking daily immunosuppressant medication(s) at designated times, getting routine bloodwork, and attending clinic visits. Although each transplant program has their own protocol for the frequency of this monitoring, physicians agree that patients who do not follow-up regularly are at risk for complications (Jarzembowski et al., 2004). Past research in solid organ transplant recipients demonstrates that variability in tacrolimus (the most commonly used medication for immunosuppression) trough levels of greater than two standard deviations over the previous four levels was significantly associated with higher rate of rejection episodes (Shemesh et al., 2004). Furthermore, inconsistent or infrequent attendance with the clinic visit schedule was associated with increased rate of late acute rejection episodes and graft loss in a sample of pediatric kidney transplant recipients (Jarzembowski et al., 2004). In combination, low immunosuppression standard deviations, keeping appointments, and obtaining routine bloodwork may be good markers of adherence and associated health outcomes.

Based on this literature, our team developed a novel and simple risk stratification score to identify patients exhibiting poor adherence behaviors at the point of care. We specifically designed it to aid clinicians in rapidly determining whether patients (a) had increased variability in immunosuppression trough levels (>2 standard deviations over 4 levels), (b) were getting laboratory monitoring according to the minimum recommended schedule (at least every four weeks), and (c) were coming in for the minimum frequency of clinic visits (at least every three months). The score is easy to obtain and simple to interpret from existing discrete elements in any electronic health record (EHR). It is based on combining the three pass/fail metrics mentioned above. The primary objective of this score was to enable clinicians to identify, and ultimately intervene on, patients who exhibited the poor adherence behaviors identified above. The objective of this study was to test this score to see if it would predict future health care utilization and associated charges.

Specifically, this pilot study aimed to (a) examine the frequency of adherence risk based on this scoring system and (b) determine whether adherence risk stratification would predict future health outcomes and related charges. Based on a systematic review of adherence studies with pediatric kidney transplant recipients (Dobbels et al., 2010), approximately 30% of patients were expected to display some level of nonadherence. It was hypothesized that patients with a higher adherence risk score (indicating more problems with adherence) would have significantly higher health care utilization and higher health care charges due to treatment of complications (e.g., rejection episodes).

Method

This study was approved by the hospital’s Institutional Review Board. Because of use of existing data, a waiver of consent was obtained. Retrospective chart review was conducted on a cohort of children and adolescent kidney transplant recipients at a children’s hospital in the Midwestern United States. Patients were included in the retrospective chart review if they (a) received a first kidney transplant between 2009 and 2012, (b) were prescribed tacrolimus during the period of 12 to 24 months posttransplant, and (c) actively received follow-up kidney transplant care at the medical center, meaning they did not transfer care to a different medical center after transplant. Patients were excluded if (a) they received another solid organ transplant in addition to a kidney (n = 4), (b) they were prescribed immunosuppression requiring therapeutic drug monitoring in addition to tacrolimus (n = 4), or (c) they transferred care to another institution during the study period (n = 8).

Measures

Health care charges.

Hospital and physician billing records containing medical encounter charges for a 24-month period (starting 24-months after kidney transplantation) were obtained from the medical center’s accounting database. This included all hospital encounters regardless of reason for the visit or discharge diagnosis. Data included charges from outpatient ambulatory care, emergency department visits, inpatient hospitalizations, infusions, laboratory tests, and diagnostic procedures. Actual reimbursement for these charges were not available. Each charge was listed separately, so research staff manually combined the charges into discrete encounters for analysis (see the table in the online supplemental material). For example, charges for medications, laboratory studies, dressing change, biopsies, physician time, and hospital charges were all combined to form a unit of “hospitalization.” Health care charges were collected across multiple years (2011–2015). To adjust for inflation, they were standardized to a common year (2017) using the Consumer Price Index Medical Care Data Table (United States Bureau of Labor Statistics, 2017). All charge variables were highly positively skewed and were thus log transformed before analyzing.

Health care utilization and outcomes.

The raw number of emergency department visits, infusion visits, and inpatient hospitalizations in the 24- to 48-month period following transplant were collected. Additionally, biopsy- proven allograft rejection episodes, allograft loss, and death were recorded as binary outcomes (1 = occurrence, 0 = nonoccurrence).

Demographic and medical characteristics.

Each patient’s EHR was accessed to collect demographic (i.e., age, sex, race, health insurance status) and clinical information (i.e., initial diagnosis requiring kidney transplantation, prior dialysis, date of kidney transplant, donor type, immunosuppression drug treatment history, allograft function markers, presence of rejection episodes).

Adherence risk score.

A novel risk stratification algorithm was developed through an interdisciplinary collaboration between Nephrology, Psychology, Information Systems, and Pharmacy. Our goal was to develop a simple score that could easily be incorporated into the clinical workflow and screen for patients who are failing to adhere to a minimum standard of care. Emphasis was on simplicity, using easily obtained discrete elements from the EHR that could be mapped to care algorithms, including the expectation that patients would have minimal variation in tacrolimus levels, receive laboratory monitoring at least monthly, and be seen in clinic at least every three months. Thus, our scoring algorithm incorporates three pass/fail markers of nonadherence collected from the EHR. Each metric was dichotomized and assigned one point for a failure for each day as follows: (a) standard deviation of the last four immunosuppression drug levels ≥2 (Shemesh et al., 2004; patients who did not have at least four drug levels in the previous six months were also considered a fail), (b) failure to get routine lab work, defined as more than two months since the previous set of labs, and (c) failure to follow up in clinic, defined as more than four months since last clinic visit. There are four possible total levels of daily risk (Level 0 = no risk factors; Level 1 = one risk factor; Level 2 = two risk factors; Level 3 = three risk factors). The daily scores were averaged across a 90-day period (15–18 months after transplant).

Procedure

A daily adherence risk score was calculated for each patient using data obtained from the EHR. These daily scores were recorded and averaged across a 90-day period (15–18 months after transplant). This time frame was selected based on the typical posttransplant course. During the first 12 months after transplant, the patient is seen frequently in clinic, many dose adjustments are made to the immunosuppression regimen, and laboratory work is monitored regularly to assess the effect of medication dose changes and kidney function. Once a patient reaches one year posttransplant, their management is considered more stable and they transition to a long-term care plan of monthly blood work and quarterly clinic visits. To capture their behavior and adherence scores after the one-year mark, we had to start at 15 months posttransplant knowing that the score was reflective of the past three months (12–15 months posttransplant). Health care charges data for each patient 24–48 months after transplant was obtained from the medical center’s billing department and was summed for each patient (See Figure 1). Demographic and health care data were collected directly from the EHR. Chart reviews were completed by two trained research assistants.

Figure 1.

Timeline of data collection.

Statistical Analyses

Descriptive data, including means, standard deviations, and frequencies, were calculated for demographic, medical, and outcomes of interest. To determine whether overall average adherence risk was associated with health care charges, we performed four linear regression analyses with maximum likelihood estimation where each of the four log-transformed charge variables was the outcome and the overall average adherence risk variable was the predictor. To determine whether overall average adherence risk was associated with health care utilization, we conducted two negative binomial regressions with maximum likelihood estimation. Utilization (as total count of encounters for hospitalizations and ED visits separately) was the outcome for each model and average adherence risk was the predictor.

Post hoc analyses were conducted with each of the three individual risk categories as predictors, instead of the overall adherence risk score, following the logic above. Specifically, for health care charges, we performed four linear regression analyses with maximum likelihood estimation where each of the four log-transformed charges was the outcome and the three individual risk categories were the predictors. For health care utilization, we conducted two negative binomial regressions with maximum likelihood estimation. Utilization (as total count of encounters for hospitalizations and ED visits separately) was the outcome for each model and the three individual risk categories were the predictors. All regression analyses were performed with Stata Version 14.

Results

There were 37 kidney transplant recipients who met inclusion criteria for the study. Participant demographics and risk scores are presented in Table 1. Notably, 78% of the sample demonstrated some level of adherence risk during the 90-day period. Of the individual factors making up the risk stratification score, the standard deviation score of tacrolimus levels was the most commonly failed component (40% of total patient days), followed by not obtaining blood work (12% of total patient days) and failure to attend clinic visits on time (3% of total patient days).

Table 1.

Participant Demographics and Risk Scores

Variable	M ± SD (range)	% of patients
Age at transplant (years)	12.1 ± 6.4 (1–21)
Sex (male)		67.5
Race
White		83.7
Black		10.8
Asian		2.7
Biracial		2.7
Transplant type (living)		62.1
Overall Adherence Risk Score (composite of individual risk scores)		78.4a
Immunosuppression drug standard deviation ≥ 2 from past 4 drug levels		67.6a
Lab risk score		43.2a
Visit risk		13.5a

^aPercentage of patients who failed the risk category for at least one day.

Association of Overall Adherence Risk With Health Care Utilization and Charges

Results are presented in Table 2. Overall average adherence risk was not significantly associated with number of ED visits, hospitalizations, or charges (inpatient, outpatient, ED, total).

Table 2.

Regression Results for Overall Adherence Risk Predicting Health Care Utilization and Charges

	Overall average adherence risk
Outcome	IRR	SE	p	95% CI
ED visits	1.02	.34	.95	[.54, 1.95]
Hospitalizations	1.58	.81	.37	[.58, 4.31]
	b	SE	p	95% CI
Total charges	.05	.36	.14	[−.65, .75]
ED charges	−.07	.96	.94	[−1.95, 1.81]
Outpatient charges	.04	.26	.14	[−.47, .55]
Inpatient charges	.52	1.66	.31	[−2.74, 3.77]

Note. ED = emergency department; IRR = incidence rate ratio; b = unstandardized regression coefficient; SE = robust standard error; 95% CI = 95% confidence interval. Total charges are in United States dollars, adjusted for inflation.

Association of Individual Adherence Risk Categories With Health Care Utilization

Given that there was no association between overall adherence risk and the health care utilization outcomes, post hoc analyses were conducted looking at each of the three individual risk categories as predictors of number of ED visits and number of hospitalizations to determine if any specific risk factor was more predictive of health care use than the overall score. Results can be found in Table 3.

Table 3.

Regression Results for Individual Risk Categories Predicting ED Visits and Hospitalizations

	Outcome
	ED visits				Hospitalizations
Predictor	IRR	SE	p	95% CI	IRR	SE	p	95% CI
Lab risk	.98	.01	.06	[.96, 1.00]	.98	.02	.17	[.94, 1.01]
Std dev risk	1.01	.01	.13	[.99, 1.03]	1.02	.01	<.001	[1.01, 1.03]
Visit risk	1.05	.01	<.001	[1.03, 1.08]	.70	.09	.003	[.55, .89]

Note. ED = emergency department; IRR = incidence rate ratio; b = unstandardized regression coefficient; SE = robust standard error; 95% CI = 95% confidence interval.

Number of ED visits was significantly predicted by visit risk (i.e., clinic follow-up appointments), such that the incident rate of an ED visit was increased 5% for every one unit increase in visit risk scores, while holding the other risk variables constant in the model. Standard deviation risk (i.e., variability of tacrolimus drug levels) and lab (i.e., blood draws) risk were not significant predictors of the number of ED visits. The set of risk variables explain 4% of the variability in ED visits (R² = .04, p < .001).

Visit risk score also predicted a decrease in the number of hospitalizations, such that there was a 30% percent decrease in the incident rate of hospitalizations for every one-unit increase in visit risk score, while holding the other risk variables constant in the model. Drug level standard deviation risk predicted an increase in hospitalizations, such that the incident rate of hospitalizations was increased 2% for every one unit increase in high standard deviation risk scores, while holding the other risk variables constant in the model. Lab risk was not significantly associated with hospitalizations. The set of risk variables explain 14% of the variability in number of hospitalizations (R² = .14, p < .001).

Association of Individual Risk Categories With Charges

High standard deviation risk was a significant predictor of increased total charges, outpatient charges, and inpatient charges. Visit risk was a significant predictor of increased ED charges. Results can be found in Table 4.

Table 4.

Regression Results for Individual Risk Categories Predicting Health Care Charges

	Outcome
	Total charges				ED charges				Outpatient charges				Inpatient charges
Predictor	b	SE	P	95% Cl	b	SE	P	95% Cl	b	SE	P	95% Cl	b	SE	P	95% Cl
Lab risk	−.01	.01	.26	[−.03, .01]	−.03	.03	.20	[−.09, .02]	−.01	.01	.26	[−.02, .01]	−.01	.05	.76	[−.10, .08]
Std dev risk	.01	.01	.008	[.003, .03]	−.01	.02	.67	[−.04, .02]	.01	.004	.02	[.002, .02]	.07	.03	.008	[.02, .12]
Visit risk	.00	.02	.999	[−.04, .04]	.15	.06	.008	[.04, .26]	.01	.02	.74	[−.03, .04]	−.11	.10	.24	[−.30, .08]

Note. ED = emergency department; b = unstandardized regression coefficient; SE = robust standard error; 95% Cl = 95% confidence interval. Total charges are in United States dollars, adjusted for inflation and log transformed because of skewness.

Discussion

Three-quarters of our sample in this pilot study demonstrated some level of nonadherence to their health care regimen based on our overall risk stratification score over a 90-day period. Of the individual factors making up the risk stratification score, erratic drug levels (i.e., standard deviation of the last four immunosuppression drug levels ≥2) was the single greatest driver of increased adherence risk stratification score, followed by not obtaining labs for blood work. Interestingly, only 14% of patients did not attend clinic visits according to the minimum expectation. These data suggest nonadherence to different components of the treatment regimen are variable, which is consistent with the broader pediatric literature (Modi et al., 2006). Our data suggest that focusing on immunosuppression drug levels and missed clinic visits may be a priority.

To our knowledge, this is the first study to examine the predictive validity of a treatment adherence risk stratification score for long-term health care utilization and health care charges in pediatric kidney transplant patients. In this pilot study, the overall composite risk stratification score did not predict health care utilization or charges. Lack of findings may be due to the small sample size of this single center pilot study.

Our results demonstrate that pediatric kidney transplant recipients with greater variation in immunosuppression blood levels (SD ≥ 2) had an increased risk of hospitalizations compared with patients with less variable immunosuppression levels. Greater variation in immunosuppression blood levels (SD ≥ 2) were also associated with significantly higher health care charges. These results are not surprising in light of existing literature in transplantation demonstrating erratic immunosuppression drug levels are correlated with rejection episodes, which are typically predictors of health care charges and utilization (Gheorghian et al., 2012; Stuber et al., 2008). Together, these findings and previous research imply that standard deviation of immunosuppression levels may serve as a good biomarker for complications that result in hospitalizations. There are a variety of factors that influence tacrolimus drug levels including timing of taking medication, consistency in taking it, diet, other medications, and diarrhea (Filler, 2007). Thus, high standard deviation of tacrolimus may serve as proxy of whether the medication is being taken consistently and at consistent times and also reflect overall variability in other life activities, which together seem to indicate a higher risk for hospitalization. Conversely, drug level standard deviation did not predict ED visits, a finding that is not surprising given the robust care coordination systems in place for patients who have undergone a kidney transplant that emphasizes outpatient management, direct admission to the hospital and avoidance of the emergency department except in case of true emergency. For instance, it is the standard practice for patients experiencing complications to be seen in clinic and directly admitted to the hospital rather than going through the ED. The intense and routine monitoring of transplant patients may result in less acute crises requiring ED services compared with other chronic conditions that are intermittent in nature (e.g., seizures, asthma attacks, ketoacidosis).

Clinic visit risk, on the other hand, was associated with seemingly contradictory outcomes. Patients not seen in the previous four months had a decreased risk of hospitalizations, but an increased risk of ED visits, and ED-related charges. Patients who are failing to follow-up routinely with their outpatient appointments may have concerns that are not detected until they become an emergency (e.g., rising creatinine, rejection). However these outcomes may reflect characteristics of the way the score is calculated; it simply assesses whether a patient had been seen in clinic within the previous four months. It may be the case that less complicated patients with fewer health problems are scheduled to be seen less frequently (every three to four months) than more complicated patients, who might need to be seen up to monthly. Thus, it is much more likely that the healthier patient could avoid being seen within four months, but would also be less likely to require hospitalization. On the flip side, these patients are more likely to use the ED when they have acute care problems because they are being seen less frequently in clinic. Unfortunately, our small sample size prevented a more sophisticated analysis stratified by disease complexity.

There are several clinical implications for the various system levels that are impacted by care for kidney transplant recipients, including the medical team, hospital administrators, and payors. Specifically, medical teams that are caring for pediatric patients often dedicate significant effort in reaching out to patients and families who are overdue for lab work or clinic visits. In fact, our team provided feedback that following up via telephone calls and e-mails took a significant amount of time. Team members required preparation and training to help families identify barriers to adherence (e.g., unreliable transportation, do not want patient to miss school for appointments) and make suggestions for overcoming these obstacles. Because our findings revealed that higher standard deviation of immunosuppression drug levels was the most common adherence behavior risk and was associated with later health care utilization, perhaps a greater emphasis and focus could be devoted to intervening with those patients as a priority. For example, pediatric psychologists are uniquely positioned to provide intervention for patients who are failing one or more of the criteria in the classification system. They can facilitate implementation of evidence-based adherence promotion strategies including technology-based solutions, such as text message or cell phone application reminder systems (Miloh et al., 2009; Pai & McGrady, 2014), or problem-solving around adherence barriers (Varnell et al., 2017). Additionally, efforts could be focused on stabilizing the home routine of taking medication on time and avoiding specific things in the diet that might affect tacrolimus metabolism. Insurance companies may be willing to invest support for adherence assessment or intervention to improve standard deviation of drug levels, if adequate control can prevent costly complications such as rejection or re-transplantation down the road. Additionally, hospital administration may provide incentives to pediatric programs who use preventive care methods (e.g., intervening on adherence barriers related to erratic immunosuppression levels) as these are more likely to save money in the long-run. There may be opportunities in the future to strengthen assessment of risk through the use of objective and validated measures of adherence, such as electronic monitoring. Future work in this area may try to expand the conceptualization of risk by incorporating other components such as socioeconomic status and psychosocial functioning (e.g., internalizing or externalizing symptoms).

There were several lessons learned regarding hospital charge data. Unfortunately, we were only able to access the amount of money that was charged by the institution, which does not represent the actual costs. It would be beneficial to prospectively assess the indirect costs (e.g., transportation, missed days from work or school) to broadly capture the economic impact of nonadherence, rather than a narrow proxy (i.e., hospital charges) for actual costs. This could potentially be included in electronic questionnaires that patients already complete when they attend a medical visit. An unexpected issue was the amount of manual reorganization of the charges data that was required. Although it was received in a very short-term frame in an electronic spreadsheet from the institution, charges were separated at a very fine level of detail (e.g., physician time for a 40 min outpatient visit, renal panel, tacrolimus level, venipuncture procedure). This required the research team to determine a strategy for combining each charge into a logical grouping, such as “emergency department visit” or “inpatient hospitalization.” Setting prespecified criteria for which charge will be included for encounter type will improve reliability with this process, such as recommended by Neff and colleagues (2004).

Findings from the current study should be considered within the context of its limitations. First, this was a small pilot study of a single transplant center; therefore, lack of statistically significant findings may reflect the small sample size rather than the lack of a true effect. Additionally, we did not make corrections for multiple tests because of the exploratory nature of this work, and thus it is possible that the some of the significant findings could be attributable to Type I errors. Future studies might combine pediatric kidney transplant recipients across multiple centers to increase statistical power and generalizability of results. A larger study would also allow for risk stratification of patient medical complexity which certainly also has a major impact on health care utilization and charges. Second, the 24-month time frame may not have been sufficient to capture low frequency events, such as rejection episodes and hospitalizations. Extending the amount of time that patients are monitored for health care utilization and associated charges would provide useful data for the long-term impact of poor adherence to immunosuppressant medication, lab work, and clinic visits. Finally, we did not collect data regarding behavioral interventions that may have been delivered to patients during the course of this study. An important future direction for research is to evaluate the potential impact of evidence-based psychological treatments on all aspects of adherence and potential offset of future economic cost to treat medical complications. This could provide leverage for pediatric psychologists to advocate for reimbursement for services by insurance companies.

In summary, we have developed an adherence risk stratification score that is easy to calculate from readily available discrete elements in the EHR and can be used at the point of care to identify poor adherence behaviors. We found that three quarters of patients exhibited some level of adherence risk in a three-month period. Although the overall score has clinical utility, it did not predict statistically significant increased health care utilization or charges in this small pilot study Rather, a standard deviation greater than two for the most recent four drug levels was associated with increased risk for hospitalization, and decreased clinic visit frequency was associated with increased utilization of the emergency department. Further research and larger studies will be necessary to refine this risk scoring system to be a valid predictor of health care utilization and charges. In the meantime, immunosuppression drug variability appears to be the most common predictor of increased utilization and charges, and may highlight a specific population of patients on which to focus interventions.

Implications for Impact Statement.

Nonadherence to immunosuppression medication is costly in terms of impact on health and financially. Having an easy-to-use system to quickly identify patients at risk for nonadherence at the point of care has clinical utility. Intervening on kidney transplant recipients who have high drug level variation or miss clinic visits may deter future costly medical care.