Utilizing home health care electronic health records for telehomecare patients with heart failure: a decision tree approach to detect associations with rehospitalizations

Abstract

Heart failure is a complex condition with a significant impact on patients’ lives. A few studies have identified risk factors associated with rehospitalization among telehomecare patients with heart failure using logistic regression or survival analysis models. To date there are no published studies that have used data mining techniques to detect associations with rehospitalizations among telehomecare patients with heart failure. This study is a secondary analysis of the home health care electronic medical record called the Outcome Assessment and Information Set (OASIS)-C for 552 telemonitored heart failure patients. Bivariate analyses using SAS™ and a decision tree technique using Waikato Environment for Knowledge Analysis were used. From the decision tree technique, the presence of skin issue(s) was identified as the top predictor of rehospitalization that could be identified during the start of care assessment, followed by patient’s living situation, patient’s overall health status, severe pain experiences, frequency of activity-limiting pain, and total number of anticipated therapy visits coombined. Examining risk factors for rehospitalization from the OASIS-C database using a decision tree approach among a cohort of telehomecare patients provided a broad understanding of the characteristics of patients who are appropriate for the use of telehomcare or who need additional supports.

Introduction

Heart failure (HF) is a complex condition that often requires rehospitalization.^1–6 Studies of the causes of rehospitalization reveal inconsistent patterns of risk factors for rehospitalizations in the home health care setting. ^7–9 Due to the complex nature of HF, evaluating individual risk factors for rehospitalization in isolation may limit the ability of clinicians to optimally target health services such as telehomecare, which is a remote monitoring intervention used in the home health care setting. Instead, understanding associations among risk factors for rehospitalization using the standardized home health care electronic health records for Medicare recipients (i.e., the Outcome and Assessment Information Set (OASIS), which may be elucidated using data mining techniques, may provide guidance for admission assessments by home health care clinicians to help target those patients most likely to benefit from telehomecare, or who may need additional interventions.

Medicare-certified home health care agencies have been required to use the OASIS system as the data-collection tool for all Medicare recipients (18 years or older) except maternity patients since 1999. ^10–12 The OASIS data set has been used for clinical assessment, care planning, outcome monitoring, and comprehensive evaluations of home health care service outcomes.¹³ A subset of the OASIS-based performance measures calculated by the Center for Medicare/Medicaid Services (CMS) are reported to the public via the Home Health Compare web site and are used in payment algorithms under the Medicare Prospective Payment System. ¹⁰

Multiple studies have utilized the OASIS as administrative data to discover risk factors for hospitalization,^7,9,14,15 and two studies have used data mining techniques to evaluate the OASIS data set.^16,17 However, to our knowledge, no studies have applied these approaches to the OASIS-C version that was initiated in 2010, which we obtained from a large home health care company to identify risk factors for rehospitalization among telehomecare patients with HF. Unfortunately, it is not possible to use the national OASIS-C sample to compare telehomecare users with non-telehomecare users because there is no specific code to identify the intervention within the data set.

Methods

This study is a secondary analysis of the OASIS-C data set, which was released in 2010. A total of 552 eligible patients were identified as having a diagnosis of HF based on the International Classification of Disease, Ninth Revision (ICD-9) coding, who had also received telehomecare and had a discharge from an in-patient facility stay within 14 days of the start of home health care.

Data analysis

To create the outcome variable (i.e., first rehospitalization during the 60 day home health care episode) and to perform bivariate analyses, SAS™ version 9.4 was used. To examine the associations between rehospitalization and OASIS-C, the following bivariate analyses including 85 OASIS-C items were generated: Chi-square/Fisher Exact analyses were used to assess for differences between rehospitalized and non-rehospitalized patients in terms of binary/categorical/count variables; t-tests were used to assess the association between rehospitalization status and normal continuous predictor variables; and Wilcoxon rank sum tests were used for non-normal continuous variables. A p-value <0.05 was considered statistically significant. Bivariate analyses were used to help identify promising variables for developing the best model from the decision tree results.

Next, a decision tree was generated using Waikato Environment for Knowledge Analysis (WEKA) software version 3.6.12 to visualize associations among risk factors and explore the profile of patients most at risk of rehospitalization at the start of care, using the tree-building technique. WEKA was developed as a form of machine learning software at the University of Waikato in New Zealand in 1993.¹⁸ In determining the number of data mining independent variables, called attributes, to consider, different techniques were attempted. The data were divided into two sets; the test dataset (368/552, 67%) was used to construct the decision tree, while the validation set (184/552, 33%) was used to confirm the results of the decision tree.

The J48 using 10-fold cross validation procedure in WEKA was used to create the decision tree. In the 10-fold cross validation process, the data were divided into 10 folds: a training dataset that represents nine folds is used to create the model being tested on the last fold.¹⁶ This step was repeated until each fold had been used as a validation set.¹⁶ The final results were averaged across the ten repetitions.¹⁶ Then the models were evaluated based on the true positive rate (sensitivity), the false positive rate (1-specificity) and the area under the receiver operating characteristic curve (AUC), which is plotted based on the sensitivity against 1-specifity for different cut-points based on a varying probability of the outcome.^19–21 The AUC is equivalent to the concordance (c) statistic.^19–21

The c-statistic is the mostly commonly used value for evaluating model discrimination, which is the ability to discriminate predictions between groups with and without the outcome.^19–21 The values of the c-statistic, the true positive rate and the false positive rate range from 0 to 1.¹⁶ The c-statistic is a rank-based test to measure how well a model discriminates between two groups (i.e., those with and without the event, or with and without an intervention) based on the outcome of interest, which reflects the discriminative ability of the model.²¹ If the value of the c statistic is greater than 0.7, the model is considered accurate; the closer the value is to 1.0, the better the model.^19–22 If the value is 0.5, the model lacks predictive accuracy and demonstrates “no discrimination.”^21,22 If the true positive rate is close to 1, and the false positive rate is close to 0, the results indicate a better decision tree.¹⁶ Finally, to select the best model, we considered the c-statistic values and the significant variables from the bivariate analyses.

Results

The associations among demographic characteristics and first rehospitalization during the 60-day home health care episode are shown in Table 1. The non-rehospitalized patients were on average one year older than the rehospitalized patients, but there was no statistically significant difference in age between the two groups (p=.19). There was no statistically significant difference in gender between the two groups in terms of rehospitalization rate (p=.85). The majority of patients were White (83%), and there was no statistically significant difference in the proportion of White patients who were rehospitalized (36%) in comparison to non-Whites (35%) (p=.80).

Table 1.

The associations among demographic characteristics and first rehospitalization

Demographic characteristics	Median (interquartile range) or Count (row % for subgroups, column % for entire group)
	Rehospitalized (N=198) (n, row%)	Non-rehospitalized (N=354) (n, row%)	Entire (N=552) (n, column%)	p-value
Age (M (IQR))	78.6 (16.7)	79.3 (14.0)	79.0 (15.0)	0.19
Gender				0.85
Male	90 (36)	158 (64)	248 (45)
Female	108 (36)	196 (64)	304 (55)
Race/Ethnicity				0.80
White	165 (36)	292 (64)	457 (83)
Non-white	33 (35)	62 (65)	95 (17)

Note: M-median; IQR- interquartile range

Decision tree

For the decision tree analysis, the data were divided into test and validation sets. The c-statistic of the best model in the validation dataset was 0.59. The true positive rate was 0.65, and the false positive rate was 0.49. Detailed descriptions of the variables (as defined in the OASIS-C dataset) that were associated with rehospitalization are presented in Table 2.²³ Figure 1 presents the decision tree derived from the best predictive model in the WEKA output, which was chosen based upon the highest value of the c-statistics and the significant variables from bivariate analyses. It provided additional information regarding associations among various risk factors instead of simply demonstrating the associations between the outcome of rehospitalization and a single risk factor. The presence of skin issue(s) was identified as the top risk factor for rehospitalization that could be identified during the start of care assessment, followed by patient living situation, patient overall health status, severe pain experiences, frequency of pain interfering with activities or movement and total number of anticipated therapy visits (e.g., total of physical, occupational and speech-language pathology visits combined).

Table 2.

Detailed descriptions of the variables (as defined in the OASIS-C dataset²³) that were associated with rehospitalization and definitions for the purpose of this study

OASIS-C questions (definition for the purpose of this study)	‘Definitions’ for the purpose of this study and Descriptions of the options from each question
Overall status: Which description best fits the patient’s overall status? (defined as patient’s overall health status)	‘Stable patients’ were defined as being stable with no heightened risk(s) for serious complications and death (beyond those typical of the patient’s age) ‘Mildly sick patients’ were defined as having high health risk(s), but they were more likely to return to stable status ‘Moderately sick patients’ were defined as still being in weak condition with ongoing high risk(s) for serious complications and death The ‘Sickest patients’ were defined as having serious ongoing condition(s) that could result in death within the next year
Patient living situation (defined as patient’s living situation)	Living alone Living with other person(s)
Has this patient had a formal pain assessment using a standardized pain assessment tool?: appropriate to the patient’s ability to communicate the severity of pain (defined as severe pain experiences)	No standardized, validated assessment conducted Assessment conducted, but it does not indicate severe pain Assessment conducted, and it indicates severe pain
Frequency of pain interfering with the patient’s activity or movement (defined as frequency of activity-limiting pain)	Patient has no pain Patient has pain that does not interfere with activity or movement Less often than daily Daily, but not consistently At all times
Does this patient have a skin lesion or open wound, excluding bowel ostomy? (defined as presence of skin issues)	No Yes
The ability to dress one’s lower body safely (with or without dressing aids) including undergarments, slacks, socks or nylons, shoes (defined as the ability to dress the lower body)	‘Independent patients’ were defined having the ability to obtain, put on, and remove clothing and shoes without assistance ‘Mildly dependent patients’ were defined as being able to dress their lower bodies without assistance, if clothing and shoes were laid out or handed to them ‘Moderately dependent patients’ were defined as requiring assistance to put on undergarments, slacks, socks or nylons, and shoes ‘Completely dependent patients’ were defined as being entirely dependent upon another person to dress their lower body
Therapy need: indicating a total of reasonable and necessary physical, occupational and speech-language pathology visits combined.	‘Total number of anticipated therapy visits combined’

Figure 1.

The decision tree for home health care providers

The decision tree model in Figure 1 also represents a clinically interpretable decision-making algorithm for home health care providers. The interpretation starts with the top attribute, then moves down to other attributes following different potential pathways. First, patients who did not have skin issue(s), who lived with other person(s) and who had severe pain were more likely to be rehospitalized when their total expected number of therapy visits combined was less than 11. On the other hand, patients who did not have skin issue(s) and who lived with other person(s) without the presence of severe pain were less likely to be rehospitalized.

Another potential pathway exists in which patients who had skin issue(s) but did not have pain interfering with activity or movement (or who had pain interfering less often than daily) were more likely to be rehospitalized, if they were considered to be in the moderately sick or sickest groups. On the other hand, patients who had skin issue(s) and who had pain interfering with activity or movement daily (but not constantly) or at all times, were more likely to be rehospitalized.

Variables associated with rehospitalization from the bivariate analyses

Five variables were found to be significantly associated with rehospitalization from the bivariate analyses: patient’s overall health status, patient’s living situation, severe pain experiences, presence of skin issue(s), and ability to dress the lower body. Table 3 presents differences in risk factors for rehospitalization at the start of home health care assessment between the rehospitalized and the non-rehospitalized groups. First, the patient’s overall health status (p=.04) was associated with rehospitalization during the 60 day home health care episode. Among rehospitalized patients, moderately sick patients had the highest rehospitalization rate (41%). Secondly, among rehospitalized patients, those who lived with other person(s) had a higher proportion of rehospitalizations (40%) than those who lived alone or who lived in a congregate environment (26%) (p <0.01). Next, severe pain experiences (p=.02) were associated with rehospitalization; patients with severe pain had a higher proportion of rehospitalizations (47%) compared to patients without severe pain (33%).

Table 3.

Risk factors for rehospitalization in bivariate analyses

Risk factors	Mean (Standard deviation) or Count (row % for subgroups, column % for entire group)
	Rehospitalized (N=198) (row %)	Non-rehospitalized (N=354) (row %)	Entire (N=552) (column %)	χ2
Overall health status				8.77*
Stable or mildly sick group	82 (31)	186 (69)	268 (49)
Moderately sick group	97 (41)	138 (59)	235 (42)
Sickest group	19 (39)	30 (61)	49 (9)
Living situation				8.77**
Live alone	37(26)	107(74)	144(26)
Live with someone	161(40)	247(61)	408(74)
Severe pain experiences#				7.90**
No severe pain from a formal pain assessment	153 (33)	305 (67)	458(84)
Severe pain from a formal pain assessment	40 (47)	46 (53)	86(16)
Presence of skin issue(s)				5.37*
No	173(34)	330(66)	503(91)
Yes	25(51)	24(49)	49(9)
Ability to dress the lower body				6.93*
Independent	24(51)	23(49)	47(9)
Mildly dependent	31(29)	76(71)	107(19)
Moderately or completely dependent	143(36)	255(64)	398(72)

Note:

^#Pain experiences-8 observations were excluded because of no standardized pain assessment tool used; M-mean; SD- standard deviation;

^*p<0.05;

^**p<0.01

Fourth, the presence of skin issue(s) (p=.02) was associated with rehospitalization. Among rehospitalized patients, those with skin issue(s) had a higher proportion of rehospitalizations (51%) than those without skin issue(s) (34%). In addition, among patients who had skin issue(s), 63% had a physician-ordered plan of care for diabetic foot care. Lastly, in terms of ability to dress the lower body, patients in the independent group had the highest proportion of rehospitalizations (51%), and those who were mildly dependent had the lowest proportion of rehospitalizations (29%) (p=.03).

Discussion

In the WEKA output, there were two main patient groups (i.e., patients with and without skin issue(s)) who were considered to be at risk for rehospitalization. There are two factors (e.g., absence of skin issue(s) and living with another person) that might be anticipated to promote good patient outcomes. However, even among those patients, severe pain was associated with a higher likelihood of rehospitalization if patients were engaged in less than 11 anticipated therapy visits. These results indicate that even with good social support, pain may be a driver of rehospitalization among telehomecare users.

Although the association between pain symptoms and rehospitalization in the HF population is understudied,²⁴ unrelieved or mistreated pain has a negative impact on quality of life, which can interfere with the performance of necessary self-care and disease management activities.^24,25 An alternative explanation for this finding is that when family members see a loved one in pain, they advocate for care that might involve rehospitalization. Finally, it also may be that more supportive therapies, such as physical therapy and occupational therapy, are indicated for these patients to address their pain, because the decision tree highlights the importance of the number of anticipated therapy visits among patients with severe pain. On the other hand, those patients who did not have skin issue(s) and who lived alone were less likely to be rehospitalized. It is possible that these patients benefitted from receiving telehomecare, since it might provide supplemental input where social support is lacking, thereby helping to keep patients out of the hospital.

We suggest providing home health care providers with a set of risk factors that would enable them to recognize high-risk patients at the start of care, not only as individual contributors to risk but also as additive effects among a combination of risk factors. This result also would be expected to facilitate earlier involvement of other home health care providers, including therapists, to optimize the patient’s care planning and initiate additional interventions for telehomecare users. Developing an automated decision support tool embedded in the electronic health record may be warranted, so that certain high-risk patient responses to standard questions upon home health care admission could trigger an alert for further action.

A decision tree showing providers a global picture of a given patient’s risk may aid in better clinical decision-making than individual predictors from a regression model. It also suggests a relative ranking of risk factors for rehospitalization, which otherwise might be incorrectly presented as individual risk factors with equal impact on patient risk. Another benefit of the decision tree method used in our study is the model’s graphic depiction, which allows for easier interpretation during application to clinical practice compared to traditional statistical results.

Despite the advantages of decision tree techniques, they have not been used frequently because of a lack of awareness about the use of decision trees in general,²⁶ and concerns about the possibility of misclassification errors, such as inaccurate calculation of cutoffs for continuous variables or inaccurate splits of categorical variables.²⁷ Some statisticians are skeptical about using decision tree techniques because of the lack of goodness of fit testing compared to traditional statistical methods.²⁷ However, the decision tree technique was considered to be a strength of this study because the OASIS-C data set may contain unknown complex interactions among a host of potential risk factors. For example, the items for functional status (i.e., grooming, ability to dress the upper body, ability to dress the lower body, bathing, toilet transferring, ambulation/locomotion and feeding or eating) may have interactions with more than two risk factors, causing difficulty in traditional modeling approaches. Our ability to account for these factors has important implications for providers attempting to target the ideal sub-population for telehomecare services among home health care patients with HF.

In this study, the AUC value of the best model was 0.59, the true positive rate was 0.65, and the false positive rate 0.49. These results are not ideal values, however, they are similar to the values obtained in other studies using OASIS data. For example, one study that evaluated two predictive models of rehospitalization using the OASIS data set reported that the values of the AUC were 0.63 and 0.59.¹⁴ Another study that identified characteristics associated with improvement either in urinary or bowel incontinence using OASIS data reported c-statistics ranging from 0.50 to 0.58.¹⁶ Poor predictive performance has been observed in other studies of HF patients. In a systematic review of statistical models used for predicting rehospitalizations in HF patients, the c-statistic values from studies looking at predictors of rehospitalization (i.e., 0.60) were consistently lower than the similar values obtained from studies evaluating predictors of mortality after hospitalization (0.67–0.81).^4,28–30

It is hoped that the study findings may lead to the development of a new screening tool or protocol for identifying patients at high risk for rehospitalization among all telehomecare users, since there are no uniform guidelines for targeting appropriate patients for telehomecare services. Presumably this is due to a lack of evidence about the patient characteristics that maximize patients’ responses to telehomecare, which is a gap filled by this study.

Implications

Practice

Upon admission to home health care, providers have to perform a full patient assessment using the OASIS-C start of care template, and order a plan of care based on that assessment. It might be difficult for home health care providers to spend additional time learning about a patient’s ability to use telehomecare, while also completing their OASIS-C start of care documentation and reviewing the patient’s discharge instructions. Thus, the decision tree developed in this study provides a simple set of rules for admitting home health care providers to identify priority patients for telehomecare placement, which are supported by findings from traditional statistical methods. For example, patients with skin issue(s) and severe pain would need to receive the highest priority for telehomecare placement. Home health care providers could use the decision tree to quickly identify multiple patient issues potentially affecting rehospitalizations, with a more comprehensive picture of patient risk within the HF population than has ever been available before.

Research

The results of this study suggest that adding decision tree analyses to the available methodologies for assessing big datasets, such as the national OASIS-C data, may provide valuable visualizations of the interactions among various risk factors for rehospitalization. They also suggest that using big datasets might be helpful for developing clinical decision support systems. Thus, using data mining techniques with future OASIS data sets including both telehomecare users and non-telehomecare users may be a worthwhile approach for evaluating risk for rehospitalization among patients with HF, which would serve as an important first step in creating guidelines or recommendations for the optimal placement of telehomecare patients.

Recognizing associations among multiple risk factors captured in the OASIS-C data set may be helpful for home health care providers developing plans of care for patients with HF. Specifically, admitting home health care providers in the post-hospital discharge setting may utilize this set of predictive factors to identify patients’ needs for more intensive and appropriate targeting of education regarding telehomecare services. For example, early education might be prioritized for patients who have skin issue(s) from diabetes or edema. Since telehomecare services are becoming increasingly attractive to home health care agencies interested in reducing rehospitalization rates and medical costs through fewer nursing visits,^31,32 it may be critical for the efficient use of telehomecare to appropriately target high-risk patients.

Health policy

Researchers using the large OASIS dataset have traditionally relied upon the national sample provided by OASIS data, but it is difficult to know which patients within the dataset used telehomecare due to a lack of items concerning the use of telehomecare. Adding an OASIS code for identifying telehomecare patients within the transfer or discharge OASIS data may be helpful for researchers to identify patients who received telehomecare when using the future versions of the OASIS data set. Such changes based on robust evidence would improve the accuracy of future studies on telehomecare in HF patients, including efforts to create a screening tool for telehomecare placement.

Limitations

One limitation of using existing data is the inability to explore other potential variables affecting the outcome that are not captured by the dataset. For example, OASIS does not include bio-markers or characteristics of medication non-compliance that may affect rehospitalization. A limitation specific to this study is that the sample size might have not been sufficient for the use of data mining techniques, although the minimum required sample size for application of data mining techniques is unclear. Also, there are disadvantages to developing a decision tree. For instance, decision tree techniques do not force variables into the model, limiting the ability to adjust for important variables that were not significant.²⁷ Consequently, the software may have excluded other potential risk factors from this analysis. Another disadvantage is that even small changes, such as transformation of the variables, could result in generation of an unstable tree, due to differences in splitting for the test and the validation sets.^26,27 In this study, there was a possibility that a tree-shaped predictive model might have not been fully developed because of the low c-statistic value, which is considered to reflect poor discriminative ability. Lastly, these study findings may not be applicable in creating a screening tool for telehomecare placement, because there was no comparison of the effect of telehomecare on rehospitalizations among users and non-users.

Conclusions

This study was a first attempt to examine risk factors for rehospitalization in the OASIS-C data set using decision tree techniques among a cohort of telehomecare patients. The results provided a preliminary understanding of the characteristics of telehomecare patients that were associated with rehospitalization. In addition, use of a decision tree in this study provided a visual depiction of the associations among risk factors, allowing a more complete exploration of the profile of patients at high risk for rehospitalization among all patients who used telehomecare. As the use of telehomecare continues to grow, it is critical for home health care agencies to develop a valid, reliable and clinically relevant screening tool that quickly identifies risk factors most likely to yield poor outcomes, in order to support better utilization of telehomecare and related interventions.