Predicting 30- to 120-Day Readmission Risk among Medicare Fee-for-Service Patients Using Nonmedical Workers and Mobile Technology

Abstract

Objective

Hospital readmissions are a large source of wasteful healthcare spending, and current care transition models are too expensive to be sustainable. One way to circumvent cost-prohibitive care transition programs is complement nurse-staffed care transition programs with those staffed by less expensive nonmedical workers. A major barrier to utilizing nonmedical workers is determining the appropriate time to escalate care to a clinician with a wider scope of practice. The objective of this study is to show how mobile technology can use the observations of nonmedical workers to stratify patients on the basis of their hospital readmission risk.

Materials and Methods

An area agency on aging in Massachusetts implemented a quality improvement project with the aim of reducing 30-day hospital readmission rates using a modified care transition intervention supported by mobile predictive analytics technology. Proprietary readmission risk prediction algorithms were used to predict 30-, 60-, 90-, and 120-day readmission risk.

Results

The risk score derived from the nonmedical workers' observations had a significant association with 30-day readmission rate with an odds ratio (OR) of 1.12 (95 percent confidence interval [CI], 1 .09–1.15) compared to an OR of 1.25 (95 percent CI, 1.19–1.32) for the risk score using nurse observations. Risk scores using nurse interpretation of nonmedical workers' observations show that patients in the high-risk category had significantly higher readmission rates than patients in the baseline-risk and mild-risk categories at 30, 60, 90, and 120 days after discharge. Of the 1,064 elevated-risk alerts that were triaged, 1,049 (98.6 percent) involved the nurse care manager, 804 (75.6 percent) involved the patient, 768 (72.2 percent) involved the health coach, 461 (43.3 percent) involved skilled nursing, and 235 (22.1 percent) involved the outpatient physician in the coordination of care in response to the alert.

Discussion

The predictive nature of the 30-day readmission risk scores is influenced by both nurse and nonmedical worker input, and both are required to adequately triage the needs of the patient.

Conclusion

Although this preliminary study is limited by a modest effect size, it demonstrates one approach to using technology to contribute to delivery model innovation that could curb wasteful healthcare spending by tapping into an existing underutilized workforce.

Background and Significance

Approximately $355 billion in healthcare spending is wasted each year in the United States as a result of failures of care delivery, poor care coordination, and overtreatment, including up to $44 billion attributable to unplanned hospital readmissions.^{1, 2} More than 34 million patients are discharged from hospitals or emergency rooms each year, and interventions that improve care transitions from one healthcare setting to another have been shown to reduce readmissions.^{3, 4} However, a major barrier to the sustainability of traditional nurse-staffed transitional care interventions is their high cost relative to the readmission penalties they are designed to prevent.^{5, 6, 7, 8, 9} Sustainability of these programs is further limited by the growing nursing shortage in the United States.¹⁰

One opportunity to overcome the threat to the sustainability of transitional care interventions is offered by leveraging the existing, underutilized workforce of more than 5 million frontline workers that provide long-term supports and services (LTSS) to help the aging population maintain function and address nonedical health determinants in the community.^{11, 12, 13, 14} This workforce, referred to as nonmedical workers in this study, includes personal care attendants, home health aides, home meal delivery drivers, health coaches, community health workers, social worker case managers, and other providers of essential nonmedical functions.^{15, 16} Nonmedical workers are involved in 8 out of 10 hours of paid services provided to the elderly and people with disabilities, and growing evidence shows that they can improve patient experience and outcomes.^{17, 18, 19, 20, 21} Furthermore, the average nonmedical worker is paid an hourly salary that is approximately 70 to 90 percent less than that of a nurse or physician, respectively.²¹ With the exception of transition models that use social workers, most transitional care interventions fail to tap into the nonmedical workforce to minimize program administration costs.^{22, 23}

A major barrier to utilizing the nonmedical workforce is determining the appropriate time to escalate care to a clinician with a wider scope of practice. An opportunity to overcome this barrier and a key function of transitional care interventions is stratifying patients on the basis of their risk of readmission.²⁴ Existing risk prediction approaches rely on hospital electronic health record and claims data which typically require an interaction with a physician.^{25, 26, 27, 28, 29, 30, 31} This approach to stratification fails to capture the dynamically changing risk for hospitalization between the doctor visits or hospitalizations. Furthermore, traditional risk prediction approaches predominantly emphasize medical risk factors, which have been shown to only account for part of the variability in hospitalization rates.^{32, 33} With the burgeoning of mobile technology, we identified an opportunity to fill the gap in predicting hospitalization risk for care transition interventions that utilized prevalent, low-cost, and high-touch nonmedical workers.^{34, 35}

The objective of this retrospective review of quality improvement data from a community-based transitional care intervention was to show how mobile technology using observations made by nonmedical workers could predict hospital readmissions. In particular, we aimed to answer the following questions: Can observations made by nonmedical workers be used to predict 30-day readmissions? Can readmission risk prediction using observations made by nonmedical workers be improved by clinician oversight? Can observations made by nonmedical workers be used to predict readmissions beyond 30 days after discharge?

Materials and Methods

Elder Services of Merrimack Valley (ESMV), an area agency on aging in Massachusetts, designed this quality improvement (QI) project to improve the agency's transitional care program with the aim of reducing 30-day hospital readmissions using mobile technology. ESMV identified suboptimal communication between nonmedical and clinical staff as a major driver impeding achievement of its aim to reduce 30-day readmissions. To address this challenge, ESMV focused its change strategies on solutions amenable to intervention by mobile technology. The strategies focused on improving communication and care coordination among staff. The primary measure of success throughout the QI project was the 30-day readmission rate among participants in the agency's care transition program.

This project was a quality improvement activity, monitored closely by the clinically responsible professionals, and abiding by the requirements of the Health Insurance Portability and Accountability Act (HIPAA), the Health Information Technology for Economic and Clinical Health (HITECH) Act, and other constraints protecting patient privacy. ESMV and the Massachusetts Executive Office of Elder Affairs determined that, as a QI project in ordinary operations, this initiative was not classified as research on human subjects, and therefore ESMV did not seek review by an institutional review board.

Setting and Participants

The QI project is ongoing, but the data reviewed for this analysis were collected by ESMV between July 10, 2013, and April 23, 2014. Eligible patients were those enrolled in the ESMV care transition program and included Medicare fee-for-service patients. Patients were enrolled into the program if they were recently admitted to the hospital and were in the process of being discharged. Patients were excluded from the study if they had observation stay status or a planned surgical readmission within 30 days of admission. All patients meeting the inclusion criteria for the care transition program were enrolled in the ESMV QI initiative. Patients were given the option to opt out of the care transition program.

The ESMV care transition program, called the mHealth Transitions Model, is an adaptation of the Care Transition Intervention (CTI) developed by Coleman et al.³⁶ While the CTI traditionally uses transition coaches with nursing degrees, the mHealth Transitions Model used nonmedical, lay health coaches in the field supervised by a nurse care manager working out of a central office.³⁷ The health coaches had at least a high school education. They all received standard training consistent with the CTI. All coaches experienced a standard 1.5-hour training in the use of the mobile technology to perform risk assessments.

Mobile Technology

During each hospital, in-home, and telephonic encounter, the health coach used a web-based application, Care at Hand, that suggested questions in lay language based on the patient's most likely risk factors for hospitalization.³⁷ For the purposes of this analysis, we reviewed only surveys collected in person in the home or telephonically rather than surveys performed in the hospital.

The questions on the Care at Hand app changed with each administration of the survey and were driven by the technology's proprietary algorithms that predict the most likely upcoming risk factors for readmission. The surveys were limited to 15 questions and were designed to take no more than two to five minutes to complete. (See Figure 1.) If the system detected an elevated risk of readmission based on the answers to the surveys, the system automatically generated a real-time alert to a nurse care manager. The nurse care manager subsequently used a different component of the software to triage the patient to the appropriate level of care and assist the health coach in care coordination and care management within 24 hours of receiving the alert.

Figure 1.

Example of Care at Hand Survey

Data Collection

The sources of data included the ESMV administrative databases and the Care at Hand system. The unit of analysis in this study was an alert generated by the technology and triaged by a nurse care manager within 30 days of discharge from an index hospital admission for an individual patient. The primary outcome measure was a readmission to the hospital by the same patient within 30 days after discharge from the index admission. An individual patient could have multiple readmissions within 30 days of discharge. A readmission counted as another index admission only if it occurred more than 30 days after discharge from the preceding index admission. An alert episode could only be associated with a single 30-day readmission. Multiple alert episodes could be associated with an index admission because each alert received its own unique triage by a nurse. A similar approach was used for calculations using 60-, 90-, and 120-day readmission rates.

The data reviewed in this QI project stemmed from episodes in which the Care at Hand technology triggered an alert based on the observations of the health coach and the alert thresholds of the mobile technology. These data included the observations of health coaches that were captured in structured form through the mobile app. The nurse care managers' responses to the alerts were captured both in structured format and through unstructured free text in a notes section within the mobile technology. Risk scores that were generated by the technology on the basis of the structured data from the health coaches' and nurse care managers' documentation were included in the analysis.

As a validity check on risk scores collected through structured data, four nursing student research assistants (RAs) performed a blinded review and codification of the unstructured nurse documentation. Each record of nurse documentation was reviewed by two RAs. The RA pairings were randomly assigned for each documentation review to ensure an equal distribution of RA pairings.

The data captured in Care at Hand were cross-referenced with the ESMV administrative databases, which captured 30-, 60-, 90-, and 120-day hospital readmission data from the six referring hospitals for the patients served by the agency.

Readmission Risk Prediction Model

The proprietary Care at Hand readmission risk prediction algorithms are based on two contributing factors: (1) observations made by the health coaches and (2) nurse care manager interpretation of the alerts triggered by the health coach observations.³⁸

The proprietary questions in the surveys for the health coaches were organized into three categories: issues intrinsic to the patient's pathophysiology, such as a heart failure exacerbation; extrinsic issues pertaining to care coordination breakdowns, such as a physician's office that never returned a phone call; and extrinsic issues pertaining to social and environmental factors, such as financial or food insecurity. (See Table 1.) The questions stem from the clinical expertise of the Care at Hand team, clinical advisors, internal data collected by the technology, and existing literature on care coordination and quality measurement.^{39, 40, 41, 42, 43, 44, 45, 46} The questions were designed to cater to the limited scope of practice of the nonmedical workers collecting data through the technology. The database of questions is vetted through a rigorous validation process including expert review by geriatricians and community nurses, psychometric evaluation among nonmedical workers, and field testing.

Table 1.

Survey Question Categories Used to Screen for Readmission Risk

	Extrinsic
Intrinsic	Care Coordination	Environmental
Worsening medical or surgical condition (chest pain, shortness of breath, etc.) Worsening mental or behavioral health problem (depression, noncompliance, etc.) Functional decline (needs help with more activities of daily living, worsening frailty, etc.)	Management of a specific condition Setting up primary care provider or specialist appointment Coordination issue remained unresolved (loop not closed) Skilled home care assessment, referral, or service Nonskilled home care assessment, referral, or service Behavioral health assessment, referral, or service Home safety assessment Other home and community services–based assessment, referral, or service Medications ordered and filled Medication reconciliation Ongoing medication management in the home (filling syringes, applying creams, etc.) Durable medical equipment ordered and filled Inadequate family or community support to help with function Patient or family education or health literacy	Financial insecurity (i.e., cannot afford basic necessities) Food insecurity (lack of access to high quality nutrition) Housing insecurity (risk of homelessness) Housing quality (bug or rodent infestations, elevator out, no heat, appliance not working, etc.) Violence or abuse Transportation (cannot get to appointments, etc.) Legal

The second component of the readmission risk score depends on the intensity of the nurse care manager's interpretation and response to the alerts. The technology captures the nurse's recommendations for subsequent care management and organizes them on the basis of the care team member assigned to perform the nurse recommendation. More than 100 interventions can be documented at the level of the patient, health coach, skilled nursing staff, nurse care manager, primary care physician, urgent or emergent services, and other home and community-based services (HCBS). The interventions include medical as well as nonmedical management, and they include a crosswalk with the HCBS taxonomy.⁴⁷ Each of these interventions add a unique value to the cumulative risk score. On the basis of clinician-informed proprietary algorithms, the Care at Hand system divides the ranges for alert and intervention-based risk scores into quartiles (baseline, mild, moderate, and high) to approximate 30-day readmission risk.

Statistical Analysis

We performed all analyses using Stata statistical software, release 11 (StataCorp; College Station, TX). We included the variables for readmission risk points derived from alerts and from nurse interpretation of alerts in a logistic regression against the binary variable, readmission within 30 days, 60, 90, and 120 days. We estimated odds ratios and 95 percent confidence intervals for readmission rates. We evaluated the resulting multivariable models using the area under the receiver operating characteristic curves (AUC) and evaluated the goodness-of-fit using the Hosmer-Lemeshow test. To compare the differences in readmission rates for risk score levels, we used a between-subjects analysis of variance (ANOVA) with 95 percent confidence intervals, and the post hoc comparison of readmission rates for different categories of risk scores was performed using paired t-tests.

Results

Overall Findings

The cohort included 2,027 unique patients with mean age of 73 years (interquartile range, 66–80). The majority of the patients were female (58 percent) and Caucasian (86 percent). The average Centers for Medicare and Medicaid Services Hierarchical Condition Categories (CMS-HCC) risk score for this cohort was 1.9. Of the patients approached to participate in the care transition program, 91 percent agreed to participate and 9 percent opted out. The cohort had 5,224 surveys performed, of which 1,202 (23.0 percent) generated alerts. (See Figure 2.) Of the 1,064 alerts with responses from a nurse care manager, 822 (77.3 percent) had no subsequent readmission within 30 days of the index hospitalization, whereas 242 (22.7 percent) had a subsequent readmission. The average monthly 30-day readmission rate for program participants during this QI project was 16.4 percent (range, 13.6–17.5 percent). The 30-day readmission rate for the three-month period preceding the start of the QI project was 17.6 percent; this rate decreased to 14.5 percent for the last month of data collection.

Figure 2.

Summary of Surveys, Alerts, and Readmissions

Predicting 30-Day Readmissions Using Observations of Nonmedical Workers

The majority of the alerts generated involved intrinsic issues (n = 661, 62 percent). (See Figure 3.) The second most common type of alert included both intrinsic issues and extrinsic care coordination breakdowns (n = 225, 21 percent). Only 12 percent of alerts (n = 123) exclusively addressed extrinsic care coordination breakdowns and only 2 percent of alerts (n = 17) simultaneously involved intrinsic issues and extrinsic environmental factors. A single alert was characterized by an extrinsic environmental issue. Thirty-seven alerts (3 percent) were classified as other.

Figure 3.

Distribution of Survey Question Categories Associated with Elevated-Risk Alerts

The risk scores for these alerts had a modest but significant association with the 30-day readmission rate with an odds ratio (OR) of 1.12 (95 percent confidence interval [CI], 1.09–1.15). The AUC for the model was 0.56 (95 percent CI, 0.54–0.58), and it satisfied the goodness-of-fit test. (See the green line in Figure 4.) When comparing 30-day readmission rates among alert-based risk score categories, a between-subjects ANOVA revealed a significant difference between the baseline, mild, and moderate risk categories (F = 24.66, p < .001). Subsequent t-tests revealed that the scores in the high-risk category had a significantly higher average 30-day readmission rate of 32 percent compared to patients in the baseline (t = 6.12, p < .001), mild (t = 3.56, p < .001), and moderate (t = 3.24, p = .001) risk categories with average readmission rates of 14 percent, 20 percent, and 19 percent, respectively. (See the green bars in Figure 5.)

Figure 4.

Receiver Operating Characteristic (ROC) Curve of Risk Scores and Readmission Risk

Figure 5.

Readmission Rate in Each Risk Quartile for Each Risk Scoring Approach with 95 Percent Confidence Intervals

Predicting 30-Day Readmissions Using Nurse Interpretation of Nonmedical Worker Observations

A nurse care manager performed the initial triage of alerts generated by the mobile technology. Multiple care team members could be involved in the care coordination and care management following an alert, and the type of care team member involved in care management was a significant factor in determining the intervention-based risk score. Of the 1,064 alerts triaged, 1,049 (98.6 percent) involved the nurse care manager communicating directly with another care team member. (See Figure 6.) The next most frequently involved care team member in response to alerts was the patient, accounting for 804 (75.6 percent) of the interventions. Next was the health coach, who was involved in 768 (72.2 percent) of the interventions. At just over half the frequency of coach involvement, skilled nursing was involved in 461 episodes (43.3 percent). The least frequently involved care team members were the outpatient physician and urgent or emergent care providers, who were involved in only 235 (22.1 percent) and 27 (2.5 percent) of the episodes, respectively.

Figure 6.

Distribution of Intervention Frequency by Care Team Member in Response to Alerts

Multiple care team members were usually involved in each episode of care in response to an alert. The most frequent combination of care team members involved the patient, health coach, and nurse care manger; this combination accounted for 207 episodes (19 percent). The second most frequent combination of care team members included the patient, health coach, nurse care manger, and skilled nursing, accounting for 185 episodes (17 percent). The third most frequent combination of care team members included the patient and nurse care manger, accounting for 113 episodes (11 percent).

The risk score derived from structured data capture of nursing interpretation and triage of alerts was significantly associated with the 30-day readmission rate with an OR of 1.25 (95 percent CI, 1.19–1.32) when alert-based risk points were included in the regression model and an OR of 1.18 (95 percent CI, 1.09–1.27) when alert-based risk points were excluded. The AUC for the model was 0.56 (95 percent CI, 0.54–0.58) and it satisfied the goodness-of-fit test. (See the orange line in Figure 4.) When comparing 30-day readmission rates among intervention-based risk score categories, a between-subjects ANOVA revealed a significant difference between the four categories (F = 22.67, p < .001) with subsequent t-tests showing that the high-risk category had a significantly higher average 30-day readmission rate of 33 percent compared to patients in the baseline (t = 5.19, p < .001), mild (t = 3.13, p = .002), and moderate (t = 2.60, p = .01) risk categories, with average readmission rates of 14 percent, 20 percent, and 22 percent, respectively. (See the orange bars in Figure 5.)

Validating Automatically Generated Risk Scores from Structured Data Capture of Nurse Documentation

To validate the nurse interpretation of alerts through structured data capture, we applied the same risk scoring algorithms to a manual review of unstructured nurse documentation in response to alerts.

Although less pronounced, the readmission risk score derived from RA review of unstructured nurse documentation was significantly associated with the 30-day readmission rate with an OR of 1.20 (95 percent CI, 1.15–1.26). The AUC for this model was comparable to the structured data capture model with an AUC of 0.56 (95 percent CI, 0.54–0.58), and it satisfied the goodness-of-fit test. (See the blue line in Figure 4.) When comparing 30-day readmission rates among risk score categories using unstructured data capture of interventions, a between-subjects ANOVA revealed a significant difference between the four categories (F = 21.95, p < .001), with subsequent t-tests showing that the high-risk category had a significantly higher average 30-day readmission rate of 30 percent compared to patients in the baseline (t = 5.04, p < .001) and mild (t = 3.03, p = .003) risk categories with average readmission rates of 14 percent and 18 percent, respectively. There was no significant difference in readmission rate between the high-risk category and moderate-risk category (t = 1.92, p = .06), with an average readmission rate of 23 percent. (See the blue bars in Figure 5.)

Predicting Readmissions beyond 30 Days after Discharge

Readmission risk prediction using mobile technology and nonmedical workers extends beyond 30 days to 60, 90, and 120 days after discharge. Based on the readmission rate in each predicted category for the intervention-based risk score from structured data capture, t-tests show that the patients who generated scores in the high-risk category had significantly higher readmission rates than those in the baseline-risk category at 60 (t = 4.68, p < .001), 90 (t = 4.33, p < .001) and 120 (t = 3.75, p < .001) days after discharge. Similarly, readmission rates of the high-risk group were significantly higher than those of the mild-risk group at 60 (t = 2.83, p < .01), 90 (t = 2.66, p < .01), and 120 (t = 2.36, p < .05) days after discharge. (See Figure 7.) The difference in the readmission rates between patients in the high-risk group compared to patients in the moderate-risk group at 60, 90, or 120 days after discharge was marginal or not significant.

Figure 7.

Readmission Rate in Each Risk Category for the Intervention-based Risk Score from Structured Data Capture with 95 Percent Confidence Intervals

Discussion

Although this preliminary approach showed only a modest effect size, our data suggest that observations made by nonmedical workers in the absence of nurse interpretation of alerts can predict 30-day readmissions. The ability of nonmedical workers to contribute to readmission risk prediction may be associated with extrinsic risk factors, which accounted for 35 percent of the elevated-risk alerts. Survey questions pertaining to extrinsic factors may require less clinical prowess to answer adequately, making these questions suitable for nonmedical workers to answer.

Another explanation for the predictive nature of nonmedical worker observations could be the way in which questions were worded. Because 85 percent of alerts involved intrinsic risk factors that traditionally fall under the broader scope of practice of a nurse, the predictive nature of the technology may be due to the accessibility of the jargon-free survey questions.

Regardless of the etiology of alert-based risk prediction, the multidomain readmission risk factors reinforce the important role of LTSS in medical services. For LTSS providers that are expanding their services to include a care transition intervention, our data suggest that nonmedical workers may be able to risk stratify patients in the absence of nurse supervision. However, 55 percent of the interventions in response to alerts involved skilled nursing or a physician, which suggests that delivery models using only nonmedical workers may be useful for risk prediction but may have an inadequate scope of practice to address all the needs identified by the elevated risk alerts.

A more balanced care transition intervention would include both nonmedical workers and nursing staff, as in the model described by Counsell et al.⁴⁸ Not only would the nursing staff be able to effectively triage alerts, but their input would also improve the predictive nature of the alerts. In particular, the OR for the intervention-based risk score using structured data capture and validated by unstructured data capture was significantly higher than the OR for the alert-based risk score. Furthermore, the OR for the intervention-based risk score using structured data improved when alert-based risk points were included in the regression model. These findings suggest that the predictive nature of the 30-day readmission risk scores is improved when both nurse and nonmedical worker inputs are used in concert, especially when nurse data are captured automatically.

Structured capture of nurse input can be applied beyond the 30-day period to also predict risk as far as 120 days after discharge, although the risk prediction may be limited to only detecting differences in readmission risk between highly elevated and mildly elevated or baseline-risk alerts. Several provisions of the Affordable Care Act (ACA), such as the bundled-payment program, offer incentives to avoid readmissions up 90 days after discharge. Other provisions of the ACA, such as the Medicare Shared Savings Program, create incentives to keep patients out of the hospital indefinitely.⁴⁹ At 120 days after discharge, the readmission risk score starts to have implications for general admission risk. One interesting area of further research would be the use of the technology as a preadmission screening or triage tool in emergency departments.

Although promising, our process for risk stratifying has its limitations. First, it is limited by a modest initial OR and AUC. We expect that as the technology is further refined, the performance measures will improve. Another limitation is the lack of differentiation between highly and moderately elevated risk alerts. Further research is needed to refine the predictive capacity of the technology to distinguish between risk levels. Despite the limitations, these data suggest that nonmedical workers could be considered for utilization in a care transition intervention with the assistance of a mobile risk prediction technology.

Conclusion

We met the objectives of this study by determining that (1) observations of nonmedical workers can be used to predict 30-day readmissions, (2) readmission risk prediction using observations of nonmedical workers can be improved by clinician oversight, and (3) observations of nonmedical workers can be used to predict readmissions beyond 30 days after discharge. This study aligns with several key national developments that support achievement of the Triple Aim for vulnerable populations.⁵⁰ First, the empowerment of the nonmedical workforce through mobile technology aligns with the aim of the Office of the National Coordinator for Health Information Technology to integrate LTSS with quality improvement through health information technology.⁵¹ Second, the identification of individuals' unique risk factors for readmission aligns with the president's Precision Medicine Initiative.⁵² Finally, if prediction of hospital readmissions using an existing low-cost asset such as the nonmedical workforce can help prevent readmissions, it offers an opportunity to increase the value of care being provided. More cost-effective care would align with the announcement by the Department of Health and Human Services to tie 50 percent of Medicare payments to value-based rather than volume-based reimbursement by the end of 2018.⁵³ This study highlights one approach to the use of technology that contributes to innovation in healthcare delivery and could help curb waste in healthcare spending.