An Evaluation of a Multisite, Health Systems–Based Direct Care Worker Retention Program: Key Findings and Recommendations

Short abstract

A mixed-methods evaluation of a program to help stem the loss of newly hired entry-level caregivers found no positive effect on retention or financial sustainability.

Abstract

The U.S. direct care workforce employs nearly 4.6 million people and represents one of the fastest growing occupations in the United States. Direct care workers, or “caregivers,” include nursing assistants, home care workers, and residential care aides, all of whom provide basic care to older adults and individuals with disabilities in various health care settings.

Despite a growing need for caregivers, supply has not kept up with demand due to high turnover and low wages. In addition, caregivers often face high levels of workplace stress, limited training and growth opportunities, and personal stressors. Ranging from 35 to 90 percent, depending on the health care setting, the turnover rates of direct care workers pose a major challenge for health systems, as well as care recipients and workers themselves.

In 2019, the Ralph C. Wilson Jr. Foundation funded three health systems to support the implementation of a new program: Transformational Healthcare Readiness through Innovative Vocational Education (THRIVE). This 12-month program was designed to help address barriers that entry-level caregivers experience and reduce turnover through a comprehensive risk assessment, training, and one-on-one coaching. Researchers from RAND conducted a process and outcome evaluation to determine whether THRIVE was meeting its goals of improving retention and achieving a positive return on investment (ROI). They also examined potential areas for program improvement.

The U.S. direct care workforce employs nearly 4.6 million people (PHI, 2021) and is one of the fastest growing occupations in the United States (U.S. Bureau of Labor Statistics, 2020). Direct care workers, referred to as “caregivers” in this article, include nursing assistants (NAs), home care workers, and residential care aides, all of whom provide basic care to older adults and individuals with disabilities in different health care settings (i.e., hospitals as well as long-term care and residential settings) (U.S. Bureau of Labor Statistics, 2020). Despite a growing need for caregivers and an increasing aging population, supply has not kept up with demand due to high turnover and low wages. In addition, caregivers often face high levels of workplace stress, limited training and growth opportunities, and personal stressors (Stone, 2004). As a result, turnover rates for direct care workers are not only quite high but also vary widely, ranging from roughly 35 percent for certified nursing assistants in hospitals (NSI Nursing Solutions, Inc., 2022) to 65 percent for home care workers (Holly, 2021) and even over 90 percent for caregivers in nursing homes (Gandhi, Yu, and Grabowski, 2020). The turnover rates of direct care workers pose a major challenge for health systems as well as care recipients and workers themselves.

In 2019, the Ralph C. Wilson Jr. Foundation funded three health systems (hereafter referred to as “sites”) to support the implementation of a new program: Transformational Healthcare Readiness through Innovative Vocational Education (THRIVE). This one-year program was designed to help address barriers to entry-level caregivers and reduce turnover through a comprehensive risk assessment, training, and one-on-one coaching. Although specific titles vary depending on the site, these entry-level caregivers include primarily NAs or those who provide basic care and assistance with activities of daily living (ADLs) to individuals in hospitals, as well as long-term and home care. The foundation and its health system partners selected the RAND Corporation (RAND) to conduct a process and outcome evaluation to determine whether THRIVE was meeting its goals of improving retention and achieving a positive return on investment (ROI) while also identifying potential areas for program improvement.

Approach

RAND utilized a mixed-methods approach for the evaluation, drawing from secondary sources, including health system administrative data and program data documented by THRIVE staff, and primary sources, including 50 interviews with THRIVE staff and 57 interviews with participating caregivers, a web-based survey of caregivers (n = 93), and 15 observations of training sessions. Administrative data were collected for Years 1 and 2 of the program—from June 2019 through May 2020 and June 2020 through May 2021, respectively—and a baseline year, referred to as Year 0, covering June 2018 to May 2019. The Ralph C. Wilson Jr. Foundation paused the program midway through Year 3 (i.e., December 2021) based on preliminary evaluation findings and feedback from sites, so data analyses reflect data collected primarily through Year 2 for administrative data and some of Year 3 for qualitative and other programmatic data.

Key Findings

The evaluation focused on five primary evaluation questions. This article details findings by evaluation question.

• Did retention rates improve during or after THRIVE implementation? Did this vary between voluntary and involuntary reasons for attrition?

  • Retention rates did not improve during THRIVE implementation (i.e., Years 1 and 2 of the program).

  • There were no significant differences in retention between Year 1 (accounting only for the nine months prior to the onset of the COVID-19 pandemic in the United States) and the same nine months in Year 0.

  • There were no identified improvements for either voluntary or involuntary reasons for attrition, and Year 2 had significantly lower voluntary retention rates compared with Year 0.

• How are program- or organizational-level factors associated with retention? How are these factors associated with short-, medium-, and long-term outcomes beyond retention?

  • Pay rate (or wage) and race emerged as significant predictors of retention in both program years; those in the lowest pay category and individuals who identified as black had a higher likelihood of termination (voluntary or involuntary) than those in the highest pay category or those who identified as white. Other predictors included part-time status, identification as “other” race, and gender for certain years or types of terminations.

  • Risk acuity level from the THRIVE risk assessment was not associated with retention.

  • Higher levels of burnout predicted caregiver intent to leave within the next six to 12 months while higher perceptions of THRIVE training were predictive of higher intent to leave.

  • Bring Back Days, six half-day curriculum-based sessions delivered throughout the year, appeared to be consistently associated with improvements in other outcomes including absenteeism, relationships with managers and other colleagues, improved commitment to the organization, and improved commitment to the caregiving field.

• What are the strengths of the THRIVE intervention, including the intervention content and modality and the skill and knowledge of those delivering the content, and what are areas for improvement?

  • Although originally designed to be in person only, THRIVE was implemented both in person and virtually with different degrees of synchronicity and levels of staff cross-training. This appears to be due to challenges around organizational staffing shortages, local requirements, and COVID-19.

  • Reported strengths of THRIVE included enthusiastic and passionate staff who provide work-related, practical, and emotional support to caregivers.

  • While there were positive and negative perceptions of the program among caregivers, there seemed to be an overall indifference to THRIVE among caregivers and a lack of understanding of what THRIVE is and the goals when beginning the program.

  • Reported areas for improvement included adjusting THRIVE activities to better target adult learners; using more respectful language; increasing empathy of THRIVE staff toward caregivers; clarifying roles and responsibilities of THRIVE staff; providing more support and engaging THRIVE staff to reduce feelings of burnout or feeling unheard; and improving program diversity in terms of demographics (e.g., race, age), additional skills, and backgrounds of THRIVE staff members.

• Can THRIVE break even or generate a positive ROI so that it is financially sustainable without philanthropic support?

  • During Years 1 and 2, THRIVE did not break even or generate a positive ROI. This was primarily due to no observed improvements in retention rates in Years 1 or 2 of the program.

  • The average total costs to deliver THRIVE per eligible caregiver was $1,772 in Year 1 and $2,263 in Year 2.

  • To break even, a retention rate of over 100 percent in a 12-month period would have been needed in Years 1 and 2, given program costs; in other words, it would not have been possible to break even, given the observed program average expenditures per caregiver.

• What lessons learned would other sites need to leverage when implementing an intervention such as THRIVE in the future?

  • From a planning and learning perspective, other sites should consider ensuring adequate planning time prior to program implementation; obtaining buy-in from leadership, managers, and other departments throughout implementation; and providing opportunities to learn about implementation from other sites.

  • In terms of implementing the program, RAND developed recommendations focused on clarifying THRIVE staff roles and ensuring diversity in backgrounds across THRIVE staff; establishing better program documentation and tracking systems; and ensuring technology is in place to support virtual or web-based options for the program.

Recommendations

RAND recommends potential actions in three different areas:

• Changes to THRIVE

  • Clearly articulate the goals to health system leadership, managers, and caregivers prior to THRIVE implementation.

  • Increase diversity of backgrounds, experiences, and demographics among those developing and delivering THRIVE.

  • Be mindful of the need for clear documentation of processes as well as program data.

• Alternative interventions to improve retention

  • Involve caregivers in the design of the intervention and throughout implementation to ensure user-centered design is embedded into the program and to help address concerns or considerations around equity for caregivers.

  • Consider alternative interventions that may be more effective for the specified goals (i.e., retention) such as pay rate increases.

  • Consider alternatives to a mandatory or “one-size-fits-all” approach, given the diverse backgrounds of caregivers throughout different health systems, and include a more explicit focus on equitable outcomes.

  • Consider the health systems’ readiness and level of supportive culture in determining appropriate interventions, including systemic issues that may be important factors related to retention.

• Future evaluation work

  • Use evaluability assessments prior to program implementation to reflect on the program theory and intended outcomes (including potential variations by job or individual characteristics), feasibility of data collection, and manage stakeholder expectations.

  • Incorporate a broader range of data collection participants to include caregivers who may not be active in THRIVE (e.g., a status of graduated or terminated) as well as caregivers’ managers.