Counselor turnover in substance use disorder treatment research: Observations from one multi-site trial

Abstract

Background:

Counselor workforce turnover is a critical area of concern for substance use disorder (SUD) treatment providers and researchers. To facilitate the adoption and implementation of innovative treatments, attention must be paid to how SUD treatment workforce issues affect the implementation of clinical effectiveness research. Multiple variables have been shown to relate to turnover, yet reasons that are specific to conducting research have not been systematically assessed.

Methods:

In a randomized clinical trial testing a sexual risk reduction counselor training intervention, sixty-nine counselors at four outpatient SUD treatment sites (two opioid treatment programs [OTP], two psychosocial) were enrolled and randomized to one of two training conditions (Standard vs Enhanced). Study counselor and agency turnover rates were calculated. Agency context and policies that impacted research participation were examined.

Results:

Study turnover rates for enrolled counselors were substantial, ranging from 33% to 74% over approximately a 2-year active study period. Study counselor turnover was significantly greater at outpatient psychosocial programs vs OTPs. Counselor turnover did not differ due to demographic or training condition assignment. Leaving agency employment was the most typical reason for study counselor turnover.

Conclusions:

This secondary analysis used data from a multi-site study with front-line counselors to provide a qualitative description of challenges faced when conducting effectiveness research in SUD treatment settings. That counselors may be both subjects and deliverers of the interventions studied in clinical trials, with implications for differential impact on study implementation, is highlighted. We offer suggestions for researchers seeking to implement effectiveness research in SUD clinical service settings.

Introduction

Counselor workforce turnover is a critical area of concern for substance use disorder (SUD) treatment providers and researchers.^1,2 To facilitate adoption and implementation of innovative treatments, past work focused on conducting efficacy/effectiveness trials in real-world settings.^3–7 For these efforts to be successful, attention must be paid to how SUD treatment counselor workforce issues impact clinical trials research.

Implementation science examines barriers and facilitators to use of evidence-based practices in organizations. One of the barriers in the behavioral health arena is staff turnover. Woltman et al.⁸ analyzed how staff turnover affected implementation outcomes of fidelity and penetration (degree to which an intervention is deployed to those eligible) of five evidence-based interventions in 52 mental health treatment agencies. Treatment team staffing was unstable across time: 5 teams had under 33% of their staff turnover, 19 had between 33% and 100% turnover, and 18 had over 100% turnover. Twenty-four-month turnover was a significant negative predictor of counselor fidelity to the interventions. Penetration of the interventions decreased over time, and 71% of teams reported that turnover affected their ability to implement the intervention. In a review of 13 randomized clinical treatment trials in community SUD treatment agencies, Baer et al.⁹ found that all 13 reported having to conduct two or more re-trainings of replacement counselors after the initial counselor cohort was trained. Similar findings have been reported in a variety of other trials and settings.^10–12

Multiple variables have been shown to relate to turnover in SUD agencies. These include organizational factors, such as openness to change,¹³ support of creativity, job autonomy and performance-based rewards,¹⁴ and quality of clinical supervision.¹⁵ Individual factors play additional roles and include burnout and job satisfaction,¹³ organizational and occupational commitment,¹⁵ perceived procedural or distributive justice,¹⁶ and turnover intentions.¹⁷

Research-specific factors may also influence counselor turnover. Although there are organizational benefits to counselors participating in research, there are also time- and energy-related costs. Research is not a typical part of counselors’ job and may bring unexpected rewards or demands.¹⁸ Counselors may not have a voice in leadership’s decision whether to partner with researchers.¹¹ Counselors may experience conflicting messages from leadership about priorities for their time, such as pressure to meet reimbursement- or clinically-related goals and pressure to use their time for research-related duties. In a study of 207 SUD treatment counselors in the National Drug Abuse Treatment Clinical Trials Network (CTN), counselor turnover intention was significantly greater when counselors believed their job demands had increased due to the research, and was significantly lower when they believed the research was leading to improvements for patients and organization.¹⁷

The purpose of this report is to provide a qualitative description of challenges related to effectiveness research in SUD treatment settings from a multi-site study with front-line counselors, and, based on these observations, offer suggestions for future trials.

Methods:

The “Being Safe in Treatment” study employed a nested 2X2 factorial repeated measures design in which SUD counselors were randomly assigned to receive Standard (2 hours) or Enhanced (10 hours plus monthly ongoing coaching) training in talking with patients about sexual risk and health.¹⁹ Volunteer patients of participating counselors completed a web-based survey about their sexual behavior and were then randomly assigned to either receive or not receive a personalized feedback report (PFR). Counselors of patients receiving PFRs also received a copy. Counselors met with a Research Coordinator and provided written consent to participate, prior to completing assessments at baseline, 1 week post-training and 3 months post-baseline. Counselor assessment results, and patient participation and outcomes are reported elsewhere. An independent research ethics committee reviewed and approved the study protocol.

Counselors and patients were nested within treatment program modality (outpatient opioid treatment program [OTP] treatment—sites A and C--and psychosocial outpatient—sites B and D) and within programs representing these modalities. This study focused on OTP and psychosocial outpatient treatment modalities because the counselor training was intended to enhance longer-term counselor-patient relationships rather than those occurring in shorter-term settings such as detoxification, inpatient, or residential treatment programs. All four sites were located in the Western United States. One psychosocial outpatient program (Site B) withdrew from the study due to counselor turnover, after counselors had been recruited and assessed. Site B’s withdrawal precluded participation in patient recruitment, but we report their counselor turnover data. The other psychosocial outpatient (site D) program included 4 locations. All programs were research-experienced, having participated in the CTN and other research.

Counselor Participants

Counselors were informed of the study through a letter from the program director, followed by a staff-meeting presentation. Those who showed interest met with the site Research Coordinator (RC) and completed informed consent. Eligible counselors had to be employed at the program, have no plans to leave the program in the next two years, have a patient caseload, and see patients for individual counseling. Initially, 46 counselors enrolled across four sites. At the three treatment programs that remained for patient recruitment, 22 counselors were enrolled 10 months later in a second round of recruitment. Following this round, one counselor was added at an OTP site (Site C). In total, 69 counselors were enrolled in the study; 5 left before completing baseline assessments or training, leaving an analysis sample of N = 64.

Measures

Counselor assessments consisted of two parts, questionnaires completed online, and a Standardized Patient Interview. Counselors received $30 for baseline, $35 for post-training, and $40 for 3-month assessment completion.

Counselors self-administered a web-based survey programmed in Qualtrics.²⁰ This included a brief demographic questionnaire that asked about age, gender, race and ethnicity, education level, chemical dependency counselor credentialing, and length of time as a counselor and working at the treatment program. Additional counselor measures not relevant to this secondary analysis will be reported elsewhere.

Turnover.

This report focuses on two outcomes: 1) study turnover, defined as the rate of study counselor turnover, and 2) organizational turnover, defined as the rate of overall counselor turnover at the agencies during the same period irrespective of study participation.

For each agency, study turnover was examined from when the first counselor was enrolled to the end of the 3-month follow-up window for all patient-participants — a 2.25-year period. Site B’s turnover rate was calculated using the 11 months they participated in the study before withdrawing. Study turnover was calculated as the ratio of the number of enrolled counselors who left the study (numerator) to the total number of counselors enrolled (denominator). “Left the study” included those who transferred to a different role that precluded continued study participation, left agency employment voluntarily or involuntarily, or withdrew study consent. Research staff asked for counselors’ primary reason for leaving. Completion of post-training and 3-month assessments did not factor into counselor turnover calculations; the study’s high rate of completion of counselorsmeasures (96.7% - 100% depending upon the measure) was independent of counselors remaining in their same roles for the purpose of recruiting their patients.

Organizational turnover is reported to provide a context against which to evaluate level of study turnover. Organizational turnover was defined as the total number of counselors who left the agency during the specified time, divided by the total number of counselor positions available at that time (either filled or unfilled) at the agency who provide direct patient care. Site staff suggested the number of counselor positions available, rather than total number of counselors at the agency, was a more accurate representation of their staffing numbers because it reflected ongoing efforts and intention to fill open positions. Therefore, we used that number in the analysis. We defined “left the agency” as moving to another program (e.g. geographic location) that was not participating in the study, voluntarily or involuntarily ending employment at the agency, or changing roles within the agency.

Agency Context Data.

Contextual, qualitative information relevant to study implementation was collected at the three sites that completed patient-participant recruitment. Site research staff’s perceptions of internal agency leadership decisions and external state and national influences that might have impacted the study were collected during periodic site calls.

Counselor Randomization

Counselors were randomized to training condition using a stratified, simple randomization approach.²¹ We first stratified counselors into type of service they provided (OTP or Psychosocial) and agency they worked in. Within each stratum, equal numbers of counselors were assigned to each of two counselor training conditions. To do this, we created a set of envelopes for each stratum. Condition assignment envelopes were randomly given to counselors after Standard training was completed.

Counselor Training

Study investigators provided in-person Enhanced and Standard training to all but one counselor at their treatment programs. Two investigators co-facilitated training for the first round of recruited counselors, while second-round training was conducted by one investigator to simplify logistics. Content and delivery method were otherwise unchanged between initial and replacement counselor training. If counselors missed a session, they viewed the slides and listened to an audio-recording independently, then had the opportunity to discuss the training with study investigators or the Site Coach. This “make-up” method was also used to train one counselor who was recruited after the second round of counselors. All counselors received Continuing Education certificates.

The Standard Training presented an overview of the study design, a rationale for the importance of addressing sexual behavior in SUD treatment, and an introduction to the PFR that Feedback-condition patients and their counselors would receive. The Enhanced Training presented four additional 2-hour sessions delivered once per week. Each enhanced session addressed a topic aimed at improving counselor skills in discussing sex with patients. Counselors received monthly coaching from a site coach during the period of patient recruitment and until the last patient completed a 6-month follow-up.

Results

Counselor characteristics

Table 1 displays demographic characteristics for counselors. Sites differed significantly only on education and years of direct patient care experience. There were no significant differences among sites on counselor gender, race, age, or credentialing status.

Table 1:

Counselor Characteristics (n = 64) At Time of Enrollment by Agency Site

	Opioid Treatment Program		Psychosocial Treatment		Difference
	Site A (n = 21)	Site C (n = 16)	Site B (n = 9)	Site D (n = 18)	Test statistic (df), p-value
Age					NS
Range	26 to 69	24 to 55	28 to 63	24 to 63
Mean (SD)	44.76 (15.45)	36.86 (8.99)	38.78 (11.94)	41.29 (12.74)
Gender					NS
Female	16 (76%)	12 (75%)	9 (100%)	13 (72%)
Male	5 (24%)	4 (25%)	0	5 (28%)
Race					NS
White	14 (67%)	14 (88%)	7 (78%)	16 (88%)
Black/African American	3 (14%)	0	0	1 (6%)
Multi-racial	2 (9.5%)	1 (6%)	1 (11%)	1 (6%)
Did not report	2 (9.5%)	1 (6%)	1 (11%)	0
Latino	1 (5%)	1 (6%)	1 (11%)	1 (6%)	NS
Education: Highest degree					X2(6) = 12.525, p = .051
Associates (AA)	9 (43%)	0	1 (11%)	2 (11%)
Bachelors (BA, BS)	5 (24%)	4 (25%)	3 (33%)	5 (28%)
Masters (MSW, MPH)	7 (33%)	10 (63%)	5 (56%)	11 (61%)
Other	0	2 (12%)	0	0
CDP or CADC*	16 (76%)	9 (56%)	7 (78%)	11 (61%)	NS
Direct patient care experience years
Mean (SD)	12.29 (13.30)a	5.89 (5.19)	2.33 (1.50)a	5.71 (5.64)	F(3,57) = 3.496, p = .021
Range	< 1 to 40	< 1 to 17	1 to 5	1 to 25
Mode	4	3	1	4
Years at current agency					NS
Mean (SD)	6.50 (9.87)	1.23 (1.54)	2.00 (1.31)	1.75 (2.96)
Range	< 1 to 29	< 1 to 4	< 1 to 4	< 1 to 10
Mode	1	< 1	1, 2, 3 same	< 1
Left study**	7 (33%)	8 (50%)	6 (67%)	12 (67%)	NS

^*CDP = Chemical Dependence Professional; CADC = Certified Alcohol and Drug Counselor

^aSite A > Site B, Tamhane test

^**Prior to study completion

Counselor Turnover

Study turnover and Organizational turnover (Table 2) cannot be statistically contrasted, as study turnover that involved a counselor leaving the organization was a part of overall organizational turnover. Both indicators of turnover varied across sites. Site A had 29% turnover in the study and 37% within the organization. Site B study (67%) and organizational (66%) rates were similar as were those in Site C (study 50%, organization 47%). Site D experienced greater (74%) turnover in the study, than in the organization (49%).

Table 2:

Organizational Turnover and Study Turnover, by Treatment Program Type

Tx Program Type	Time Period (months)	Organizational Turnover Rate (# who left/total # counselor positions)	Study Turnover Rate (# who left study/total # study counselors enrolled)
OTP
Site A	25	37% (11/30)	33% (7/21)
Site C	26	47% (14/30)	50% (8/16)
Psychosocial
Site B	11	66% (10/15)	67% (6/9)
Site D	26	49% (23/47)	74% (17/23*)
Average Turnover		48%	56%

^*5 of 23 Site D counselors left the study prior to Baseline assessment so were not counted in Table 1

We compared those who left the study to those who remained on demographic variables of age, years at the agency, years of direct patient care experience, race, gender, chemical dependency counseling certification, and education. No group differences were found.

Next, we compared study turnover rate as a function of training condition and type of program. Turnover did not differ between Standard (n=16 of 32, 50%) and Enhanced (n=14 of 29, 48%) training conditions (X² (2) = 2.975, p = NS). However, type of program (OTP vs psychosocial) was related to turnover. A significantly greater proportion of outpatient psychosocial treatment program counselors left the study (n = 18 of 27, 67%) compared to OTP counselors (n = 15 of 37, 41%) (X²(1) = 4.266, p = .039).

Reasons for Leaving the Study

Counselors left the study for various reasons (Table 3). From most to least common, the four reasons identified included leaving the agency voluntarily (e.g. job elsewhere), transfer to a different role within the agency that changed their ability to participate, withdrawal of consent to participate in the study for any reason (e.g. perceived burden of study participation), and termination from the agency.

Table 3:

Reasons for Counselor Attrition from Study

	Opioid Treatment Program		Psychosocial Treatment
Reason for leaving	Site A (n = 7)	Site C (n = 8)	Site B (n = 6)	Site D (n = 17)
Left agency voluntarily	5	5	5	10
Transferred to a different agency role	1	1	1	2
Withdrew consent to participate in study	0	0	0	5
Terminated from agency	1	2	0	0

Reports about organizational context

Site research staff provided anecdotal reports about programmatic shifts occurring during the study that may have contributed to accelerated turnover. These included increased productivity expectations, increases in caseload, and decreases in administrative time, all occurring in the context of the increased demand for treatment services.

Timing of study.

The study was operating amidst the Opioid epidemic, contributing to increased pressures to expand treatment. One site underwent accelerated growth, the overall census rising by 40% during the 6 months just prior to study start, as well as a new clinic added. The study also took place during a time of change in the health care landscape. Sites were negotiating integration of SUD treatment and mental health services, changing funding structures, and the cascade of new requirements which focused on treatment staff.

Reports of context changes within agencies.

Two sites experienced a programmatic change at the time counselor recruitment began. To increase counselor productivity, they implemented an incentive program measured by weekly completed billable hours that led to increased agency pressures on counselors to perform. The program was in place for the first six months of the study. After the initial six months, incentives for productivity were removed but the expectation for billable hours remained. In addition to increased productivity expectations, counselors at the time of recruitment were adjusting to a new requirement of collaborative documentation. Implementation of this new documentation method led to less time scheduled for counselors’ documentation and paperwork.

Impact of Study Turnover

Though this study lacked a direct variable measuring the impact of study turnover on implementation, several proximal indicators were evident. First, the loss of, on average, 48% of counselors necessitated a second round of counselor recruitment and training that was costly to schedule and implement. Based on the training literature and investigator experience, replacement counselor training was delivered using the same face-to-face methods as the initial training.⁹ While ensuring methodological consistency, it nevertheless consumed significant logistical, time, and budgetary resources for the study and trainers. Second, patients’ eligibility to enroll in the study’s patient arm was based on having a counselor enrolled in the study. Attrition of counselors therefore was directly related to sites’ patient recruitment capacity: fewer available study counselors meant fewer patients could be enrolled, which slowed study progress. Third, 14.8% (n = 71) of enrolled patient participants had counselors who left the agency or changed roles. Eleven percent of enrolled patient participants (n = 53) were re-assigned to remaining study counselors, if their caseloads had room. Although efforts were made to keep patients within the same counselor condition as their original counselor (Enhanced or Standard training), this was not possible for 64.1% (n = 34) of those re-assigned. Patient re-assignment therefore became a potential threat to internal validity that required tracking to monitor and examine during data analysis. Fourth, at Site B the loss of 67% of counselors in the first eleven months of the study, combined with the agency’s difficulty in hiring and retaining replacements, directly led to withdrawal from the study prior to recruiting any patients. Thus, significant resources were lost to the project.

Discussion

Counselor turnover affected implementation of a randomized clinical trial at SUD treatment sites, and highlighted the important distinction between two roles counselors may play in clinical trials: research participant versus an agent of intervention delivery. We recruited 69 counselors at two outpatient psychosocial and two medication-assisted SUD treatment programs. From counselor enrollment to the end of the 3-month follow-up window for patient-participants, 54% of counselors (N=37) left the study for a variety of reasons. Past findings show that this is a common and expected factor in clinical trials implementation.^8–10,22 Indeed, this report and past work^23–28 clearly suggest that behavioral intervention effectiveness trials must prioritize counselor and supervisor retention as highly as patient retention.

Our work extends past research by revealing specific and varied reasons for turnover which, importantly, may be anticipated ahead of time by researcher-agency collaboration. For example, a number of counselors in our study left the project due to a change in role (e.g., no longer seeing patients, or changing their assigned treatment modality from individual to group therapy). These counselors would be missed in a simple overall turnover rate because they remained with the agency, yet their role change effectively removed them from the study.

This report also highlights an important distinction between retaining counselors for completing research assessments versus for their study role. Research staff are charged with retaining participants through various means. In the BEST study, research staff efforts resulted in high rates of assessment completion for both patients and counselors. Research staff maintained contact with both participant groups regardless of whether patients left treatment or counselors changed roles or organizations. Still, they had little control when it came to losing counselor participants due to role/job change or withdrawal from the study. These departures created a significant re-training burden for the study and negatively affected the timeline. A high degree of cooperation among agency administrators, counselors, and the research team is necessary to minimize the impact of counselor turnover on clinical trials.

This study revealed important clinical trial implementation challenges regarding the contexts within which SUD treatment organizations exist. The opioid crisis placed pressure on treatment programs to expand quickly. Such rapid expansion applies a new set of stressors on the organization via additional staffing, shifting caseloads, increased volume of patients and new clinic locations. These stressors appeared to affect counselor turnover as well as overall efficiency of research operations. Increased demands for greater integration of mental health and substance use disorder treatment also led to changes in agency processes and documentation requirements, both of which may have contributed to burnout, turnover, or willingness to participate in new research activities. While the abovementioned examples may be specific to the time and place of this study, we can anticipate ongoing changes in health care and drug trends, and researchers should assess and plan for how larger contextual factors might interact with one’s study.

This study highlighted the importance of considering counselor turnover in study design, and its impact on statistical power, study momentum, training, and re-training. Ideally, study design should take into account chronic turnover²² and should assume a given level of turnover. Turnover may be reduced through incentives or increased agency buy-in, however it will never be eliminated. Researchers should consider planning discussions with agency administrators and supervisors to anticipate turnover, in its many forms, prior to study initiation. In particular, retention of treatment providers and competing priorities they face are key topics that could be addressed. This study also illuminated the benefits and drawbacks of prioritizing replacement counselor recruitment and training as highly as initial recruitment and training. In the BEST study we retained similar methods for initial and replacement counselors. Despite this, the magnitude of counselor attrition highlights the importance of Baer et al.’s⁹ findings on this issue. Re-training typically receives little or no attention during study planning, which Baer et al. argues is a methodological flaw in many clinical trials because such differences in training content and delivery may contribute to varying levels of counselor engagement with the research study. Instead, training for new hires should match the level of rigor of initial training, so fidelity to the study design remains intact. Nevertheless, its impact on study budget and resources must be considered.

Finally, this study illustrates the need to elevate counselor retention to the same level of importance as patient retention. Researchers might brainstorm with agency leaders about agency or external factors that could influence counselor retention, a financial incentive for study participation, “branding” of the study to increase its cache for those who participate, or other creative benefits for participating. Clinical trial study design and training typically devotes significant attention to recruitment—and critically, retention--of study participants,^23–28 yet parallel efforts for counselors are rarely present. Consequently, counselors may be retained from the perspective of completing research assessments (e.g., a counselor leaves the agency or changes roles but is willing to complete follow-up assessments), but this differs from meaningful retention wherein counselors continue to perform study duties (e.g. delivering an intervention to patients). Although researchers should exercise caution in over-incentivizing, there remains room for increased attention.

Limitations and scope.

This was a retrospective and observational study. Because we did not plan to assess counselor turnover in our parent clinical trial, we are limited to reporting observations over time and cannot draw causal conclusions. However, our results are similar enough to those reported in other studies that they lend credence to our conclusions. Second, we cannot determine what contribution counselor turnover makes to the validity of patient outcomes, because our study did not test this relationship. Instead, our qualitative report of experiences with counselor retention informs the level of difficulty researchers may experience in implementing a clinical trial as planned. Finally, we acknowledge that this paper focuses on counselor turnover and clinical trial effectiveness research in the specific setting of OTP and outpatient psychosocial SUD treatment agencies, not on all workplaces or organizations. Our findings may not generalize to other SUD treatment settings such as residential, detoxification, or inpatient.

Conclusions.

Researchers, insurers and public agencies are increasingly asking counselors and treatment organizations to mount complicated evidence-based practices, resulting in multiple sources of performance pressure and competing priorities. Effectiveness research is an indispensable step in moving an innovation into practice. It is incumbent upon researchers to understand and articulate the challenges in integrating behavioral research into daily provision of care and apply those lessons productively going forward.