Research participation and turnover intention: An exploratory analysis of substance abuse counselors

Abstract

Clinical research is increasingly being conducted in community-based addiction treatment settings. While the primary focus of such research is on the development of effective clinical interventions, less attention has been paid to the potential impact of these projects on counseling staff who are involved in their implementation. Such involvement may be perceived as stressful or rewarding, and these perceptions may be associated with counselors’ turnover intention. Using data from 207 counselors involved in research projects conducted within the National Institute on Drug Abuse’s Clinical Trials Network (CTN), this study examines the associations between counselors’ reactions to research experiences and turnover intention. When counselors perceived that research projects resulted in organizational benefits, turnover intention was significantly lower. However, there was a positive association between perceptions of research-related stressors and turnover intention. These findings suggest that the impact of clinical trials on treatment organizations and staff members warrants continued study.

1. Introduction

Since the publication of the Institute of Medicine’s report, Bridging the Gap between Practice and Research: Forging Partnerships with Community-Based Drug and Alcohol Treatment (Lamb, Greenlick, & McCarty, 1998), considerable resources in the drug abuse specialty treatment field have been devoted to increasing the amount of clinical research conducted in real-world treatment settings. A major mechanism for this type of research has been the National Institute on Drug Abuse’s Clinical Trials Network (CTN), which was designed to test promising interventions in multi-site clinical trials and to promote the transfer of evidence-based treatment techniques into routine practice (NIDA, 2006; Hanson, Leshner, & Tai, 2002). Selection, design, and implementation of these clinical trials relies on a collaborative process between researchers and clinical providers (Reback, Cohen, Freese, & Shoptaw, 2002; Saxon & McCarty, 2005). In contrast to efficacy studies conducted in traditional, laboratory-based research settings, these trials are largely staffed by community treatment centers’ existing counseling staff (Carroll et al., 2002; Miller, Bogenschutz, & Villarreal, 2006). Thus far, the CTN has completed multi-site clinical trials of medications as well as psycho-social interventions (Amass et al., 2004; Carroll et al., 2006; Ling et al., 2005; Peirce et al., 2006; Petry et al., 2005).

Although the primary goal of research conducted within the CTN (as in clinical trials generally), is to improve the quality of treatment, little is known about the reactions of the clinical staff who are actively involved in implementing the trials. Such concerns are highlighted in a recent call for recognizing the human subject protection issues related to staff involvement in research (Hilton, 2006). While there is an emerging literature on counselors’ attitudes toward various treatment techniques (Arfken, Agius, Dickson, Anderson, & Hegedus, 2005; Ball et al., 2002; Knudsen, Ducharme, Roman, & Link, 2005; McGovern, Fox, Xie, & Drake, 2004; Thomas, Wallack, Lee, McCarty, and Swift, 2003) and their beliefs about how to ethically conduct research (Forman et al., 2002), there is virtually no research about whether involvement in clinical trials or similar activities is associated with counselors’ attitudes towards their jobs. As more research is conducted in community-based treatment facilities, both via the CTN and other federally-funded research projects (Guydish, Sorensen, Rawson, & Zweben, 2003), the issue of how the clinical research experience affects staff members will only increase in significance for program managers.

An overriding management issue is that the high rate of counselor turnover faced by many substance abuse treatment centers is costly and potentially threatens the quality of care received by the center’s clients (Barak, Nissly, & Levin, 2001; Lum, Kervin, Clark, Reid, & Sirola, 1998; McLellan, Carise, & Kleber, 2003). Given that turnover intention is the strongest work-related predictor of actual turnover (Griffeth, Hom and Gaertner 2000; Tekleab, Takeuchi, & Taylor, 2005), understanding the potential linkages between clinical trial research participation and turnover intention is of practical significance. This article examines the experiences of 207 counselors who have been actively involved in the CTN’s research activities. We specifically consider whether the quality of counselors’ experiences in research implementation is associated with turnover intention.

Conventional models of job stress can be readily applied to an examination of the implications of research involvement for counselors’ job-related affect. There is a long tradition of research documenting how job demands and rewards are associated with attitudinal outcomes, such as turnover intention (Barling, Kelloway, & Frone, 2005). Given that research is not a normal part of their job descriptions or training, counselors involved in clinical research activities may face both unexpected demands and rewards, particularly when research studies require substantial changes in how counselors perform their jobs (Rawson & Branch, 2002).

Polcin (2004) notes that research projects bring with them a range of new requirements for the clinicians who participate in study implementation. For example, clinicians may be required to use new intake and assessment measures, which may increase the amount of time spent on these activities (Reback et al., 2002). More specifically, psycho-social intervention trials often rely on manualized therapies, which may introduce not only new techniques but specific structured strategies that involve a substantially different way of conducting sessions with clients. Requirements regarding high-quality data collection may also introduce new administrative demands that may vary considerably from counselors’ routine experiences with paperwork. Within a treatment center, there is also the potential for interpersonal conflict between counselors involved in the trial and those who are not, particularly in terms of tensions over perceived pseudo-prestige ascribed to those implementing the research protocols (Guydish, et al., 2005). In addition, counselors may find themselves faced with clinical trials that are having significant implementation problems, such as clients not being recruited quickly enough or high rates of study dropout.

However, it is also possible that counselors involved in clinical research activities may experience new and unexpected rewards from their participation. For example, clinicians involved in implementing a new therapy may see noticeable improvements among their clients, which may boost job satisfaction (Kellogg et al., 2005). Learning new techniques may enhance their occupational self-efficacy as well as break up long-term routines of treatment delivery. Counselors may be able to perform their jobs more efficiently and effectively, which may also improve morale (Carise, Cornely, & Gurel, 2002). They may also develop a special sense of camaraderie with other clinicians affiliated with the research. These counselors may also feel that they will be able to train other counselors in the technique once the trial ends. Thus, involvement in clinical research may present counselors with new challenges and burdens, but it may also involve them in activities that are highly rewarding.

This article has three aims. First, it examines the factor structure and validity of a set of new measures of counselors’ experiences with clinical trials. Second, it describes the experiences of counselors involved in CTN-related research in terms of these measures. Finally, it estimates the associations between perceived quality of research experiences and turnover intention, while controlling for counselors’ socio-demographic characteristics.

2. Materials and Methods

2.1. Sample

Between late 2002 and mid 2004, all of the community-based treatment programs (CTPs) affiliated with NIDA’s Clinical Trials Network were invited to participate in a health services research project examining the organizational factors associated with the adoption of evidencebased treatment practices. At the time of data collection, the CTN consisted of 17 research “nodes.” Each node was comprised of at least one university-based research center and several CTPs. In total, the CTN included 109 distinct treatment provider organizations; embedded within these organizations were 262 treatment centers. For the purpose of this study, a treatment center was defined as an organizational unit having an autonomous administrator with discretionary control over the unit’s budget. In most CTPs, these centers were equivalent to service lines; for example, one CTP consist of distinct cost centers devoted to adolescent services, methadone maintenance services, and adult outpatient services. The definition used in this study does not necessarily translate into either a “service delivery unit” or a physical location. The purpose of this definition was to consider all of the resources available to staff and clients within a distinct treatment center.

To be eligible for inclusion in this study, CTPs were required to provide a minimum level of care at least equivalent to ASAM-defined outpatient services (Mee-Lee, Gartner, Miller, Shulman, & Wilford, 1996) or methadone maintenance treatment (MMT). CTN-affiliated units that were dedicated to prevention/education/outreach services, correctional services, or assessment services were not included in the sample as these programs were unlikely to have direct involvement in CTN-related research protocols. During the study period, administrators of 240 centers (embedded within 104 larger CTPs) were interviewed, representing a response rate of 91.6% of all eligible centers. Face-to-face interviews were conducted with center administrators and/or clinical directors in order to collect information regarding organizational structure and service delivery.

Upon completion of the face-to-face interview, center administrators were asked to provide a list of counselors who provide direct services to substance abuse clients at the center. All listed counselors were mailed a paper-and-pencil questionnaire and an informed consent form. Counselors who completed and returned the questionnaire received a $40 incentive payment. These counselor-level data are the focus of these analyses. A total of 1,643 questionnaires were mailed and 1,001 were received (60.9%). These rates of participation are similar to other recent studies of addiction treatment professionals (Forman, Bovasso, & Woody, 2001; Knudsen et al., 2005; Mark, Kranzler, & Song, 2003; Thomas et al., 2003).

To assess response bias in the returned questionnaires, aggregated data on responding counselors were compared to aggregated data on the characteristics of counseling staff obtained from center administrators. Notably, the characteristics of responding counselors were similar to the centerlevel staffing characteristics, suggesting that response bias was not a significant problem in the resulting data set. On average, program administrators reported that 61.2% of the counselors employed in their centers were women, 36.1% were racial/ethnic minorities, 43.1% held a Master’s-level degree or higher, 43.6% were certified in addictions counseling, and 43.1% were in recovery from substance abuse. Among counselors responding to the mail questionnaires, 62.3% were women, 36.7% were minorities, 48.9% held a Master’s level degree, 45.7% were certified, and 41.2% were in recovery.

The following analyses focus on a sub-sample of counselors who reported being involved in the operation of a CTN-related clinical trial and provided complete data on all measures. These 207 counselors (20.7% of respondents) worked in 69 of the 104 larger treatment organizations affiliated with the CTN. This large reduction in sample size was not unexpected given reports by program leaders about the number of clinical staff actually involved in the day-to-day implementation of the protocols. These leaders indicated that on average, only about 4.34 clinical staff members were directly involved in the protocol at their center (SD = 3.90); this represents about 22.6% of the average center’s clinical staff, as measured by its number of counselors, nurses, and physicians employed by the center (mean number of clinical staff = 19.24, SD = 21.12).

Chi-square tests and t-tests were used to compare the 207 counselors involved in clinical trials with the counselors not involved in CTN research activities. There was no significant difference in turnover intention between research-involved and non-involved counselors. There were no significant differences between these two groups of counselors with regard to racial/minority status, educational attainment, personal recovery status, or age. CTN counselors involved in clinical trials were significantly less likely to be female (54.6% of trial-involved counselors vs. 63.2% of counselors not involved in the trials; χ² = 5.14, df = 1, p<.05). In addition, counselors involved in clinical trials (54.6%) were significantly more likely to be certified in addictions counseling than non-involved counselors (44.4%; χ² = 6.77, df = 1, p<.01). Research-involved counselors’ annual earnings were about $1,774 greater than non-involved counselors (t = −2.03, df = 995, p<.05). These latter two findings likely indicate that treatment organizations select more experienced counselors to implement clinical research protocols; this line of reasoning is supported by additional data (not shown) that revealed significantly longer tenure among research-involved counselors.

Finally, we used chi-square tests and t-tests to compare the 207 counselors who provided complete data with those counselors reporting research involvement who were excluded due to missing data (n = 39). These two groups did not differ with regard to turnover intention or the three dimensions of research experience. There were two significant differences in demographic characteristics between these two groups of counselors. First, counselors providing complete data were more likely to be in recovery (44.4%) than those excluded due to missing data (24.3%; χ² = 5.24, df = 1, p<.05). The other difference was that included counselors were more likely to be certified (54.6%) than excluded counselors (35.1%; χ² = 4.76, df = 1, p<.05).

2.2. Measures

The questionnaires contained numerous items about counselors’ jobs, caseloads, work experience, and CTN involvement. The principal independent variables for these analyses relate to counselors’ experiences with implementing the study protocols. Counselors were asked if they had personally been involved in the implementation or operation of a CTN clinical trial; if so, they were asked to respond to a series of items about their experiences with that clinical trial. The wording of these items appears in Table 1. Each item was measured on a six-point Likert scale (0 = no extent, 5 = very great extent).

Table 1.

Factor Analysis of Counselor’s Self-Reported Involvement in Clinical Research (n = 207)

“With 0 representing no extent and 5 representing a very great extent, to what extent has the implementation of this clinical trial…”	Factor 1	Factor 2	Factor 3
…generated administrative burdens for you?	.763	.189	.008
…enhanced the efficiency of your job performance?	.146	.749	−.148
…generated conflict between staff members affiliated with the trial and other staff members who are not involved in the trial?	.406	.188	−.508
…generated conflict between staff members who are affiliated with the trial?	.409	.135	−.599
…built up a spirit of teamwork among staff members affiliated with the trial?	.071	.775	.009
…created a “core” of clinical staff members that will train other non-trial involved staff in the components of the protocol after the trial is completed?	.155	.727	−.189
…resulted in the loss (turnover) of clinical staff?	.168	.106	−.733
…resulted in a decline in the quality of care received by clients enrolled in the protocol?	.119	.129	−.911
…resulted in lower rates of client retention over the course of treatment?	.157	.112	−.847
…resulted in higher rates of client retention over the course of treatment?	.097	.737	−.196
…have the regulations associated with the trial added to your workload?	.824	.187	−.101
…have the regulations associated with the trial increased emotional stress for you?	.828	.186	−.171
…do the eligibility criteria of the study make client enrollment difficult to achieve?	.711	.045	−.263
…is client dropout a significant problem with this protocol?	.575	−.031	−.421
…is it difficult to adhere to the requirements of the protocol?	.732	−.099	−.423

The dependent variable of interest was turnover intention. It was measured by three items that were adapted from Walsh, Ashford, and Hill (1985). These items asked counselors if they were seriously thinking about quitting their jobs, actively searching for other jobs, and planning to quit once they found a better job. Counselors indicated their agreement using a Likert response format where 1 represented “strongly disagree” and 7 represented “strongly agree.” The Cronbach’s alpha reliability for this mean scale was .87.

Seven socio-demographic characteristics were included as control variables in the models. These measures were gender (1 = female, 0 = male), racial/ethnic minority background (1 = minority, 0 = white), age in years, educational attainment (1 = Master’s level degree or higher, 0 = less than Master’s level degree), certification in addictions counseling (1 = certified, 0 = not certified), personal recovery status (1 = in recovery from substance abuse, 0 = not in recovery), and earnings. The measure of earnings consisted of nine categories (less than $15,000, $15,000-$20,000, $20,000-$25,000 etc.). Each category was recoded to its midpoint, and Parker and Fenwick’s (1983) method for estimating the value of the final open-ended category (greater than $50,000) was used for analytic purposes. The resulting earnings measure is expressed in thousands of dollars.

3. Results

3.1. Factor Analysis of Counselor Involvement in Clinical Research

Table 1 presents the results from a factor analysis of the items describing counselors’ involvement in CTN clinical trials, which used principal component analysis with varimax rotation. The factor analysis yielded a three-factor solution. The first factor consists of five items measuring research-related stressors (α = .87). The second factor consisted of four items related to organizational benefits yielded by the implementation of the trial (α = .78). In the third factor, three items measure the extent to which the trial had resulted in organizational costs (α = .86) in terms of increased staff turnover, declining quality of patient care, and reductions in client retention. Three additional items cross-loaded, and therefore, were not included in the mean scales that were calculated for the ordinary least squares (OLS) regression analysis.

3.2. Descriptive Statistics

Descriptive statistics for all variables appear in Table 2. The three mean scales of involvement in clinical research each ranged from 0 to 5. While counselors perceived that the clinical trials had yielded organizational benefits to some extent (mean = 2.10, SD = 1.24), involvement in the trials was also a source of stress for some counselors (mean = 1.75, SD = 1.27). Counselors reported very low levels of perceived organizational costs resulting from the clinical trials being conducted within their treatment centers (mean = 0.78, SD = 1.06).

Table 2.

Descriptive Statistics of CTN-Affiliated Counselors Involved in Clinical Research (n = 207)

	% (N)	Mean (SD)
Counselor Characteristics
Gender
Female	54.6% (113)
Male	45.1% (94)
Racial/Ethnic Minority
Minority	38.2% (79)
White	61.8% (128)
Age in Years		44.32 (10.83)
Educational Attainment
Master’s Degree or Higher	47.8% (99)
Less than Master’s Degree	52.2% (108)
Certification in Addictions Counseling
Certified	54.6% (113)
Not Certified	45.4% (94)
Personal Recovery Status
In Recovery	44.4% (92)
Not in Recovery	55.6% (115)
Earnings (in thousands)		32.76 (10.66)
Quality of Experiences with Clinical Research
Research-Related Stressors		1.75 (1.27)
Organizational Benefits		2.10 (1.24)
Organizational Costs		0.78 (1.06)
Turnover Intention		2.80 (1.85)

3.3. OLS Regression Model of Turnover Intention

Table 3 presents the results of the ordinary least squares (OLS) regression model of turnover intention on counselor involvement in clinical trials, net of the socio-demographic control variables. Two of the three scales measuring research experiences were significantly associated with turnover intention. First, there was a negative relationship between organizational benefits and turnover intention (β = −.24, p<.01). Counselors who more strongly perceived that the trial was yielding organizational benefits were less likely to express that they were intending to quit their jobs. However, the degree to which counselors perceived additional stressors due to implementing the trial was positively associated with turnover intention (β = .22, p<.01). The association between perceived organizational costs and turnover intention approached, but did not achieve, statistical significance (β = .13, p=.09).

Table 3.

OLS Regression of Turnover Intention on Counselor Involvement in Clinical Research (n = 207)

	b (SE)	β
Counselor Characteristics
Female (vs. Male)	.054 (.253)	.015
Racial/Ethnic Minority (vs. White)	−.088 (.284)	−.023
Age in Years	−.037 (.013)	−.218**
Master’s Degree or Higher (vs. < Master’s)	.144 (.290)	.039
Certified in Addictions Counseling (vs. Non- Certified)	−.171 (.269)	−.046
Personally In Recovery (vs. Non-Recovering)	.026 (.301)	.007
Earnings	−.022 (.012)	−.125
Quality of Experiences with Clinical Research
Research-Related Stressors	.314 (.113)	.216**
Organizational Benefits	−.357 (.106)	−.239**
Organizational Costs	.230 (.135)	.132
Adjusted R2		.142

^*p<.05

^**p<.01 (two-tailed tests)

Of the control variables, the only significant association was between age and turnover intention. Older counselors reported significantly lower turnover intention than younger counselors (β = −.22, p<.01). The association between earnings and turnover intention approached, but did not achieve, statistical significance (β = −.13, p = .08).

4. Discussion

With an increasing amount of clinical research being conducted in community-based treatment organizations, it is important to consider if staff members are impacted by their involvement in implementing clinical trials. To date, there have been no studies examining whether the quality of counselors’ experiences with research are associated with attitudinal outcomes such as turnover intention. Using data from 207 counselors involved in NIDA’s Clinical Trials Network (CTN), we found that counselor perceptions about these research activities were associated with turnover intention. Turnover intention was significantly greater when counselors perceived that their job demands had increased due to the research. However, turnover intention was significantly lower if counselors perceived that the research was resulting in improvements for clients and the organization.

These results represent an exploratory examination of how experiences with implementing clinical trials may be meaningful for counselors as they consider whether to remain or depart from their employing treatment center. However, there are numerous questions that remain unanswered. First, this research piloted a new instrument for measuring counselors’ experiences with clinical research. Additional research is needed to further validate these measures and establish the stability of the three factors identified using these data. Future research should consider other attitudinal outcomes, such as job satisfaction and organizational commitment, which are more proximate measures of job-related affect and may also influence decision-making about turnover. Finally, panel longitudinal data are necessary to examine whether these research experiences and turnover intention are associated with actual turnover among counselors. These are important avenues for future research.

There are several limitations that must be noted. First, the analysis is based on cross-sectional data, which limits the ability to establish causality. Although turnover intention is consistently associated with actual turnover in the literature (Griffeth et al., 2000), panel longitudinal data is needed in order to establish the associations between research involvement, turnover intention, and actual turnover.

Second, the sample size is rather small, which reflects the limited number of trials that had been conducted by the CTN at the time of data collection and the relatively small number of counselors within a given treatment center that were involved in those protocols. The sample size was further reduced by limiting the analysis to counselors who provided complete data, and there were significant differences between included and excluded counselors in terms of recovery status and certification status. However, neither of these characteristics were associated with turnover intention, which is consistent with other published work on turnover intention among addiction counselors (Knudsen, Ducharme, & Roman, 2006; Knudsen, Johnson, & Roman, 2003). We are currently collecting another round of questionnaire data from CTN-affiliated counselors, which will allow us to determine whether these findings can be replicated.

An additional limitation is that respondents were not asked to identify the type of clinical trial in which they had participated. It is not known if counselors’ experiences – and their impacts on work-related affect – differ based on the type of intervention under investigation. For example, there may be differences in counselors’ experiences with medication trials (in which they less integrally involved) versus trials of psycho-social interventions (in which their job tasks are directly and substantially altered). Future research should consider whether there are measurable differences in counselors’ evaluation of their research experiences based on the type of treatment intervention being studied.

Finally, there are some organizational differences between treatment centers affiliated with the CTN and those outside this research network (Ducharme, Knudsen, Roman, & Johnson, in press). It is not known how these findings might generalize to research being conducted in clinical settings outside the CTN. However, these dimensions of research experiences are not unique to the CTN; they reflect of the challenges of the clinical research endeavor, particularly when such activities are conducted in real-world addiction treatment settings. Future research should consider if research involvement is associated with turnover intention in other research contexts.

These findings provide some support for Hilton’s (2006) argument that researchers may need to consider whether clinical staff involved in research should be covered by human subjects protections. Although counselors generally reported modest levels of stress due the implementation of the clinical trial, this facet of research involvement is linked with greater turnover intention. There may be other negative affective responses among counselors when they are involved in research that is not going well. The use of alternative and possibly more proximate outcomes such as burnout, job satisfaction, and organizational commitment would help elucidate the extent to which research participation might adversely impact counselors. However, these data also suggest that turnover intention is lower when research involvement is perceived to yield organizational benefits, such as more effective job performance and improved treatment response among clients. Further investigation is clearly warranted, but these exploratory analyses suggest that both researchers and treatment programs interested in conducting clinical research in “real world” settings need to consider not only the potential benefit for the program’s clients, but also the potential impact on the program’s clinical staff.