Adoption of Evidence-Based Practices Among Substance Abuse Treatment Providers

Abstract

This research was conducted at a Substance Abuse Forum designed to address local community needs by focusing on Evidence-Based Practices (EBPs) in addiction treatment. The purpose of the study was to assess substance abuse treatment professionals’ readiness to adopt EBPs, experience with EBPs, and attitudes toward EBPs, as well as agency support for EBPs. A total of 119 addiction treatment providers completed pre-test measures, and 82% completed a post-test. Eighty-three percent of participants reported using some EBPs in the past year, and 75% reported currently using EBPs. Participants who were currently licensed or certified in addictions had less negative attitudes toward EBPs than those without credentials. While respondents reported agency support for EBPs, most expressed interest in further training. This study underscores the movement toward EBPs in addiction treatment and the need for effective dissemination and training in this area.

Accountability, constrained budgets and a growing demand for more effective services highlight the need for Evidence-Based Practices (EBPs) in addiction treatment. EBPs integrate the best available research with clinical expertise, taking into account patient characteristics, culture, and preferences (Levant, 2005). Effective substance abuse treatments are available, but difficult to disseminate into routine clinical settings (Ball et al., 2002; McGovern & Carroll, 2003). Few studies have systematically tracked how EBPs are transferred to the field (McGovern & Carroll, 2003), and community addiction providers’ experiences, beliefs, and readiness to adopt EBPs is largely unknown.

Current research suggests that community treatment providers’ experience with EBPs is limited (Ball et al., 2002), and when there is some exposure to EBPs, certain EBPs (e.g., Motivational Interviewing, Twelve-Step Facilitation) are more accepted than others (e.g., Contingency Management, addiction-specific medications) (McGovern et al., 2004). Lack of basic knowledge and skills required to assimilate EBPs into daily practice is often cited as a barrier to dissemination and implementation (Corrigan et al., 2001). Organizational characteristics and dynamics may also impact clinician adoption of EBPs (Gotham, 2004).

These findings are consistent with a survey of a large national sample of licensed psychologists regarding their attitudes and beliefs about the role of treatment manuals in clinical practice (Addis & Krasnow, 2000). Thirty-seven percent of the sample did not have a clear idea of what a treatment manual was, and 47% reported never using a treatment manual. In addition, some participants held negative views toward treatment manuals, expressing concerns regarding the constraints treatment manuals place on client-therapist relationships and actual impact on improved treatment outcomes. In order to disseminate EBPs effectively, it is useful to identify which treatments clinicians are likely to accept, and the attitudes they have toward EBPs.

The San Francisco Treatment Research Center (SFTRC) conducts dissemination activities, including community forums designed to address the needs of the local community. The topic of evidenced-based practices was selected based on Federal, state, and community treatment provider interest and expressed needs. Specifically, treatment program directors and clinicians reported their desire for training in a number of innovative EBPs in the substance abuse treatment field. Our focus on a local community and EBPs has implications at the national and state levels because of current initiatives to disseminate EBPs and a growing interest in the widespread implementation of EBPs in diverse settings. The purpose of our research was: 1) to assess experience with EBPs by substance abuse treatment professionals, 2) to examine clinician attitudes regarding use of EBPs, and 3) to investigate agency support for EBPs.

Method

Participants

Participants included a convenience sample of 119 drug abuse treatment counselors, clinical staff and executive directors of substance abuse treatment programs in the Bay Area (i.e., counselors, social workers, psychologists and therapists). A wide range of program types was represented, including inpatient, outpatient, residential, therapeutic community, methadone, detoxification, aftercare, as well as programs with a specific focus such as dual diagnosis, mental health, women, youths, family, ethnic minorities (e.g., Latino) and HIV/AIDS. All participants attended a daylong conference in May 2006 entitled, “Treatments that Work: A Substance Abuse Forum. Since the conference addressed clinicians who were willing to obtain additional training and potentially motivated to make changes, we believe it is meaningful to study this group as they may represent dissemination leaders in the community and change agents within their respective organizations. The University of California, San Francisco Institutional Review Board approved study methods.

Participants received a packet in the morning when they registered for the conference, containing both pre- and post-test measures. We told participants that the purpose of the questionnaire was to assess baseline knowledge of EBPs. Conference staff collected the pre-test questionnaires immediately before the actual conference events commenced, and collected the post-test questionnaires after participants attended workshops on specific EBPs. Participants who completed both pre- and post-test questionnaires were eligible for a lottery drawing at the end of the conference for one of six gift certificates to local merchants worth $25.

Conference Content

A team of researchers and practitioners in the field of substance abuse planned the community forum. The committee selected topics based on community treatment provider interest expressed by clinicians participating in the NIDA Clinical Trials Network (CTN) as well as a survey of executive directors from six local substance abuse treatment programs about their needs regarding staff training. The conference was advertised to potential attendees through a mailed brochure, flyers, and electronic mail. Participants could register by mail or online and were offered continuing education credit in their respective discipline for conference attendance. The purpose of the conference was to provide introductory training and resources (e.g., manuals, trainers, consultation, websites) on a broad range of topics to clinicians in the field of substance abuse treatment. The forum also offered clinicians an opportunity for in-depth exploration of one of four specialized clinical interventions with an expert trainer.

Measures

Study measures included the Pacific Southwest Addiction Technology Transfer Center (ATTC) National Pre-Event Training Form and Post-Event Training Form. The Pre-Event Training Form covered demographic information, education, work setting, responsibilities, and training goals. The Post-Event Training Form included satisfaction with the quality of the training and materials, and level of skills in the topic area.

We created an additional 20-item post-test survey, the SFTRC Course Evaluation, which defined EBPs as “treatments with a clearly defined intervention guide (i.e., manual), which have been scientifically tested in a formal research study and shown to improve outcomes.” This instrument assessed clinician stage of change for using EBPs, clinician experience with specific EBPs, clinician attitudes regarding EBPs, organizational barriers to adopting EBPs, and agency management strategies to support EBPs.

The clinician stage of change items used a categorical classification system, previously employed in the assessment of stage of change for quitting smoking (DiClemente et al., 1991) and the adoption of dietary behaviors (Ling & Horwath, 2000). The clinician experience item and agency management strategies item were checklists the authors devised based on EBP literature. We adapted 6 attitude items (i.e., 3 Positive Outcome and 3 Negative Process) from a study of psychotherapy treatment manuals (Addis & Krasnow, 2000), and we also created 6 items on organizational barriers to adoption of EBPs. These 12 items were rated on a 5-point Likert scale from 1 (strongly disagree) to 5 (strongly agree).

Data Analysis

An anonymous linkage code was used to link pre- and post-test measures. Questionnaire responses were compiled to produce descriptive statistics. Cronbach’s alpha was calculated for the 12-item attitude and organizational barriers scale. Independent t-tests were conducted to examine the relationship between job responsibility, work setting, discipline, and certification status with attitudes towards EBPs and organizational barriers to adopting EBPs.

Results

A total of 119 participants completed the pre-test of approximately 130 individuals who attended the conference (not including the conferences organizers), yielding a 92% survey response rate. Ninety-eight participants (82%) completed the post-test measures. However, missing data varied throughout the questionnaire items, with range of 1% to 18% missing from the pre-test and a range of 18% to 34% missing from the post-test measures. A majority of the missing post-test data was from the SFTRC Course Evaluation items, which appeared as the second half of the questionnaire on the back side of the last page. No differences were found on demographic and professional characteristics between those who completed the SFTRC items and those who did not complete the items.

Participant Demographics

The mean age of participants was 49.8 (SD = 11.8) and 68% were women. Only 8% described themselves as having Hispanic or Latino ethnicity. The racial breakdown was 22% African-American, 10% Asian-American, 62% White/Caucasian, and 6% mixed race. The educational level of participants varied: no high school degree (1%); high school degree only (3%); some college but no degree, (31%); Associate’s degree (6%); Bachelor’s degree (6%); Master’s degree (33%); doctoral degree (17%); other degree (4%).

Professional Characteristics

The mean years of experience in the addictions field was 11.9 (SD = 8.4) with a range of 0 – 32 years. Fifty-eight percent of participants were currently certified or licensed in the addictions field, while 4% had previous licensure; 6% were interns, and 32% were not licensed. A majority of respondents described their discipline as addiction counseling (58%), psychology (26%), or administration (22%). Other disciplines included other counseling (14%), social work (13%), medicine or nursing (8%), criminal justice (6%), education (5%), vocational rehabilitation (3%) and student (1%). Since the discipline item categories were not mutually exclusive, participants may have reported more than one discipline. Approximately half of the sample reported working for a substance abuse treatment agency (49%). Primary job responsibilities included line staff (41%), supervision (35%), administration (21%), and training/education (14%).

Clinician Readiness to Adopt EBPs

Of those who responded to the EBP adoption section of the questionnaire (n = 85), 75% (n = 64) reported that were currently using EBPs in their work (action stage of change). Among participants who were not currently using EBPs (n = 21), 14 reported that they were seriously considering adopting EBPs in the next 30 days (preparation stage), 3 reported that they were seriously considering adopting evidence-based practices in the next 6 months (contemplation stage) and 4 participants were not considering incorporating EBPs into their work (precontemplation stage). Figure 1 shows the stages of change to adopt EBPs.

Figure 1.

Stages of Change to Adopt EBPs

Experience with EBPs

Sixty-six of 80 (83%) respondents reported EBP use in the past year. The most frequently employed EBPs were Cognitive-Behavioral Therapy (61%), Motivational Enhancement Therapy or Motivational Interviewing (53%), and Twelve-Step Facilitation (31%). Less frequently used EBPs were Psychodynamic-Expressive-Supportive Therapy (14%), Multi-dimensional Family Therapy (11%), Contingency Management (9%), Brief Strategic Family Therapy (6%), and Behavioral Couples Therapy (2%).

Eighty-six percent of participants reported on the post-test that they were interested in obtaining additional training in EBPs.

Attitudes Toward EBPs

The mean score for the Positive Outcome items was 3.51 (SD = 0.71), indicating moderate agreement that EBPs enhance treatment outcome. The Negative Process items was 2.02 (SD = 0.67), indicating disagreement among clinicians that EBPs negatively impact the therapeutic process. Table 1 shows the means and standard deviations for the items and subscales. Independent t-tests were performed to examine the relationship between work setting, job responsibility, discipline, and certification status with attitudes toward EBPs. Participants who were currently certified in the addictions field had significantly lower scores on Negative Process (t = −2.34, p < .05; M = 1.83 vs., 2.17) compared with those who were not certified. None of the other comparisons were statistically significant.

Table 1.

Practitioner Attitudes and Organizational Barriers^a

Item	M	SD	n
Attitudes – Positive Outcomeb	3.51	0.71	80
Using a treatment manual helps a therapist to evaluate and improve his or her clinical skills.*	3.84	0.98	81
Following a treatment manual will enhance therapeutic outcomes by insuring that the treatment being used is supported by research.*	3.75	0.84	81
If a treatment has been shown scientifically to be effective, then the counselor is ethically obligated to use the treatment as opposed to one that has not been studied.*	2.94	1.05	80
Attitudes – Negative Processb	2.02	0.67	80
Evidence-based practices make counselors more like technicians than caring human beings.	1.90	0.73	81
Treatment manuals are appropriate for research clients but not “real world” clients.	2.12	0.90	81
Using evidence-based practices detracts from the authenticity of the therapist interaction.	2.04	0.91	80
Organizational Barriersb	2.34	0.41	76
Evidence-based practices seem overly complicated and hard to put into practice.	2.14	0.68	83
There are influential clinicians at my program that are definitely against evidence-based treatments.	2.27	1.07	81
It would take some very strong incentives, such as restricting our funding, before our treatment program would use evidence-based practices.	2.36	1.06	78
The idea of evidence-based practices sound good in “theory,” but in reality, it’s virtually impossible to scientifically test a phenomenon as complex as substance abuse treatment.	2.11	0.87	80
The treatments that we do at our program may not be “evidence-based,” but they work just as well, or better.	2.88	0.81	78
As long as they don’t conflict with treatments already in place at our program, I don’t see any problem with using a few procedures that are evidence-based.**	3.76	1.13	79

Note. Respondent scale: 1 = Strongly Disagree, 2 = Disagree, 3 = Not sure, 4 = Agree, 5 = Strongly Agree

^aCronbach’s alpha (n = 76) for 12-item scale = .63; standardized alpha = .67

^bRepresents subscale statistics.

^*Item is reverse-scored in calculating scale reliability.

^**Item is reverse-scored in calculating scale reliability and subscale mean.

Organizational Barriers

The mean score for the organizational barriers items was 2.34 (SD = 0.41), suggesting that agencies in which the participants worked were ready to adopt EBPs (see Table 1). Participants held this view regardless of work setting, job responsibility, discipline, and certification status (p values > .05).

Management Strategies

Participants were asked to check off the management strategies utilized in their agency to support the use of EBPs. Of the 85 participants who responded to this item, 92% reported that their agency used at least one strategy to support use of EBPs. Many strategies were being incorporated (see Figure 2). The highest frequencies were reported for off-site trainings (67%) and on-site trainings (66%).

Figure 2.

Management Strategies Used to Support EBPs

Discussion

The present results suggest that substance abuse treatment providers in the San Francisco Bay Area, including addictions counselors, are willing to learn about EBPs and most employ EBPs in their work. Although we did not assess previous formal training in EBPs, the addictions field has increasingly emphasized the importance of higher education and credentialing. A recent survey of substance abuse treatment providers demonstrated that substance abuse counselors are now more educated and have more certifications and substance abuse licenses (Mulvey et al., 2003). In our sample, 58% of participants were currently licensed or certified in addictions and 50% held advanced degrees. Participants who were currently licensed or certified in the addictions field were less likely to have negative attitudes toward EBPs, compared with those who did not have such credentials. Practitioners may also be more open to providing clinical services with a research base because managed care has encouraged the use of treatment manuals and EBPs (Strosahl, 1998). In addition, the dissemination activities of federal, state and local agencies (e.g., NIDA Clinical Trials Network [CTN]) have aided addiction counselors in becoming more familiar with EBPs (Ball et al., 2002). Study findings also indicate that many agencies in the San Francisco Bay Area are using management strategies to support EBPs, and may facilitate adoption of EBPs.

Regarding attitudes toward EBPs, in general, clinicians tend to hold positive attitudes toward treatment manuals and EBPs. They view EBPs as facilitating treatment outcomes, and do not perceive EBPs as artificial or interfering with the client-counselor relationship. Participants tended to hold these positive attitudes regardless of work setting, job responsibility, and discipline. Interestingly, our 2006 sample of substance abuse providers held more positive views toward the use of psychotherapy manuals than a group of psychologists from all over the United Stated surveyed in 1998 (Addis & Krasnow, 2000). This difference in attitudes may be reflective of the recent dissemination movement toward EBPs in substance abuse treatment within our geographic region.

Our results support previous findings regarding the use of specific EBPs (McGovern et al., 2004). We showed that clinicians were more likely to use “an amalgam” of Motivational Interviewing, Cognitive-Behavioral Therapy, and Twelve-Step Facilitation rather than, Contingency Management or other EBPs. Poor utilization of Behavioral Couples Therapy, which has very strong research support, has been reported in substance abuse treatment due to being too intensive (i.e., too many sessions) and a stand-alone intervention (Fals-Stewart & Birchler, 2001). Substance abuse treatment providers may be more interested in using EBPs as an adjunct to traditional treatment (McGovern et al., 2004). Others conclude that substance abuse treatment agencies may not yet be ready to embrace EBPs, and that programs should address more fundamental aspects of care before trying to implement EBPs (Amodeo et al., 2006). Recent research suggests that an understanding of the technology transfer process (i.e., variables such as exposure, organizational resources, nature of innovations, and stage of diffusion) is critical to moving evidence-based practices from research settings into standard clinical care (Ducharme et al., 2007).

The current study provides descriptive data on readiness to adopt EBPs, experience with EBPs and attitudes toward EBPs by substance abuse treatment providers. Study findings may be limited by missing data, which was higher for specific sets of questions on the post-test: stages of change (29% missing), management strategies (29% missing), attitudes toward EBPs (33% missing) and organizational barriers (34% missing). Participants may have skipped items when their answer was “no” or “not applicable,” selectively answered questions based on their level of understanding, or neglected items due to fatigue. In addition, these items were on the back side of the last page of the questionnaire and may have been overlooked. Analysis comparing demographic and professional characteristics of those who did and did not complete survey items showed no differences, suggesting that missing data was not due to systematic differences on those characteristics. Main findings were that individual participants and their agencies were interested in, were positive about, and were in most cases using one or more EBPs. As the direction of bias associated with missing data is unknown, we have no reason to expect this general pattern of findings is inaccurate. It is possible that more complete responding would result in attenuation, although not reversal, of these findings, and additional and future research will inform on this point.

Next, most of our measurement tools were not examined for validity and reliability in this study. The 12-item scale we created to assess practitioner attitudes and organization barriers showed fair but not optimal internal reliability (alpha = .63), and there were too few items to calculate subscale reliabilities. Further refinement to the instruments is needed to obtain more in-depth information on EBPs. Another limitation was that participants voluntarily attended a conference to learn about and receive training on EBPs and, thus, may have been affected by social desirability response bias. Specifically, participants may have felt some pressure to state they were using, and positive about, EBPs even if they were using them minimally or had reservations. However, responses were anonymous and not linked to individuals, and this may mitigate responding on the basis of social desirability. Finally, we did not directly observe clinical practice, and cannot report actual clinical practices with certainty.

Our findings suggest that dissemination activities in the San Francisco Bay Area are reaching addiction treatment providers, and that agencies are supportive of offering training to their employees on EBPs. We are also encouraged that 86% of participants reported interest in obtaining additional training in EBPs. This study has implications for those in the practice settings such that becoming certified or licensed in addictions may lead to greater acceptance and use of EBPs. In addition, widespread dissemination of EBPs has potential for improving clinical care and changing treatment standards among substance abuse treatment professionals.

The next step for research in this area could involve independent observation of clinicians and addictions counselors who self-report use of EBPs. It is critical to assess how clinicians implement and maintain EBPs once they are trained. Increased dissemination and additional time for professional development may lead to discernable change or use of EBPs if programs are organized to allow counselors to practice new skills (Kerwin et al., 2006).