Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Feb 1.
Published in final edited form as: Addiction. 2010 Oct 6;106(2):428–441. doi: 10.1111/j.1360-0443.2010.03135.x

Teaching Community Program Clinicians Motivational Interviewing Using Expert and Train-the-Trainer Strategies

Steve Martino 1, Samuel A Ball 1, Charla Nich 1, Monica Canning-Ball 1, Bruce J Rounsaville 1, Kathleen M Carroll 1
PMCID: PMC3017235  NIHMSID: NIHMS225881  PMID: 20925684

Abstract

Aims

The effectiveness of expert-led (EX) and train-the-trainer (TT) strategies was compared to a self-study approach (SS) for teaching clinicians motivational interviewing (MI).

Design

Twelve community treatment programs were randomly assigned to the three conditions. EX and TT conditions used skill-building workshops and three monthly supervisions sessions guided by treatment integrity ratings, performance feedback and coaching techniques. Trainers in TT were first trained and certified in MI and then carefully prepared to deliver the workshops and supervise MI at their programs. Clinicians in SS only received the training materials.

Setting

Licensed outpatient and residential addiction and mental health treatment programs in the U.S. State of Connecticut were involved in the study.

Participants

Ninety-two clinicians who provided addiction treatment within these programs and had limited experience with MI participated in the study.

Measurements

Primary outcomes were the clinicians’ MI adherence and competence and the percentage of clinicians meeting clinical trial standards of MI performance. Assessments occurred at baseline, post-workshop, post-supervision, and a 12-week follow-up.

Findings

The study found EX and TT, in comparison to SS, significantly improved clinicians’ adherence and competence, with higher percentages of clinicians reaching clinical trial standards of MI performance and few differences between EX and TT.

Conclusions

This study supports the combined use of workshops and supervision to teach community program clinicians MI and suggests training-the-trainer may be a feasible and effective strategy for disseminating empirically supported treatments.

Keywords: dissemination, implementation science, motivational interviewing, substance abuse, technology transfer, training strategies

INTRODUCTION

International and U.S. policymakers have strongly encouraged clinicians to learn empirically supported mental health and addiction treatments to improve client care [1-4]. How to best train clinicians to use these practices with adherence and competence (i.e., how much and how well clinicians use specific strategies) has begun to be systematically evaluated [5-7]. This study evaluates the effectiveness of expert-led and train-the-trainer approaches for teaching motivational interviewing (MI) [8], an empirically supported treatment for addictions [9].

Two common strategies for training clinicians in empirically supported treatments include self-study materials and workshops. Clinical trials of psychotherapies usually add post-workshop supervision using feedback and coaching based on performance ratings [10-12]. Experts in those therapies typically conduct workshops and subsequently supervise clinicians. Self-study, a common way clinicians learn new treatments [3], has been found to be ineffective [13,14]. Similarly, workshops are of limited value in developing or sustaining counselors’ therapeutic skills [13,15-20]. In contrast, several studies [13,14,21-24] have demonstrated that expert-delivered workshops, coupled with post-workshop supervision that employs rating-based feedback and coaching, improves clinicians’ ability to implement empirically supported approaches.

While effective, expert-driven training approaches are limited as technology transfer vehicles. Experts often are few in number, expensive to employ, and not readily available to support on-site practice. Treatment implementation models emphasize establishing program-based advocates who first become knowledgeable and committed to an empirically supported treatment, and then actively champion its use within their agencies [25-28]. One method for establishing program-based advocates is to use a train-the-trainer (TT) approach [32-49] in which an expert trains practitioners how to teach a designated intervention to others. Subsequently, the practitioners return to their settings and then train, supervise, and monitor staff members’ treatment implementation. This approach is popular in the medical education [29-35], primary/secondary prevention [36-42], and mental health and addiction fields [43-46].

No prior controlled randomized trial of TT for training clinicians in empirically supported treatments has been conducted. In this report, we describe a controlled randomized trial of expert and TT strategies, using workshop training and post-workshop supervision, to teach community program clinicians MI. Relative to a self-study condition, we hypothesized that both the expert and TT conditions would: 1) improve community program clinicians’ adherence and competence in using MI following workshop training and supervision; and 2) result in a higher percentage of clinicians performing MI adequately. In addition, by virtue of providing durable on-site resources and support, we expected the TT condition to sustain these gains through the 12-week follow-up, and the expert condition to show diminished MI performance in the absence of continuing expert consultation.

METHODS

Study design

Twelve outpatient substance abuse treatment programs within Connecticut (USA) were randomly assigned to either a 1) self-study (SS), 2) expert (EX), or 3) train-the-trainer (TT) condition using a random sample program to select 4 programs per condition.1 Participating programs were licensed addiction treatment facilities and had not previously implemented MI workshop or supervision training. Clinicians completed baseline, post-workshop, post-supervision, and 12-week follow-up assessments. Primary outcomes were 1) clinician MI adherence and competence in client and role-played sessions and 2) the percentage of clinicians performing MI adequately.

Participant clinicians

Clinicians were required to 1) be employed ≥ 20 hours/week, and 2) treat English-speaking substance-using clients. Clinicians who had received recent formal MI workshop training or supervision were excluded. Research assistants met with clinicians who expressed interest in the study and obtained written informed consent approved by Yale University School of Medicine Human Investigation Committee.

Training conditions

Self-Study (SS)

Clinicians were provided with the MI textbook [8], training videotapes [47], and a treatment manual.2 They were asked to spend 20 hours reviewing these materials over the next 12 weeks.

Expert (EX)

One expert (first author) provided all EX training to clinicians. Training included a 15-hour workshop per Motivational Interviewing Network of Trainers recommendations [8]. Next, clinicians audiotaped client sessions in which they practiced MI. These were rated for MI adherence and competence using the Independent Tape Rater Scale (ITRS described below). Clinicians had 3 monthly individual face-to-face supervisions, which involved receiving rating-based feedback about their use of MI strategies and role-play practice.

Train-the-Trainer (TT)

Two clinicians per program (n = 8) became MI trainers and supervisors at their program. They all worked full-time, had master’s degrees or higher, and were licensed professionals. Most were program directors/managers (75%) and had been supervising on average 14.0 years (SD = 9.4). None had received prior MI training.

Trainers learned MI using the workshop and supervision procedures described above. Trainers had to meet adequate standards of MI performance, consistent with those used to certify clinicians in prior clinical trials of MI [48-50]. Thereafter, they completed another 15-hour workshop in which they learned how to conduct the MI workshop and rating-based supervision. A TT manual guided their preparation.2 Trainers received monthly consultation calls from the expert and received $25 per completed supervision session to support supervision implementation.

Assessments procedures

Client sessions

Clinicians submitted audiotaped 40-minute sessions in which they were instructed to conduct MI with clients who had substance use problems. Clinicians, not research staff, selected clients based on clinical judgment of the suitability of substituting MI for their standard practice and the client’s willingness to be audiotaped (signed consent obtained). These audiotapes were rated to assess their MI skills in clinical practice.

Role-played sessions

To control for program-specific client populations and variations in clinical presentations, clinicians completed a 40-minute videotaped standardized role-played session with a client-actor. The same scenario (i.e., a client presenting in an intake as ambivalent about stopping marijuana and alcohol use) was used across assessment points. The ‘client’ in the assessment session was played by a project staff member who had previously enacted a “client” in a professionally produced training video. Similar methods have been shown to be reliable stimuli for MI skills assessment [17].

Independent tape rating

Twelve raters, blind to training condition, program and assessment points, rated the client and role-played sessions. All raters attended seminars in which they learned the ITRS system. Next, each rated an identical set of 18 calibration tapes randomly selected from the larger pool of session tapes, which were used to evaluate inter-rater reliability. Raters received approximately 44 hours of training.

Measures

Independent Tape Rater Scale

The ITRS assesses clinicians’ adherence and competence using MI and other strategies inconsistent with MI (e.g., direct confrontation) or common to drug counseling (e.g., assessing substance use). This study used 15 ITRS items: ten evaluated strategies that characterize MI and five that were inconsistent with MI. For each item, raters evaluated the extent (adherence; 1 = not at all, to 7 = extensively) and skill (competence; 1 = very poor, to 7 = excellent) with which the clinician delivered the strategy. Confirmatory factor analyses have supported a two-factor model for the 10 MI consistent items (corresponding to “fundamental” and “advanced” MI strategies) [51,52]: five fundamental MI strategies that underpin the client-centered stance of MI and five items that involved advanced strategies for evoking client motivation for behavior change (e.g., heightening discrepancies). Each scale was averaged to obtain adherence and competence scores (see Table 1 for item descriptions). In addition, five MI inconsistent items (unsolicited advice, therapeutic authority, direct confrontation, emphasizing total abstinence, asserting disease concepts of addiction) were averaged to determine the impact of training on this area. Intra-class correlation coefficients (ICC) for the fundamental and advanced MI strategy adherence and competence scores showed good to excellent inter-rater reliability (adherence ICC: fundamental = .88 and advanced = .87; competence ICC: fundamental = .87 and advanced = .68) 3 [53]. Interrater reliability estimates for the MI-inconsistent adherence items were excellent (ICC = .91).

Table 1.

Independent Tape Rater Scale item definitions for the fundamental and advanced MI strategy factors

Fundamental MI Strategies Definition
 Open Questions Questions that elicit more than yes/no responses and encourage the client to talk about his/her problems, motivation, change efforts, and plans
 Reflective Statements Repeating (exact words), rephrasing (slight rewording), paraphrasing (e.g., amplifying the thought or feeling, use of analogy, making inferences) or making reflective summary statements about what the client said
 Affirmations Verbal reinforcement of the client’s strengths, abilities, or efforts to change his/her behavior
 Fostering a Collaborative Relationship Statements or questions that convey that the therapy is a collaborative relationship in contrast to one where the therapist is in charge
 Motivational Interviewing Style/Spirit Demonstration in the session of the therapist’s supportive, warm, and non-judgmental stance, empathic sensitivity to the client’s experiences, and attentiveness to the client’s motivation for change
Advanced MI Strategies
 Client-centered Problem Discussion and Feedback Explicit facilitation of a discussion about the problems for which the client entered treatment, as well as the provision of personalized feedback about the client’s substance abuse and the evidence or indications of related problems in other life areas
 Pros, Cons, and Ambivalence Exploration of the positive and negative effects of the client’s substance use, what might be gained and lost by abstinence or reduction in substance use, and reflections capturing the client’s ambivalence about the change process
 Heightening Discrepancies Efforts made to heighten the internal conflict a client’s continuing substance use poses to the client’s self perceptions, life goals or values
 Motivation for Change Efforts made to elicit client discussion of change (self-motivational statements) through questions or comments designed to promote greater awareness/concern for the problem, increase intent/optimism to change, or encourage elaboration on a topic related to change
 Change Planning Discussion with a client about his or her readiness to prepare a change plan and facilitation of a discussion about the client’s self-identified goals, steps for achieving those goals, supportive people available to help the client, what obstacles to the change plan might exist, and how to address impediments to change
Open in a new tab

To determine if clinicians’ met adequate standards of MI performance based on the ITRS, the same threshold was used as in prior clinical trials of MI with community program clinicians, that is, the clinicians conducted sessions in which they were rated as having at least half of the MI consistent scale items rated average or above in terms of adherence and competence [48-50].

Evaluation Form (workshop and supervision versions)

This 12-item form asks clinicians to rate how much the workshop trainers or supervisors covered core MI skills/principles and their overall skillfulness. Items used 7-point Likert-rating scales, with 1 = not at all, to 7 = extensive.

Data analysis

We used a random effects regression model approach, with two contrasts for training condition (EX vs. SS; TT vs. SS), inclusion of all assessment points, and condition by time interaction in the model. The prediction was that at each of the 3 time points, the MI scores would be higher for TT and EX than for SS compared to the baseline MI scores. Separate analyses were conducted for each of the five primary outcomes (fundamental and advanced MI strategies adherence and competence, MI inconsistent adherence). All models were initially run separately for each condition to assess differences in outcome by program site and time; because program effects were not significant within conditions, program site was dropped in the final models. The d formula by Raudenbush and Lui [54] was used for estimating effect magnitude in hierarchical linear regression models [55]. Chi-square likelihood ratios were conducted to test predicted differences in the percentage of clinicians meeting adequate standards of MI performance at each assessment time point. Chi-square and independent t-tests were used to evaluate baseline differences between conditions and satisfaction with workshop training/supervision.

RESULTS

Participants

Of the 97 clinicians screened, 5 were excluded (1 anticipated medical leave, 1 was leaving the agency, and 3 did not complete baseline assessments), leaving a total of 92 clinicians allocated to three conditions across programs (SS = 31; EX = 32; TT = 29) (see Figure 1). Clinicians were primarily Caucasian (83%) female (65%) alcohol/drug counselors (71%). About half had master’s degrees and professional licenses. Most worked as frontline clinicians (83%) in outpatient settings (66%). A year prior to study involvement, clinicians had little MI training (see Table 2).

Figure 1.

Figure 1

Open in a new tab

CONSORT Flowchart

Table 2.

Clinician demographic variables and experience levels by training condition

Condition
Variable SS EX TT
Female, n (%) 24 (77%) 23 (72%) 13 (45%) *
Mean (SD) age 45.9 (12.5) 42.2 (10.9) 46.3 (10.3)
Ethnicity, n (%)
 African-American 4 (13%) 1 (3%) 5 (17%)
 Caucasian-American 25 (81%) 29 (91%) 22 (76%)
 Hispanic-American 2 (6%) 2 (6%) 1 (3%)
 Other 0 0 1 (3%)
Mean (SD) Years Education 17.4 (2.4) 17.3 (1.9) 17.3 (3.5)
Highest Degree Earned, n (%)
 Doctorate 1 (3%) 1 (3%) 3 (10%)
 Master’s 17 (55%) 21 (63%) 11 (31%)
 Bachelor’s 10 (32%) 5 (34%) 7 (59%)
 Associates 2 (6%) 3 (9%) 3 (10%)
 High School 1 (3%) 2 (6%) 5 (17%)
Discipline, n (%)
 Alcohol/Drug Counseling 23 (74%) 21 (66%) 21 (72%)
 Marriage and Family Counseling 2 (6%) 2 (6%) 0
 Social Work 9 (29%) 11 (34%) 6 (21%)
 Psychology 0 0 1 (3%)
 Nursing 0 1 (3%) 0
 Internal Medicine 0 0 1 (3%)
Licensure Status, n (%)
 Alcohol/Drug Certification 12 (39%) 12 (38%) 11 (38%)
 Licensed Professional 15 (48%) 17 (53%) 9 (31%)
Primary Role *
 Clinician 25 (81%) 21 (66%) 28 (97%)
 Program Manager/Director 6 (19%) 11 (34%) 1 (3%)
Mean (SD) Years Experience
 General Counseling 10.7 (1.0) 9.1 (6.5) 8.9 (8.3)
 Substance Abuse Counseling 9.2 (8.1) 7.6 (6.0) 7.7 (8.0)
Past substance abuse problem, n (%) 10 (32%) 10 (31%) 12 (41%)
Level of Clinical Care, n (%)
 Outpatient/intensive outpatient 23 (74%) 20 (62%) 18 (62%)
 Residential 8 (26%) 12 (38%) 11 (38%)
Past year MI training
 Mean (SD) hours 1.1 (2.5) 0.4 (2.1) 1.3 (3.6)
 Lecture/presentation, n (%) 5 (16%) 1(3%) 5 (17%)
 Workshop, n (%) 4 (13%) 1 (3%) 3 (10%)
 Supervision, n (%) 1 (3%) 0 (0%) 1 (3%)
 Read 2002 MI Textbook 2 (6%) 2 (6%) 3 (10%)
 Used MI Manuals 6 (19%) 3 (9%) 3 (10%)
Open in a new tab

Note. SS = Self-Study, EX = Expert, TT = Train-the-Trainer. For the Discipline category, the percentages in the SS and EX conditions exceed 100% because some clinicians belonged to more than one discipline.

*

p < .05.

Randomization and baseline equivalence

Randomization at the program level resulted in equivalent groups of clinicians across conditions at baseline. Significant between-group baseline differences were only observed for fundamental MI competence within the role-play assessment (F(2,87) = 4.70, p = .02). Post hoc Tukey tests showed EX clinicians had significantly higher competence (but not adherence) ratings than TT clinicians, with SS clinicians falling in between the two. In addition, TT in comparison to SS or EX, had significantly fewer women (X2(2) = 7.97, p = .02) and program managers/directors (X2(2) = 6.57, p = .04).

Training exposure

All EX and TT clinicians attended the MI workshop. EX clinicians attended slightly more supervision sessions than those in TT (2.9 vs. 2.5; t(59) = 2.62, p = .01). Clinicians rated the expert trainer as significantly more skilled and more extensively covering several MI strategies during the workshops and supervision than TT trainers (see Table 3), though ratings were generally high in both conditions. SS clinicians indicated they had spent on average 6.7 hours (SD = 7.9) using the MI training materials. TT clinicians reported they continued to receive MI supervision on average 6.3 times (SD = 5.4) during the 12-week follow-up.

Table 3.

Coverage of MI strategies in workshops and supervision by training conditions

Workshop Supervision
MI Skill EX TT EX TT
MI spirit 6.8 (0.4) 6.1 (0.8) * 6.3 (0.8) 5.3 (1.3) **
Open questions 6.7 (0.5) 6.5 (0.6) 6.2 (1.0) 5.4 (1.6) *
Reflections 6.9 (0.3) 6.4 (0.7) * 6.2 (0.9) 5.0 (1.5) *
Affirmations 6.1 (0.8) 5.9 (0.8) 6.1 (1.1) 5.2 (1.7) *
Fostering collaboration 6.4 (0.6) 6.1 (0.7) 5.8 (1.1) 5.1 (1.6) *
Motivation for change 5.9 (0.8) 6.0 (0.8) 5.5 (0.9) 5.5 (1.5) *
Client-centered discussion/feedback 6.4 (0.8) 6.1 (1.0) 6.0 (1.1) 5.0 (1.9) **
Exploring pros/cons/ambivalence 6.5 (0.6) 6.3 (0.8) 6.1 (0.9) 5.2 (1.7) **
Developing discrepancies 6.1 (0.9) 5.9 (1.0) 5.6 (1.2) 4.8(1.5) *
Change planning discussion 5.8 (0.9) 5.4 (1.2) 5.2 (1.2) 5.0 (1.6)
Reducing MI inconsistent strategies 6.6 (0.6) 6.0 (0.9) ** 5.0 (1.4) 5.4 (1.5)
Overall skillfulness 6.8 (0.4) 6.4 (0.6) ** 6.6 (0.8) 6.1 (1.0) *
Open in a new tab

Note. SS = Self-Study, EX = Expert, TT = Train-the-Trainer. Coverage was rated 1=not at all, 2=a little, 3=infrequently, 4=somewhat, 5=quite a bit, 6=considerably, 7=extensively.

*

p < .05.

**

p < .01.

Completeness of Follow-up Data

Participant retention across study assessment points was: 98% post-workshop, 90% post-supervision, and 82% 12-week follow-up. For client sessions, the percentage of clinicians having samples was: 95% baseline, 92% post-workshop, 85% post-supervision, and 72% 12-week follow-up. For role-played sessions, the percentages were: 98% baseline, 89% post-workshop, 89% post-supervision, and 78% 12-week follow-up. There were with no significant differences across conditions for any follow-up data.

MI adherence and competence

Overall, across conditions, assessment points, and type of session, clinicians typically used fundamental MI strategies about twice as often as advanced ones and with adequate levels of competence (see Table 4). MI inconsistent strategies seldom occurred. Table 5 presents the results of the random regression analyses.

Table 4.

Primary measures of clinician MI performance in client and role-played sessions

Condition
SS EX TT
Variable M SD N M SD n M SD N
Client Sessions
 Fundamental MI Strategies
  Adherence
   Baseline 3.7 0.9 29 4.1 0.9 30 4.1 1.0 27
   Post-workshop 3.7 1.1 27 4.5 0.8 31 4.5 0.9 27
   Post-supervision 4.0 1.2 26 4.6 0.8 31 4.4 1.1 21
   12-week follow-up 3.9 0.9 22 4.6 0.8 27 4.4 0.9 17
  Competence
   Baseline 3.8 0.8 29 4.1 0.7 30 4.0 0.7 28
   Post-workshop 3.8 0.9 27 4.8 0.9 31 4.4 0.9 27
   Post-supervision 4.3 0.9 26 4.6 0.8 31 4.8 0.7 21
   12-week follow-up 3.9 0.8 22 5.0 0.9 27 4.4 0.9 17
 Advanced MI Strategies
  Adherence
   Baseline 2.3 0.8 29 2.6 0.8 30 2.5 1.1 28
   Post-workshop 2.5 1.1 27 2.7 0.9 31 2.6 1.2 27
   Post-supervision 2.7 1.0 26 2.9 0.9 31 2.9 1.2 21
   12-week follow-up 2.2 0.7 22 2.7 0.8 27 2.6 1.1 17
  Competence
   Baseline 4.1 0.8 28 4.1 0.7 30 4.0 0.9 26
   Post-workshop 3.9 0.9 26 4.4 0.8 31 4.2 0.7 26
   Post-supervision 4.3 0.9 26 4.5 0.6 31 4.6 0.6 21
   12-week follow-up 4.0 0.7 22 4.9 0.7 25 4.3 0.7 16
 MI Inconsistent Adherence
   Baseline 2.4 0.8 29 1.9 0.7 30 2.2 0.7 28
   Post-workshop 2.0 0.9 27 1.4 0.4 31 1.6 0.8 27
   Post-supervision 1.8 0.5 26 1.4 0.5 31 1.8 0.8 21
   12-week follow-up 1.8 0.9 22 1.5 0.5 27 1.8 0.7 17
 % Meeting MI Performance Standards
   Baseline 24% 43% 32%
   Post-workshop 33% 55% 41%
   Post-supervision 42% 68% 57%
   12-week follow-up 18% 59% 53%
Role-Played Sessions
 Fundamental MI Strategies
  Adherence
   Baseline 4.0 0.8 31 4.0 0.9 31 3.8 1.0 28
   Post-workshop 4.0 1.0 30 4.5 0.7 25 4.6 0.6 27
   Post-supervision 4.2 0.8 29 4.5 0.6 32 4.4 0.8 21
   12-week follow-up 4.2 0.7 23 4.5 0.7 30 4.1 1.0 19
  Competence
   Baseline 3.8 0.9 31 4.3 0.8 31 3.6 1.0 28
   Post-workshop 4.1 0.9 30 4.7 0.8 25 4.5 0.8 27
   Post-supervision 4.1 1.0 29 4.9 0.8 32 4.4 0.7 21
   12-week follow-up 4.2 1.0 23 4.6 0.7 30 4.5 0.9 19
 Advanced MI Strategies
  Adherence
   Baseline 2.9 1.1 31 2.7 1.0 30 2.7 0.8 28
   Post-workshop 2.8 0.9 30 3.3 0.9 25 3.3 0.9 26
   Post-supervision 2.7 0.9 29 3.3 0.8 32 3.4 1.1 21
   12-week follow-up 2.8 1.1 23 3.1 0.8 30 3.2 1.0 19
  Competence
   Baseline 4.0 0.6 30 4.2 1.0 31 3.7 1.0 28
   Post-workshop 4.1 0.9 30 4.6 0.7 25 4.3 0.6 27
   Post-supervision 4.1 0.9 29 4.6 0.6 32 4.4 0.6 21
   12-week follow-up 4.3 0.9 23 4.6 0.7 30 4.4 0.9 19
 MI Inconsistent Adherence
   Baseline 2.2 1.0 31 2.2 0.9 31 2.1 1.1 28
   Post-workshop 1.9 0.9 30 1.5 0.5 25 1.8 1.1 27
   Post-supervision 2.1 1.1 29 1.3 0.4 32 2.0 0.9 21
   12-week follow-up 1.8 0.6 23 1.4 0.5 30 1.9 0.8 19
 % Meeting MI Performance Standards
   Baseline 39% 45% 29%
   Post-workshop 33% 76% 67%
   Post-supervision 31% 78% 71%
   12-week follow-up 48% 63% 53%
Open in a new tab

Note. SS = Self-Study, EX = Expert, TT = Train-the-Trainer. Adherence was rated 1=not at all, 2=a little, 3=infrequently, 4=somewhat, 5=quite a bit, 6=considerably, 7=extensively. Competence was rated 1=very poor, 2=poor, 3=acceptable, 4=adequate, 5=good, 6=very good; 7=excellent.

Table 5.

Effects of training condition on primary outcomes at post-workshop, post-supervision, and 12-week follow-up points: results of random regression analyses

Fundamental MI Strategies Advanced MI Strategies MI Inconsistent Strategies
Adherence Competence Adherence Competence Adherence
t p t p t p t p t p
Client Sessions
Intercept 20.94 .000 28.03 .000 14.40 .000 27.62 .000 17.38 .000
Training Condition
 EX vs. SS 1.39 .167 1.18 .239 1.25 .214 0.26 .794 -2.16 .034
 TT vs. SS 1.21 .231 0.68 .496 0.77 .44 -0.46 .650 -0.83 .410
Time (baseline to)
 Post-workshop -0.04 .967 -0.21 .835 0.56 .575 -0.87 .386 -2.12 .036
 Post-supervision 0.86 .391 2.66 .009 1.35 .181 1.03 .306 -3.24 .002
 12-week follow-up 0.46 .643 0.26 .792 -0.39 .695 -0.28 .781 -3.02 .003
Training Condition by Time (baseline to)
 EX vs. SS at post-workshop 1.36 .175 2.67 .009 -0.31 .760 1.57 .120 -0.74 .458
 EX vs. SS at post-supervision 0.86 .391 0.20 .839 -0.16 .878 0.75 .453 0.08 .934
 EX vs. TT at 12-week follow-up 1.25 .214 2.84 .005 0.38 .705 2.92 .004 0.33 .744
 TT vs. SS at post-workshop 1.30 .197 1.58 .109 -0.16 .873 1.30 .196 -1.08 .282
 TT vs. SS at post-supervision 0.43 .667 1.51 .166 0.22 .828 1.62 .108 0.56 .577
 TT vs. SS at 12-week follow-up 0.56 .577 1.01 .296 0.72 .475 1.40 .165 0.51 .609
Role-Played Sessions
Intercept 26.71 .000 24.36 .000 16.52 .000 25.77 .000 12.74 .000
Training Condition
 EX vs. SS 0.16 .876 2.01 .047 -0.74 .462 1.18 .241 -0.30 .768
 TT vs. SS -1.07 .289 -1.17 .244 -0.87 .387 1.34 .182 -0.59 .554
Time (baseline to)
 Post-workshop 0.17 .866 1.48 .142 -0.61 .541 0.90 .368 -1.72 .087
 Post-supervision 0.82 .416 1.28 .203 -0.84 .401 0.44 .664 -0.73 .466
 12-week follow-up 1.26 .211 1.80 .076 -0.50 .617 1.34 .182 -2.22 .028
Training Condition by Time (baseline to)
 EX vs. SS at post-workshop 1.73 .086 0.67 .503 2.31 .022 0.99 .324 -1.03 .307
 EX vs. SS at post-supervision 1.15 .253 1.40 .163 2.47 .015 1.18 .241 -2.60 .010
 EX vs. TT at 12-week follow-up 0.93 .355 0.08 .935 1.50 .135 0.29 .772 -1.26 .210
 TT vs. SS at post-workshop 3.00 .003 2.60 .011 2.39 .018 1.77 .080 0.31 .755
 TT vs. SS at post-supervision 2.03 .045 2.60 .011 2.74 .007 2.66 .009 0.06 .952
 TT vs. SS at 12-week follow-up 0.48 .635 2.16 .033 1.75 .083 1.87 .064 0.60 .546
Open in a new tab

Note. SS = Self-Study, EX = Expert, TT = Train-the-Trainer

Client sessions

EX compared to SS clinicians significantly increased their fundamental strategy competence from baseline to post-workshop (d = 1.05) and 12-week follow-up (d = 1.06) and advanced strategy competence from baseline to 12-week follow-up (d = 1.19). Across all conditions, clinicians showed increased fundamental strategy competence from baseline to post-supervision and decreased use of MI inconsistent strategies at each time point. EX clinicians used MI inconsistent strategies less frequently than SS clinicians. Significant differential training effects between TT and SS were absent.

Role-played sessions

EX clinicians showed significantly increased advanced strategy adherence from baseline to post-workshop (d = .82) and post-supervision (d = .89) and decreased use of MI inconsistent strategies from baseline to post-supervision (d = -.78) compared to SS clinicians. TT compared to SS clinicians showed significantly increased fundamental strategy adherence from baseline to post-workshop (d = .95) and post-supervision (d = .63) and competence from baseline to post-workshop (d = .71), post-supervision (d = .76), and 12-week follow-up (d = .61). TT relative to SS clinicians also showed significantly more use of advanced strategies from baseline to post-workshop (d = .92) and post-supervision (d = 1.17) and competence from baseline to post-supervision (d = .72) and 12-week follow-up (d = .54). Across all conditions, use of MI inconsistent strategies declined from baseline to 12-week follow-up.

MI performance standards

No significant baseline differences existed between conditions in the percentage of clinicians who met adequate standards of MI performance in client or role-played sessions. Accordingly, Chi-square likelihood ratios were conducted at each assessment point. However, to further rule out the potential of baseline performance affecting subsequent clinician achievement of adequate standards of MI performance, we also compared the rates of adequate standards of achievement for each time point for the subgroup of clinicians who did not meet this criterion at baseline.

Client sessions

A significantly higher percentage of EX clinicians achieved adequate standards of MI performance than SS clinicians post-supervision (68% vs. 42%, X2(1) = 3.72, p = .05, ϕ = .26) and 12-week follow-up (59% vs. 18%, X2(1) = 8.47, p = .01, ϕ = .42). A significantly higher percentage of TT than SS clinicians achieved adequate MI performance standards at the 12-week follow-up (53% vs. 18%, X2(1) = 5.21, p = .02, ϕ = .37) (see Figure 2). When these analyses were repeated only for clinicians who did not meet these standards at baseline, only the EX percentages remained higher than SS at the 12-week follow-up.

Figure 2.

Figure 2

Open in a new tab

Percentage of clinicians meeting MI performance standards in client and role-played sessions

Role-played sessions

EX and TT, in comparison to SS, had significantly higher percentages of clinicians reach or surpass adequate MI performance standards post-workshop (76% vs. 33%, X2(1) = 9.96, p = .01, ϕ = .43; and 67% vs. 33%, X2(1) = 6.32, p = .01, ϕ = .33, respectively) and post-supervision (78% vs. 31%, X2(1) = 13.67, p = .00, ϕ = .47; and 71% vs. 31%, X2(1) = 7.96, p = .01, ϕ = .40, respectively) (see Figure 2). Significant difference were absent at the 12-week follow-up point. Repeating these analyses only for clinicians who fell below standards at baseline showed that the results remained unchanged.

Expert vs. train-the trainer

Analyses were re-run to explore differences between EX and TT conditions. No significant condition or condition by time effect was present for the clinicians’ MI adherence and competence ratings in the client sessions. In the role-played sessions EX, not TT, clinicians reduced their use of MI inconsistent strategies from baseline to post-supervision (t(60) = -2.84, p = .006). EX clinicians used fundamental strategies more competently than TT clinicians (t(60) = 3.11, p = .003), although TT clinicians became more competent in their use of these strategies over time (t(60) = -2.64, p = .011). No differences occurred in the percentage of clinicians who met adequate standards of MI performance in client or role-played sessions.

DISCUSSION

The study’s main finding is that EX and TT strategies, using the same clinician training methods, resulted in improvements in clinicians’ MI performance at different assessment points when compared to a self-study approach. Consistent with its hypotheses, clinicians exposed to expert training 1) became more competent with fundamental and advanced MI strategies in client sessions, 2) used more advanced MI strategies and fewer MI inconsistent strategies in role-played sessions, and 3) performed MI adequately with clients and within role plays in percentages higher than clinicians studying MI on their own. Clinicians trained by program-based trainers, in comparison to those who self-studied, 1) improved their fundamental and advanced adherence and competence in role-played sessions, and 2) had a higher percentage performing MI adequately in role-played sessions. This study adds to the literature supporting the use of workshop training followed by rating-based feedback and coaching in supervision as a method for training clinicians in addiction treatments [13,14].

Contrary to the hypothesis that TT would continue to sustain these MI performance gains better than EX during the 12-week follow-up period, the predicted advantage did not occur. This was most surprising in the client sessions where the program-based trainers reported continued supervision of MI practice and yet, relative to SS, did not achieve significantly higher percentages of clinicians achieving adequate standards of MI performance among those who had not performed to this level at baseline. In retrospect, the expectation of better TT outcomes at follow-up was not well-founded given that program-based trainers had no prior MI training, were evaluated as less extensively and skillfully covering MI material in the workshops and supervision (albeit still positive), and had MI training activities added to their workloads rather than supplanting other tasks. Monthly consultation calls, financial incentives, and data collection during the study’s supervision phase supported implementation. Once these supports ended, trainers may have returned to supervision-as-usual practices, excluding the critical techniques of observation, feedback, and coaching. At the 12-week follow-up, TT trainers and clinicians anecdotally confirmed this speculation. The challenge of sustaining program-based trainers’ fidelity to empirically supported training methods parallels the challenge of sustaining clinicians’ proficiency using empirically supported treatments. How to best select, prepare, certify, and support TT trainers requires future investigation.

Nonetheless, this study suggests that rigorously trained program-based trainers may be able to achieve clinician training effects consistent with those of experts and supports the use of a TT strategy for disseminating MI [8]. Direct comparison of TT to EX across time points showed similar levels of clinician MI adherence and competence and percentage meeting adequate standards of performance. Moreover, relative to EX, TT clinicians significantly improved their competence using fundamental MI strategies over time. These training gains were achieved using a systematic and intensive process for preparing the trainers and suggests that a substantial amount of training and implementation resources may be required to adequately prepare program-based trainers to teach clinicians MI. Similarly, clinicians may need more supervision over time to improve upon the less than 60% rate at which EX and TT clinicians met adequate standards of MI performance at the 12-week follow-up.

This study also raises the issue of how to best evaluate clinicians’ MI adherence and competence. Standardized role-played sessions have the advantage of providing consistent stimuli across programs, clinicians, and time points, making adherence and competence assessments more attributable to changes resulting from clinician training efforts [17]. However, role-played clients may not vary their responses like actual clients [13] and these interactions might not reflect how clinicians implement MI in practice. In this study, results based on ratings from actual and role-played client assessments were not interchangeable. In comparison to SS, clinicians in EX showed significantly more competent performance in client sessions whereas those in TT showed more adherent and competent performance in role-played sessions. If only client or role-played assessments had been used, one training strategy might have appeared better than the other when compared to a self-study approach. Using multiple methods (client and role-played assessments) to examine different facets of clinician fidelity may be the most prudent way to understand the effectiveness of clinician training strategies. Future research is needed to determine why clinician performance may vary with the type of assessment methods used.

Limitations

This study’s limitations include: 1) unequal control for attention and time in the self-study condition; 2) attrition of recorded client and role-played sessions; 3) use of only one expert trainer in the EX condition that confounds the condition with the trainer’s skills and shortcomings; 4) self-selected client sessions that might not reflect clinicians’ use of MI more broadly within their programs or over-estimate the actual effects obtained in this study. In addition, we did not assess 1) the impact of training on the clients’ verbal statements within sessions (e.g., frequency of statements that favor or disfavor change), a good proxy for behavior change [56], 2) organizational features that might have moderated training effects [26,28], or 3) independent fidelity checks on the expert and program-based trainers in the workshops and supervision.

Conclusion

This study lends additional empirical support for the use of workshops followed by clinical supervision as an effective way to train community program clinicians in addiction treatments. It also provides the first evidence that program-based trainers, adequately prepared to teach MI, can help clinicians learn MI with training outcomes similar to those achieved by an expert. The train-the-trainer approach continues to hold promise as an effective strategy for disseminating empirically supported practices in community program settings.

Acknowledgments

This study was funded by the U.S. National Institute on Drug Abuse (R01 DA16970 awarded to Steve Martino, with additional support provided by U10-DA013038 and P50-DA09241). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of NIDA. The authors acknowledge the invaluable support of program executive directors (Richard Bilangi, Judith Bouchard, Ronald Fleming, Christopher Leary, Barry Kasdan, Diane Manning, Mark Muradian, Joan Pesce, Joseph Puzzo, Mwamburi Shake, Joseph Sullivan, Kathleen Ulm), trainers in the TT condition (Cliff Briggie, Gary Freudenthal, James Garland, Jodi Giorlando, Nancy Hunter, Robert Lambert, Ciro Massa, Jennifer Mendes-Hramiak, Rolando Martinez, John Reed), study personnel (Francis Giannini, Monica Canning-Ball, Regina Wright), and independent raters (Theresa Babuscio, Amy Blakeslee, Jeffrey Burda, Joanne Corvino, Carolyn Haller, Karen Hunkele, Demetrios Kostas, Elaine LaValle, Martha Staeheli Lawless, Mark Lawless, Katy McRae, Manuel Paris, Nora Rightmer, Douglas Slawin, Gia Syracuse-Siewart, Elizabeth Trainor) involved in this study.

Footnotes

1

One program initially randomized to the TT condition experienced a one-month employee strike during the time in which two program clinicians were being prepared as trainers, disrupting the trainer preparation process and delaying the study’s implementation at this site. Subsequently, the agency experienced substantial staff turnover, which resulted in the trainers having their caseloads increased and no time to conduct supervision for the study. Because of these issues, this program was removed from the study. Another three programs interested in study participation were recruited. One of these programs was randomized to the TT condition as the replacement site.

2

The MI treatment and train-the-trainer manuals are available upon request from Steve Martino.

3

ICCs were calculated using Shrout and Fleiss [57] intraclass correlation coefficients (ICC’s) two-way mixed model (3.1) with item ratings as the random effect and raters as the fixed effect. ICC reliabilities for the ITRS are reported for the mean adherence and competence ratings because the primary outcomes involve combinations of ratings across items.

References

  • 1.Giuseppe C, Clerici M. Dual diagnosis—policy and practice in Italy. Am J Addict. 2006 Mar-Apr;15(2):125–30. doi: 10.1080/10550490500528340. [DOI] [PubMed] [Google Scholar]
  • 2.Hintz T, Mann K. Co-occurring disorders: policy and practice in Germany. Am J Addict. 2006 Jul-Aug;15(4):261–7. doi: 10.1080/10550490600754275. [DOI] [PubMed] [Google Scholar]
  • 3.Miller WR, Sorensen JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: a review with suggestions. J Subst Abuse Treat. 2006 Jul;31(1):25–39. doi: 10.1016/j.jsat.2006.03.005. [DOI] [PubMed] [Google Scholar]
  • 4.Reickmann TR, Kovas AE, Fussell HE, Stettler NM. Implementation of evidence-based practices for treatment of alcohol and drug disorders: the role of the state authority. J Beh Health Ser Res. 2009 Oct;36(4):407–19. doi: 10.1007/s11414-008-9122-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Beidnas RS, Kendall PC. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. doi: 10.1111/j.1468-2850.2009.01187.x. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Madson MB, Loignon AC, Lane C. Training in motivational interviewing: A systematic review. J Subst Abuse Treat. 2009 Jan;36(1):101–9. doi: 10.1016/j.jsat.2008.05.005. [DOI] [PubMed] [Google Scholar]
  • 7.Martino S. Evidence-based training strategies for evidence-based treatments: A beginning. Addict Sci Clin Pract. In press. [PMC free article] [PubMed] [Google Scholar]
  • 8.Miller WR, Rollnick S. Motivational interviewing: preparing people for chang. 2. New York: Guilford Press; 2002. [Google Scholar]
  • 9.Lundahl BW, Tollefson D, Gambles C, Brownell C, Burke BL. A meta-analysis of motivational interviewing: Twenty five years of empirical studies. Res on Soc Work Pract. In press. [Google Scholar]
  • 10.Baer JS, Ball SA, Campbell BK, Miele GM, Schoener EP, Tracy K. Training and fidelity monitoring of behavioral interventions in multi-site addictions research: A review. Drug Alcohol Depend. 2007 Mar 16;87(2-3):107–18. doi: 10.1016/j.drugalcdep.2006.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Carroll KM. New methods of treatment efficacy research: Bridging clinical research and clinical practice. Alcohol Health Res World. 1997;21(4):352–9. [PMC free article] [PubMed] [Google Scholar]
  • 12.Cucciare MA, Weingardt KR, Villafranca S. Using blended learning to implement evidence-based psychotherapies. Clin Psychol Sci Pract. 2008 Oct 23;15(4):299–307. [Google Scholar]
  • 13.Miller WR, Yahne CE, Moyers TE, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational interviewing. J Consult Clin Psychol. 2004 Dec;72(6):1050–62. doi: 10.1037/0022-006X.72.6.1050. [DOI] [PubMed] [Google Scholar]
  • 14.Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. J Consult Clin Psychol. 2005 Feb;73(1):106–15. doi: 10.1037/0022-006X.73.1.106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Davis D. Does CME work? An analysis of the effect of educational activities on physician performance or health care outcomes. Int J Psychiatry Med. 1998 Jun 7;28(1):21–39. doi: 10.2190/UA3R-JX9W-MHR5-RC81. [DOI] [PubMed] [Google Scholar]
  • 16.Walters ST, Matson SA, Baer JS, Ziedonis DM. Effectiveness of workshop training for psychosocial addiction treatments: A systematic review. J Subst Abuse Treat. 2005 Dec;29(4):283–93. doi: 10.1016/j.jsat.2005.08.006. [DOI] [PubMed] [Google Scholar]
  • 17.Baer JS, Rosengren DB, Dunn CW, Wells EA, Ogle RL, Hartzler B. An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. Drug Alcohol Depend. 2004 Jan 7;73(1):99–106. doi: 10.1016/j.drugalcdep.2003.10.001. [DOI] [PubMed] [Google Scholar]
  • 18.Baer JS, Wells EA, Rosengren DB, Hartzler B, Beadnell B, Dunn C. Agency context and tailored training in technology transfer: A pilot evaluation of motivational interviewing for community counselors. J Subst Abuse Treat. 2009 Sep;37(2):191–202. doi: 10.1016/j.jsat.2009.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Mitcheson L, Bhavsar K, McCambridge J. Randomized trial of training and supervision in motivational interviewing with adolescent drug treatment practitioners. J Subst Abuse Treat. 2009 July;37(1):73–8. doi: 10.1016/j.jsat.2008.11.001. [DOI] [PubMed] [Google Scholar]
  • 20.Moyers TB, Manuel JK, Wilson PG, Hendrickson SML, Talcott W, Durand P. A randomized trial investigating training in motivational interviewing for behavioral health providers. Beh Cog Psychother. 2008 Nov 22;36(2):149–62. [Google Scholar]
  • 21.Andrzejewski ME, Kirby KC, Morral AR, Inguchi MY. Technology transfer through performance management: The effects of graphical feedback and positive reinforcement on drug treatment counselors’ behavior. Drug Alcohol Depend. 2001 Jul 1;63(2):179–86. doi: 10.1016/s0376-8716(00)00207-6. [DOI] [PubMed] [Google Scholar]
  • 22.Henggeler SW, Melton GB, Brondino MJ, Scherer DG, Hanley JH. Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. J Consult Clin Psychol. 1997 Oct;65(5):821–33. doi: 10.1037//0022-006x.65.5.821. [DOI] [PubMed] [Google Scholar]
  • 23.Keller DS, Galanter M. Technology transfer of network therapy to community-based addictions counselors. J Subst Abuse Treat. 1999 Mar;16(2):183–9. doi: 10.1016/s0740-5472(98)00044-0. [DOI] [PubMed] [Google Scholar]
  • 24.Morgenstern J, Morgan TJ, McCrady BS, Keller DS, Carroll KM. Manual-guided cognitive behavioral therapy training: A promising method for disseminating empirically supported substance abuse treatments to the practice community. Psychol Addict Beh. 2001 Jun;15(2):83–8. [PubMed] [Google Scholar]
  • 25.Addis ME, Wade WA, Hatgis C. Barriers to dissemination of evidence-based practices: Addressing practitioners’ concerns about manual-based psychotherapies. Clin Psychol Sci Pract. 1999 May 11;6(4):430–41. [Google Scholar]
  • 26.Fixsen DL, Naoom SF, Blasé KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231); 2005. [Google Scholar]
  • 27.Simpson DD, Flynn PM. Moving innovations into treatment: A stage-based approach to program change. J Subst Abuse Treat. 2007 Sept;33(2):111–20. doi: 10.1016/j.jsat.2006.12.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Squires DD, Gumbley SJ, Storti SA. Training substance abuse treatment organizations to adopt evidence-based practices: The Addiction Technology Transfer Center of New England Science to Service Laboratory. J Subst Abue Treat. 2008 Apr;34(3):293–301. doi: 10.1016/j.jsat.2007.04.010. [DOI] [PubMed] [Google Scholar]
  • 29.Coogle CL. The families who care project: Meeting the educational needs of African American and rural family caregivers dealing with dementia. Educ Gerontology. 2002 Jan;28(1):59–71. [Google Scholar]
  • 30.Gabel LL, Pearsol JA. The twin epidemics of substance use and HIV: A state-level response using a train-the-trainer model. Fam Pract. 1993 Dec;10(4):400–5. doi: 10.1093/fampra/10.4.400. [DOI] [PubMed] [Google Scholar]
  • 31.Jason LA, Lapp C, Kenney KK, Lupton T. An innovative approach in training health care workers to diagnose and manage patients with CFS. In: Alexandra AM, editor. Advances in psychology research. Hauppauge, NY: Nova Science Publishers; 2008. pp. 273–81. [Google Scholar]
  • 32.Nyamathi A, Vatsa M, Khakha DC, McNeese-Smith D, Leake B, Fahey JL. HIV knowledge improvement among nurses in India: Using a train-the-trainer program. J Assoc Nurses AIDS Care. 2008 Nov-Dec;19(6):443–49. doi: 10.1016/j.jana.2008.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Pullum JD, Sanddal ND, Obbink K. Training for rural prehospital providers: A retrospective analysis from Montana. Prehosp Emerg Care. 1999 Jul-Sep;3(3):231–38. doi: 10.1080/10903129908958942. [DOI] [PubMed] [Google Scholar]
  • 34.Tribbles LR, Smith AE, Manzi SC. Train-the-trainer for hospital-wide safety training. J Nurs Staff Dev. 1993 Nov-Dec;9(6):266–69. [PubMed] [Google Scholar]
  • 35.Virani R, Malloy P, Ferrell BR, Kelly K. Statewide efforts in promoting palliative care. J Palliat Med. 2008 Sep;11(7):991–96. doi: 10.1089/jpm.2007.0291. [DOI] [PubMed] [Google Scholar]
  • 36.Both-Kewley S, Gilman PA, Shaffer RA, Brodine SK. Evaluation of sexually transmitted disease/human immunodeficiency virus prevention train-the-trainer program. Mil Med. 2001 Apr;166(4):304–10. [PubMed] [Google Scholar]
  • 37.Coogle CL, Osgood NJ, Parham IA. A statewide model detection and prevention program for geriatric alcoholism and alcohol abuse: Increased knowledge among service providers. Community Ment Health J. 2000 April;36(2):137–48. doi: 10.1023/a:1001864926148. [DOI] [PubMed] [Google Scholar]
  • 38.Gadomski AM, Wolff D, Tripp M, Lewis C, Short LM. Changes in health care providers’ knowledge, attitudes, beliefs, and behaviors regarding domestic violence, following a multifaceted intervention. Acad Med. 2001 Oct;76(10):1045–52. doi: 10.1097/00001888-200110000-00015. [DOI] [PubMed] [Google Scholar]
  • 39.Moon RY, Calabrese T, Aird L. Reducing the risk of sudden infant death syndrome in child care and changing provider practices: Lessons learned from a demonstration project. Pediatrics. 2008 Oct;122(4):788–98. doi: 10.1542/peds.2007-3010. [DOI] [PubMed] [Google Scholar]
  • 40.Tziraki C, Graubard BI, Manley M, Kosary C, Moler JE, Edwards BK. Effect of training on adoption of cancer prevention nutrition-related activities by primary care practices: Results of a randomized, controlled study. J Gen Intern Med. 2000 Mar;15(3):155–62. doi: 10.1046/j.1525-1497.2000.03409.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Well M, Stokols D, McMahan S, Clitheroe C. Evaluation of a worksite injury and illness prevention program: Do the effects of the REACH OUT training program reach the employees? Journal of Occup Health Psychol. 1997 Jan;2(1):25–34. doi: 10.1037//1076-8998.2.1.25. [DOI] [PubMed] [Google Scholar]
  • 42.Zapata LB, Coreil J, Entrekin N. Evaluation of Triple Touch: An assessment of program delivery. Cancer Pract. 2001 Mar-Apr;9(Suppl 1):S23–S30. doi: 10.1046/j.1523-5394.2001.95105.x. [DOI] [PubMed] [Google Scholar]
  • 43.Rogers ES, Cohen BF, Danley KS, Hutchinson D, Anthony WA. Training mental health workers in psychiatric rehabilitation. Schizophr Bull. 1986 Dec 4;12(4):709–19. doi: 10.1093/schbul/12.4.709. [DOI] [PubMed] [Google Scholar]
  • 44.Hawkins KA, Sinha R. Can line clinicians master the conceptual complexities of dialectical behavior therapy? An evaluation of a state department of mental health training program. J Psychiatr Res. 1998 Sep 1;32(6):379–84. doi: 10.1016/s0022-3956(98)00030-2. [DOI] [PubMed] [Google Scholar]
  • 45.Hein D, Litt LC, Cohen L, Miele GM, Campbell A. Trauma services for women in substance abuse: an integrated approach. Washington DC: American Psychological Association; 2009. Preparing, training, and integrating staff in providing integrated treatment; pp. 143–161. [Google Scholar]
  • 46.Martino S, Brigham GS, Higgins C, Gallon S, Freese TE, Albright LM, et al. Partnerships and pathways of dissemination: The NIDA-SAMHSA Blending Initiative in the Clinical Trials Network. J Subst Abuse Treat. doi: 10.1016/j.jsat.2009.12.013. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Miller WR, Rollnick S, Moyers TB. Motivational interviewing (6-tape series) Albuquerque, New Mexico: University of New Mexico; 1998. [Google Scholar]
  • 48.Ball SA, Martino S, Nich C, Frankforter TL, Van Horn D, Crits-Christoph P, et al. Site matters: Motivational enhancement therapy in community drug abuse clinics. J Consult Clin Psychol. 2007 Aug;75(4):556–67. doi: 10.1037/0022-006X.75.4.556. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.Carroll KM, Ball SA, Nich C, Martino S, Frankforter TL, Farentinos C, et al. Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: A multisite effectiveness study. Drug Alcohol Depend. 2006 Feb 28;81(3):301–12. doi: 10.1016/j.drugalcdep.2005.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Carroll KM, Martino S, Suarez-Morales L, Ball SA, Miller WR, Añez L, et al. Multisite Randomized Controlled Effectiveness Trial of Motivational Enhancement for Spanish-Speaking Substance Users. J Consult Clin Psychol. 2009 Oct;77(5):993–9. doi: 10.1037/a0016489. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug Alcohol Depend. 2008 Jul 1;96(1-2):37–48. doi: 10.1016/j.drugalcdep.2008.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Santa Ana EJ, Carroll KM, Añez L, Paris M, Ball SA, Nich C, et al. Evaluating motivational enhancement therapy adherence and competence among Spanish-speaking therapists. Drug Alcohol Depend. 2009 Jul 1;103(1-2):44–51. doi: 10.1016/j.drugalcdep.2009.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol Assess. 1994 Dec;6(4):284–90. [Google Scholar]
  • 54.Raudenbush SW, Liu X. Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychol Methods. 2001 Dec;6(4):387–401. [PubMed] [Google Scholar]
  • 55.Feingold A. Effect sizes for growth-modeling analysis for controlled clinical trials in the same metric as for classical analysis. Psychol Methods. 2009 Mar;14(1):43–53. doi: 10.1037/a0014699. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Amrheim P, Miller WR, Yahne CE, Palmer M, Fulcher L. Client commitment language during motivational interviewing. J Consult Clin Psychol. 2003 Oct;71(5):862–78. doi: 10.1037/0022-006X.71.5.862. [DOI] [PubMed] [Google Scholar]
  • 57.Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–29. doi: 10.1037//0033-2909.86.2.420. [DOI] [PubMed] [Google Scholar]

RESOURCES