Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Nov 1.
Published in final edited form as: Am J Drug Alcohol Abuse. 2015 Aug 18;41(6):527–534. doi: 10.3109/00952990.2015.1062894

Adherence and competence in two manual-guided therapies for co-occurring substance use and posttraumatic stress disorders: Clinician factors and patient outcomes1

Andrea Meier a, Mark P McGovern b, Chantal Lambert-Harris a, Bethany McLeman c, Anna Franklin d, Elizabeth C Saunders c, Haiyi Xie e
PMCID: PMC4698972  NIHMSID: NIHMS744129  PMID: 26286351

Abstract

Background

The challenges of implementing and sustaining evidence-based therapies into routine practice have been well-documented.

Objectives

This study examines the relationship among clinician factors, quality of therapy delivery, and patient outcomes.

Methods

Within a randomized controlled trial, 121 patients with current co-occurring substance use and posttraumatic stress disorders were allocated to receive either manualized Integrated Cognitive Behavioral Therapy (ICBT) or Individual Addiction Counseling (IAC). Twenty-two clinicians from seven addiction treatment programs were trained and supervised to deliver both therapies. Clinician characteristics were assessed at baseline; clinician adherence and competence were assessed over the course of delivering both therapies; and patient outcomes were measured at baseline and 6-month follow-up.

Results

Although ICBT was delivered at acceptable levels, clinicians were significantly more adherent to IAC (p<.05). At session 1, gender (p<.05) and education level (p<.05) were predictive of clinician adherence and competence across both therapies. Adherence and competence at session 1 in either therapy were significantly predictive of patient outcomes. ICBT adherence (p<.05) and competence (p<.01) were predictive of PTSD symptom reduction, whereas IAC adherence (p<.01) and competence (p<.01) were associated with decreased drug problem severity.

Conclusions

The differential impact of adherence and competence for both therapy types is consistent with their purported primary target: ICBT for PTSD and IAC for substance use. These findings also suggest the benefits of considering clinician factors when implementing manual-guided therapies. Future research should focus on diverse clinician samples, randomization of clinicians to therapy type, and prospective designs to evaluate models of supervision and quality monitoring.

Keywords: Therapist adherence and competence, Clinician characteristics, Integrated psychosocial treatments

1. Introduction

For persons with substance use disorders, psychosocial therapies, such as motivational interviewing, cognitive behavioral therapy, twelve-step facilitation, and contingency management have demonstrated consistent effectiveness in research trials and meta-analyses (1-4). Comorbidity with posttraumatic stress disorder (PTSD) is prevalent in treatment settings, asssoicated with negative outcomes, and integrated interventions have been developed (5-7). However, these evidence-based therapies are not widely available in routine practice (8-10). Barriers to implementation seem formidable (11-14). Even if clinicians are adequately trained in evidence-based therapies, sustaining acceptable quality is challenged by the required clinical supervision and fidelity monitoring (15-17).

A potential motivating connection for clinicians is the association between therapy fidelity and actual patient outcomes. In other words, “if I deliver it as designed, my patients get better.” Such information could bolster enthusiasm to alter a clinician's typical practice (18). Unfortunately, studies on treatment fidelity have shown mixed results on patient outcomes (19, 20). Some have indicated a positive relationship between adherence and/or competence on patient outcomes (17, 21-24). Other studies have found more mixed effects (25-28). Lastly, studies have also found negative relationships (29, 30), curvilinear relationship (31) or no relationship at all (21, 32, 33).

Among many factors that could account for these equivocal results, variability in the type of clinicians delivering the interventions could be a major factor (17, 20). Clinician attitudes have been found to be positively associated with implementing evidence-based treatments (34, 35), but for other characteristics, such as demographics and professional background, findings are less conclusive. Some studies have found that clinician gender (36), interpersonal skills (37, 38), years of experience (35, 39), or congruence with existing practice (36, 40, 41) positively impact adoption and fidelity. Others studies have found no association related to professional background, theoretical orientation, or clinician demographic factors such as age and gender (38, 39, 42). The role of clinician factors in therapy research has been inadequately addressed and warrants further study (43, 44).

The present study evaluates typical community addiction treatment program clinicians who were trained and supervised to deliver two treatments: one more congruent with their existing practice (substance use-focused) and one less congruent (integrated mental health and substance use-focused).

The current study addresses the following questions:

  1. Does clinician adherence and competence vary by therapy type?

  2. Do baseline clinician factors, such as gender and education level, predict adherence and competence?

  3. Are adherence and competence ratings associated with patient outcomes, such as reduction in drug or alcohol problem severity, or psychiatric symptoms?

2. Material and Methods

2.1 Design

This is a two-group repeated measures design to evaluate clinicians who served as study therapists in a randomized controlled trial (RCT). Clinicians (n=22) treated randomized patients (n=121) who met diagnostic criteria for both a substance use disorder and posttraumatic stress disorder (PTSD). The RCT compared two manual-guided therapies: one focused on substance use disorders only, and the other on co-occurring substance use and posttraumatic stress disorders (PTSD). Clinicians were assessed at baseline on demographic and work experience factors. They were trained and supervised in the delivery of two manual-guided therapies: Integrated Cognitive Behavioral Therapy (ICBT) and Individual Addiction Counseling (IAC). The same clinicians delivered both therapies to study patients. Repeated measures included ICBT and IAC adherence and competence ratings of audio-recorded sessions, as well as patient outcomes at 6-month post-baseline follow-up. Data analyses focused on: 1) differential adherence and competence by the two therapy types (ICBT and IAC), 2) predictors of adherence and competence, and 3) the relationship among clinician adherence and competence to patient outcomes.

2.2 Sample and Settings

Patient and clinician data were obtained from seven public non-profit community outpatient addiction treatment agencies located within two US states. The study sample consisted of one hundred twenty-one patients meeting current diagnostic criteria and admitted for substance use disorder treatment, and meeting additional current diagnostic criteria for PTSD. Participants consented to receive one of two evidence-based psychosocial treatments as an adjunct to standard intensive outpatient program care. Twenty-two clinicians who were employees of the programs delivered the two interventions.

2.3 Therapy interventions

Integrated Cognitive Behavioral Therapy (ICBT), is an 8-12 session individual format, manual-guided therapy consisting of three learning and skill components designed to reduce PTSD symptoms and substance use: 1) patient education about PTSD, substance use, and their interaction; 2) mindful relaxation: An anxiety reduction technique focused on centering and breathing retraining; and 3) cognitive restructuring: A cognitive behavioral approach evaluating the interactions of cognitions, emotions, and behaviors. Research has supported ICBT's feasibility in community settings (45-48) and efficacy in reducing PTSD and substance use symptoms (7, 49, 50). ICBT's primary focus on PTSD, is different, or less congruent, with community addiction clinicians’ typical practice.

Individual Addiction Counseling (IAC), is an 8-12 session individual format, manual-guided therapy targeting substance use in a stage-based approach (Treatment initiation, Early abstinence, Maintaining abstinence, and Recovery). IAC is an adaptation of the Individual Drug Counseling (IDC) treatment used in the National Institute on Drug Abuse Cocaine Collaborative Study and the Twelve Step Facilitation (TSF) therapy used in the National Institute on Alcohol Abuse and Alcoholism Project Match study (51, 52). IAC focuses on addiction only and is more congruent with community addiction clinicians’ typical practice.

Treatment length was determined by the number of sessions it took to complete all of the modules in ICBT or IAC, which ranged from 8-12 sessions. Patient attrition in less than 8 sessions was always patient determined, not clinician recommended. The variation in dose from 8-12 was based on an individual patient's pace to complete all modules, which was determined by a collaborative process between clinician and patient

2.4 Measures

2.4.1 Clinician characteristics

Clinician Background Form

The Clinician Background Form is a 32-item self-report form on clinician characteristics including demographics (age, gender, race, and ethnicity), education level, work experience, years of experience, and discipline. This form was constructed specifically for this study.

2.4.2 Adherence and competence ratings

ICBT Adherence and Competence Rating Scale

This scale consists of 13 items on ICBT specific skills on a 7-point Likert scale (Adherence: 1 = not at all to 7 = extensively; Competence: 1 = very poor to 7 = excellent). The items rated are based on the specific module(s) covered in the therapy session. The scale also includes four items on generic skills (Competence: 1 = very poor to 7 = excellent). The cutoff for adequate adherence and competence are an average rating of “4” and above. This scale has acceptable inter-rater reliability based on this randomized controlled trial (Adherence Cronbach's alpha: 0.85, ICC: 0.72; Competence Cronbach's alpha: 0.83, ICC: 0.72). It has been used in previous studies of ICBT (7, 49, 50).

IAC Adherence and Competence Rating Scale

This scale has two items on IAC global fidelity on a 7-point Likert scale (Adherence: 1 = not at all to 7 = extensively) and skill (Competence: 1 = very poor to 7 = excellent). Ratings are not specific to module(s) covered in the therapy session, but rather on global ratings of IAC. The scale also includes four items on generic skills (Competence: 1 = very poor to 7 = excellent). The cutoff for adequate adherence and competence are an average rating of “4” and above. This scale also has acceptable inter-rater reliability based on the current randomized controlled trial (Adherence Cronbach's alpha: 0.80, ICC: 0.67; Cronbach's alpha: 0.83, Competence ICC: 0.71). It has been used in a previous study of IAC (47).

2.4.3. Patient outcome measures

Clinician Administered PTSD Scale (CAPS)

The CAPS (53) is a structured diagnostic interview for PTSD severity assessment. The CAPS total score is a reliable and valid measure of PTSD symptom severity.

Addiction Severity Index (ASI)

The ASI is a self-administered standardized assessment that is reliable and valid, and used to measure Drug Problem Severity (ASI Drug Severity Composite Score) and Alcohol Problem Severity (ASI Alcohol Severity Composite Score) (54-56).

2.5 Procedure

Clinicians volunteered to serve as study therapists in a randomized controlled trial comparing two manual-guided therapies (one focused on PTSD and substance use disorders and one focused on substance use disorders only) both with treatment-as-usual, and to treatment-as-usual alone. Any front-line clinician who volunteered was included, regardless of education level, background, or discipline. There were no exclusion criteria based on prior manualized therapy experience, certification, or licensure.

Clinicians received an initial on-site, 2-hour training in both interventions led by an expert and member of the research team. Training consisted of a didactic overview of substance use disorders and PTSD, walk through of the ICBT and IAC clinician manuals and patient workbooks, and included an emphasis on practice-based learning. Clinicians completed the Clinician Background Form accompanied by an information sheet describing the form as voluntary and confidential. Immediately after training, clinicians began meeting with patients to deliver both interventions. All therapy sessions were audio-recorded and transmitted securely to the research team.

Five, independent therapist-blind raters audited and rated 25% of full audio-recorded sessions (sessions 1 and 4) using the ICBT and IAC Adherence and Competence Rating Scales. Raters included three undergraduate interns, one bachelor's-level intern, and a member of the research team. They attended an initial two-day training led by one of the developers of ICBT and IAC, which consisted of review of the adherence and competence rating scales and instructional manuals, and in vivo ratings of audio-recorded ICBT and IAC sessions. After the training, raters independently rated ten ICBT and ten IAC sessions, and scores were matched with expert ratings for inter-rater reliability. Raters met with an expert in the rating scales once weekly for fidelity and quality monitoring to the rating scales.

Clinical supervision was also led by an expert and member of the research team. Supervision consisted of weekly alternating on-site group supervision and individual telephone supervision. Supervision included review of technique, monitoring clinician and patient alliance, focusing on aspects of engagement and setting mutual goals, reviewing session audio-recordings, and providing feedback using the adherence and competence rating scales.

Patients completed therapy outcome measures (CAPS and ASI) at baseline and 6-months post-baseline follow-up. Measures were administered by research staff at program sites.

The study was conducted in accordance with all human subject protections and good clinical practices (e.g., Helsinki Declaration, Belmont Principles, and Nuremberg Code). The Trustees of Dartmouth College Committee for the Protection of Human Subjects (CPHS) (i.e., institutional review board) approved the collection, analysis, and reporting of these data.

2.6 Data analysis

IBM SPSS 21.0 was used to conduct statistical analyses (57). Adherence and competence scores were rated for each clinician for each treated case. Therefore, each row in the database represented a client and included adherence and competence scores for their individual sessions with the clinician (at session 1 and 4). In addition, clinicians’ caseloads varied within sites.

Clinician characteristics were examined using descriptive statistics. We used simple mean difference tests (t-test) to determine differences in adherence and competence scores among gender, education levels, and treatment type. For this set of analyses, adherence or competence scores were the outcome variables, and gender, education level, or treatment type were the predictor variables.

To determine whether or not treatment completion status was influenced by clinician fidelity, we tested group difference between completers (ICBT: N=62; IAC N=58) and dropouts (ICBT: N=29; IAC N=32), and we found no significant differences in clinician adherence and competence scores by treatment completion status. We also found no significant differences in treatment completion status between treatment groups. Therefore, we did not include treatment completion status as a covariate in our analyses.

Simple linear regression analyses were utilized to determine if clinician characteristics were predictive of clinician adherence and competence. In order to simplify the analyses and enhance statistical power, we examine clinician characteristics across therapies. To further examine the relationship between clinician characteristics with adherence or competence scores, multiple regression analyses were used.

Finally, the relationship between adherence and competence scores and patient-level change in PTSD and substance use severity by therapy type was examined. This included six independent linear regressions with adherence/competence scores at sessions 1 and 4 as the predictor variable of patient outcomes. The majority (79.6%) of participants had not completed ICBT or IAC by the 3-month mark. Therefore, the 6-month follow-up became the de facto measure of post-treatment effects.

The Generalized Estimating Equations (GEE) method was used to account for clustering effects of site and clinician.

3. Results

3.1 Clinician characteristics

All clinicians were Caucasian (Non-Hispanic) and predominantly female. Clinician characteristics are presented in Table 1. This is a relatively experienced sample of clinicians, as the mean number of years in the profession was slightly over seven years. Consistent with community addiction treatment workforce, the majority of clinicians focused on substance use only in their practice (63.6%).

Table 1.

Clinician characteristics (n=22)

Female (n=18) Male (n=4) Overall (n=22) χ2/t-value
Work experience, n (%)
    Substance abuse and mental health 4 (22.2%) 0 (0.0%) 4 (18.2%) 1.09
    Mental health only 3 (16.7%) 1 (25.0%) 4 (18.2%) 0.15
    Substance abuse only 11 (61.1%) 3 (75.0%) 14 (63.6%) 0.27
Education, n (%)
    Bachelor's 2 (11. 1%) 0 (0.0%) 2 (9.1%) 1.09
    Master's 14 (77.8%) 4 (100.0%) 18 (81.8%)
    Doctorate 2 (11. 1%) 0 (0.0%) 2 (9.1%)
Mean [sd] Mean [sd] Mean [sd]
Age 43.8 [10.3] 44.0 [5.0] 43.9 [9.5] 0.05
Years in profession 7.8 [5.3] 5.3 [3.2] 7.4 [5.0] −0.94
Open in a new tab

3.2 Differences in adherence and competence ratings between ICBT and IAC

Both ICBT and IAC therapies were delivered on average above the adequate adherence and competence level (≥4 on the ICBT and IAC adherence and competence rating scales) (see Table 2). IAC was delivered with slightly greater adherence at session 1 than ICBT (6.49(0.79) vs. 6.18(0.90)) (t=−1.99, df=116, p<.05). This differential rating would appear associated with the primary focus in practice of the participating clinicians.

Table 2.

Adherence and competence ratings by treatment session and therapy type (n=121 patients)

ICBT IAC t-value
Session 1 m[sd] m[sd]
Adherence 6.18 [0.90] 6.49 [0.79] −1.99*
Competence 5.67 [1.21] 5.95 [1.19] −1.95
Session 4 m[sd] m[sd]
Adherence 6.12 [1.04] 6.46 [0.41] −1.25
Competence 6.00 [0.96] 6.08 [0.67] −0.42
Open in a new tab
*

p=.05

There was no significant difference in patient dropout by treatment type. We also found no significant differences in patient treatment completion by clinician adherence and competence scores for either ICBT or IAC.

3.3 Clinician factors predicting adherence and competence ratings

Gender was predictive of higher adherence at session 1 (Beta=0.21, p<.05) and competence (Beta=0.18, p<.05) ratings for both ICBT and IAC therapies. Education level was also predictive of higher adherence and competence ratings at session 1 (Beta=−0.21, p<.05; Beta=−.240, p<.01). Clinicians with a bachelor's or master's degree were more competent at session 1 than doctorate-level clinicians delivering the ICBT and IAC therapy (t=4.81, df=22, p<.001; t=2.48, df=97, p<.05). Session 1 adherence scores among bachelor's-level clinicians were statistically greater than master's-level clinicians (t=2.37, df=54.65, p<.05). No statistical differences were found on gender or education level in adherence and competence at session 4.

In multiple regression analyses, at session 1, gender and education accounted for 8% of the variance in adherence (F=5.08, df=2, p<.01) and 8% of the variance in competence (F=5.22, df=2, p<.01). Gender predicted adherence (Beta=.190, p<.05), but did not predict competence scores (Beta=.161, p=.08). Education predicted adherence scores (Beta=−.197, p<.05), and competence scores (Beta=−.227, p≤.01). At session 4, gender and education accounted for 11% of the variance in adherence (F=4.88, df=2, p≤.01) and 8% of the variance in competence (F=3.09, df=2, p≤.05). Gender predicted adherence (Beta=.296, p<.01) and competence (Beta=.274, p<.05). Education did not significantly predict adherence or competence to the therapies.

3.4 Relationship between adherence and competence, and patient outcomes

Higher ICBT adherence (Beta=0.36, p<.05) and competence (Beta=0.44, p<.01) ratings at session 1 predicted greater reductions in PTSD. Alcohol problem severity reduction was predicted by lower ICBT competence ratings at session 4 (Beta=−0.40, p<.05). A negative relationship existed between ICBT session 4 adherence and competence ratings and ASI drug severity reduction (Beta=−0.42, p<.05; Beta=−0.37, p<.05).

There were no significant relationships between IAC adherence and competence and PTSD symptom severity. But higher IAC adherence and competence ratings at session 1 predicted greater drug severity reduction (adherence: Beta=0.42, p<.01; competence: Beta=0.50, p<.01).

The GEE method controls for clustering by site and clinician. The findings remained consistent across the simple linear regression and GEE analyses with the following exceptions: 1) for ICBT, adherence/competence at session 4 lost statistically significant predictive potential of drug problem severity reduction (Slope=−0.05, χ2=3.53, df=1, p=.060; Slope=−0.05, χ2=3.64, df=1, p=.056). But clinician adherence to ICBT at session 4 predicted reductions in alcohol problem severity (Slope=−0.12, χ2=3.73, df=1, p≤.05); and 2) for IAC, we found that competence at session 4 predicted greater drug severity reductions (Slope=0.04, χ2=4.62, df=1, p≤.05).

4. Discussion

4.1 Summary of findings

These data revealed that with training and supervision, clinicians employed by community treatment programs were able to deliver either a primary substance use-focused intervention or an integrated PTSD and substance use therapy with acceptable quality. Overall, both therapies were delivered with at least adequate adherence and competence.

With respect to the first research question, clinician adherence did vary by therapy type. Clinicians delivered the Individual Addiction Counseling (IAC) with greater adherence than Integrated Cognitive Behavioral Therapy (ICBT). This is consistent with previous research suggesting clinicians have more difficulty with therapies incongruous or different from their typical practice (12, 14, 58). Cognitive behavioral approaches have been especially challenging for addiction counselors to adequately implement and sustain (59, 60). However, it is conceivable that over the 10-15 years since these studies were conducted, cognitive behavioral strategies have become more acceptable to addiction counselors, resulting in adequate ICBT fidelity findings. Furthermore, in the present study, clinician competence was not found to vary by type of therapy.

In addressing the second research question, adherence and competence outcomes were predicted by clinician factors, specifically by clinician gender and education level. Female clinicians had higher adherence and competence ratings. Bachelor's-level clinicians had significantly higher ratings (across both therapies), which is contradictory to a recent study that found clinicians with graduate degrees have higher fidelity to manualized interventions (61). The few studies that have examined the relationship between clinician factors and fidelity have found the strongest and most consistent predictor to be clinician experience, particularly the intervention's compatibility with the clinician's identified theoretical orientation (36, 40, 42). This was not supported by our findings.

In regard to the third and final research question, we found clinician adherence and competence were both positively and negatively associated with patient outcomes. Previous research on the association of adherence and competence with patient outcomes has likewise been equivocal perhaps because of the multitude of variables that may affect outcomes (19, 20). The majority of our findings are congruent with positive relationships of adherence and competence on outcome (17, 22-24). Higher ICBT adherence and/or competence were associated with greater reductions in PTSD symptom severity. IAC adherence and competence were not correlated with PTSD symptom improvement, but were related to improvements in drug use problem severity. This is an important finding. ICBT is a therapy primarily designed to address PTSD symptom severity (in addition to substance use). Whereas IAC does not target PTSD but instead is exclusively focused on drug and alcohol issues. The relationship between ICBT adherence and PTSD outcomes, and IAC adherence and drug use outcomes, supports the targeted efficacy of both therapies. However, perhaps related to the primary focus of the ICBT therapy, and consistent with literature supporting a negative relationship between adherence and/or competence on patient outcomes (29, 30), higher ICBT adherence and competence were associated with less improvement in substance use.

Clinician characteristics appear to be especially predictive of adherence and competence at the initial session (session 1 compared with session 4). In addition, higher adherence and competence ratings at earlier sessions appear to be particularly predictive of patient-level outcomes across both therapies, indicating that fidelity to interventions early on is important. Using data that demonstrates a causal effect of therapist adherence and competence in a manual-guided approach on patient outcomes is an important tool to reinforce clinicians. This would clearly appeal to a value of providing the best care possible for their patients, and therapy that produces the best outcomes possible.

4.2 Limitations

The study's limitations are related to the sample, measures, and research design. Although we sampled 121 patients across seven community addiction treatment programs, data were limited to 22 clinicians for adherence and competence outcomes. These clinicians were primarily Caucasian females with master's-level of education. Patients were also primarily Caucasian. A more diverse and broadly sampled group of clinicians and patients may expand confidence in the generalizability of our findings.

Also related to clinician sampling issues, we did not stratify in recruiting clinicians for baseline factors such as addiction vs. mental health, traditional Twelve-Step or cognitive behavioral orientations, or for years of professional experience. Furthermore, clinicians were assigned cases by their program coordinator and were not randomly assigned cases by therapy type: ICBT or IAC. Although we found no anecdotal evidence for therapist preference to take one or the other type of case, only by experimental design could this potential bias be addressed.

Sampling of therapy sessions for clinician adherence and competence was limited to 25% of recorded sessions. Although there are no clear guidelines on how to audit for fidelity in therapy research, the convention is 25% of sessions. However, it is unknown whether this is 25% of all sessions or 25% of each session.

While the study deployed two “gold standard” measures to evaluate PTSD (CAPS) and substance use (ASI) outcomes, the assessments took place at baseline and 6-months post-baseline intervals. Within-treatment patient outcome measures could be more clearly linked to therapy sessions. These may be a more accurate test of the effect of fidelity and skill. In addition, the patients enrolled in this trial were receiving other treatment services within an intensive outpatient program, including group and individual therapies, addiction and/or psychotropic medication, case management services, and were also asked to attend peer recovery support meetings in the community. These other services confound the interpretation of any causal link between either therapy adherence or competence with patient-level outcomes. These data were gathered as part of a randomized controlled trial. Clinicians volunteered so may have been a biased sample (i.e., more experienced with or interested in evidence-based treatments). They were also exposed to a high level of scrutiny and oversight that may not be generalizable to routine, community practice. Furthermore, the study was designed to evaluate patient-level outcomes, not to study supervision or clinician-level factors. A study designed to experimentally evaluate supervision may yield different findings.

4.3 Future research

Over the past 20 years, systems, organizations, and clinicians have been motivated to integrate evidence-based treatments for co-occurring disorders (41, 62). Historically, addiction and mental health services have been split into separate silos based on policy, workforce, treatment philosophy, and financing. Our data show typical community clinicians can deliver both an integrated and addiction focused therapies with acceptable quality; furthermore, the quality of delivery directly impacts patient-level substance use and psychiatric outcomes.

Additional exploration of the mediators and moderators of adherence and competence, beyond clinician factors, could further interpretation. More research on organizational-level factors (15, 34, 63) and patient-level factors, including symptom severity and other comorbid conditions (19, 26, 27), is needed to understand the determinants of clinician adoption and sustainment of evidence-based therapies.

Future implementation research should focus on utilizing these front-line clinicians and gathering data on clinician-level attitudes and experiences with the therapies being delivered to aid in the generalizability of findings and sustainability of research efforts (13). Research should account not only for therapist selection, but also the requisite training, supervision, and fidelity monitoring models necessary to deliver the therapy in question (17, 64). There are no standard approaches to this fundamental process at the present time.

Furthermore, supervision and fidelity monitoring models are not addressing sustainability (e.g., what is realistic beyond or at the conclusion of clinical trials). The field would benefit from experimental studies focused on community clinician samples, training, and supervision models; sustainable fidelity monitoring strategies; and moderators of outcomes to facilitate the effective and economical implementation of evidence-based treatments into routine care.

Table 3.

Overall adherence and competence ratings by clinician gender and education level

Adherence
Session 1 β Session 4 β
Gender .206* .307
Female 6.38 (0.79) 6.35 (0.70)
Male 5.75 (1.36) 5.42 (1.55)
Education −.213* −.163
Bachelor's 6.63 (0.44) 6.42 (0.50)
Master's 6.31 (0.90) 6.30 (0.82)
Doctorate 5.60 (0.89) 5.68 (1.36)
Competence
Session 1 β Session 4 β
Gender .180* .274
Female 5.87 (1.06) 6.10 (0.72)
Male 5.10 (2.18) 5.25 (1.58)
Education −.240** −.014
Bachelor's 6.21 (0.71) 5.97 (0.58)
Master's 5.80 (1.24) 6.08 (0.85)
Doctorate 4.40 (0.89) 5.73 (1.23)
Open in a new tab
*

p<.05

**

p<.01

Table 4.

Relationship between clinician adherence and competence ratings and patient outcomes: Comparison between therapy type (n=121 patients)

ICBT IAC
Outcomesa Session 1 (n=61) Session 4 (n=41) Session 1 (n=57) Session 4 (n=38)
Adherence
β β β β
CAPS total score 0.36* 0.07 0.21 0.22
Alcohol composite score ASI 0.05 −0.33 0.22 −0.25
Drug composite score ASI −0.04 −0.42* 0.42** 0.18
Competence
β β β β
CAPS total score 0.44** 0.09 0.19 0.19
Alcohol composite score ASI −0.03 −0.40* 0.06 −0.17
Drug composite score ASI −0.05 −0.37* 0.50** 0.29
Open in a new tab

Linear regression Beta:

*

p<.05

**

p<.01

a

Change score for baseline to 6-month follow up (baseline value - 6-month follow up value = change score)

Acknowledgments

The authors are grateful to the administration, clinicians and patients from Central Vermont Substance Abuse Services, Clara Martin Center's Quitting Time, Dartmouth-Hitchcock Addiction Treatment Program, Health Care and Rehabilitation Services, HowardCenter, Rutland Mental Health Services’ Evergreen Program, and Starting Now at the Brattleboro Retreat. We also acknowledge the assistance of Melissa An, Anna Franklin, Brandon Harrington, and Eunhee Kim who served as adherence and competence raters.

Role of Funding Source

This research was supported by NIDA R01 DA027650 (McGovern, PI).

Footnotes

1

Presentation at the College on Problems of Drug Dependence Annual Meeting in San Diego, California (June 17, 2013)

Contributors

Meier, McGovern, and Lambert-Harris wrote the manuscript. McGovern designed the study and wrote the protocol. Meier, Franklin, and Saunders managed the literature searches and summaries of previous related work. Lambert-Harris, McLeman, and Xie performed the statistical analyses. All authors contributed to and have approved the final manuscript.

Conflict of Interest

All authors report no conflicts of interest.

Clinical trials NCT01457391.

REFERENCES

  • 1.Ferri M, Amato L, Davoli M. Alcoholics Anonymous and other 12-step programmes for alcohol dependence. The Cochrane Database of Systematic Reviews. 2006;3(3):CD005032. doi: 10.1002/14651858.CD005032.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Hofmann SG, Asnaani A, Vonk IJ, Sawyer AT, Fang A. The efficacy of cognitive behavioral therapy: A review of meta-analyses. Cognitive Therapy and Research. 2012;36(5):427–440. doi: 10.1007/s10608-012-9476-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Magill M, Ray LA. Cognitive-behavioral treatment with adult alcohol and illicit drug users: A meta-analysis of randomized controlled trials. Journal of Studies on Alcohol and Drugs. 2009;70(4):516–527. doi: 10.15288/jsad.2009.70.516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Prendergast M, Podus D, Finney J, Greenwell L, Roll J. Contingency management for treatment of substance use disorders: A meta-analysis. Addiction. 2006;101(11):1546–1560. doi: 10.1111/j.1360-0443.2006.01581.x. [DOI] [PubMed] [Google Scholar]
  • 5.Brady KT, Dansky BS, Back SE, Foa EB, Carroll KM. Exposure therapy in the treatment of PTSD among cocaine-dependent individuals: Preliminary findings. Journal of Substance Abuse Treatment. 2001;21(1):47–54. doi: 10.1016/s0740-5472(01)00182-9. [DOI] [PubMed] [Google Scholar]
  • 6.Najavits LM, Weiss RD, Shaw SR, Muenz LR. Seeking Safety: Outcome of a new cognitive-behavioral psychotherapy for women with posttraumatic stress disorder and substance dependence. Journal of Traumatic Stress. 1998;11(3):437–456. doi: 10.1023/A:1024496427434. [DOI] [PubMed] [Google Scholar]
  • 7.McGovern MP, Lambert-Harris C, Xie H, Meier A, McLeman B, Saunders EC. A randomized controlled trial of treatments for co-occurring substance use disorders and PTSD. Addiction. 2015 doi: 10.1111/add.12943. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Carroll KM. Dissemination of evidence-based practices: How far we've come, and how much further we've got to go. Addiction. 2012;107(6):1031–1033. doi: 10.1111/j.1360-0443.2011.03755.x. [DOI] [PubMed] [Google Scholar]
  • 9.Finney JW, Hagedorn HJ. Introduction to a special section on implementing evidence-based interventions for substance use disorders. Psychology of Addictive Behaviors. 2011;25(2):191–193. doi: 10.1037/a0023949. [DOI] [PubMed] [Google Scholar]
  • 10.Garner BR. Research on the diffusion of evidence-based treatments within substance abuse treatment: A systematic review. Journal of Substance Abuse Treatment. 2009;36(4):376–399. doi: 10.1016/j.jsat.2008.08.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Comer JS, Barlow DH. The occasional case against broad dissemination and implementation: Retaining a role for specialty care in the delivery of psychological treatments. The American Psychologist. 2014;69(1):1–18. doi: 10.1037/a0033582. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Gallo KP, Barlow DH. Factors involved in clinician adoption and nonadoption of evidence-based interventions in mental health. Clinical Psychology: Science and Practice. 2012;19(1):93–106. [Google Scholar]
  • 13.Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning clinical science: Unifying the discipline to improve the public health. Clinical Psychological Science. 2014;2(1):22–34. doi: 10.1177/2167702613497932. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Stirman SW, Crits-Christoph P, DeRubeis RJ. Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice. 2004;11(4):343–359. [Google Scholar]
  • 15.Manuel JK, Hagedorn HJ, Finney JW. Implementing evidence-based psychosocial treatment in specialty substance use disorder care. Psychology of Addictive Behaviors. 2011;25(2):225–237. doi: 10.1037/a0022398. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Olmstead TA, Abraham AJ, Martino S, Roman PM. Counselor training in several evidence-based psychosocial addiction treatments in private US substance abuse treatment centers. Drug and Alcohol Dependence. 2012;120(1-3):149–154. doi: 10.1016/j.drugalcdep.2011.07.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Schoenwald SK, Garland AF. A review of treatment adherence measurement methods. Psychological Assessment. 2013;25(1):146–156. doi: 10.1037/a0029715. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Carroll KM, Rounsaville BJ. A vision of the next generation of behavioral therapies research in the addictions. Addiction. 2007;102(6):850–862. doi: 10.1111/j.1360-0443.2007.01798.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: Issues and research recommendations. Clinical Psychology: Science and Practice. 2005;12(4):365–383. [Google Scholar]
  • 20.Webb CA, Derubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2010;78(2):200–211. doi: 10.1037/a0018912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Barber JP, Crits-Christoph P, Luborsky L. Effects of therapist adherence and competence on patient outcome in brief dynamic therapy. Journal of Consulting and Clinical Psychology. 1996;64(3):619–622. doi: 10.1037//0022-006x.64.3.619. [DOI] [PubMed] [Google Scholar]
  • 22.Kuyken W, Tsivrikos D. Therapist competence, comorbidity and cognitive-behavioral therapy for depression. Psychotherapy and Psychosomatics. 2009;78(1):42–48. doi: 10.1159/000172619. [DOI] [PubMed] [Google Scholar]
  • 23.Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug and Alcohol Dependence. 2008;96(1-2):37–48. doi: 10.1016/j.drugalcdep.2008.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Trepka C, Rees A, Shapiro DA, Hardy GE, Barkham M. Therapist competence and outcome of cognitive therapy for depression. Cognitive Therapy and Research. 2004;28(2):143–157. [Google Scholar]
  • 25.Feeley M, DeRubeis RJ, Gelfand LA. The temporal relation of adherence and alliance to symptom change in cognitive therapy for depression. Journal of Consulting and Clinical Psychology. 1999;67(4):578–582. doi: 10.1037//0022-006x.67.4.578. [DOI] [PubMed] [Google Scholar]
  • 26.Huppert JD, Barlow DH, Gorman JM, Shear MK, Woods SW. The interaction of motivation and therapist adherence predicts outcome in cognitive behavioral therapy for panic disorder: Preliminary findings. Cognitive and Behavioral Practice. 2006;13(3):198–204. [Google Scholar]
  • 27.Imel ZE, Baer JS, Martino S, Ball SA, Carroll KM. Mutual influence in therapist competence and adherence to motivational enhancement therapy. Drug and Alcohol Dependence. 2011;115(3):229–236. doi: 10.1016/j.drugalcdep.2010.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Strunk DR, Brotman MA, DeRubeis RJ, Hollon SD. Therapist competence in cognitive therapy for depression: Predicting subsequent symptom change. Journal of Consulting and Clinical Psychology. 2010;78(3):429–437. doi: 10.1037/a0019631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Barber JP, Gallop R, Crits-Christoph P, Barrett MS, Klostermann S, McCarthy KS, Sharpless BA. The role of the alliance and techniques in predicting outcome of supportive-expressive dynamic therapy for cocaine dependence. Pychoanalytic Psychology. 2008;25(3) doi: 10.1037/0736-9735.25.3.483. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Svartberg M, Stiles TC. Therapeutic alliance, therapist competence, and client change in short-term anxiety-provoking psychotherapy. Psychotherapy Research. 1994;4(1):20–33. [Google Scholar]
  • 31.Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase M, Weiss RD, Connolly MB. The role of therapist adherence, therapist competence, and the alliance in predicting outcome of individual drug counseling: Results from the NIDA Collaborative Cocaine Treatment Study. Psychoanalytic Psychology. 2006;16:229–240. [Google Scholar]
  • 32.Loeb KL, Wilson GT, Labouvie E, Pratt EM, Hayaki J, Walsh BT, Agras WS, Fairburn CG. Therapeutic alliance and treatment adherence in two interventions for bulimia nervosa: A study of process and outcome. Journal of Consulting and Clinical Psychology. 2005;73(6):1097–1107. doi: 10.1037/0022-006X.73.6.1097. [DOI] [PubMed] [Google Scholar]
  • 33.Svartberg M, Stiles TC. Predicting patient change from therapist competence and patient-therapist complementarity in short-term anxiety-provoking psychotherapy: A pilot study. Journal of Consulting and Clinical Psychology. 1992;60(2):304–307. doi: 10.1037//0022-006x.60.2.304. [DOI] [PubMed] [Google Scholar]
  • 34.Aarons GA, Glisson C, Green PD, Hoagwood K, Kelleher KJ, Landsverk JA. The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science. 2012;7(1):56. doi: 10.1186/1748-5908-7-56. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Ball S, Bachrach K, DeCarlo J, Farentinos C, Keen M, McSherry T, Polcin D, Snead N, Sockriter R, Wrigley P, Zammarelli L, Carroll KM. Characteristics, beliefs, and practices of community clinicians trained to provide manual-guided therapy for substance abusers. Journal of Substance Abuse Treatment. 2002;23(4):309–318. doi: 10.1016/s0740-5472(02)00281-7. [DOI] [PubMed] [Google Scholar]
  • 36.James IA, Blackburn IM, Milne DL, Reichfelt FK. Moderators of trainee therapists’ competence in cognitive therapy. The British Journal of Clinical Psychology. 2001;40(Pt 2):131–141. doi: 10.1348/014466501163580. [DOI] [PubMed] [Google Scholar]
  • 37.Moyers TB, Miller WR. Is low therapist empathy toxic? Psychology of Addictive Behaviors. 2013;27(3):878–884. doi: 10.1037/a0030274. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Najavits LM, Weiss RD. Variations in therapist effectiveness in the treatment of patients with substance use disorders: An empirical review. Addiction. 1994;89(6):679–688. doi: 10.1111/j.1360-0443.1994.tb00954.x. [DOI] [PubMed] [Google Scholar]
  • 39.Huppert JD, Bufka LF, Barlow DH, Gorman JM, Shear MK, Woods SW. Therapists, therapist variables, and cognitive-behavioral therapy outcome in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology. 2001;69(5):747–755. doi: 10.1037//0022-006x.69.5.747. [DOI] [PubMed] [Google Scholar]
  • 40.Barber JP, Foltz C, Crits-Christoph P, Chittams J. Therapists’ adherence and competence and treatment discrimination in the NIDA Collaborative Cocaine Treatment Study. Journal of Clinical Psychology. 2004;60(1):29–41. doi: 10.1002/jclp.10186. [DOI] [PubMed] [Google Scholar]
  • 41.McGovern MP, Fox TS, Xie H, Drake RE. A survey of clinical practices and readiness to adopt evidence-based practices: Dissemination research in an addiction treatment system. Journal of Substance Abuse Treatment. 2004;26(4):305–312. doi: 10.1016/j.jsat.2004.03.003. [DOI] [PubMed] [Google Scholar]
  • 42.Siqueland L, Crits-Christoph P, Barber JP, Butler SF, Thase M, Najavits LM, Onken LS. The role of therapist characteristics in training effects in cognitive, supportive-expressive, and drug counseling therapies for cocaine dependence. The Journal of Psychotherapy Practice and Research. 2000;9(3):123–130. [PMC free article] [PubMed] [Google Scholar]
  • 43.Carroll KM. Constrained, confounded and confused: Why we really know so little about therapists in treatment outcome research. Addiction. 2001;96(2):203–206. doi: 10.1046/j.1360-0443.2001.9622032.x. [DOI] [PubMed] [Google Scholar]
  • 44.Miller WR, Moyers TB. The forest and the trees: Relational and specific factors in addiction treatment. Addiction. 2014;110(3):401–413. doi: 10.1111/add.12693. [DOI] [PubMed] [Google Scholar]
  • 45.Lu W, Yanos PT, Gottlieb JD, Duva SM, Silverstein SM, Xie H, Rosenberg SD, Mueser KT. Using fidelity assessments to train clinicians in the CBT for PTSD program for clients with serious mental illness. Psychiatric Services. 2012;63(8):785–792. doi: 10.1176/appi.ps.201000458. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Mueser KT, Rosenberg SD, Jankowski MK, Hamblen JL, Descamps M. A cognitive-behavioral treatment program for posttraumatic stress disorder in persons with severe mental illness. American Journal of Psychiatric Rehabilitation. 2004;7(2):107–146. [Google Scholar]
  • 47.Mueser KT, Rosenberg SD, Xie H, Jankowski MK, Bolton EE, Lu W, Hamblen JL, Rosenberg HJ, McHugo GJ, Wolfe RS. A randomized controlled trial of cognitive-behavioral treatment for posttraumatic stress disorder in severe mental illness. Journal of Consulting and Clinical Psychology. 2008;76(2):259–271. doi: 10.1037/0022-006X.76.2.259. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48.Rosenberg SD, Mueser KT, Friedman MJ, Gorman PG, Drake RE, Vidaver RM, Torrey WC, Jankowski MK. Developing effective treatments for posttraumatic disorders among people with severe mental illness. Psychiatric Services. 2001;52(11):1453–1461. doi: 10.1176/appi.ps.52.11.1453. [DOI] [PubMed] [Google Scholar]
  • 49.McGovern MP, Lambert-Harris C, Acquilano S, Xie H, Alterman AI, Weiss RD. A cognitive behavioral therapy for co-occurring substance use and posttraumatic stress disorders. Addictive Behaviors. 2009;34(10):892–897. doi: 10.1016/j.addbeh.2009.03.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.McGovern MP, Lambert-Harris C, Alterman AI, Xie H, Meier A. A randomized controlled trial comparing Integrated Cognitive Behavioral Therapy versus Individual Addiction Counseling for co-occurring substance use and posttraumatic stress disorders. Journal of Dual Diagnosis. 2011;7(4):207–227. doi: 10.1080/15504263.2011.620425. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Mercer D, Woody G. Individual Drug Counseling. National Institute on Drug Abuse; Rockville, MD: 1999. [Google Scholar]
  • 52.Nowinski JK, Baker S, Carroll KM. Twelve-step facilitation therapy manual: A clinical research guide for therapists treating individual with alcohol abuse and dependence. National Institute on Alcohol Abuse and Alcoholism (NIAAA); Rockville, MD: 1992. [Google Scholar]
  • 53.Blake DD, Weathers FW, Nagy LM, Kaloupek DG, Gusman FD, Charney DS, Keane TM. The development of a Clinician-Administered PTSD Scale. Journal of Traumatic Stress. 1995;8(1):75–90. doi: 10.1007/BF02105408. [DOI] [PubMed] [Google Scholar]
  • 54.Cacciola JS, Alterman AI, McLellan AT, Lin YT, Lynch KG. Initial evidence for the reliability and validity of a “Lite” version of the Addiction Severity Index. Drug and Alcohol Dependence. 2007;87(2-3):297–302. doi: 10.1016/j.drugalcdep.2006.09.002. [DOI] [PubMed] [Google Scholar]
  • 55.Rikoon SH, Cacciola JS, Carise D, Alterman AI, McLellan AT. Predicting DSM-IV dependence diagnoses from Addiction Severity Index composite scores. Journal of Substance Abuse Treatment. 2006;31(1):17–24. doi: 10.1016/j.jsat.2006.03.003. [DOI] [PubMed] [Google Scholar]
  • 56.Rosen CS, Henson BR, Finney JW, Moos RH. Consistency of self-administered and interviewer-based Addiction Severity Index composite scores. Addiction. 2000;95(3):419–425. doi: 10.1046/j.1360-0443.2000.95341912.x. [DOI] [PubMed] [Google Scholar]
  • 57.IBM Corporation . IBM SPSS Statistics for Windows, Version 21.0. IBM Corporation; Armonk, NY: 2012. [Google Scholar]
  • 58.Becker CB, Zayfert C, Anderson E. A survey of psychologists’ attitudes towards and utilization of exposure therapy for PTSD. Behaviour Research and Therapy. 2004;42(3):277–292. doi: 10.1016/S0005-7967(03)00138-4. [DOI] [PubMed] [Google Scholar]
  • 59.Morgenstern J, Morgan TJ, McCrady BS, Keller DS, Carroll KM. Manual-guided cognitive-behavioral therapy training: A promising method for disseminating empirically supported substance abuse treatments to the practice community. Psychology of Addictive Behaviors. 2001;15(2):83–88. [PubMed] [Google Scholar]
  • 60.Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don't train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology. 2005;73(1):106–115. doi: 10.1037/0022-006X.73.1.106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Campbell BK, Buti A, Fussell HE, Srikanth P, McCarty D, Guydish JR. Therapist predictors of treatment delivery fidelity in a community-based trial of 12-step facilitation. The American Journal of Drug and Alcohol Abuse. 2013;39(5):304–311. doi: 10.3109/00952990.2013.799175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 62.McGovern MP, Xie H, Segal SR, Siembab L, Drake RE. Addiction treatment services and co-occurring disorders: Prevalence estimates, treatment practices, and barriers. Journal of Substance Abuse Treatment. 2006;31(3):267–275. doi: 10.1016/j.jsat.2006.05.003. [DOI] [PubMed] [Google Scholar]
  • 63.McGovern MP, Lambert-Harris C, McHugo GJ, Giard J, Mangrum LF. Improving the dual diagnosis capability of addiction and mental health treatment services: Implementation factors associated with program level changes. Journal of Dual Diagnosis. 2010;6:237–250. [Google Scholar]
  • 64.Baer JS, Ball SA, Campbell BK, Miele GM, Schoener EP, Tracy K. Training and fidelity monitoring of behavioral interventions in multi-site addictions research. Drug and Alcohol Dependence. 2007;87(2-3):107–118. doi: 10.1016/j.drugalcdep.2006.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES