Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Mar 1.
Published in final edited form as: Psychol Addict Behav. 2014 Aug 18;29(1):106–113. doi: 10.1037/adb0000008

The Relationship of Therapeutic Alliance and Treatment Delivery Fidelity with Treatment Retention in a Multisite Trial of Twelve-Step Facilitation

Barbara K Campbell 1, Joseph Guydish 2, Thao Le 2, Elizabeth A Wells 3, Dennis McCarty 1
PMCID: PMC4739723  NIHMSID: NIHMS754993  PMID: 25134056

Abstract

Objective

This study examined associations of therapeutic alliance and treatment delivery fidelity with treatment retention in Stimulant Abusers to Engage in Twelve-Step (STAGE-12; Donovan et al., 2013), a community-based trial of 12-Step Facilitation (TSF) conducted within the National Drug Abuse Treatment Clinical Trials Network (CTN).

Method

The STAGE-12 trial randomized 234 stimulant abusers enrolled in 10 outpatient drug treatment programs to an 8-session, group and individual TSF intervention. During the study, TSF participants rated therapeutic alliance using the Helping Alliance questionnaire-II. Following the study, independent raters evaluated treatment delivery fidelity of all TSF sessions on adherence, competence and therapist empathy. Poisson regression modeling examined relationships of treatment delivery fidelity and therapeutic alliance with treatment retention (measured by number of sessions attended) for 174 participants with complete fidelity and alliance data.

Results

Therapeutic alliance (p = 0.005) and therapist competence (p = .010) were significantly associated with better treatment retention. Therapist adherence was associated with poorer retention in a non-significant trend (p = .061).

Conclusions

Stronger therapeutic alliance and higher therapist competence in the delivery of a TSF intervention were associated with better treatment retention, while treatment adherence was not. Training and fidelity monitoring of TSF should focus on general therapist skills and therapeutic alliance development to maximize treatment retention.

Keywords: Treatment Retention, Fidelity, Therapeutic Alliance, Twelve -Step Facilitation

Longer stays and better attendance are associated with enhanced outcomes in substance abuse treatment (Hubbard, Craddock, Flynn, Anderson, & Etheridge, 1997; Simpson, 1981; Zhang, Friedmann, & Gerstein, 2003). Treatment retention, however, remains a challenge and premature termination remains a common problem (Brorson, Arnevik, Rand-Hendriksen, & Duckert, 2013; Stark, 1992; Swift & Greenberg, 2013). Research has typically examined patient characteristics associated with retention or, conversely, with drop-out. Few patient characteristics, however, show consistent associations with treatment drop-out in either addictions treatment (Brorson et al., 2013) or the broader psychotherapy domain (Swift & Greenberg, 2012). Reviews note the lack of research on the relationship of treatment variables with retention and recommend this area be addressed (Brorson et al. 2013; Swift & Greenberg, 2012). Among the limited studies of treatment variables associated with retention, manualized and time-limited treatments (Swift & Greenberg, 2012) and higher therapeutic alliance (Brorson et al., 2013; Sharf, Primavera, & Diener, 2010) have been associated with better retention.

Treatment Retention and Therapeutic Alliance

Therapeutic alliance, the collaborative relationship between therapist and patient, has been conceptualized as a common treatment factor present across treatment orientations (Horvath & Luborsky, 1993). It has predicted attendance (Fiorentine, Nakashima, & Anglin, 1999; Simpson et al., 1997), retention (De Weert-Van Oene, Schippers, De Jong, & Schrijvers, 2001; Meier, Donmall, McElduff, Barrowclough, & Heller, 2006; Knuuttila, Kuusisto, Saarnio, & Nummi, 2012; Ruglass et al., 2012) and outcomes (Connors, Carroll, DiClemente, Longabaugh, & Donovan, 1997; Gillaspy, Wright, Campbell, Stokes, & Adinoff, 2002; Crits-Christoph, Johnson, Connolly Gibbons, & Gallop, 2013) for patients in substance abuse treatment. Early therapeutic alliance, particularly, is associated with treatment engagement and retention (Meier et al., 2006; Brorson et al., 2013). This robust finding supports focusing retention research on treatment variables, including both common and specific treatment factors. To do so, it may be useful to investigate the degree to which delivery of specified treatments as intended, known as treatment delivery fidelity, is associated with retention.

Treatment Retention and Fidelity

Treatment delivery fidelity has received considerable research attention as a method of ensuring internal trial validity (Bellg et al., 2004; Gearing, El-Bassel, Ghesquiere, Baldwin, Gilles, & Ngeow, 2011). Fidelity measures most commonly involve observer ratings of adherence to specified treatment content, as well as ratings of therapist competence or skill delivering the treatment (Borrelli, 2011). Research on the relationship of therapist adherence and competence to patient outcomes in substance abuse treatment has produced mixed findings (Webb, DeRubeis, & Barber, 2010). Some studies have identified fidelity-outcome relationships only after controlling for the effects of therapeutic alliance (Barber et al., 2006; Gibbons et al., 2010; Hogue et al., 2008). Although numerous studies have investigated fidelity-outcome relationships, we identified only one study that examined the relationships between therapist adherence and competence and treatment retention in substance abuse treatment. The study found no significant association between adherence to a three-session motivational interviewing (MI) intervention and days of outpatient treatment enrollment, while competence in advanced motivational interviewing skills (measured using the entire therapist sample, including therapists conducting treatment as usual) was negatively associated with retention in outpatient treatment at 4-weeks post intervention (Martino, Ball, Nich, Frankforter, & Carroll, 2008). There were no significant competence-retention relationships within the MI therapist-only sample. Unexpected competence-retention results are difficult to understand, particularly given significant effects often shown for MI in increasing treatment retention (Hettema, Steele & Miller, 2005). This may be a spurious result given the relatively high number of analyses conducted in the study. An alternative explanation may be that clients at risk for treatment disengagement prompted therapists to use motivational strategies with greater skill in an effort to build motivation for treatment (Martino, personal communication, 2014). Results point to the need for further research to clarify relationships among adherence, competence, and retention in treatments for substance use disorders. Fidelity-retention relationships should be studied across different manual-guided treatments, particularly given the finding that overall retention rates have been found to be superior for manualized versus non-manualized treatments (Swift & Greenberg, 2012).

Treatment Retention and Outcomes in TSF

Twelve-Step Facilitation (TSF) is a manual-guided treatment for alcohol and substance use disorders that seeks to increase clients’ engagement in 12-Step activities outside of formal treatment sessions. Since Project Match, which found TSF to be comparable in outcomes to MI and cognitive behavioral therapy (Project Match Research Group, 1997), empirical support for TSF has accumulated (Brown, Seraganian, Tremblay, & Annis, 2002; Carroll, Nich, Ball, McCance, & Rounsaville, 1998; Kaskutas, Subbaraman, Witbrodt, & Zemore, 2009; Timko, DeBenedetti, & Billow; 2006; Timko & DeBenedetti, 2007). Research to date has shown that retention in TSF has generally been comparable with other treatments (Carroll et al., 1998; Project Match Research Group, 1997) and that retention in TSF is associated with better outcomes. (Katsukas et al., 2009; Timko, Sutkowi, Cronkite, Makin-Byrd & Moos, 2011).

Stimulant Abusers to Engage in Twelve Step (STAGE-12), the parent study for the current analysis, examined the efficacy/effectiveness of TSF for stimulant abusers conducted in community treatment programs (Donovan et al., 2013). The study trained outpatient counselors in 10 treatment centers to deliver a group-plus-individual TSF treatment that was integrated into treatment as usual (TAU) and compared to a TAU-only condition. TSF retention was comparable to TAU retention as measured by self-reported, group session attendance within a 30-day period and was higher for number of individual sessions reported by participants. TSF participants had a higher likelihood of abstinence from stimulants during treatment, although there were no differences at follow-up. TSF participants had higher rates of attendance and involvement in 12-Step programs post-treatment and at 6-month follow-up. The relationship of number TSF treatment sessions attended (using a dichotomous measure called high versus low exposure) to participant outcomes was also examined (Wells et al., under review). High exposure to treatment, defined as attendance at two or more (out of 3) individual sessions plus three or more (out of 5) group sessions, was achieved by 77% of TSF participants and was associated with a) significantly higher odds of abstinence from stimulants during treatment and across 4 months of follow-up, b) significantly lower rates of stimulant use for non-abstinent participants and non-stimulant drug use during treatment, but not after, and c) more days of attending 12-Step meetings and engaging in duties during meetings through 90-days post treatment (Wells et al., under review).

In an ancillary study, we assessed the reliability and concurrent validity of the Twelve Step Facilitation Adherence Competence Empathy Scales (TSF ACES), a ratings measure of treatment delivery fidelity based on an expansion of the adherence rating scales used in STAGE-12 (Campbell, Manuel et al., 2013). Trained, independent raters evaluated fidelity of all audio-recorded TSF sessions. The availability of comprehensive fidelity ratings for the entire TSF sample provided an opportunity to study relationships of fidelity with other variables, including predictors and outcomes. A prior report (Campbell, Buti, et al., 2013) found that therapists reporting self-efficacy in basic counseling skills had higher adherence, competence, and empathy delivering the TSF intervention and those with graduate degrees had higher adherence. In contrast, therapists with more positive attitudes towards 12-Step groups and self-efficacy in addiction-specific counseling skills had lower adherence ratings. In a study of fidelity-patient outcomes relationships, greater therapist empathy was significantly associated with fewer days of self-reported drug use at 3-months post-treatment; greater competence was associated with this outcome in a non-significant trend (p = 0.06) and there was no association of adherence with days of drug use. All three fidelity measures were associated with better employment outcomes on the Addiction Severity Index (ASI), but worse drug composite scores at 3-months post-treatment. Analysis of ASI drug use items, which include both days of use and how troubled the respondent is by use, showed that greater fidelity was associated with fewer days of use, but an increased sense of being troubled by use (Guydish et al., 2014). The authors noted that different types of ASI items had different relationships to the same predictor, and posited that maxims such as “one day at a time” kept the risk of drug use at the fore even as actual drug use declined. The current study examined the relationships of treatment retention in STAGE -12 TSF, as measured by number of sessions attended, with treatment delivery fidelity (i.e., therapist adherence, competence, and empathy) and therapeutic alliance, as reported by participants at the second treatment visit.

Methods

Overview of STAGE-12 Trial

STAGE-12 was a multi-site, randomized trial conducted in 10 outpatient community treatment centers across the United States. Patient participants were adults with stimulant abuse/dependence who were enrolled or seeking treatment admission. Participants were randomly assigned to either STAGE-12 TSF (i.e., 5-session group plus 3-session, individual, intensive referral sessions) plus TAU (N = 234) or TAU (TAU; 5–15 hours of weekly treatment; N = 237). The STAGE-12 study was approved by the University of Washington Institutional Review Board (IRB), as well as IRBs of all academic institutions affiliated with participating sites. See Donovan et al. (2013) for a complete description of STAGE-12 study participants, procedures and the TSF treatment.

Participants

The STAGE-12 trial randomized 234 participants to the TSF intervention. Two-hundred (85.5%) completed the therapeutic alliance measure, Helping Alliance questionnaire-II (HAq-II; Luborsky et al., 1996) at 2 weeks post-randomization. We excluded the following due to incomplete measures: a) 4 who did not have fidelity ratings; b) 1 who completed the Haq-II before attending any treatment sessions thus invalidating the measure and c) 6 who had more than 20% missing HAq-II items (i.e., 4 or more items). We also excluded 15 participants who had missing baseline ASI alcohol composite or ASI drug composite scores. Our analysis included 174 participants who met the following conditions: completed Haq-II at week 2 post-randomization and attended at least 1 session to produce a valid measure of therapeutic alliance, were rated for treatment fidelity, had completed at least 80% of Haq-II items, and had ASI alcohol and drug composite scores at baseline.

Study Therapists

All therapists (N = 106) at study sites were considered for inclusion based on four eligibility criteria: (a) credentialed to provide substance abuse treatment, (b) approved by the treatment program’s administration, (c) willing to participate and to be randomized, and (d) familiar with the 12-Step orientation. There were 39 therapists (37%) who met criteria and were included in the study pool; two from each site were chosen at random from the pool to conduct the TSF treatment. Remaining therapists were available for training as replacement therapists, four of whom became TSF therapists. Supervisors were trained as back-up TSF therapists. In total, there were 34 therapists and supervisors who conducted the TSF intervention. We obtained demographic information from 33 of the 34 therapists; they were predominantly Caucasian (70%) women (67%) with a mean age of 52 years (SD = 9.2). Most (82%) had at least five years of counseling experience and 55% had masters degrees or above. During the trial, therapists were trained in the TSF intervention, certified and monitored for adherence by on-site supervisors and expert raters (4 clinicians experienced in substance abuse treatment and trained in the TSF intervention; 1 masters’ level, 1 doctoral candidate, and 2 doctoral level). The STAGE-12 trial audio recorded all TSF sessions but did not record TAU sessions.

Independent Fidelity Raters

We recruited separate raters from local graduate programs to conduct ratings of all audio-recorded, STAGE-12 TSF sessions. The nine raters (seven with masters’ degrees and two with doctoral degrees) averaged five years of clinical experience (SD = 4.05), seven years of research experience (SD = 5.96) and one year of rating experience (SD = 2.95). A doctoral level psychologist with extensive experience in fidelity monitoring served as an expert rater and trainer/ratings supervisor.

Measures

TSF ACES

TSF ACES measures five dimensions of fidelity using 6-point scales, three of which were used in our analysis: 1) adherence – delivery of specific treatment content; 2) competence - the skill of content delivery; and 3) global empathy – the therapist’s effort to understand the clients’ perspectives (adapted from the Motivational Interviewing Treatment Integrity scale; Moyers, Martin, Manuel, Hendrickson, & Miller, 2005). There are four content rating forms, one for group sessions (10 items) and three corresponding to STAGE-12 individual session 1 (10 items), session 2 (4 – 5 items) and session 3 (8 – 9 items). Summary measures derived for each session had modest to excellent inter rater reliabilities, with intraclass correlations of .91 for mean adherence, .90 for mean competence, and .69 for global empathy. Internal consistencies computed with Cronbach’s alpha for summary measures based on multiple items were .69 for mean adherence and .71 for mean competence. In assessing TSF ACES convergent validity with the HAq-II, all correlations were in the expected directions (e.g., negative correlation of Haq-II with proscribed therapist behaviors); there were no significant correlations for mean adherence, mean competence, or global empathy with HAq-II scores collected at week two. See Campbell, Manuel, et al. (2013) for a further description of the psychometric characteristics of the ratings scale and for sample items. TSF ACES ratings manual and forms are available at: http://ctndisseminationlibrary.org/PDF/795_TSFACES.pdf.

Helping Alliance Questionnaire-II

Therapeutic alliance was assessed using the patient version of the HAq-II (Luborsky et al., 1996). This self-report measure assesses the degree to which patients experience therapist and treatment as collaborative and helpful. The HAq-II had good test-retest reliability (.78), internal consistency (.90), and convergent validity on a normative sample of cocaine abusers (Luborsky et al., 1996) and is frequently used in alliance research with substance-abusing samples (see Meier, et al., 2005 for a review). The instrument contains 19 items measured on a 6-point Likert scale; the sum of the items (with negative items reverse scored) forms the total score. STAGE-12 study participants completed the HAq-II at week two of treatment and week eight (i.e., end of treatment). We used week two scores for our analysis in order to employ a measure that temporally preceded our outcome measure and based on previous robust findings of early alliance predicting engagement and retention.

Addiction Severity Index (ASI) - Lite

The ASI-Lite (McLellan et al., 1992) was administered at baseline and follow-up in STAGE-12. ASI composite scores measure problem severity in seven areas (medical, employment, legal, alcohol, drug, social, psychological; McLellan et al., 1985). Scores are derived from questions in each area measuring problem severity within the prior 30-day period. We used baseline ASI drug and alcohol composite scores as measures of substance use severity at treatment entry.

Treatment Retention

Number of TSF sessions attended (ranging from 0 – 8) was the measure chosen for treatment retention. Session attendance was reviewed and recorded weekly during the treatment phase by the therapist.

Procedures for Independent Fidelity Ratings

Raters viewed the STAGE-12 TSF therapist training video and completed a one-day training. Prior to rating study sessions, raters achieved a criterion level of inter-rater reliability with the ratings expert on audio recorded, practice sessions conducted by STAGE-12 therapists. Audio recordings of all TSF group (n=512) and individual (n=487) sessions were randomly assigned to certified raters in sets of 20; one session per set was randomly assigned to the study expert for co-rating to monitor ratings consistency. There were 33 incomplete or poor audio quality recordings, leaving 966 rated sessions. The University of California, San Francisco and Oregon Health and Science University IRBs approved the procedures for the fidelity study. See Campbell et al. (2013) for more detail.

Data Analysis

Descriptive statistics, including means, standard deviations, and percentages, were used to summarize characteristics of TSF participants at baseline. Comparisons between TSF participants included in our analysis and those excluded from analysis due to missing data were conducted with t-tests for continuous variables and with chi-square tests for categorical variables.

We first tested univariate associations of therapeutic alliance and treatment delivery fidelity (mean adherence, mean competence, mean empathy) with treatment retention (i.e., number of sessions attended). Poisson regression modeling examined the multivariate relationship between these predictor variables and the retention outcome measure. Age, gender, race and baseline values of ASI drug and alcohol composite scores were included in the Poisson model. Nesting of clients within site was controlled for as well. Nesting of clients within counselor was not controlled because clients received STAGE-12 group sessions from more than one counselor. Analyses were conducted using SAS software version 9.3 (SAS Inc. NC, USA).

Results

Participant Characteristics

The mean age of participants in the analytic sample was 38.1 (SD = 10.2) years and 62% were women. White participants accounted for 44% of the sample and African Americans accounted for 37%. More than half (52.6%) had never married, 23.7% were divorced and 14.5% reported being married. About half were high school graduates and 29% had some college education. The majority were working (35.1% full time and 23.6% part time). Participants included in the analyses (n = 174) were similar to those excluded (n = 60) on these demographic characteristics. However, participants included in the analysis received significantly more STAGE-12 sessions (mean = 5.6, SD = 2.0) than those not included in the analysis (mean = 2.2, SD = 2.3, p < 0.001), primarily due to inclusion criterion of having completed a Haq-II after attending at least one session. See Table 1 for participant characteristics.

Table 1.

Baseline demographic characteristics for STAGE-12 TSF participants included (n=174) and not included (n=60) in the analysis.

Included in Analysis (N=174) Not Included (N=60) p values1
Age – Mean (SD) 38.1 (10.2) 38.6 (9.7) 0.717
Gender 0.956
 Female 108 (62.1%) 37 (61.7%)
 Male 66 (37.9%) 23 (38.3%)
Hispanic or Latino Ethnicity 12 (6.9%) 3 (5.1%) 0.624
Race 0.246
 White 76 (43.7%) 30 (50.9%)
 African American 64 (36.8%) 23 (39.0%)
 Other2 34 (19.5%) 6 (10.2%)
Highest Education 0.582
 Less than High School 37 (21.3%) 15 (25.0%)
 High School graduate 86 (49.4%) 25 (41.7%)
 Some College 51 (29.3%) 20 (33.3%)
Employment 0.629
 Full time 61 (35.1%) 22 (36.7%)
 Part time 41 (23.6%) 17 (28.3%)
 Unemployed 64 (36.8%) 17 (28.3%)
 Other3 8 (4.6%) 4 (6.7%)
Marital status 0.374
 Married 25 (14.5%) 11 (18.3%)
 Widowed 1 (0.6%) 1 (1.7%)
 Separated 15 (8.7%) 9 (15.0%)
 Divorced 41 (23.7%) 15 (25.0%)
 Never married 91 (52.6%) 24 (40.0%)
No. of STAGE-12 sessions Mean (SD)4 5.6 (2.0) 2.2 (2.3) <0.001
Open in a new tab
1

p values from t-test for continuous variables and chi square test for categorical variables.

2

Other includes: Alaskan=3, Asian=3, Mixed=19, Native HI or Pacific Islander=2, other=13.

3

Other employment categories were student (1), retired (7), and living in a controlled environment (4)

4

19 TSF participants received 0 sessions, and 0 values are included in the mean calculations.

Number of Sessions Attended

See Table 2 for a distribution of session attendance for the 174 participants included in our analysis. Approximately 5% attended only one session, while 14% attended all 8 sessions. The mean number of sessions was 5.6 (sd = 2); the majority of participants (88%) attended 4 – 8 sessions.

Table 2.

Distribution of Number of TSF Counseling Sessions Attended

Number of counseling sessions Frequency Percent
1 9 4.6
2 20 10.2
3 14 7.1
4 15 7.7
5 25 12.8
6 41 20.9
7 44 22.5
8 28 14.3
Open in a new tab

Relationship between Therapeutic Alliance, Treatment Fidelity, and Treatment Retention

Results of univariate and multivariate analyses are shown in Table 3. In the univariate analysis, there was a statistically significant association between therapeutic alliance (Haq-II) and treatment retention (β=0.142, p=0.002), and no significant relationships of fidelity variables with retention. In the multivariate analysis, controlling for age, gender, race, baseline ASI drug composite, baseline ASI alcohol composite, adherence, competence, and empathy scores, an increase in therapeutic alliance by one unit resulted in an increase in the number of sessions attended by 14% (exp(β)=1.14; p = 0.005). Likewise, there was a significant association of therapist mean competence with retention; an increase in therapist mean competence by one unit resulted in an increase in session attendance by 36% (exp (β) = 1.36; p = .010). The association between number of sessions attended and mean adherence, while controlling for all other variables, approached significance (p=.061); for every increase of one unit in mean adherence, session attendance decreased by 20% (exp (β) = 0.80). There was no significant association between session attendance and empathy scores when controlling for other variables and none of the patient-characteristic control variables were associated with retention.

Table 3.

Parameter estimates (β), exponential β (95%CI) and p values of univariate and multivariate analysis examining associations between therapeutic alliance, TSF fidelity predictors, and treatment retention (N=174)1

Predictors Outcome variable (Number of sessions attended)
Univariate analysis Multivariate analysis1
Parameter estimates (β) Exp β (95%CI) p value Parameter estimates (β) Exp β (95%CI) p value
Therapeutic Alliance2 0.142 1.15 (1.05, 1.26) 0.002 0.130 1.14 (1.04, 1.25) 0.005
Adherence 0.039 1.04 (0.89, 1.21) 0.615 −0.220 0.80 (0.64, 1.01) 0.061
Competence 0.078 1.08 (0.97, 1.21) 0.171 0.307 1.36 (1.07, 1.72) 0.010
Empathy 0.019 1.02 (0.93, 1.21) 0.689 −0.106 0.90 (0.78, 1.03) 0.132
Open in a new tab
1

Poisson regression model assessing relationships between therapeutic alliance (HAq-II), fidelity ratings (Adherence, Competence, Empathy) and treatment retention variable (number of attended sessions). Model controlled for age, gender, race, and baseline value of ASI alcohol composite and ASI drug composite, with clustering within site.

2

Therapeutic alliance as measured by the Helping Alliance Questionnaire (Haq-II) at 2 weeks post-randomization.

Discussion

The robust association of longer retention with better outcomes in substance abuse treatment has been extended to TSF treatment (Katsukas et al., 2009; Wells et al., under review), supporting the importance of treatment retention for TSF. The current study contributes to recommended research on treatment variables as predictors of retention (Brorson et al. 2013; Swift & Greenberg, 2012). Our findings indicated that early, participant–rated, therapeutic alliance was significantly associated with retention in TSF in both univariate and multivariate analyses. To our knowledge, it is the first study to show a relationship of therapeutic alliance with retention in TSF with substance abusers and corroborates a previous finding from Project Match showing a relationship of therapist-rated, therapeutic alliance with outpatient, TSF retention for alcohol dependent participants (Connors et al., 1997). Results are also the first to identify a significant fidelity-retention relationship for manual-guided, TSF treatment, a finding that has important implications for treatment delivery. Therapist competence was associated with higher session attendance when therapeutic alliance and other fidelity variables were controlled. Unexpectedly, the multivariate model suggested a relationship between higher adherence and poorer retention, that, while not significant in this analysis (p = .061), may bear additional attention in future research.

Results suggest that variables related to general therapist skill, which facilitate development of positive therapeutic alliance and are associated with competent TSF delivery, may improve attendance more than strict intervention adherence. Results are consistent with findings from Guydish et al. (2014) indicating that therapist empathy and competence were associated with better patient outcomes, while adherence was not, although the lack of a significant empathy finding in the current study is inconsistent with this pattern of results. It may be that the relationship of empathy with retention is accounted for mostly in facilitating therapeutic alliance, such that, when alliance is controlled, differences in therapist empathy do not affect retention. Overall, findings lend support to the “common factors” (Castonguay, 1993) hypothesis regarding treatment effectiveness, suggesting that variables present across treatments, such as a positive therapeutic alliance and competent delivery of treatment content may be central to increasing retention and improving outcomes.

Competence-retention findings in the current study are compatible with research showing that more experienced therapists had lower dropout rates (Swift & Greenberg, 2012) and that more advice giving was associated with worse outcomes in group counseling (Crits- Christoph, Johnson, Connolly Gibbons, & Gallop, 2013). Swift & Greenberg (2012) suggested that more experienced therapists may be more responsive and have a greater relationship focus, which may explain their ability to retain clients in treatment. Therapist responsiveness to client presentation may also be relevant for adherence results. Adherence may provide intervention structure that ensures the inclusion of empirically-supported practices. However, departures from strict adherence based on therapist responsiveness to changes in client presentation (i.e., therapist attunement) may improve alliance, address client need more effectively, and appropriately individualize treatment in community settings serving heterogeneous clients with multiple comorbidities. It has been argued that flexible application of manual-guided treatments, including training about when and how to be flexible, optimizes the use of empirically-supported treatments in clinical practice (Kendall, Gosch, Furr, & Sood, 2008). Use of a mean adherence measure in our study may have obscured precise adherence information needed to show a relationship with retention. If the therapist responsiveness (i.e., flexible fidelity) hypothesis is correct, then variations in strict adherence based on therapist-client interactions may be associated with improved retention and better outcomes, and may require more finely-tuned measurement.

Use of fidelity ratings of TSF sessions, using an instrument with known psychometric properties (Campbell, Manuel, et al., 2013) and independent raters who had undergone rigorous training are strengths of the current study. The inclusion of a measure of therapeutic alliance is also a strength, not only to assess its relationship with retention, but as a variable to control when examining the fidelity- retention relationships. Missing data that eliminated approximately 25% of TSF participants from the current analysis is a study weakness, although the excluded sample did not differ demographically from the sample included in our analysis. Participants who did not attend any sessions did not have fidelity or valid therapeutic alliance data, thus were omitted from our analysis. This is a study limitation that prohibits us from identifying any variables associated with immediate drop-out, an important treatment consideration. Lack of measurement of early symptom improvement among participants may also be considered a study limitation. Early participant improvement may be confounded with therapeutic alliance as well as a predictor of retention itself (Crits-Christoph, Connolly Gibbons & Hearon, 2006; Webb et al., 2010). The use of a measure of therapeutic alliance after only week two of treatment may mitigate this concern, given the limited time for improvement to occur. Also, limited research has shown that, although early alliance may be affected by symptom improvement, alliance remains a significant predictor of positive outcomes when early symptom improvement is controlled (Barber, Connolly, Crits-Christoph, Gladys, & Siqueland, 2000).

Recommendations

Clinicians should be trained and monitored in general therapy skills, not simply adherence, in both clinical trials and community implementation of TSF and other behavioral interventions. This includes training designed to facilitate the therapeutic alliance, several interventions for which have been developed. Campbell et al. (2009) developed a brief intervention specifically designed to foster alliance development and found that it increased clients’ continuation in outpatient treatment following detoxification in a randomized trial. A preliminary study of Alliance Fostering Therapy added to supportive-expressive therapy resulted in depressed patients’ reports of positive changes in quality of life (Crits-Christoph et al., 2006). The complex topic of training and intervention characteristics that facilitate alliance development and general therapist competence clearly requires further study. Studies should also examine therapist adherence variations during manual-guided treatment. Adherence flexibility may be a component of therapist competence that is superior to strict adherence, although this may vary depending on the specified treatment and client-related variables. Despite the need for further study, accumulating evidence, including the present study’s findings, suggests that training empirically-supported treatments such as TSF should emphasize general therapist and alliance-developing skills to improve retention and outcomes.

Acknowledgments

Awards from the National Institute on Drug Abuse (R01 DA025600, U10 DA015815 and P50 DA009253) supported this investigation. We appreciate the generous assistance of Dennis Donovan, Dennis Daley and the STAGE-12 study team in providing access to STAGE-12 study data. We’d like to thank Kevin Delucchi, University of California, San Francisco, for consultation regarding statistical analysis and Emma Passalacqua for her formatting assistance. Special thanks to our raters and the staff at all the participating treatment sites.

References

  1. Barber JP, Connolly MB, Crits-Christoph P, Gladys L, Siqueland Alliance predicts patients’ outcome beyond in-treatment change in symptoms. Journal of Consulting and Clinical Psychology. 2000;68:1027–1032. doi: 10.1037//0022-006x.68.6.1027. [DOI] [PubMed] [Google Scholar]
  2. Barber JP, Gallop R, Crits-Christoph P, Frank A, Thase ME, Weiss RD, Gibbons MB. The role of therapist adherence, therapist competence, and the alliance in predicting outcome of individual drug counseling: Results from the NIDA Collaborative Cocaine Treatment Study. Psychotherapy Research. 2006;16:229–240. [Google Scholar]
  3. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, … Czajkowski S. Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH Behavior Change Consortium. Health Psychology. 2004;23:443–451. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
  4. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. Journal of Public Health Dentistry. 2011;71:S52–S63. doi: 10.1111/j.1752-7325.2011.00233.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Brorson HH, Arnevik EA, Rand-Hendriksen K, Duckert F. Drop-out from addiction treatment: A systematic review of risk factors. Clinical Psychology Review. 2013;33:1010–1024. doi: 10.1016/j.cpr.2013.07.007. [DOI] [PubMed] [Google Scholar]
  6. Brown TG, Seraganian P, Tremblay J, Annis H. Process and outcome changes with relapse prevention versus 12-step aftercare programs for substance abusers. Addiction. 2002;97:677–689. doi: 10.1046/j.1360-0443.2002.00101.x. [DOI] [PubMed] [Google Scholar]
  7. Campbell BK, Buti A, Fussell H, Srikanth P, McCarty D, Guydish JR. Therapist predictors of treatment delivery fidelity in a community based trial of 12-Step Facilitation. American Journal of Drug and Alcohol Abuse. 2013;39(5):304–311. doi: 10.3109/00952990.2013.799175. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Campbell BK, Fuller BE, Lee ES, Tillotson C, Woelfel T, Jenkins L, Robinson J, Booth RE, McCarty D. Facilitating outpatient treatment entry following detoxification for injection drug use: a multi-site test of three interventions. Psychology of Addictive Behaviors. 2009;23:260–270. doi: 10.1037/a0014205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Campbell BK, Manuel JK, Manser S, Peavey M, Stelmokis J, McCarty D, Guydish J. Assessing fidelity of treatment delivery in group and individual 12-Step facilitation. Journal of Substance Abuse Treatment. 2013;44:169–176. doi: 10.1016/j.jsat.2012.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Carroll KM, Nich C, Ball SA, McCance E, Rounsaville BJ. Treatment of cocaine and alcohol dependence with psychotherapy and disulfiram. Addiction. 1998;93:713–27. doi: 10.1046/j.1360-0443.1998.9357137.x. [DOI] [PubMed] [Google Scholar]
  11. Crits-Christoph P, Connolly Gibbons MB, Hearon B. Does the alliance cause good outcome? Recommendations for future research on the alliance. Psychotherapy: Theory, Research, Practice, Training. 2006;43(3):280–285. doi: 10.1037/0033-3204.43.3.280. [DOI] [PubMed] [Google Scholar]
  12. Crits-Christoph P, Connolly Gibbons MB, Crits-Christoph K, Narducci J, Schamberger M, Gallop R. Can therapists be trained to improve their alliances? A preliminary study of alliance-fostering psychotherapy. Psychotherapy Research. 2006;16(3):268–281. [Google Scholar]
  13. Crits-Christoph P, Johnson JE, Connolly Gibbons MB, Gallop R. Process predictors of the outcome of group drug counseling. Journal of Consulting and Clinical Psychology. 2013;81(1):23–34. doi: 10.1037/a0030101. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Carroll KM, Nich C, Ball SA, McCance E, Rounsaville BJ. Treatment of cocaine and alcohol dependence with psychotherapy and disulfiram. Addiction. 1998;93(5):713–727. doi: 10.1046/j.1360-0443.1998.9357137.x. [DOI] [PubMed] [Google Scholar]
  15. Castonguay LG. “Common factors” and “nonspecific variables”: Clarification of the two concepts and recommendations for research. Journal of Psychotherapy Integration. 1993;3:267–286. [Google Scholar]
  16. Connors GJ, Carroll KM, DiClemente CC, Longabaugh R, Donovan DM. The therapeutic alliance and its relationship to alcoholism treatment participation and outcome. Journal of Consulting and Clinical Psychology. 1997;65:588–598. doi: 10.1037//0022-006x.65.4.588. [DOI] [PubMed] [Google Scholar]
  17. De Weert-Van Oene GH, Schippers GM, De Jong CA, Schrijvers GJ. Retention in substance dependence treatment: The relevance of in-treatment factors. Journal of Substance Abuse Treatment. 2001;20:253–261. doi: 10.1016/s0740-5472(01)00160-x. [DOI] [PubMed] [Google Scholar]
  18. Donovan DM, Daley DC, Brigham GS, Hodgkins CC, Perl HI, Garrett S, … Zamarelli L. Stimulant abuser groups to engage in 12-Step (STAGE-12): A multisite trial in the NIDA Clinical Trials Network. Journal of Substance Abuse Treatment. 2013;44:103–114. doi: 10.1016/j.jsat.2012.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Fiorentine R, Nakashima J, Anglin MD. Client engagement in drug treatment. Journal of Substance Abuse Treatment. 1999;17:199–206. doi: 10.1016/s0740-5472(98)00076-2. [DOI] [PubMed] [Google Scholar]
  20. Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gilles J, Ngeow E. Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clinical Psychology Review. 2011;31:79–88. doi: 10.1016/j.cpr.2010.09.007. [DOI] [PubMed] [Google Scholar]
  21. Gibbons CJ, Nich C, Steinberg K, Roffman RA, Corvino J, Babor TF, Carroll CM. Treatment process, alliance and outcome in brief versus extended treatments for marijuana dependence. Addiction. 2010;105:1799–1808. doi: 10.1111/j.1360-0443.2010.03047.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Gillaspy JA, Wright AR, Campbell C, Stokes S, Adinoff B. Group alliance and cohesion as predictors of drug and alcohol abuse treatment outcomes. Psychotherapy Research. 2002;12:213–229. [Google Scholar]
  23. Guydish J, Campbell BK, Manuel JK, Delucchi K, Le T, Peavy KM, McCarty D. Does Treatment Fidelity Predict Client Outcomes in 12-Step Facilitation for Stimulant Abuse? Drug and Alcohol Dependence. 2014;134:330–336. doi: 10.1016/j.drugalcdep.2013.10.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Hettema J, Steele J, Miller WR. Motivational interviewing. Annual Review of Clinical Psychology. 2005;1:91–111. doi: 10.1146/annurev.clinpsy.1.102803.143833. [DOI] [PubMed] [Google Scholar]
  25. Hogue A, Henderson CE, Dauber S, Barajas PC, Fried A, Liddle HA. Treatment adherence, competence, and outcome in individual and family therapy for adolescent behavior problems. Journal of Consulting & Clinical Psychology. 2008;76:544–555. doi: 10.1037/0022-006X.76.4.544. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Horvath AO, Luborsky L. The role of the therapeutic alliance in psychotherapy. Journal of Consulting and Clinical Psychology. 1993;61(4):561–573. doi: 10.1037//0022-006x.61.4.561. [DOI] [PubMed] [Google Scholar]
  27. Hubbard RL, Craddock SG, Flynn PM, Anderson J, Etheridge RM. Overview of 1-year follow-up outcomes in the Drug Abuse Treatment Outcome Study (DATOS) Psychology of Addictive Behaviors. 1997;11:261–278. [Google Scholar]
  28. Imel ZE, Baer JS, Martino S, Ball SA, Carroll KM. Mutual influence in therapist competence and adherence to motivational enhancement therapy. Drug and Alcohol Dependence. 2011;115:229–236. doi: 10.1016/j.drugalcdep.2010.11.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Kaskutas LA, Subbaraman MS, Witbrodt J, Zemore SE. Effectiveness of making Alcoholics Anonymous easier: A group format 12-step facilitation approach. Journal of Substance Abuse Treatment. 2009;37:228–239. doi: 10.1016/j.jsat.2009.01.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kendall PC, Gosch E, Furr JM, Sood E. Flexibility within fidelity. Journal of the American Academy of Child & Adolescent Psychiatry. 2008;4(9):987–993. doi: 10.1097/CHI.0b013e31817eed2f. [DOI] [PubMed] [Google Scholar]
  31. Knuuttila V, Kuusisto K, Saarnio P, Nummi T. Early working alliance in outpatient substance abuse treatment: Predicting substance use frequency and client satisfaction. Clinical Psychologist. 2012;16(3):123–135. [Google Scholar]
  32. Luborsky L, Barber JP, Siqueland L, Johnson S, Najavits LM, Frank A, Daley D. The revised Helping Alliance questionnaire (HAq-II): Psychometric properties. Journal of Psychotherapy Practice & Research. 1996;5:260–271. [PMC free article] [PubMed] [Google Scholar]
  33. Martino S, Ball SA, Nich C, Frankforter TL, Carroll KM. Community program therapist adherence and competence in motivational enhancement therapy. Drug & Alcohol Dependence. 2008;97:37–48. doi: 10.1016/j.drugalcdep.2008.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McLellan AT, Kushner H, Metzger D, Peters R, Smith I, Grissom G, Pettinati H, Argeriou M. The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment. 1992;9:199–213. doi: 10.1016/0740-5472(92)90062-s. [DOI] [PubMed] [Google Scholar]
  35. McLellan AT, Luborsky L, Cacciola J, Griffith J, McGahan P, O’Brien C. Guide to the Addiction Severity Index: Background, administration, and field testing results. U.S. Department of Health & Human Services; Washington, DC: 1985. Publication No. ADM 85–1419. [Google Scholar]
  36. Meier PS, Donmall MC, McElduff P, Barrowclough C, Heller RF. The role of the early therapeutic alliance in predicting drug treatment dropout. Drug and Alcohol Dependence. 2006;83:57–64. doi: 10.1016/j.drugalcdep.2005.10.010. [DOI] [PubMed] [Google Scholar]
  37. Moyers TB, Martin T, Manuel JK, Hendrickson SML, Miller WR. Assessing competence in motivational interviewing. Journal of Substance Abuse Treatment. 2005;28:19–26. doi: 10.1016/j.jsat.2004.11.001. [DOI] [PubMed] [Google Scholar]
  38. Project Match Research Group. Matching alcoholism treatments to client heterogeneity: Project MATCH post treatment drinking outcomes. Journal of Studies on Alcohol. 1997;58:7–29. [PubMed] [Google Scholar]
  39. Ruglass LM, Miele GM, Hien D, Campbell A, Hu MC, Caldeira N, … Nunes EV. Helping alliance, retention, and treatment outcomes: A secondary analysis from the NIDA Clinical Trials Network women and trauma study. Substance Use and Misuse. 2012;47(6):695–707. doi: 10.3109/10826084.2012.659789. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Sharf J, Primavera LH, Diener M, Dropout J therapeutic alliance: A meta-analysis of adult individual psychotherapy. Psychotherapy: Theory, Research, Practice, Training. 2010;47(4):637–645. doi: 10.1037/a0021175. [DOI] [PubMed] [Google Scholar]
  41. Simpson DD. Treatment for drug abuse follow-up: Outcomes and length of time spent. Archives of General Psychiatry. 1981;38(8):875–880. doi: 10.1001/archpsyc.1981.01780330033003. [DOI] [PubMed] [Google Scholar]
  42. Simpson DD, Joe G, Broome K, Hiller M, Knight K, Rowan-Szal G. Program diversity and treatment retention rates in the Drug Abuse Treatment Outcome Study (DATOS) Psychology of Addictive Behaviors. 1997;11:279–293. [Google Scholar]
  43. Stark M. Dropping out of substance abuse treatment: A clinically oriented review. Clinical Psychology Review. 1992;12:93–116. [Google Scholar]
  44. Swift JK, Greenberg RP. Premature discontinuation in adult psychotherapy: A meta-analysis. Journal of Consulting and Clinical Psychology 2012. 2012;80(4):547–559. doi: 10.1037/a0028226. [DOI] [PubMed] [Google Scholar]
  45. Timko C, DeBenedetti A. A randomized controlled trial of intensive referral to 12-Step self-help groups: One-year outcomes. Drug and Alcohol Dependence. 2007;90:270–279. doi: 10.1016/j.drugalcdep.2007.04.007. [DOI] [PubMed] [Google Scholar]
  46. Timko C, DeBenedetti A, Billow R. Intensive referral to 12-Step self-help groups and 6-month substance use disorder outcomes. Addiction. 2006;101:678–688. doi: 10.1111/j.1360-0443.2006.01391.x. [DOI] [PubMed] [Google Scholar]
  47. Timko C, Sutkowi A, Cronkite RC, Makin-Byrd K, Moos RH. Intensive referral to 12-step dual-focused mutual-help groups. Drug and Alcohol Dependence. 2011;118:194–201. doi: 10.1016/j.drugalcdep.2011.03.019. [DOI] [PubMed] [Google Scholar]
  48. Webb CA, DeRubeis RJ, Barber J. Therapist Adherence/Competence and Treatment Outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology. 2010;78:200–211. doi: 10.1037/a0018912. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Wells EA, Donovan DM, Daley DC, Doyle S, Brigham G, Garrett SB, … Walker R. Is level of exposure to a 12-Step facilitation therapy associated with treatment outcome? doi: 10.1016/j.jsat.2014.06.003. under review. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Zhang Z, Friedmann PD, Gerstein DR. Does retention matter? Treatment duration and improvement in drug use. Addiction. 2003;98:673–684. doi: 10.1046/j.1360-0443.2003.00354.x. [DOI] [PubMed] [Google Scholar]

RESOURCES