Therapist Predictors of Treatment Delivery Fidelity in a Community Based Trial of 12-Step Facilitation

Abstract

Background and aims

Therapist characteristics may be associated with variation in consistency, quality and effectiveness of treatment delivery. We examined associations between treatment fidelity and therapist education, experience, treatment orientation, and perceived skills in a randomized, multi-site trial of Twelve Step Facilitation (TSF).

Methods

Raters scored audio-recorded, TSF sessions (n = 966; 97% of TSF sessions) from 32 community-based, trained therapists for adherence, competence, empathy, and global session performance.

Results

Therapists with graduate degrees had significantly higher adherence and global performance fidelity ratings. Therapists reporting more positive attitudes towards 12-Step groups had lower adherence ratings. Being in recovery was associated with lower fidelity in univariate tests, but higher adherence in multivariate analysis. Fidelity was higher for therapists reporting self-efficacy in basic counseling skills and lower for self-efficacy in addiction-specific counseling skills. Fidelity was also superior in group relative to individual TSF sessions.

Conclusions

Results have implications for therapist selection, training and supervision in community-based, effectiveness trials and community implementation of evidence-based treatments. To obtain high fidelity and improve outcomes, it may be preferable to choose masters level therapists who are open to learning new treatments and have good, general counseling skills.

Introduction

The delivery of treatment as intended, known as treatment fidelity, is essential for validity in clinical trials and for implementation of evidence-based treatments. The "tool kit" of fidelity procedures includes use of treatment manuals, intervention training, therapist certification, fidelity ratings of treatment sessions, and supervision informed by ratings. Ratings should measure adherence to and competence in treatment delivery, including measurement of non-specific therapist skills, such as empathy, that facilitate the therapeutic alliance (1–3). Fidelity procedures used during clinical trials or for community implementation of evidence-based treatments (EBTs) are necessary to sustain treatment quality and consistency across diverse therapists and settings over time.

The relationship of fidelity to patient outcomes is complex. In a meta-analytic review, Webb et al. (4) found no overall adherence- or competence-outcomes relationships, but wide heterogeneity across studies. Within substance abuse trials, adherence-outcome relationships have been generally positive. Higher levels of treatment adherence have predicted positive outcomes for motivational interviewing (MI) (5), brief strategic family therapy (6), and multidimensional family therapy (7). Other studies have shown adherence-outcome relationships only when therapeutic alliance was low, (8,9) or when therapeutic alliance was controlled (10). Higher levels of adherence have also predicted better outcomes in community implementation following adoption of EBTs (11). Complex fidelity-outcome findings are not surprising, since the relationship between therapist behavior and patient outcomes is affected by interactions among many variables (4). Nonetheless, it is important to understand the role of fidelity in treatment process and outcomes, and the ability of community based therapists to achieve fidelity as they implement new treatments.

Therapist Characteristics and EBT Implementation

Although efficacy trials typically employ therapists selected on the basis of relevant expertise (12), effectiveness trials approximate community treatment by training existing staff who often differ in education, training, treatment orientation, and general counseling skills (13) Identifying therapist characteristics that predict readiness to learn and higher fidelity to treatment innovations could empirically guide therapist selection and training, and assist organizational choices about treatment adoption.

A few studies have examined the effects of therapist characteristics on treatment delivery fidelity in addictions research. Therapist education predicted competence, but not adherence, in behavioral couples therapy with alcoholic men; masters-level counselors showed higher competence than bachelors-level counselors (14). Therapist pre-training characteristics did not affect post-training treatment fidelity in two MI training studies (15,16). A third study found that pre-training MI skills were superior for therapists with higher education and lower endorsement of the disease model, differences that remained at 3-month follow-up (17). Community therapists, most of whom espoused a 12-Step approach, demonstrated adequate levels of cognitive behavioral treatment (CBT) skills following training (18). Among those trained, 70% endorsed CBT as one component of effective treatment, although about half indicated that use of CBT only would conflict with their beliefs regarding effective treatment.

Clinicians' treatment orientation, experience, and education may affect readiness to implement EBTs. Clinicians who endorsed 12-Step as a primary treatment model were more likely to endorse use of Twelve Step Facilitation (TSF) and less likely to endorse use of relapse medications than clinicians who did not report 12-Step as a primary approach (19). Divergence from usual practice was identified as an important dimension in the Evidence Based Practice Attitudes Scale (20). Numerous studies have found a link between education level and readiness to implement EBTs, generally showing higher educational attainment to be associated with greater readiness to learn and implement treatment innovations (18, 21, 22).

Therapist Selection and Fidelity in Twelve Step Facilitation

TSF is an emerging EBT that seeks to increase clients’ engagement in 12-Step activities beyond formal treatment (23). TSF has been successfully delivered in individual (24–26) and group formats (27,28). While many community treatment providers report using a 12-Step orientation and even TSF (19), ability to accurately implement empirically-supported TSF has not been systematically assessed. Since openness to different EBTs has been shown to vary based on treatment orientation, implementation may be facilitated when a new treatment is similar to current practice. Thus, endorsement of a 12-Step orientation may facilitate learning manualized TSF, a premise used for therapist selection in the current study and others (25, 29). It is also possible that counselors may contaminate TSF with their usual 12-Step practice, resulting in lower fidelity. Perepletchikova & Kazdin observed that, "experience may have solidified therapists' working styles and, therefore, can hinder new learning" ((30), p. 369).

We examined this question in a study of treatment fidelity by community therapists in a multisite trial of TSF conducted within the National Drug Abuse Treatment Clinical Trials Network (CTN). The parent study, Stimulant Abuser Groups to Engage in 12-Step (STAGE-12), compared combined group and individual TSF to treatment as usual in community outpatient settings (29). In a corollary study, we evaluated the relationship of therapist characteristics with TSF treatment delivery fidelity and the relationship of fidelity with patient outcomes. Before the STAGE-12 trial, therapists completed surveys providing demographics, education, experience, treatment approaches, counseling self-efficacy, views about 12-Step treatment, and commitment to post-trial, TSF implementation. After the trial ended, we conducted independent fidelity ratings (not those used during the study) via ratings of audio files of all TSF sessions (31).

We found mixed results for fidelity and outcomes relationships; greater fidelity was associated with better employment outcomes, but worse drug composite scores on the Addiction Severity Index (ASI) at three months post-treatment. Analysis of drug composite scores showed that greater fidelity was associated with fewer days of drug use, but an increased sense of being troubled by use (32). Perhaps TSF and associated 12-Step participation promoted subjective concern about use even as amount of use decreased.

This paper focuses on therapist characteristics that predicted fidelity. Based on evidence suggesting that endorsement of and experience with 12-Step oriented treatment in current practice facilitates openness to learning manualized TSF, we hypothesized that several therapist characteristics would be associated with higher levels of adherence and competence; (a) being in recovery, (b) being an experienced addictions therapist, (c) frequent use of a 12-Step approach, (d) positive attitudes towards 12-Step groups, and (e) commitment to post-study, TSF implementation.

Methods

STAGE-12 Study Overview

STAGE-12 was a randomized, clinical trial conducted at 10 community-based, outpatient addiction treatment programs. Study participants (N = 471) were adults with stimulant abuse/dependence as either a primary or secondary drug of abuse, seeking admission or enrolled in outpatient treatment. Participants were randomly assigned to either treatment-as-usual (TAU; 5–15 hours of weekly treatment) or TAU plus STAGE-12 Substitution in which five group and three individual TSF sessions replaced the same number of TAU sessions (33). See Donovan et al. (29) for a complete description of STAGE-12 study procedures and the TSF treatment. Results of STAGE-12 were mixed; TSF participants were significantly more likely to be abstinent during the 8-week treatment phase, but not at 3- or 6-month follow-ups. TSF participants also had significant reductions in ASI Drug Composite scores from baseline to 3-month follow-up and engaged in more types of 12-Step activities throughout treatment and follow-ups (29).

Study Therapists

All therapists (N=106) at study sites were screened for inclusion according to the following criteria: (a) credentialed to provide substance abuse services, (b) willing to participate (c) approved by the treatment program’s administration, (d) willing to be randomized, and (e) familiar with the 12-Step orientation. There were 39 therapists (37%) who met criteria and were included in the study pool; two from each site were chosen at random from the pool to conduct the TSF treatment. Remaining therapists were available to be trained as replacement therapists, four of whom did so. Supervisors were also trained as back-up TSF therapists. In total, there were 34 therapists (including back-up supervisors) who conducted the TSF intervention, 32 of whom completed pre-trial therapist surveys and were included in our analysis. They were predominantly Caucasian (69%) women (69%) with a mean age of 51 years (SD=9.4). Most (81%) had at least 5 years of counseling experience and 53% had a graduate degree.

Fidelity Raters

We recruited raters from local graduate programs to conduct ratings of all STAGE-12, TSF sessions. The nine raters (seven with masters’ degrees and two with doctoral degrees) averaged five years of clinical experience (SD =4.05), seven years of research experience (SD =5.96) and one year of rating experience (SD=2.95). The expert rater was a doctoral level psychologist with extensive experience in fidelity monitoring procedures (e.g., (34)).

Measures

Fidelity Ratings

Raters used the Twelve Step Facilitation Adherence Competence Empathy Scales, (TSF ACES; 31) to evaluate fidelity to the manualized treatment. Ratings assessed five dimensions of fidelity using 6-point scales: 1) adherence – delivery of specific treatment content; 2) competence - the skill of content delivery; 3) global empathy – the therapist's effort to understand the clients’ perspectives (adapted from the Motivational Interviewing Treatment Integrity scale) (35); 4) proscribed therapist behaviors – behaviors that detract from general therapist skill and should not occur (e.g., excessive self-disclosure); and 5) global session rating – overall session performance. TSF ACES has one content-specific scale that measures adherence and competence for groups and one scale each for individual session, 1–3. Empathy, proscribed behaviors and overall session performance were assessed for all group and individual sessions using the same scale items. Sample items are shown in Appendix 1.

Five summary measures derived for each session have good to excellent inter rater reliability, with intraclass correlations of .91 for mean adherence, .90 for mean competence, .80 for global session score, .83 for mean proscribed behaviors, and .69 for global empathy. Internal consistency computed with Cronbach's alpha for summary measures that are based on multiple items is acceptable for adherence (.69) and competence (.71) and low (.47) for proscribed behaviors. See Campbell et al. for a further description of psychometric characteristics of the ratings scale (31).

Therapist Surveys

Therapist surveys assessed demographics, workforce characteristics (e.g., education, experience, certifications/ licensures) and treatment orientations (13). Study therapists also completed a 3-item measure of Attitudes and Beliefs about 12-Step Groups scale (general attitudes about 12-Step groups) (36,37) and a 9-item measure of attitudes toward controversial aspects of 12-Step groups (36,37). The STAGE-12 Integration Goal Commitment Scale, adapted from Hollenbeck et al. (38), assessed intention to integrate the TSF intervention into ongoing treatment practices. The Addiction Counseling Self-Efficacy Scale (39) measured counselors’ confidence in basic counseling skills, addiction-specific counseling skills, and group counseling skills.

Procedures

Therapist Survey Procedures

Study procedures were approved by The University of California, San Francisco and Oregon Health and Science University Institutional Review Boards (IRBs). All available outpatient therapists clinical supervisors, directors, and clinical, support staff (N=115) at the 10 participating sites completed workforce surveys prior to randomization of therapists to STAGE-12 TSF. Therapists and supervisor/directors convened with study staff in groups at each site to receive instructions, provide consent and complete surveys. Eligible staff who were unavailable completed consent procedures via telephone and surveys left with a clinical manager.

Fidelity Ratings Procedures

Raters viewed the STAGE-12 therapist training video and completed a one-day training. Prior to rating study sessions, raters achieved a criterion level of inter-rater reliability with the ratings expert on audio recorded, practice sessions conducted by STAGE-12 counselors. The STAGE-12 trial audio recorded all TSF sessions; TAU sessions were not recorded. Audio recordings of all TSF group (n=512) and individual (n=487) sessions were randomly assigned to certified raters for review Of the recorded sessions, 33 were either incomplete or of poor audio quality, leaving 966 rated sessions. Sessions were assigned to each rater in sets of 20; one session per set was randomly assigned to the study expert for co-rating to monitor ratings consistency. See Campbell et al. (31) for more detail.

Data Analysis

Therapist Predictor Variables

The analysis included four categorical variables: a) therapist experience (< 5 years/≥ 5 years), b) graduate degree (yes/no), c) recovery status (yes/no), d) frequent use of 12-Step (yes/no), and six continuous therapist variables: a) commitment to STAGE-12 TSF in ongoing clinic practice, b) general attitudes toward 12-Step groups, c) attitudes toward controversial aspects of 12-Step groups, d) self-efficacy for basic counseling skills, e) for addiction counseling skills and f) for group counseling skills. Scales were reverse-scored for the goal commitment and attitudes toward controversial aspects of 12-Step; higher scores indicated better goal commitment and more positive attitudes.

Fidelity Outcome Variables

Four TSF ACES summary measures were calculated for all TSF sessions and used as outcome variables: 1) mean adherence, 2) mean competence, 3) global empathy rating, and 4) global session score. The proscribed behaviors summary measure was removed from analysis due to low internal consistency.

Statistical Analysis

Given the lack of studies in this area of fidelity research, as well as the therapist sample size relative to the number of therapist variables of interest, the analysis plan was designed to balance the need to restrict the size of the statistical models against the goal of identifying important relationships. We used a three stage model building approach. First, relationships between therapist predictors and fidelity measures were assessed using ANOVAs for the categorical predictors and Spearman Rank Correlation coefficients for continuous variables. Second, to control for possible correlations among the predictors, all predictors that reached a significance level of p ≤ 0.2 in the univariate analyses were entered into a regression model using a forward, stepwise model selection process. Third, variables that reached the 0.05 significance level in the multivariate, stepwise model for each fidelity outcome variable were used in multi-level regression models to adjust for correlations within sites and within counselors. Session type (i.e., group, individual sessions 1, 2, and 3) was evaluated for use as a covariate, using alpha less than .05 for model inclusion.

Results

Univariate Analyses

Table 1 summarizes mean fidelity ratings by therapist characteristics. Therapist mean scores ranged from 4.76 (SD = 0.84) for adherence items to 5.08 (SD = 1.08) for the global empathy rating (a rating of 5 = satisfactory). Therapists reporting graduate degrees and less than five years of experience had significantly higher fidelity ratings on each measure. Contrary to our hypothesis, therapists in recovery had significantly lower fidelity ratings on adherence, competence and empathy than non-recovering therapists. Frequent use of a 12-Step orientation was associated with significantly lower ratings on all fidelity measures. Fidelity ratings also differed by session type; group sessions had higher adherence, competence and global session scores than individual sessions.

Table 1.

Mean scores on fidelity summary measures for therapist predictor variables and session type

		Fidelity Summary Measure
Characteristics	%	Adherence M (SD) F value	Competence M (SD) F value	Empathy M (SD) F value	Global M (SD) F value
Overall	N=32	4.76 (0.84)	4.81 (0.97)	5.08 (1.08)	4.91 (1.03)
Education
No Graduate Degree	50.2	4.62 (0.79)	4.64 (1.00)	4.92 (1.15)	4.71 (1.07)
Graduate Degree	49.8	4.91 (0.79)	4.98 (0.92)	5.24 (0.97)	5.12 (0.94)
		27.06**	28.66**	21.49**	38.63**
Work Experience
< 5 years	17.9	5.01 (0.81)	5.24 (0.83)	5.38 (0.92)	5.33 (0.92)
≥ 5 years	82.1	4.71 (0.84)	4.71 (0.98)	5.02 (1.10)	4.82 (1.03)
		17.75**	40.64**	15.71**	34.42**
Recovery
No	42.3	4.84 (0.85)	4.90 (0.99)	5.18 (1.02)	4.98 (1.04)
Yes	57.2	4.71 (0.83)	4.74 (0.96)	5.01 (1.11)	4.86 (1.02)
		4.92*	6.61*	5.52*	3.07
12-Stepa
Frequent use	84.5	4.69 (0.84)	4.71 (0.97)	4.97 (1.11)	4.82 (1.03)
Infrequent use	15.5	5.14 (0.72)	5.35 (0.79)	5.69 (0.56)	5.43 (0.82)
		35.93**	56.09**	57.24**	45.00**
Session Type
Group	51.1	4.99 (0.73)	5.16 (0.84)	5.06 (1.11)	5.12 (0.95)
Individual 1	21.7	4.67 (0.80)	4.52 (0.92)	5.00 (1.13)	4.72 (1.04)
Individual 2	16.4	4.45 (0.89)	4.43 (0.98)	5.23 (0.96)	4.66 (1.06)
Individual 3	10.9	4.38 (0.98)	4.32 (1.07)	5.12 (1.08)	4.69 (1.11)
		28.48**	48.82**	1.45	14.11**

^aClassification of therapy approach was defined as a respondent reporting “much” or “very much” use (frequent use) of the specified therapy.

^*p-value < 0.05

^**p-value < 0.01

Higher commitment to implementing TSF post-clinical trial was associated with lower adherence, competence, and global session rating (See Table 2). There was also an inverse relationship between attitudes towards 12-Step treatment and fidelity -- more positive general attitudes were correlated with lower fidelity on all four fidelity measures. Similarly, more positive attitudes towards controversial aspects of 12-Step were associated with lower competence, empathy and global session rating. These findings were contrary to expectations. Therapists who reported higher self-efficacy in basic counseling skills tended to have higher ratings in all fidelity categories. Conversely, reporting higher self-efficacy in addiction-specific, counseling skills was correlated with lower adherence, competence, and global session rating, as was reporting higher self-efficacy in group counseling. Contrary to this general pattern, higher ratings of self-efficacy in addiction-specific counseling skills were associated with higher empathy ratings.

Table 2.

Correlation matrix: therapist survey scales by fidelity summary measure

	Fidelity Summary Measure
Scale	Adherence	Competence	Empathy	Global
Goal commitment	−0.12**	−0.09**	0.02	−0.08*
General attitudes toward 12-step groups	−0.21**	−0.18**	−0.10**	−0.14**
Attitudes toward controversial aspects of 12-step groups	−0.05	−0.11**	−0.13**	−0.09**
Self-efficacy in basic counseling skills	0.07*	0.23*	0.30**	0.20**
Self-efficacy in addiction-specific counseling skills	−0.15**	−0.07*	0.09**	−0.09**
Self-efficacy in group counseling skills	−0.14**	−0.11**	0.01	−0.11**

^*p-value < 0.05

^**p-value < 0.01

Multivariate Analysis

In the hierarchical models, after controlling for other predictors and the session type covariate, graduate degree had a significant effect (Table 3). Therapists with graduate degrees had higher adherence (β = 0.37, p < 0.05), and global performance (β = 0.54, p < 0.05) ratings than those without graduate degrees. Recovery status findings differed from univariate results; therapists reporting recovery had higher adherence ratings (β = 0.35, p < 0.05). Reporting more positive general attitudes toward 12-Step groups was associated with lower adherence (p < 0.05); for every one unit increase in positive attitudes there was a 0.15 decrease in adherence ratings. Self-efficacy in basic counseling skills was consistently predictive of higher fidelity (p < 0.05); each one unit increase in mean self-efficacy score corresponded to a 0.38 unit increase in adherence, a .52 increase in competence, and a .56 increase in empathy ratings. In contrast, self-efficacy in addiction-specific counseling skills was associated with lower adherence (p < 0.05); each unit increase in addiction-specific skills self-efficacy was associated with a .38 decrease in adherence.

Table 3.

Mixed effects model estimates for fidelity summary measures (N=32)

	Fidelity Summary Measure
	Adherence		Competence		Empathy		Global
Therapist Variable	Coef.	|z|	Coef.	|z|	Coef.	|z|	Coef.	|z|
Graduate Degree	0.37	2.31*	0.38	1.76	0.33	1.39	0.54	2.14*
≥ 5 Years Counseling	0.27	1.42	0.13	0.52	0.27	0.93	0.22	0.74
In Recovery	0.35	2.15*	0.29	1.29	0.34	1.35	0.48	1.84
Goal Commitment	--	--	--	--	0.20	1.09	--	--
General Attitudes Towards 12-Step	−0.15	1.99*	−0.13	1.31	−0.19	1.52	--	--
Frequent Use 12-Stepa	−0.17	0.76	−0.26	0.88	−0.25	0.72	−0.50	1.87
Self-efficacy in Basic Counseling Skills	0.38	2.29*	0.52	2.24*	0.56	2.14*	0.52	1.91
Self-efficacy in Addiction-Specific Counseling Skills	−0.38	1.99*	−0.39	1.47	−0.31	1.04	−0.43	1.37
Session Typeb
Individual 1	−0.32	5.04**	−0.64	9.64**	--	--	−0.41	5.48**
Individual 2	−0.53	7.68**	−0.73	9.96**	--	--	−0.46	5.51**
Individual 3	−0.63	7.68**	−0.86	9.94**	--	--	−0.45	4.64**

^aReference is infrequent use of therapy approach.

^bReference is group session.

^*p-value ≤ 0.05

^**p-value ≤ 0.01

Discussion

Therapist characteristics predicted variations in TSF treatment delivery. Univariate analyses found that possessing a graduate degree, less than five years of addictions counseling experience, and self-efficacy in basic counseling skills were associated with higher fidelity. Contrary to expectation, lower adherence and competence were associated with therapist recovery status, frequent use of 12-Step, positive attitudes towards 12-Step and commitment to TSF implementation. In multivariate analysis, positive attitudes towards 12-Step remained predictive of lower adherence, although recovery status shifted with those in recovery showing higher adherence. Selecting therapists who are familiar with and endorse 12-Step oriented treatment may actually interfere with adherence to empirically-based TSF, perhaps due to the intrusion of usual practice into manualized procedures. All STAGE-12, TSF therapists achieved satisfactory fidelity to become certified to deliver TSF in the parent study (29). Supervision and periodic fidelity review occurred throughout the study, but there was no de-certification procedure. It is possible that therapists who a endorsed 12-Step approach included their usual practice in treatment delivery over time, which they may have believed fit the TSF treatment because of its perceived similarity to TSF. Moreover, their reported use of 12-Step treatment as a frequent approach may not have accurately reflected the techniques they actually used. A study of treatment as usual found that clinicians did not demonstrate frequent use of interventions they reported using and that there were few significant relationship between their stated orientations and techniques they actually used (40).

Multivariate analysis showed that graduate education remained significantly predictive of higher adherence and global session performance. This finding supports others showing superior fidelity (14, 17) and more positive attitudes towards EBTs (18, 21, 22, 41) among therapists with graduate training. Advanced education may expose therapists to a variety of treatment orientations, increasing flexibility to learning new treatments or new variations of standard clinical practice. Emphasizing the evidence base for any treatment may also be important in graduate education. Hershenberg et al. (42) propose that graduate training that consistently integrates science and practice will produce clinicians equipped to use evidence-based practices. Calls for advanced education among the addictions treatment work force have increased to meet the demands of treating complex problems (e.g., co-occurring disorders) with evidence-based practices (43,44).

Self-efficacy was significantly predictive of fidelity. Therapists reporting higher self-efficacy in basic counseling skills demonstrated higher adherence, competence and empathy, while therapists reporting higher self-efficacy in addiction-specific counseling skills had lower adherence to manualized TSF. Findings suggest that therapists may have accurately perceived their general skill and that such skill enhanced their ability to deliver a new treatment with greater accuracy and quality. Conversely, self-efficacy for addiction-specific counseling skills decreased adherence, a finding in keeping with the interpretation that, when usual practice is similar to a new treatment, it may more easily intrude.

Fidelity was significantly associated with session type. Generally, group session fidelity was higher than fidelity for individual sessions. Individual session fidelity may have suffered due to limited practice (i.e., therapists conducted groups using the same basic format more frequently). It is also possible that elements of individual sessions were difficult to deliver. For example, individual sessions included interventions for participants who had not followed through with 12-Step meeting attendance. These treatment elements were sometimes awkwardly modified or omitted by therapists, decreasing fidelity for these sessions. Detailed fidelity ratings allow for this kind of in-depth examination to inform training, supervision and future treatment refinement, such as examining whether individual sessions add to a group treatment's effectiveness.

Strengths and Limitations

The current study used a rating scale that reliably assesses multiple facets of fidelity across group and individual sessions and independent raters to assess fidelity of all available TSF treatment sessions, approximately 97% of the trial's sessions. The completeness of our fidelity data and the relatively large number of therapists conducting sessions (N=32) in this multisite trial provided an unusual opportunity to study individual differences in therapists' treatment delivery fidelity. However, the sample size is small for conducting analyses and our results should be interpreted with caution. The study examined TSF fidelity in community treatment sites employing agency therapists for intervention delivery, both hallmarks of an effectiveness trial. Thus, results may generalize to therapists in community treatment. However, limiting factors include use of CTN participating sites, whose therapists may differ from their colleagues in non-participating agencies (45). In addition, we do not know how many therapists met study criteria for inclusion, but were disinterested in participation, although STAGE-12 study staff believed that number to be small (S. Garrett, personal communication, March 21, 2012). These considerations may limit generalizability of findings. Another limitation was the absence of fidelity ratings for the comparison (TAU) condition. As a result, we were unable to determine discriminant validity for TSF ACES adherence items and unable to identify how TAU did or did not differ from the manualized TSF treatment. Correlations among our fidelity outcome measures were high, ranging from .35 (empathy with adherence) to .82 (adherence with competence; (31)). Despite the lack of fully independent information in each measure, we conducted separate analyses of adherence and competence because they are considered important dimensions of treatment delivery fidelity (1) and have shown differential predictions of patient outcomes (8–10). We retained empathy as a measure of nonspecific therapist skill and the global session rating because it was the criterion measure for STAGE-12 therapist certification.

Conclusions

TSF is an emerging, evidence based treatment (46), that is compatible with community treatment, increasing its potential for adoption. Comprehensive fidelity data affords the opportunity to identify predictors of fidelity, including therapist characteristics, that may inform community treatment programs in hiring and training therapists, and in monitoring treatment delivery. Results support the recommendation that "SUD treatment practitioners should continue to earn degrees in higher education" (44) (p.7). They also indicate the benefit of basic counseling skills as a foundation for learning treatment innovations. Consistency and quality of treatment delivery may be enhanced by hiring and training masters level therapists who report and demonstrate good general skills. Training and supervision may be improved by fostering openness to new treatments, clarifying differences between current practice and new treatments, especially those perceived to be similar, and addressing non-specific treatment skills in the training of manualized treatments (47). If we are able to identify therapist characteristics and training that facilitate fidelity, as well as determine the "core" ingredients (48) of TSF treatment, we will be able to develop a practical treatment for implementation in community addictions treatment.

Appendix 1. TSF ACES Sample Items and Scalesa

1.A. Adherence and Competence Items

1.A.1 Group Session Item:

a. Group check-in, reviewing reaction to recovery tasks: To what extent did the group counselor review members’ reactions to last week’s group session recovery tasks (meetings, readings, sponsor, using telephone to contact 12-Step peers, and completing written assignments)?

1.A. 2. Individual Session Item:

a. Session 1, 12-Step philosophy: To what extent did the individual counselor review and discuss the 12-Step program's philosophy of recovery, structure and terminology of meetings, and any concerns of the participant regarding participation?

Adherence Scale:

Not At All		Somewhat				Considerably				Extensively
□	1	□	2	□	3	□	4	□	5	□	6

Competence Scale: How well did the counselor handle this item?

Unsatisfactory		Fair				Good				Excellent
□	1	□	2	□	3	□	4	□	5	□	6

1.B. Global Skills Items

1.B.1. Empathy. Overall, how well did the counselor understand or make an effort to grasp the client's perspectives?

1.B2. Global Session Rating: Overall, how well did the counselor conduct this specific session?

Unsatisfactory		Fair				Good				Excellent
□	1	□	2	□	3	□	4	□	5	□	6

^a The full TSF ACES Rating Scale and Manual can be downloaded from the CTN Dissemination Library at http://ctndisseminationlibrary.org/PDF/795 TSFACES pdf.