What is usual about “treatment-as-usual”? Data from two multisite effectiveness trials

Abstract

Despite increased emphasis on broadening the implementation of empirically supported therapies (ESTs) to improve standard clinical practice and patient outcomes, objective descriptions of what actually constitutes standard practice in community-based drug abuse treatment do not exist. We present data from independent ratings of 379 audiotapes drawn from the “treatment-as-usual” arm of two multisite randomized effectiveness trials in the National Institute on Drug Abuse Clinical Trials Network. As expected, the most frequently occurring strategies involved assessing the participant’s substance use and social functioning, asking open-ended questions, discussing problems and feedback, and giving advice and direction. However, a number of interventions associated with ESTs were very rarely implemented in these early sessions. These data suggest missed opportunities for optimally engaging patients in the early stages of treatment and enhancing substance use outcomes and only moderate success to date of efforts to bridge the gap between research and practice.

1. Introduction

What therapeutic approaches and techniques do clinicians commonly use in community-based programs to help patients address their addiction problems? Recent commentaries on this issue (Miller, 2007) highlight the dearth of data regarding what clinicians actually do during counseling sessions. The National Institute on Drug Abuse (NIDA) Blue Ribbon Task Force on Health Service Research has highlighted the need for more studies to evaluate and describe commonly used interventions and practices in standard treatment to determine whether new interventions will lead to improved outcomes relative to practices that are traditionally used (Compton et al., 2005; NIDA, 2004). Moreover, given the increasing pressure from funding agencies and third-party insurer mandates on treatment professionals to institute quality improvement initiatives and to provide empirically supported therapies (ESTs; McGovern, Fox, Xie, & Drake, 2004; McLellan & Meyers, 2004; Miller, Zweben, & Johnson, 2005), it is striking that there are virtually no objective data documenting what actually constitutes treatment-as-usual (TAU) in community programs (Miller, 2007).

Existing information about what constitutes standard practice typically comes from surveys asking program directors and clinicians about what treatment approaches they use with their clients. Self-report data often indicate clinicians report using high levels of ESTs and doing so effectively (Ball et al., 2002; McCarty et al., 2007) even if they have not actually changed their usual practice. There is some evidence that when clinicians were trained to use ESTs, they typically rate themselves as having higher levels of treatment fidelity than as evaluated by independent raters (Carroll, Nich, & Rounsaville, 1998; Miller, Yahne, & Tonigan, 2003). What actually occurs during substance abuse counseling sessions remains largely unknown (Carroll & Rounsaville, 2007; Miller, 2007).

The lack of objective information on what interventions comprise TAU also presents several problems for addiction treatment research. Without meaningful independent evaluations of standard practice, it is difficult to evaluate the appropriateness and feasibility of testing ESTs in community settings, the extent to which ESTs have been adopted in dissemination efforts, or what type of training and supervision is needed or most likely to alter practice. Without demonstrations that ESTs are more effective than treatment as usually provided, policy makers, payors, and clinicians are unlikely to embrace these treatments (Carroll & Rounsaville, 2007).

To date, there have been only a handful of randomized trials that have compared ESTs for substance use disorders to standard practice using TAU as a comparison condition (e.g., McLellan, Grissom, Zanis, & Randall, 1997; Miller et al., 2003; Morgenstern, Blanchard, Morgan, Labouvie, & Hayaki, 2001; Rawson et al., 2004). This may be in part because TAU comparisons raise a host of methodological challenges, such as the difficulty in specifying interventions and inherent variability of unstructured approaches (Bickman, 2002). Most existing effectiveness studies typically describe TAU in very general terms (e.g., weekly group or individual drug counseling) or as consisting of a myriad of programmatic activities, including encouragement to attend 12-step programs, group addiction-focused therapy and lectures, “relapse prevention” counseling, and case management (Morgenstern et al., 2001; Rawson et al., 2002). Systematic and independent evaluation of the types of techniques used by clinicians or the skill in which they implemented them in TAU has never been reported in the literature despite that assessment of treatment fidelity is now a requirement of efficacy studies (Chambless & Hollon, 1998).

In this report, we provide a description of the therapeutic strategies and techniques used by clinicians delivering the TAU comparison condition during the early phase of treatment within nine community programs participating in two multisite randomized effectiveness trials in the NIDA Clinical Trials Network (CTN; Ball et al., 2007; Carroll et al., 2006). One protocol incorporated motivational interviewing (MI; Miller & Rollnick, 2002) into a one-session standard intake assessment (Carroll et al., 2006), henceforth referred to as motivational interviewing assessment (MIA). The other protocol (Ball et al., 2007) examined the effectiveness of a three-session adaptation of the Motivational Enhancement Therapy manual used in Project MATCH (Miller, Zweben, DiClemente, & Rychtarik, 1992), henceforth referred to as MET. In both protocols, the effectiveness of MIA/MET was compared with outpatient individual TAU sessions that were equal in length and format. In both protocols, all sessions were audiotaped, and independent raters blind to study condition evaluated counselor adherence and competence for a range of clinical interventions.

This report addresses three questions regarding the nature and delivery of TAU in these studies. First, what are the primary counseling strategies or practices that characterize TAU? We were particularly interested in the extent to which interventions associated with ESTs were evident, given that, prior to the start of the trials, systematic reports from the participating site directors and clinicians collected prior to protocol initiation and clinician training indicated they believed they were using a variety of ESTs in their standard practice (Ball et al., 2002; Carroll et al., 2002). Therefore, based on their reports, we anticipated that in both protocols TAU would be characterized by assessment/evaluation of substance use and psychosocial functioning, a discussion of the problems for which the client sought treatment, and formal treatment planning. We also expected TAU clinicians to focus their interventions on orienting participants to the program, providing case management, and promoting 12-step program principles and involvement (e.g., attending meetings, emphasizing spirituality/higher power, promoting total abstinence and the concepts of powerlessness and loss of control).

Second, are a broader range of strategies being used in the three-session MET in comparison with the one-session MIA protocol? Because of the two additional sessions in the MET protocol, we expected that TAU clinicians in this protocol would use a greater variety of drug counseling strategies and more of the EST-based interventions such as coping skills training, working with cognitions, addressing ambivalence, discussing change plans, and addressing HIV and risk behavior reduction than TAU clinicians in the one-session MIA protocol.

Third, what relationships exist between key clinician characteristics (e.g., education, licensure, and recovery status), counseling orientation, and the treatment strategies they used in TAU? We expected that clinicians with higher levels of education (e.g., master’s degrees) and who were licensed would make use of interventions associated with ESTs more frequently than clinicians with lower levels of education and those who were not licensed, whereas clinicians who reported having had substance abuse problems themselves or are oriented primarily to the 12-step approach to counseling would make greater use of interventions associated with disease model/Twelve-Step Facilitation approaches.

2. Methods

2.1. Overview of the MIA/MET protocols

The MIA protocol (Carroll et al., 2006) was designed for sites that typically offer group treatment only. Participants seeking outpatient substance abuse treatment at one of four participating sites were randomized to either the initial intake/assessment session as typically conducted or a parallel session in which MI techniques had been integrated. Following the single protocol session, participants then received standard treatment as practiced at that site. In the MET protocol, participants seeking outpatient substance abuse treatment at one of five participating sites were randomized to receive either three individual TAU or MET sessions (Ball et al., 2007). Clinician training, inclusion/exclusion criteria, and assessments were parallel in both protocols. In terms of outcomes, the MIA protocol resulted in significantly better 4-week client retention and reduced days of primary substance use in participants whose primary substance was alcohol rather than drugs (Carroll et al., 2006). The MET protocol did not show significant differences in retention or reduced substance use at the end of the 4-week treatment phase. However, there was a statistically significant difference during follow-up, where participants assigned to MET sustained reduced primary substance use over a 12-week follow-up period, whereas TAU increased during the follow-up weeks (Ball et al., 2007).

2.2. Participants

2.2.1. Clinicians

In both protocols, clinicians who volunteered to participate in the protocol were randomized to deliver either MIA/MET or TAU as currently practiced at the site at which they were employed. Random assignment of clinicians to interventions was done to balance level of clinician skill, experience, and commitment to ESTs across treatment conditions. Clinicians provided written informed consent for participation when required by the site’s local institutional review board. Data presented here are based on session tapes from the 25 clinicians who implemented TAU in the two protocols. All were employed in one of nine outpatient community treatment programs that served diverse samples of substance users. Programs were located in California, Connecticut, Oregon, Pennsylvania, Maryland, and Virginia in rural, suburban, and urban settings. Fourteen clinicians from four programs implemented a one-session TAU in the MIA protocol, and 11 clinicians from the other five programs implemented a three-session TAU in the MET protocol.

2.2.2. Patient participants

In both protocols, participants were required to be (a) English speaking, (b) 18 years or older, (c) seeking outpatient treatment for any substance use problem with use of alcohol or any illicit drug at least once in the 28 days prior to randomization, and (d) willing to participate in all study procedures. Minimal exclusions (severe medical or psychiatric instability prohibiting outpatient treatment enrollment, residential instability or imminent incarceration, seeking detoxification) were placed on potential participants to obtain representative community samples. The participants in the MIA protocol (Carroll et al., 2006) included 423 outpatients with mixed primary substance abuse problems (48% alcohol, 6% cocaine, 21% marijuana, 5% opiates, 19% methamphetamines, and 1% benzodiazepines). The MET protocol (Ball et al., 2007) included 461 outpatients with a similar range of substance use problems (29% alcohol, 23% cocaine, 16% marijuana, 9% opiates, 4% methamphetamines, 11% alcohol and drug, and 8% other). On average, TAU participants from the MIA/MET protocols were in their early to mid-30s (MIA = 31.9, SD = 9.8; MET = 35.5, SD = 9.9). Proportionally more men (MIA = 58%, MET = 72%, χ²(1) = 10.4, p < .001) and African Americans (MIA = 9.3%, MET = 45%, χ²(1) = 94.5, p < .000) participated in the MET than in the MIA protocol; the remaining ethnic demographic makeup of both groups was primarily Caucasian/European American. Participants in both protocols had completed on average 12 years of education, with less than half of each group currently being employed (MIA = 40.2%, MET = 42.9%).

2.3. Treatment as usual

Clinicians randomly assigned to deliver TAU in either protocol followed standard counseling procedures according to their agency guidelines. Sessions in the MIA protocol were about 2 hours in duration, whereas sessions in the MET protocol were about 45 to 55 minutes in duration. To minimize the impact of the research protocol on TAU at the sites, we asked TAU clinicians not to change their standard intake/counseling procedures, and as such, they did not receive any additional training or supervision beyond that which was already in place at each program. Clinicians assigned to conduct MIA/MET were trained following a detailed protocol with regular clinical supervision that focused on supporting fidelity to the MIA/MET treatment manual and refining MI skills (Ball et al., 2007; Carroll et al., 2006; Martino, Ball, Nich, Frankforter, & Carroll, in press). In both conditions, all sessions were audiotaped and sent to the coordinating center for independent rating.

2.4. Process assessments

Fifteen independent tape raters were trained to assess clinician adherence and competence (Martino et al., in press). Raters were blind to treatment protocol (MIA vs. MET), treatment condition (MIA/MET vs. TAU), session number, and site. The extent to which a range of counseling interventions were delivered in the protocol (i.e., adherence) and the skill with which the clinicians delivered those interventions (i.e., competence) were assessed using a process rating system adapted from the Yale Adherence and Competence Scale (YACS; Carroll et al., 2000), a reliable and valid system for evaluating clinician adherence and competence across several types of ESTs and commonly used interventions in addiction treatment. Each intervention was rated on two dimensions using a seven-point Likert scale. First, raters indicated the extent to which the clinician delivered the intervention (adherence; 1 = not at all to 7 = extensively). Second, they rated the skill with which the clinician delivered the intervention (competence; 1 = very poor to 7 = excellent). Item definitions and rating decision guidelines were specified in a detailed rating manual (Ball, Martino, Corvino, Morganstern, & Carroll, 2002).

This report focuses on the 31 items that address specific therapeutic strategies. Ten items detailed MI-consistent interventions central to the delivery of the MIA and MET interventions. Confirmatory factor analysis of these MI-consistent items supported a two-factor model (Martino et al., in press): 5 items included basic MI skills that underpin the empathic and collaborative stance of MI such as use of open-ended questions, reflective statements, and MI style or spirit. The remaining 5 items involved advanced MI skills for evoking client motivation and commitment to behavior change, such as heightening discrepancies and change planning. Twenty-one items assessed a broad range of addiction counseling interventions. These items included 10 interventions seen as antithetical to MI (e.g., confrontation of denial, therapeutic authority, unsolicited advice, emphasis on abstinence) or associated with theoretical orientations or approaches other than MI (e.g., skills training/cognition, disease model, psychodynamic) and 11 general addiction counseling items generated from systematic interviews conducted with program directors and clinicians’ self-report of their usual practice and therapeutic strategy at the nine participating programs prior to the initiation of the study (Ball et al., 2002). Interrater reliability estimates (Shrout & Fleiss, 1979) indicated that 29 of 31 adherence items showed good to excellent reliability (intraclass correlations [ICCs] ranged from .66 to .99) with only two items (skills training and psychodynamic interventions) having fair reliability (ICCs = .55 and .57, respectively), which reflected their very low frequency during sessions (Martino et al., in press). A detailed description of the rater training process, psychometric analysis of the rating instrument, and description of the levels of MIA/MET fidelity is described in another report (Martino et al., in press).

In this report, we focused solely on the TAU condition as delivered in these protocols. For the MIA protocol, 423 participants were assigned to treatment (209 MIA/214 TAU); of these, 377 completed their protocol session (186 MIA/ 191TAU). All recorded and audible TAU sessions from the MIA/one-session protocol (n = 160; 84% of the treatment-exposed participants) were rated. Because of the much larger number of tapes generated in the MET protocol, we elected to randomly select tapes only from those participants who had completed all three standard counseling sessions (59% of all study participants). We selected a minimum of 10 sets of three-session tapes from within each of the five sites in the MET protocol. We then randomly selected additional tapes from any clinician who was not included through this process to ensure we had ratings for all clinicians in the study. This procedure resulted in 425 tapes (206 MET/219 TAU), representing 178 participants (83 MET/95 TAU).

2.5. Data analyses

TAU is described in terms of the means and standard deviations for the adherence and competence ratings and percent of sessions during which interventions occurred for the 31 items. Simple analysis of variances were conducted to analyze differences by session in the MET protocol for all 31 items and to evaluate mean differences in counseling interventions and general clinician ratings based on education (dichotomized as less than a master’s degree or a master’s degree or above), licensure and recovery status, and clinician orientation (dichotomized as a scale score > 4 on a five-point scale of treatment allegiance).

3. Results

3.1. Clinician characteristics and orientation

As shown in Table 1, clinicians delivering TAU in the two protocols were predominantly women, Caucasian, and on average about 38 years old. They had been employed at their agencies for an average of 3.2 years, had almost 6 years of substance abuse counseling experience, and completed approximately 14 years of education. Most had degrees in counseling, social work, psychology, or marriage and family therapy. Approximately 40% reported having had substance use problems in the past. Prior to the initiation of the protocol, clinicians reported that their usual practice included a wide range of orientations, typically blending 12-step/disease model and cognitive-behavioral therapy (CBT) together or with other treatment approaches (see Ball et al., 2002). For example, a large proportion of clinicians in each protocol endorsed an eclectic counseling approach utilizing “much of everything” as opposed to relying on one dominant theoretical orientation (MIA, 27.3%; MET, 28.6%).

Table 1.

Characteristics of clinicians delivering TAU by protocol

Characteristics	1-Session protocol, MIA (n = 14)	3-Session protocol, MET (n = 11)
Gender, female	9 (64.3)	9 (81.8)
Ethnicity
Caucasian	11 (78.6)	8 (72.7)
African American	0	1 (9.1)
Hispanic	2 (14.3)	1 (9.1)
Native American	1 (7.1)	0
Middle Eastern	0	1 (9.1)
Highest Degree Completed
High School/associates	8 (57.1)	2 (18.2)
Bachelors	2 (14.3)	4 (36.4)
Masters	4 (26.6)	5 (45.5)
Holds license	11 (78.6)	5 (45.5)
Self-report: being “in recovery”	5 (41.7)	3 (42.9)
Age, M (SD)	40.4 (9.2)	35.6 (10.0)
Years of education	14.4 (5.7)	13.9 (5.6)
Years employed at agency	3.6 (3.5)	2.7 (3.8)
Years counseling experience	5.5 (3.7)	6.2 (5.1)
Years held highest degree	8.7 (10.7)	7.6 (5.9)
Treatment orientation
DC	2 (14.3)	0
CBT	1 (7.1)	1 (9.1)
Motivational interview	0	1 (9.1)
Rogerian	1 (7.1)	0
Psychodynamic	0	0
“A little of everything”	1 (7.1)	1 (9.1)
Much of everything	4 (28.6)	3 (27.3)
DC/CBT mostly	1 (7.1)	2 (18.2)
DC and others	0	2 (18.2)
CBT and others	3 (21.4)	0
DC/CBT and others	1 (7.1)	1 (9.1)

Note. Values are presented as number (percentage) unless otherwise indicated. DC = disease model.

3.2. Item frequency (adherence) ratings in TAU

Table 2 (MIA protocol) and Table 3 (MET protocol) present means and standard deviations for all TAU adherence and competence rating items, collated by type of intervention (addiction counseling interventions, basic and advanced MI skills), in descending order of their frequency of occurrence within sessions. For the one-session MIA protocol, the most frequently occurring interventions were assessing and monitoring the participant’s substance use (M = 5.8) and social functioning (M = 5.5) in different life areas (e.g., work, family, partner, social network, legal). These items were present in 98% of TAU sessions. The next most frequently occurring interventions (i.e., present in between 78% and 93% of sessions) included asking open-ended questions (M = 4.2), discussing the client’s medical problems or use of medications for the treatment of substance abuse or psychiatric problems (M = 3.8), reviewing symptoms related to psychopathology (M = 3.7), discussing problems for which the client entered treatment (M = 3.7), reflective listening (M = 3.5), and program orientation (M = 3.5). Interventions that occurred at a lower intensity (i.e., mean adherence less than 3), but which were present between 47% and 76% of sessions, included therapist self-disclosures and discussions unrelated to any problems for which the participant may have been seeking treatment (M = 2.8), giving unsolicited advice (M = 2.6), asserting therapeutic authority (M = 2.5), use of motivational interviewing style (M = 2.5), self-help group involvement (M = 2.4), case management (M = 2.2), and psychoeducation (M = 2.2). Interventions rated as occurring very infrequently (i.e., adherence ratings less than 2 and present to any extent in less than 20% of sessions and generally occurring very briefly if at all) included formal treatment planning (M = 2.0), addressing ambivalence (M = 1.4), any discussion of HIV/STD risk behavior or reduction (M = 1.3), confrontation of denial or defensiveness (M = 1.2), psychodynamic interventions (M = 1.1), discussion of powerlessness and loss of control (M = 1.1), reality therapy principles (M = 1.1), cognitions (M = 1.0), skills training (M = 1.0), and heightening discrepancies (M = 1.0).

Table 2.

Mean adherence and competence ratings and item frequencies in the TAU conditions in the MIA (one-session) protocol (n = 160)

	Adherence			Competence a
Item rated	% of sessions	M	SD	M	SD	Competence n
Addiction counseling interventions
1. Assessing/Monitoring substance use	0.98	5.8	1.3	4.6	0.9	157
2. Social functioning and factors	0.98	5.5	1.4	4.5	0.9	157
3. Assess medical issues and medication	0.93	3.8	1.6	4.2	0.8	149
4. Psychopathology	0.92	3.7	1.4	4.2	0.9	147
5. Program orientation	0.78	3.5	2.0	4.1	1.0	125
6. General discussion and self-disclosure	0.60	2.8	2.0	3.6	1.0	96
7. Give unsolicited advice and direction	0.58	2.6	1.9	4.1	0.9	93
8. Assert therapeutic authority	0.54	2.5	1.9	4.1	0.9	86
9. Self-help group involvement	0.76	2.4	1.1	4.0	0.8	122
10. Psychoeducation about substances	0.47	2.2	1.7	4.1	0.9	75
11. Case management	0.50	2.2	1.6	4.2	1.0	80
12. Formal treatment planning	0.38	2.0	1.5	4.1	0.9	61
13. Spirituality/Higher power	0.47	1.7	0.9	3.8	0.8	75
14. Emphasis on abstinence	0.25	1.4	0.8	3.8	0.8	40
15. Risk behavior reduction	0.19	1.3	0.9	3.6	0.9	30
16. Confront denial or defensiveness	0.13	1.2	0.8	4.3	0.7	20
17. Psychodynamic interventions	0.06	1.1	0.5	4.0	0.5	9
18. Powerlessness and loss of control	0.04	1.1	0.5	4.0	0.8	7
19. Reality therapy principles	0.06	1.1	0.4	3.7	0.8	10
20. Cognitions	0.03	1.0	0.2	3.8	0.5	5
21. Skills training	0.02	1.0	.01	4.0	0.0	3
Basic MI skills
22. Open-ended questions	0.88	4.2	1.8	4.1	0.9	141
23. Reflective statements	0.79	3.5	1.9	4.2	0.9	126
24. Motivational interviewing style	0.53	2.5	1.7	4.1	1.1	85
25. Fosteri collaborative atmosphere	0.46	2.0	1.3	4.1	0.9	74
26. Affirm strength and self-efficacy	0.43	1.8	1.3	4.3	0.8	69
Advanced MI skills
27. Problem discussion and feedback	0.89	3.7	1.6	4.2	0.9	142
28. Motivation to change	0.45	1.8	1.1	3.7	0.9	72
29. Change planning discussion	0.28	1.5	1.0	3.8	0.8	45
30. Pros, cons, and ambivalence	0.19	1.4	1.0	4.0	0.8	30
31. Heightening discrepancies	0.03	1.0	0.3	4.5	0.6	5

^aCompetence ratings are on a seven-point Likert scale: 1 = very poor, 2 = poor, 3 = acceptable, 4 = adequate, 5 = good, 6 = very good, 7 = excellent. Mean competence ratings are made only when an item occurs within a session. Thus, the sample sizes for the item competence ratings vary from n = 3 for skills training and n = 157 for assessing/monitoring substance use and social functioning and factors.

Table 3.

Mean adherence and competence ratings and item frequencies in the TAU conditions in the MET (three-session)^a protocol (presented by frequency, n = 219)

	Adherence			Competenceb
Item rated	% of sessions	M	SD	M	SD	Competence n
Addiction counseling interventions
1. Social functioning and factors	0.95	4.7	1.8	4.3	0.7	206
2. Give unsolicited advice and direction	0.64	3.1	2.0	4.1	0.9	141
3. Assessing/Monitoring substance use	0.67	2.8	1.7	4.2	0.7	145
4. Assert therapeutic authority	0.53	2.5	1.8	4.2	0.9	117
5. Self-help group involvement	0.55	2.4	1.7	4.2	0.7	119
6 General discussions and self-disclosure	0.48	2.3	1.8	3.8	1.0	105
7. Medical/Medication	0.28	1.5	1.1	4.0	1.1	63
8. Psychopathology	0.24	1.5	1.0	3.9	1.0	55
9. Program orientation	0.31	1.6	1.2	4.1	1.0	68
10. Psychoeducation about substances	0.25	1.6	1.2	4.3	1.0	55
11. Case management	0.23	1.4	0.9	4.1	0.8	52
12. Formal treatment planning	0.14	1.4	1.0	4.1	1.1	33
13. Spirituality/Higher power	0.17	1.4	1.0	4.2	1.1	38
14. Confront denial or defensiveness	0.18	1.4	1.1	4.2	1.2	41
15. Reality therapy principles	0.14	1.3	0.8	4.1	1.2	33
16. Emphasis on abstinence	0.12	1.2	0.6	3.9	1.0	29
17. Psychodynamic interventions	0.10	1.2	0.7	4.1	1.2	22
18. Powerlessness and loss of control	0.11	1.2	0.6	3.9	0.9	25
19. Cognitions	0.11	1.2	0.8	3.9	1.2	25
20. Risk behavior reduction	0.03	1.1	0.4	3.3	1.7	9
21. Skills training	0.04	1.1	0.3	3.6	1.5	10
Basic MI skills
22. Open-ended questions	0.91	4.4	1.8	4.4	0.8	202
23. Reflective statements	0.81	3.7	1.9	4.3	0.9	179
24. Motivational interviewing style	0.57	2.6	1.7	4.1	0.9	126
25. Fostering collaborative atmosphere	0.42	1.9	1.3	4.3	0.8	94
26. Affirm strength and self-efficacy	0.51	2.0	1.3	4.3	0.7	114
Advanced MI skills
27. Problem discussion and feedback	0.77	3.1	1.7	4.3	0.8	168
28. Motivation to change	0.28	1.5	0.9	4.3	0.7	63
29. Change planning discussion	0.19	1.4	1.1	4.4	1.0	42
30. Pros, cons, and ambivalence	0.08	1.2	0.6	3.9	1.2	21
31. Heightening discrepancies	0.12	1.2	0.6	4.3	1.2	27

^aTAU delivered across three sessions within a 28-day period.

^bCompetence ratings are on a seven-point Likert scale: 1 = very poor, 2 = poor, 3 = acceptable, 4 = adequate, 5 = good, 6 = very good, 7 = excellent. Mean competence ratings are made only when an item occurs within a session. Thus, the sample sizes for the item competence ratings vary from n = 9 for risk behavior reduction and n = 206 for social functioning and factors.

In the three-session MET protocol, the most frequently occurring interventions (i.e., present between 64% and 95% of sessions) were assessment of the participant’s social functioning (M = 4.7), asking of open-ended questions (M = 4.4), reflections (M = 3.7), direct inquiries about problems for which the client entered treatment (M = 3.1), and giving unsolicited advice (M = 3.1). Interventions that occurred to a lesser degree (i.e., adherence ratings less than 3 and present between 48% and 67% of sessions) included assessment and monitoring of substance use (M = 2.8), use of a motivational interviewing style (M = 2.6), self-help group involvement (M = 2.4), and therapist self-disclosures and discussions unrelated to any problems for which the participant may have been seeking treatment (M = 2.3). Interventions rated at a very low level (i.e., mean adherence ratings less than 2 and occurring in less than 20% of sessions) included change planning discussion (M = 1.4), confrontation of denial or defensiveness (M = 1.4), formal treatment planning (M = 1.4), discussion of the concept of spirituality/higher power (M = 1.4), psychodynamic reality therapy principles (M = 1.3), emphasis on abstinence (M = 1.2), powerlessness and loss of control (M = 1.2), cognitions (M = 1.2), heightening discrepancies (M = 1.2), HIV/STD risk behavior reduction (M = 1.1), skills training (M = 1.1), and discussion of pros, cons, and ambivalence (M = 1.1).

3.3. Competence ratings in TAU

Mean competence scores for each of the items are also presented in Table 2 and Table 3 (note the sample sizes vary for these ratings because the competence dimension can be rated only if the intervention occurred in the session rated). These ratings, with few exceptions, centered within an “average” range of competence at the rating scale’s midpoint. That is, the mean competence scores ranged from a low of 3.3 to a high of 4.3. Thus, although a number of interventions were used very infrequently, in those cases when they were implemented, the ratings indicated that the clinicians tended to do so at reasonable levels of competence in both protocols.

3.4. Change across the three MET sessions

Although we hypothesized that the MET protocol would include increasing frequency and a larger range of strategies across sessions, this did not occur. Although there was a statistically significant session effect for some interventions (assessing/monitoring substance use, F[2, 215] = 10.6, p = .00; assessment of social functioning and factors, F[2, 215] = 9.4, p = .00; reviewing symptoms related to psychopathology, F[2, 214] = 6.9, p = .00; program orientation, F[2, 215] = 5.5, p = .01; fostering a collaborative atmosphere, F[2, 216] = 3.8, p = .02; and affirming strength and self-efficacy, F[2, 215] =5.5, p = .00, with the exception of one counseling intervention [affirming strength and self-efficacy]), each of these techniques actually decreased in frequency across the three sessions. Most other strategies were used relatively consistently across the sessions. Moreover, other interventions that might be expected to increase in frequency as the clinician developed greater familiarity with the participant, such as formal treatment planning or case management, as well as other evidence-based interventions, such as skills training, assessment of cognitions, and risk behavior reduction, were largely absent in the later sessions.

3.5. Association between clinician characteristics and counseling strategies in TAU

Analysis of the relationship of clinicians’ licensure, recovery, and educational status to ratings of their adherence and competence revealed few significant relationships between these variables. In the MIA protocol, clinicians who were drug and alcohol certified or licensed were rated as significantly more likely to encourage their participants to get involved in 12-step groups (F[1, 11] = 16.1, p = .00), to assess medical and medications issues (F[1, 11] = 6.6, p = .03), and to assess and monitor client substance use (F[1, 11] = 4.6, p = .05) as compared with clinicians who were unlicensed or not certified. Clinicians who identified themselves as being in recovery in the MIA protocol were rated as more likely to provide psychoeducation about substances (F[1, 9] = 6.18, p = .04) as compared with clinicians who did not self-identify as being in recovery. No other significant differences in interventions were found for clinicians based on their characteristics.

In the MET protocol, clinicians describing themselves as oriented primarily to a 12-step approach were rated as less likely to use formal treatment planning in their counseling (F [1, 8] = 29.5, p = .00) relative to clinicians who did not identify themselves with the 12-step approach. There were no other significant differences in the extent of TAU counseling interventions and clinician orientation in the MIA and MET protocols.

4. Discussion

Several important findings emerged from this objective description of TAU as practiced in the early phase of individual substance abuse treatment in community programs affiliated with the CTN, which is, to our knowledge, the first study to use independent ratings of a large number of audiotaped standard treatment sessions. First, in the one-session MIA protocol, the ratings indicated that TAU was characterized by comparatively high levels of assessment of participant’s substance use and social functioning, as well as moderate levels of problem discussion, program orientation, case management, use of open-ended questions, assessment of participant’s medical and medication issues, psychiatric symptoms, and self-help group involvement. In the three-session MET protocol, TAU was predominantly characterized by assessment of social functioning, use of open-ended questions, reflections, problem discussion, and to a lesser degree, motivational interviewing style and self-help group involvement. Assessment of substance use was less frequent, and the provision of unsolicited advice was more frequent than in the one-session MIA protocol. Second, in both protocols, a number of interventions that the program directors and clinicians indicated they used frequently prior to the initiation of the protocol, including skills training, focus on cognitions, HIV risk behavior reduction, and emphasis on abstinence, were present in a very few of the sessions and very infrequently implemented. Third, contrary to our expectations, the frequency or range of different treatment strategies did not increase across sessions in the three-session MET protocol. Fourth, although some interventions were implemented much less frequently than we anticipated, the competence ratings indicated that clinicians did use them with a reasonable level of skill. Finally, there were very few significant relationships between clinicians’ stated theoretical orientations and their use of techniques associated with these orientations or between their professional (i.e., educational and licensure) status or history and use of specific techniques.

Our expectations regarding the “content” of TAU as practiced in the participating sites was only partially supported by the session tape ratings. As anticipated, these early sessions consisted largely of assessment of substance use and related problems and a variety of nonspecific counseling strategies (e.g., problem discussion, psychoeducation). However, the only interventions consistently present and that were associated with a specific EST were those associated with basic MI skills (asking open-ended questions, reflective listening). Most surprising was the relative absence in these tapes of other interventions the program directors and clinicians had indicated they used frequently, including those associated with CBT (skills training, cognitive restructuring) and traditional 12-step/disease models (e.g., emphasizing total abstinence as a treatment goal, discussing concepts such as spirituality, powerlessness, and loss of control). Moreover, recognizing the limited time for intervention in the one-session protocol, we anticipated that in the three-session protocol, there would be more opportunity for the clinicians to use a variety of interventions associated with ESTs, but the frequency of several rated interventions actually decreased rather than increased across sessions.

Thus, the interventions actually provided in these sessions were largely inconsistent with our expectations, the program directors’ descriptions of the nature of counseling conducted at their sites, and the clinicians’ descriptions of their therapeutic orientations prior to the initiation of the trials, who reported they used “moderate or high levels” of ESTs in their standard practice (Ball et al., 2002; McCarty et al., 2007). Comparatively, the overall levels of adherence were markedly lower than that found for similar or identical items assessing general, nonspecific drug counseling interventions and ESTs such as Twelve-Step Facilitation and CBT in the initial sessions of several previous efficacy trials that have used the YACS rating system (e.g., Carroll et al., 1998, 2004) when delivered by masters- or doctoral-level therapists trained to adhere to manual guidelines. Although not directly addressed by these protocols, these differences may suggest that training and supervision are likely to be important factors in increasing adherence for providing interventions associated with ESTs.

Why were interventions associated with ESTs and even some basic drug counseling strategies, such as formal treatment planning, addressing ambivalence, and emphasis on abstinence, relatively infrequent in these sessions? First, given that these data were drawn from early sessions where it was important to evaluate the participant, to conduct extensive assessments, and to complete agency paperwork, the opportunity to implement other interventions may have been limited. As noted earlier, however, data from the later sessions of the three-session protocol did not indicate that the level of general counseling or ESTs increased across sessions when clinicians may have had the opportunity to shift in focus from assessment to more direct intervention on the participant’s substance use and related problems. Second, clinicians assigned to the TAU condition in these protocols were instructed to deliver standard interventions according to usual practice at their agencies and generally did not receive formal clinical supervision focused on the specific technical delivery of their individual counseling. As such, although the clinicians were aware they were being audiotaped, there was little formal oversight of treatment delivery or criterion-based evaluation of evidence-based interventions (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005) and limited incentive to implement or adopt new strategies (McLellan, Carise, & Kleber, 2003; McLellan & Meyers, 2004). Third, in environments where supervision and checks on quality of treatment are unavailable, clinicians may be less likely to maintain or refine their skills over time unless they are highly motivated to seek training in ESTs on their own time and expense. Recent studies evaluating strategies for training and encouraging clinicians to use ESTs suggest that participation in intensive workshops coupled with individual supervision, including performance feedback and individualized coaching, results in the greatest increase in the ability of clinicians to learn and implement new practices (Miller, Yahne, Moyers, Martinez, & Pirritano, 2004; Sholomskas et al., 2005). However, this type of training and supervision is rarely available in most community programs (Fixsen et al., 2005).

Of note, one intervention was found to be present at higher rates than we expected and more frequently than a number of interventions associated with ESTs: clinician-initiated discourse and self-disclosures unrelated to problems for which the participant sought treatment. This item occurred at a higher mean level (one-session protocol = 2.8; three-session protocol = 2.3) and in a larger proportion of sessions (one-session = 60%; three-session = 48%) than several other ESTs that we expected would occur on a moderate level. Although this type of discussion may have been done in the spirit of relationship building in the early phases of treatment, its relative level of intensity compared with other interventions suggests there may have been some missed opportunities to implement more focused interventions early in treatment.

It is also not clear why, contrary to expectations, many interventions were more frequently observed in the one-session MIA protocol than in the three-session MET protocol. It is possible that because the length of TAU sessions in the MIA/single session protocol was twice the length of sessions in the MET protocol (i.e., 2 hours vs. 45–55 minutes), clinicians implemented a larger number of counseling interventions simply because they had more time during the session. Alternatively, because the MIA protocol involved the initial triage/assessment session and in many sites required the clinicians to complete a set of specific intake forms and evaluations (e.g., the Addiction Severity Index and agency-specific documentation), the clinicians may have had clearer guidelines and more structure that promoted more types of counseling interventions. Conversely, clinicians providing TAU in the three-session protocol did not have a structured agenda and were relatively free to conduct sessions as they saw fit.

There were also very few strong or consistent relationships between clinician characteristics (e.g., licensure, recovery, and educational status), clinician orientation (i.e., 12-step or motivational interviewing), and use of specific counseling strategies, suggesting that clinicians’ stated theoretical orientation or beliefs about what they do in therapy may have little relationship to the nature of the treatments they actually provide. The audiotapes also suggest that of the 31 interventions that were evaluated, those that were MI-consistent were present at comparatively higher levels relative to other interventions observed, including 12-step approaches (e.g., emphasis on abstinence, self-help group involvement, spirituality/higher power, confrontation of denial, powerlessness, and loss of control), CBT (risk behavior reduction, skills training, cognitions), or other basic drug counseling (formal treatment planning, case management). However, the MI interventions that occurred were primarily basic MI skills (e.g., open-ended questions and simple reflective statements) rather than advanced ones (e.g., pros, cons, and ambivalence, heightening discrepancies, motivation to change, change planning discussion) and were consistently implemented at lower levels compared with levels rated in the MIA/MET experimental conditions (Martino et al., in press). Thus, data from these TAU sessions suggest that clinicians in TAU may have “picked up” some MI interventions, but in the absence of structured supervision and feedback, they did not acquire the skill sets necessary to deliver MI with full proficiency (Miller et al., 2004).

This study has several limitations. First, community-based programs associated with the CTN are largely self-selected and are likely to differ from other community treatment programs in the United States. Hence, their practices may vary widely from other agencies in the United States and from those in other countries. On the other hand, the CTN sites’ willingness to open their agencies to a range of research trials, as well as their stated willingness to adopt ESTs via participating in protocols, suggests that levels of ESTs in other programs may be different, possibly even lower, than those observed here. Moreover, given that ratings were based on a large number of session audiotapes from nine widely different programs in a range of geographic areas and settings, the data presented here probably reflect a reasonable representation of TAU in the United States during the time the trials were conducted (2001–2004). Second, the observations presented here are based only on individual protocol sessions; participants in both conditions were involved in a range of other group sessions and activities associated with each program, and it is possible that they were exposed to other interventions in their group, family, or other types of sessions. Finally, the data presented here are based on the 31 types of counseling interventions assessed, and it is possible that some interventions were not captured by this rating system. However, many of the items were drawn from a fidelity rating system that had been used in multiple previous addiction studies and identified comparatively high levels of adherence to general drug counseling interventions as well as ESTs (Carroll et al., 1994, 2000) and were supplemented with other interventions described by program administrators and staff prior to the start of these protocols. Moreover, the ratings had high levels of interrater reliability for both the adherence and competence domains, suggesting that this very experienced group of raters were able to describe clinician behaviors accurately and very consistently.

The primary aim of this report was to provide an objective description of what constitutes TAU in the early phase of outpatient addiction treatment in a range of programs in the United States and hence serve as a benchmark for future effectiveness and dissemination studies. This study is, to our knowledge, the first to have objectively examined standard treatment based on audio-recorded sessions across a diverse sample of community addiction treatment programs. Through the process of objectively evaluating what actually occurs in standard treatment, researchers and clinicians will be better able to determine the degree to which standard practice is transformed over time by efforts to encourage broader use of techniques and practices that are empirically based. Although the outcomes for both trials suggest that outcomes were relatively good in the TAU conditions in most sites, the significantly better retention outcomes in the MI protocol and drug use outcomes in the MET protocol suggest that relatively minor changes to standard treatment can have a meaningful effect on outcome. Moreover, the low observed frequency of interventions associated with ESTs other than basic MI or those interventions one would reasonably expect to be present in the early phases of treatment (e.g., treatment orientation, emphasis on abstinence, case management, discussion of treatment goals) suggest there is ample opportunity for dissemination and training efforts to address how clinicians manage the early phase of substance abuse treatment.