The Role of Therapist Characteristics in Training Effects in Cognitive, Supportive-Expressive, and Drug Counseling Therapies for Cocaine Dependence

Abstract

The role of therapist characteristics in therapy training was examined for 62 therapists in a multisite psychotherapy outcome study that included cognitive therapy (CT), supportive-expressive (SE) psychodynamic therapy, and individual drug counseling (IDC) for cocaine-dependent patients. Demographic variables and experience and competence ratings prior to training were correlated with measures of change in competence during the training phase. Higher competence ratings before training were associated with greater change in competence for SE and higher average competence for IDC. More years of experience were associated with greater change in competence for CT therapists, but more hours of pre-training supervision in the CT treatment modality were associated with less change.

Training of therapists has become more important as the use of therapy manuals has increased in research, in training programs for therapists, and more recently in clinical practice.¹ There is limited evidence that the skills taught in training programs translate into performance in sessions.^2,3 Only a few studies of psychotherapy training have assessed the use and application of previously taught skills in actual therapy sessions. Furthermore, these studies have yielded mixed results,⁴ some showing that learning has occurred,^5,6 others showing no learning,⁷ and some suggesting possible negative effects of training.⁶

There are two likely explanations for these conflicting results. First, there could be variability in therapists' ability to learn to implement manualized treatments. Alternatively, training effects may differ across different therapies. Therapist characteristics, such as background and training experience, may help identify which therapists can benefit from training. Henry et al.⁸ found that neither therapist gender nor professional affiliation affected the outcome of training in brief psychodynamic therapy. However, amount of previous training received and competence as rated prior to beginning the study were correlated with changes in competence from pre- to post-training. Therapists with more previous hours of supervision (the effect being stronger than for years of experience) were less likely to show changes in adherence to the model post-training. However, existing competence was associated with improvement in interviewing style after training.⁸ Rounsaville et al.⁷ found that general therapist factors were strongly related to level of competence in interpersonal therapy, suggesting that much of the variance in therapist competence post-training could be related to a “good therapist” factor prior to training.

This report will address the role of therapist characteristics (e.g., demographics, level of experience) on training effects in the largest study of training of which we are aware, involving over 60 therapists and over 200 patients.⁴ In that study, Crits-Christoph et al.⁴ found that therapists in all three conditions—individual drug counseling, cognitive therapy, and supportive-expressive psychotherapy—showed increased competence across the sessions of a particular case. However, only cognitive therapists showed learning across training cases—that is, showed higher competence ratings in later training cases compared with earlier training cases, or began the next case at a higher level of competence than the previous case. Although other studies have found that pre-training competence predicted better response to training, the present study breaks down the pre-training competence variable into its different components (years of experience, supervision received, and others) to assess them individually. In addition, we can compare the role of therapist characteristics across three different treatments. Cognitive therapy and dynamic therapy are among the most commonly used approaches in psychotherapy, and drug counseling is one of the primary modalities in substance dependence treatment.

Given the literature, we hypothesized that pre-training ratings of competence and more years of general clinical experience would be associated with higher ratings of competence or change post-training in all treatments. In addition, we hypothesized that more experience with substance abuse patients would be associated with higher ratings of competence post-training because therapists/counselors comfortable with this population could focus on refining the specific techniques of the therapy model; no differences in this regard were predicted between treatment conditions.

METHODS

Patient Sample

The patient sample included 202 patients (out of the total 312 randomized patients) who had a minimum number of sessions rated for adherence/competence. Thirty-one percent of the patients were female; 57% were Caucasian, 40% African American, and 3% Hispanic or Native American. The majority of patients (73%) were crack cocaine smokers. Thirty-seven percent of the patients met DSM-III-R criteria for current alcohol dependence, 28% had at least one Axis I disorder other than substance dependence, and 44% qualified for a personality disorder diagnosis (see Crits-Christoph et al.⁴ for more detailed description of study design).

Therapists

The three individual treatments studied were supportive-expressive therapy,⁹ (SE) a psychodynamic treatment derived from Luborsky;¹⁰ cognitive therapy¹¹ (CT); and Individual Drug Counseling (IDC; D. Mercer, G. Woody, unpublished manuscript), based on the 12-step addiction model. Therapists were recruited primarily through newspaper advertising or from the staff of the study sites. Therapists were selected by the supervisors for each therapy modality on the basis of a combination of previous education and training as stated in the therapist's application; letters of reference; and two audiotaped samples of their therapy work. All therapists had previous experience in their treatment modality (but not in the specific techniques taught in this study) and had to identify themselves as primarily practicing within the specific modality.

Fifty-four SE applications were submitted, 27 (50%) were accepted, and 26 SE therapists started the training. Twenty-nine CT applications were submitted, 25 (86%) were accepted, and 21 CT therapists started the training. Thirty-nine IDC applications were submitted, 29 (74%) were accepted, and 24 IDC counselors started the training.

Training Process

Therapists selected to begin the training process were first given a copy of the manual and other training materials to review prior to treating cases. Therapists attended four training workshops over the course of the 1½-year training period that consisted of didactic presentations, role-plays, and discussion of case examples and difficulties encountered implementing the treatment. All therapists, supervisors, and training unit directors attended these two-day workshops.

The first workshop was held prior to the assignment of the first training case. It focused on the theoretical foundations of the treatment and review of the treatment manual outline and goals. The second workshop, and more especially the third and fourth workshops, focused on reviewing techniques and case formulations, using patients with cocaine dependence who had been or were currently in treatment with the training study therapists. Videotapes and audiotapes of problematic and successful cases were reviewed and discussed.

Each therapist was assigned 4 training cases over the course of the training phase. The treatment was 6 months long and included twice-weekly sessions for 3 months and then once-weekly sessions for 3 months. To be counted as a training case, the patient had to remain in therapy at least 1 month.

All therapy sessions were audiotaped and mailed to the supervisor within 3 days. The therapist's supervisor rated specific sessions, using the adherence/competence scales described below, after listening to the audiotape of the whole session. Supervisors were permitted to listen to and rate sessions out of the above sequence if there was a clinical crisis or a specific training need. Supervision, conducted by telephone, was focused on specific feedback from the ratings of adherence and competence for that particular session and over the course of training. The average patient in this study had 5.3 (SD=1.7) sessions rated by the supervisor. The total number of sessions rated per therapist ranged from 8 to 25 across the 4 training cases.

At the end of the training phase, therapists who achieved adequate levels of competence were invited to participate in a subsequent clinical trial evaluating the efficacy of three psychosocial treatments for cocaine dependence (see Crits-Christoph et al.¹²).

Raters and Clinical Supervisors

The heads of the training units chose the clinical supervisors for their units. Most supervisors were co-authors of the treatment manuals and had previous experience in the training of therapists within the specific modality, as well as extensive general supervision experience. The supervisors and heads of the training units coordinated and conducted the training workshops. Each supervisor rated only the therapy modality for which he or she provided clinical supervision. There were three SE supervisors, four CT supervisors, and four IDC supervisors.

Measures of Demographics and Pre-training Experience and Competence

A therapist application form was designed for this study to gather the following demographic information from therapists/counselors in all treatment conditions: age, race, gender, and professional degree (e.g., M.A., Ph.D.). Although some questions varied by treatment modality, the following information was also available for all three treatment conditions: years of experience (number of years since degree for CT and SE; number of years as an addictions counselor for IDC) and number of substance dependence cases treated in the modality. The number of supervision hours in the treatment modality (CT or SE) was requested only on the CT and SE applications and is not available for IDC. Table 1 presents demographic and descriptive information on the therapists included in each treatment modality.

TABLE 1. Descriptive characteristics of therapists and counselors.

In addition, pre-training level of competence was rated on the basis of an average of the adherence/competence ratings for the two application audiotapes. These application tapes were rated on the same adherence/competence measures described below that were used to measure competence during the training.

Adherence/Competence Ratings

The Penn Adherence and Competence Scale for Supportive-Expressive Therapy (PACSE) was modified for this cocaine study from a general adherence and competence scale designed for supportive-expressive therapy.¹³ Only the Expressive subscale was used for the current study, because therapists generally began training with adequate levels of use of supportive techniques, and training/supervision focused on the use of expressive techniques. In addition, the use of supportive techniques has not been found to be specific to SE therapy.^13,14

Frequency (adherence) and quality (competence) of interventions are rated separately for each item. The Expressive subscale includes items related to the focus on interpersonal relationships and interpretation of the core conflictual relationship themes. Recent research has shown a relationship between competence in the use of expressive techniques, but not supportive techniques, and the outcome of short-term SE therapy for depression.¹⁵ Item scores range from 1 (not present or poor quality) to 7 (high frequency or excellent quality). Based on preliminary reliability analyses of items, a final scale of 15 items for adherence and 6 items for competence was used in the SE condition for the current analysis.

The Cognitive Therapy Scale (CTS; J. Young, A.T. Beck, unpublished) employed in the Treatments for Depression Collaborative Research Project¹⁶ was used in rating competence in the CT condition. The CTS has been shown to adequately assess variations in session quality.¹⁷ This scale includes items specific to cognitive therapy (cognitive conceptualization, strategy, and technique) as well as general therapeutic skills. CTS scores range from 0 (poor) to 6 (excellent) for each item, for a possible total of 66. Preliminary reliability analyses with the judges and patients used for the present study indicated that 4 of the original 11 items could not be rated reliably; therefore only the remaining 7 items were retained for the analysis of competence in the training phase. In order to be consistent with the dynamic therapy, only items assessing change in specific CT skills were used in the analysis. Thus, a total of 6 CT-specific items were used in this analysis (items #1 and 7–11 on the original CTS scale). Scores were prorated on the original 0 to 66 scale to allow for comparison with previous studies.

The Adherence and Competence Scale for Addiction Counseling¹⁸ was developed for this project and was used to rate competence in the IDC condition. It includes 17 items categorized into six primary sections: monitoring drug use behaviors, encouraging abstinence, encouraging 12-step participation, preventing relapse, supportively confronting negative behaviors, and educating the client. Each item was rated for both adherence and competence on scores ranging from 1 (not present or poor quality) to 7 (highly frequent or excellent quality). Adherence and competence summary scores were calculated by averaging over the 17 items.

For more information on the internal consistency and the item and interjudge reliability of the adherence/competence scales for all three treatments, see Crits-Christoph et al.⁴

Data Analysis and Transformation of Variables

A total of 62 therapists were included in the analysis, with 19 therapists in CT, 23 in SE, and 20 in IDC. To be included, the therapist had to have at least 1 training case with 3 sessions where adherence/competence ratings were available. This meant the training case had to complete at least 8 sessions, because the following sessions were rated for competence: sessions 2, 4, 8, 12, 18, 24, 30, and the last session. A minimum of 3 sessions rated was set for the analysis of a training effect so that a slope (and associated standard error) could be estimated across sessions. Therapists were included irrespective of whether the therapist subsequently dropped out of the study or was not certified for continuing into the main clinical trial. Including all selected therapists allowed us to assess the impact of training and provided the greatest variability in therapist skill level.

Using hierarchical linear modeling,¹⁹ the slopes representing change in competence scores across sessions within a case (within-case competence) and across training cases (across-case competence) were calculated for each therapist. Each treatment had its own unique competence scale, so data could not be pooled across the three treatment conditions and instead were analyzed within each treatment type. We were interested in looking at within-case change to see if therapists could incorporate supervision from previous sessions and use it in later sessions with the same patient or training case. One would expect to see higher competence scores in later sessions compared with earlier sessions. Across-case change assesses whether learning from previous cases is applied to new cases by looking for increasing competence ratings from training case 1 to training case 3. Applying training within and across cases could be considered two different skills, with the latter more likely demonstrating a clear training effect or generalizability of training. Across-case change would suggest that therapists start at a higher level of competence with later cases compared with the first case or earlier cases. In addition, an average competence score was also calculated for each therapist by taking an average of all competence ratings for all patients treated.

Professional degree was recoded into two levels. For CT and SE, one level included master's-level training and the other level included Ph.D., Psy.D., or psychiatrist. For IDC, the two levels were bachelor's-level training compared with master's-level training. Race (Caucasian or African American) and gender were entered as dichotomous variables, and age in years as a continuous variable. Given problems in the normality of the distribution of some variables, transformations were made to certain variables as follows. A square-root transformation was performed on the number of years of experience variable for all conditions, on number of supervision hours in CT, and on the number of substance-dependent patients treated in SE and CT. The variable reflecting the number of substance-dependent patients treated by IDC counselors was modified by using a logarithmic transformation.

RESULTS

Each of the therapist characteristics was examined to ascertain whether there was sufficient variability to test its effect on training. Almost all of the therapists in CT and SE were Caucasian, despite attempts to recruit a racially diverse group. Only IDC had a significant number of African American therapists. Therefore, we could test the role of race only in IDC. No significant relationship of race of therapist and competence ratings was found in IDC. Correlations had to be calculated for each modality separately, given the different adherence/competence scales used. A preliminary test was run to ascertain whether there were differences in competence across the other demographic descriptors. None of the therapist demographic variables (race, degree, age, and gender) were significantly correlated with the three ratings of a training effect.

Pearson correlations of a training effect (average competence, within-case change, and across-case change) with measures of therapist experience (years experience, number of substance-dependent patients treated, supervision hours) and pre-training competence (competence ratings of application tapes) can be found in Table 2. Pre-training ratings of specific competence in the treatment modality tended to be associated with higher competence ratings during training.

TABLE 2. Pearson correlations of training effects and therapist characteristics.

In SE, therapists with higher competence ratings on their application audiotapes showed more change in competence ratings across sessions of the same training case.

In IDC, there was a trend (P=0.06) for counselors with higher application competence ratings to have higher average competence ratings on their training cases.

CT therapists with more years of experience showed greater change in competence within a specific case. In contrast, CT therapists with more hours of supervision in CT showed less within-case change.

DISCUSSION

None of the therapist demographic variables (race, degree, age, or gender) were significantly correlated with training competence, a finding consistent with most other studies.²⁰ However, experience as a therapist in general and experience in treating patients in the particular treatment modality had an impact on change in therapeutic competence over training. CT therapists with more overall experience as therapists may be better able to use the feedback from supervision in their work with a particular patient because many of the same CT skills are used with different patient populations. However, CT therapists with more hours of supervision in CT showed less change with training. Perhaps therapists with more experience treating the same type of patients in “their own style” of CT have a harder time adjusting to a new model. It is important to note that years of experience and number of supervision hours were negatively correlated in the present study. Examination of the data suggested that more experienced therapists (10–15 years of experience) reported fewer hours of supervision than therapists newer to the field, who reported more, and likely recent, supervision hours.

Our results for cognitive therapy are very similar, however, to those in the study by Henry et al.⁸ of psychodynamic therapists treating general outpatients. Henry et al. found that more years of general therapy experience predicted more change in training, but that more supervision in a particular modality had a negative effect on training. The findings from the present study, as well as reports from Henry et al.⁸ and Rounsaville et al.,⁷ all show that competence prior to beginning training predicts competence in the training phase. This relationship suggests that investigators may gain the most information about how therapists will perform in future trials from audiotaped samples of the therapist's work that can be rated for adherence to the model being delivered. For clinical trials, researchers may want to choose therapists who are already proficient at what they do, and whose own style of therapy is consistent with the model being tested. Because we were unable to assess general therapeutic skill separately, we cannot be certain that those therapists who had higher competence ratings prior to training had good general therapeutic skills that could be improved or sharpened for the particular modality. There is some literature from graduate training²¹ that those therapists who have good therapeutic skills are the most likely to benefit from additional training, and this is likely true for professional psychologists as well.

However, the finding that more supervision in a particular modality predicts less change with training suggests that in future studies investigators may need to find a way to assess personality variables such as therapist flexibility and willingness to use a new, and likely different, model from their usual work. One would expect that the therapists who volunteered for the present study were open to training, since they agreed to participate in a project with such intensive monitoring and evaluation of their work. Given the current focus on whether one can train therapists to deliver empirically validated treatments,¹ understanding more about how to train and supervise therapists and predicting who can master these skills is especially important for the field.

Despite being one of the largest training studies to date, our examination of therapist characteristics was still limited by having only about 20 therapists in each modality group. The data could not be pooled across the three treatment conditions because each treatment had its own unique competence scale, leading to reduced statistical power and an increased number of significance tests being performed. Thus, replication of the findings will be needed. In addition, the effects of training were seen primarily in cognitive therapy, so it may be difficult to identify predictors of training in SE and IDC.⁴ Finally, a number of therapists who wanted to participate in the training were rejected at the application phase, perhaps truncating the range of therapist skill or ability to benefit from training. These results would therefore have to be replicated in other studies before definitive statements about who can be trained could be made.

Acknowledgments

The NIDA Collaborative Cocaine Treatment Study is a National Institute on Drug Abuse (NIDA)–funded cooperative agreement involving four clinical sites, a coordinating center, and NIDA staff. The coordinating center at the University of Pennsylvania includes Paul Crits-Christoph, Ph.D., Principal Investigator (PI); Lynne Siqueland, Ph.D., Project Coordinator; Karla Moras, Ph.D., Assessment Unit Director; Jesse Chittams, M.A., Director of Data Management/Analysis; and Larry Muenz, Ph.D., Statistician. The collaborating scientists at the Treatment Research Branch, Division of Clinical and Research Services at NIDA, include Jack Blaine, M.D., and Lisa Simon Onken, Ph.D. The participating clinical sites are: 1) University of Pennsylvania (U. of P.): Lester Luborsky, Ph.D., PI; Jacques P. Barber, Ph.D., Co-PI; Delinda Mercer, Ph.D., Project Director (PD); 2) Brookside Hospital/Harvard Medical School: Arlene Frank, Ph.D., PI; Stephen F. Butler, Ph.D., Co-PI/Innovative Training Systems; Sarah Bishop, M.A., PD; 3) McLean/Massachusetts General Hospital, Harvard Medical School: Roger D. Weiss, M.D., PI; David R. Gastfriend, M.D., Co-PI; Lisa M. Najavits, Ph.D., and Margaret L. Griffin, Ph.D., PDs; 4) U. of Pittsburgh/Western Psychiatric Institute and Clinic: Michael E. Thase, M.D., PI; Dennis Daley, M.S.W., Co-PI; Ishan M. Salloum, M.D., Co-PI; and Judy Lis, M.S.N., PD. The Training Unit includes Heads of Cognitive Therapy Training Unit: Aaron T. Beck, M.D. (U. of P.), and Bruce Liese, Ph.D. (U. of Kansas Medical Center); Heads of Supportive-Expressive Therapy Training Unit: Lester Luborsky, Ph.D., and David Mark, Ph.D. (U. of P.); Heads of Individual Drug Counseling: George Woody, M.D. (Veterans Administration/U. of P. Medical School) and Group Drug Counseling Unit: Delinda Mercer (Head), Dennis Daley (Assistant Head; U. of Pittsburgh/Western Psychiatric Institute and Clinic), and Gloria Carpenter, M.Ed. (Assistant Head; Treatment Research Unit, U. of P.). The Monitoring Board includes Larry Beutler, Ph.D., Jim Klett, Ph.D., Bruce Rounsaville, M.D., and Tracie Shea, Ph.D. The contributions of John Boren, Ph.D., and Deborah Grossman, M.A., NIDA, the project officer for this cooperative agreement, are also gratefully acknowledged.

Preparation of this manuscript was funded in part by Grants NIDA U18-DA07090, NIMH Clinical Research Center P50-MH-45178, and NIMH Career Development Award K02-MH00756, NIMH Career Development Award K05-DA00168, NIDA Career Development Award DA 00326, NIDA U18-DA07663, NIDA U18-DA07673, NIDA U18-DA07693, NIDA U18-DA07085. Findings were previously presented to the Society for Psychotherapy Research, Geilo, Norway, June 27, 1997.²²