Theoretical and Practical Barriers to Practitioners’ Willingness to Seek Training in Empirically Supported Treatments

Abstract

To identify barriers to the dissemination of empirically supported treatments (ESTs) a random sample of psychologists in independent practice (N = 1291) was asked to complete measures of attitudes towards ESTs and willingness to attend a 3-hour, 1-day, or 3-day workshop in an EST of their choice. The strongest predictor of unwillingness to obtain EST training was the amount of time and cost required for the workshop, followed by objections to the need for EST training. Psychodynamic (compared to cognitive-behavioral) and more experienced practitioners agreed more strongly with the objections to ESTs overall, as did those whose graduate schools had not emphasized psychotherapy research. Results suggest that both practical and theoretical barriers are significant obstacles to EST dissemination.

Evidence Based Practice (EBP) principles encourage clinicians to integrate the best available research evidence regarding possible treatment for a patient with their clinical expertise and consideration of the patient’s characteristics and values (Sackett, Straus, Richardson, Rosenberg, & Haynes, 2000). At the core of most work on EBP in clinical psychology is the focus on empirically supported treatments (ESTs). The EST effort represents one common approach to defining the research evidence arm of EBP¹ and is organized according to particular psychotherapy procedures (e.g., interpersonal therapy, cognitive-behavioral therapy) for specific psychological problems and disorders (e.g., major depression and panic disorder (Task Force, 1995). In light of recent evidence indicating that efforts to promote ESTs have had minimal impact on the practice of front-line practitioners (Arnow, 1999; Becker, Zayfert, & Anderson, 2004; Crowe, Mussel, Peterson, Knopke, & Mitchell, 2000; Goisman, Warshaw, & Keller, 1999; Haas & Clopton, 2003; Mussell et al., 2000; von Ranson & Robinson, 2006; Simmons, Milnes, & Anderson, 2008; Stewart & Chambless, 2007) dissemination of ESTs has emerged as an important area for study.

Much has been written about objections that practitioners may hold regarding ESTs, but there is little empirical research on what specifically clinicians² may find objectionable about ESTs, or on what factors might influence practitioners’ willingness to gain training in ESTs, which is the focus of the present paper. Given that the list of ESTs developed as a result of the EBP movement, and the question of what qualifies as evidence appears to be at the heart of both controversies (Norcross, Beutler, & Levant, 2006; Sackett & Wennberg, 1997), the available research on practitioners’ attitudes towards EBP is likely to shed light on dissemination and training issues for ESTs. Moreover, it has been demonstrated that practitioners do not necessarily differentiate between EBP and ESTs (Aarons, 2004; Hays et al., 2002; Pagoto et al., 2007). Accordingly, we surveyed the literature including research and opinion pieces on both EBP and ESTs, and have identified two broad challenges to EBP and EST dissemination: theoretical obstacles and practical obstacles. ³

Theoretical Obstacles

There have been many thorough and pointed discussions about ESTs in the literature. These discussions primarily focus on the rejection of ESTs on ideological grounds, postulating concerns raised on behalf of practitioners. These will be reviewed next.

EST efficacy does not generalize to real-world clinical practice.

One objection that repeatedly emerges is the generalizability concern, or the belief that efficacy outcome data (used to identify ESTs) do not generalize to clinical practice, and that patients in practice are more troubled, complex, and difficult to treat than patients treated in psychotherapy outcome trials (Westen & Morrison, 2001). Using focus groups of 19 child and adolescent clinicians to examine attitudes towards EBP, Nelson, Steele, & Mize (2006) found that clinicians reported concerns regarding the applicability of controlled research treatments to their work in community health centers, and were unconvinced EBPs are appropriate or flexible enough for their complex and comorbidly diagnosed patients. Similarly, in a survey of 239 practicing psychologists randomly selected from a nationwide mailing list (http://www.psychlist.com) about their attitudes towards a newly developed website to promote ESTs (therapyadvisor.com), 16% of clinicians reported being deterred from the adoption of ESTs because of concerns about generalization from patients in research settings to clinical practice (Riley et al., 2007). How widely results from these small samples can be generalized to the greater population of practicing psychologists is unclear.

Clinical experience and manuals.

Another potential charge is that dissemination of psychotherapy research findings and identification of ESTs are unnecessary because practicing clinicians know best how to treat their patients based on their clinical training and expertise. In a survey of 591 psychologists in independent practice, clinicians described their clinical experiences as having the greatest impact on their treatment decisions, an impact significantly greater than that of research evidence (Stewart & Chambless, 2007). In a telephone survey of 52 clinicians who treat eating disorders, von Ranson and Robinson (2006) found that clinicians’ most common reason for selecting a particular therapeutic approach was that their clinical experience indicated the treatment was effective. Similarly, Raine et al. (2004) found that clinicians were open to and accepting of research evidence only if it was supported by their experience in clinical practice. These studies may suggest that clinician reject ESTs in lieu of guidance by previous clinical experiences.

There also appear to be concerns regarding manual-based treatments (on which ESTs are based), and the belief that psychotherapists who use ESTs must follow treatment manuals in a robotic fashion, robbing them of creativity and other therapeutic skills (Addis, Wade, & Hatgis, 1999; Silverman, 1996). Pagoto et al. (2007) asked behavioral professionals (N = 37) through professional listservs to nominate barriers to the adoption of EBP. Of the 84 barrier nominations, 32% were related to negative attitudes about EBP, specifically the belief that EBP forces clinicians to make decisions based solely on irrelevant research evidence, with little regard for clinical experience and the human side of therapy such as empathy and creativity. Moreover, clinicians may hold negative attitudes regarding the idea of a “manual” informing treatment. Borntrager, Chorpita, Higa-McMillan, and Weisz (2009) found that therapists’ attitudes towards EBP were significantly more positive when the use and mention of manuals was de-emphasized.

All treatments are equivalent and the relationship is what matters.

Another potential objection held by clinicians is that all psychotherapies are equally effective. Under these circumstances there is clearly no need to focus on or utilize particular treatments for particular disorders (Ahn & Wampold, 2001; Luborsky et al., 2002; Wampold et al., 1997). In the previously mentioned survey, Riley et al. (2007) found that 23% of clinicians indicated that their belief in treatment equivalence deterred them from the adoption of ESTs. Similarly, clinicians may subscribe to the belief that non-specific factors (such as the relationship with the therapist, hope, expectation of change) are all that is necessary for effective treatment.

EST lists are unfair and overly focused on diagnosis.

Another objection to EST research offered in the opinion literature is that EST lists are unfair, disproportionately favoring CBT and discriminating against other treatments that may not be as amenable to controlled research (Stiles et al., 2006). A related criticism concerns the common linkage of EST research and Diagnostic and Statistical Manual of Mental Disorders (American Psychological Association, 1994) diagnoses. Some argue that the focus on DSM diagnostic categories constrains how we think about, question, and test clinical problems (Goldfried & Wolfe, 1998; Silberschatz in Persons & Silberschatz, 1998; Silverman, 1996). To our knowledge there are no empirical data on the prevalence of this notion. However, it is possible that such beliefs about ESTs may lessen clinicians’ interest in learning or utilizing these treatments.

Practical Obstacles

Some have suggested that logistical and systemic obstacles to EST training pose a formidable challenge (e.g., Addis et al., 1999; Barlow, Levitt, & Bufka, 1999), and the scant research on this topic supports this argument. Although the 239 practicing psychologists in the Riley et al. (2007) study rated the website promoting ESTs positively, they strongly endorsed the concept that limited time and resources were barriers to greater evidence-based practice generally (52% rating 7-10 agreement on a 0-10 scale). Moreover, in their qualitative study of EBP, Nelson et al. (2006) reported that practitioners often shared practical concerns about EBPs, noting repeatedly that they did not have the time necessary to learn such new approaches. In the Pagoto et al. (2007) study, when behavioral professionals were asked to nominate top barriers to EBP, concerns about lack of training constituted 23% of the barrier nominations, and 19% of the barrier nominations concerned logistical barriers required for the implementation of EBP such as time, cost and access to manuals.

Clinicians have a fair point. Standard training workshops in ESTs are time-consuming (Craske & Barlow, 2006), and it may be difficult for practitioners to reschedule (or cancel) patient appointments or otherwise find the time to attend training in a new psychotherapy. Moreover, workshops can be expensive, especially if clinicians must forgo income by attending workshops in lieu of seeing patients. Such continued training is rarely compensated for clinicians in private practice. More research is needed to assess the impact of practical barriers on EST implementation.

Goals of Study

The overall purpose of the present study was to glean information on both theoretical and practical barriers to dissemination of EST research. The first goal of the study was to develop a measure of theoretical objections to EST research based on our review of the literature and to provide descriptive data on how much practitioners subscribe to various objections. Although many theoretical objections have been voiced in the opinion papers, and some have been confirmed in the few empirical studies, we still do not know how broadly practitioners subscribe to any or all of these beliefs. Moreover, the inferences we can draw from the studies above are limited due to non-representative sampling, small sample sizes, and for two of studies, the limited sample of practitioners (in our experience) who are reached via Web-based research. Conversely, we were also interested in examining what clinicians might find to be advantageous aspects of EST adoption, a topic entirely ignored in the research heretofore.

It is plausible that practical barriers such as time and resources are a large impediment to the dissemination of ESTs and if so, this variable has been largely ignored in dissemination and research efforts. The second goal of the present study was therefore to examine factors that influence practitioners’ willingness to pursue training in an EST of their choice, focusing not only on theoretical but also on practical barriers, as well as important biographical variables such as therapeutic orientation and emphasis on psychotherapy research in graduate school.

The third goal of this study was to examine whether attitudes towards the theoretical obstacles and advantageous aspects of EST adoption vary in relation to theoretical orientation, years of experience, graduate school emphasis on research, and other demographic and descriptive variables. Some evidence indicates that clinicians with less clinical experience, more emphasis or training in ESTs in graduate school, and a non-psychodynamic theoretical orientation (Aarons, 2004; Nelson & Steele, 2007; Stewart & Chambless, 2007) hold more pro-research attitudes and therefore may subscribe less to the theoretical objections.

Lastly, we were interested in examining whether attitudes towards the theoretical obstacles might vary with the logistical demands of the workshop. In our view, psychologists in practice are caught in a difficult dilemma. On one hand, the American Psychological Association Ethics code (APA, 2002) dictates that psychologists undertake efforts to maintain and develop competence, and evidence-based practice is very much in the air (e.g., the American Psychological Association’s endorsement of a report advocating evidence-based practice, Levant, 2005). On the other hand, in a time of falling incomes for practitioners, allotting time and money for training may prove difficult. These seem like ideal circumstances for the creation of cognitive dissonance, which might understandably be resolved by rejecting the notion of ESTs on theoretical grounds. Thus, a fourth and final goal was to investigate the hypothesis that, as demands for time and resources to obtain training are increased, practitioners will be more likely to subscribe to ideological objections to ESTs.

Method

Participants

Using mailing labels purchased from APA, we sent a cover letter and survey to a randomly selected sample of 4,000 members of the American Psychological Association (APA) who specified themselves as practitioners in private practice. Front-line clinicians practice in a number of settings, including, for example, community agencies and hospitals as well as private practice. However, private practitioners are those who are least likely to receive employer-provided reimbursement for EST training. To achieve greater sample homogeneity in light of limited resources, this study was limited to private practitioners. Eight envelopes were returned to the sender due to faulty mailing addresses. Of the 1,289 respondents, 28 were unusable (i.e., participants returned blank surveys indicating they did not have time, were not currently in practice, or had retired). We obtained a total of 1,261 useable responses to the mailing, for an effective response rate of 32%.

Measures

The survey was divided into five sections. Section 1 assessed practitioners’ demographic information, theoretical orientation, clinical experience, emphasis on empirical research on psychotherapy outcomes in graduate school, primary employment setting, current average number of psychotherapy hours, and number of psychotherapy hours desired. In Section 2, participants read a definition of ESTs and some examples of ESTs representing a variety of theoretical orientations for depression, panic disorder, borderline personality disorder, and bulimia. So that clinicians would have a workshop in mind when answering subsequent questions, they were asked to consider a disorder for which they would like to pursue EST training (of any theoretical orientation available) in a workshop and indicate which disorder they had chosen. In Section 3, according to random assignment, subjects received one of three possible workshop scenarios: (a) low demand for time and resources (3-hour workshop during a time when patients are not scheduled), (b) medium demand for time and resources (1-day workshop during a day when patients are scheduled), and (c) high demand for time and resources (3-day workshop during days when patients are scheduled). These scenarios were developed to be ecologically valid for practitioners seeking continuing education.

In Section 4, participants were then asked to assess the probability they would attend their selected workshop (1-100%). Section 5 was comprised of 12 questions assessing practitioners’ agreement on a 1-7 Likert-type scale regarding objections to and positive attributes of the EST workshop they had envisioned. The eight negative questions include the objections described previously (e.g., I am not interested in learning an EST because most therapies have equivalent efficacy, and so it is not worthwhile to obtain training in one particular treatment). The four positive questions reflect advantageous aspects of attending a workshop (e.g., I am interested in learning an EST because this may make it easier for me to get managed care to pay for my client’s treatment). Items are listed in Table 2.

Table 2.

Factor Loadings of Perceived Barriers and Advantages of EST Workshops

	Factor
Barriers	1	2	3
I am not interested in learning an EST because most therapies have equivalent efficacy, and it is therefore not worthwhile to obtain training in one particular treatment	.52	-.20	-.06
I am not interested in leaning an EST because a good working relationship with my client is more important than learning how to do a specific treatment	.66	-.18	-.12
I am not interested in learning an EST because therapy is an art form that can never be manualized as it is in EST research.	.73	-.15	-.18
I am not interested in learning an EST because my patients are different in important ways (e.g., more troubled, complex, and difficult to treat) from patients treated in psychotherapy outcome studies.	.66	-.09	-.06
I am not interested in learning an EST because I know better than academic researchers how to care for my clients.	.67	-.22	-.09
I am not interested in learning an EST because the treatment I favor has not been tested in controlled research.	.54	-.27	-.03
I am not interested in learning an EST because clinical experience is more important as a guide to treatment than research evidence.	.66	-.20	-.14
I am not interested in learning an EST because I use individual case formulations to determine my treatment approach, not diagnoses, and therefore a workshop such as described would not be worthwhile	.69	-.22	-.20
Advantages
I am interested in learning an EST because I believe it is my professional duty to keep up with new developments in treatment.	-.29	.63	.16
I am interested in learning an EST because I believe it is important to incorporate scientific findings into my everyday practice.	-.23	.70	.13
I am interested in learning an EST because this may make it easier for me to get managed care to pay for my clients’ treatment.	-.04	-.20	.55
I am interested in learning an EST because I may attract new patients who are seeking these treatments.	-.19	-.22	.60

Note. Highest factor loadings are indicated with boldface type.

Results

Sample Characteristics

Of the practitioners, 60% were female. The mean age of the sample was 55.28 (SD = 8.44, range = 29-80). In terms of highest professional degree, 83% of the practitioners had earned a Ph.D., 14% a Psy.D., and 3% an Ed.D. Practitioners had been in practice an average of 21.6 (SD = 8.36) years. They saw patients an average of 24.65 (SD = 11.13) hours a week. The large majority (95%) of the practitioners worked in private practice. The two most common self-described primary theoretical orientations were cognitive-behavioral (46%) and psychodynamic (23%). An additional 19% reported themselves as eclectic, 5% described themselves as humanistic/experiential, 3% subscribed to family systems, and 4% chose the category other as their primary theoretical orientation. To evaluate the representativeness of the sample, the above characteristics were compared to data on the 19,660 practitioners in APA (American Psychological Association, 2009) who report private practice as their primary employment setting. The sample collected was virtually identical to overall sample in terms of age, gender, professional degree, and number of years in practice.⁴ Information on theoretical orientation and place of employment was not available from the APA.

Power analyses were conducted with the G*Power program (Faul, Erdfelder, Buchner, & Lang, 2009) for the main regression analyses: variables affecting EST objections and advantageous qualities. With an expected small effect size of f² = .02, alpha set at .05, and 10 predictors, a sample size of 921 provides greater than 99% power to detect a significant effect for a single predictor given the 9 other predictors.

Perceived Barriers and Advantageous Qualities of EST Workshops

A primary focus of this study was gleaning information from clinicians about their agreement with theoretical objections to the concept of ESTs. For all of the barriers, on average practitioners held no opinion or mildly to moderately disagreed that the proposed objection lessened their interest in learning an EST. A repeated measures ANOVA was conducted to test whether there were significant differences in the importance practitioners assigned to the eight barriers, F(7, 1,224) = 97.12, p < .0001, η² = .35. Post hoc tests were conducted with repeated measures t-tests with a Bonferroni corrected p-value of .0018 (.05/28 tests).

The results are reported in Table 1. Items are listed according to rank order agreement (lower scores indicate higher agreement). Included for each item is a list of those items from which this item is significantly different and a range of effect sizes for those differences. For example, practitioners agreed most strongly with the statement “A good working relationship with my client is more important than learning how to do a specific treatment.” Differences between the endorsement of this item and other items were significant for five of seven comparisons (those items ranked 4-8 in Table 1). They agreed least with the statement “Most therapies have equivalent efficacy, and it is therefore not worthwhile to obtain training in one particular treatment.” This item received significantly lower endorsement than all other items (those ranked 1-7 in Table 1). All effect sizes were small to medium.

Table 1.

Agreement with Theoretical Obstacles with Lower Ranks Indicating Higher Agreement with the Barrier Listed (N = 1249)

Rank	Barriers: “I am not interested in learning an EST because…”	M	SD	Differs	d range
1	A good working relationship with my client is more important than learning how to do a specific treatment	4.81	1.81	4-8	0.13-0.58
2	Clinical experience is more important as a guide to treatment than research evidence.	4.87	1.62	4-8	0.11-0.58
3	My patients are different in important ways (e.g., more troubled, complex, and difficult to treat) from patients treated in psychotherapy outcome studies.	4.94	1.74	5-8	0.10-0.51
4	Therapy is an art form that can never be manualized as it is in EST research.	5.05	1.76	1,2, 6-8	0.11-0.44
5	I use individual case formulations to determine my treatment approach, not diagnoses, and therefore a workshop such as described would not be worthwhile	5.11	1.65	1-3, 6-8	0.10-0.42
6	I know better than academic researchers how to care for my clients.	5.37	1.60	All	0.11-0.33
7	The treatment I favor has not been tested in controlled research.	5.54	1.54	All	0.11-0.43
8	Most therapies have equivalent efficacy, and it is therefore not worthwhile to obtain training in one particular treatment	5.74	1.37	All	0.14-0.58

Note. Lower numbers denote greater agreement. (1 = Strongly Agree, 2 = Moderately Agree, 3 = Mildly Agree, 4 = No Opinion, 5 = Mildly Disagree, 6 = Moderately Disagree, 7 = Strongly Disagree).

Difference numbers signify from which items the primary item differs and the range of effect sizes.

We calculated the average objection score for each practitioner across the barriers. We found that 81.7% of practitioners mildly to strongly disagreed that the barriers deterred their interest in EST training, whereas only 6.4% of practitioners strongly to mildly agreed on average with the barriers. However, it is possible the average score on the theoretical barriers factor is an insensitive indicator. It may be sufficient that one endorses any one of the theoretical barriers strongly to lead to rejection of EST training. Indeed, the barrier with the strongest endorsement tended to have an especially extreme rating. When we plotted participants’ mean standardized item score against the within-subject item rank for the eight objections, we observed a clear linear relationship across subjects for seven of the eight objections. The eighth objection (the one subjects agreed with the most) was an obvious outlier from this linear trend. This suggests that subjects provided extreme responses to one objection, although the specific strongest objection selected differed across participant. Nonetheless, overall participants held no opinion to mild agreement (M = 3.54, SD = 1.72) that their most EST-rejecting item deterred their interest in learning an EST.

Regarding the positive items, on average practitioners moderately agreed that it is their professional duty to keep up with new developments in treatment (1-7 scale, lower numbers denote greater agreement, M = 2.03, SD = 1.39) and that it is important to incorporate scientific findings into everyday practice (M = 2.13, SD = 1.44). Practitioners mildly disagreed that they would be interested in learning an EST to get managed care payments (M = 5.15, SD = 1.78), and they held no opinion as to whether learning an EST might help attract new patients (M = 4.08, SD = 1.77).

Exploratory factor analysis.

The correlation matrix for the 12 items was assessed using Bartlett’s chi-square criteria (Geweke & Singleton, 1980), which suggested the presence of at least one common factor (p < .0001). Parallel analysis (Hayton, Allen, & Scarpello, 2004) suggested as many as 5 dimensions might be extracted, but both the Kaiser criterion and an examination of the scree plot suggested a three-factor solution. We applied common factor analysis rotating according to promax criteria. The 3-factor solution accounted for 46% of the variance (presented in Table 2). Factor 1 (Theoretical Barriers) accounted for 29% of the variance and consisted of 8 items (α = .87). Item themes reflect arguments posited against the concept of ESTs. The second factor (2 items, r = .54) accounted for 10% of the variance and demonstrated openness to incorporating scientific findings and believing it is a professional duty to keep up with treatment developments (Empirically Guided Professional Development). The third factor (2 items, r = .35) accounted for 7% of the variance and appeared to reflect the business aspect of a private practice such as attracting and retaining patients (Business Development and Patient Retention). Due to the focus of the paper on barriers to EST adoption, only the first factor, Theoretical Barriers, will be used for further analyses.

Factors Affecting Likelihood of Obtaining Training

Consistent with the second goal of our study, we aimed to examine the influence of theoretical barriers and the practical barriers of time and resources (workshop assignment) on willingness to obtain training. We initially tested what other variables might be related to our dependent variable of likelihood of training: These were gender (women were more willing to obtain training, r_pb = -.06, p < .05), years in clinical practice (r = -.12, p < .01), and graduate training emphasis on research (r = .15, p < .01).

We conducted multiple regression analysis with likelihood of workshop attendance as the criterion. The predictors were: workshop assignment, the theoretical barriers score, the significant biographical variables identified above, and theoretical orientation. Workshop demand comprised three ordered levels (see Methods) but in accordance with Dawes & Corrigan (1974) was treated as continuous to conserve power. To represent the categorical variable of theoretical orientation, we assigned a series of dichotomous indicator variables (dummy codes) to indicate group membership with CBT as the reference group. Residual and influence diagnostics yielded no evidence of any problems, indicating these data were suitable for regression. The overall model was significant, F(10, 1,204) = 51.08, p < .0001. The semi-partial correlations for the predictors are reported in Table 3. These effect sizes are considered small at .01, medium at .06, and large at .13 (Harlow, 2005).

Table 3.

Semipartial Correlations of Predictors with Likelihood of Attending an EST Workshop (N = 1249)

Predictor	Likelihood
Gender	-.04
Graduate training in research	.05*
Years in clinical practice	-.06*
Workshop demand	-.40***
Disagreement with theoretical barriers	.28***
Theoretical Orientationa
Psychodynamic	-.10**
Eclectic	.05*
Humanistic	.02
Family Systems	.02
Other	-.05

Note. Semipartial rs are derived from simultaneous multiple regression of likelihood of attending on the predictors. When correlations carry a negative sign, higher scores on the predictor are associated with lower likelihood of attending, where

^*p < .05

^**p < .01

^***p < .001.

^aTheoretical orientation was dummy coded with cognitive-behavior orientation as the reference category. A negative correlation indicates that a particular orientation group was less likely than the CBT group to report willingness to attend.

Workshop assignment was a significant predictor of likelihood of attendance with a large effect size: When the time and resource demands were higher for a workshop, clinicians rated themselves less likely to attend. In addition, clinicians who subscribed more to the theoretical barriers rated themselves as less likely to attend, with a large effect size. Clinicians with more graduate school emphasis on psychotherapy research were more likely to report intentions to attend, whereas those with more years of clinical experiences were less likely. Psychodynamic practitioners were less likely to attend than CBT practitioners whereas the opposite was the case for eclectic practitioners. Practitioners espousing humanistic, family systems, and “other” orientations did not differ significantly from CBT practitioners in their willingness to attend; however, the sample sizes for these orientations were small.

Variables Affecting EST Objections

We examined factors that may influence practitioners’ attitudes about the theoretical obstacles to EST research. The theoretical barriers score was calculated using the simple sum of the items loading on Factor 1, where lower numbers denote greater agreement with the barriers. Preliminary to the analysis, we tested what other demographic and substantive variables might be related to our dependent variable. These proved to be gender, (r_pb = .08, p < .01), graduate school emphasis on treatment research (r = .26, p < .01), years in clinical practice (r = -.12, p < .01), and average hours in practice a week (r = -.11, p < .01). Men and practitioners with more years in practice and heavier case loads agreed more with proposed barriers than women and those who were relatively less experienced and had somewhat lower case loads. Those whose graduate schools had emphasized treatment research were less likely to agree with barriers than those from less treatment research-oriented programs.

We conducted multiple regression with the theoretical barriers score (Factor 1) as the dependent variable. Theoretical orientation, workshop assignment, and the significant biographical variables from the correlational analyses were entered as simultaneous predictors. The five theoretical orientations were dummy coded with CBT as the reference group. We conducted residual analysis to determine if the assumptions of multiple regression were met, and to ensure no outliers unduly influenced our results. There was no evidence of any problems, and we determined the data were appropriate for regression. The overall model was significant, F(10, 1,208) = 34.07, p < .0001, accounting for 21% of the variance in scores on theoretical barriers. The semipartial correlations of the predictors are provided in Table 4.

Table 4.

Semipartial Correlations of Predictors with Disagreement with Theoretical Barriers

Predictor	Barriers Summary Score	Most Extreme Barrier Response
Gendera	.07**	--
Graduate training in research	.16***	.15***
Years in clinical practice	-.06*	-.06*
Workshop demand	-.02	-.05*
Average hours per week	-.06*	.05
Theoretical Orientationb
Psychodynamic	-.38***	-.32***
Eclectic	-.21***	-.17***
Humanistic	-.18***	-.16***
Family Systems	-.05	-.02
Other	-.10***	-.11***

Note. Semipartial rs are derived from simultaneous multiple regression of the Theoretical Barriers factor score on the predictors. When correlations carry a negative sign, higher scores on the predictor are associated with more agreement with the perceived barriers to ESTs, where

^*p < .05

^**p < .01

^***p < .001.

^aA positive correlation indicates men were more likely than women to agree with the theoretical barriers. Gender was not significantly correlated with the extreme barrier score and was removed from the model.

^bTheoretical orientation was dummy coded with cognitive-behavior orientation as the reference category. A negative correlation indicates that a particular orientation group agreed more than the CBT group with the theoretical barriers to ESTs.

All predictors were significant with the exception of workshop demand. Psychodynamic, eclectic, humanistic, and “other” practitioners all subscribed more than CBT practitioners to the proposed objections to EST research. The effect size for the “other” practitioners was medium; all other theoretical orientation differences were large. Practitioners with less graduate training in research were more likely to agree with EST objections than those with more graduate training in research. This difference was also large. Male practitioners were more likely to agree with the proposed barriers than female practitioners, as were practitioners with more years in clinical practice and those who reported more hours of patient contact a week. The effect sizes for these predictors were medium in magnitude.

Contrary to our hypothesis that practitioners would reduce cognitive dissonance by agreeing more with the theoretical barriers when workshop demands were higher, workshop demand was not a significant predictor in this model. However, given the average theoretical barriers score may be an insensitive indicator, on a post hoc basis we utilized the dependent variable described above representing each subject’s most EST-rejecting item score out of the eight items. We regressed these responses on all covariates that predicted this new dependent variable (same as above with the exception of gender) but were unrelated to the experimental manipulation. The results supported the post hoc hypothesis that as workshop time and resource demands increased, practitioners would be more likely to agree strongly with at least one of the theoretical barriers. See Table 4, second data column.

Discussion

One of the primary purposes of this investigation was to identify and assess private practitioners’ attitudes towards ESTs. We endeavored to measure attitudes towards ESTs in two ways: endorsement of barriers and stated willingness to obtain training. Overall, psychologists in private practice held no opinion or mildly to moderately disagreed that each proposed barrier lessened their interest in learning an EST. Moreover, 81.7% of practitioners mildly to strongly disagreed on average that the objections deterred their interest in training. These results suggest that this sample was more positive about EST research than clinicians are often depicted to be, and that clinicians may not subscribe to the objections raised on their behalf in the literature. Only a minority (6.4%) of practitioners mildly to strongly agreed with the barriers, and practitioners on average held no opinion or mild agreement towards their most strongly endorsed objection.

Where specific barriers were concerned, clinicians were least likely to agree that they were not disinterested in learning an EST because they believed that all treatments were of equal efficacy. They were most likely to agree that they were not interested in learning an EST because “a good working relationship with my client is more important than learning how to do a specific treatment.” A considerable number of studies have shown that the therapeutic alliance correlates positively with therapeutic change across a multitude of treatment modalities and clinical issues (Castonguay, Constantino, & Holtforth, 2006). It is a rare psychologist (EST advocate or not) who would argue that better treatment results are obtained by having a poor rapport with one’s clients.

It is possible that clinicians believe that psychotherapists who use ESTs must follow treatment protocols in a robotic fashion, robbing them of empathy and the ability to build a therapeutic alliance. This conclusion may result from a misunderstanding about what psychotherapy researchers mean by manualized treatments. Although manuals provide extensive descriptions of specific procedures to be utilized, flexibility, creativity, and therapeutic skills are inherently necessary so that the therapist can be effective while deploying specified procedures (Kendall, Chu, Gifford, Hayes, & Nauta, 1998). Moreover, although cognitive-behavioral researchers have not always included measures of the therapeutic relationship in their research trials, when they have, any obtained superiority of CBT to an alternative treatment has not been accounted for by alliance differences (e.g., Clark et al., 2006). Nevertheless, if beliefs that EST utilization is antithetical to maintaining a solid therapeutic alliance abound and deter interest in EST training, it may be important for EST researchers to address this misconception not only in further research, but directly to practitioners through communication avenues that are likely to reach clinicians such as national workshops and clinical newspapers (Beutler, Williams, Wakefield, & Entwistle, 1995).

Regarding the advantageous aspects of ESTs, practitioners moderately agreed that it is their professional duty to keep up with new developments in treatment and that it is important to incorporate scientific findings into everyday practice. Overall, practitioners mildly disagreed that they would be interested in learning an EST to get managed care payments, and they had no opinion as to whether learning an EST might help attract new patients. These were exploratory items to investigate possible endorsements of ESTs. It is surprising that clinicians were not motivated by managed care payments. However, few insurance companies currently require ESTs for payment, and recent qualitative research indicates that practicing clinicians strongly reject managed care’s utilization of EST lists for reimbursement (Stewart, Stirman, & Chambless, 2010). More research is necessary to determine what might motivate clinicians to gain training in the current and future healthcare environment.

The results from this study suggest that attitudes towards ESTs vary in relation to biographical factors, specifically theoretical orientation, graduate training, and length in practice. Practitioners of different theoretical orientations differ in their attitudes towards the proposed theoretical obstacles. Related to the findings of Stewart and Chambless (2007) and Nelson and Steele (2007) on EBP/EST attitudes and usage, cognitive-behavioral practitioners were more likely than eclectic and psychodynamic practitioners to disagree with the notion that theoretical barriers deterred their interest in EST training. Of these three groups, evidenced by the large effect size, psychodynamic researchers agreed most with the theoretical obstacles. Theoretical orientation also affected willingness to attend an EST workshop. Psychodynamic practitioners were also less likely to attend than cognitive-behavioral and eclectic practitioners.

Several explanations are possible for this finding. EST research may be inconsistent with many psychodynamic psychotherapists’ conceptualizations of psychopathology and treatment (Strupp, 2001). Psychodynamic psychotherapy is traditionally considered a long-term treatment focused on character change, alleviation of unconscious conflicts, and increasing insight (Strupp, 2001). In contrast, ESTs primarily involve focused interventions geared to symptom relief, and psychodynamic clinicians may perceive the EST model to be irrelevant to the psychotherapy they practice. Nelson and Steele (2007) found that practitioners who viewed treatment research as directly relevant to their work were more likely to use EBPs than those who view such research as irrelevant to their practice.

Moreover, the preponderance of therapy manuals and psychotherapy research has been in CBT (Wilson, 1996) and this is not surprising given the traditional emphasis of CBT researchers on specifying procedures and identifying symptoms and treatment goals. Although this lack is being rapidly reversed (e.g., Leichsenring, Rabung, & Leibing, 2004), and it is likely short-term psychodynamic treatments will appear more on EST lists in the future, it is possible that psychodynamic practitioners perceive more obstacles to use of ESTs because their treatments are not well represented on EST lists – an issue of fairness. However, in the descriptive portion of our questionnaire, we provided a sample of theoretically diverse ESTs, including brief psychodynamic therapy for depression and interpersonal therapy for bulimia. Moreover, clinicians were queried on their willingness to attend a workshop of their choice, and presumably they could select an EST that is consistent with their theoretical orientation. Eclectic practitioners reported more willingness to attend an EST workshop than CBT practitioners. Although the difference was small, it is possible that eclectic practitioners were moved by the diversity of the list of ESTs in the questionnaire and were more interested in training as a result, or that eclectic practitioners had received less training to date in ESTs than CBT practitioners but unlike the psychodynamic practitioners were open to it.

Graduate training proved to be an important biographical variable related to EST attitudes. Our data indicate that those with more research training about psychotherapy in graduate school disagreed more with the theoretical obstacles, and were more likely to report intentions to attend an EST workshop, even after controlling for theoretical orientation. This is consistent with past data indicating that graduate school treatment research training was correlated with pro-EST research attitudes (Stewart & Chambless, 2007). Similarly, Nelson and Steele (2007) found that practitioners who report taking a class in EBPs are more likely to report EBP use. Together, these data suggest that exposure to EBP and EST principles in graduate school can lead to more positive attitudes about EST utilization and training. Aarons (2004) described graduate education and internship as particularly opportune times in which to convey the value of EBPs and ESTs. More research is needed on EBP and EST training at these stages in professional development.

Lastly, clinicians with more years in clinical practice agreed more with the theoretical obstacles and reported being less likely to obtain EST training. Practitioners with more years of experience are farther away from graduate training, and there are data to suggest these practitioners hold less positive attitudes toward adoption of EBPs (Aarons, 2004). However, these findings have not been consistent (Stewart & Chambless, 2007). Another possibility is that clinicians in practice longer necessarily have more clinical experience and may be more subject to the belief that clinical experience trumps empirical findings, despite evidence to the contrary (e.g., Garb, 1998; Hannan et al., 2005; Kadden, Cooney, Getter, & Litt, 1989; Schulte, Kunzel, Pepping, & Schulte-Bahrenberg, 1992). In the present sample, the concept of clinical experience as a more important guide to treatment than research evidence was the second most endorsed barrier. This is consistent with other research indicating that clinicians consider clinical experience to be a highly influential practice factor (e.g., Riley et al., 2007; Stewart & Chambless, 2007), whereas research takes less precedence (Cook, Schnurr, Biyanova, & Coyne, 2009; Nelson et al., 2006). Clinicians’ belief in the primacy of unsystematically recorded clinical experience is a critical factor to consider in any EST dissemination effort.

Our data indicate that practical and theoretical barriers are of paramount importance as predictors, with large and medium effect sizes respectively. Practitioners were more likely to report willingness to attend an EST workshop when there was less demand on their time and resources. As expected, practitioners who agreed more with the validity of the theoretical barriers reported less willingness to obtain EST training.

Our hypothesis that cognitive dissonance due to the conflict between the push for EBP and the costs of obtaining EST training would lead to stronger adherence to theoretical objections to ESTs when workshop demands were higher was not supported when we tested practitioners’ average endorsement of eight EST objections. However, when we tested this hypothesis by looking at the strongest of the eight objections for each participant, significant findings emerged: Respondents were more likely to endorse a negative belief about ESTs as the time and financial cost of the workshop increased. This analysis was post hoc, and any interpretation must very cautious until the findings have been replicated. Tentatively, these findings suggest that practitioners may subscribe to theoretical objections to ESTs in part because these barriers provide a more comfortable reason to ignore EST findings than admitting that one is reluctant or unable to commit the time and resources to obtain EST training.

Limitations

One limitation of this study is that the sample may not be representative of practicing psychologists. Not all doctoral-level psychologists join APA, and doctoral level psychologists are only one group among those who provide mental health services, which includes Masters’ level psychologists, social workers, and counselors of various sorts. Moreover, because only 32% of the sample responded, there is no way if knowing if the responses can be generalized to describe the initial sample, although the characteristics of the sample collected were virtually identical to data on the 19,660 practitioners in APA who report private practice as their primary employment setting. APA does not provide data on theoretical orientation, but our sample is roughly comparable to the most recent survey of the theoretical orientations of psychologists who are members of APA Division of Clinical Psychology (Norcross, Karpiak, & Santoro, 2005). The exception is that Norcross et al. (2005) found a higher percentage of practitioners with an eclectic orientation, perhaps because we pushed strongly for practitioners to select the eclectic label only if they did not have a primary orientation. It is possible that clinicians who respond to a survey describing an EST may be more sympathetic to EST research than non-responders, who may have disregarded the survey given its focus.

A second limitation of the present study is our use of questions about attending a specific EST workshop as a proxy for willingness to gain training in ESTs in general. Although some research indicates that workshops are not effective in changing practitioner behavior (Davis et al., 1999; Jameson, Stadter, & Poulton, 2007), a workshop may offer an introduction to a treatment and serve as a motivational gateway to gaining further treatment. Nonetheless, even a 3-day workshop (the longest workshop we described) is hardly sufficient to train therapists in a new treatment, and supervision is necessary beyond didactics in order to ensure minimum levels of competence (Sholomskas et al., 2005). Thus, although our study indicates willingness to attend one particular workshop, it may not be measuring practitioners’ willingness to submit themselves to the amount of training that is required when learning a new EST.

A third limitation of this study is the small number of positive items about ESTs. These loaded onto two factors, thus limiting the reliability of measurement. Accordingly, we did not conduct hypothesis testing with factors representing incentives to learn ESTs. Additional development of items concerning affirmative reasons to engage in EST training might prove valuable to EST dissemination research.

Conclusions

The present research suggests that there exist both theoretical and practical barriers to EST dissemination and training. This study provides a measure of theoretical obstacles that can be used in further research on practitioner characteristics and attitudes regarding EST dissemination and training. Understanding pertinent barriers to EST dissemination could aid its endorsement and implementation by focusing research efforts to investigate practitioners’ concerns. Indeed, many concerns such as generalizability to clients in one’s practice (e.g., (Stewart & Chambless, 2009; Stirman, DeRubeis, Crits-Christoph, & Brody, 2003; Stirman, DeRubeis, Crits-Christoph, & Rothman, 2005), and flexibility (Gibbons, Crits-Christoph, Levinson, & Barber, 2003) are already being considered in the research literature. However, given that clinicians may not have access to or interest in academic journal articles, it is unlikely that they have been exposed to these rebuttals. This is part of a larger issue in the field of how to disseminate research findings to clinicians through avenues that are likely to reach and interest clinicians. More research is needed on how to appeal directly to practitioners and whether EST promotional efforts can change attitudes or translate to training interest.

Moreover, this study underscores the importance of practical barriers to EST dissemination, a point that has largely been ignored in the literature. Indeed, only 35% of practitioners overall reported willingness to attend our most time- and resource demanding workshop of 3 days, and it likely that even fewer practitioners would be willing to complete full training with supervision to reach competence in an EST. More research is needed on how to make EST training more appealing and worthwhile of the necessary resource demands, particularly the expense and time necessary to gain training. Potential motivators to be investigated could include organizational or insurance requirements, cost-effectiveness, professional development ethics, and client demand. Moreover, policies and avenues need to be developed to assist people already in practice to obtain EST training should they desire it. Our research suggests that practitioners might be more open to receiving training than it appears if it were more feasible for them to do so. Web strategies for training are a particularly promising avenue (Riley et al., 2007). Such training could be combined with supervision provided by telephone. More research on dissemination considering both practical and theoretical obstacles is a critical next step for the field.