Pre-Practicum Training in Professional Psychology to Close the Research-Practice Gap: Changing Attitudes Towards Evidence-Based Practice

Abstract

Despite the rapid proliferation of mental health interventions with proven benefit for youth, empirically supported interventions (ESIs) are underutilized in most service settings. Treatment outcome studies in these community-based settings suggest that the majority of youth do not show improvement, underscoring the importance of addressing the gap between research and practice. Clinician attitudes toward evidence-based practice (EBP) may limit the use of ESIs, and efforts to address these attitudes with post-graduate training pose significant challenges. Pre-practicum training in EBP may address these challenges by familiarizing students with the framework of EBP as well as with the current youth treatment evidence base and the theories and strategies of well-supported interventions. We describe a required EBP course within a professional psychology doctoral program. Forty-two students in two class cohorts completed a measure of attitudes toward EBP prior to the first class and after the final class lecture. Students were predominantly Caucasian women with bachelor's degrees. As expected, over the course of the class, student attitudes became significantly more favorable toward EBP. Students who had previously received a master's degree had more favorable attitudes prior to the class, and students with a prior bachelor's degree showed the greatest change in attitude. The results support the use of pre-practicum training in EBP to improve attitudes toward EBP, which may lead to use of effective practices with clients following training.

The gap between science and service in youth mental health treatment poses a perplexing dilemma. On the one hand, there is an ever-expanding list of protocols in the treatment evidence base shown to be efficacious for a host of child mental health diagnoses and difficulties (see Silverman & Hinshaw, 2008). On the other hand, these interventions are not well represented in community-based “usual care (UC)” (Garland et al., 2010), and there is evidence that the majority of youth treated with UC do not show improvement (Manteuffel, Stephens, Sondheimer, & Fisher, 2008; Warren, Nelson, Mondragon, Baldwin, & Burlingame, 2010). This so-called “gap” has provoked efforts to increase the use of effective, evidence-based interventions in service settings; despite these efforts, challenges related to training, dissemination, support and sustainability remain (Weisz, Ng, & Bearman, 2014). We propose that an increased focus on predoctoral training in professional psychology could address critical barriers proposed to slow the advancement of evidence-based practice into service settings. Results will be presented examining change in doctoral student attitudes before and after a required course on evidence-based practice.

Evidence-Based Practice and Training in Professional Psychology

Evidence-based practice (EBP) in psychology has been defined as “the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences” (American Psychological Association (APA), Presidential Task Force on Evidence Based Practice, 2006, p. 273). Empirically supported interventions (ESIs) are treatment protocols that have demonstrated efficacy in a scientifically controlled study, most often a randomized clinical trial (see Chambless & Ollendick, 2001). While the terms are not synonymous, ESIs are an inherent part of any focus on EBP, since the results of randomized clinical trials represent at least a component of the “best available research.” In apparent congruence, the APA Guidelines and Principles for Accreditation of Programs in Professional Psychology (APA, 2009) state that training programs must “provide a wide range of training and educational experiences through applications of empirically supported intervention procedures” (p. 8); training programs in professional psychology are likewise tasked with ensuring that students develop competency in “evaluating the efficacy of interventions (p. 7). Thus, it would seem that some exposure to and experience with ESIs specifically, and EBP more generally, would be uniform across professional psychology training programs.

Research examining professional psychology training programs call into question whether the efforts related to EBP training meet the goals of the APA guidelines. A survey of 172 students from 60 APA-accredited clinical, counseling, and school psychology graduate programs revealed that a majority of students were unfamiliar with ESI literature or manuals (Karekla, Lundgren, & Forsythe, 2004). A separate study of 227 child psychologists and doctoral student members of APA Division 53 showed that one-third reported minimal exposure, experience, or expertise in ESIs during graduate training (Pidano & Whitcomb, 2012). School psychology directors (n=97) reported that their students' familiarity with ESIs was relatively low and reflected didactic presentations rather than practical experiences (Shernoff, Kratochwill, & Stoiber, 2003). Only 20% of Psy.D. programs—which enroll the largest number of students and produce the majority of doctoral-level direct service providers (Cook & Coyne, 2005)—required clinical supervision in CBT, which comprise the majority of ESIs (Weissman et al., 2006). Taking a step back from specific interventions, fewer than 5% of doctoral students in a study by Luebbe, Radcliffe, Callands, Green, and Thorn (2007) were able to accurately define EBP. Of note, clinically oriented students reported less agreement with the principles of EBP than their more research-oriented counterparts.

Professional psychology training programs may assume that students will receive exposure to and training in EBP in the required community placements or practica that are a part of their training. However, a recent survey found that although three-quarters of professional psychology program directors felt that using scientific and professional literature to inform direct service was highly valued, only one-third of the directors of the practicum sites where those students trained shared this priority (Hatcher, Wise, Grus, Mangione, & Emmons, 2012). Thus, relying on practica experiences to fill in holes in student exposure to EBP may be a risky gamble.

The benefit of training programs to adequately address EBP is multiple; we have already noted that it is necessary in order to fulfill the training guidelines. More generally, an understanding of EBP and of specific ESIs may enable psychologists to comply with the ethical guidelines set forth by APA, including Standard 10.01(b), Informed Consent to Therapy, of the APA Ethics Code (APA, 2002), as noted by Babione (2010), which states “psychologists inform their clients/patients of the developing nature of the treatment, the potential risks involved, and alternate treatments that may be available” (p. 1072). This standard suggests that psychologists are obligated to inform clients of the scientific support for the proposed treatment, as well as information about interventions that have been found to be clinically efficacious for a specific disorder or concern, which would require some knowledge of established ESIs in the literature for particular clinical conditions, or an awareness of how to find such information.

Barriers to Evidence-Based Practice

In order to better understand, and ultimately address the gap between research and practice for youth mental health, research has focused on identifying barriers to EBP use for mental health service providers. A number of ideological barriers, or “attitudes towards evidence based practice” are posited to decrease the likelihood of clinicians adopting EBP, and specific ESIs. For example, some studies have shown that clinicians worry that using ESIs, which are often manualized, will interfere with the development of the therapeutic alliance, prevent genuine connection and dampen creativity (Addis, Wade, & Hatgis, 1999; Pagoto et al., 2007; Stewart, Stirman, & Chambless, 2012). Despite some evidence that the use of manuals may help rather than hinder therapeutic alliance (Langer, McCleod, & Weisz, 2011), clinician concerns about this and other relationship variables may pose a threat for EBP in service settings.

Misconceptions or incomplete knowledge about the evidence base may also pose an obstacle. Many clinicians believe that treatment trials cannot generalize to their client group (Addis et al., 1999; Pagoto et al., 2007). In fact, the growing evidence base of the past decade more accurately reflects the diversity and diagnostic complexity of clients seen in routine usual care, and ESIs outperform UC even when controlling for comorbidity, ethnic minority representation, and symptom severity (Weisz, Jensen-Doss, & Hawley, 2006). There is also a mounting literature on the use of EBP for ethnic minority youth (for a review, see Huey & Polo, 2008). Beyond treatments developed for or tested with ethnic minority youth, thoughtful frameworks have been published for adapting specific interventions for clients in a way that addresses cultural diversity (e.g. Bernal, Jiménez-Chafey, & Domenech Rodriguez, 2009; Feldstein Ewing, Wray, Mead, & Adams, 2012; González Castro, Barrera, & Holleran Steiker, 2010). Nonetheless, lack of attention to client diversity remains a criticism of EBP, and to the extent that clinicians believe that the best evidence excludes their clients, they are not likely to use the treatments recommended by that evidence.

Finally, a major challenge to the use of EBP in service settings is a practical one: the logistical barriers associated with learning ESIs via continuing education and other similar trainings. A large survey of 1,630 clinicians identified that the most frequently reported barrier to adopting new treatments was related to the high cost of training, insufficient time to attend trainings, and the lack of accessible or local training opportunities (Cook, Biyanova, & Coyne, 2009). Post-graduate training is costly and seems an inefficient way to address wide-scale “up-skilling,” since the availability and quality of trainings from one region to another is likely to vary widely and would be difficult to regulate. Furthermore, reviews by Beidas and Kendall (2010) and Herschell, Kolko, Baumann, and Davis (2010) suggest that training workshops alone fail to change provider behavior or facilitate proficiency with new interventions.

Training in Professional Psychology to Address EBP Barriers

There is reason to suspect that providing EBP training in graduate school for professional psychology could address some of the barriers listed above. First, knowledge of EBP appears to be associated with more favorable attitudes towards these practices (Ashcraft et al., 2011; Nakamura, Higa-McMillan, Okamura, & Shimabukuro, 2011). Likewise, having a CBT or a behavioral orientation was also associated with more EBP use, as was a more positive attitude towards research (Nelson & Steele, 2007). Although it is possible to develop knowledge of EBP, a CBT orientation, and positive attitudes towards research at any point in the career continuum, graduate training provides a natural opportunity to present the sort of information to students that would foster a more positive attitude towards EBP. In fact, training received in graduate school and the influence of a significant mentor were among the greatest determinants of current practice in a survey of 2,607 providers (Cook, Schnurr, Biyanova, & Coyne, 2009), suggesting that graduate training can have a substantial impact on practice.

Graduate training programs in professional psychology are also well-positioned to provide clear and coherent information in order to address myths and misperceptions widely held about research, and treatment research in particular. Moreover, graduate schools provide a combined focus on research and practice that could facilitate the systematic introduction of treatment outcome literature and its application. Indeed, even the most clinically-oriented training programs require some research-related competencies in order for psychologists to be consumers of research, allowing for many opportunities to integrate the practice and research perspectives.

The prohibitive financial and time-related challenges associated with postgraduate EBP training could also be addressed by more complete coverage of EBP by graduate training programs in professional psychology. Graduate training programs, more so than brief CE workshops, may be better able to link theories with specific practices to promote a more complete, conceptual understanding of how to best use the strategies associated with ESIs for maximal benefit for specific clients (Hoffman, 2013). Graduate programs are also able to evaluate competency with required curriculum and identify the need for additional training. In contrast, CE workshops must rely on the “train and hope” paradigm wherein there is little capacity to evaluate the impact that training has had on behavior, and even less potential for recommending additional efforts in the event that proficiency with a given practice is not met.

Taken together, we believe a strong case can be made for an increased pre-practicum focus on EBP by graduate training programs in professional psychology. As described elsewhere, (Lee, 2007), such courses would include presentation of research evidence coupled with an emphasis on the underlying theory of the interventions and the disorders. In order to facilitate actual practice, courses should include ample observation of and experience with the practices taught via engaging and interactive techniques such as use of video, modeling, and role-play. Course topics should include issues of therapeutic alliance in EBP, the use of EBP with diverse populations, and how to balance fidelity and flexibility in EBP use. Perhaps most critically, students should develop an understanding of how to continually evaluate interventions with regards to their efficacy, and revise their practices as a result of the evidence.

Consistent with this description, we describe a required course on evidence-based practice offered to students in a combined school-clinical child doctoral training program. In an effort to explore whether such a course could address some of the barriers to EBP described above, measures of student attitudes towards evidence-based practices were assessed at the start and conclusion of the semester-long course in order to examine change in attitudes. Because characteristics such as age and training background have been found to predict differences in EBP attitudes and use in other studies (Ashcraft et al., 2011; Bearman et al., 2013; Nakamura et al., 2011), we investigated whether these variables were related to student attitudes towards EBP.

Method

Participants

Participants were students (N = 42) who took a course entitled “Foundations and Applications of Empirically Supported Practices for Youth” in Spring 2012 (n = 24) and 2013 (n = 18) during training in a school-clinical child psychology Psy.D program (for which it is a required course) or a clinical psychology Psy.D. program (for which it is an elective course) at a Northeastern graduate school of psychology. Characteristics of the participating students are listed in Table 1.

Table 1. Characteristics of 42 participating students.

Characteristic	N	%
Female	38	90
Male	4	10
Age(M±SD)a	25.7±2.6
Race-ethnicity
Caucasian	11	26
Asian	1	2
Other	10	24
Unknown	20	48
Highest degree earned
Bachelor's	34	81
Master's	5	12
Unknown	3	7
Class year
2012	24	57
2013	18	43
MPAS
T1 (M±SD)b	2.8±.49
T2 (M±SD)c	3.2±.58

^aRange: 22-32

^bRange: 1.7-3.6

^cRange: 1.6-3.9

Procedure

Students completed a measure of attitudes towards EBP prior to the first class lecture, and at the end of the semester after the final lecture. Students used a unique, anonymous ID for the pre and post questionnaire. The instructor (Bearman) asked students to complete the form in order to assess beliefs about EBP, and presented these data to students at the end of the final class.

Course Description

The graduate level course entitled “Foundations and Applications of Empirically Supported Practices” was intended to provide an introduction to treatments and intervention strategies with scientific support, largely cognitive behavioral therapy (CBT) and behavioral parent training (BPT), with a particular emphasis on child and adolescent therapy. There were 14 two-hour classes, with one class used for a final exam. The first class introduced the definition of evidence-based practice and the nature and history of ESIs, while the following three classes introduced the theoretical underpinnings of CBT and BPT (learning theory, cognitive theory, and theory of emotion processing), as well as the definitions and history of the “evidence based” and “empirically supported” taxonomy. Class five was devoted to an overview of the common structure of ESIs and the role of therapeutic alliance, and the following six classes provided a more detailed review of treatment elements common across ESIs for commonly occurring youth disorders. This included identifying and reappraising thoughts (class 6 and 7), behavior change with an emphasis on exposure therapy (class 8), behavior change with an emphasis on behavioral activation (class 9), behavior change with use of habit reversal techniques (class 10), and skill building techniques including problem-solving and relaxation (class 11). The final classes highlighted cultural and developmental responsiveness in EBP (class 12) and integrating information about ESIs into the more general EBP framework (class 13). Didactic instruction was accompanied by recordings of treatment sessions, skill modeling and student role-plays.

Resources and assignments

Course grades were based on class participation (10%), weekly thought papers (30%), a written report of a behavioral self-experiment (20%), and a final exam (40%). Class participation and thought papers were based on weekly reading assignments, which included two textbooks (O'Donohue & Fisher, 2009; Szigethy, Weisz, & Findling, 2012) and peer-reviewed articles presenting treatment outcomes for youth or issues related to the use, dissemination, and sustainability of ESIs. To highlight the applicability of ESIs, care was taken to assign case studies that described the use of ESIs for diagnostically complex and culturally diverse clients. For the behavioral self-experiment, students identified an observable behavior of their own to be modified, and developed a monitoring plan for that behavior using a functional behavior analysis. The final exam asked students to apply the definition of “Evidence Based Practice” as the confluence of research, client characteristics, and clinical judgment to several scenarios, and to describe key characteristics of evidence-based interventions.

Measures

Demographics

Demographic information for participants was obtained with IRB approval through the registrar's office in order to match student responses to key demographic variables while keeping student identity confidential.

Attitudes towards EBP

The Modified Practice Attitudes Scale (MPAS) is an eight-item self-report measure of provider attitudes towards evidence-based practice (Borntrager, Chorpita, Higa-McMillan, & Weisz, 2009). On the MPAS, participants respond on a four-point Likert-scale (0 = not at all, 1 = to a slight extent, 2 = to a moderate extent, 3 = to a great extent, 4 = to a very great extent) to indicate agreement with statements regarding evidence based practice. Five items were reverse scored, and items were averaged, with higher scores indicating more favorable attitudes. The MPAS was administered on the first and last days of class and demonstrated acceptable internal consistency in this sample (Time 1 α = .64; Time 2 α = .78).

Analytic Plan

Several independent samples t-tests were conducted to determine whether there were significant differences in Time 1 or Time 2 attitudes between women and men, BA versus MA degree, or class year. A one-way analysis of variance was conducted to determine if there were differences on Time 1 or Time 2 attitudes between ethnic groups. The correlation between age and Time 1 and Time 2 attitudes was calculated to determine whether they were significantly related. To assess attitude change, a repeated measures ANOVA was conducted including any demographic variables associated with the outcome as covariates. To examine change on individual MPAS items, we conducted paired sample t-tests for each item at Time 1 and Time 2.

Results

Pre- and post-course data were available for 42 students who took the course in 2012 (n = 24) or 2013 (n = 18). Demographic characteristics, as well as mean MPAS scores at Time 1 and Time 2 are reported in Table 1.

Age was not significantly correlated with MPAS score at Time 1 [r (42) = .22, p > .05] or Time 2 [r (42) = -.23, p >.05]. Results of independent t- tests revealed that there were not significant differences between women and men on pre- or post-scores, but there were significant differences on Time 1 MPAS scores between the two different class years [t(43) = -2.59, p = .01]. The 2012 class reported significantly less favorable attitudes before the course (M₂₀₁₂ = 2.64 versus M₂₀₁₃ = 3.01). An additional t-test revealed that there was not a significant difference between the two class years on change in MPAS score, t(40) = 0.12, p > .05. On average, the students with a BA reported a significantly larger change in attitudes (M_BA = 0.52) than students with an MA (M_MA = -0.23). At Time 1, students with an MA degree (n = 5) reported significantly more favorable attitudes toward evidence-based practice than students with a BA degree (n = 37), t(37) = -2.23, p = .03 (M_MA = 3.28 vs. M_BA = 2.74). The master's level students MPAS score did not change significantly from T1 to T2, t(5) = .537, p = .62. There were not significant differences in mean MPAS score between the degree groups at Time 2. An additional t-test revealed that there was a significant difference between the two degree groups on change in MPAS score, t(37) = -2.23, p = .03. The results of a one-way ANOVA indicated that there were not significant differences in Time 1 or Time 2 scores reported by students from different ethnic groups (n = 22). The values of all t-tests and ANOVAs are reported, respectively, in Tables 2 and 3.

Table 2. Comparisons of Attitudes towards EBPs by Participant Characteristics.

Measure	Mean (SD)		t	p-value
	Men	Women
MPAS T1	2.70 (.53) n = 5	2.80 (.50) n = 40	-0.43	.672
MPAS T2	3.19 (.55) n = 4	3.20 (.59) n = 42	-0.03	.979
MPAS T1-T2	0.31 (.46) n = 4	0.41 (.74) n = 38	-0.27	.790
	2012	2013
MPAS T1	2.64 (.53) n = 27	3.01 (.37) n = 18	-2.59	.013*
MPAS T2	3.06 (.64) n = 28	3.40 (.39) n = 18	-2.03	.049*
MPAS T1-T2	0.42 (.88) n = 24	0.39 (.41) n = 18	0.12	.903
	BA	MA
MPAS T1	2.74 (.49) n = 37	3.28 (.30) n = 5	2.39	.022*
MPAS T2	3.26 (.56) n = 37	3.05 (.75) n = 5	-0.76	.450
MPAS T1-T2	0.52 (.66) n = 34	-0.23 (.94) n = 5	-2.23	.032*

^*p < .05.

Table 3. Analyses of Variance for Attitudes towards EBP by Participant Ethnicity and Time Point.

Measure	Mean (SD)			df	F	p-value
	Asian (n = 1)	White (n = 11)	Other (n = 11)
MPAS T1	3.00	2.64 (.42)	2.68 (.64)	22	0.20	.82
MPAS T2	2.75	3.25 (.61)	3.14 (.66)	22	0.31	.73
MPAS T1-T2	-0.25	0.61 (.44)	0.37 (1.07)	21	0.63	.54
	Time 1	Time 2
MPAS	2.82 (.49)	3.22 (58)		38	10.3 5	.003*

^*p < .05.

A repeated measures ANOVA was conducted to determine whether MPAS scores changed over time before and after taking the course. Because change in MPAS score was significantly related to highest degree held, highest degree held was entered as a covariate. The results indicate a significant time effect, F (1, 37) = 10.35, p =.003. The students reported significantly more positive views toward evidence-based treatment at the end of the course. The results also indicated there was a significant interaction between time and highest degree held, F(1, 37) = 4.97, p = .03.

Paired sample t-tests compared student responses to each individual MPAS item at Time 1 and Time 2. All items reflected a change towards a more favorable attitude, and three items showed significant change: Item 3 (“Clinical experience and judgment are more important than using evidence-based treatments.”), t(40) = -2.766, p = .009; Item 4 (“I like using evidence-based treatments because of the structure they provide.”), t(40) = -4.039, p < .001 and Item 6 (“Evidence-based treatments allow clinicians to respond to important events in therapy as they come up.”), t(41) = -5.617, p < .001. Item 2 (“Evidence-based treatments do not allow me to tailor my therapy to each client's individual needs.”) also changed to reflect a more favorable attitude, and approached significance, t(41) = -2.006, p = .052.

Discussion

The current study examined the impact of a doctoral-level course on EBP for child and adolescent therapy on the attitudes that pre-practicum graduate students held towards EBP. Treatments that have been rigorously tested and found to be effective are underutilized in community settings, where most clients are seen for services. At the same time, many youth treated in these settings do not appear to benefit from treatment as usual (Manteuffel et al., 2008; Warren et al., 2010). Utilization of treatments supported by research, or ESIs, is a key component of EBP and may result in improved outcomes, since the number of psychosocial treatments with established efficacy is large and growing (Silverman & Hinshaw, 2008). Many efforts to train providers in ESIs focus on postgraduate workshops; however, these may not change provider behavior (Herschell et al., 2010). Even if workshops can lead to proficient delivery of ESIs, attitudes about these practices may influence who seeks out and benefits from such trainings (Stewart et al., 2012). Thus, predoctoral training in ESIs may be especially important. In particular, training provided as part of standard curriculum in professional psychology doctoral programs could improve attitudes towards EBP by increasing knowledge of the treatment evidence base and challenging misapprehensions.

As expected, attitudes towards EBP were more positive after the course than prior to the course, and this difference was significant. In particular, student attitude change on key items suggests that exposure to a course on EBP challenged commonly held misperceptions that have been shown to reduce clinician interest in adopting these practices. Practicing clinicians have reported that barriers to using ESIs include worry that the structure of manualized ESIs would interfere with their ability to meet the needs of their clients and that the inclusion/exclusion criteria used in studies testing ESIs would limit their utility for addressing the types of concerns clients present in typical treatment (Addis et al., 1999; Pagoto et al., 2007). The results of the present study illustrate that as students are exposed to information about ESIs and EBP, they become less concerned about the limitations related to ESI structure and more confident that they can adequately meet the needs of their clients while using interventions consistent with the evidence base. Other research has shown that clinicians with more positive attitudes reported more actual use of treatments with empirical support (Kolko, Cohen, Mannarino, Baumann, & Knudsen, 2009; Nelson & Steele, 2007), suggesting that including this type of course in graduate curriculum may lead to changes in practice behavior.

The current study sample is consistent with other studies that have examined therapist attitudes towards EBP using the MPAS (Bornatrager et al., 2009; Nakamura et al., 2011). Prior to the course, the average student score on the MPAS was most similar to those of doctoral students and doctoral-level trainees prior to a state-funded training on ESIs for youth (Nakamura et al., 2011), and were also similar to—though slightly more favorable than—clinicians of varying training backgrounds prior to an EBP training for a treatment effectiveness trial (Borntrager et al., 2009). The sample in the current study had substantially higher scores on the MPAS following the course than clinicians in the study by Borntrager and colleagues, who attended a six-day EBP training. This may reflect the relatively more intensive presentation of information about EBP and ESIs, given that the course extended across an entire academic semester and included readings, student assignments and tests. More favorable attitudes towards EBPs in the current study at post-training might also be due to the fact that our sample consisted of students, whereas the sample in the study by Borntrager et al. were practicing clinicians.

In the present study, trainees with master's degrees had more favorable attitudes towards EBP prior to the start of class than their BA-level counterparts and showed no significant change in attitudes towards EBP, whereas BA-level trainees showed change towards more favorable attitudes over the course of the class. Studies by Ashcraft et al. (2011) and by Nakamura and colleagues (2011) have found that clinicians with doctoral degrees held more positive attitudes than clinicians with master's and bachelor's degrees; our results may indicate a similar trend in which a prior degree from a mental health training program resulted in more favorable attitudes initially. Due to the very small subset of students with a master's degree who participated, these results should be interpreted cautiously.

Pre-practicum training makes use of existing resources within professional training programs and does not pose barriers related to time and cost, in contrast with efforts to train clinicians after graduate training is complete and they are part of the existing workforce. Embedding courses, such as the one described, into program requirements would also decrease the reliance on practica experiences that are beyond the direct control of the training program and are likely to vary widely in terms of the emphasis that is placed on use of research to influence practice. This variation may result in uneven student preparation that may not align with program priorities (Hatcher et al., 2012). Required EBP courses within graduate programs would also allow programs to assess student EBP competency as part of routine student evaluation. Although the APA Guidelines and Principles for Accreditation of Programs in Professional Psychology (APA, 2009) clearly direct training programs to incorporate “empirically supported intervention procedures” (p. 8) into graduate curriculum, the available reviews of programs over the past decade suggest that there is room for improvement (Karekla et al., 2004; Luebbe et al., 2007; Pidano & Whitcomb, 2012).

Required courses on EBP could also encourage students to adopt a framework of “evidence-based practice,” making use of the best available research in tandem with information about client variables (characteristics, culture, and preferences) in the context of clinical expertise, as encouraged by APA (2006). The ability to practice within this framework presupposes an understanding of the treatment evidence base—and, since the evidence base is ever evolving, an understanding of how to identify and critically digest new research. Because the participants in treatment research trials may differ from a given client in important ways, students may also benefit from the opportunity to consider the complexities of applying research syntheses to individual clients. Although this study did not expressly test whether students were more able to integrate the three components of EBP following the EBP course, they showed more agreement with the statement, “Clinical experience and judgment are more important than using evidence-based treatments,” prior to the class than at its conclusion. It is possible that exposure to course material, and in particular an emphasis on culturally and developmentally responsive use of ESIs, helped students to strike the balance of clinical judgment, client variables, and treatment outcome research that evidence-based practice requires.

Limitations and Future Directions

Results of this study must be tempered by several limitations: most notably, without a comparison group, we cannot be certain that the change in attitudes occurred as a result of exposure to the course material. The small sample size, and particularly the small number of men and master's level students, suggests that results should be interpreted cautiously. Many students did not report their ethnicity, resulting in missing data for this variable. Although students' survey responses were anonymous, they may nonetheless have felt pressure to report more favorable attitudes towards EBP given that they were in an evaluative environment, in a course taught by a professor whose preference for EBP was well known.

In addition, reviews on the impact of trainings in ESIs have shown that didactic presentation may improve attitudes and increase declarative knowledge, but do little to change actual clinician behavior or increase proficiency (Beidas & Kendall, 2010; Herschell et al., 2010). Although the course discussed incorporated active learning strategies such as role-plays, modeling, and observations of recorded sessions, an important next step in this line of research will be to assess whether coursework and practicum experiences in ESIs can have an impact on trainee behavior with clients. This will be especially important in the context of work with multicultural communities who may require culturally responsive adaptation. Lastly, although Psy.D. programs produce the majority of practicing psychologists (Cook & Coyne, 2005), the bulk of the front-line workforce for youth mental health treatment is comprised of master's level clinicians (Glisson et al., 2008). Determining whether pre-service training in EBP can influence the attitudes and behaviors of students in master's programs would, therefore, be of great value.

Conclusion

Because re-training a clinical workforce via post-graduate training and mentoring is costly and inefficient (Hoagwood, Atkins, & Ialongo, 2013), efforts to improve practices should focus attention on how professional psychology training programs can systematically address the research-practice gap. The current study provides encouraging evidence that a course covering common theories of empirically-supported interventions, therapeutic strategies frequently used in ESIs for youth, youth treatment outcome research, and issues related to EBP such as therapeutic alliance and client diversity can improve trainee attitudes towards EBP, potentially paving the way for additional training and increased use of interventions with proven benefit for youth and families.