Evidence-based Practices Addressed in Community-based Children’s Mental Health Clinical Supervision

Abstract

Context

Clinical supervision is the principal method of training for psychotherapeutic practice, however there is virtually no research on supervision practice in community settings. Of particular interest is the role supervision might play in facilitating implementation of evidence-based (EB) care in routine care settings.

Objective

This study examines the format and functions of clinical supervision sessions in routine care, as well as the extent to which supervision addresses psychotherapeutic practice elements common to EB care for children with disruptive behavior problems, who represent the majority of patients served in publicly-funded routine care settings.

Methods

Supervisors (n=7) and supervisees (n=12) from four publicly-funded community-based child mental health clinics reported on 130 supervision sessions.

Results

Supervision sessions were primarily individual in-person meetings lasting one hour. The most common functions included case conceptualization and therapy interventions. Coverage of practice elements common to EB treatments was brief.

Discussion

Despite the fact that most children presenting to public mental health services are referred for disruptive behavior problems, supervision sessions are infrequently focused on practice elements consistent with EB treatments for this population. Supervision is a promising avenue through which training in EB practices could be supported to improve the quality of care for children in community-based “usual care” clinics.

Introduction

Clinical supervision is the principal method of training for psychotherapeutic practice (Lambert & Ogles, 1997). A national survey of 200 clinic directors in community-based child mental health care settings confirmed that clinical supervision is ubiquitous across geographic areas and provider disciplines (Schoenwald et al., 2008). Despite the prominent role of clinical supervision in practice settings and relatively extensive existing literature defining theoretical and conceptual approaches to supervision (Aten, Strain, & Gillespie, 2008; Atkinson & Woods, 2007; Friedberg, Gorman, & Beidel, 2008; Hernández, 2008; Holloway, 1995; Kilminster & Jolly, 2000; Moffett, 2009; Morgan & Sprenkle, 2007), it is the “least investigated” aspect of clinical training and education (Ellis, Ladany, Krengel, & Schult, 1996; Kilminster & Jolly, 2000), with only minimal empirical research on supervision processes and/or effectiveness. The important emerging research that has been done on supervision processes and outcomes has been conducted primarily within the context of evidence-based intervention implementation and/or dissemination efforts (e.g., Schoenwald, Sheidow, & Chapman, 2009), or within the context of an academic clinical training clinic (Callahan, Almstrom, Swift, Borja, & Heath, 2009). However, to our knowledge, there has been no research on supervision conducted in routine care settings in the community, outside the context of an intervention trial. Given that many, if not most trainees across mental health disciplines receive practicum and/or internship training in community-based settings, research describing the supervision structures and content in these settings is needed.

Research on supervision in community-based care can also contribute to our understanding of factors associated with treatment effectiveness (or lack thereof). Based on a study of treatment effectiveness in an academic training clinic, Callahan and colleagues (2009) estimated that supervision may account for 16% of the variance in youth treatment outcome. Given that specific psychotherapeutic interventions have been associated with only half as much variance in one meta-analysis (Wampold, 2001), this is particularly impressive and reinforces the need to better understand how supervision processes may be associated with improved treatment effectiveness. Improved knowledge about effective clinical supervision methods has broad implications for the national mental health workforce, which has been described as lacking competence in delivering effective interventions (Schoenwald, Hoagwood, Atkins, Evans, & Ringeisen, 2010).

Clinical supervision (defined here as ongoing individual or group training/mentoring on psychotherapy) can be broad in character and focus. A number of theories about clinician development and models of supervision exist across fields of psychology, counseling, social work, nursing, education, and medicine (Kilminster & Jolly, 2000). All recognize that clinical supervision has at least three major functions, including administrative (e.g., monitoring case documentation, legal/ethical issues), supportive/restorative (e.g., providing support for emotional impact of clinical work, developing confidence), and educational/formative (e.g., fostering clinical skill development) (Holloway, 1995; Kadushin, 1976; Proctor, 1987). However, little is known about the relative emphasis given to these functions in routine care supervision.

The limited available reports on supervision approaches common in routine care settings identify limitations in the supervision infrastructure to support therapeutic skill development (Carroll & Rounsaville, 2007; Hurlburt & Knapp, 2008). Specifically, Carroll and Rounsaville (2007) report that supervision in community-based substance abuse treatment focuses primarily on administrative functions (e.g., paperwork), which is consistent with anecdotal reports from therapist participants in our research-practice partnership team, who provide treatment to children across a range of disorders (described in Garland, Plemmons, & Koontz, 2006).

Carroll & Rounsaville (2007) conclude that supervision based on objective practice standards is “virtually nonexistent” in the United States, and there is no evidence to suggest that this is less true for other mental health areas. Indeed, a study surveying program managers of community-based clinics in California found that supervision in publicly-funded outpatient clinics is relatively unlikely to include structured, therapeutic skill-building components, such as use of videotape or live observation, or use of any structured checklists for performance-based skill development (Hurlburt & Knapp, 2002). Given that there are few structured guidelines and/or resources for effective supervision, there is likely wide variation in the structure and quality of supervision practice in community settings.

With ongoing national calls for implementation of evidence-based (EB) practices in routine care settings, it is also critical to explore the potential role of clinical supervision in supporting implementation of EB practices. Workshop training has been a common model for teaching EB practices, but workshops alone often have limited sustainable impact on routine care (Beidas & Kendall, 2010; Kavanagh et al., 2008). Community mental health practitioners indicate that supervision is crucial to successful adoption of EB practices (Gioia & Dziadosz, 2008). Indeed, ongoing supervision has been associated with more successful intervention implementation (Beidas & Kendall, 2010; Heaven, Clegg, Maguire, 2006). However, intervention implementation is likely to be most effective when it is tailored to knowledge about the current service context (Casper, 2008; Hoagwood & Kolko, 2009; Perkins et al., 2007). Therefore, it is important first to learn more about the existing functions, format, and content of routine care supervision. The extent to which discrepancies exist between supervision approaches in routine care and those in EB treatment models could point to logical and potentially potent quality improvement efforts (Garland, Bickman, & Chorpita, 2010).

Unfortunately, the literature on clinical supervision focuses heavily on theoretical models of supervision without directly testing these models (see Kilminster & Jolly, 2000 for a review). The limited available empirical research has been largely conducted in the context of intervention trials testing the efficacy or effectiveness of specific treatment protocols. Some of these studies have demonstrated that the quality of supervision (defined as demonstrating strong fidelity to the intended intervention model) is associated with clinical outcomes for patients (Callahan et al., 2009; Ng, 2005; Schoenwald et al., 2009). For example, supervisor adherence to multisystemic therapy principles predicted greater supervisee adherence to these principles in therapy, which in turn was associated with significantly greater reductions in youth externalizing and internalizing behavior problems (Schoenwald et al., 2009). Supervisor focus on clinical development and adherence to a specified supervision structure/process was also associated with improvements in youth behavior problems. Other research has also shown that supervisors account for a significant amount of variability in client outcomes (Callahan et al., 2009; Ng, 2005).

Our research focus in clinical supervision is broader, examining the extent to which routine clinical supervision (i.e., outside the context of an intervention trial or an academic training clinic) addresses therapist skill-building and delivery of care consistent with EB practice. Multiple EB treatment protocols have been developed and empirically supported for many diagnostic groups, including children with disruptive behavior problems (Eyberg, Boggs, & Nelson, 2008). Despite differences in features and format, there is substantial overlap among EB treatments for disruptive behaviors. For example, training parents to properly use positive and negative reinforcement with their child is often an important component of EB parent training programs for children with disruptive behavior disorders (Eyberg, Boggs, & Algina, 1995; Patterson, Chamberlain, & Reid, 1982; Webster-Stratton & Hammond, 1997). Likewise, child-focused anger management strategies are common in EB child psychotherapy for this patient population (Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008). Thus, there are often common “practice elements” shared across individual EB treatment models for particular clinical problems (Chorpita, Daleiden, & Weisz, 2005; Garland et al., 2008). A set of practice elements common across EB treatments for children with disruptive behavior problems has been previously identified (Garland et al., 2008).

There are a number of arguments for identifying and selecting individual elements from empirically supported treatments that are common across multiple individual treatment protocols (Chorpita & Regan, 2009; Garland et al., 2010; McHugh, Murray, & Barlow, 2009). Rather than learning multiple individual treatment protocols, it may be more practical for clinicians to acquire critical EB treatment skills that could be delivered flexibly to patients with different and/or multiple presenting problems. This more flexible approach to practice may fit better with the diagnostic complexity and comorbidity so common in community-based patient populations (Chorpita et al., 2005). Furthermore, implementation models perceived by clinicians as more flexible may inspire stronger intentions to change practice, thereby increasing the likelihood that EB treatment strategies are actually adopted and sustained over time (Perkins et al., 2007). Finally, this approach could provide clinicians from diverse training backgrounds with a common foundation for improved practice. Overall, a common elements approach to implementation of EB practice may fit more readily within the existing service context (e.g., patient characteristics, clinicians’ preferences, and organizational structures).

This study addresses gaps in literature by gathering detailed data on a sample of 130 supervision sessions conducted in publicly-funded routine care, community-based clinical settings. Descriptive data regarding the format and functions of supervision sessions are presented, followed by data on participants’ satisfaction with the amount of time spent on different supervision session functions. The extent to which clinical supervision sessions addressed a set of psychotherapeutic practice elements that are common to EB practice for the majority of the patient population (i.e., children with disruptive behavior problems) is also examined. The resulting data can be used to identify potential quality improvement targets.

Methods

This study examined supervision practices in four publicly-funded community-based child mental health clinics in a large Southwestern county using self-report event-contingent, web-based surveys of supervisors and supervisees.

Participants

Written informed consent was obtained from supervisors and supervisees. All participants were financially compensated for their participation in research interviews and protocols were approved by the appropriate human subjects committees.

Supervisors

The sample included supervisors (n=7) from the four largest publicly-funded outpatient mental health clinics serving children in San Diego County. All the supervisors in these clinics (except one who was retiring during the study period) were identified for recruitment in 2009 and all agreed to participate. The mean age of supervisor participants was 46 years (SD=12.4), and 6 (86%) were female. By self-report, six (86%) were Caucasian and one (14%) was Asian American/Pacific Islander. Supervisors reported an average of nine years (SD=7.9) experience supervising. About half of the sample had a doctoral degree in Psychology (n=4, 57%) while the other three (43%) had a master’s degree in Marriage and Family Therapy/Counseling. Primary theoretical orientations included cognitive-behavioral (n=3, 43%), family systems (n=1, 14%), eclectic (n=1, 14%), and other (n=2, 29%). Supervisors were supervising an average of two supervisees at the time of the study.

Supervisees

All supervisees whose supervisors were participating in the study were recruited for participation and all agreed to participate Participating supervisees (n=12) were mostly female (n=11, 92%) with an average age of 26 years (SD=3.7), nine (75%) of whom were trainees and three (25%) of whom were unlicensed staff. Race/ethnicity was mixed, with 58% Caucasian, 17% Latino, 8% Asian American/Pacific Islander, and 17% Mixed/Other. Regarding mental health discipline, 50% endorsed Marriage and Family Therapy, 42% endorsed Psychology, and 8% endorsed Social Work. Primary theoretical orientations included cognitive-behavioral or behavioral (33%), family systems (25%), eclectic (25%), humanistic (8%), and other (8%). Most had a master’s degrees (67%) while others had a bachelor’s degree (33%). The average number of years of therapy experience was 3 years (SD=2.4).

Procedures

Survey data were gathered from participating clinical supervisors and their supervisees to assess the content of individual supervision sessions and the extent to which practice elements (i.e., psychotherapeutic strategies) common in EB practice were discussed in supervision sessions. Data collection occurred over 12 weeks of supervision sessions and included the administration of two measures. The secure, online survey program “Vovici” was utilized for all survey administrations and submissions to support efficient and secure data collection (e.g., reducing opportunities for data entry errors). These web-based surveys ensured timeliness of data collection by sending survey prompts the day of the scheduled supervision session and reminders at pre-determined, intervals after the supervision session had occurred. This program supported the creation of a secure submission gateway that allowed only approved, identified users to receive and submit survey data. Each week, on the day of their scheduled supervision session, participants received a link to the Supervision Process Questionnaire and the Treatment Strategy Questionnaire (described below) via email. If the survey was not completed within three days of the request, an email reminder was automatically generated by the software program and sent to participants. The response rate of survey submission for supervisors was 65% and for supervisees was 73%. Data were obtained for 130 total supervision sessions, with reports from both respondents for 102 sessions and reports from only one respondent for 28 sessions.

Measures

Given that no established measures to assess supervision session format and content could be identified, measures were developed for this study by the research-practice partnership team, consisting of researchers and one senior clinician from each of six publicly-funded clinics in this county (for a description of the partnership team, see Garland et al., 2006). These measures were initially piloted and refined by team members over the course of three months. Measures are available from the authors.

Supervision Process Questionnaire

The Supervision Process Questionnaire was developed to assess supervision functions in routine care clinics. Members of the partnership group used their own existing supervision checklists, supervision contracts and examples of their own supervision sessions to inform the development of this measure. Members evaluated the measures for comprehension, face validity, acceptability and potential for broad use across disciplines and levels of experience. The final measure assessed 1) total duration of the supervision session; 2) session format (i.e., in-person, live, individual, group); 3) session data source (i.e., videotape, audiotape, skill checklist, progress notes); 4) and potential supervision functions, including crisis assessment, administrative tasks, case management issues, case conceptualization/formulation, therapy interventions/approaches, client relationship/alliance, supervisory relationship/process, and supervisee’s professional/academic roles. Respondents indicated how many minutes were devoted to each of these functions and whether the amount of time devoted to each function was “too little,” “about right,” “or too much.”

Inter-rater reliability

Supervisee-supervisor reliability for the amount of time spent on each content area was moderate to substantial for all areas except case conceptualization and alliance. The following rule of thumb was used to evaluate intraclass correlation coefficients (ICCs): <.40 (poor reliability), .40 to .59 (fair reliability), and .60 to .74 (good reliability) (Cichetti, 1994). The ICC values are the following: crisis assessment (ICC=.446), administrative tasks (ICC=.401), case management issues (ICC=.406), case conceptualization/formulation (ICC=.223), therapy interventions/approaches (ICC=.441), client relationship/alliance (ICC=.246), supervisory relationship/process (ICC=.712), and supervisee’s Professional/Academic Roles (ICC=.660). This level of agreement is expected given the specificity of the data collected (i.e., number of minutes spent on various functions). Although most ICCs are in the fair range, the relative amount of time spent on each function as reported by supervisors and supervisees is comparable. For example, the ICCs for case conceptualization and alliance represent poor reliability, but the relative amount of time devoted to case conceptualization (23.7% v. 25.5% by supervisor and supervisee report respectively) and alliance (11.8% v. 11.3%) across all sessions was relatively comparable between supervisors and supervisees. Aggregated data indicate a general consensus on rank order of relative time spent on supervision functions between supervisors and supervisees as well.

Treatment Strategy Questionnaire

The Treatment Strategy Questionnaire assessed the extent to which the supervision session addressed practice elements that have been identified as common to EB practice for children with disruptive behavior problems (Garland et al., 2008). Members of the research-practice team refined the descriptions of the practice elements and structure of the measure as discussed earlier. The practice elements were divided into treatment techniques (i.e., delivering positive reinforcement, delivering limit-setting, modeling, role playing, assigning/reviewing homework, psychoeducation, establishing/reviewing treatment goals), and treatment content areas (i.e., principles of positive reinforcement, principles of limit-setting, parent/child relationship, affect/anger management, affect education, anticipating setbacks/relapses). Respondents were asked to rate the extent to which these strategies were discussed in the supervision session on the following Likert type scale: 1 (not at all), 2 (very briefly), 3 (somewhat thoroughly), 4 (thoroughly).

Inter-rater reliability

Agreement between supervisees and supervisors was not strong regarding whether or not particular practice elements were addressed in supervision sessions (i.e., ICCs ranged from .02 to .54). Supervisees were more likely than supervisors to indicate that particular practice elements were discussed at all in supervision sessions, suggesting that supervisors were more conservative in determining whether or not elements were discussed. Given the low reliability across the informant groups, we report these data by supervisor and supervisee informants separately. However, consistency in interpretation of the practice elements across informant groups is supported by the fact that the ranking of the frequency of discussion of individual practice elements by each informant group, which is relatively consistent. For example, “homework” was reportedly discussed least often and “parent-child relationship” was discussed most often according to both supervisees and supervisors.

Data Analysis

Descriptive data regarding supervision format, functions, and coverage of EB practice elements were calculated with SPSS (v. 15.0: Release 15.0.1., 2007). Follow-up analyses examined supervisor/supervisee characteristics associated with the extent of discussion of EB practice elements. For these analyses, the 3 (somewhat thoroughly) and 4 (thoroughly) ratings were grouped into one category because there were so few ratings of ‘4’. This combined rating is referred to as “at least somewhat thoroughly”.

Results

Format

Supervision sessions occurred every one to two weeks and ranged in duration from 40 to 100 minutes, with a mean of 59.4 minutes (SD=9.4). Supervision largely occurred in person (96.9%) and was primarily individual supervision (74.8%). The majority of supervision sessions utilized progress notes as the data source (83.2%). Other data sources/aides included the use of skill/therapy checklists (4.6%), videotape (12.2%), and audiotape (0.8%). Few (1.5%) sessions were categorized as “live supervision.”

Supervision Functions

Table 1 presents mean data for each informant group on percentage of time per supervision session devoted to different supervision functions. There was relatively high concordance between supervisees and supervisors regarding the relative percentage of time spent devoted to different functions. Approximately 25% of time in supervision sessions was spent discussing therapy interventions and approaches, while another 25% was spent discussing case conceptualization/formulation. Client alliance, case management issues, and administrative tasks each occupied about 11% of supervision time. Supervisee professional/academic roles (≈6%), crisis assessment (≈5%), and supervisory relationship (≈4%) occupied relatively little time in supervision.

Table 1.

Percentage of time spent in supervision sessions on content areas.

	Total time in % (SD)		Sessions in which dissatisfaction with amount of time is reported (%)
			“too little”		“too much”
	supervisors	supervisees	supervisors	supervisees	supervisors	supervisees
Therapy Interventions/Approaches	27.2 (15.4)	25.6 (18.1)	23	9.5	0.0	0.0
Case Conceptualization/Formulation	23.7 (15.2)	25.5 (16.7)	11.3	3.0	4.8	0.0
Client Relationship/Alliance Building	11.8 (11.2)	11.3 (12.0)	18.0	11.1	1.6	0.0
Case Management Issues	11.3 (10.9)	11.3 (12.2)	6.9	1.0	4.6	0.0
Administrative Tasks	10.9 (10.3)	11.4 (13.6)	2.3	3.0	7.0	1.0
Crisis Assessment	5.6 (13.3)	4.7 (9.6)	2.3	2.0	0.8	0.0
Supervisee’s Professional/Academic Roles	5.9 (14.1)	6.2 (12.8)	14.5	16.5	0.0	0.0
Supervisory Relationship/Process	3.9 (8.3)	4.1 (11.1)	16.1	9.0	0.0	0.0

Participants’ satisfaction with amount of time devoted to different functions is presented in the second set of columns of Table 1. Overall, the majority of supervisees and supervisors were satisfied with the amount of time spent on each of these functions. When dissatisfied, supervisees and supervisors largely reported devoting “too little” time to a particular function. For example, “too little” time was spent discussing therapeutic interventions in 9.5% of supervision sessions according to supervisees and 23% of sessions according to supervisors. Interestingly, neither supervisees nor supervisors reported spending too much time discussing this topic during any supervision session. Supervisees also felt that “too little” time was spent addressing professional/academic roles (16.5%), and client alliance (11.1%), while supervisors felt that “too little” time was spent discussing client alliance (18.0%), supervisory relationship/process (16.1%), supervisee’s professional/academic roles (14.5%), and case conceptualization/formulation (11.3%).

EB Practice Elements Addressed

Overall, both sets of respondents indicated that practice elements common to EB treatments for children with disruptive behavior problems were discussed, but they were not necessarily discussed thoroughly, nor in all supervision sessions. Figure 1 presents data on supervisees’ and supervisors’ reports of the extent to which supervision sessions included thorough discussion of EB practice techniques, while Figure 2 examines supervisees’ and supervisors’ reports of EB practice content areas. For example, the top bar in Figure 1 indicates that supervisors reported that delivering positive reinforcement was addressed briefly in 49% of sessions, somewhat thoroughly in 15% of sessions, and thoroughly in 5% of sessions. Thus, this element was addressed in 70% of all sessions but usually briefly.

Figure 1.

Percentage of supervision sessions addressing techniques common to EB practice.

Figure 2.

Percentage of supervision sessions addressing content areas common to EB practice.

According to supervisee report, the average individual practice element was discussed thoroughly in 13.8% of supervision sessions, discussed somewhat thoroughly in 21.6% of sessions, discussed briefly in 29.4% of sessions, and not discussed at all in 35.2% of sessions. Supervisors were more conservative in their reports, describing the average individual EB practice element being addressed thoroughly in 4.7% of supervision sessions, discussed somewhat thoroughly in 17.8% of sessions, discussed briefly in 35.1% of sessions, and not discussed at all in 42.5% of sessions.

Out of seven possible techniques common to EB practice, an average of 0.3 (SD=0.8) to 1.0 (SD=1.5) were covered thoroughly per supervision session according to supervisors and supervisees, respectively. The most common techniques reportedly addressed were principles of positive reinforcement, principles of limit-setting/discipline, and delivering limit-setting/discipline. Out of six possible content areas common to EB practice, an average of 0.3 (SD=0.6) to 0.8 (SD=1.3) were covered at least somewhat thoroughly, according to supervisees and supervisors, respectively. The most common EB content elements addressed were parent-child relationship, treatment goals/agenda, and psychoeducation.

Discussion

This study examining supervision represents one of the first efforts to provide relatively detailed information about supervision sessions from supervisor/supervisee dyads in routine care settings. Supervision sessions were primarily individual face-to-face meetings lasting approximately one hour, utilizing progress notes as the primary data source. Supervision sessions addressed many different functions, with the majority of time devoted to therapeutic intervention approaches, case conceptualization, case management and alliance building. Practice elements common in evidence-based treatments were addressed in supervision, but were rarely addressed thoroughly.

These results provide more detail regarding some existing data and assumptions about clinical supervision. For example, while a national study found that about one-quarter of 200 community-based child mental health centers reported reviewing videotapes and just under one-fifth reported reviewing audiotape in supervision (Schoenwald et al., 2008), this study suggests that regular use of videotape (12.2%) or audiotape in supervision sessions (0.8%) is relatively rare. This discrepancy may reflect the fact that while many clinics do use video or audio tape, they do not necessarily use it frequently (i.e., the different results may reflect percentage who ever use these methods, versus the percentage of supervision sessions in which the methods are used). In addition, routine care therapists in a research-practice partnership group (see Garland et al., 2006) reported that administrative tasks take up a large amount of time in supervision, interfering with the ability to discuss information directly related to treatment. However, these data suggest that administrative tasks actually occupy relatively little time in supervision, with approximately half of the time devoted to discussion of case conceptualization and therapy interventions. Furthermore, supervisors and supervisees were generally satisfied with the amount of time devoted to the different supervisory functions. When dissatisfied, they mostly indicated that “too little” time, as opposed to “too much,” was available for discussion of the various functions.

Participants indicated that practice elements common in EB treatments were discussed in supervision sessions, but thorough coverage of practice elements common to EB care was relatively rare. The average practice element was discussed thoroughly in less than 5% of sessions according to supervisors and less than 15% of sessions according to supervisees. Techniques most frequently discussed thoroughly included principles of positive reinforcement, principles of limit-setting/discipline, and delivering limit-setting/discipline. Content areas most frequently discussed thoroughly included parent-child relationship, treatment goals/agenda, and psychoeducation. However, some elements key to many EB treatments were addressed infrequently. For example, assigning/reviewing homework was not addressed at all in more than half of supervision sessions.

Study Strengths and Weaknesses

This is the only study we have identified that presents relatively detailed data on the functions of clinical supervision sessions in community-based “usual care” clinics. Although the number of participants is small, the participants represent multiple mental health disciplines and were recruited from different organizations so as to maximize diversity. The sample is relatively representative of the therapists in these settings, the majority being female and Caucasian, with master’s degrees (Glisson et al., 2008). The sample of supervision sessions is relatively large and data collection was timely and efficient, contributing to good response rate and minimal missing data. Because data collection relied on participants’ self-report, it is subject to potential cognitive biases and the measures need to be validated through additional research.

The poor reliability across informants on the Treatment Strategy Questionnaire, in particular, is concerning. Supervisees consistently rated the extent to which common elements of EB practice were addressed in supervision sessions higher than supervisors. This may reflect a social desirability effect whereby supervisees may have been reluctant to report critically on supervision sessions despite assurances of confidentiality. It is also possible that supervisees had lower expectations about the extent to which these practice elements should or could have been addressed in supervision sessions, therefore perceiving them to be covered thoroughly relative to their expectations. In addition, it is possible that the data collection procedures may have influenced supervision practices. However, we are optimistic that this method—assessing for supervision activities the day of the session and aggregating across multiple sessions—was likely more valid than a more general survey about supervision practices in the abstract. Certainly, future studies would benefit from increase sample sizes and generalizability.

This study represents a preliminary investigation of the extent to which practice elements common to EB treatment are addressed in usual care supervision. The criterion set of common elements of EB practices for children with disruptive behavior problems was used because the vast majority of children in community-based outpatient care present with these types of problems (Garland et al., 2001; Kazdin & Wassell, 2000; Offord, Boyle, & Racine, 1991). Thus, the odds that supervisees had patients with disruptive behavior problems to discuss in clinical supervision were extremely high. In addition, several of these elements have been identified as common to EB care for other clinical problems (e.g., anxiety: Kendall & Barmish, 2007; depression: Weersing, Rozenman & Gonzalez, 2009). It is important to note, however, that supervisors and supervisees may have been discussing EB practice elements outside this particular criterion set for patients with other types of clinical problems. Thus coverage of EB practice elements may represent an underestimate.

Clinical Implications

Supervisee/supervisor dyads reported that discussion of therapy interventions and case conceptualization takes up slightly more than half of the supervision session, or about 30 minutes. Within this amount of time however, thorough coverage of practice elements consistent with EB care for children with disruptive behavior problems was minimal. Even though the vast majority of children presenting to public mental health services are referred for disruptive behavior problems, this study found that supervision sessions often only briefly address practice elements consistent with EB treatments for this population. Despite the national importance of implementing EB treatments in usual care clinics (National Advisory Mental Health Council, 2001), a large gap between knowledge about EB treatments and delivery continues to exist in these settings (Perkins et al., 2007).

Organizational culture and climate may be contributing to this gap since they are associated with usual care therapists’ attitudes toward adoption of EB practice (Aarons & Sawitzky, 2006). Lack of structural support in adopting EB practices, such as clinical supervision, may be an especially important barrier to enacting change. As a structure that is already an integral part of usual care settings, supervision could serve in part to train and support therapists’ implementation of EB principles. With regular supervision focusing on increasing the intensity of common elements of EB practice, quality of patient care could potentially improve, as well as therapists’ perceptions of specific EB treatments (Perkins et al., 2007). Nevertheless, the question still remains regarding how to effectively train supervisors in EB practice. Although a common way to disseminate EB practice, workshop training for supervisors has been found to minimally impact supervision practice (Kavanagh et al., 2008).

The importance of effective clinical supervision for effective, sustainable implementation of specific evidence-based treatments has been highlighted in recent research (e.g., Schoenwald et al., 2009), but the role of clinical supervision in ongoing routine care has been largely ignored. This is particularly surprising given that many mental health clinicians across disciplines receive practicum or internship training in these settings. Academic accreditation and licensing organizations provide some structural guidelines for amount of supervision and supervisor qualifications, but there is limited guidance on the nature of content of supervision. Relatively recent attention to minimal competency standards for clinical skills may reflect some progress towards greater attention to clinical skill development (The Association of Directors of Psychology Training Clinics Practicum Competencies Workgroup, The Council of Chairs of Training Councils Practicum Competencies Workgroup, Hatcher, & Lassiter, 2006).

This study provides an unusual glimpse into “usual care” clinical supervision, both with respect to time management of the different functions discussed and the intensity with which elements common to EB practice are addressed. Overall, discussion of techniques and content common to EB practice was minimal. Although there is much to be learned about the quality and impact of supervision, it is a promising avenue through which training and continued support of EB practices could be supported.