Using modular psychotherapy in school mental health: Provider perspectives on intervention-setting fit

Abstract

Objective

The “fit” or appropriateness of well-researched interventions within usual care contexts is among the most commonly-cited, but infrequently researched, factors in the successful implementation of new practices. The current study was initiated to address two exploratory research questions: (1) How do clinicians describe their current school mental health service delivery context? and (2) How do clinicians describe the fit between modular psychotherapy and multiple levels of the school mental health service delivery context?

Method

Following a year-long training and consultation program in an evidence-based, modular approach to psychotherapy, semi-structured qualitative interviews were conducted with seventeen school-based mental health providers to evaluate their perspectives on the appropriateness of implementing the approach within a system of school-based health centers. Interviews were transcribed and coded for themes using conventional and directed content analysis.

Results

Findings identified key elements of the school mental health context including characteristics of the clinicians, their practices, the school context, and the service recipients. Specific evaluation of intervention-setting appropriateness elicited many comments about both practical and value-based (e.g., cultural considerations) aspects at the clinician and client levels, but fewer comments at the school or organizational levels.

Conclusions

Results suggest that a modular approach may fit well with the school mental health service context, especially along practical aspects of appropriateness. Future research focused on the development of methods for routinely assessing appropriateness at different stages of the implementation process is recommended.

The past four decades have seen the development of many evidence-based treatments (EBT) for child and adolescent mental health problems. These efforts have produced an impressive array of interventions with substantial empirical support for their efficacious use with youth and families (Weisz & Kazdin, 2010). Simultaneously, there has been growing recognition that efforts to promote EBT use among community-based service providers often fail to result in changes in providers’ clinical practice (Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). Observational evaluations of usual care treatment practices support this assertion and demonstrate that EBTs are used infrequently or at lower intensity than would be expected to produce positive therapeutic effects (e.g., Garland et al., 2010).

As a consequence, research to identify effective methods of reducing this gap between “typical” and “optimal” care has increased quickly and involves identifying barriers to successful EBT implementation and developing methods to overcome them. Among the most commonly-cited barriers are concerns about the “fit” of well-researched interventions within usual care practice contexts (Durlak & DuPre, 2008; Greenhalgh, Robert, McFarlane, Bate, & Kyriakidou, 2004; Stith et al., 2006). Indeed, controlled research settings and usual care contexts generally differ in characteristics of clinicians, treatment settings, and service recipients, calling into question the generalizability of research findings to community treatment settings (Ehrenreich-May et al., 2011; Hoagwood, Hibbs, Brent, & Jensen, 1995; Weisz, Jensen Doss, & Hawley, 2005; Southam-Gerow, Chorpita, Miller, & Gleacher, 2008). Community agencies and providers themselves vary considerably in their receptivity to the implementation of EBT; for instance, Aarons, Sommerfeld, and Walrath-Greene (2009) found that private mental health agencies are more likely than those in the public sector to support the use of evidence-based practices.

Recently, Proctor et al. (2011) presented a heuristic framework for identifying and studying key implementation processes and outcomes related to research-supported intervention practices. Among the implementation outcomes described, the appropriateness of innovations within specific practice contexts is particularly relevant to the issue of intervention-setting fit and the success of efforts to install EBT in new service organizations. Appropriateness was defined as “the perceived fit, relevance, or compatibility of the innovation or evidence-based practice for a given practice setting, provider, or consumer; and/or perceived fit of the innovation to address a particular issue or problem” (Proctor et al., 2011, p.69). As indicated in its definition, appropriateness is nearly synonymous with Rogers’ (2003) notion of compatibility, which he identified as one of the five core characteristics that influence program diffusion, and a key component of Diffusion Theory. Tornatzky and Klein (1982) further detailed that compatibility may refer to either value compatibility, or the congruence of the innovation with the values or norms of adopters, or practical compatibility, which represents a fit with existing practices. Below, the term “appropriateness” (as well as its subtypes; value appropriateness, practical appropriateness) will be used to refer to intervention-setting fit.

Most contemporary implementation frameworks conceptualize the introduction of new programs as an iterative and adaptive process informed by ongoing assessment of intervention appropriateness and effectiveness (e.g., Aarons & Sommerfeld, 2012; Damschroder et al., 2009; Southam-Gerow, Rodríguez, Chorpita, & Daleiden, 2012). There is also increasing awareness that appropriateness is a multi-level construct, which may manifest differently among the range of stakeholders and systems that characterize service organizations. Indeed, many models and frameworks suggest that appropriateness should be evaluated at multiple levels, such as organizational, managerial, provider, and consumer (e.g., Aarons, Hurlburt, & Horwitz, 2011; Fixsen et al., 2005; Southam-Gerow et al., 2012).

Understanding and maximizing the extent to which service providers believe new practices are relevant to and compatible with the multiple levels of their service delivery contexts is essential to promoting initial uptake and sustained use. Although there are a few emerging approaches to enhance appropriateness through systematic program adaptation (e.g., Aarons et al., 2012), successful use of these approaches may benefit from a clear understanding of the implementation context and end-users’ perceptions about the match of specific innovations to specific settings. The primary aim of this paper is to ascertain provider perspectives on the appropriateness of implementing an evidence-based, modular therapeutic approach within a school-based health clinic.

Modular psychotherapy approaches are comprised of “self-contained functional units [modules] that connect with other units, but do not rely on those other units for their own stable operations” (Chorpita, Daleiden, & Weisz, 2005, p.142). In this way, modular approaches have the potential to be more flexible than traditional manualized treatment protocols. Empirical support for the effectiveness of modular approaches is emerging, including a recent randomized controlled trial in which a Modular Approach to Therapy for Children with Anxiety, Depression, and Conduct (MATCH-ADC) outperformed traditionally-structured EBT as well as usual care (e.g., Weisz et al., 2012). In their review of research on factors affecting implementation success, Durlak and DuPre (2008) identified that flexibility and adaptability are generally expected to enhance the fit of an intervention with a given setting. For this reason, multiple authors have suggested that the flexibility of the modular approach may allow for greater compatibility with various levels of service organizations (e.g., Aarons et al., 2011; Chorpita et al., 2005). Some initial evidence in support of this assertion comes from a study by Borntrager, Chorpita, Higa-McMillan, and Weisz (2009), who documented greater acceptability for the modular approach than for standard-arranged EBT manuals at the clinician level. Due in large part to their adaptability and relevance to applied settings, emerging research has also begun to support the feasibility and potential utility of modular interventions delivered by mental health providers working in school settings (Lyon, Charlesworth-Attie, Vander Stoep, & McCauley, 2011; Stephan, Wissow, & Pichler, 2010; Stephan et al., 2012; Weist et al., 2009).

Although schools provide between 70 and 80% of all mental health services delivered to youth (Farmer, Burns, Philips, Angold, & Costello, 2003), schools have lagged behind the mental health system as a whole in the implementation of EBT (Evans & Weist, 2004; Lyon, McCauley, & Vander Stoep, 2011; Rones & Hoagwood, 2000). The education sector also has a number of unique features that pose challenges to the implementation of particular EBTs. For example, a study by Langley, Nadeem, Kataoka, Stein, and Jaycox (2010) recently identified competing responsibilities, lack of parental engagement, logistical barriers, and a lack of support from school administrators and teachers as barriers to the implementation of the Cognitive Behavioral Intervention for Trauma in Schools (CBITS), an evidence-based intervention for middle school students. The traditional structure of many EBT (e.g., 12–16 weekly sessions lasting 50 minutes) also may not fit with the unpredictable and often shorter-term nature of service provision within school settings (Lyon, Charlesworth-Attie et al., 2011).

School-based health centers (SBHCs) are a service delivery model in place in nearly 2,000 schools in the United States that provide an array of health and psychosocial services, including primary care, reproductive health, and mental health care, regardless of insurance status (Brown & Bolen, 2003; National Association of School Based Healthcare [NASBH], 2008). As with education sector services in general, SBHCs represent a proven structure in which service-access disparities (e.g., those based on ethnicity or socioeconomic status) are substantially reduced (Gance-Cleveland & Yousey, 2005; Wade, Mansour, Line, Huentelman & Keller, 2008; Walker, Kearns, Lyon, Bruns, & Cosgrove, 2010).

This study was designed to advance implementation science by documenting the perceptions of mental health counselors regarding the appropriateness of implementing modular psychotherapy within the SBHC setting. Two exploratory research questions addressed features of the school context that affect delivery of modular psychotherapy to middle and high school students: (1) How do clinicians describe their current school mental health service delivery context? and (2) How do clinicians describe the fit between modular psychotherapy and multiple levels of the school mental health service delivery context?

The study was conducted following implementation of a year-long pilot consultation program to introduce modularized psychotherapy into the practice of school-based mental health clinicians (see Lyon, Charlesworth-Attie et al., 2011, for a complete description). Providers reported their perspectives about delivery of modular psychotherapy including its relevance to their client population and fit with aspects of the school-based service setting.

Methods

The current study utilized a qualitative approach. Because the study of modular psychotherapy in schools is in its infancy, qualitative studies can be useful for describing in detail early efforts at fitting this evidence-based intervention approach within the school context.

Participants

Seventeen school-based mental health providers (out of 18 invited) participated in the current study. Participants were 94% female, 88% Caucasian, and were an average of 40.2 years old (SD = 9.9). All but one of the participants had a Masters degree in social work, education, or counseling. They had worked as therapists for between 0 and 30 years (mean = 11.0, SD = 9.1) and had held their current positions for 0 to 17 years (mean = 4.9, SD = 4.3). The majority of participating clinicians indicated that their primary theoretical orientation was “integrative/eclectic,” suggesting that their clinical practice was derived from a number of different orientations (n = 11), with smaller numbers endorsing cognitive-behavioral (n = 3), behavioral (n = 1), interpersonal (n = 1), and systems orientations (n = 1). Most participating providers were the only full-time mental health provider in their respective schools. Of the 17, 7 clinicians participated fully in the consultation program over one full academic year, while 10 either began participation and then discontinued or never engaged in the program. Data from the original study indicated that levels of practitioner knowledge and attitudes about evidence-based practices (using the Knowledge of Evidence Based Services Questionnaire; Stumpf, Higa-McMillan, & Chorpita, 2009; Evidence-Based Practice Attitudes Scale; Aarons, 2004) were comparable to those in national norming samples.

Modular Psychotherapy Intervention

School-based mental health counselors were trained to carry out modular psychotherapy using a subset of the tools available through the PracticeWise Managing and Adapting Practice (MAP) system (Chorpita, Becker, Phillips, & Daleiden, 2009; PracticeWise, 2011). The MAP system is designed to build evidence-informed treatment through the application of a variety of professional development supports and clinical tools. MAP’s major components include: (1) A computerized PracticeWise evidence-based services (PWEBS) database that contains information therapists use to select treatment modules that have the strongest evidence for being helpful for a particular presenting problem; (2) A set of easy-to-use practice guides for each treatment module that give step-by-step instructions for implementing the key elements; (3) A “dashboard” tracking system to monitor use of treatment elements and track a student’s clinical course using standardized and ideographic measures; (4) The MAP Professional Development Program (PDP), which structures training and support at multiple levels to help individuals select, organize, and deliver common elements of EBT and provides a framework for scaling up MAP within organizations; and (5) A therapist portfolio, which includes records of learning and direct service goals and performance standards for MAP trainees.

In the 2009/2010 implementation, an adapted, streamlined version of the MAP system was introduced which differed from the standard MAP system and training process in the following key ways: (a) consultants, rather than providers, accessed the PWEBS database to identify appropriate modules based on data collected about student characteristics and clinical presentations; (b) only modules relevant to depression and anxiety were selected; and (c) elements were introduced gradually in the context of existing consultation throughout the academic year rather than utilizing the standard MAP training and consultation approach (i.e., without the MAP Professional Development Program or Therapist Portfolio). Nevertheless, dashboards were used consistently by providers to track practice elements and client outcomes, transmitted to consultants prior to each consultation session, and served as the foundation for all case discussion and problem solving. Although the current program used clinical modules from the MAP system, the overall professional development approach was somewhat less comprehensive. In this way, service delivery based on the MAP modules was more similar to the MATCH-ADC trial mentioned previously (Weisz et al., 2012). Adaptations were made a priori to maximize efficiency and the relevance of the training and consultation to the school context as well as to make use of existing consultation structures and resources (see Lyon, Charlesworth-Attie et al., 2011, for a full discussion of adaptations and consultation procedures). All clinicians were provided with a brief overview and description of modular psychotherapy at a training that occurred at the beginning of the year when they were given the opportunity to participate. Study procedures were conducted with approval from the local institutional review board.

School Setting

Participating SBHCs were located in middle schools (grades 6–8) and high schools (grades 9–12) in a large urban area in the Pacific Northwest. The health centers generally have one mental health counselor per school. The participating school district has approximately 48,000 students with 360 to 1700 students per school. During the 2009–2010 school year, the district enrolled 9,401 students grades six to eight and 13,230 students in grades nine to twelve. The population was 2% American Indian, 11% Hispanic/Latino, 22% Black/African American, 23% Asian/Pacific Islander, and 40% White. Forty-two percent of middle school students and 40.7% of high school students received free or reduced price lunch.

Data Collected

Data discussed in the current study were collected following implementation of the version of the MAP consultation program described above. In all, clinicians participated in semi-structured qualitative interviews lasting approximately one hour each (average length was 52 minutes). Two interviewers, one of whom had helped to provide the training and consultation, conducted all interviews. For the current project, responses to a subset of questions from the full interviews were explored. These questions generally focused on the ways in which the school context influences psychotherapy practice (e.g., “How does the school environment affect your job and the way you practice?) and their perspectives on the fit of modular approaches with their practice in schools (e.g., “How well do you believe modular psychotherapy matches the needs and characteristics of your overall client population?”). Follow-up probes were used routinely to elicit more detailed responses and specific examples.

Data Management and Analysis

Data for the current study were analyzed qualitatively. Interviews were audio recorded, transcribed, and then coded using a combination of conventional and directed content analysis (Hsieh & Shannon, 2005) and qualitative coding software (Atlas.ti; Muhr, 2004). Content analysis is typically used to derive the contextual meaning of communications and differs from Grounded Theory (Strauss & Corbin, 2008), a related approach to qualitative data analysis, in that it seeks only to understand the meaning of the communication in detail, rather than establish new theory about the ways qualitatively-derived constructs are related (Hsieh & Shannon, 2005). Conventional content analysis, which focuses on simply describing phenomena of interest, was used to evaluate clinicians’ descriptions of the school-based service delivery context as well as modular psychotherapy. Directed content analysis, in contrast, makes explicit use of existing theoretical or empirical frameworks to identify initial coding categories (Potter & Levine-Donnerstein, 1999). This approach was used to evaluate clinicians’ discussions about the appropriateness of modular psychotherapy in schools while drawing from theories describing appropriateness as multilevel (e.g., Aarons et al., 2011) and consisting of value and practical components (Tornatzky & Klein, 1982). These a priori codes are presented in the results section.

Conventional content analysis coding began with four coders reviewing clinician responses to each question from the same three transcripts, identifying potential codes and then meeting to discuss the codes and produce an initial codebook that represented a combination/consolidation of the codes identified. The two researchers who conducted the interviews also served as coders for the qualitative analysis. This codebook was then trialed independently through multiple iterations in which all team members coded one to three transcripts and met for discussion. During this process, new codes relevant to the research questions were added, others removed, and some codes merged or split into sub-codes. This process continued over six iterations until a stable set of codes was reached. A priori codes for the directed content analysis were then integrated into the final codebook.

Following the completion of the codebook, coding occurred using a consensus process similar to that described by Hill et al. (1997; 2005), in which each transcript was recoded independently by two different raters who then met to arrive at consensus judgments through open dialogue (DeSantis & Ugarriza, 2000; Hill et al., 1997; 2005). The two coding team members who had completed the original interviews were split into different groups for consensus coding. The consensus coding process is designed to circumvent some researcher biases while being more likely to capture data complexity, avoid errors, and reduce groupthink. Consensus coding was performed as an alternative to the calculation of inter-rater reliability and is considered by many qualitative researchers (e.g., Hill et al., 1997) to be a more valid method of analyzing human communication. Instead of assuming an objective “reality” that coders must match, consensus coding makes explicit use of differences in opinion and coding ambiguities to prompt discussion and increase confidence in findings.

Results

School as a Context for Modular Psychotherapy Implementation

RQ1 investigated how clinicians describe their practice context. Table 1 displays rank-ordered codes identified in clinicians’ descriptions as well as code descriptions. Codes identified include characteristics of the services they provided/clinician practice, the setting in which they practiced, and the youth served.

Table 1.

Rank-Ordered Codes Identified in Clinicians’ Descriptions of the School-Based Mental Health Service Delivery Context

Code	Description	Percent
Clinician Practice
Approach	Treatment approach within therapy sessions	94%
Referral	Outbound referral processes, agency support (or lack thereof), and youth most appropriate for external services	88%
Teamwork	Team approach to intervention and school or SBHC staff impacting services	77%
Consult/Supervision	Desire for consultation or support and/or having another therapist on staff	77%
Scope	Scope of practice and what clinicians provide to the school	53%
School Setting
Accessibility	Access provided by school based mental health services	82%
Need-Availability	High service need and limited time and staff to provide indicated services	71%
Unpredictability	School environment is unpredictable and impacts service provision (e.g., school schedule, absences, crisis)	71%
Confidentiality	Issues of confidentiality in school mental health	59%
Information	Information available from or about other students, teachers, etc.	59%
Client Characteristics
Engagement	Engagement in services, relationship building/support, and expectations for treatment success	82%
Population	Characteristics of the diverse student/client population	71%

Clinician practice

Related to their clinical practice, respondents discussed their intervention approach, outbound referral processes, teamwork, consultation and supervision experiences, and practice scope. Intervention approaches were typically varied, with references to both evidence-based and non-evidence-based treatments and a frequent use of psychoeducation. Fifty-nine percent of participants described routine use of practices that are common components of cognitive-behavioral therapy (CBT). Furthermore, most clinicians emphasized the importance of therapeutic alliance in their work (“having a good relationship”).

Many providers (88%) discussed processes related to making determinations about which youth should be seen outside the school context and then connecting them to those services. Some counselors felt that more intensive cases, such as eating disorders, self-injury and trauma, were the most appropriate candidates for outside referral (e.g., “anytime I have a kid hurting themselves, an eating disorder, cutting, any of those get referred out,” “If it’s something trauma related then it can feel overwhelming to address that in a school environment”). These determinations appeared to stem from worries that the school allowed for an inadequate transition from the therapy room to the classroom (e.g., “to have them sort of walk in here and, open up and address all these intense therapy issues, and then walk out and see their friend about a second later is hard”). Other comments suggested that youth who missed considerable amounts of school, or for whom family therapy was indicated, might be best treated in the community. For this reason, some counselors described their role as an “adjunct” to non-school services. Barriers to linking youth to appropriate outside services included little support from sponsor organizations, insurance status (“do they have insurance that would cover them elsewhere”), the likelihood that the student would be able to follow through on a referral, and the quality of services in the community (e.g., “that’s the hardest piece…not having good…services out there”).

As members of their larger school communities, clinicians commonly described advantages to working as a team with other school, student, and health center staff. Increased communication and engagement was frequently noted by clinicians experiencing more connection and positive relationships with school staff and other healthcare providers within the school (“working pretty closely with teachers”). Despite valuing other professionals working in schools, many clinicians mentioned feeling isolated (“1800 kids by myself”).

Related to their scope of practice, about half the clinicians described their responsibilities as variable (“I feel like the beauty of [working in a school] is that we can be whatever they need in that moment”). Descriptions of their roles included, “jack of all trades,” “a go-to guy,” “a pressure valve,” as well as being “a resource for the faculty and for the staff” and a case manager (“a lot of times we are therapists trying to be case managers”). Because of the wide and variable scope of their responsibilities, clinicians reported struggling with caseload pressures and confusion about their role and effectiveness (“a lot of the times I feel like I’m sort of practicing out of the scope of what I will be really effective at”), but enjoyed the flexible practice afforded by the school context (“I’m able to provide interventions that are, perhaps, more flexible”).

School setting

In addition to those directly related to their clinical practice, other relevant characteristics of the school setting reported by respondents included the accessibility to students of services provided by SBHCs, the high mental health service needs of their school population, the unpredictability of day-to-day events, confidentiality, and use of information not typically available in other contexts.

Despite the challenges associated with working in schools, many of the clinicians (82.4%) discussed the advantages of students being able to access services within the school setting (“you really do get to see kids that normally would never get to be seen”). Related to this accessibility, a large percentage of the clinicians (71%) reported that more students in their schools needed mental health services than they were able to treat (“we tend to get over-booked,” “I can easily be filled in September”), and suggested solutions for managing the high need included running groups, referring out, and conducting time-limited therapy.

Unpredictability and a lack of control of events during the school day was reported by 71% of clinicians and included scheduling difficulties (“I can’t assume a teacher is going to let them out,” “they may not show up that day”), limitations of the school year (“on break,” “summer approaching”), environmental stress (“fire drill,” “a fight in the hall”), and crisis intervention (“community-wide crisis,” “teacher finds them crying in the hall”).

Several clinicians also discussed the role of confidentiality in school. Although confidentiality is often promised to students within the school setting (“I let them know everything in here is 100% confidential”), some reported that this confidentiality limits collaboration (“Sometimes the school staff kind of got in the way because they didn’t understand privacy issues the way that we do as mental health counselors”) and affects access to services when teachers are unwilling to release students from class. Conversely, clinicians explained the advantage of obtaining information about their clients by observing them within the school context (“the ability to see them in their setting is huge”) and receiving information from the school personnel (“I get all the emails from the student intervention team”). As one clinician stated, “I’m really good at getting information and not giving it.”

Client characteristics

Although they were not specifically asked about their clients, respondents commonly described the diverse characteristics of the clients with whom they worked (“a lot of our parents are immigrants,” “We work with a very low end economically”), as well as the potential impact of those factors on therapy engagement. Common issues facing the clients included truancy, emotion regulation (e.g., “anger issues, fighting”), difficulties with mood and anxiety (“depression and anxiety are probably our two biggest ones”), adjustment issues, and grief and loss (“I do a fair amount of work…around loss”).

Modular-School Appropriateness

Drawing from the theoretical and empirical findings discussed earlier, directed content analysis was used to evaluate the research question addressing the appropriateness of modular psychotherapy for school mental health. These analyses only included clinicians who reported some familiarity with modularity principles, typically from exposure to the MAP consultation or the initial project introduction to the full group (n = 12). However, unlike other coding in the current study, initial a priori codes addressing appropriateness at the client, provider, school, and organizational levels were developed with specific sub-codes for value and practical components of fit. Table 2 displays these codes, organized by level, and their frequency of occurrence. As is readily apparent from the table, clinicians primarily discussed the appropriateness of the MAP intervention within schools at the levels of the client and provider. Across levels, practical themes (i.e., those related to concrete processes or practices) were mentioned by more providers than themes addressing values or norms.

Table 2.

Percentage of Clinicians Mentioning Appropriateness Themes at Different Contextual Levels in Relation to the Modular Approach

Code	Description	Percent
Client Value	Fit with values/norms of target population	33%
Client Practical	Fit with practical factors related to target population (e.g., effectiveness, developmental appropriateness)	75%
Client Other	Uncategorized comments at the client level	92%
Provider Value	Fit with values/norms of provider	33%
Provider Practical	Fit with practical factors of the provider (e.g., clinical practices/approach, knowledge gaps)	66%
Provider Other	Uncategorized comments at the provider level	25%
School Value	Fit with values/norms of school/school clinic	0%
School Practical	Fit with practical factors in a particular school (e.g., school schedule)	8%
School Other	Uncategorized comments at the school level	0%
Organization Value	Fit with values/norms of external organization/agency	0%
Organization Practical	Fit with practical factors of the organization/larger system	8%
Organization Other	Uncategorized comments at the organization level	0%

Client level

At the client level, statements about the practical appropriateness of the modular approach for schools were both positive and negative. Many statements related to the perceived effectiveness of the approach with adolescents and the mental health problems most commonly encountered by the respondents in schools (i.e., depression and anxiety). Some comments indicated that the modular approach was flexible enough to be compatible with student absences and other interference (“Whatever length or depth of the relationship you have with your client…there were discrete pieces that you could use in a session”). Simultaneously, its structure was seen as generally realistic and advantageous for the client population (“those modules give them some tools to work with their feelings”). Comments about specific modules also related to this theme, such as one that suggested psychoeducation was particularly applicable “because a number of my clients haven’t ever had therapy or counseling.” Most practical comments that were critical of the modular approach referenced student need hierarchies, stating that sometimes needs more pressing than depression/anxiety warranted prioritization (e.g., “we needed to address other stuff first and I’m not sure how to get around that…horrible family relationships…needing to find somewhere to live or food or something like that”). Additional comments related to the developmental appropriateness of the intervention for youth of different ages (“some of the kids that are more…cognitively-oriented”).

In contrast to practical appropriateness, nearly all providers who discussed value appropriateness made comments about the lack of fit between the modular approach and the youth they see in schools. Value/norm appropriateness included general comments about youth engagement, some of which were tied to client cultural factors. Statements here included those that many youth were not sufficiently engaged in services, generally, for the approach to be effective (e.g., “our population is…not always the most, engaged or committed to counseling… [so] trying to make that transferable was challenging”), or that there were characteristics of the modular approach that made it less appropriate for certain clients. One therapist commented that the approach is “very in-tune with the white population” to whom “science and therapy…makes a lot of sense…so when you do psychoeducational stuff, it’s very interesting [to them].”

Clinicians also made a large number of uncategorized, general comments about the fit of the modular approach, most of which were unelaborated statements that the approach fit their population “well.” However, perhaps not surprisingly, a number of clinicians indicated that the fit for their entire caseload was incomplete (e.g., “I don’t think it’s useful for every person,” “I think it probably meets [the needs of] about 75% of the overall population”).

Clinician level

Positive value comments at the service provider level included descriptions of personal preferences for characteristics of the modular approach (e.g., “I like concrete things”). Practical comments were often related to providers’ own learning styles or knowledge gaps in that MAP provided an opportunity to learn more about how to intervene with youth presenting with “things I didn’t really know how to treat well.” Providers also differed with respect to their opinions about how much effort was required to use the modular approach. Although many indicated it was “pretty simple [and], easy to learn,” others stated that the actual application of the tools was more difficult and that “it takes a lot of practice” to master them.

Multiple clinicians discussed how the modular approach “matches my practice,” contains “things that we were familiar with anyways,” and is consistent with how many providers operate (e.g., “we all do it all the time in bits and pieces, but this was really helpful in that it just put it in a small package and organized it”). On the negative side, one provider commented that she thought that the modular approach could interfere with her existing practices (“I didn’t feel like I could use my self-relaxation techniques and use the MAP stuff”).

School and organization levels

As can be seen in Table 2, few comments were made related to the school or broader organizational levels, and the comments that were made represented only practical issues. One clinician described ambivalence surrounding whether or not conducting trauma intervention (i.e., via a trauma narrative/exposure) was appropriate in her school setting. Another expressed appreciation for the consultation structure that accompanied implementation of the adapted MAP system because of a lack of support at the organizational level (“I wasn’t getting much support anywhere else [in my agency]”).

Improving fit

Clinicians were also specifically asked if there were any ways to improve the fit of the modular approach with their service setting and caseload. Some clinicians reiterated their previous comments about the deficits in fit they had observed (e.g., addressing its applicability with a low-resource population; developmental appropriateness). Others commented on the written materials themselves, suggesting more handouts could be provided or that they could be revamped in some way (e.g., “groovier worksheets”).

Discussion

This study was designed to inform future dissemination and implementation of evidence-based practice within the school setting by documenting some of the ways a modular approach may be compatible and incompatible with school-based service delivery. Results highlighted the variability that characterizes school-based mental health service delivery at multiple levels. When discussing the fit between the modular approach and the school context, clinicians most commonly emphasized factors operating at either the level of the client or provider. Although policy-level influences have been identified as an important factor in implementation (e.g., Noonan et al., 2009), no providers made comments about appropriateness that could be classified at that level. This may be due to the fact that organizational policies may not explicitly impact day-to-day clinical care in these settings. Below, key elements of fit at the client and clinician levels are summarized, elaborated, and integrated using themes identified when discussing the appropriateness of the modular approach for the school context. Recommendations for evaluating and maximizing program appropriateness during implementation are also discussed.

Client Appropriateness

General comments about the degree of fit between the modular approach, as represented in the adapted MAP program, and the SBHC client population indicated relatively high perceived appropriateness (e.g., “it probably meets [the needs of] about 75% of the overall population”) and effectiveness. Nevertheless, clinicians also described exceptions to that fit, especially surrounding client values, noting specific subpopulations for which the modular approach may be less appropriate as implemented. As described in the results, these comments focused primarily on aspects of client engagement. First, the flexibility afforded by a modular approach generally seemed to be a good match for the varied, and sometimes unpredictable, ways that students tend to use services, in that it allowed clinicians more freedom to work within the practice constraints inherent to the education sector (e.g., missed sessions due to absences). Second, engagement comments referenced the applicability of more structured and directive psychotherapy techniques to ethnic minority and low-income groups. Indeed, the relevance and effectiveness of “traditional” western psychosocial intervention approaches for ethnically and culturally diverse individuals and communities has long been a focus of discussion and skepticism (Bernal & Scharron-del-Rio, 2001). Despite this, research on the effectiveness of EBT with diverse groups of youth, though limited, has generally been encouraging (Huey & Polo, 2008). In the current study, cultural value appropriateness included general comments about service engagement and notions that ethnic minority clients may be less engaged in more traditional psychotherapy services. As a function of its built-in adaptability, a modular approach may be one method through which practitioners working with ethnically and culturally diverse populations can overcome some, but likely not all, of the commonly-cited tension between model fidelity and flexible service delivery (Kendall & Beidas, 2007; Lyon, Lau, McCauley, Vander Stoep, & Chorpita, under review). In light of recent findings that attention to a greater number of components of cultural adaptation (e.g., language, content, goals) is associated with increased intervention effectiveness (Smith, Domech Rodriguez, & Bernal, 2011), future research should study whether clinicians implementing modular psychotherapy in schools are able to address those components.

Information about which client presentations are most appropriate for school-based service delivery also provides some insight into the types of clients who might be best served by a modular approach in the education sector. Based on a variety of factors (e.g., greater school absences among high severity students), some clinicians described their services as most appropriate for low and moderate severity youth or as adjunctive to external services for more severe youth. For example, multiple clinicians stated a belief that significant trauma was an inappropriate target for school-based intervention. Although trauma is often an important focus of clinical practice, our adapted version of the MAP system did not explicitly address trauma outside of the generic anxiety modules. Interestingly, trauma was also not a separate component of Weisz and colleagues’ (2012) large-scale randomized trial of the modular approach to youth psychotherapy. Subsequent to the start of the original trial, however, trauma was given more explicit emphasis with the addition of a set of trauma-oriented intervention modules (Chorpita & Weisz, 2009). Importantly, other effective methods are already available for addressing trauma in the school setting, and some researchers in this area have conducted groundbreaking work surrounding education sector evidence-based practice implementation (e.g., Kataoka, et al. 2003; 2011; Langley et al., 2010). In consideration of these findings, it may be the case that school-based trauma treatment is most likely to be perceived as appropriate when it is a primary or more explicit focus of intervention.

Clinician Appropriateness

Clinicians generally described their practice as highly variable in response to the needs of their clients and the school context. Much of this variability appeared to be driven by the accessibility of services inherent in the education sector. In this way, the majority of the contextual and practice themes described suggested a need for expansion in the size and scope of practitioner caseloads and clinical responsibilities. For instance, clinicians described spending considerable time working to refer some students to outside services, but those services were frequently unavailable or of largely inadequate quality. These factors necessitate that school-based clinicians, as the default provider for many youth, are equipped to address the needs of caseloads of considerable size, diversity, and acuity, often on an ad hoc basis. Because it was designed to facilitate use of a more comprehensive evidence-based approach for a fairly wide variety of client diagnoses (Chorpita, Daleiden, & Weisz, 2005), the modular approach seems particularly relevant to filling gaps in clinician training. In the current study, this aspect of appropriateness was supported by clinician statements that the modular approach effectively increased their ability to treat client problems they had previously felt less confident addressing.

The practical appropriateness of the modular approach for school clinicians may also have been facilitated by the fact that some clinicians already utilized cognitive-behavioral therapy (CBT) and other components of treatment in their practice that were consistent with some of the practices contained in the adapted MAP system. Although only three clinicians described CBT as their primary theoretical orientation, many more clinicians described using psychoeducation in their practice, a key element of many CBT approaches. Further, 59% explicitly mentioned using some CBT components. Even beyond CBT, many noted that the MAP system made use of various techniques that they already used. The consistency of new training with existing practices is an important, but understudied, element of EBT implementation. Indeed, it has been suggested that trainings should present a somewhat balanced combination of novel and familiar practices to be seen as accessible and relevant to practitioners (Lyon, Stirman, Kerns, & Bruns, 2011). More routine and systematic focus on increasing effective treatment elements that are already used at sub-optimal levels by practitioners (e.g., via shaping toward their more frequent/intensive application) may be one key approach to improving service quality. This approach could also help assuage clinician concerns that implementation of new techniques interferes with the application of other strategies that clinicians already use and believe to be effective. Use of unstudied clinical practices, some of which have been developed organically in response to needs of the local context, may be more possible within a modular framework than in other service delivery models. Indeed, findings from the recent MATCH-ADC trial indicated that clinicians participating in the modular condition were more likely to use evidence-based, as well as non-evidence-based, intervention components than clinicians in the standard manual condition (Weisz et al., 2012).

Summary and Recommendations

The current study evaluated clinician perspectives on the appropriateness of modular psychotherapy for use in schools as an illustration of the importance of intervention-setting fit for successful implementation of new practices. Findings suggested that a modular approach is generally appropriate for implementation within education sector mental health, but that there may be opportunities for adaptations or refinements to further increase fit. Because the study included a sample of clinicians working in one particular type of school-based service delivery model (SBHCs), caution is warranted when generalizing findings to other models or settings. Many districts contract with one or more external public mental health agencies to provide services to their students (Foster et al., 2005). Nevertheless, considering that many of the implementation barriers in schools (e.g., logistical barriers; difficulties with parental engagement; Langley et al., 2010) are similar to those encountered by community-based service providers in other settings, the current findings may have relevance to additional contexts.

In addition, the current project used a modular approach that was streamlined to include only those practices that were relevant to depression and anxiety. The findings should therefore be interpreted with caution when applied to the larger literature on modular interventions, which often include a broader array of client presentations (including behavior problems) (Weisz et al., 2012). The limited diagnostic scope in the current paper may also have influenced clinician sentiment that the model met the needs of “75% of the overall population.” It may be that a more comprehensive version of the model would have been seen as even more appropriate.

Future implementation research should focus on careful evaluation and enhancement of appropriateness as a method to improve the uptake of evidence-based practices in service settings. Considering that there are currently few measures available that explicitly evaluate appropriateness (Lewis, Borntrager, Martinez, Fizur, & Comtois, 2011), it may be that development of new assessment tools for this construct is indicated. In the current study, information about appropriateness was largely retrospective, collected following the completion of program implementation. Appropriateness has also been identified as a construct that could be effectively evaluated prior to initial adoption (Proctor et al., 2011). At early stages of the implementation process, prospective assessment of appropriateness could drive development of contextually-fitting program adaptations.

Finally, as more research focuses on the implementation of evidence-based programs in settings that differ from those in which they were originally developed, it may be necessary to clearly differentiate appropriateness from other identified implementation outcomes, such as the closely-related construct of acceptability. Acceptability can be defined as “the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory” (Proctor et al., 2011, p.67). It remains unclear if appropriateness and acceptability uniquely influence implementation or whether they can be measured in a way that allows each to contribute independent variance. Comments from school-based clinicians in the current study suggested that many found the components of the adapted MAP approach acceptable (e.g., the modular structure and influence of a CBT framework), but that they nevertheless had questions about its appropriateness for subgroups of their client population.

In sum, this study adds to the emerging literature examining intervention-setting fit and the construct of contextual appropriateness, using modular psychotherapy and school-based health centers as exemplars. Beyond evaluations of its appropriateness in schools, additional studies testing the effectiveness of the modular approach are needed. The first large-scale randomized trial of a modular approach has recently been published (Weisz et al., 2012), but many questions remain surrounding the extent to which the approach is effective in improving mental health outcomes in schools, with virtually no information available about its impact on academic or other school outcomes (e.g., attendance). Evidence for positive academic effects is likely to significantly enhance the perceived appropriateness of any mental health intervention delivered in schools, thereby facilitating implementation.