Adapting Evidence-based Mental Health Treatments in Community Settings: Preliminary Results from a Partnership Approach

Abstract

This paper describes the application of a university-community partnership model to the problem of adapting evidence-based treatment approaches in a community mental health setting. Background on partnership research is presented, with consideration of methodological and practical issues related to this kind of research. Then, a rationale for using partnerships as a basis for conducting mental health treatment research is presented. Finally, an ongoing partnership research project concerned with the adaptation of evidence-based mental health treatments for childhood internalizing problems in community settings is presented, with preliminary results of the ongoing effort discussed.

Using Partnerships to Adapt Evidence-based Mental Health Treatments for use Outside Labs

Recent findings suggest that up to 20% of youth experience a mental disorder and a large portion of these youth do not receive adequate treatment (Hoagwood & Olin, 2002;U.S. Public Health Service [USPHS], 2000). Many of the problems children experience can be classified as either internalizing or externalizing disorders. Internalizing disorders such as anxiety and depressive disorders (e.g., Gotlib, Lewinsohn, & Seeley, 1995; Lewinsohn, Hops, Roberts, Seely, & Andrews, 1993; Shaffer et al., 1996; Silverman & Ginsburg, 1998) cause considerable impairment and have been linked to psychopathology in adulthood (e.g., Costello, Angold, & Keeler, 1999; Pine, Cohen, Gurley, Brook, & Ma, 1998). Additionally, externalizing disorders such as childhood conduct problems affect between 5 to 10% of children (Costello, Mustillo, Erkanli, Keeler, & Angold, 2003; Nolan, Gadow, & Sprafkin, 2001) and represent a majority of referrals to mental health clinics (Loeber, Burke, Lahey, Winters, & Zera, 2000; Wilens et al., 2002).

Over the past three decades, clinical researchers have accumulated evidence in support of the efficacy of psychosocial treatments tested in research settings (Chorpita & Southam-Gerow, 2006; Compton, Burns, Egger, & Robertson, 2002; Weisz, Weiss, Han, Granger, & Morton, 1995), and identifying evidence based therapies (EBTs) effective in treating childhood mental health problems has become a federal public health policy priority (USPHS, 2000). Unfortunately, EBTs are not usually used in “real-world” settings such as community mental health clinics (Weiss, Catron, & Harris, 2000; Weisz, Weiss, & Donenberg, 1992) and it is only recently that efforts have been made to deploy EBTs in such settings (Chorpita et al., 2002).

Accounts for the discrepancy between the scientific support for EBTs and the paucity of their use by community clinics are numerous and thoroughly addressed elsewhere (e.g., Schoenwald & Hoagwood, 2001a; Southam-Gerow, 2004; Weisz, 2000). However, this paper will focus on one common explanation for this gap: EBTs have almost exclusively been developed and tested in university-based research settings that appear to differ from typical service clinic settings (e.g., Hammen, Randolph, & Weisz, 1999; Southam-Gerow, Weisz, & Kendall, 2003; Southam-Gerow, Chorpita, Miller, & Gleacher, 2008). This research-clinic based method of treatment development ignores the fact that mental health treatment occurs in and is affected by a multi-level system including clients, providers, agencies, and the broader mental health system (Schoenwald & Hoagwood, 2001b; Southam-Gerow, Ringeisen, & Sherrill, 2006), and it has proceeded with limited to no involvement with many of the relevant stakeholders, including community clinic staff (e.g., providers, administrators) and families.

Attention to the multi-systemic context of community mental health may improve uptake of EBTs in community settings. The current paper describes an approach to adapting EBTs that considers the context of treatment delivery (Southam-Gerow et al., 2006) and involves a high degree of collaboration between researchers and mental health stakeholders. In the next sections of the paper we will describe this collaborative partnership between clinical scientists and community stakeholders. We will define what is meant by partnership research and briefly describe different ways to accomplish such research. Finally, we will provide preliminary results from our ongoing partnership research effort occurring in a mental health program for children and adolescents in public mental health agency.

We begin by discussing what we view as the spectrum of clinical studies, presented in Table 1. The first three columns in the table are defined using relatively traditional terms and progression (cf. Friedman, Furberg, & DeMets, 1998), whereas the latter two columns are more reflective of work by Schoenwald, Hoagwood, and Weisz (e.g., Schoenwald & Hoagwood, 2001b; Weisz, Southam-Gerow, Gordis, & Connor-Smith, 2003). We briefly describe each stage with a focus on the research questions and research designs common to each.

Table 1.

A continuum of treatment development stages

Stage name	Early clinical studies (cf. Phase I & II trials)	Efficacy studies (cf. Phase III trials)	Effectiveness studies	Transportability studies	Dissemination studies
Focus of inquiry	Safety, feasibility of the treatment in tightly controlled setting, estimate of the effect size	Comparative effects of the treatment in tightly controlled setting	Comparative effects of treatment in a new, less controlled setting; cost effectiveness	Identifying factors related to successful implementation of the treatment in a variety of settings	Testing different strategies to get the end-user to use your intervention, including the implementation strategy
Primary research questions	-Does the treatment work? -Can the treatment be delivered reliably? -What is the expectable effect size? -Is the treatment safe?	-Is the treatment superior to a control treatment?	-Does the treatment work in a practice setting? -Is the treatment cost effective?	-What processes are needed to make the treatment work in practice settings?	-What dissemination strategies are most effective? -How does the implementation model fare when disseminated on a large scale (e.g., service system wide)?
Outcomes	-Preliminary, safe, feasible treatment model	-Treatment model with an evidence base	-Treatment model with cost-effectiveness evidence -Treatment model with evidence of effectiveness in an alternate setting	-Implementation model	-Dissemination model
Research design(s)	Case reports, single case design, multiple baseline, pre-post, open trials	RCT	RCT, pre-post, benchmarking, cost-effectiveness designs	Surveys, qualitative studies, correlational research, single-case studies, pre-post, RCT	RCT, pre-post, benchmarking

According to the model presented in Table 1, ”early clinical research,” using single case and open trial studies, establishes the safety and preliminary effects of a new treatment. If the treatment passes this first “test,” efficacy studies represent the next step. The primary goal of efficacy studies is to determine if the treatment produces good outcomes in controlled settings when compared to some control group. Using randomized controlled trial (RCT) methodology, the passage of time (i.e., a waitlist) or a placebo represents an initial control group. If the treatment proves effective against a passive control, active treatments are then used as comparators. Once efficacy studies have generated supportive evidence, the treatment is tested in less-controlled contexts such as practice or school-based settings in what is known as effectiveness studies.

The efficacy/effectiveness distinction has generated controversy (cf. Barlow, 1996; Donenberg, Lyons, & Howard, 1999; Nathan, Stuart, & Dolan, 2000; Wells, 1999). While the primary research question for both efficacy and effectiveness studies remains on the effects of the treatment for the client/patient, effectiveness studies go beyond the efficacy studies by testing the mettle of the treatment under real-world conditions, including the use of therapists who work in the setting (vs. graduate students in research labs), inclusion of clients referred through typical channels (vs. research recruitment procedures), and therapy provided in clinical settings (vs. research settings). Effectiveness studies also utilize research designs beyond RCT and often include an analysis of cost-effectiveness.

Once effectiveness studies have demonstrated that a treatment can produce beneficial effects in a cost-effective manner in community service settings, a possible next step is wide-scale dissemination. Indeed, for many products (e.g., medications, cell phones), this level of research and development is adequate to move to wide-scale dissemination. However, some have argued that an intermediate step between effectiveness and dissemination is needed for psychosocial treatments: namely, transportability studies (Chorpita & Nakamura, 2004; Schoenwald & Hoagwood, 2001b). Transportability studies examine the processes involved in deploying the treatment in a community setting. Questions of interest include which clinician, organizational, or service system variables impact the execution and outcomes of an innovation in usual care settings? The goal of transportability studies is to identify the strategies needed to encourage the adoption and effective execution of innovations (Schoenwald & Hoagwood, 2001a). Such strategies include: (a) identifying settings appropriate for dissemination, (b) securing and maintaining funding and referral streams, (c) identifying agency- and system-level changes that will be needed to execute the treatment program in the setting, (d) implementing training and supervision procedures for therapists and supervisors, and (e) recognizing administrative supports needed to monitor outcomes. The primary outcome of transportability research is an implementation intervention, an elaboration of the methods and procedures needed for treatment adoption new settings. Without evidence regarding critical contextual factors and a subsequent set of procedures to implement the treatment with the best fit considering those factors, the process of devising and testing dissemination strategies for a particular treatment program may be a fruitless effort. Such hastiness may yield an array of problems and decrease the chances that a treatment is integrated into the new system (Schoenwald & Hoagwood, 2001b).

Refinement of an implementation intervention involves a move away from the exclusive focus on client-level outcomes characteristic of the first three stages of treatment development. Schoenwald and Hoagwood (2001a; Southam-Gerow, et al, 2006) have elaborated a model that specifies important variables to consider at multiple levels, including client, therapist, treatment delivery model, agency, and systems levels. A researcher must adopt a variety of research designs to explore these topics because the central questions vary and can be exploratory, a topic we return to later.

The outcome of transportability studies is a set of strategies, potentially including modifications to the treatment itself, that appear necessary in implementing and disseminating an evidence-based treatment effectively in multiple diverse settings. The researcher’s next step is the dissemination stage, with a focus on how to disseminate both the treatment and the implementation strategies to achieve wide-spread adoption. Thus, another intervention is being developed and tested: a dissemination intervention. This intervention consists of procedures and methods that encourage adoption of both the treatment and the implementation procedures that appear needed. During the dissemination phase, the treatment program (pre-tested across four phases) and the implementation intervention are tested, with primary focus on the combined adoption of both the treatment and implementation intervention.

Dissemination research poses design challenges because there are numerous variables of interest that could be experimentally manipulated for desirable effects in various settings or circumstances. For example, the identification of a dissemination strategy that works in urban agencies may not necessarily apply to rural and suburban agencies; different implementation strategies may be needed in these different locales. The researcher’s challenge during the dissemination stage is very similar to that of the marketer—namely, to have as many “end-users” as possible adopt the product. As such, dissemination research requires an integration of services and intervention research traditions (Southam-Gerow et al., 2006). In other words, the researcher must focus on the whole ecology of the treatment system, ranging from the model of treatment to the models of implementation and dissemination.

In summary, Table 1 presents a treatment development model that guides starting with small-scale safety studies to test the effects of a new treatment and ending with large-scale dissemination studies where the effectiveness of the means of disseminating the treatment is as important an outcome as that of the client. A final note about Table 1 is warranted. The linear layout suggests systematic progression through the stages. Although such an outcome is possible in theory, progress is more likely to occur “in fits and starts”. As an example, one may proceed to the effectiveness stage only to find that the evidence for the treatment program is weak. Additionally, one may have also identified some moderators of treatment that suggest adaptations of the program. It may be desirable to go back and test the adapted program at the efficacy (and in some cases safety) “level” before returning to the effectiveness level. Given the complexity of treatment development research, it is likely that similar events will occur throughout all stages, making the overall process more iterative and recursive than linear in nature.

Given the nature of the later stages of treatment development, research often necessitates input from individuals working in the communities and agencies in which the treatment is being tested. The researcher investigating the implementation of a treatment in a novel setting is well-advised not to “go it alone,” especially when seeking to disseminate mental health treatments to non-research settings. We now turn our attention to partnership models that involve stakeholders affected by the implementation of mental health innovations in community health care centers in the research process through partnerships.

Partnership Research Models

Models emphasizing partnerships between researchers and community members have existed for many years across diverse fields, with recent examples coming from education (Adelman & Taylor, 2004; Harkavy, 1998; Stein et al., 2002) and the treatment or prevention of medical illnesses (e.g., sickle cell disease, HIV/AIDS) in community settings (Harper et al., 2004; Radda, Schensul, Disch, Levy, & Reyes, 2003; Sullivan & Kelly, 2001). Relevant to mental health, the majority of partnership efforts to date have been focused on substance abuse and prevention of childhood behavior problems. Dissemination of EBTs for substance abuse has been the focus of many research enterprises (e.g., Backer, 2003; Backer, Liberman, & Kuehnel, 1986; Gotham, 2004). In recent years, there has been a growing emphasis on increasing stakeholder involvement in research, with university-community partnerships becoming more common (e.g., Backer, 2003; Lamb, Greenlick, & McCarty, 1998; Sorensen, Rawson, Guydish, & Zweben, 2003). In a similar fashion, prevention researchers have chosen partnership models to identify adaptation directions for applying prevention programs in novel settings or with an aim toward building necessary capacity for a broader dissemination of prevention services (e.g., Leff, Costigan, & Power, 2004; Spoth, Greenberg, Bierman, & Redmond, 2004).

Many scientists and policymakers have advocated university-community partnerships to promote successful and sustained dissemination, particularly once the initial research project has ended (Backer, 2003; Hoagwood, Burns, & Weisz, 2002). Because successful partnerships involve prolonged engagement with relevant stakeholders, they often result in the co-creation of a plan for the dissemination process that includes consideration of contextual issues that will impact implementation of the innovation. Because these factors may not be relevant in laboratory-based research, they are often not identified until the dissemination program is already in place, making change difficult. Furthermore, a successful partnership can help to translate the innovation (in the case of mental health, a treatment or prevention program) into the languages of the stakeholders, increasing the likelihood that the innovation will be accepted or at least understood.

Finally, partnership engagement helps to identify and cultivate stakeholders with the skills and interests needed to champion the innovation in the setting, lessening the load that the researcher must carry (Rogers, 2003). As some readers may know, a “champion” in this sense is an individual within the organization whose enthusiasm about the innovation persuades others in the organization to “adopt” or facilitate the innovation’s adoption. In the end, a successful partnership—one that involves and engages stakeholders in the work of dissemination—may lead to what has been called capacity-building within the organization. In other words, partnerships may increase the community agency’s ability to sustain the innovation through (a) increased staff expertise and (b) identification of reliable funding sources (Backer, 2003; Spoth et al, 2004).

Participatory action research

Some partnership efforts use a model called participatory action research (PAR; Jason, Keys, Suarez-Balcazar, Taylor, & Davis, 2004; Kemmis & McTaggart, 2000). PAR is a diverse set of strategies that collectively have carried many titles (e.g., action research, collaborative action research, etc.; see Kemmis & McTaggart, 2000). Basically, the goal of PAR is to empower and give voice to a group or groups of citizens with the goal of creating social action (Taylor et al., 2004). Hence, researchers engaging in PAR view stakeholders not as participants but as partners in the research process. Further, projects applying PAR are designed to create change at an organizational or systemic level. In the study we discuss shortly, we sought to implement EBTs for youth with specific problems in a large public outpatient service but not before engaging in a prolonged partnership with the agency stakeholders to co-create an adaptation of the EBTs.

It is important to consider that within PAR studies, varying degrees of stakeholder involvement are present. As described in Suarez-Balcazar et al (2004), there are several dimensions of involvement including: (a) degree of partner control over research process (ranging from no control to equal partnership in the process); (b) amount of collaboration (from minimal to highly active); and (c) degree of partner commitment (from none to full ownership of the process by the partner). Hence, at low levels of involvement, for example, a small number of stakeholders may serve as advisors to the research group. In contrast, at high levels of involvement, stakeholders are active in the planning and implementation of the research project and have ongoing input into the research.

In practice, PAR studies end up at various places along this continuum, largely driven by the goals of the project as well as practical aspects (e.g., funding limitations, time restrictions). The continuum of stakeholder involvement has implications for research planning and implies tradeoffs to which researchers are accustomed. At greater levels of stakeholder involvement, sustainability of the project is more likely, but control over its direction less certain. As stakeholder commitment and involvement decreases, researcher control increases, but sustained change becomes less likely. The project described below incorporated a moderate level of involvement, given relatively limited funding and a somewhat tight timeline.

Pros and cons of the partnership approach

Considering the positives of partnership research, one would expect many research projects to involve PAR. However, that is not the case and one of the primary reasons deserves some consideration. Partnerships similar to those previously discussed will necessarily result in the loss of investigator control over the research process. In partnership research, the project takes its direction not from the PI alone, but from the partners. Although this yields significant benefits, many researchers (and perhaps their funding sources) are unwilling to relinquish that much control. Lack of control is not a common (or comfortable) experience for many scientists. The potential for numerous concerns, including internal validity threats can easily discourage investigators from pursuing a partnership project in favor of a more traditional research endeavor.

To summarize, research that involves an active partnership between researchers and stakeholders offers many benefits but poses many challenges. In the case of our project, a central goal was to improve mental health services for children and families in community settings. We wanted our work to lead to long-standing changes that would be sustained after we left the setting. Consequently, a partnership research program that involved stakeholders and leveraged their strengths and expertise seemed the best path.

Case Study: Application of Partnership Model

We have been engaged in the Chesterfield-VCU Adaptation of Depression and Anxiety Psychological Treatments for Youth (ADAPT) project for more than three years (as of May 2008), though as we will shortly recount, our work with our partner is now more than six years old. Our aim was to form a partnership with a community-based mental health provider agency in a Richmond, VA suburb and then to adapt EBTs designed for youth with internalizing problems (i.e., anxiety and/or depression) and comorbid externalizing behavior problems for use in that setting. We all wanted an adapted treatment that (a) “fit” the context of the clinic (e.g., the problems facing the families and youth, the training and constraints of the therapists), (b) was sustainable in the setting long-term, and (c) yielded positive outcomes. ADAPT was designed as a mixed method, qualitative/quantitative research project. Furthermore, ADAPT incorporates several features of PAR that make it relatively unique among effectiveness studies. On the continuum of participation discussed above, ADAPT falls somewhere in the high-middle range, with considerable involvement and investment from multiple stakeholders at our partner clinic.

We began ADAPT with a few central assumptions. First, treatment adaptation should involve a consideration of client, provider, agency, and service system factors (Schoenwald & Hoagwood, 2001b; Southam-Gerow et al., 2006) because most EBTs are developed and tested in research clinics with very different profiles than the typical “real-world” clinic (Southam-Gerow, et al., 2003). Further, we assumed that stakeholder involvement would be critical in identifying relevant adaptations to EBTs. Although stakeholders in children’s mental health include a wide range of individuals and groups including families, providers, clinical administrators, other agency employees, agency administrators, local, state, and federal policymakers, and payor agencies (e.g., insurance companies, government programs like Medicaid), we focused on involving parents of clients served by the agency, therapists, and agency administrators (clinical, fiscal, etc.).

The literature on involving multiple stakeholders in the deployment of EBTs to children’s mental health agencies is almost non-existent. Although there is a small body of literature on providers’ attitudes toward EBTs, (Addis & Krasnow, 2000; Becker, Zayfert, & Anderson, 2004; Kazdin, Siegel, & Bass, 1990; Norcross, 1999), there is no study we found examining how to involve and learn from therapists to conduct a treatment study. Because of this paucity of previous work, qualitative research methods offered advantages as a means to generate hypotheses and ensure that a broad variety of perspectives were gleaned. ADAPT was designed to inter-lace qualitative and quantitative methods to (a) reinforce the partnership through ongoing bi-directional information flow and (b) provide a broad scientific examination of the research questions. In this next section, we briefly describe the qualitative methods in ADAPT as these elements most clearly illustrate the partnership aspects of the project. Later, we describe the quantitative studies involved in ADAPT later.

Partnership elements of the ADAPT research plan

The ADAPT project includes two formal qualitative research studies, both involving focus group methodology. In addition, ADAPT has a built-in design element inspired by PAR methodology designed to facilitate the partnership: creation of an ongoing advisory board. Finally, the project had an additional serendipitous element that, in retrospect, has roots in the qualitative research tradition. We discuss each of these briefly below, beginning with the element born of serendipity.

Prior ethnography

Many qualitative research endeavors involve what is termed “prior ethnography”, a term that refers to the process of becoming a participant observer in the situation under study for a lengthy period before the study is actually undertaken (Rodwell, 1998). A goal of prior ethnography is to reduce the researcher’s obtrusiveness. In addition, the process helps to integrate the researcher into the culture of the setting and to appreciate the nuances of the setting and how those nuances might influence those already immersed in the setting. In ADAPT, we had not intended to engage in prior ethnography. However, as the project required grant funding, we found that we were meeting regularly with therapists and administrators at the clinic while revisions of the grant were written and its funding status was decided. These meetings ranged from case consultation to informal interviewing to attending in-service trainings to discussing additional possible research collaborations. Over the course of what amounted to just over two years of collaborating before grant funding, a real sense of partnership on the project emerged.

In retrospect, this prior ethnography was also invaluable to the evolving research design. Listening to the therapists, administrators and the stories of the families, we began to understand what issues these agency stakeholders were facing and how those issues might impact the goals of ADAPT. Indeed, although the official launch of the project occurred in 2005, in actuality, the “project” has been ongoing since 2002.

Focus group study I

Our first focus group study¹ was designed to assess the perceptions and opinions of several stakeholder groups (parents, therapists, administrators) as a means of guiding (a) the adaptation of the two EBTs under study and (b) the implementation of the research process. Through these dual purposes, we sought to increase the likelihood that the adapted treatment would lead to better outcomes at our partner clinic as well as fostering the success of the research studies through high levels of involvement, participation, and cooperation across stakeholder groups. In this way, the focus groups represent a form of PAR because they serve as a means to guide the research in directions that the stakeholder groups deem relevant.

The first goal of the focus groups was to identify facilitative and obstructive factors relevant to the implementation and testing of EBTs for children and adolescents with comorbid internalizing and externalizing disorders. For example, stakeholders were asked to (a) describe ideal treatments for anxiety, depression, and conduct disorder and (b) describe barriers to implementing these ideal treatments. A major objective of all focus groups was to identify possible adaptations to the EBTs so that their applicability in the setting is increased.

Quantitative outcome studies

At the conclusion of the first focus group study, we began the first of two quantitative clinical studies testing the effects of the adapted EBT. This study, a single-case series, was designed to allow further adaptation and “tweaking” of the treatment program with a small number of cases (n = 10). The final case in that series is midway through treatment as of May 2008. Although the series is a quantitative study, we remain actively engaged in the partnership processes that have been the theme of the project. As an example, while our supervision meetings center on case discussions, as would be expected, we regularly discuss how to improve the “manual” for the larger upcoming trial. All members of the team make contributions to this “final” product that we plan to roll out in the pilot clinical trial designed to test the feasibility of the adapted treatment. After we complete the single case series, we will conduct a small-scale clinical trial using an updated, adapted version of the manual.

Advisory board

In addition to the first focus group study, we added another element to ADAPT that involves partnership-building and incorporates some PAR methodology. Early in the collaboration, we created an advisory board comprised of nominated individuals from the focus group studies that represented different stakeholder groups. The advisory board meets quarterly to consider study design issues and oversees the research projects. Including this additional element of stakeholder involvement further increases the extent of the partnership and creates a formal means for ongoing dialogue among the stakeholder groups. The VCU researchers are equal partners in the advisory board structure but are charged with implementing the research-related tasks and working with others to execute aspects of the research plan. The advisory board serves as expert members of the research team, offering advice based on the board members’ keen understanding of the contextual factors at our partner clinic.

Focus group study II

After the two clinical studies, we will conduct a second series of focus group studies as a means to assess stakeholder perceptions of the treatment program, the partnership process, and the research process. These data will serve as another way to assess the feasibility of the treatment program and the method of adaptation. As noted, a long-term goal of ADAPT is to determine the feasibility of using a partnership model to adapt treatment programs at other sites beyond our partner clinic. In the second focus group study, we plan to use some of the same questions from the first set of focus groups as well as questions pertaining to the stakeholders’ impressions on the research process.

Preliminary Findings

As noted earlier, the first series of focus group sessions and interviews have been conducted, and data collection for the single case series is nearly complete. A brief review of methods and preliminary findings from both of these studies will be discussed with an emphasis on qualitative findings and the modifications made to the treatment and implementation plan as a result of the collaboration among stakeholders and researchers.

Preliminary results of focus group study I

Potential focus group participants employed by our partner clinic (i.e., therapists providing services to families with children ages 9–14 and clinic administrators) were invited to participate through announcements in staff meetings and invitations sent via email and US mail. Clinic consumers identified as a good match for the study (i.e., families with children receiving once-weekly outpatient psychotherapy focusing on anxiety, depression, or conduct problems) were informed of the study by staff therapists and given the option of contacting study staff for more information. One group of administrators and two groups of therapists were convened. Unfortunately, however, we were unable to recruit a large enough group of interested caregiver participants at one time to conduct a focus group and instead interviewed caregivers individually. Focus group sessions and individual interviews were audio recorded, transcribed, and analyzed using a thematic analysis approach within a grounded theory perspective to identify themes relevant to treatment modification and implementation (Miles & Huberman, 1994; Strauss & Corbin, 1998).

Although data analyses are ongoing, the focus group and interview data reveal a number of salient themes relevant for treatment implementation. Among these are the expected findings that stakeholders perceive the kinds of problems in the community setting to be complex and involve not only child mental health problems, but familial and community problems as well, including parental mental health and substance use issues and child involvement across multiple government systems, including juvenile justice and drug court. A few themes that also emerged were less expected. Parents in particular expressed confusion and a lack of information about what exactly was “wrong” with their children, what the treatment plan was, and what the long-term prognosis was. Many parents and therapists indicated that information provided to parents by the service system was confusing and at times contradictory. Another theme that was heard across all stakeholder groups concerned access to mental health care. Because all stakeholder groups saw the problems facing children in the system extended beyond Axis I DSM disorders, collaboration and cooperation among service systems was viewed as an important feature of quality mental health care. However, that “cooperation” was more frequently wished for than achieved, according to stakeholders. The result was many problems in accessing mental health care and needed adjunctive services. As noted, these are preliminary findings. Once data analyses are complete, a full report of the results will be available.

Treatment adaptations

Based on the qualitative data, along with input from the advisory board, we developed a tentative treatment manual to use for the single-case series. Because space limitations preclude our describing this process in much detail, we will provide illustrations of modifications we made in response to data from the focus group and individual interviews. First, our qualitative data and ongoing consultation with therapists suggested that therapists were confused with the large number of manuals involved. At the outset and based on the literature, we envisioned three manuals, one for each problem area. However, we also saw the need to add additional treatment content. Thus, we would go beyond the three manuals but questioned how many “manuals” would we need. We decided to move away from using separate manuals and instead adopted the modular approach described by Chorpita, Daleiden, and Weisz (2005; see also Chorpita, 2007). While a full discussion of the modular approach is beyond the scope of this paper, briefly, modularity allows for individualization of treatment content to specific client problems. This is accomplished through the use of a single “manual” that is actually comprised for each client from series of modules that possess a “standardized interface” such that they can be ordered in any fashion that suits the client’s needs (Chorpita et al., 2005). How to order modules was the next challenge we faced; we describe our solution shortly. Adopting the modular approach made adding new treatment content easier, as new “content” could be added as a new module. Further, once we trained therapists how to use the overall modular approach, incorporating changes in the form of new modules was much easier than training the therapist in an entirely new manual.

An important second modification was the method for ordering the modules, and more broadly, for identifying the focus and structure of treatment in multi-problem families. As described earlier, the issue of complex problems was identified by many stakeholders in our study. Agency administrators noted that maintaining staff expertise to deal with multiple problems posed great difficulties. Therapists identified confusion and expressed feeling overwhelmed because many cases had numerous problems, all meriting treatment. Parents reported feeling unclear regarding what problems the treatment they were receiving was meant to remediate. Considering these data, we saw an emergent need for a method that would guide treatment focus throughout treatment. In the language of modularity (Chorpita et al., 2005), we needed a method to create the default modular pathway for each client. Our review of the literature suggested that the functional analytic approach used in multi-systemic therapy (MST) by Henggeler, Schoenwald, Borduin, Rowland, & Cunningham, 1998, Henggeler, Schoenwald, Rowland, & Cunningham, 2002) was well-suited to our purposes. Considerable evidence supported the MST approach in helping multi-problem youth in the juvenile justice system (e.g., Borduin et al., 1995; Henggeler, Melton, & Smith, 1992). Further, our partners were familiar with the MST model and appreciated the ecological frame that MST used. Thus, we decided to use the MST-style function analytic framework to guide treatment planning and module selection.

A final area of modification concerned the “content” of the treatment modules that would serve as our manual. Our initial plan was to rely on the modules derived from standard evidence-based protocols for the three problem areas that served as entry criteria into the clinical studies: anxiety disorders, depressive disorders, or conduct disorders. However, stakeholders identified the need for treatment content that addressed issues that were not in any of these manuals. We compiled a “laundry list” of “missing” content and then consulted the literature for possible ways to address those problems. We worked with the therapists to develop new treatment content, based on the literature and the clinicians’ experiences, and then piloted the new modules. For example, we identified the need to focus on parent-child communication and accordingly developed a module to address the problem. Another example is our creation of two assessment modules that guided therapists in their initial assessment and then later in a re-assessment. These two modules represent the formalization of the MST-style function analytic assessment process described earlier.

In short, among our modifications were (a) the use of a modular approach to treatment, (b) the guiding of that treatment via an MST-style function analytic framework, and (c) the creation of several modules that were “missing” from standard evidence-based protocols but were deemed “necessary” based on our data. These missing elements included modules on assessment and reassessment (i.e., how to conduct intakes so that module selection is simplified), emotion regulation, and parent-child communication. We made these modifications in collaboration with our advisory board and as a result of our qualitative research process that involved a wide range of stakeholders.

Clinical studies

The second phase of ADAPT involves two clinical studies, a single case series and a small pilot trial. These studies will test the effects of the adapted treatment. The single case series is nearly complete. In that study, families with children between the ages of 9 and 14 who presented to the clinic with a combination of internalizing and externalizing symptoms were invited to participate. After an initial assessment confirming the diagnosis of at least one anxiety or depressive disorder and the presence of disruptive behavior disorder symptoms, families were randomly assigned to a waitlist of three to five weeks before their first therapy session during which time weekly baseline assessments were administered. Assessments were administered every six weeks during active psychological treatment and every six weeks after the termination of treatment until a total of ten assessments (including those during baseline) had been completed. At the time of publication, six youth had completed the single case series and one remained in treatment. Preliminary analyses of pre- and post-treatment measures of clinical symptoms by multiple raters reveal support for the effectiveness of the treatment in community clinic settings. According to both self-report and parent-report measures, all six youth have experienced a reduction in symptom severity from baseline to post-treatment, with all youth reporting no mood or anxiety symptoms in the clinical range at post-treatment. Analysis of parent report revealed five of six parents reported their child did not meet criteria for a mood or anxiety disorder at follow-up. The remaining parent reported that the child no longer met criteria for GAD at follow up but continued to meet criteria for Social Anxiety Disorder and Specific Phobia. Two youth retained their prior diagnoses of ADHD.

Conclusion

The ADAPT project applies mixed qualitative and quantitative research methods and a participatory action research (PAR) approach in an effort to foster sustainable dissemination of EBTs in non-research community mental health settings. Because such field studies typically involve a high degree of commitment and work from the stakeholders, we selected these methodologies, with their emphases on partnership-building, to facilitate the prolonged engagement and joint ownership needed to maximize the chances that the dissemination would be both successful and sustained. The initial qualitative study has led to what are potentially important adaptations to the evidence-based treatment approach we planned to use in the setting. Preliminary data suggest that these changes have led to good outcomes for children and families receiving the services. The longer-term hope is that these studies will also yield a modified treatment program and the outline of an implementation model that can be tested in future studies and in other settings. Ideally, we hope to be able to identify potential principles for implementation and dissemination of mental health treatments, principles that can be tested and further refined in other settings. Thus, whereas ADAPT considers local concerns as paramount, the project was designed to help the field “think globally” as well.

The science-practice gap in children’s mental health has been a long-pondered problem. We have described that one possible way to remedy the gap may lie in partnerships with community service-providing agencies and their stakeholders. This path is certainly not the only feasible way to close the gap nor is it one that will necessarily result in the successful and sustained implementation of EBTs. Furthermore, even if ADAPT is a successful project and leads to considerable progress in closing the gap, children’s mental health problems will remain a major public health concern. Indeed, although the research in which we are engaged moves us away from the lab, we still strongly advocate the importance of a continued focus on “lab-based” clinical research, particularly as advocated by Kazdin (2003). Our current evidence-base is clearly lacking insofar as there are many problems for which there are no identified EBTs. Furthermore, even treatments with strong empirical support typically are not effective for as many 40–45% of those treated. Finally, we are still only beginning to understand the mechanisms of change in the treatments we use. Clearly, then, there is work to do on all fronts to help families and children with mental health problems. It is our hope and belief that partnership-based projects like ADAPT will contribute to the work we all are doing: improving mental health services for children and their families.