Cultural Adaptation and Implementation of Evidence-Based Parent-Training: A Systematic Review and Critique of Guiding Evidence

Abstract

With advances in knowledge regarding efficacious evidence-based interventions, there have been significant attempts to culturally adapt, implement, and disseminate parent training interventions broadly, especially across ethnic and cultural groups. We sought to examine the extent to which researchers and developers of evidence-based parent training programs have used cultural adaptation models, tested implementation strategies, and evaluated implementation outcomes when integrating the interventions into routine care by conducting a systematic review of the literature for four evidence-based parent training interventions: Parent-Child Interaction Therapy (PCIT), The Incredible Years (IY), Parent Management Training-Oregon Model (PMTO™), and the Positive Parenting Program (Triple P). A total of 610 articles across the four programs were identified. Of those, only eight documented a rigorous cultural adaptation process, and only two sought to test the effectiveness of implementation strategies by using rigorous research designs. Our findings suggest that there is much work to be done to move parent-training intervention research towards a more rigorous examination of cultural adaptation and implementation practices.

Child disruptive behavior is a public health concern and costly issue in the U.S. (Honeycutt, Khavjou, Jones, Cuellar, & Forehand, 2013) and, if left untreated, can lead to delinquency later in life (Fergusson, Horwood, & Ridder, 2004). With the assumption that parents' behaviors mediate children behavior, parent training programs have been created to prevent and/or intervene on child disruptive behavior (e.g., Hagen, Ogden, & Bjørnebekk, 2011; Honeycutt et al., 2013; Presnal, Webster-Stratton, & Constantino, 2014). Considering that there are now a number of evidence-based parent training programs that could be readily implemented in community settings (Substance Abuse and Mental Health Services Administration, 2012; The California Evidence-Based Clearinghouse for Child Welfare, 2012), it should follow that parent training programs are disseminated and implemented in usual care¹ to prevent and intervene on child disruptive behavior. However, evidence-based care is still the exception rather than the rule in usual care settings serving children, youth, and families (Kohl, Schurer, & Bellamy, 2009; Raghavan, Inoue, Ettner, & Hamilton, 2010). Indeed, an evaluation of parent-training programs in one midsized Midwestern city revealed that only about 11% of agencies had adopted evidence-based programs (Kohl et al., 2009). The low rates at which evidence-based parenting interventions are delivered suggests that simply publishing reports on their availability and effectiveness, while necessary, is not sufficient given the myriad of barriers at the client, clinician, team, organizational, policy, and funding-levels (e.g., Flottorp et al., 2013; Powell, Hausmann-Stablile, & McMillen, 2013). This signals a need to study the implementation of evidence-based parenting interventions, and to evaluate strategies that can facilitate the uptake of such interventions in usual care.

The research-to-practice gap is even larger for racial and minority groups. Compared to non-Hispanic Whites, racial and ethnic minority groups in the U.S. tend to underutilize mental health services, to discontinue treatment prematurely, and to receive poor care (Institute of Medicine, 2003, 2009; United States Department of Health and Human Services, 2001). Even with comparable insurance, needs, attitudes toward treatment, and beliefs about treatments, African Americans and Latinos are less likely than their European counterparts to use mental health services (Alegría et al, 2008; Chow, Jaffee, & Snowden, 2003). As Kazdin and Blase (2013) articulate, the lack of services for most people in need has direct implications for models of treatment delivery. The current methodology to provide mental health services has not been successful in improving mental health in the U.S.; a shift and expansion in intervention research and practice is needed to be able to prevent and treat mental illness and decrease health and mental health disparities (Kazdin & Blase, 2013). A powerful solution may be found in bringing together the fields of cultural adaptation research and implementation research. In the cultural adaptation, wherein program modifications are intended to increase the fit of the intervention to the target population while protecting scientific integrity (Castro, Barrera, & Martinez, 2004; Kumpfer et al., 2002). In implementation research, methods aim to promote the systematic uptake of research findings and evidence-based practices into routine practice (Eccles & Mitttman, 2006). In combination, these can be sources for thinking about designs and methodologies that can accelerate the spreading of evidence-based prevention and intervention for those in need. In fact, the Healthy People 2020 report challenges researchers and practitioners to eliminate disparities and improve the health of all groups (United States Department of Health and Human Services, & Office of Disease Prevention and Health Promotion, 2012). To achieve such a goal, scholars tend to focus on adapting evidence-based interventions to clients' culture with the premise that, to be effective, an intervention should be responsive to the cultural practices and worldview of the target population (Domenech Rodríguez & Bernal, 2012). Similarly, implementation researchers aim to promote the systematic uptake of evidence-based interventions to improve the quality and effectiveness of health services (Eccles & Mittman, 2006).”

The Interventions

This review focuses on four parent training interventions that have been given the highest possible rating as “well supported by research evidence” by the California Evidence-Based Clearinghouse for Child Welfare (http://www.cebc4cw.org): Parent-Child Interaction Therapy (PCIT), The Incredible Years (IY), Parent Management Training-Oregon Model (PMTO^R), and the Positive Parenting Program (Triple P). Our sample was selected from the list of 22 interventions indicated by SAMSHA's National Registry of Evidence-based Programs and Practices (NREPP) to have focused on mental health promotion and treatment in early childhood, to have been funded by the National Institutes of Health, and have been evaluated in comparative effectiveness research studies. From there, we selected our sample based on the ratings from the California Evidence-Based Clearinghouse, as programs that have (a) no case data suggesting a risk of harm; (b) a well-defined treatment manual and strong empirical evidence demonstrating their ability to change parenting behaviors and reduce child behavior problems; and (c) demonstrated efficacy across a variety of populations and in multiple settings. Triple P is a continuum of parent support and training (Sanders, Markie-Dadds, & Turner, 2003), so we focus here on Level 4 Triple P, which is most comparable to the other interventions under consideration.

Cultural Adaptation of Evidence-Based Parenting Interventions

There are several types of intervention adaptation (e.g., adapting the training, the agency; Stirman, Miller, Toder, & Calloway, 2013). We focus here on cultural adaptation, defined as “the systematic modification of an evidence-based treatment (EBT) to consider language, culture, and context in such a way that it is compatible with the client's cultural patterns meanings and values” (Bernal, Jimenez-Chafey, & Domenech Rodríguez, 2009, p. 362). Cultural adaptation is an important part of the implementation process (Cabassa & Baumann, 2013), and considering the current diversity of the U.S. population, attention to how cultural factors affect the implementation of parent trainings in usual care is crucial.

Meta-analyses in the cultural adaptation field have indicated that adapting interventions to clients' cultural backgrounds by explicitly integrating cultural factors such as language, cultural beliefs, and explanatory models into the intervention improves the relevance, acceptability, effectiveness, and sustainability of the intervention by the providers and target populations (e.g., Benish, Quintana, & Wampold, 2011; Huey & Polo, 2008; Griner & Smith, 2006; Smith, Domenech Rodríguez, & Bernal, 2011). Care should be taken, however, as these meta-analyses reflect great variability in effect sizes, study designs, populations, and interventions sampled (Cabassa & Baumann, 2013).

There is no single, correct way to culturally adapt interventions and there is no rule that states that every EBT should be adapted (Domenech Rodríguez & Bernal, 2012). When considering whether to culturally adapt an intervention, one should carefully consider what evidence about the intervention is available (e.g., what information does the literature provide about the EBT?), the target population (e.g., who was the original target population? To whom will the intervention be delivered?), and what is the target domain of the intervention (e.g., changing parenting practices). Domenech Rodríguez and Bernal (2012) provide guidelines to support the decision of whether to adapt an intervention, which involve assessing whether (a) the EBT is accessible to the providers who will be delivering the intervention, (b) the underlying mechanism of change of the intervention is a good fit for the target population, and (c) the EBT is acceptable for the target population. If the decision is to culturally adapt an intervention, the next step is to decide which framework will guide the process.

There are two sets of cultural adaptation frameworks: those that inform modification to the content of the intervention and those that inform the process of adaptation (Ferrer-Wreder, Snudell, & Mansoory, 2012). One model that informs what to adapt in the delivery and content of the intervention is the Ecological Validity Model (EVM) by Bernal, Bonilla, and Bellido (1995). The EVM specifies eight domains: language, persons, metaphors, content, concepts, goals, methods, and context. Another content model is the cultural sensitivity model, which distinguishes deep versus surface adaptations (Resnicow, Soler, Braithwaite, Ahluwailia, & Butler, 2000).

The second set of frameworks focus on the process of adaptation, where decisions about when to adapt, how to adapt, and which stakeholders should be involved in the process are outlined. A number of frameworks fall into this category and vary in how prescriptive (i.e., have a set of a priori steps that guide the process) or specific (i.e., focused on the adaptation of one specific EBT) they are (Ferrer-Wreder et al., 2012). Several of them have been described elsewhere (Bernal & Domenech Rodríguez, 2012). Generally, these models recommend adaptations to be informed by the expertise of stakeholders, use formative research methods, and conduct formal evaluations of the adapted intervention (Cabassa & Baumann, 2013; Domenech Rodríguez & Bernal, 2012). It is important to assess the extent to which adaptations to parent training interventions have been guided by cultural adaptation frameworks in order to document the types of adaptation that are being conducted and examine how these adaptations impact intervention and implementation outcomes.

Implementation of Evidence-Based Parent Training Interventions

Implementation refers to the process of integrating the intervention within a setting (Rabin & Brownson, 2012). It extends efficacy and effectiveness research that focuses on discovering what works to understanding how the implementation works in specific contexts (Damschroder, Peikes, & Peterson, 2013). This paper focuses on empirical tests of strategies used to implement the aforementioned evidence-based parent training programs. We define an implementation strategy as a “systematic intervention process to adopt and integrate evidence-based health innovations into usual care” (Powell et al., 2012, p. 124). The literature reflects a wide range of different implementation strategies (Powell et al., 2012); yet, the evidence to support the use of specific implementation strategies in mental health and social service settings has lagged behind other fields such as nursing and medicine (Powell, Proctor, & Glass, 2013). The recent prioritization of implementation research by the National Institutes of Health (2013) and the Institute of Medicine (2009) will undoubtedly increase the number of empirical studies testing innovative approaches to implementation. Developing a robust evidence-base for specific implementation strategies and learning more about how they interact with contextual elements of the settings in which they are deployed will give implementers the tools they need to improve the quality of care in social service settings. In other words, the testing of implementation strategies will attempt to answer the question set forth by Asgary-Eden and Lee (2011): “So now we've picked an evidence-based program, what's next?” (p. 169).

Empirical tests of implementation strategies should be guided by available theories, conceptual models, and/or frameworks, as they can ensure that essential contextual and process elements related to implementation are not overlooked (Proctor, Powell, Baumann, Hamilton, & Santens, 2012; Tabak, Khoong, Chambers, & Brownson, 2012). Moreover, frameworks provide a systematic way of evaluating the interventions and facilitating replication of the implementation process in different settings. While there are over 60 dissemination and implementation frameworks and models (Tabak et al., 2012), evidence from healthcare suggests that theory is drastically underutilized in implementation studies (Colquhoun et al., 2013).

It is also critical that implementation studies evaluate implementation outcomes. Proctor and colleagues (Proctor et al., 2011; Proctor & Brownson, 2012) have suggested a taxonomy of implementation outcomes, including acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. Evaluation of implementation outcomes can help investigators disentangle implementation effectiveness from treatment effectiveness and to know, for example, if an intervention failed because it was ineffective or it was implemented incorrectly (Proctor et al., 2011). Furthermore, assessing implementation outcomes may improve our understanding of which implementation strategies work best with given interventions, settings, and conditions.

Purpose

Cultural adaptation and implementation processes are inextricably linked and both processes should be planned and documented to facilitate future replication (Cabassa & Baumann, 2013). The purpose of this article is to review and assess the literature to determine the extent to which studies of evidence-based parent training programs have (a) used cultural adaptation models to guide the adaptation process, (b) tested implementation strategies, and (c) evaluated implementation outcomes.

Methods

Inclusion and Exclusion Criteria

Cultural adaptation studies

Our review includes empirical studies of cultural adaptation that explicitly accounted for clients' culture, ethnicity or race, and that used and report data from experimental designs (e.g., RCTs). We excluded studies that simply translated the materials into a different language and those that tested diagnosis-specific adaptations (e.g., adaptations for populations with autism).

Implementation studies

We included implementation studies that were empirical and utilized comparative designs that tested the effectiveness of an implementation strategy or set of strategies. To be included, studies also had to meet the standards of rigor set forth by the Cochrane Effective Practice and Organisation of Care Group (2002), which specifies four research designs: (a) randomized controlled trials, (b) controlled clinical trials, (c) interrupted time series, and (d) controlled before and after studies. This inclusion criteria has been used in other reviews of implementation studies in social service settings (e.g., Landsverk et al., 2011).

Search Strategy

In order to locate empirical papers meeting our inclusion criteria we used a four-step process that included (a) a database search, (b) a search of the authors' personal libraries, (c) an examination of the website for each intervention, and (d) a query to treatment developers to identify articles that we may have missed. For the database search, we took an inclusive approach in generating search terms, using multiple variants of the four parent training programs: (a) “Parent-Child Interaction Therapy” OR PCIT OR “Parent Child Interaction Therapy”, (b) “Incredible Years”, (c) “PMTO” OR “Parent Management Training-Oregon Model” OR “Parent Management Training”, and (d) “Positive Parenting Program” OR “Triple P”.

We searched the following databases: PsycINFO, Medline/Pubmed, CINAHL, and SocIndex in January of 2013. After articles from the first three phases of the search strategy were coded, we emailed the treatment developers or spoke with members of their team to ask whether we had identified all of the published articles that were potentially relevant to our study objectives.

Selection of Studies

Five authors reviewed the articles; each was the primary coder for one intervention (two authors reviewed Incredible Years). After potentially relevant articles were identified, full-text review was completed by at least two authors. Any discrepancies were handled through discussion until consensus was reached.

Data Extraction and Coding Procedures

Coding for cultural adaptation

We coded two aspects of the adaptation process. To evaluate what adaptations had been made to meet the needs of particular cultural groups, we used the eight categories of the EVM framework: language, persons, metaphors, content, concepts, goals, methods, and context (Bernal et al., 1995). To assess how the adaptation was made, we extracted any descriptions of the process of adaptation from articles that fit our criteria as well as from articles that solely described the process of adaptation.

Coding for implementation

For the implementation studies, we coded study designs, implementation strategies, implementation outcomes, and whether or not a conceptual model or framework was used to guide the study and/or implementation effort. For the implementation outcomes, we relied upon Proctor and colleagues' (2011) taxonomy that includes: acceptability, adoption, appropriateness, cost, feasibility, fidelity, penetration, and sustainability. To assess the implementation process, we extracted descriptions from articles that fit our criteria as well as from articles that described the implementation process.

Data Synthesis

Given the small number of studies that met our inclusion criteria we relied primarily upon tabulation and narrative summaries to describe our results.

Results

The database search yielded a total of 67 articles related to Triple P, 137 articles related to PCIT, 72 articles related to Incredible Years, and 62 articles related to PMTO. Additional articles were added from the authors' personal libraries, intervention websites, and intervention developer queries, yielding a total of 106 articles for Triple P, 192 articles for PCIT, 209 articles for IY, and 103 articles for PMTO.

Cultural Adaptation Articles

Only one article from Triple P, three articles from PCIT, and four articles from PMTO met our criteria. No articles from IY fit our criteria. The results of our coding about cultural adaptation are shown in Table 1. Two interventions were adapted to be delivered to Latinos (Martinez & Eddy, 2005; Parra Cardona et al., 2012), including Puerto Rican families (Matos, Bauermeister, & Bernal, 2009; Matos, Torres, Santiago, Jurado, & Rodriguez, 2006). Two articles reported adaptations for use with indigenous populations. Triple P was adapted for use with Aboriginal and Torres Strait Islander families in Australia (Turner, Richards & Sanders, 2007). Another article reported adaptation made to PMTO for use with Somalis and Pakistanis in Norway (Bjørkness & Manger, 2013). McCabe and colleagues' articles (McCabe & Yeh, 2009; McCabe, Yeh, Lau, & Argote, 2012) reported the intervention of GANA, a PCIT intervention adapted for Mexican American families. The adaptation of GANA was described in McCabe, Yeh, Garland, Lau, and Chavez (2005).

Table 1. Results of the Coding for Cultural Adaptation Studies.

	Triple P	PCIT	IY	PMTO	Total
Total #	106	192	209	103	610
Cultural adaptation articles	1	3	0	3	8
EVM categories
Language	1	3		2	8
Content		2		2	6
Persons		1		2	5
Metaphors	1	2			5
Methods	1	3		1	7
Concepts		1		1	4
Goals		1		1	3
Context

All articles reported changes in language with the exception of Parra Cardona et al. (2012), because the authors were adapting a manual that had already been translated to Spanish. All but Parra Cardona and colleagues (2012) and McCabe et al. (2005) described changes in the persons category, such as the inclusion professionals serving this population and parents (Turner et al., 2007; Turner & Sanders, 2006) and the homogenization of gender in the parenting groups (Bjørkness & Manger, 2013). Changes in metaphors included insertion of idiomatic expressions, as well as changing the name of the program and pictures of the manuals to fit the needs of the target population (Matos et al., 2006; McCabe et al., 2005; Turner et al., 2007).

Changes in content were reported in all studies, including sessions with content about biculturalism (Parra Cardona et al., 2012). Context and service delivery methods were also altered for Triple P in that group sessions were restructured to “allow time to discuss the social and political context for parenting, develop trust, slow the pace of presentation, and share personal stories” (Turner et al., 2007, p. 5).

One of the most common changes in concept was the expansion of family to include other family members besides parents and children, and changes in goals to increase the fit of the intervention for the target population (e.g., framing the intervention as an educational and skill-building intervention; McCabe et al., 2005). All but Matos et al. (2006), Parra Cardona et al. (2012) and Martinez and Eddy (2005) reported changes in method, including increases in session time (McCabe et al., 2005). Finally, Matos et al. (2009), Matos et al. (2006), and McCabe et al. (2005) reported changes such as giving additional time for parents to build rapport with therapists, adapting the data collection process to include meals and child care, as well as explicitly mentioning issues regarding immigration aspects during the intervention.

One of our questions was whether the articles identified a framework for informing the adaptation, specifically to inform what about the intervention should be adapted and how that adaptation should be made. Only Matos and colleagues (2006) explicitly mentioned a framework that guided their process of content adaptation: Bernal and colleagues' (1995) Ecological Validity Model.

More authors reported frameworks that guided their process of adaptation. Turner et al. (2007) reported that the process of adaptation was made with extensive community consultation, but the authors do not give specific details of the steps taken to establish collaboration between researchers and community members. McCabe et al. (2005) mentioned that they used a mixed method design to adapt the intervention, and Matos et al. (2006) report that they based their adaptation process on Rounsaville, Carroll, and Onken (2001) stage model of behavioral therapies. Domenech Rodríguez et al. (2011) used the Cultural Adaptation Process model (Domenech-Rodríguez & Wieling, 2004) to inform their adaptation process of PMTO. Parra-Cardona and colleagues (2011) adapted PMTO using community-based participatory research principles, where the adaptation involved translation and cultural adaptation of the materials, qualitative study of the community, and a test of the adapted intervention. Finally, no framework was reported by Bjørkness & Manger (2013) about the process of adaption of PMTO for Somalis and Pakistanis in Norway or by Matos et al. (2009), who adapted PCIT for Puerto Rican preschool children with ADHD and behavior problems.

Some lessons can be learned about the cultural adaptation process from studies that did not fit our criteria but that reported the adaptation process of the parent trainings. While the emphasis on the following components varied, in general the adaptation of the parent trainings involved: (a) review of the core components of the EBT; (b) survey of relevant information from the clinical literature, from the empirical literature, expert opinion (cultural adaptation experts), and qualitative data collected from target population and potential providers to assess the fit of the EBT with target population; (c) preliminary adaptation of EBT manual and training components; (d) feedback from consultants (cultural adaptation experts), practitioners, community members, and treatment developer and his/her team; (e) in-depth interviews with parents and providers and/or evaluation of data for further refinement of the intervention (e.g., Bigfoot & Funderbrook, 2001; Dionne, Davis, Sheeber, & Madrigal, 2009; Domenech Rodríguez et al., 2011; Matos et al., 2006; McCabe et al., 2005).

Table 2 shows the target population, the design and comparison groups and the main results of these studies. Overall, adapted interventions are associated with great retention and satisfaction. Data also indicate the feasibility and effectiveness of the adapted interventions as compared to control groups or waiting list.

Table 2.

Main Results of the Cultural Adaptation Studies.

Study	Intervention	Target population	Design and comparison groups	Results
Turner, Richards & Sanders, 2007	Triple P	Indigenous families living in Australia	Repeated measures randomized group; culturally sensitive Triple P with waiting list	Good retention rate of families in intervention and high satisfaction by families; small drop out of families in waiting list prior to engaging them in intervention. Decrease in child problem behavior; improvement parenting practices (decrease on parent verbosity).
Matos, Bauermeister, & Bernal, 2009	PCIT	Puerto Ricans families living in Puerto Rico	Repeated measures randomized group; adapted PCIT with waiting list	Significant reduction on mother's depression and child outcome (hyperactivity, inattention, aggression and oppositional behaviors), and increase in the use of adequate parenting practices on the intervention group.
Matos, Torres, Santiago, Jurado, & Rodriguez, 2006	PCIT	Puerto Ricans famlies living in Puerto Rico	Pre-post assessment; single group	High retention rates and satisfaction level from parents. Significant reduction in children‘s’ externalizing problems, reduction of parenting stress, and improvement in parenting practices.
McCabe & Yeh (2009)	PCIT	Mexican American families living in the US	GANA (PCIT adapted), standard PCIT or treatment as usual	Both GANA and PCIT produced significant decrease in child externalizing behavior, increase in parent practices, and decrease in parent distress TAU. GANA was not superior to PCIT.
Martinez & Eddy, 2005	PMTO	Latino families in the U.S. (50% target children were U.S.-born, 50% were foreign born)	RCT; PMTO vs. No intervention	High retention rates; high satisfaction from parents and strong support for group intervention. Significant improvement in parenting practices (general parenting, skill encouragement, overall effective parenting), as well as significant decrease in youth aggression, externalizing behaviors, likelihood of smoking and use of alcohol, marijuana and other drugs.
Parra Cardona et al., 2012	PMTO	Latino families living in the US	RCT; CAPAS-Enhanced (with two extra sessions about bicultural experience) vs. CAPAS-Original (PMTO translated to Spanish)	High engagement, retention and parent satisfaction in both arms. Participants from CAPAS-Enhanced emphasized the importance of devoting time to reflect about cultural themes.
Bjørkness & Manger, 2013	PMTO	Muslim families from Somalia and Pakistan living in Norway	RCT; adapted PMTO and waiting list	Moderate retention rate; high satisfaction. Improvement in parenting practices (decrease in hash discipline and increase in positive parenting), and reduction on child behavior problem as reported by mothers. No difference in child behavior as reported by teachers.

Implementation

Out of the 610, only two (0.32%) studies from PCIT met our criteria. Chaffin et al. (2009) manipulated the presence of motivational interviewing aimed at improving parenting retention in one of the PCIT studies implemented in child welfare with parents of children between 2.5 and 12 years of age. A double-randomized design was employed in which participants were first randomized to either the standard orientation or to the self-motivation orientation, and then were again randomized to either PCIT or to a standard didactic group. Their results showed that motivational interviewing improved retention only when combined with PCIT, but only for low to moderately motivated child welfare clients. Herschell and colleagues (2009) evaluated whether simply giving the manual to therapists would be sufficient to master PCIT knowledge and skills. They also tested whether either didactic or experiential training was more effective in improving a number of outcomes, including therapists' attitudes toward treatment manuals, skill acquisition, knowledge gain, and satisfaction with the intervention. Their results show that reading a treatment manual increased both knowledge and skills related to PCIT; however, was not sufficient for clinicians to obtain mastery. Both the experiential and didactic (control) group improved clinicians' attitudes toward manualized treatments, PCIT knowledge and skill, and satisfaction with the training; however, few clinicians reached mastery after the two-day training.

Several articles that did not fit our criteria described the experiences of implementing parent trainings. Some interventions did follow a framework (e.g., Bekkema, Wiefferink, & Mikolajczak, 2008; Ogden, Hagen, Askeland, & Christensen, 2009), but the majority of the authors simply described the steps taken to implement the interventions: (a) standardization of treatment delivery via manuals; (c) standardization of training for providers delivering the intervention; and (d) ongoing fidelity monitoring. All authors highlighted the importance of peer support amongst providers during training, certification of providers, and constant supervision for fidelity checking. Fidelity was assessed during training and up to certification, and at different times after certification to ensure maintenance and quality of intervention delivery. Moreover, authors unanimously mentioned that sustainability and success of the intervention involved agency support, which includes acceptance of the EBT, integrating EBT with caseloads, and procuring ongoing funding (e.g., Webster-Stratton, 2009; Ogden et al., 2009; Turner & Sanders, 2006).

Discussion

The goals of this review were to determine the extent to which researchers and developers of evidence-based parent training programs (a) used cultural adaptation models, (b) tested implementation strategies, and (c) evaluated implementation outcomes. Our results were discouraging. Out of 610 published reports, only eight studies fit our criteria for cultural adaptation studies and two fit our criteria for implementation studies.

We used the Ecological Validity Model (EVM; Bernal et al., 1995) as a framework to code the cultural adaptation of content of the parent trainings as it provides researchers a comprehensive list of potential targets of cultural adaptation and identifies critical areas where cultural factors can play a role in psychosocial interventions. Another advantage of EVM is that it helps identify surface and deep level adaptations (Resnicow et al., 2000). Surface modifications, such as translation of the materials, increase the feasibility of the program. Deep modifications, such as changes in methods and content, enhance the program impact (Resnicow et al., 2000). Oftentimes the translation of the materials (surface modifications) could be sufficient to produce impact of the culturally adapted program but we advocate for these to be carefully thought and planned (Cabassa & Baumann, 2013).

There is, however, debate regarding deep adaptations. Some scholars advocate for a rationale for cultural adaptations prior to adapting EBTs given concerns regarding an absence of data that confirms that such adaptations are needed, as well as concerns regarding feasibility, cost, and other resources needed to properly test the effectiveness of adapted treatments (Domenech Rodríguez & Bernal, 2012). Others also advocate that making adaptations: (1) delays implementation/delivery of the intervention unnecessarily (while adaptations are made); (2) is based on a potentially false assumption that people will not benefit from an intervention; (2) results in delivering an untested form of an intervention (National Advisory Mental Health Council's Workgroup, 2010).

However, arguments have also been advanced on the need to assume that culture must be taken into account on the basis of ethical and professional standards on the premise that adaptations are not needed when they, in fact, could require researchers to conduct research that harms vulnerable (e.g., marginalized) individuals (Domenech Rodríguez & Bernal, 2012). Moreover, as expressed in the Surgeon's General report ([USDHHS], 2001), ‘culture counts’ in mental health care, as it shapes how people seek help and engage in health behaviors and shapes how providers communicate with clients and deliver services. Considering that the population of ethnic minorities in the U.S. is projected to be 50% of the U.S. population by 2060 (Colby & Ortman, 2015), and that service delivery in its current form has not been efficient in reaching those in need, a careful assessment of the methodologies currently being used to provide service delivery may be important so as to “shift” current thinking and promote greater reach to those in need (Kazdin & Blasé, 2011). Implementation science, with methodology that aims to promote the spread of EBTs can be a powerful resource for helping accelerate the uptake of evidence-based parent interventions in usual care.

One problem when deciding whether or not to adapt a parent training is that few clinical trials have included a sufficient number of ethnic minority families to permit generalization across cultures (McCabe et al., 2005), and the gap in utilization of EBTs by minority populations still exists (Institute of Medicine, 2009). Moreover, the field of cultural adaptation still needs to address questions such as “what (if any) are the adaptations necessary to achieve cultural relevance and treatment efficacy?” (italics added) and “What are the most relevant procedures that should be undertaken in any process of cultural adaptation?” (Parra Cardona et al., 2012, p.3). While the answers to these questions are not straightforward, one methodology to answer these questions is to use differential research designs. This would entail testing two differentially culturally adapted versions of an existing EBT to examine their efficacy, feasibility and cultural acceptability (Martinez & Eddy, 2005; Parra Cardona et al., 2012). An alternative way to address whether or not to adapt an intervention is to bridge the fields of cultural adaptation and implementation whereby the evaluation of whether, when, and how to adapt a parent training would be part of the implementation process (Cabassa & Baumann, 2013; Chambers, Glasgow, & Stange, 2013). To do so, one should carefully choose the frameworks that would guide the process.

Our review revealed that few scholars were using cultural adaptation frameworks. Cultural adaptation frameworks are important to promote effective implementation research as it helps maintain high fidelity and avoid decrements in intervention impact (Allen et al., 2012). We advocate for the explicit description of what has been adapted, why it was adapted, and how it was adapted. The documentation of the process is important to inform stakeholders on how to (a) identify the diversity of adaptations being made; (b) examine the impact of the intervention on different populations; (c) help provide technical assistance to community providers, and (d) scale up programs (Stirman et al., 2013).

While no IY study fit our criteria, we note that there is a large amount of work by Webster-Stratton and her team to implement IY with minority groups (Webster-Stratton, 2009; Reid, Webster-Stratton, & Beauchaine, 2001). Cultural adaptation efforts for multicultural groups may be difficult to implement as the very pluralism of the group creates challenges for specific tailoring efforts. Future research could provide clarity on what kinds of adaptations may be made in multicultural contexts to address the diversity within groups.

The surprisingly low number of studies (n = 2) that fit our criteria of implementation indicates that the field of parent training, to date, has focused primarily on determining the effectiveness of the interventions. That is not to say that investigators are not assessing implementation outcomes. For example, other types of articles described findings on fidelity (e.g., Sigmarsdóttir & Gudmundsdóttir, 2013) or cost effectiveness (e.g., Foster, Olchowski, & Webster-Stratton, 2004). It seems, however, that the field is poised to engage in more rigorous research involving comparative tests of implementation strategies and evaluating implementation outcomes.

Our review has several limitations. First, there was a discrepancy between the numbers or articles from database search and from our personal libraries and inquiries to treatment developers and their team. Some of the articles that were captured from our personal libraries and/or from indications from the treatment developers did not contain the names of the interventions in the titles, and one intervention had several articles on their website that were also indexed by database. This poses a challenge for database searches; thus, we call for scholars to carefully consider titles, key words, and abstracts when publishing study results.

Second, our criteria could have been too stringent considering current state of research on parenting intervention. For example, articles such as those using observational or case study designs were excluded. An analysis of the excluded articles may provide an interesting picture of the adaptation and implementation work being done in the field. For example, while Domenech Rodríguez et al. (2011) provide a detailed description of how they used the EVM model to adapt the PMTO intervention,; their study was excluded from our analyses because they did not include data from the randomized controlled trial. Third, we were also stringent on our selection of parent trainings. Future studies could assess whether other interventions would provide a different picture of the cultural adaptation and the implementation process. Finally, while our team reached consensus about articles to include in this review, others may have a different perspective about our selection criteria. In terms of treatment adaptations, we focused on cultural adaptation. However, we recognize that other types of adaptations may be necessary during implementation in order to fit the intervention to the setting, such as adaptions in context, training, and modifications at the agency level and system level (Stirman et al., 2013).

In summary, our data indicate that the field of prevention science is ripe for more studies documenting the content and process of cultural adaptations and empirically testing different approaches to implementation. As the Institute of Medicine report (2009) points out: much research is needed to identify essential aspects of implementation, especially when implementing interventions whose evidence may be “significantly affected or impeded by aspects of the ethnic, linguistic, and cultural environment in which they are implemented” (p.4). In short, efforts to implement evidence-based parent training programs must be complemented with evidence-based approaches to cultural adaptation if they are to be adopted, implemented, sustained, and scaled-up in community settings (Cabassa & Baumann, 2013).

Highlights.

• We examined the extent to which researchers and developers of evidence-based parent training programs have used cultural adaptation models, tested implementation strategies, and evaluated implementation outcomes

• Four evidence-based parent interventions

• Much work to be done to move parent-training intervention research towards a more rigorous examination of cultural adaptation and implementation practices