Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Aug 8.
Published in final edited form as: Child Youth Serv Rev. 2014 Apr;39:160–168. doi: 10.1016/j.childyouth.2013.10.005

Interagency Collaborative Team Model for Capacity Building to Scale-Up Evidence-Based Practice

Michael Hurlburt 1,2,*, Gregory A Aarons 2,3, Danielle Fettes 2,3, Cathleen Willging 4, Lara Gunderson 4, Mark J Chaffin 5
PMCID: PMC4976939  NIHMSID: NIHMS531388  PMID: 27512239

Abstract

Background

System-wide scale up of evidence-based practice (EBP) is a complex process. Yet, few strategic approaches exist to support EBP implementation and sustainment across a service system. Building on the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework, we developed and are testing the Interagency Collaborative Team (ICT) process model to implement an evidence-based child neglect intervention (i.e., SafeCare®) within a large children’s service system. The ICT model emphasizes the role of local agency collaborations in creating structural supports for successful implementation.

Methods

We describe the ICT model and present preliminary qualitative results from use of the implementation model in one large scale EBP implementation. Qualitative interviews were conducted to assess challenges in building system, organization, and home visitor collaboration and capacity to implement the EBP. Data collection and analysis centered on EBP implementation issues, as well as the experiences of home visitors under the ICT model.

Results

Six notable issues relating to implementation process emerged from participant interviews, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes. These issues highlight strengths and areas for development in the ICT model.

Conclusions

Use of the ICT model led to sustained and widespread use of SafeCare in one large county. Although some aspects of the implementation model may benefit from enhancement, qualitative findings suggest that the ICT process generates strong structural supports for implementation and creates conditions in which tensions between EBP structure and local contextual variations can be resolved in ways that support the expansion and maintenance of an EBP while preserving potential for public health benefit.

Keywords: Implementation, Sustainment, Teams, Process Model, Evidence-based practice

1 INTRODUCTION

Introduction of evidence-based practices (EBPs) can lead to substantial public health benefits. However, the implementation process can shape whether intended outcomes are actually achieved (Aarons & Palinkas, 2007; Allen, Brownson, Duggan, Stamatakis, & Erwin, 2012; Crea, Crampton, Abramson-Madden, & Usher, 2008; Fixsen, Naoon, Blase, Friedman, & Wallace, 2005; Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004; Palinkas & Aarons, 2009). Well-established practice models, implemented poorly or not sustained, will fail to achieve intended goals despite research evidence supporting their clinical effectiveness (Backer, 2000; Bond, Drake, McHugo, Rapp, & Whitley, 2009). Thus, an effective implementation approach is often as important as the practice to be utilized.

Several conceptual models describe factors that can influence implementation effectiveness. Some models emphasize structural features hypothesized to be core components of effective implementation (Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009; Feldstein & Glasgow, 2008; Greenhalgh et al., 2004; Mendel, Meredith, Schoenbaum, Sherbourne, & Wells, 2008). Other models emphasize implementation processes, outlining key steps (and their timing) hypothesized to contribute to successful implementation of service innovations (Glisson & Schoenwald, 2005; Sosna & Marsenich, 2006; Stetler, McQueen, Demakis, & Mittman, 2008). Structural and process implementation models are often conceptually aligned. For example, both types of models address the central importance of issues such as strong and effective leadership to support change initiatives, establishing a strong fit between change efforts and organizational and service system culture and values, creating methods for ensuring quality program delivery (i.e., fidelity), and clarifying/addressing financial supports for a change initiative.

This paper describes the Interagency Collaborative Team (ICT) implementation process model. The ICT model provides an approach to support successful roll-out of human service innovations in large geographic areas, particularly change efforts involving EBPs. It is directly relevant to improving outcomes of service enhancements in child and family service systems. The ICT model is designed to enable organizations to work together in ways that generate the structural and process supports associated with successful implementation and sustainment of innovations. We discuss some core areas of difference and similarity between the ICT model and other implementation strategies, connecting core features to one structural implementation framework, the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework (Aarons et al., 2011). Qualitative data from the scale-up of an EBP in one large county illustrate areas of strength and some limitations in the ICT model and provide perspective on other process models of EBP implementation.

1.1 Interagency Collaborative Team (ICT) Model in the EPIS Framework

Like a number of implementation frameworks, the EPIS framework summarizes variables that can positively or negatively affect the implementation of an evidence-based practice. The EPIS framework is unusual in identifying key variables thought to particularly affect implementation efforts during each of four major implementation stages in public sector child welfare and mental health settings. For example, some key variables identified as influencing the preparation and early implementation stages of a quality improvement effort include strength of the leadership supporting change (Aarons, 2006; Edmondson, 2004; Klein, Conn, & Sorra, 2001), the degree of fit of an innovation with the service system context (Klein & Sorra, 1996), clarity of financial support for proposed changes (Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009; Frambach & Schillewaert, 2002), level of involvement of practice developers in the implementation process (Aarons et al., 2011), and the presence of cross-organizational knowledge of and commitment to the new practice (Glisson & Schoenwald, 2005; Sosna & Marsenich, 2006).

The ICT implementation process model outlines steps designed to lead directly to the kinds of key implementation supports described in the EPIS framework. The model takes its name from the fact that it emphasizes the key role of collaboration among stakeholders and staff members at the system level, from multiple partnering organizations, and of developing or utilizing a local “seed” team to embody and support promotion and maintenance of expertise and ongoing fidelity in the practice to be implemented. Inter-agency collaboration and willingness to share expertise is central to multiple steps in the implementation process and across organizational levels. Conceptually, the ICT model has much in common with other implementation process models (Chamberlain, Price, Reid, & Landsverk, 2008; Glisson & Schoenwald, 2005; Sosna & Marsenich, 2006), which describe logically ordered sets of activities designed to create a context in which EBP implementation occurs effectively and intended public health benefits are realized.

1.1.1 ICT Processes and Action Steps

Figure 1 provides a graphical representation of key implementation processes included in the ICT model, with the stages of the EPIS framework listed temporally down the left side of the figure. In the ICT model, a process is considered to be a goal-driven domain of focus that extends over a period of time within the longer implementation effort. For example, the initial EBP education and stakeholder development and alignment processes involve an initial phase of identifying community-based stakeholders with interests in a particular practice change effort, and discussions and education efforts designed to lead to joint selection of and commitment to a common practice change initiative. The practice fit assessment process involves a careful analysis by key stakeholders at system and organizational levels of EBPs under consideration to identify aspects of practices that fit with existing policies, contracting, and service routines and those where modifications might be required. Brief descriptions of each ICT process are provided at the bottom of Figure 1.

Figure 1.

Figure 1

Open in a new tab

Implementation processes emphasized by the ICT, including structural supports hypothesized to emerge from following ICT model steps listed in Table 1.

Specific ICT model action steps are listed in Table 1 that animate the processes shown in Figure 1. Their contributions to each implementation process are noted in the figure. For example, the Initial EBP Education process occurs as part of ICT action steps A (convening of stakeholders) and B (soliciting expertise). Education about the EBP becomes an intense process focus that occurs in the context of meetings among interested stakeholders, supplemented by expertise about the EBP solicited from appropriate sources. Sources may be multiple, including EBP developers, other users of the EBP, researchers having familiarity with the practice, and/or materials available from sources such as journals or intermediary organizations that summarize information about EBPs. Structural supports designed to arise from the ICT processes are represented as planks beneath the model processes that generate them. We represent the ICT model in this manner because it is best conceptualized as a series of major actions that address core implementation processes. Specific action steps animate these processes and give rise to or strengthen key structural supports viewed as creating an environment that can sustain an innovative practice as it is scaled up.

Table 1.

Steps involved in the Interagency Collaborative Team (ICT) implementation process model.

ICT Steps

  1. Identify and convene stakeholders with likely interests in a shared quality improvement initiative (may be iterative)

  2. Solicit relevant expertise required to address questions about selected quality improvement directions and EBP alternatives

  3. Develop commitment and direction among stakeholders to a jointly supported, EBP-centered change effort

  4. Create an interagency seed team to:

    1. learn the EBP

    2. conduct initial local delivery of the EBP

    3. train new local EBP practitioners

    4. serve in a liaison role with external EBP developers/trainers

    5. monitor and provide feedback about quality of EBP delivery

    6. communicate and support a commitment to quality EBP delivery

    7. communicate with stakeholders about implementation progress

  5. Form additional interagency training teams that:

    1. deliver the EBP

    2. relay feedback about implementation to the seed team

    3. share information with one another about implementation progress

  6. Plan a phased reduction in EBP developer involvement
Open in a new tab

1.1.2 ICT Initial Steps: Exploration / Adoption Decision

The ICT model initially revolves around a service system and multi-agency commitment to invest in the long-term viability of an EBP-centered quality improvement initiative, with an ultimate goal to improve selected client level outcomes. Partnering agencies may include a range of stakeholder organizations, but particularly involve funding, administration, and service delivery organizations from the outset. During an initial exploration phase, stakeholders convene and meet to discuss need for a practice change effort that involves investment by multiple individuals and organizations. Although no specified leader is required to initiate such meetings, it is expected that one local or regional organization will often take responsibility for convening and leading such discussions. For example, a health and human service administration may convene discussions around maltreatment prevention, reduction in delinquency, or some other practice change effort. Within an ICT model-guided implementation, convening of stakeholders should include efforts to identify those stakeholders with substantial interests in the identified substantive area (e.g., child neglect).

A second important step in the process of exploring a possible practice change involves concentrated efforts to obtain wide-ranging factual information about the costs, benefits, and tradeoffs associated with specific practice changes. Outside expertise is identified and sought to help answer questions and reduce uncertainty about the change effort under discussion. The joint process of participating in education about possible practice change efforts and discussing the advantages and disadvantages of various options is aimed at developing a shared commitment and direction among stakeholders at an inter-agency level to a jointly supported EBP implementation.

1.1.3 Interagency Seed Team Development: Preparation and Implementation

Once a specific EBP is selected as the focal point for a broad practice change effort, stakeholders in the ICT process initiate implementation of the EBP by creating a formative interagency collaborative “seed” team (or ICST), which may consist of employees from several different local organizations that form a core unit of expertise in the selected service model. A seed team intentionally involves multiple organizations in the maintenance of innovation expertise to build broader investment in, commitment to, and communication about an innovation among invested stakeholders and subsequently trained practitioners. The seed team becomes a repository of local expertise for an EBP. It is designed to serve as the ongoing support structure for continued EBP training, coaching, and roll-out across a geographic area and as a facilitator of minor practice adjustments that help to fit a practice to diverse regional contexts (Aarons et al., 2012). Members of the seed team maintain a central liaison role between the EBP developer and other actors at multiple levels within the service system so that issues, decisions, and adaptations can be negotiated during initial experimentation with implementation.

1.1.4 Seed Team: Ongoing Responsibilities

Following inception of the seed team and initial EBP training, certification, and service delivery, the team is then responsible for training and supporting additional teams of individuals that can then implement the selected EBP as it is scaled up across a service system. The seed team assumes responsibility for ongoing training of new teams of practitioners that may consist of employees from several local non-profit organizations, hereafter referred to as interagency collaborative teams (ICTs). These ICTs form for the express purpose of learning and mastering delivery of the EBP to be implemented, under the guidance of the original seed team. Although members include staff from multiple organizations, they meet together with a seed team coach during a supervision and knowledge transfer phase. ICTs trained by the seed team are responsible for the primary delivery of the EBP. This implementation structure, with regional teams having interagency composition, results in a network of local providers that allows for high inter-agency communication, and information and possible workload sharing.

The seed team maintains relationships with ICTs following EBP training. In order to maintain and continually enhance quality delivery of the EBP, newly trained providers continue to receive constructive support and feedback in the form of supervision and coaching from seed team members for a defined period of time, which may vary by practice or situation. In the case of SafeCare, ongoing fidelity monitoring and coaching are integral components of the EBP. Organizing supervision and coaching through the seed team has many potential benefits in the short and longer term. This structure is designed to provide a clear source of leadership and information to newly trained providers. Ideally having the seed team serve as the source of ongoing training and coaching facilitates a gradual reduction of EBP developer involvement whereby the local service system and its contracted agencies become the repository of expertise in the practice being adopted. Based on its initial key role within a multi-organizational implementation effort, the seed team continues to serve as a locus of information about needs for adaptations to make a practice work within a particular local context. Such adaptations may involve changes to aspects of the intervention itself or to the structure of the service setting in which the EBP is delivered (e.g., Finno-Velasquez, Fettes, Aarons, & Hurlburt, under review). Cross-organizational membership on the seed team contributes to ensuring a continuing locus of expertise available to all organizations within the ICT partnership, reducing the kinds of expertise loss that regularly occur within individual organizations and agencies due to staff turnover and organizational changes.

As noted above, the ICT model seeks to foster implementation supports in the areas of practice fit, leadership, communication, expertise distribution, EBP quality (fidelity) management, effective developer involvement, and program adaptation. A large California county used the ICT model to implement an evidence-based neglect prevention program throughout the county. Qualitative inquiry into this effort helps to illustrate several key elements of the implementation approach.

1.2 The Present Study

Implementation of the SafeCare neglect prevention model occurred in one large California county. With approximately 3 million total residents, the county population is similar in size or larger than that of many smaller U.S. states. The county encompasses urban, semi-urban, and rural areas that are home to a diverse cultural mix of residents, including significant Mexican-American and Native American populations. Planning for many aspects of public human services in the county is organized into local planning regions, each with some of its own local history, demographic and cultural characteristics, climate, and topography. Implementation of any new practice at a county level represents a large-scale system and organizational change effort that occurs across the planning regions.

In 2007, the Department of Health and Human Services (DHHS) agency and the local chapter of a national foundation embarked upon an effort to transition one category of county maltreatment prevention services toward an EBP. County DHHS leaders (responsible for child welfare services), members of the local branch of a national foundation, and research partners convened to consider three different child focused EBPs to improve outcomes for children and families involved with the child welfare system. After consideration of research evidence, programmatic fit, and financial resources required, SafeCare®, an evidence-based child neglect prevention program utilizing home visiting (Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012; Lutzker et al., 1998), was selected for implementation.

Qualitative interviews and focus groups allowed us to document the roll-out of SafeCare, provided insight into how the ICT model generated key structural supports for implementation, and helped to identify process issues worthy of more careful consideration. The following section summarizes at a general level what we learned from that qualitative work about themes related to implementation process, including: (a) initial commitment and collaboration among stakeholders, (b) leadership, (c) communication, (d) practice fit with local context, (e) ongoing negotiation and problem solving, and (f) early successes.

2 METHODS

2.1 Overview

From August, 2008 to January, 2009 we undertook in-depth qualitative interviews with key stakeholders involved in the early stages of system-wide implementation of SafeCare that followed the ICT implementation model. Data collection and informed consent procedures were approved by the appropriate Institutional Review Boards.

2.2 Participants

Participants in this study included 27 stakeholders involved in various facets of the early implementation process. Participants were recruited through an initial telephone call or email describing the study purpose and participation. One of the authors either made the initial contact or was available to answer questions about participation. Our purposive sample consisted of all individuals who took part in initial EBP planning meetings, including representatives from the county (n=3) a foundation supporting part of the implementation effort (n=9), and the executive directors of the community-based organizations (n=3) that were eventually contracted to deliver SafeCare. Next, we interviewed key individuals involved in supporting delivery of the EBP, including SafeCare supervisors, trainers, and coaches (n=6) and front line providers (n=6).

2.3 Semi-Structured Interviews

Interview guides consisted of open-ended questions that were tailored to each stakeholder group. The interviews with representatives of the county, the foundation, and community-based organizations focused on the initial planning process, their roles and responsibilities and interactions with one another, and perceptions overall of SafeCare implementation. These interviews also sought to capture data on organizational- and system- level factors affecting implementation. The interviews with supervisors, trainers, coaches, and seed team members centered on each person’s involvement in the ICT approach, knowledge of and experiences with SafeCare, and the “fit” of the intervention with local populations and service delivery contexts. All participants agreed to recorded interviews, which lasted approximately 60 minutes, and were professionally transcribed. Transcriptions were reviewed for accuracy by a research assistant. In addition, ethnographer interview notes were typed and uploaded to an electronic database.

2.4 Data Analysis

We employed an iterative process to review the textual data from interviews and utilized NVivo 9 (2009) qualitative data analysis software to facilitate this work. Data analysis proceeded first by engaging in an open coding approach to locate the themes and issues that emerged from the interview transcripts. Focused coding was then used to determine which of these themes emerged frequently and which represented unusual or particular concerns (Emerson, Fretz, & Shaw, 1995). In this staged approach to analysis, our research team coded sets of transcripts, created detailed memos linking codes to each theme and issue, and then passed their work to other team members for review. Discrepancies in coding and analysis were identified and resolved through consensus during team meetings (Sandelowski & Barroso, 2003). Themes emerging from analyses were cross-walked with objectives of the ICT model to identify areas in which language used by participants might differ from that used in the model but have similar meaning. Principal themes from the interviews are presented, supplemented by relevant comments from participants. In some cases, quotations are edited slightly to smooth readability.

3 RESULTS

Results confirmed the significance of a number of the processes and structural supports outlined in the ICT model, including initial commitment and collaboration, cross-level leadership, and practice fit to the local context. Other key themes also arose, such as the importance of early successes, and of negotiating roles and responsibilities among stakeholder organizations. Interconnections among these themes are highlighted below.

3.1 Commitment and Collaboration

The first step in implementing the ICT model involves identifying and then bringing together key stakeholders invested in a change effort in order to discuss shared interests in undertaking a particular initiative. In this case, interest from stakeholders in the child welfare system and the foundation chapter in supporting a system-wide improvement in the area of child maltreatment led to the creation of a “Council” focused on a possible quality/capacity enhancement effort that eventually centered on SafeCare. The Council included representatives not simply from county child welfare services and the local foundation, but from community-based non-profit organizations, advocacy organizations, a children’s hospital, EBP developers, and researchers. Initial conversations about directions for the intervention were critical in setting the foundation for the long-term collaboration among stakeholders that would be required to implement and sustain SafeCare.

A potentially underappreciated aspect of building initial commitment is the fact that stakeholders do not necessarily share the same organizational culture or values when they begin to collaborate. In the case of SafeCare, some stakeholders worked in large government organizations, others in small non-profit organizations. Consequently, stakeholders often had very different ideas about how to pursue change and their respective roles in this process, which at times led to tension and conflict that added complexity to the implementation, some examples of which are described in a later section. Nonetheless, the stakeholders pushed forward, buoyed by the belief that the specific EBP they had agreed to implement would improve child welfare services and reduce neglect. County officials, for example, were enthusiastic about ensuring that services provided in the child welfare system were evidence-based. Their confidence in moving forward with SafeCare was bolstered after a presentation and discussion period with a research team knowledgeable about the EBP, its underlying evidence base, and the benefits of the intervention.

The initial commitment process involved frank discussion and evaluation of whether interests were broadly shared among stakeholders amidst differences in organizational directions, cultures, and values. Although this process of group reflection may not have been sufficient to guarantee the overall success of the implementation effort, it did appear to have been a central component in laying an appropriate foundation for positive outcomes. This process put the diverse interests of the stakeholders on the table, fostered commitment to a common direction, and engendered a sense of top-level leadership support for the change effort that was ultimately reinforced by additional leadership tiers.

3.2 Leadership

Once an intervention is selected, the ICT model calls for development of a “seed team,” an initial cadre of service providers responsible for acquiring expertise in the service model, for transmitting this knowledge to other teams of individuals involved in day-today service provision, and for providing ongoing fidelity assessment and support. By virtue of their roles as trainers and supervisors of future cohorts of SafeCare-trained home visitors, the nine seed team members were placed in a structural position of leadership. As the implementation progressed, three of the original seed team members were selected to assume the roles of trainers/coaches, and one emerged as the team leader and SafeCare supervisor. As noted by several participants, these individuals could be counted on to guide newly trained home visitors in consistent SafeCare practice and thus were paramount to implementation success.

As noted, one SafeCare supervisor became the primary identified team leader and source of support for home visitors. Several home visitors commented on the support provided by the SafeCare supervisor in particular. One home visitor stated, “She’s very good at answering our questions, while a second added, “I find her very helpful and she’s available if I need her.” Although not expressed in terms of leadership from the home visitor perspective, we interpreted the regular comments regarding the support and information provided by the SafeCare supervisor as a reflection of the clinical leadership provided by the seed team, as viewed by home visitors.

Strong leadership was also evident from the directors of each community-based provider organization, the local foundation, and county child welfare services. The fact that the provider organizations collaborated from the outset to respond to the local foundation’s Request for Proposals to deliver SafeCare from a multi-agency position, and then facilitated involvement of their staff within a single seed team, reinforced a broad sense of cross-level leadership commitment to SafeCare. During the implementation phase, the local foundation, in partnership with the county, also spearheaded organizational meetings for planning purposes. This higher-level buy-in, commitment, and support communicated a message that this new EBP was not the “flavor of the day” and that there was an expectation for effective implementation and ongoing use of SafeCare.

3.3 Communication

The majority of participants suggested that communication was crucial to successful implementation, but attributed problems encountered during the roll out of SafeCare to communication challenges. Interviewees reported that the communication structures around implementation were initially insufficient. Some stakeholders were privy to misinformation or to no information regarding issues impacting implementation. In one example, a county staff member incorrectly informed some supervisors and the local foundation that home visitation caseloads pre-SafeCare were half of what they were in reality (typically 20 vs. 10 cases per home visitor). Such unintentional misinformation altered the course of project planning and fueled concerns that implementation of the new EBP would prove to be too expensive in the long run. Problems disseminating information among stakeholders across all levels were also common. Email, in particular, did not function as a dependable mode of communication and information sharing. There were times when stakeholders felt that only a limited subset of individuals were receiving needed information about program implementation.

Stakeholders also described discussing or even deliberating on SafeCare issues individually outside of group meetings, while alluding to underlying power dynamics that influenced communication. One participant lamented about being left of out of these informal communications about SafeCare: “When I hear [other stakeholders] talking to each other, I can tell that [they] have talked on the side…. It looks like they’re able to access the information they need, and [can] find things out.” A second participant reported running into others involved in implementation in non-SafeCare milieus and felt freer to share ideas about the intervention and its roll out. They admitted, “We talk about [SafeCare] a bit. [If] I am with the county person, I’m like, ‘Don’t ask me [later at a Council meeting], I’ll talk to you here. I won’t talk to you in the meetings when there are other county people [present].” Such comments suggest that perceived power imbalances among stakeholders sometimes interfered with candid dialogue.

Structurally, the SafeCare supervisor was expected to serve as a locus of communication and information exchange between home visitors newly trained in SafeCare and the other stakeholder groups (e.g., county child welfare and the local foundation). The SafeCare supervisor commented on the challenges she experienced mediating between the home visitors and upper-level leadership: “Right now…we’re in the early stages [of implementation] so there’s a lot of things that are changing everyday…. [There’s a need to] maintain that open communication…. That’s really the big part of it, as far as ‘we want you guys to do this’ or ‘we want you guys to do this differently’ or ‘don’t use this form’ or ‘use this form.’ I tell the team, ‘There’s always going to be some changes and as soon as I know something you guys will know as well. So just bear with me. The first few months are going be like this.’” In some respects it was difficult for the SafeCare supervisor and the home visitors to keep up with frequent changes initiated from above, due to potential lack of clearly structured communication channels. Apart from the SafeCare supervisor, the home visitors also struggled to some degree with whom else they should be communicating regarding SafeCare implementation. In particular, they were often unsure whether they should turn to or report to their immediate team supervisor, who was not trained in SafeCare, but who managed their workaday lives, or to others involved in the roll-out, such as the researchers or trainers.

3.4 Fit With Existing Practice and Fidelity

The ICT model created an active, functional process for addressing the fit of SafeCare to the structure, culture, and local needs of service populations and the organizations delivering those services. This occurred at several points. During the initial discussion and commitment phase, possible EBPs were considered with respect to their target audiences and outcomes, modes of delivery, and training and resource requirements. SafeCare emerged from this phase as a top candidate. It had a jointly held focus of interest to Council members (child neglect), only involved retooling of the curricular component of existing home visitation services rather than a more substantial reformulation of service models, and had implementation costs viewed as manageable within a large-scale roll out of the practice. This first participatory phase put stakeholders on the path to selecting a model perceived to have good fit with organizational structure, values and needs.

At the practice level, the structure of the seed team created an ideal framework for adapting an EBP as it was progressively implemented across a larger scale service area. A major theme, widely expressed in our interviews, was concern for whether SafeCare could be applied to all families. The seed team described the service population as having multiple needs, not necessarily consistent with the SafeCare curriculum. Families, for example, struggled with serious problems, ranging from the procurement of basic necessities to shelter, critical issues that fell outside the scope of SafeCare. Commenting on the difficulties of carrying out SafeCare in such circumstances, one seed team member explained, “[Families] are having a hard time. How can somebody want to learn about safety or health when they don’t know what they’re going to feed their kids [or] have the money to pay their rent?” This individual admitted to cutting short her visits with families in order “to help them with resources or whatever they’re going through.” One solution advanced during the early implementation period was not to begin SafeCare when families were in crisis. In such cases, the seed team member only initiated SafeCare after he or she had the opportunity to help the family address other issues. Over time, the seed team helped to shift home visitation practice so that such issues could occur within the context of ongoing SafeCare visits. In fact, such issues were fit within the broader problem-solving framework utilized within SafeCare. The seed team played an instrumental role in identifying this issue and facilitating incremental solutions that helped tailor SafeCare to the local context.

The seed team was also able to undertake deliberative steps to improve the cultural responsiveness of SafeCare to populations in the local region. For example, the seed team actively reviewed and translated SafeCare materials to improve their relevance to Latino, particularly Mexican-American, families. The seed team also discussed and adapted child health focused sessions and materials to the customs of Latino immigrant families, who sometimes expressed values and preferences for homeopathic remedies not originally considered within the health module of the SafeCare EBP (Finno-Velasquez, Fettes, Aarons, & Hurlburt, under review).

The ICT model requires that the seed team become the local cross-agency repository of expertise in an EBP for a group of collaborating organizations. At the outset, this involved the seed team learning and utilizing the new practice model until they reached a level of expertise meriting certification by the model developers. While mastering the EBP during early service delivery, the seed team was also immersed in delivery of the practice in the local context. In their designated roles as future trainers, coaches, and supervisors, the seed team members were tasked with internalizing the knowledge of and expectations of program developers, but also with a high level of responsiveness to local population needs. As noted above, the pressure to resolve tensions between existing SafeCare structure and knowledge of the local service population put the seed team in the position of developing a locally refined expertise that fit the practice to the local area and allowed for a planned decrease in the involvement of the original EBP developer in supporting sustainment of SafeCare. Documentation of the specific adaptations made by the seed team are discussed by Finno et al. (under review). Participants viewed the gradual decrease in developer involvement as proceeding effectively, as originally planned.

3.5 Negotiation of Rights, Roles, Responsibilities, and Interests

In addition to a need to incorporate further structure in the area of communication patterns, as alluded to by earlier qualitative findings, the ICT model might benefit from further attention to enabling methods for facilitating negotiation of differences among organizations and their members. Although many possible differences among partners may emerge during the implementation of any new change effort, several specific examples from the SafeCre implementation experience illustrate the kinds of issues that arise and require negotiation during a large-scale collaborative EBP initiative. For example, the appropriate pace for SafeCare implementation and documentable change was one area in which stakeholders held differing expectations and opinions. It arose because the collaborating partners had different needs and expectations influencing their participation. The local foundation supporting initial training and development of the seed team had interests in seeing measurable outcomes quickly in order to show progress to donors supporting the organization, in addition to their core interest in improving outcomes for children and families. Other stakeholders did not share the pressure to accelerate change to the same degree. Discordant expectations led to some tensions related to this issue.

Differences also emerged with regard to expected roles and responsibilities. For example, one important decision during implementation revolved around the order in which the seed team trained new home visitor teams in the various county regions. As a new partnership in which different parties jointly contributed to the SafeCare implementation, issues around rights and responsibilities needed to be negotiated. County representatives had the authority and perceived responsibility to make decisions about ordering of SafeCare roll-out, since they were entrusted by the public to provide oversight of child welfare services. However, foundation staff also sought to exert authority in this area because the foundation had committed funds to support the seed team. As shared authority was being established, power struggles between parties occasionally ensued. These tensions were recognized by others involved in the implementation. One community-based provider organization executive stated, “It’s no one’s fault, but I think the foundation and the county still have to have some meetings about whose role is what, and who has decision-making authority on certain things. And I think right now they are still a little messy.”

One further example may also be illustrative. Having significant investment in the success of the SafeCare implementation, especially given that this was the first time the foundation was investing in a single major capital improvement as opposed to multiple smaller local grants, foundation representatives indicated a need and a right to work directly at times with the local community-based organizations contracted by the county to deliver SafeCare services. County representatives, again having direct responsibility for management and oversight of child welfare services, also asserted a duty to be involved in conversations and communications around service initiatives under their purview, sometimes leading to tensions around roles and responsibilities among collaboration partners. One participant described the resulting tension as a “strange triangulation” between the entities involved. However, “What I think helped move it [the intervention] along was the enthusiasm of everyone on the Council, [and] the enthusiasm of the County, for transitioning into the SafeCare model.”

3.6 Early Successes

The ICT model intentionally involves the staged deployment of an EBP, beginning with the experience of the seed team prior to subsequent roll out to and support of other practitioners. Participants in qualitative interviews consistently mentioned that the purposeful transition of SafeCare from an initial seed team did lead to early signs of success that were interpreted as facilitating and supporting efforts to implement and sustain SafeCare. One participant noted that stakeholders across all levels were “…seeing it is working. They are seeing that they’re not getting cheated or embarrassed. They are seeing that there is a system in place. They’re just calmer. And so their control needs are a little less.” Markers of success appeared to help stakeholders begin to overcome power struggles. Evidence of success took various forms, including positive experiences with initial training and delivery of services and encouraging reports by the home visitors about how families were responding to the intervention. A county official noted, “The training went well and implementation was successful…. The reception actually at the line level has exceeded my expectations…. Home visitors are comfortable with one, embracing more of a script and, you know, more structure within the visit and that they’re receptive to the coaching…” Undertaking implementation in a phased roll out, beginning with exploration, appropriate preparation, and planned implementation phases and involving the seed team as a central training and support component, created opportunities for shared successes and further commitment to ongoing problem solving.

4 DISCUSSION

This paper describes the ICT model for EBP implementation, which is designed to facilitate development of many supports hypothesized to be central to successful quality improvement efforts organized around EBP implementation. Qualitative data from implementation of SafeCare in one large geographic area provided the opportunity to reflect on the strengths and limits of the ICT model and to consider it relative to other process models of quality improvement and EBP implementation.

The ICT model departs from traditional service structure and process by distributing local expertise across service teams, and more focally in a seed team, in a way that takes into account challenges faced by real-world public social service systems (Aarons, Hurlburt, & Horwitz, 2011; Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009). Drawing on computer science and engineering theories of distributed expert systems (Dai, Xie, Poh, & Liu, 2003) and team decision making (Hollenbeck et al., 1995), the ICT model aims to increase effective team functioning through building greater systemic and cross agency trust (Edmondson & Roloff, 2009) and collaboration (Bertram, 2008). This structuring occurs at multiple levels, including among administrative and funding stakeholders with interest in a practice change initiative, and at the level of local clinical leadership (i.e., the seed team). The goal of the ICT process is to build interagency relationships at both levels, and between levels, creating the structural supports central to effective adoption, implementation, and sustainment of an EBP with positive public health effects. With the seed team playing a substantial role in operational implementation, the ICT model seeks to build structures and processes that enable the fitting of an EBP to the local context as outside developer involvement is reduced, and potential for EBP sustainment is increased.

Other implementation strategies directly relevant to EBP implementation include such models as the Availability, Responsiveness and Continuity (ARC) and Community Development Team (CDT) models. Each arises out of somewhat different theoretical frameworks than the ICT model. The ARC model emerged from organizational development (Burke, 1993; Nadler & Tushman, 1977; Porras & Robertson, 1992) and interorganizational domain development (Gray, 1990; Trist, 1985) theories. The ARC model focuses strongly on improving organizational culture and climate and organizational processes to support effective care and more effective EBP implementation. ARC relies on an organizational change agent to work with the organization to effect intraorganizational change to improve care. The CDT model was developed based on the experiences of the California Institute for Mental Health (CiMH), a training and technical assistance organization supported by county mental health agencies and child welfare systems. The CDT approach focuses on developing supportive collaborations among stakeholders, often in different counties, that are considering and implementing EBPs. The CDT model, like the ARC, relies heavily on the involvement of an outside consultant to structure communications among stakeholders, set priority topics for discussion, and foster organizational problem solving around issues that arise during implementation. The outside consultant brings collected expertise acquired from extensive communications with multiple EBP program developers and local stakeholders to his/her role in supporting EBP implementation efforts (Sosna & Marsenich, 2006).

Considerable common ground exists among these models, although there are also some areas of difference, both qualitatively and in overall emphasis. For example, the ICT model devotes considerably less attention to efforts to change intraorganizational culture and climate than the ARC model. However, both have many similarities including processes specifically targeting initial collaborative work and joint decision making among stakeholders, and the development of mechanisms for monitoring and providing feedback about implementation progress and quality, and for fitting practices to be implemented to the local geographical and cultural context. The ICT and CDT process models also share similarities. Both include processes specifically targeting initial collaborative work among adopting stakeholders and organizations, using education to reduce uncertainty about EBP adoption during the exploration and preparation stages, and developing a core focus on EBP fidelity as part of implementation and sustainment. The models differ, however, in how many of the implementation processes are organized by an external change agent as opposed to individuals within the service system. Both the ARC and CDT models include a much more extensive role for an external change agent than the ICT model, which proposes that the planned actions and processes illustrated in Figure 1 will result in the development of structural supports at the core of effective implementation. The ICT model relies more on establishing a process map for developing local structures to support implementation, and on facilitating desired interagency relationships, than on external consultant support.

Given notable common procedural aspects of different implementation process models, the rich qualitative data gathered around implementation of SafeCare in this study informs the ICT model and, to some degree, other implementation process models as well. One key finding from this qualitative study was the interconnectedness of the process components and the implementation supports they facilitated. Initial collaborative efforts among stakeholders, including recognition and discussion of differences, facilitated later problem solving and negotiation around areas of disagreement and potential conflict. Staged roll out of implementation created opportunities for early success, which in turn facilitated communication and problem resolution and helped build “buy-in” and enthusiasm for the EBP. The presence of the seed team solidified perceptions of leadership at multiple levels and directly addressed other key implementation drivers, including sustaining a focus on fidelity and fitting of the practice to be implemented to key local contextual variations. Our qualitative work supports arguments made by others that multi-component approaches to implementation that address inter- and intra-organizational contextual issues are necessary to create an environment conducive to strong implementation and sustainment (Aarons, Hurlburt, & Horwitz, 2011; Damschroder et al., 2009; Ferlie & Shortell, 2001; Fixsen, Blase, Naoom, & Wallace, 2009; Glisson & Schoenwald, 2005; Grimshaw et al., 2001; Grol & Grimshaw, 1999). Results from interviews clearly connected various aspects of the implementation process to the overall supports for implementation. The interviews also are consistent with the idea that core implementation drivers reinforce and support one another (Fixsen et al., 2009).

The ICT model includes notably less involvement of external change agents, such as intermediary or brokering organizations, or outside organizational development consultants than other process models. The foundation and its willingness to support initial implementation phases were important to SafeCare implementation in San Diego because the foundation served as the initial convening agent and was responsible for providing financial support to initiate implementation of SafeCare. However, the ICT model relies more heavily on organizing a series of actions and local structures designed to create inherent inter-organizational interaction and mutual support around an EBP implementation. Such structures can include service system contracts with community-based organizations that deliver services, or memorandums of agreement that support the ICT structure. The leadership at the system level, and at the organizational level is needed to facilitate formal agreements for the ICT structure. In addition, the ICT model presumes that the presence of a local seed team will create a locus of leadership at the practice level that reinforces initial commitments from higher-level administrators and accountability to service provision and ongoing intervention fidelity. The model presumes that the role of the seed team, both to learn and to transmit a practice to other colleagues, will require the team to identify and resolve salient adaptation issues that arise in the local context. The existence of the seed team, with its coaching and supervision responsibilities, is designed to create a focus on fidelity that is transmitted directly to practitioners working in different organizations delivering the new practice. The seed team is placed in a structural role to carry a voice of leadership and fidelity focus throughout the course of implementation, rather than having this spearheaded by an external change agent. It is worth noting that the ICT model seeks to create conditions for successful implementation of an identified practice change, not to influence how all practice changes occur, although it is presumed that experience with the model with improve local change efforts more broadly.

Initial results suggest that the ICT model steps do lead to many of the intended structural supports or drivers for effective implementation. However, they also reveal that the ICT process may not sufficiently foster some of the supports that receive direct attention in other process models. Our qualitative data suggest that including explicit processes for regular communication in the ICT model is crucial to improve the ability of involved stakeholders to identify and address potential conflicts in ways that build trust and continued cooperation. Participants’ experiences suggested that there are likely to be unexpected areas of conflict and disagreement in any implementation effort and that clear communication and a problem solving orientation will facilitate effective resolution of such issues.

The ICT model does appear to have resulted in a systemic focus on fidelity in a manner that facilitates appropriate local adaptation. For many years, the potential inflexibility and lack of local cultural relevance of EBPs have been significant limiting concerns (Bernal & Scharron-del-Rio, 2001; Bernal, 2006; Bernal, Jimenez-Chafey, & Rodriguez, 2009; Castro, Barrera, & Martinez, 2004; Kumpfer, Alvarado, Smith, & Bellamy, 2002; Lau, 2006; Matos, Torres, Santiago, Jurado, & Rodriguez, 2006). Experiences from participants using the ICT model with SafeCare suggest that such concerns have some validity, but that they are addressable through planned implementation structures and processes. Questions did arise about the fit of SafeCare with local cultural nuances, particularly for Latino families. The seed team, with a liaison role between the EBP developer and trained home visitors and families receiving services, did address issues of how to adapt SafeCare to local conditions while remaining true to core components of the practice. The ICT model envisions this as a negotiated process between a locally based team and EBP developers who know that this team will assume responsibility over time for system-wide fidelity maintenance. These kinds of negotiations were observed as the seed team recognized the need for SafeCare adaptation for the county and worked with SafeCare developers to make appropriate adaptations (Finno-Velasquez, Fettes, Aarons, & Hurlburt, under review).

At a broad level, our qualitative results largely support the role that ICST model processes play in generating structural supports for implementation and sustainment of a systemwide EBP-driven quality improvement effort. Several areas were identified as likely needing further attention in the implementation model, including a process for detailing appropriate communication patterns early during implementation and a process for creating forums in which differences among participating organizations can be identified, discussed, and resolved.

In the presence of multiple implementation models a natural question that arises concerns the conditions under which a particular implementation model is particularly relevant. The ICT model seems particularly well suited to circumstances in which an EBP is planned for roll-out in a given organization or across a contiguous geographic area in which authority for initiating and supporting the effort falls under the domain of a small number of administrative entities, in this case county child welfare services. As opposed to focusing on collaborative information sharing among organizations implementing an EBP in different locations, or emphasizing the internal organizational cultural and climate of implementing organizations, the ICT model is relevant when local organizations are in a position to work together to develop a shared core infrastructure for implementing and sustaining expertise in a practice that will be scaled up across a broad geographic area. The ICT model is somewhat unique in its emphasis on creating implementation supports through formal and strategic structuring and staging of the implementation process rather than through the extensive involvement of an outside organization that organizes and pushes the implementation process forward.

5 CONCLUSIONS

The ICT implementation process model developed out of the collaborative experience of researchers and local agency partners. Use of its implementation processes has led to sustained and widespread use of SafeCare, an evidence-based neglect prevention model, in one large county, and resulted in the phased transitioning of expertise from model developers to the local context. Although some aspects of the implementation model may benefit from enhancement, results suggest that the process model generates strong structural supports for implementation and creates conditions in which tensions between EBP structure and local contextual needs can be resolved in ways that support the expansion and maintenance of the EBP while preserving its potential for public health benefit.

Acknowledgments

This study was supported by National Institute of Mental Health grants 2R01MH072961 and P30MH074678. The authors thank the community-based organizations, case-managers, and supervisors that made this study possible.

Footnotes

The authors declare no other conflicts of interest.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Aarons GA. Transformational and transactional leadership: Association with attitudes toward evidence-based practice. Psychiatric Services. 2006;57(8):1162–1169. doi: 10.1176/appi.ps.57.8.1162. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons GA, Green AE, Palinkas LA, Self-Brown S, Whitaker DJ, Lutzker JR, Chaffin MJ. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science. 2012;7(32):1–9. doi: 10.1186/1748-5908-7-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in child welfare. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38(1):4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Aarons GA, Palinkas LA. Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34:411–419. doi: 10.1007/s10488-007-0121-3. [DOI] [PubMed] [Google Scholar]
  5. Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health. 2009;99(11):2087–2095. doi: 10.2105/AJPH.2009.161711. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Allen P, Brownson RC, Duggan K, Stamatakis KA, Erwin PC. The makings of an evidence-based local health department: Identifying administrative and management practices. Frontiers in Public Health Services and Systems Research. 2012;1(2):2. [Google Scholar]
  7. Backer TE. The failure of success: Challenges of disseminating effective substance abuse prevention programs. Journal of Community Psychology. 2000;28(3):363–373. [Google Scholar]
  8. Bernal &, Scharron-del-Rio Are empirically supported treatments valid for ethnic minorities? Toward an alternative approach for treatment research. Cultural Diversity and Ethnic Minority Psychology. 2001;7:328–342. doi: 10.1037/1099-9809.7.4.328. [DOI] [PubMed] [Google Scholar]
  9. Bernal G. Intervention development and cultural adaptation research with diverse families. Family Process. 2006;45(2):143–151. doi: 10.1111/j.1545-5300.2006.00087.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bernal G, Jimenez-Chafey MI, Rodriguez MMD. Cultural Adaptation of Treatments: A Resource for Considering Culture in Evidence-Based Practice. Professional Psychology-Research and Practice. 2009;40(4):361–368. [Google Scholar]
  11. Bond G, Drake R, McHugo G, Rapp C, Whitley R. Strategies for improving fidelity in the national evidence-based practices project. Research on Social Work Practice. 2009;19(5):569. [Google Scholar]
  12. Burke MJ. Computerized psychological testing: Impacts on measuring predictor constructs and future job behavior. In: Schmitt N, Borman WC, editors. Personnel Selection in Organizations. Jossey-Bass; 1993. pp. 203–239. [Google Scholar]
  13. Castro F, Barrera M, Martinez C. The cultural adaptation of prevention interventions: Resolving tensions between fidelity and fit. Prevention Science. 2004;5(1):41–45. doi: 10.1023/b:prev.0000013980.12412.cd. [DOI] [PubMed] [Google Scholar]
  14. Chaffin M, Hecht D, Bard D, Silovsky JF, Beasley WH. A statewide trial of the SafeCare home-based services model with parents in Child Protective Services. Pediatrics. 2012;129(3):509–515. doi: 10.1542/peds.2011-1840. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Chamberlain P, Price J, Reid J, Landsverk J. Cascading implementation of a foster and kinship parent intervention. Child Welfare. 2008;87(5):24–48. [PMC free article] [PubMed] [Google Scholar]
  16. Crea TM, Crampton DS, Abramson-Madden A, Usher CL. Variability in the implementation of Team Decisionmaking (TDM): Scope and compliance with the family to family practice model. Children and Youth Services Review. 2008;30(11):1221–1232. [Google Scholar]
  17. Damschroder L, Aron D, Keith R, Kirsh S, Alexander J, Lowery J. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science. 2009;4:50. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Edmondson AC. Learning from failure in health care: Frequent opportunities, pervasive barriers. Quality and Safety in Health Care. 2004;13(suppl 2):ii3–ii9. doi: 10.1136/qshc.2003.009597. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Emerson R, Fretz R, Shaw L. Writing ethnographic fieldnotes. Chicago: University of Chicago Press; 1995. [Google Scholar]
  20. Feldstein A, Glasgow R. A practical, robust implementation and sustainability model (PRISM) Joint Commission Journal on Quality and Patient Safety. 2008;34(4):228–243. doi: 10.1016/s1553-7250(08)34030-6. [DOI] [PubMed] [Google Scholar]
  21. Ferlie EB, Shortell SM. Improving the quality of health care in the United Kingdom and the United States: A framework for change. Milbank Quarterly. 2001;79(2):281–315. doi: 10.1111/1468-0009.00206. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Finno-Velasquez MF, Fettes D, Aarons GA, Hurlburt MS. Cultural adaptation of an evidence-based home visitation program: Latino clients’ experiences of service delivery during implementation under review. [Google Scholar]
  23. Fixsen D, Blase K, Naoom S, Wallace F. Core implementation components. Research on Social Work Practice. 2009;19(5):531. [Google Scholar]
  24. Fixsen DL, Naoon SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa: University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network; 2005. [Google Scholar]
  25. Frambach RT, Schillewaert N. Organizational innovation adoption: A multi-level framework of determinants and opportunities for future research. Journal of Business Research Special Issue: Marketing theory in the next millennium. 2002;55(2):163–176. [Google Scholar]
  26. Glisson C, Schoenwald S. The ARC organizational and community intervention strategy for implementing evidence-based children’s mental health treatments. Mental Health Services Research. 2005;7(4):243–259. doi: 10.1007/s11020-005-7456-1. [DOI] [PubMed] [Google Scholar]
  27. Gray B. Building interorganizational alliances: Planned change in a global environment. Research in organizational change and development. 1990;4:101–140. [Google Scholar]
  28. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly. 2004;82(4):581–629. doi: 10.1111/j.0887-378X.2004.00325.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Grimshaw J, Shirran L, Thomas R, Mowatt G, Fraser C, Bero L, O’Brien M. Changing provider behavior: An overview of systematic reviews of interventions. Medical Care. 2001;39(8):II2–II45. [PubMed] [Google Scholar]
  30. Grol R, Grimshaw J. Evidence-based implementation of evidence-based medicine. Journal on Quality Improvement. 1999;25:503–513. doi: 10.1016/s1070-3241(16)30464-3. [DOI] [PubMed] [Google Scholar]
  31. Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: An organizational analysis. Journal of Applied Psychology. 2001;86(5):811–824. doi: 10.1037/0021-9010.86.5.811. [DOI] [PubMed] [Google Scholar]
  32. Klein KJ, Sorra JS. The challenge of innovation implementation. Academy of Management Review. 1996;21(4):1055–1080. [Google Scholar]
  33. Kumpfer KL, Alvarado R, Smith P, Bellamy N. Cultural sensitivity and adaptation in family-based prevention interventions. Prevention Science. 2002;3(3):241–246. doi: 10.1023/a:1019902902119. [DOI] [PubMed] [Google Scholar]
  34. Lau AS. Making the case for selective and directed cultural adaptations of evidence-based treatments: Examples from parent training. Clinical Psychology: Science and Practice. 2006;13(4):295–310. [Google Scholar]
  35. Lutzker JR, Bigelow KM, Doctor RM, Gershater RM, Greene BF. An ecobehavioral model for the prevention and treatment of child abuse and neglect. In: Lutzker J, editor. Handbook of Child Abuse Research and Treatment. Springer; 1998. pp. 239–266. [Google Scholar]
  36. Matos M, Torres R, Santiago R, Jurado M, Rodriguez I. Adaptation of parent-child interaction therapy for Puerto Rican families: A preliminary study. Family Process. 2006;45:205–222. doi: 10.1111/j.1545-5300.2006.00091.x. [DOI] [PubMed] [Google Scholar]
  37. Mendel P, Meredith L, Schoenbaum M, Sherbourne C, Wells K. Interventions in organizational and community context: A framework for building evidence on dissemination and implementation in health services research. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35(1–2):21–37. doi: 10.1007/s10488-007-0144-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Nadler DA, Tushman ML. A diagnostic model for organizational behavior. In: Hackman RJ, Lawler EE, Porter LWN, PS, editors. Perspectives on behavior in organizations. New York: McGraw-Hill; 1977. [Google Scholar]
  39. Palinkas LA, Aarons GA. A view from the top: Executive and management challenges in a statewide implementation of an evidence-based practice to reduce child neglect. International Journal of Child Health and Human Development. 2009;2(1):47–55. [Google Scholar]
  40. Porras J, Robertson P. Organizational development. In: Dunnette M, Hough L, editors. Handbook of industrial and organizational psychology. London: Consulting Psychologists Press; 1992. pp. 719–822. [Google Scholar]
  41. Sandelowski M, Barroso J. Classifying the findings in qualitative studies. Qualitative Health Research. 2003;13(7):905–923. doi: 10.1177/1049732303253488. [DOI] [PubMed] [Google Scholar]
  42. Sosna T, Marsenich L. Community development team model: Supporting the model adherent implementation of programs and practices. California Institute of Mental Health; Sacramento, CA: 2006. [Google Scholar]
  43. Stetler CB, McQueen L, Demakis J, Mittman BS. An organizational framework and strategic implementation for system-level change to enhance research-based practice: QUERI Series. Implementation Science. 2008;3:30. doi: 10.1186/1748-5908-3-30. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Trist E. Intervention strategies for interorganizational domains. In: Tannenbaum R, Margulies N, Massarik F, editors. Human systems development. San Francisco: Jossey-Bass; 1985. [Google Scholar]

RESOURCES