Collaboration, Negotiation, and Coalescence for Interagency-Collaborative Teams to Scale-up Evidence-Based Practice

Abstract

Objective

Implementation and scale-up of evidence-based practices (EBPs) is often portrayed as involving multiple stakeholders collaborating harmoniously in the service of a shared vision. In practice, however, collaboration is a more complex process that may involve shared and competing interests and agendas, and negotiation. The present study examined the scale-up of an EBP across an entire service system using the Interagency Collaborative Team (ICT) approach.

Methods

Participants were key stakeholders in a large-scale county-wide implementation of an EBP to reduce child neglect, SafeCare^®. Semi-structured interviews and/or focus groups were conducted with 54 individuals representing diverse constituents in the service system, followed by an iterative approach to coding and analysis of transcripts. The study was conceptualized using the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework.

Results

Although community stakeholders eventually coalesced around implementation of SafeCare, several challenges affected the implementation process. These challenges included differing organizational cultures, strategies, and approaches to collaboration, competing priorities across levels of leadership, power struggles, and role ambiguity. Each of the factors identified influenced how stakeholders approached the EBP implementation process.

Conclusions

System wide scale-up of EBPs involves multiple stakeholders operating in a nexus of differing agendas, priorities, leadership styles, and negotiation strategies. The term collaboration may oversimplify the multifaceted nature of the scale-up process. Implementation efforts should openly acknowledge and consider this nexus when individual stakeholders and organizations enter into EBP implementation through collaborative processes.

The implementation of evidence-based practice (EBP) is a priority for improving the quality of health, allied health and social services in the United States (U.S.) and other countries (Aarons, Hurlburt, & Horwitz, 2011). The National Institutes of Health, the Centers for Disease Control and Prevention, and the Agency for Healthcare Research and Quality have all sponsored research to develop and improve effective approaches to EBP implementation and sustainment. This emphasis is not unique to the U.S. and similar initiatives exist in the United Kingdom, Canada, Australia and other countries (Bero et al., 1998; Brownson, Colditz, & Proctor, 2012). Developing effective approaches to EBP implementation is an important initiative for improving health care and social services globally and locally (Pelletier, Porter, Aarons, Wuehler, & Neufeld, 2013).

A number of implementation approaches invoke collaboration as a key element to support effective EBP instantiation in service systems, organizations, and teams. For example, the Community Development Team (CDT) model developed by the California Institute for Mental Health (CIMH) utilizes a change agent, that is, an individual or team that helps to facilitate innovation, and foster communication, collaboration, and problem solving between counties and/or organizations considering EBP implementation (CIMH, 2006). The Institute for Healthcare Improvement’s Breakthrough Series describes collaboration as the key to quality improvement and organizational change (IHI, 2003). The Availability, Responsiveness, and Continuity (ARC) quality improvement model also utilizes a change agent and includes developing collaboration as a key early stage in the implementation process (Glisson & Schoenwald, 2005). The four-phased Exploration, Preparation, Implementation, and Sustainment (EPIS) framework emphasizes the importance of collaboration throughout all phases of the implementation process (Aarons et al., 2011).

Collaboration refers to a variety of relationships among stakeholders, although it is generally understood as individuals or groups working together to achieve a common goal (Butterfoss & Kegler, 2009; Butterfoss, 2007). Other definitions have been proposed, including working together to resolve issues of trust, turf, resources and conflict of interest (Shane, 1982) that may be affected by history or relationships, differing professional “languages,” aims and agendas, and power inequalities and struggles (Ranade, 1998; Salmon, 2004). However, it is also the case that collaboration is not wholly under the control of the participating stakeholders (Huxham & Vangen, 2000).

In the broader literature, a precondition for successful collaboration is for each member to expect to share authority, resources, responsibilities, and rewards, and also to respect each other’s own “structure, agenda, values, and culture” (Butterfoss, 2007, p. 27). Stakeholders must be willing to help create and operate within a structural arrangement that may be different from those to which they are ordinarily accustomed. This requires clear communication, comprehensive planning, and an understanding that the continuum of power and responsibilities will likely change depending on the stage of implementation and collaboration (Butterfoss, 2007, pp. 26–28). Diverse stakeholders are also influenced by different organizational cultures and values, which have been shown to influence the implementation of EBPs (Aarons & Sawitzky, 2006). Any one of these factors can result in challenges when participants in a collaborative undertaking do not share the same understandings and expectations, or are guided by competing organizational and individual goals and agendas.

There is a tendency to underestimate the complexity involved in collaboration and to minimize the subtext of stakeholder perceptions, priorities, agendas, and idiosyncrasies that may influence a change process such as EBP implementation (Green & Aarons, 2011). And yet, collaboration is considered an important element in change processes that involve communities and organizations. For example, part of the collaboration process involves complex negotiation related to potentially competing demands, stakeholder organizational cultures, world views, and power and control (Kramer & Messick, 1995). When multiple stakeholders with diverse views are at the table, negotiations that take problem-solving, pro-social, and less egoistic orientations may facilitate collaboration and achievement of joint outcomes (Campbell & Mark, 2006; De Dreu, Weingart, & Kwon, 2000). Studies also suggest that negotiations involving both cooperative and individualistic stakeholders may lead to more positive process and outcomes in contrast to having only individualistic stakeholders (Schei, Rognes, & Shapiro, 2011).

A weakness in the literature on collaboration is a general lack of a deeper exploration of the kinds of tensions that can arise during a change initiative, and the ways in which such tensions are, or are not, resolved. Often, when the term collaboration is invoked, the complexity of negotiation processes may be understated. While several studies document tensions in collaborative processes (Hersch, 1970; Hopkins, Monaghan, & Hansman, 2009; Sowa, 2008), such studies are few and typically focus on change initiatives other than EBP implementation. Implementation plans that rely on building collaborations are not risk-free and have the potential to impede or undermine implementation as well as potentially support it (Kano, Willging, & Rylko-Bauer, 2009). Thus, studies are needed that take a more in-depth look inside the black box of collaboration in major system change and EBP scale-up efforts.

The Present Study

The present study examines the role of collaboration in a large-scale countywide implementation of an EBP to reduce child neglect, SafeCare^®. SafeCare is designed for families involved in or at-risk for child neglect and child welfare system involvement. It has demonstrated effectiveness with diverse populations and also improves service provider burnout and staff retention (Aarons, Fettes, Flores, & Sommerfeld, 2009; Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009; Chaffin, Bard, Bigfoot, & Maher, 2012; Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012; Damashek, Bard, & Hecht, 2012; Silovsky et al., 2011). SafeCare incorporates a generalized problem-solving approach into three modules: 1) home safety/accident prevention, 2) child health, and 3) parent-child and/or parent-infant interaction. In contrast to services as usual, SafeCare is highly structured and manualized and is typically delivered by Bachelor’s and Master’s degree level service providers. More detailed information can be found in the SafeCare manual (Lutzker & Bigelow, 2002).

The study implementation conceptual model is the four-phased Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (Aarons et al., 2011). Similar to some other models the EPIS framework encompasses factors likely to influence implementation in the outer (e.g., service system) and inner (e.g., organizational) contexts (e.g., Damschroder et al., 2009; Proctor et al., 2009; Schoenwald & Hoagwood, 2001; Southam-Gerow, Rodríguez, Chorpita, & Daleiden, 2012) through the four phases and identifies variables that can positively or negatively affect the progress of implementation. For example, key variables that may influence the preparation and early implementation phases include strong leadership supporting change (Aarons, 2006; Edmondson, 2004; Klein, Conn, & Sorra, 2001); strong fit of an innovation within the service system context (Klein & Sorra, 1996); clarity of financial support for proposed changes (Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009; Frambach & Schillewaert, 2002); effective involvement of practice developers in the implementation process (Aarons et al., 2011); and development of cross-organizational knowledge of and commitment to the new practice (CIMH, 2006; Glisson & Schoenwald, 2005).

This paper draws upon qualitative data collected via semi-structured interviews and focus groups to describe collaborative processes unfolding among multiple organizations and stakeholders implementing SafeCare in a county-wide service system. Consistent with the EPIS framework, the effort to implement SafeCare proceeded through a common series of stages, including initial Exploration of a range of EBPs to determine which would best fit the system, provider organizations, service providers, and clients. In the Preparation Phase, collaborators developed strategies, processes, and contracts to support EBP implementation and sustainment. In the Implementation Phase, a “seed team” of trainers and coaches was developed. The seed team is comprised of the local EBP experts (trained by the intervention developer) that could train, coach, and certify others to deliver the EBP. The seed team, once certified as coaches and trainers, trained subsequent interagency service provider teams, defined as units of employees from one or more CBOs. Because the Implementation Phase had just concluded at the time of this writing, the Sustainment Phase will be examined in future studies.

The implementation occurred within the context of the Interagency Collaborative Team model (Hurlburt et al., In review) that involves stakeholders across organizations working together in cross-organization working groups. The ICT implementation framework itself was developed as an emergent process of child-welfare system (CWS), foundation (FDN), and community-based organization (CBO) stakeholders in consultation with academic collaborators.

The ICT model recognizes and supports implementation in complex service systems that involve multiple stakeholders who interact and work together. The model necessitates inter-organizational relationships across governmental, philanthropic, and academic entities, and CBOs. At the service delivery level, CBOs work together to facilitate training, coaching, and a qualified and well-trained workforce to deliver one or more EBPs to clients. The ICT model was utilized to facilitate the effective implementation and diffusion of SafeCare throughout the service system. The ICT model, by its very nature, necessitates that stakeholders affiliated with different systems, implementing organizations, and teams work together to implement an EBP. Thus, collaboration is an important element of the model, which consists of steps designed to lead directly to the kinds of key implementation supports described in the EPIS framework. The ICT model has some elements in common with other implementation process models such as the ARC (Glisson & Schoenwald, 2005) and the CDT models (CIMH, 2006; Chamberlain, Price, Reid, & Landsverk, 2008). Each of these models describe logically ordered sets of activities designed to create a context in which EBP implementation occurs effectively and the intended public health benefits are realized.

Methods

Overview

The study took place in a large county in the southwestern United States as part of a larger long-term mixed methods implementation study. The county encompasses urban, semi-urban, and rural areas that are home to a diverse cultural mix of residents, including significant Mexican-American and American Indian populations. The county is comprised of specific geographic regions, each with its own local history, demographic and cultural characteristics, climate, and topography. Implementation of any new public human service practice within the county represents a large-scale system and organizational change effort that occurs across all of the regions. We conducted in-depth qualitative individual and small group interviews and focus groups with key stakeholders involved in the early stages of a system-wide implementation of an EBP to reduce child neglect that followed the ICT implementation model. These particular qualitative methods are useful for understanding intervention implementation processes from the vantage points of multiple stakeholders, eliciting perceptions and experiences of outer and inner contextual factors that bear upon these processes, and generating rich, deep, and holistic descriptions documenting both the intended and unintended consequences of implementation.

Qualitative methods were selected and utilized to better understand the perspectives of participants in the scale-up process and so that data would not be bound by investigator conceptions of the process or rigid conceptions about what should be measured. We utilized individual interviews with key stakeholders at higher levels in organizations and service systems in order to allow for candid responses without other’s present. The goal here was to allow each respondent to be comfortable responding to questions and to assure confidentiality. We elected to utilize small group interviews and focus groups with participants from within similar organizational levels (e.g., home visitors without supervisors present). This approach provides confidentiality for staff in relation to their supervisors or others higher up in the organization.

In regard to data analysis, we utilized a combination of open coding and focused coding and we coded data to identify a-priori (e.g., from interview guides) and emergent (e.g., not directly asked about and emerging from the data) themes. Open coding was utilized to identify overall and broad themes and issues. Focused coding was then used to delve more deeply into particular concerns. We describe this process in more detail in the Data Analysis section.

Participants

We invited all outer and inner context stakeholders directly involved in SafeCare implementation in the county to participate. We successfully sampled the entire universe of potential participants. The outer context participants, or service-system stakeholders, included executive staff from the County Child Welfare System (CWS), CBOs providing home visitation services under contract to CWS, and a local foundation (FDN). The inner context stakeholders included key CBO employees such as case manager supervisors, SafeCare trainers/coaches, the seed team (comprised of supervisors and home visitors from two CBOs), and the service provider team members trained by the seed team.

A total of 54 participants were engaged in semi-structured individual interviews, small group interviews (i.e., < 5 participants) or focus groups (i.e., 5 or more participants). The different qualitative approaches (i.e., individual and small-group interviews, focus groups) were utilized for efficiency and to be sure that there was adequate representation of the various stakeholders. Fifteen outer context participants took part in interviews, including CWS administrators (n=3), CBO executive directors (n=3), and FDN leaders and advisors (n=9). Thirty-nine inner context participants took part in focus groups or small group interviews. These included five focus groups with home visitors/seed team members (n=32) and two small group interviews with supervisors (n=4) and trainers/coaches (n=3).

Data Collection

Interviews and Focus Groups

The data collection and informed consent procedures were approved by the University of California, San Diego and University of Oklahoma Institutional Review Boards. The interview guides consisted of open-ended questions tailored to each stakeholder group and interviews were conducted over a two-year period during the Implementation Phase. The interviews with representatives of CWS, FDN, and CBOs assessed the initial planning process, their roles and responsibilities and interactions with one another, and perceptions of overall SafeCare implementation. For example, one interview prompt was: “Describe the relationships and interactions of your organization and the other stakeholders involved in the SafeCare implementation.” Through these interviews data were collected in regard to organizational- and system- level factors affecting implementation. The individual and small group interviews with supervisors, trainers, coaches, and seed team members centered on involvement in the ICT approach, knowledge of and experiences with SafeCare, and the “fit” of the intervention with local populations and service delivery contexts. An example question and probe were: “How has the team been working out since it was formed?” On average, each interview took 60 minutes to complete.

Focus groups or small group interviews were conducted with teams of providers trained in and implementing SafeCare. The group/discussion guides consisted of open-ended questions tailored to providers and the seed team, largely centering on knowledge, acceptance, and experience with SafeCare, and perceptions of the interagency collaborative team and how the team members worked together. For example, one question was: “In what ways do the agencies that employ the home visitors work together to support SafeCare?” The groups/interviews were conducted at CBOs or at locations where team meetings were usually held and lasted approximately 60–90 minutes.

Handwritten notes from the focus groups, small group interviews, and interviews were typed and uploaded into an electronic database. All data collection events were digitally recorded and professionally transcribed. Transcriptions were reviewed for accuracy by at least one of the authors or a research assistant.

Data Analysis

In keeping with conventional approaches to qualitative data analysis (Emerson, Fretz, & Shaw, 1995; Patton, 2002; Strauss & Corbin, 1998), we employed an iterative process to review the textual data from interviews and utilized NVivo9 (2010) qualitative data analysis software to facilitate this work. Data analysis proceeded first by engaging in open coding to locate the themes and issues that emerged from the interview transcripts (Strauss & Corbin, 1998). The empirical material contained in the interviews was independently coded by the project investigators to condense the data into analyzable units. Segments of text ranging from a phrase to several paragraphs were assigned codes based a priori on the particular topical domains and questions that made up the interview and focus guides. During our review of the transcripts, new codes were subsequently identified and defined to capture information on emergent themes. Focused coding was then used to determine which of these themes emerged frequently and which represented unusual or particular concerns to the research participants (Emerson et al., 1995). In the context of the research presented here, focused coded was systematically employed to identify and highlight examples of specific themes related to the topics of collaboration, partnering and partnerships, and interactions between and across different stakeholder groups and organizational entities as part of the SafeCare implementation process.

In this staged approach to analysis, our research team coded sets of transcripts, created detailed memos that both described and linked codes to each theme and issue, and then passed their work to other team members for review. Through the process of constantly comparing and contrasting our codes with one another (Glaser & Strauss, 1967; Strauss & Corbin, 1998), we were able to group together codes with similar content or meaning into broad themes, which were linked to segments of text within the NVivo database. The final list of codes, constructed through a consensus of research team members, consisted of a numbered list of themes which, for the present analysis, placed SafeCare implementation within a framework for understanding collaborations, negotiations, and resolutions while simultaneously considering outer and inner contextual characteristics.

As part of our analytic process, we also triangulated the findings by source, referring here to participant type: CWS administrators, CBO executive directors, and FDN leaders and advisors (Patton, 2002). More specifically, we created a matrix detailing the specific themes pertinent to collaboration and the supporting data provided by participants across types. We then engaged in a side-by-side comparison of the various perspectives, in order to identify points of convergence and divergence in all participant statements related to collaboration, and to avoid privileging voices associated with one participant type over another. Quotations that exemplify the views and concerns of study participants by type as well as commonly shared experiences related to SafeCare implementation are provided below to illuminate the dominant themes related to collaboration. Some quotations were edited to enhance readability.

Results

The major themes that emerged from the analyses addressed a number of areas related to interagency collaboration process and outcomes: organizational culture; changes in organizational strategy; leadership forms and roles; shared authority and responsibility; power struggles and their resolution; role ambiguity; effectiveness of communications; and keys to overcoming implementation challenges. We address each theme in greater detail, highlighting various types and levels of collaboration between CWS and FDN, outer and inner context organizations, and seed teams and provider teams.

System and Organizational Culture

Most stakeholders entered into this initiative with different understandings or assumptions regarding the collaboration. The stakeholders were part of diverse organizational cultures, including those found in large government bureaucracies, large foundations, and small non-profits. For example, CWS is part of a large county health and human services agency that answers to a board of supervisors. As such, there are multiple layers of policies and regulations that influence the flexibility with which CWS stakeholders respond to opportunities and changes in service provision. The CBOs are much smaller organizations that are reliant on multiple funding streams and, while contracts place constraints on their activities, there is typically more flexibility in how these private non-profit organizations operate (Aarons, Sommerfeld, & Walrath-Greene, 2009). Due to the different organization cultures that shaped their work day lives, stakeholders often maintained very different ideas about their respective roles as the collaborative was taking shape, which, at times, led to tension and conflict.

As with many large service system entities, CWS is a large, complex and highly regulated government organization, and therefore change often moves slowly. This collaboration began with the CWS and FDN. Past FDN collaborations had involved smaller initiatives with much lower fiscal commitment. As part of this current collaboration, FDN convened a “Council” composed of community leaders and representatives, including individuals who had worked with CWS in the past or had a strong interest in child welfare concerns. Some stakeholders attributed the implementation slowness to the fact that SafeCare centered on a sensitive topic—child neglect. Others perceived the CWS as assuming a more dominant position because day-to-day services are supported and monitored by county officials. Reticence within CWS to move forward with implementation too quickly was the source of some tension within the broader collaborative. This tension may be explained by the different ways in which in public bureaucracies vs. non-profit organizations operate. A FDN leadership stakeholder observed:

With Child Welfare Service…I know the…Council has lots of interaction with the County…. They’ve worked with the County before…. I would think they would have an understanding of how long County projects might take to change. [The FDN] certainly didn’t have an understanding of the pace that the County moves and how slowly things move through a large bureaucracy.

Change in Organizational Strategy

As part of this implementation initiative, the FDN was experimenting with an alternative business model. Rather than provide small grants to various organizations, FDN officials made the strategic decision to allocate a larger sum of money to the startup of SafeCare in order to eventually achieve greater public health impact through reduction of child neglect. Monies for service provision were to be provided by CWS. Thus, a novel collaboration between funding sources was forged, as the FDN was becoming accustomed to their new model. Commenting on the changes taking place, one Council member noted:

[T]hey (FDN) got [an] enormously valuable lesson from this experience…. I give [the FDN] a lot of credit for staying the course. Gee, it would have been easier just to give away all these [smaller] grants. And the money is gone, and we don’t have to worry about it. Send out one [larger] check and it’s a lot harder. It’s a lot better, but it's a lot harder. Harder work.

The introduction of this new strategy necessitated a shift in organizational culture for the FDN that influenced how the FDN collaborated with the CWS and CBOs contracted to provide the EBP to families in the community. The CBOs charged with implementing SafeCare via their home visitors were both affected by the different organizational cultures/operational styles of the CWS and FDN, in combination with their own particular organizational cultures/styles. For example, communication and contracting approaches of the CWS and FDN differed in regard to flexibility and deliverables. This led to some adaptation and/or accommodation in both outer and inner contexts. For example, a CBO stakeholder noted that:

[It] was three entities having to learn to work together. Just kind of, kind of feel our way along. And the [CWS] has always had some high control needs. And [one CBO] had said to [CWS] a number of times, this is not your issue. This is a contract with [the FDN].

From the FDN perspective, this was also seen as a developmental process. For example, one FDN stakeholder noted that:

[It] was not an easy partnership to start with… [It] was new for [the FDN], and then it was new for the [CWS]…. The County has very strict guidelines on trying to be as fair and equitable [to contractors] as possible. So [the FDN] does business very differently. They’re much more donor focused and board focused. I mean I loved it because it was really a partnership where both sides had to learn a very different culture. They start with the centerpiece, which everyone wholeheartedly agreed was a great model that everybody wanted, so they made it work.

Leadership in Perspective

In an interagency initiative, there may be strong leaders who may need to take super-ordinate or subordinate positions in the social network tasked with EBP implementation. As such, one of the challenges in the ICT project was that many key stakeholders were in positions of power and accustomed to leading. One FDN stakeholder observed:

It’s just that there are a lot of strong personalities involved and there’s a lot of history with [particular people in CWS and CBOs], [so] there’s constantly conflict. And that’s just the way they work together. So some of this is never gonna go away. It’s just how they do business.

One FDN Council member went so far as to describe the early collaboration as: “Politics, politics, and personality and threats.” Such comments underscore some of the difficulties that stakeholders encountered when it came to sharing authority or relinquishing autonomy. These struggles were most evident in the Preparation Phase, but moved toward resolution as the process progressed into and through the Implementation Phase.

Shared Authority and Responsibilities

Related to leadership issues, stakeholders involved in the early stages of SafeCare implementation appeared to have different ideas regarding what shared authority might look like. Some experienced power struggles or lack of agreement on authority over particular aspects of the implementation process. The CWS stakeholders considered themselves responsible for certain aspects of the implementation. For example, regarding the decision as to which team would next receive SafeCare training, meaning the next “cascade,” a CWS participant stated:

We don’t always talk about it with everybody because not everybody gets to make that decision. I think that’s one of the problems, because everybody thinks they can make that decision, but where the cascade is really decided [is] by the County. Period.

In contrast, a FDN stakeholder stated:

The other thing that the County doesn’t want [the FDN] to do, which had a lot of bad feelings in [the FDN], is to meet with any of the partners without County presence.

These statements demonstrate a lack of clear consensus or agreement on the nature of “shared authority” or what aspects of the implementation process were to involve collaborative decision making, and how shared authority might be put in place. Each entity in the collaborative could claim legitimate authority for the right to involvement in aspects of decision making. For the FDN, it was the responsibility to donors for the financial resources expended. For CWS, it was the statutory obligations to manage the system and accountability in the public eye. For the CBOs, it was the reality of staffing their individual programs and clinical duties to their clients.

Role Ambiguity

Another challenge to the early stage collaboration concerns the ambiguous nature of roles and responsibilities. There is a vast literature on the negative impacts of role ambiguity and lack of role clarity in business and management (Bliese & Castro, 2000). Ambiguity in the Preparation Phase led to some members asserting that roles were not previously agreed-upon, which caused some resentment between stakeholders. For example, while all of the selected agencies held CWS contracts for the provision of services, some also had contracts for ongoing training and coaching functions. Thus, contracts involving the FDN, the CWS, and CBOs were complicated and the complexity led to different interpretations about how to collaborate on roles and responsibilities.

The contract negotiation process and instantiated contracts served as key mechanisms for defining organizational roles, facilitating implementation, and working through the complexities. Yet, both the process and contracts caused a strain on stakeholder and organizational relationships. For example, early roles and fiscal responsibilities and obligations were negotiated with regard to service providers reducing clinical duties and increasing training and coaching duties, thus reducing caseloads and the number of families served. These fiscal and operational concerns were resolved through a decision-making process that allowed for FDN funding supporting a large proportion of early implementation activities, particularly training and coaching, with the CWS assuming financial responsibility in the latter part of the Implementation Phase and into the Sustainment Phase.

In the Preparation Phase, contracts involving the FDN, the CWS, and the CBOs were developed and modified through negotiation. In the beginning, there was lack of clarity regarding which of the initial responsibilities were under the purview of the FDN and CWS. Complicating matters, the service contracts were between the CWS and CBOs, while the training and seed team contracts were between the FDN and CBOs. There were additional contracts and training/coaching support from academic trainers and the NSTRC that may have resulted in some role confusion for the seed team providers. For example, it may not have been clear to them where ultimate expertise regarding SafeCare resided (academic partners, NSTRC coach, their supervisor, the CWS, the FDN). This created ambiguity regarding to whom the CBOs answered. A CWS stakeholder described the tension as a “strange triangulation” between stakeholders:

Where the problem has been is really sort of the County—the County thinking that they own the [FDN] contract. And it’s [the FDN]’s money. The contract I have is with [the FDN]. That’s who I need to be answerable to.

A FDN stakeholder also weighed in:

There needs to be some clarification of what everybody’s role is, [in] particular what the County’s role is…. But when it comes down to it, our relationship with Agency A is between [the FDN] and Agency A…. And the County [has been] very upset about that. They wanted to approve the invoices first and then just tell us it was okay to pay. Being stewards of our donors’ money, I wasn’t comfortable with that. But they have been very adamant that they don’t ever want us to have a conversation with [the CBO] or anybody else if they’re not in the room.

As is common in community-based services, simultaneous sharing and working within separate spheres, i.e., private foundation, public bureaucracy, non-profit service organizations, presented challenges for the ICT stakeholders. For example, the FDN required the creation of measureable goals, documentation, and data to report to their donors. However, there was a perception that it was not within the purview of the CWS to provide those FDN specific measures. One CWS stakeholder suggested that it was important for her and her colleagues to control dissemination of the information related to the implementation as the CWS was liable for political fallout that could result from data being reported inaccurately or inappropriately (e.g., identifying data). In contrast, FDN stakeholders expressed the need for basic outcome data (e.g., client recidivism rates) to answer to and satisfy their donors. Still, the academic partners had the role of advising in regard to empirically defensible approaches to collecting, analyzing, and reporting data.

Communication Effectiveness

Another factor that emerged as important to the collaborative process was identifying communication breakdowns related to ambiguous roles and responsibilities. Despite a history of work in the same outer context of the service system, there was no clear structure of communication between all stakeholders throughout the phases of Exploration, Preparation, and Implementation. For example, the types and schedules of communications, such as in-person meetings, conference calls, or emails, were not consistently specified and decided upon in advance. Communication breakdowns were most apparent during the Implementation Phase. The problems caused by the lack of such a structure filtered all the way down to the seed team, whose members did not know who to contact to ask questions about implementation or report information. Other stakeholders reported feeling “out of the loop” due to the absence of a clear communication structure. Assumptions that people will get copied on, or forwarded, emails and find out information, were not always borne out in reality. While some stakeholders had long established relationships with one another, others were less embedded in the communication network. In one instance academic partners arrived for a meeting that had been canceled. Communications were so poor that not all stakeholders were informed of the cancellation, illustrating that some stakeholders had an assumed position on the periphery of the inter-organizational network.

Surmounting Implementation Challenges

A number of factors were important in overcoming challenges to collaboration. First, the stakeholders all felt very strongly about the appropriateness of, and their support for SafeCare. Within the outer policy context, the CWS had a strong agenda to move toward evidence-based approaches, and of the models presented, SafeCare was seen as the best fit. However, although there were a number of smaller scale and single case design studies, at the time of the implementation, definitive effectiveness (i.e., large-scale controlled trial) data for SafeCare was still forthcoming. The academic partner presenting SafeCare was perceived by the stakeholders as authoritative, balanced, and convincing in the description of the intervention and its potential to reduce child neglect. This assessment was also supported by an independent clearing house. One participant noted:

When [he/she] speaks, people listen because [he/she's] so good. And so the fact that [he/she] was a major leader…and it was ranked again, not by the council but by this independent clearing house. So we decided to go with that, and test that model…

Second, the stakeholders were able to make the conceptual leap from the idea of SafeCare to the implementation and practice of SafeCare. This was facilitated by the high degree of potential for fit with the CBOs organizational processes, home visitors need for appropriate structure in service delivery, and the needs of client population (i.e., prevalence of child neglect). A CWS leadership stakeholder noted,

The training went well and implementation was successful… The reception actually at the line level has exceeded my expectations that these home visitors are comfortable with embracing more of a script and more structure within the visit and that they’re receptive to the coaching. And from the supervisor to the manager in the field that they’re happy with it, I mean that to me is a gift.

The stakeholders appeared to have shared a strong common vision and a realistic view that it takes time to implement an EBP effectively, in order to see the training through and see results, despite the initial challenges. One CBO director stated:

I think [the training went] as well as they would have expected. [Nothing] happened that I was really shocked [about]—nothing that doesn’t usually happen when you’re trying to get something going and [you have] a lot of players and it’s something new. And in this piece you had people [i.e., intervention developers and academic partner trainers] come in from out of town. So that even went smoother than I thought it would be. We’re all used to our own kinds of [interactions] and then you throw in someone from the outside, and it seems to go fine.

One of the key factors in any large-scale change is the perseverance needed to see things through. System and organizational culture change takes time and there were many aspects to culture change. The CWS intended to move to a culture that welcomes and encourages EBP, the FDN changed their culture regarding the types and scope of initiatives to support, while CBOs and their providers began to embrace a culture that utilizes and values more structured and evidence-based interventions in their direct work with children and families. The importance of perseverance was remarked upon by one CBO stakeholder:

But you know there were some things where you couldn’t get straight answers about things in the very beginning, but just like everything else, once you start going and you start doing things, the answers come.

Resolution of Power Struggles

Despite some power struggles that were challenging for the implementation and collaboration process, CWS stakeholders relaxed some oversight after there were indications that the initial EBP implementation was progressing well. For example, concerns with the implementation process (i.e., training completion, ongoing coaching, and meeting caseload requirements, having appropriate referrals) eased once implementation targets were beginning to be achieved. As one CBO stakeholder observed:

Their [CWS officials’] fear has subsided. They’re seeing it [the intervention] is working. They are seeing that they’re not getting cheated or embarrassed. They are seeing that there is a system in place. They’re just calmer. And so their control needs are a little less.

Thus, while preparation and early implementation were characterized by some tensions, these eased over the latter part of the Implementation Phase.

Finally, this implementation was judged by the investigative team to be “effective” because the appropriate reach/penetration of the intervention in the service system was realized (i.e., eligible providers in participating CBOs in the system were trained and certified, had capacity to deliver the EBP, and eligible clients with open cases deemed appropriate for the EBP were receiving the EBP). In addition, there were sufficient coaches so that the required fidelity monitoring and coaching (monthly for established providers) was being conducted and the level of fidelity (according to systematically collected client report) was high.

DISCUSSION

Several challenges affected stakeholder collaborations across the outer and inner implementation context. As a function of alignment with their respective organizations and constituents, stakeholders experienced initial shifting of leadership, meeting organizational needs, and shared responsibility for the implementation process. Negotiations were complicated by differing interpretations of each stakeholder’s role in the implementation process, ambiguity and some contestation over decision-making authority, communication breakdowns, and variability in levels of trust. While the process resulted in eventual stakeholder coalescence and collaboration, each of these factors impacted how service provider organizations approached the EBP implementation process.

We found that, in practice, the notion of collaboration actually represents a complex process of burgeoning interest in an initiative that moves through discussion and negotiation, entails consideration of competing priorities, accurate or inaccurate assumptions, and structures and processes that can either facilitate or hinder effective movement through the EPIS model phases of Exploration, Preparation, and Implementation (Aarons et al., 2011). Stakeholders had different motives for collaboration. For example, those from the CWS were particularly concerned with reducing the likelihood of child neglect, operating in a way consistent with the mission of CWS, and maintaining ongoing working relationships with academic, FDN, and CBO stakeholders. The FDN was embarking on a philanthropic strategy with a higher degree of risk than their usual approach of providing smaller targeted funds and utilizing a more sweeping and potentially higher impact strategy for improving the lives of children and families. The CBOs were able to join in providing services with scientific evidence of effectiveness and at the same time attain funding to support their organizational missions. Collaboration and negotiation were key elements in navigating these missions and relationships.

While many issues emerged during the Exploration Phase, they were most apparent during the Preparation Phase and peaked during the early Implementation Phase. Multi-staged models that address conflict and resolution in implementation consistent with EPIS occur across different types of innovations and change (Robey & Farrow, 1982). A rapprochement and coalescence became more evident as implementation progress was being made, and additional cascades (i.e., sequential training of new teams) occurred so that the ICT implementation approach was seen as successful in scaling up SafeCare and services were being provided to clients across the entire service system.

As noted earlier, collaboration can be complicated when stakeholders emphasize different sets of values, do not participate in the same organizational cultures, or when collaboration appears to require some individual stakeholders to sacrifice valued organizational habits, interests, and beliefs in favor of the collective whole. Development of a distinctly identifiable organizational culture is believed to be a primary key to the success of any business or, in this case, a collaborative group. Strong organizational cultures can reduce uncertainties, create group identity and cohesion, and facilitate the long-term effectiveness (i.e., sustainment) of initiatives undertaken by a collaborative (Alvesson & Willmott, 2002; Cameron & Quinn, 2011). Research in public sector allied health settings illustrates the influence of organizational leadership, cultures, and climates on attitudes and approaches to the adoption of EBPs (Aarons, 2006; Aarons & Sawitzky, 2006; Glisson et al., 2012). Collaborative enterprise can articulate an overarching culture or set of shared beliefs and values, that individual stakeholders and organizations can all buy into, and under which the collective effort of diverse stakeholders can occur. This process could define “common ground,” or it could indicate a majority view that the corresponding minority views may come to accept or embrace, or will at least tolerate. Thus, perceived cultures of collaboration depend on the perspectives of participating stakeholders or stakeholder groups.

Collaboration may be constrained, particularly in situations where one partner is a highly regulated governmental agency vested with statutory responsibilities for certain decisions, with limited degrees of freedom, or with high potential public visibility in the event of poor outcomes or failure. For example, child-welfare systems are intrinsically responsible to directives within a larger bureaucratic structure (e.g., federal mandates, governors, state legislatures, health and human services agencies, boards of supervisors, commissions, etc.). In these circumstances, clear communications and negotiations are needed to determine where, when, on what issues, and how much latitude for decision-making and collaboration is needed. One challenge in these kinds of collaborations can be discriminating whether a stakeholder is reluctant to share decision making because they are uniquely bound by law or policy, or risking public exposure, or because of other operational types of reasons such as contracting processes and cycles.

There is an ongoing and increasing level of governmental contracting with private and public non-profit organizations. This mutual dependence between nonprofit service organizations and government agencies can blur the lines between public and private (Smith & Lipsky, 1993). In regard to EBP adoption and implementation there is evidence that the type of contracting organizations, level of organizational support for EBP, and specific structures and processes that support an EBP are critical to adoption and use of EBP (Aarons, Sommerfeld, & Walrath-Greene, 2009). Thus, it is not only collaboration and communication, but also how individual CBOs instantiate EBP within their organization under varied regulatory and funding constraints.

While stakeholders in the outer context (in this case CWS and FDN) may desired a particular change in practice, all stakeholders participated so that even in the Exploration Phase, systemic changes were more likely to translate appropriately from outer to inner context through policies, funding, and contracting. That is, policies that exist and are developed, or are modified in the outer context, are often translated into memorandums of agreement, memoranda of understanding, and/or service contracts that specify what functions, processes and deliverables are expected through the implementation phases. Such a process can contribute to the instantiation and institutionalization of an EBP into the inner context in a sustainable way (Steckler, Goodman, McLeroy, Davis, & Koch, 1992).

Although outer and inner context relationships can be reciprocal, there is an element of “following the money” that impacts what specific services are available to serve clients in public sector services. That is, non-profit contracting organizations and CBOs rely on a number of funding sources including service system and foundation grants and/or contracts. In the present implementation study, some non-profit CBOs tasked with learning, implementing, and delivering SafeCare were beholden both to CWS and to the FDN, while others were beholden only to the CWS with associated differences in responsibilities. Although this dual-sourced funding increased the complexity of stakeholder relationships, the buy-in and support from the CWS (for service provision funds) and the FDN (for training and coaching funds) resulted in a relatively effective move from Exploration through Implementation Phases, and made possible the county-wide implementation of SafeCare in this large public service system. Recommendations for building capacity are relevant for this discussion and include specifying appropriate governance structures, developing decision-making frameworks, aligning stakeholder interests with a larger vision, balancing short-term and long-term objectives, and securing resources for sustainment (Alexander, Christianson, Hearld, Hurley, & Scanlon, 2010). Consistent with these recommendations and within the four EPIS phases, it became evident that ongoing communication, a problem-solving orientation, buy-in, enthusiasm, and support for the ultimate and overriding goal of improving care for children and families were key factors in the resolution of power struggles.

Limitations

Some limitations of the current study should be noted. First, we focused primarily on stakeholder relationships at the organizational level. While there are also issues of collaboration at the clinical and service provider level, the purview of this study is also on those larger system and organizational issues that facilitate successful collaboration, EBP implementation, and use of the ICT scale-up strategy. Further study is needed to identify how these larger organizational factors influence stakeholder relationships and also service providers and clients. Second, this study was conducted in one large service system with a group of stakeholders focused on EBP implementation and improving outcomes for children and families. The degree to which the findings will generalize to other settings, groups of stakeholders, or EBP implementation efforts is not clear. Further studies are needed to determine the degree to which these results are consistent across settings. Finally, the scale-up is a work in progress. During this stage of our research, conclusions regarding the degree to which the ICT approach actually facilitated implementation would be premature. It could be that more challenges would have been experienced or that coalescence might have been more difficult to achieve without it and this should be examined in future analyses. In addition, the comparative effectiveness of the ICT approach versus other strategies (e.g., ARC, CDT) is unknown. Future studies should examine these relationships and issues through the end of the Implementation Phase and through the Sustainment Phase.

Recommendations

There is a consistent demand for service improvement and a growing call for EBP implementation in federal, state, and local service systems. In some cases service systems set agendas, issue requests for proposals, and award contracts for new services. It is sometimes the case that these edicts are fast-moving and require quick response from CBOs that seek ongoing funding through contracts. We recommend that the process include space for reflection and recalibration along the way. Stakeholders can build in time and processes to reflect and consider factors in both the outer and inner context that may affect implementation in each EPIS phase (Aarons et al., 2012). For example, in the Exploration and Preparation Phases, there is need for a thorough consideration of the fit of a particular EBP with the outer context of the service system and structures and processes within CBOs comprising the inner context (Greenhalgh, Robert, Macfarlane, Bate, & Kyriakidou, 2004; Klein & Sorra, 1996). It may be that adaptations are needed to service system and organizational structures and processes to support the selected EBP (Aarons et al., 2012). There should also be preparation and planning for how ongoing training and fidelity monitoring will be built into services. Ongoing training is needed to address the ever-present staff turnover that is seen in service systems and to ensure that fidelity monitoring and coaching can be utilized to support staff in delivering EBPs in a practical way that will lead to better staff and client outcomes (Aarons, Sommerfeld, Hecht, et al., 2009). Because of this, service systems, funders, and CBOs need to partner to address implementation in a way that is workable and facilitative for all involved.

Clear structures and processes for communications that keep all stakeholders informed and up to date should help to address concerns with players being central or peripheral in the implementation process. It is important to clearly articulate who answers to whom, what communication methods are to be utilized, and how to best facilitate a sense of inclusion and support from the beginning. This includes a clear order and process of communication for stakeholders that directly relates to the each one’s different ideas about how to support collaboration. It is important to recognize that scale-up takes time and patience to effectively implement one or more EBPs. During this study, we documented diverse stakeholders overcoming their challenges to collaboration and eventually coalescing around a common goal despite differing roles, responsibilities, and organizational realities.

Future research should develop and test approaches to facilitate effective collaboration and negotiation during system and organizational change. Some extant approaches include community-based participatory research (Roberts, 2013), or community partnered research (Wells & Jones, 2009). These approaches are often focused on academic-public collaborations and, while this implies a consideration of both organizations and individuals, an expansion of their purview may be fruitful. One future direction might be to delve more deeply into system and organizational concerns as well as the social and cognitive processes that may improve the efficiency of collaborative efforts. Another is to delve further into the negotiation process as it influences systems, organizations, and individuals on interpersonal and intrapersonal levels (Thompson, Wang, & Gunia, 2010). While some measures exist, the further development of methods and metrics to assess the presence, strength, or effectiveness of collaborations might also advance implementation science (Cross, Dickmann, Newman-Gonchar, & Fagan, 2009; Dedrick & Greenbaum, 2011). Such directions may help to illuminate the “black box” of collaboration.

Clinical relevance

The uptake and use of EBPs in usual care clinical settings is being strongly influenced by policies set by federal, state, and county mental health and social services agencies. The complex interrelationships among policy, administration, clinical supervisors and service providers comprise the backdrop for improving clinical services through EBP implementation (Green & Aarons, 2011). Implementation can be more effective when the agendas and key issues near and dear to the hearts of various stakeholders have greater congruence. In addition, when agendas and interests in the outer context are congruent with inner context issues of EBP fit with the values and needs of clinicians and clients, implementation will be more likely to be effective (Klein & Sorra, 1996). For example, service system stakeholders in the outer context may have identified a need to reduce child neglect. Engaging stakeholders (including clinical supervisors, providers, and client groups) in a collaborative process to identify and vet EBP selection may help to provide a better “innovation values fit,” signifying the appropriateness of the particular EBP for the outer system issues of funding, quality assurance, contracting, reimbursement for services, and inner context issues of CBO reimbursement, service context (e.g., clinic-based vs. home-based services), and providing an appropriate model that will meet the needs and diversity of clients.

Conclusions

System wide scale-up of EBPs involves multiple stakeholders in a complex process that provides a nexus for differing agendas, priorities, leadership styles, and negotiation strategies. The term “collaboration” oversimplifies the multifaceted nature of the scale-up process. Implementation efforts should consider the complex agendas, priorities, and interaction styles of organizations and individual stakeholders and allow for facilitative resolution of the concerns of each participant in the process. However, for implementation science a redefinition of the term “collaboration” is required. Collaboration should denote the entire process of coming together to collaborate, identifying differences in needs and agendas of stakeholders, processes by which differences are, or are not, resolved, and the impact of these processes on implementation. Multiple terms and expressions can capture such aspects or differing processes and tensions inherent in collaboration including cooperation, confrontation, negotiation, compromise, concession, conciliation, finding a middle ground, and give and take. This list is by no means comprehensive and it is difficult to identify a single term to characterize the complexity of the collaborative process. Our findings support the need for a deeper understanding of collaboration that can help to illuminate EBP implementation planning processes across the EPIS phases that occur in the outer and inner contexts of complex human service settings.