The power of a collaborative relationship between technical assistance providers and community prevention teams: A correlational and longitudinal study

Abstract

Background

Historically, effectiveness of community collaborative prevention efforts has been mixed. Consequently, research has been undertaken to better understand the factors that support their effectiveness; theory and some related empirical research suggests that the provision of technical assistance is one important supporting factor. The current study examines one aspect of technical assistance that may be important in supporting coalition effectiveness, the collaborative relationship between the technical assistance provider and site lead implementer.

Methods

Four and one-half years of data were collected from technical assistance providers and prevention team members from the 14 community prevention teams involved in the PROSPER project.

Results

Spearman correlation analyses with longitudinal data show that the levels of the collaborative relationship during one phase of collaborative team functioning associated with characteristics of internal team functioning in future phases.

Conclusions

Results suggest that community collaborative prevention work should consider the collaborative nature of the technical assistance provider – prevention community team relationship when designing and conducting technical assistance activities, and it may be important to continually assess these dynamics to support high quality implementation.

Background

There has been increasing use of community collaborations and teams as organizing units to implement prevention programs and advocate for change in policies related to the prevention of health risk behaviors over the last 20 years (Butterfoss, Goodman, & Wandersman, 1996; Chinman et al., 2004; Greenberg & Feinberg, 2002; Hallfors, Cho, Livert, & Kadushin, 2002; Hawkins et al., 2008). Indeed, federal dollars to implement prevention programming or to work for changes in policies (e.g., Weed and Seed and Drug Free Communities) have required the formation of community coalitions as part of the implementation process (Community Capacity Development Office, 2005; Substance Abuse and Mental Health Services Administration, 2014). However, the effectiveness of these efforts has been somewhat mixed, which is likely in part due to the quality of team processes limiting or supporting what the coalition can achieve (Hallfors et al., 2002).

Recent research has started to link the quality of prevention team functioning to outcomes concerning the quality of work products and sustainability efforts (Brown, Feinberg, & Greenberg, 2010; Perkins et al., 2011; Spoth, Clair, Greenberg, Redmond, & Shin, 2007). Given the possible connections between prevention team functioning and outcomes of the team's efforts, a valid question is: what factors promote high quality community prevention team functioning? In the study described in this paper, we analyzed data collected over the first 4.5 years of the PROSPER (PROmoting School-university-community Partnerships to Enhance Resilience) trial (Spoth, Greenberg, Bierman, & Redmond, 2004) to closely examine a key potential predictor of community prevention team functioning. Specifically, we examined the degree to which a collaborative approach to technical assistance was related to the quality of team functioning.

Defining Technical Assistance

Technical assistance (TA), or the support and assistance that a prevention effort receives from someone or some organization that is not a part of a community team, has been theorized as very important in supporting high quality implementation of prevention programs specifically, and prevention systems more generally (Chinman et al., 2005; Forman, Olin, Hoagwood, Crowe, & Saka, 2009; Mitchell, Florin, & Stevenson, 2002; Wandersman & Florin, 2003; Wolff, 2001). A TA provider typically has specialized knowledge, experience, and expertise in the issues that are salient to such efforts that likely would support improved outcomes. Despite apparent consensus that technical support is an important aspect of prevention programming, there is less agreement on exactly what types of activities technical support should include. A review of the literature related to TA for prevention programming reveals that TA providers commonly employ a wide variety of techniques including, but not limited to: training, coaching, consulting, supervising, modeling, problem solving, providing feedback, supporting, instructing, demonstrating, and assisting with evaluations (Becker, Bradshaw, Domitrovich, & Ialongo, 2013; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005; Pas, Bradshaw, & Cash, 2014; Stormont & Reinke, 2014). Creating supportive interpersonal relationships seems to be assumed across each of these aspects of TA (Kilburg, 1996), yet the degree to which supportive interpersonal relationships occur likely varies. Consequently, the current study examined the quality of the collaborative relationship between TA providers and lead prevention implementers that were part of the PROSPER project (Spoth et al., 2004).

Effects of Technical Assistance

There is relatively strong evidence that implementation of prevention programs is of higher quality when supported by TA (Catalano, Berglund, Ryan, Lonczak, & Hawkins, 2004; Durlak & DuPre, 2008; Fixsen et al., 2005; Kelly et al., 2000; Olson, 2010; Rabin et al., 2010; Spoth et al., 2013). Preliminary research on the effectiveness of TA systems has focused largely on the degree to which such support affects both the quality of program implementation and overall program effectiveness. Results of such studies have suggested that a wide variety of forms of TA have been associated with improved program implementation. Although implementation quality could be operationalized broadly, and could include multiple characteristics such as overall quality of instruction within a prevention program, time management, and individual capacity or preparation to implement prevention strategies (Becker et al., 2013; Chinman et al., 2008), it has most commonly been defined as the degree to which a program has been implemented with fidelity to the original program model (Fox, Hemmeter, Snyder, Binder, & Clarke, 2011; Noell et al., 2005). Research on the links between TA and implementation quality have yielded mixed results, with some researchers finding that more TA is better (Chinman et al., 2008; Stevenson, Florin, Mills, & Andrade, 2002), and others finding a more complicated relationship between TA and outcomes (Becker et al., 2013; Feinberg, Chilenski, Greenberg, Spoth, & Redmond, 2007; Feinberg, Ridenour, & Greenberg, 2008; Mihalic & Irwin, 2003; Mitchell, Stone-Wiggins, Stevenson, & Florin, 2004). Given that empirical research focused on the mechanisms through which positive effects of TA might occur is in its formative stages, it is not yet possible to make firm research-based decisions on how exactly to structure the TA process for community collaborative prevention efforts.

Despite limited information regarding the exact components of successful TA efforts, evidence seems to indicate that such support holds promise as a way to improve both the quality and effectiveness of evidence-based programming efforts. Nevertheless, many prevention stakeholders remain resistant to adopting formal TA systems. One primary reason for such resistance is likely due to the fact that providing TA, either at the program- or the community coalition-level, can add substantial costs to an already significant investment in implementing new evidence-based programs. Indeed, our experience with PROSPER suggests that many stakeholders continue to question the cost-effectiveness of adding these “infrastructure” costs to direct program implementation costs.

Implementation-related outcomes

A growing body of literature has focused on the effects of TA on the quality of implementation of packaged prevention programs. Most of these studies have found small to moderate positive relations between TA activities and implementation with fidelity to the underlying program model. For example, Becker and colleagues (Becker et al., 2013) examined the effects of coaching on teachers' implementation of the Good Behavior Game. Coaches engaged in a wide variety of TA activities, including regular check-ins with teachers, needs assessments, modeling of proper implementation, and other forms of rapport building and supplemental support. Quality of implementation was assessed through subjective ratings of independent observers using a 29-item rubric. Results indicated small but statistically significant improvement in implementation fidelity among teachers who received TA that was tailored to their unique needs. Effects of coaching appeared to be particularly strong among teachers who started out with implementation scores that were at the lower end of the spectrum.

Other studies have revealed similar findings. For example, Fox and colleagues (Fox et al., 2011) found that TA in the form of professional development for teachers such as workshops, implementation support, and performance feedback— was associated with ratings of improved program fidelity. However, such findings may not be broadly generalizable, given that the study focused on a very small sample of three teachers.

There is some evidence that the effects of TA on implementation vary based on the intensity of the support offered. For example, Noell and colleagues (Noell et al., 2005) found higher levels of implementation fidelity associated with a school-based behavioral intervention among teachers who received more intensive support as compared to those who received simple weekly check-ins. In this study, the intensive TA included tailored performance feedback in which consultants worked closely with teachers to assess implementation of the intervention and recommend strategies to improve treatment integrity.

Behavioral Outcomes

Several researchers have focused on relationships among various types of TA and program outcomes. To date, results have been mixed, with findings from some studies indicating significant relationships between TA and positive outcomes among program participants. Other studies, however, have revealed no or mixed effects. For instance, one study found that a combination of consultation with teachers and performance feedback related to intervention implementation resulted in a variety of positive outcomes among those who were implementing classroom management strategies (Reinke, Lewis-Palmer, & Merrell, 2008). Specifically, teachers who received tailored feedback were more likely to use behavior-specific praise and less likely to use reprimands in their classrooms. Furthermore, higher levels of consultation and tailored feedback were associated with decreased levels of disruptive behaviors among students.

Another study examined outcomes associated with the coaching of teachers who were implementing the All Stars prevention curriculum (Dusenbury et al., 2010). Results of the study indicated that coaches engaged in a wide variety of TA activities, and that many of these activities were well-received by the teachers. Follow-up analyses revealed mixed findings related to the effects of coaching on student outcomes. Several coaching topics, including how to get parents involved and how to reach high-risk youth, were related to decreased drug use among program participants. However, coaching also seemed to have small iatrogenic effects on student commitment to the program. More detailed analyses of similar data revealed no meaningful effects of coaching on program participant outcomes (Ringwalt et al., 2009).

Mechanisms underlying the effects of technical assistance

Given the relatively recent emergence of this line of inquiry, researchers have yet to examine which of the many aspects of TA are most critical in promoting positive outcomes. Based on the reviewed studies, it appears as if program-level TA is most likely to impact both program implementation and behavioral outcomes when the dosage of support is beyond a certain threshold. Indeed, the positive effects noted above only occurred under conditions of intensive coaching with feedback that was tailored to individual teacher needs (Dusenbury et al., 2010; Noell et al., 2005; Reinke et al., 2008).

Results of studies focused specifically on the effects of TA dosage on community prevention board functioning have also been mixed. For example, Feinberg and colleagues (2008) found that dosage of TA had little effect on overall board functioning, although it did appear to have some positive effects among newly formed boards. Similarly, Mitchell and colleagues found little evidence of a link between amount of TA and the functioning of community health coalitions, although general coalition functioning did improve over time when supported by TA efforts, regardless of dosage (Mitchell et al., 2004).

In light of the equivocal findings related to dosage of TA, it appears as if other characteristics of TA are important. In recent years, a growing body of literature has focused on the nature of the relationship between the TA provider and program implementer. Indeed, some scholars have underscored the importance of developing strong connections and rapport with stakeholders in similar contexts (Kilburg, 1996; Wasylyshyn, 2007). Consequently, TA may be more effective when providers approach interactions with stakeholders in a positive and friendly manner that is encouraging, supportive, and sensitive to unique circumstances (Fixsen et al., 2005; Stormont & Reinke, 2012). In the business literature, research has suggested that executive coaching is more likely to be successful within the context of a supportive, collaborative relationship (Kilburg, 1996; Wasylyshyn, 2007). However, research in fields such as education and prevention science are just beginning to emerge (Domitrovich et al., 2015). The purpose of the study described in this paper was to increase our understanding of the degree to which a collaborative approach to TA can promote well-functioning prevention coalitions.

Current Study

The current study uses 4.5 years of data from the PROSPER project (Spoth et al., 2004) to closely examine the TA process over time. The PROSPER delivery system for empirically-validated prevention programs connects local Cooperative Extension System Educators with a representative of the local public school system to build a community team that generally assesses the health and well-being of their youth and families. This community team meets monthly and is connected to appropriate education and prevention resources at the university and state-level by Extension Prevention Coordinators (PCs) who provide a wide variety of TA to the community team (see Figure 1).

Figure 1. The three-tiered PROSPER organizational structure.

*Note: Dashed lines represent intermittent direct contact; solid lines represent regular direct contact.

This is largely an exploratory study given that the empirical investigation on TA in collaborative community prevention initiatives is in its formative stages. Specifically, this paper will examine the contribution of the collaborative relationship between the TA provider and the community prevention team to the quality of team functioning across different phases of team development: the organizational or planning phase; the operations / implementation phase; and the sustainability phases of collaborative team functioning.

We chose to investigate how the collaborative relationship between the TA provider and the community prevention team associates with team functioning at different phases for multiple reasons. First, as others do, we conceptualize that community prevention teams go through a developmental process that is somewhat predictable (Livet & Wandersman, 2005; Stevenson & Mitchell, 2003). The first phase, the organizational phase, typically lasts 6-8 months. This is where members are recruited, the vision and mission are formed, and basic operating procedures are decided upon and put into place. Next comes the operations / implementation phase, where team member efforts are focused on tangible work products, whether it be implementing programs, or work to change policies, or other activities. Next comes sustainability, or some refer to this next stage as institutionalization. In this phase team member efforts continue to implement programs or policies that were started in the earlier stage and have started to become part of regular procedures; implementation typically becomes much more efficient in this stage. The new work of this stage focuses on taking steps to ensure that the team's programs or policies are sustained well into the future. This work frequently entails searching for reliable and consistent funding mechanisms, and coordinating with historically strong organizations.

Consequently, the work of a community team when it is just beginning to form is quite different from the work of a community team that has been in existence for a period of time. Likewise, the relationships and trust among community team members (and likely their TA providers) change over time as team members build a history of working together, and different members cycle on and off the community team. To a degree, earlier research has shown that different community, team member, or team characteristics are more important in predicting high quality team functioning when community prevention teams are in different phases (Feinberg et al., 2007; Greenberg, Feinberg, Meyer-Chilenski, Spoth, & Redmond, 2007). This study continues and expands this work to investigate the role of technical assistance.

Team functioning is an important proximal outcome of community teams that has been shown to relate to more distal indicators of effectiveness (Brown, Feinberg, Shapiro, & Greenberg, 2013; Feinberg, Bontempo, & Greenberg, 2008; Perkins et al., 2011; Spoth, Clair, Greenberg, Redmond, & Shin, 2005). Internal team functioning and relationships with organizations external to the team are the outcomes examined in this study. We hypothesized that the collaborative relationship between the TA provider and the community team/team leader will predict better internal and external team relationships. The structure of the data allows us to follow all teams from the inception of the project and allows us to investigate how TA during the organizational and implementation phases relates to outcomes in each subsequent phase of team development.

Method

The 14 communities involved in the current study were the intervention communities that were randomly selected from a pool of 28 potential communities to participate in the PROSPER project in Pennsylvania and Iowa (Spoth et al., 2004). Eligible communities had: (a) total school district enrollment between 1301-5200 students located in a non-metropolitan area; (b) a minimum of 15% of students eligible for free and/or reduced lunches; (c) less than 50% of the population employed by or attending a university; (d) not been involved in any other university-affiliated youth-prevention research projects; and (e) to have both a school district and a county Extension Educator that were willing to be involved in PROSPER programming. The participating universities' Institutional Review Boards approved the study. Measures of community characteristics, team processes, implementation quality, and student outcomes were collected from multiple reporters and at multiple levels at multiple occasions. To date, PROSPER intervention communities have shown significant improvements in family functioning, student attitudes, student problem behaviors relative to control communities (Redmond et al., 2009; Spoth, Redmond, et al., 2011), improvements in indicators of a community's social capital (Chilenski, Ang, Greenberg, Feinberg, & Spoth, 2014; Mincemoyer et al., 2008), and success with family recruitment, implementation fidelity and sustainability (Greenberg et al., 2015; Spoth, Guyll, Redmond, Greenberg, & Feinberg, 2011). The current study extends these analyses to assess the TA process that is likely a key component to achieving these outcomes.

Technical Assistance Model

Across all phases of team development, PROSPER intervention community teams participated in a series of formal and informal activities as part of a proactive TA model. The components for this model were drawn from a variety of sources, including the proactive TA model developed and used by the Blueprints for Violence Prevention initiative (Mihalic, Fagan, Irwin, Ballard, & Elliott, 2004). In the PROSPER model (see Figure 1), Prevention Coordinators (PCs) are generally conceived to be the linking agents between the community-level teams and state/university-level knowledge and resources (Spoth et al., 2004). More specifically, the job description of the PC entailed seven main components outlined below. First, each team was assigned their own primary TA contact, called a Prevention Coordinator (PC), who was an expert in implementing evidence-based programs in a variety of settings. Teams were introduced to their PC at the first PROSPER model training and allotted approximately 25% FTE to support each of their assigned community teams. Second, Prevention Coordinators were instructed to attend the monthly meetings of each of their assigned community teams. Their role at these meetings was to provide support to the team leaders and other community team members by answering questions that team leaders could not answer, providing new information from state-level project leadership, and addressing questions or concerns that developed during the meeting that required further input from the state-level leadership.

Third, Prevention Coordinators were instructed to have, at a minimum, a regularly-scheduled contact with their community teams every other week. This type of regularly-scheduled contact was a fundamental component of the Blueprints TA model, and is considered to be a key component that facilitated timely progress and increased success in implementing prevention programs as a part of the Blueprints initiative (Mihalic et al., 2004). The Blueprints team found that the proactive nature of their TA efforts, in which TA providers actively reach out to prevention teams rather than waiting for questions from the teams, was a key contributor to implementation success. Within PROSPER, this type of proactive TA was facilitated and documented by a biweekly semi-structured report that would often be conducted as an informal discussion over the phone between the PC and Team Leader. Generally, team leaders provided updates on implementation progress during these exchanges, including reviewing major accomplishments, discussing problems and challenges, and planning for next steps, and conclude with the opportunity to mention any TA needs that had not yet been broached through the course of the conversation. Fourth, team leaders and PCs were expected to have additional unscheduled communications; this occurred often.

Fifth, all available intervention community teams within a state came together once each year for a statewide meeting. Wise and Ezell (2003) discuss the importance of celebrating success as a way to mark specific learning outcomes and to incorporate lessons learned into future programming. Within PROSPER, teams shared implementation successes and challenges with each other, their PCs and other experts; problems were solved; upcoming relevant information and research findings were shared; and planning and skills-building for the next year occurred during these meetings.

Sixth, into the second year of the project, team leaders, Prevention Coordinators, and key applied prevention researchers and Extension faculty gathered together between 4-6 times each year to participate in a small Learning Community for continued professional development. Zhao and Bryant (2006) found that small-group mentoring and follow-ups to formal training sessions are related to teachers' use of new classroom technologies. To promote a similar type of acceptance of new approaches to youth-focused programming within PROSPER, TA providers created small-group meetings that included updates on progress, sharing creative and successful implementation activities and learning more about special topics from outside experts on skills or knowledge which was expected to support success with sustaining programs after grant funds ceased. These topics included understanding data and evaluation, including basic statistical analysis, evaluation design, and relevant evaluation findings; fundraising and generating in-kind resources; communicating prevention science concepts to communities; stakeholder analysis and alignment with mission; creative and effective ways to recruit families to universal, voluntary programs; marketing and communication strategies; and other topics. Finally, PCs were supported by university/state-level applied prevention researchers and Extension Faculty on a biweekly schedule via an in-person meeting or conference call where questions about the implementation could be answered and challenges with teams could be discussed, thereby promoting shared learning and growth and development.

As mentioned earlier, these activities were distributed across the phases of team development (see Table 1). We conceptualize that the organizational and planning phase occurred over a nine-month period that started when the team leaders were trained and oriented to the project, team members were recruited and the whole team oriented to the project, the initial EBP was selected, and the planning for the first round of program implementation occurred. The second phase, the implementation and operations phase, started as teams began implementing their selected programs. Teams also selected a second program to implement during this time, kept up with implementation of the first program while received training in the second program and planning for its implementation and completing implementation of this second program, as well. The sustainability phase, then, began as the community teams started their third year of program implementation and started to cover their program costs. Principal investigators created an incentive system to encourage each community team to cover 100% of their program costs over a period of three years, which started at the beginning of the sustainability phase.

Table 1. Overview of PROSPER project timeline and waves of data collection.

Table Notes:

^∼The first two assessment points of the team member interviews, Wave 1 and 2, were not included in this current study, however, we chose to include them here to be consistent with earlier papers.

^*Assessment point predicting team functioning during the implementation phase, with TA Collaboration during the organizational phase, W1-W3 TA Collaboration;

⁺Assessment points predicting team functioning during sustainability, with TA Collaboration during the implementation phase, W4-W9 TA Collaboration.

Research Participants & Procedures

Data for this project were collected from three different sources: 1) Prevention Coordinators, 2) community prevention team members, and the 3) US Census. Table 1 also includes an overview of the assessment schedule.

Prevention Coordinators

Ten individuals were involved as Prevention Coordinators for the 14 community teams across the 4.5 year time period for the study. Thirty percent of the PCs were male and all were White. They had, on average, 19.4 years (range 7 to 31 years) of experience implementing evidence-based programs within the Extension System or other community and school settings prior to their work with PROSPER. PCs responded to several questions in a web-based questionnaire assessing the quality of the collaborative relationship with each of their teams at nine different time points during the organizational and planning phase, and implementation and operations phase. One PC responded to questions about each team at each measurement occasion.

Team Members

The Team Member sample included 208 individuals that participated in one or more waves of data collection over a 4-year period between the Fall of 2002 through the Fall of 2006. As noted earlier, team members consisted of local stakeholders recruited for the PROSPER local teams: representatives of the Cooperative Extension System, middle school teachers or staff, local mental health and substance abuse agency representatives, parents, and other community stakeholders involved on the PROSPER Community Team. Recruitment of participants started at the beginning of the project and continued throughout the project as individuals left and were replaced in their respective positions. This study draws specifically from four datapoints, waves 3 – 6.

Respondents ranged in age from 24 to 62 years (M = 42.1, SD = 8.90), 39.8% were male, and 99.5% self-identified as White. Ninety one percent of respondents indicated having obtained a minimum of a college degree, and 76.1% lived in or near the school district that organized the PROSPER intervention team. The average number of respondents per community over the four waves of data collection was 15 (ranging from 12 to 22). At each wave of data collection individuals participated in one-hour computer-assisted, face-to-face interviews and were compensated with $20.

Measures

Several constructs describing the relationship with the TA provider or the quality of internal and external team functioning were created. Unless otherwise noted, response options for the scales range on a four-point Likert-type scale from Strongly Disagree to Strongly Agree. All team member scales were formed by taking the mean of the scale items and then aggregated to the team level; analyses were conducted at the team-level. Descriptive statistics of all variables are reported in Table 2.

Table 2. Descriptive statistics of all measures at each measurement occasion.

	n	Mean	SD	Min	Max	Skewness	Kurtosis	Phase of Team Functioning
TA Collaboration*
Wave 1	11	0.04	1.00	-2.54	1.03	-1.83	4.33	Organizational
Wave 2	11	0.22	0.53	-0.65	0.82	-0.52	-1.47	Organizational
Wave 3	10	0.05	0.31	-0.42	0.48	-0.13	-1.63	Organizational
Stage Mean	12	0.10	0.48	-1.03	0.63	-1.17	1.26	Organizational
Wave 4	14	0.17	0.86	-1.68	0.94	-1.06	-0.02	Implementation
Wave 5	14	-0.07	0.57	-1.20	0.94	-0.40	0.25	Implementation
Wave 6	14	0.08	0.67	-1.09	1.03	-0.66	-0.48	Implementation
Wave 7	14	-0.03	0.86	-1.83	1.03	-0.58	-0.35	Implementation
Wave 8	14	-0.28	0.74	-1.78	0.94	-0.41	-0.15	Implementation
Wave 9	14	0.02	0.74	-1.25	1.03	-0.09	-0.89	Implementation
Stage Mean	14	-0.02	0.56	-1.06	0.74	-0.49	-0.49	Implementation
Internal Team Functioning*
Leadership
Wave 3	14	3.70	0.17	3.37	3.93	-0.90	-0.06	Implementation
Wave 4	14	3.68	0.21	3.15	3.86	-1.63	2.33	Early Sustainability
Wave 5	14	3.69	0.20	3.13	3.96	-1.54	4.43	Mid-Sustainability
Wave 6	14	3.67	0.26	2.88	3.95	-2.37	7.62	Late Sustainability
Culture
Wave 3	14	3.58	0.25	3.11	3.86	-1.17	1.60	Implementation
Wave 4	14	3.49	0.28	2.80	3.83	-2.21	6.37	Early Sustainability
Wave 5	14	3.54	0.36	2.46	3.89	-0.87	-0.08	Mid-Sustainability
Wave 6	14	3.39	0.44	2.48	3.83	-0.72	-1.06	Late Sustainability
Goals
Wave 3	14	3.50	0.40	2.56	3.93	-1.22	1.11	Implementation
Wave 4	14	3.48	0.27	3.06	4.00	0.61	-0.39	Early Sustainability
Wave 5	14	3.47	0.41	2.40	3.90	-1.56	2.68	Mid-Sustainability
Wave 6	14	3.47	0.40	2.50	3.94	-1.18	1.50	Late Sustainability
Focus on Work
Wave 3	14	0.91	0.09	0.70	1.00	-1.13	0.88	Implementation
Wave 4	14	0.93	0.11	0.64	1.00	-2.16	4.14	Early Sustainability
Wave 5	14	0.93	0.12	0.56	1.00	-2.54	7.31	Mid-Sustainability
Wave 6	14	0.88	0.16	0.40	1.00	-2.11	5.43	Late Sustainability
Tension
Wave 3	14	1.66	0.37	1.00	2.38	0.11	-0.01	Implementation
Wave 4	14	1.83	0.50	1.00	3.00	0.81	1.23	Early Sustainability
Wave 5	14	1.87	0.44	1.00	2.67	-0.08	-0.08	Mid-Sustainability
Wave 6	14	1.73	0.46	1.00	2.40	0.15	-1.07	Late Sustainability
External Team Functioning*
Community Support
Wave 3	14	3.09	0.25	2.60	3.42	-0.23	-0.86	Implementation
Wave 4	14	2.99	0.24	2.67	3.50	0.66	-0.06	Early Sustainability
Wave 5	14	2.88	0.37	2.36	3.50	0.39	-1.00	Mid-Sustainability
Wave 6	14	2.96	0.27	2.47	3.33	-0.32	-1.03	Late Sustainability

Note:

^*The project experienced some omissions of data reporting early in project implementation, and two coalitions also had a slightly delayed start, hence the lower n for the data points during the organizational stage.

TA Collaboration

Though TA collaboration is likely a multidimensional construct, we used one measure to describe the degree to which the team communicated with and worked collaboratively and effectively with the Prevention Coordinator, TA Collaboration (7 items; α = .84). Given the competing demands of the larger research project, the scale used in this study focused on asking the TA provider to rate different observable behaviors of the team leader or team, rather than also complete many subjective ratings of attitudes or feelings. The PC first rated the timeliness of reports, applications, and materials; and then the team's cooperation with TA on a 1 (Poor) to 7 (Excellent) scale. In the other 5 items, the PC rated how often (1-5 scale, rarely to always) the team leaders promptly returned calls and emails; the team leaders informed the PCs of developments; the PC experienced resistance to suggestions (reversed); the PC experienced resistance from the team to other project intervention staff and procedures (reversed); and how often the PC experienced resistance from the team to project research protocols. The last three resistance items, when reverse scored, describe the team leader and team's receptivity to TA, which can be considered part of a collaborative relationship. An exploratory factor analysis revealed that all seven items best formed one scale. Consequently, all seven items were standardized (M=0; SD=1) before averaging to create the scale value.

Team Functioning

Four measures assessed by team members described the internal team functioning of PROSPER community teams. Team leadership (8-items, α range .78 to .85 adapted from (Kegler, Steckler, McLeroy, & Malek, 1998) assessed the degree to which team leadership encourages input and consensus, along with promotes a friendly work-environment; an example item is, “the team leadership… gives praise and recognition at meetings.” Team culture (8-items, α range .80 to .92 adapted from (Kegler et al., 1998) assessed the team atmosphere; an example item is, “there is a strong feeling of belonging in this team.” Team goals (2-items, r range 0.31 to .69; (Perkins et al., 2011) assessed the degree to which the PROSPER community team has developed clear goals and governance procedures; an example item is, “The PROSPER team has agreed on how it will govern itself and make decisions.” Team focus on work (5-items; α range .66 to .72; adapted from (Moos & Moos, 1998) assessed the work-orientation of the team; an example item is, “People pay a lot of attention to getting work done.” One item assessed the degree of conflict and tension in the PROSPER team, team tension. This item was on a four-point scale ranging from 1 = no tension to 4 = a lot of tension (Feinberg et al., 2007).

One measure described the external relationships of PROSPER community teams. Community support (5-items, four point scale 1 = Not at all, to 4 = A great deal; α range .64 to .78 (Perkins et al., 2011) assessed the degree to which school administrators and agency leaders in the community were perceived to be committed to and champion the PROSPER initiative; an example item is, “Do influential community leaders understand PROSPER and why it is important?”

Analysis Strategy

TA Collaboration was our independent variable in these models. Ratings of team functioning were our dependent variables. We tested the predictive power of TA Collaboration in one phase on internal and external team functioning in the next phase using team-level Spearman Correlations. We selected this analysis strategy for three reasons. First, although we had longitudinal data available for both the independent variable (TA Collaboration) and dependent variables (ratings of team functioning), the assessment timeline of the two measures did not match (see Table 1), such that a multilevel model in which TA Collaboration was a time-varying covariate would not have been appropriate. Second, as described earlier in the paper, we had interest in understanding TA Collaboration as it maps onto the phases of collaborative team functioning; this more parsimonious statistical model makes this investigation more straightforward. Third, a team-level correlational analysis simplifies the merging and analysis of data from two different reporters. Fourth, a team-level n of 12-14 generally has insufficient power to detect significant team-level associations; a correlation's effect size is more easily interpretable when traditional significance criteria are not enough to understand potential statistical associations. In addition, the distribution of a few variables became slightly skewed at later time points.

After making the above decision, analyses followed five steps: First, we averaged scores of TA Collaboration across Waves 1-3 and Waves 4-9 to create scores that described the average level of team-PC collaboration that occurred within the organizational and implementation phases. Second, we created team-level values for each dependent variable by averaging individual team member responses together for each DV time point. Third, we estimated Spearman Correlations with the average level of team-PC Cooperation during the organizational phase with team member ratings of team functioning in the implementation stage. Fourth, we estimated Spearman Correlations with the average level of team-PC Cooperation during the implementation phase with team member ratings of team functioning at the three different time points during sustainability (early, mid, and late). We kept the dependent variable values distinct during the sustainability phase for two reasons. It is an extremely long time-period and we wanted to look for the possibility of effect decay over time. The financial responsibilities of the teams also grew at each time point, hence there were some qualitative differences of team activities at each time point during sustainability. All correlations controlled for the percent of families living below the federal poverty threshold in 2000 (National Center for Education Statistics, 2003; US Census, 2000). Community poverty tends to be a good measure of community stress. Community poverty has also been important in predicting team functioning in prior research (Feinberg et al., 2007; Greenberg et al., 2007).

Following prior research, we used a measure of effect size to determine substantively meaningful associations among variables and balance our need to limit both Type 1 and Type 2 errors with such a small sample size (n range 12-14). We selected a level of r = +/-.35 in all correlation models as our minimum level of effect size.. A correlation of this size explains slightly more than 10% variance in the dependent variable.

Results

Preliminary Analyses

Descriptive statistics of all measures are listed in Table 2. There is some fluctuation of reported TA Collaboration over time, with lower averages appearing during the implementation phase compared to the organizational and planning stage. The communities involved in the study have a relatively low percentage of families that are living below the federal poverty threshold. Overall, levels of internal team functioning are quite positive over time, but they seem to drop slightly across the 4.5 years studied as levels of tension seem to increase slightly and levels of community buy-in seem to be decrease slightly as well.

TA Collaboration Predicting Team Functioning During Implementation/Operations

Table 3 summarizes the results of the Spearman Correlations between levels of TA Collaboration during the organizational and planning stage with the quality of team functioning during the implementation and operations stage. Average levels of TA Collaboration during the organizational and planning stage meaningfully associated with higher team member ratings of leadership (r = .55, p < .10), culture (r = .40, p > .10), and goals (r = .38, p > .10) at Wave 3, the team functioning assessment point during the implementations and operations stage.

Table 3.

Results from Spearman correlation analyses controlling for poverty exploring the association between the quality of the TA collaborative relationship during the organizational and implementation stages with internal team functioning and community support during the implementation and sustainability phases, respectively.

	TA Collaboration during the Organizational Phase (n=12)	TA Collaboration during the Implementation Phase (n=14)
Implementation / Operations Stage
Wave 3: Internal Team Functioning
Leadership	.55*	---
Culture	.40+	---
Goals	.38+	---
Focus on Work	.28	---
Tension	.07	---
Wave 4: Community Support	.08	---
Early Sustainability
Wave 4: Internal Team Functioning
Leadership	---	.33
Culture	---	.47*
Goals	---	.70***
Focus on Work	---	.32
Tension	---	-.20
Wave 4: Community Support	---	-.14
Mid-Sustainability
Wave 5: Internal Team Functioning
Leadership	---	.07
Culture	---	.46+
Goals	---	.49*
Focus on Work	---	.15
Tension	---	-.03
Wave 5: Community Support	---	.41+
Later-Sustainability
Wave 6: Internal Team Functioning
Leadership	---	.25
Culture	---	.27
Goals	---	.31
Focus on Work	---	.41+
Tension	---	-.25
Wave 6: Community Support	---	.30

⁺r => +/- .35;

^*p ≤ .10;

^**p ≤ .05;

^***p ≤ .01

TA Collaboration Predicting Team Functioning During Sustainability

Table 3 summarizes the results of the Spearman Correlations between levels of TA Collaboration during the implementation and operations stage with the quality of team functioning during early, mid, and later sustainability. Average levels of TA Collaboration during the implementation and operations phase meaningfully associated with higher team member ratings of culture (r = .47, p = .10) and goals (r = .70 p < .01) at Wave 4, the assessment point during early sustainability. Average levels of TA Collaboration during the implementation and operations stage meaningfully associated with higher team member ratings of culture (r = .46, p > .10) and goals (r = .49, p < .10) at Wave 5, the assessment point during mid-sustainability. TA Collaboration during the implementation and operations stage did not meaningfully associate with the quality of team functioning during later-sustainability.

Discussion

This study examined the relationship between TA provision and the community prevention team, focusing on its impact on the quality of the team functioning. Specifically, we examined the contribution of the collaborative relationship between TA provision and the community prevention team in predicting high-quality team functioning, an important proximal outcome that predicts later success (Feinberg, Bontempo, et al., 2008; Greenberg et al., 2014; Perkins et al., 2011; Spoth et al., 2005). The findings of this study provide an initial understanding of the importance of the relationship between TA providers and the implementing team; higher levels of a collaborative relationship associated with more positive aspects of select characteristics of future internal team functioning.

The findings from this study support the importance of TA providers employing collaborative teaching, professional development, and problem-solving techniques (Becker et al., 2013; Fixsen et al., 2005; Pas et al., 2014; Stormont & Reinke, 2014). The substantively meaningful associations were replicated at three different time points covering two broad phases of team development. These findings will hopefully help TA providers to remember to be encouraging, supportive, and empowering during the organizational and planning phase, a phase which tends to include a fair amount of instruction. Though it can seem easier to direct step-by-step instructions to the prevention team leader and team members in order to progress smoothly through to the next phase, it may be more beneficial to use a different approach. TA providers can support building the capacity of the team leader and team members with an empowerment approach that includes asking open-ended questions which encourage them to critically reflect on their knowledge and experience, encouraging teams to brainstorm pros and cons when assessing solutions to the many challenges that they will face, providing positive yet constructive feedback, helping the team leader and team set realistic goals, and encouraging team-centered accountability. In addition, it is important for TA providers to continue these types of patient and empowering interactions during the inherently challenging and stressful implementation and operations phase. These interactions are found within a motivational interviewing technique (Miller, 1983); this evidence-based treatment strategy has begun to be referred to as a professional development resource for TA providers using Communities that Care (Hawkins, Catalano, & Arthur, 2002).

The various factors affecting these team dynamics is unclear. That is, the reported study did not allow disentangling the team dynamics effect of TA provider characteristics, specific lead implementer characteristics, and combined individuals' characteristics. Other work has assessed how personality characteristics of team members associate with implementation quality in these settings and found that levels of team member openness to experience was negatively associated with team functioning, whereas conscientiousness positively related to team functioning (Feinberg, Kim, & Greenberg, 2008). Future work should further investigate these relationships.

In examining the collaborative relationship between TA providers and lead implementers, the presence of substantively meaningful associations between the collaborative relationship with leadership, culture, goals, work, tension, and community support at various time points may provide some insights into the dynamics related to those relationships. For instance, we found that TA collaboration meaningfully predicted the quality of the leadership of teams most strongly in the beginning. Meaningful associations with a team's culture and goal directedness continued longer over time. Perhaps the collaboration between the TA provider and the lead implementer increases the effectiveness of the leader to manage the team, thereby increasing the confidence of the team members in the leader's management of the effort and supporting positive interpersonal dynamics among team members.

TA collaboration is consistently meaningfully associated with team goal directedness. This pattern is consistent with other research about coaching and problem solving (Becker et al., 2013; Fixsen et al., 2005; Pas et al., 2014; Stormont & Reinke, 2014; Wasylyshyn, 2007). Indeed, a collaborative coaching style of teaching and problem solving typically is described as goal-oriented and includes continuous examination of objectives to reach goals (Kilburg, 1996). Implementation goals and ideal timelines were communicated throughout the implementation of the PROSPER effort; emphasis on these goals intensified and became more crucial to meet as teams became responsible for carrying the full financial burden of their programming. Thus, benchmarks were developed to guide sustainability, especially the early sustainability actions of teams.

On the contrary, a team's work orientation reported by team members is not associated with TA collaboration in the early stages, but is seems to become more important during late sustainability when all team's supported 100% of their program cost. The skills associated with achieving sustainability are unique and may require more intensive collaboration between the TA provider and coalition leader. This more intensive collaboration may support the team leader's managing of a highly work-focused team, as the team is managing program implementation, fidelity assessments, pre-post evaluations, fundraising, and generating other in-kind resources, all simultaneously during the sustainability phase. Successful coordination of these efforts may account for higher reports of work orientation by team members, which leads to coalition success.

The longitudinal associations found in this study are especially remarkable because of the time lag between measurement time points. Levels of collaboration between the TA provider and prevention team during the implementation and operations stage, reported by the TA provider, strongly associated with different aspects of internal team functioning 1-2 years later. They continued to be meaningful predictors even three years later in this study. Note that the quality of internal team operations was rated independently of prevention team members' ratings of collaboration. This may indicate that the quality of the TA–lead implementer relationship during the implementation and operations stage is crucial in supporting future success. For this reason, future research on this question should continue to investigate future associations while controlling for concurrent associations. Furthermore, pinpointing the developmental processes and determining whether collaborative relationships can be built regardless of personal characteristics with the use of different kinds of teaching and feedback strategies is a critical area for investigation. For instance, future research questions include: How much of a high-quality collaborative relationship is determined by characteristics of the TA provider? How much is determined by lead implementer characteristics? How important is it to consider matching certain TA staff with certain qualities with different kinds of lead implementers? These questions also have been suggested in reviews of the relevant literature (Durlak & DuPre, 2008; Fixsen et al., 2005; Spoth et al., 2013). Future research that includes investigating how dosage of TA may affect TA collaboration, and how they interact to predict team functioning would also be worthwhile.

On a different note, the prevention team's relationships with community organizations and leaders outside of the effort were not at all associated with the collaborative relationship between the TA provider and lead implementer. Other factors may be more important in determining the quality of relationships with community organizations external to the prevention coalition. This lack of a relationship also may suggest that the primary driver of the collaborative relationship is not the prevention team leader. If the driver of the collaborative relationship was the team leader, levels of community support also likely would be rated as high and a moderate to strong positive association would be expected.

Limitations

There are a number of limitations in this study. This sample includes rural and semi-rural communities participating in the same community prevention team model, all from the project's inception. The teams were all focused on reducing and delaying early adolescent substance use and used similar strategies to achieve their goal. This narrow sample and the timeline reduced potential noise in the data and enabled us to investigate associations at different stages of team development, but it could also limit variability and consequently weaken statistical power. In addition, the generalizability of these findings to other coalition and community contexts is unknown. There may also have been some unmeasured differences in implementation that affected the quality of the collaborative relationship.

Our sample is small and there were some early omissions in data reporting by some TA providers. Consequently, our statistical analysis was limited to Spearman Correlations. Our statistical power is lower than what typically is desired, which increases our chances of making a Type 2 error. We compensated by using a measure of effect size rather than traditional measures of significance when interpreting our results. We were limited to the number of variables that we could investigate simultaneously; as a result, we constrained our analyses to controlling for one important community context variable, community poverty, rather than also controlling for other potentially important predictors, such as prior levels of team functioning or other characteristics of the community context. Future research with a larger number of teams and a variety of team types is desirable. The plethora of community teams and coalitions across the country would seem to be supportive of composing this kind of sample. Given these limitations, this study's conclusions are tentative and need replication. Yet the longitudinal data and different reporters contribute to this study's strength.

Conclusion

This study showed that the quality of the relationship between the lead implementer and TA provider is quite important for prevention teams to achieve proximal outcomes that are indicators of effectiveness. Specifically, a relationship characteristic of collaboration between the TA provider and lead implementer was supportive of high-quality internal team functioning. Future research should continue investigating these relationships. Future community collaborative prevention work also should consider these types of findings when designing and conducting TA activities, and could benefit from considering assessment of these relationships, along with reviewing them with TA providers in a systematic way to support high quality implementation. Given these findings, it is possible that a collaborative relationship is a key factor to delivering effective TA; more research is necessary to further understand this developmental process.

Highlights.

• - We examine the relationship between technical assistance providers and community prevention team leaders

• - Higher levels of a collaborative relationship relate to better internal team functioning

• - This pattern is found in longitudinal analysis

Biographies

Sarah M. Chilenski, PhD is a Research Associate at the Prevention Research Center at the Pennsylvania State University. Her primary interest is examining how communities, schools, and universities can collaborate in the pursuit of quality youth prevention and health promotion programming. To this end, her focus includes researching the association of community characteristics with adolescent outcomes, the predictors and processes of community prevention team functioning and prevention systems. She frequently employs multiple methods in her research.

Daniel F. Perkins, PhD is Professor of Family and Youth Resiliency and Policy at the Pennsylvania State University and is Director of Penn State Clearinghouse for Military Family Readiness. Currently, Dr. Perkins is examining the transitioning of evidence-based programs and practices to their large scale expansion into real-world settings. The Clearinghouse is an applied research center with a mission of fostering and supporting interdisciplinary applied research and evaluation, translational and implementation science, and outreach efforts that advances the health and well-being of Military service members and their families.

Jonathan R. Olson, PhD is on the faculty at The Pennsylvania State University. His research focuses on predictors of internalizing and externalizing outcomes among young people, positive youth development, and prevention programming for youth and families.

Lesa Hoffman, PhD is the Scientific Director of the Research Design and Analysis (RDA) Unit of the Lifespan Institute and Associate Professor of Quantitative Methods in the Child Language Doctoral Program. Her program of research seeks to empirically examine and to thoughtfully disseminate how developments in quantitative psychology can best be utilized to advance empirical work in psychology, human development, and other social sciences.

Mark Feinberg is Research Professor at the Bennett Pierce Prevention Research Center at Penn State. He has two main areas of research, including community prevention systems and family prevention. His current methodological interests focus on how interventions work with multiple family members in the context of prevention trials. He is the PI on the NIH-funded Military Family Foundations project.

Mark Greenberg is the Bennett Chair of Prevention Research, and Professor of Human Development and Psychology at Penn State. His research interests include intervening in the developmental processes in risk and non-risk populations with a specific emphasis on aggression, violence, and externalizing disorders; promoting healthy social and emotional development through school-based prevention; the study of community partnerships and the diffusion of evidence-based programs; and the study of contemplative practices and mindfulness interventions.

Janet Welsh is Research Assistant Professor at the Bennett Pierce Prevention Research Center at Penn State. Her research interests primarily involve development, implementation, and evaluation of school based prevention programs for children and youth. She is involved in a number of school readiness initiatives for young children, including Head Start REDI and the Focus on Learning project.

Max Crowley is a decision scientist and health policy researcher studying how to effectively and efficiently protect children and prevent risky health behaviors. His work seeks to design community prevention efforts that strategically invest in youth to reduce risky health behaviors as well as the future burden on social service systems. This includes developing performance-based financing strategies to ensure high-quality program delivery.

Richard Spoth, PhD is the F. Wendell Miller Senior Prevention Scientist and the Director of the Partnerships in Prevention Science Institute at Iowa State University. He provides oversight for an interrelated set of projects addressing a range of research questions on prevention program engagement, program effectiveness, culturally-competent programming, and dissemination of evidence-based programs through community-university partnerships. Among his NIH-funded projects, Dr. Spoth has received multiple awards for his work from the National Institute on Drug Abuse and The Society for Prevention Research.