Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 1.
Published in final edited form as: Prev Sci. 2015 Jan;16(1):101–109. doi: 10.1007/s11121-013-0447-x

Reciprocal Relations between Coalition Functioning and the Provision of Implementation Support

Louis D Brown 1, Mark E Feinberg 2, Valerie B Shapiro 3, Mark T Greenberg 2
PMCID: PMC4051854  NIHMSID: NIHMS547913  PMID: 24323363

Abstract

Community coalitions have been promoted as a strategy to help overcome challenges to the dissemination and implementation of evidence-based prevention programs. This paper explores the characteristics of coalitions that enable the provision of implementation support for prevention programs in general, and for the implementation of evidence-based prevention programs with fidelity. Longitudinal cross-lagged panel models were used to study 74 Communities That Care (CTC) coalitions in Pennsylvania. These analyses provide evidence of a unidirectional influence of coalition functioning on the provision of implementation support. Coalition member knowledge of the CTC model best predicted the coalition’s provision of support for evidence-based program implementation with fidelity. Implications for developing and testing innovative methods for delivering training and technical assistance to enhance coalition member knowledge are discussed.

Keywords: Communities That Care, substance-use prevention, delinquency prevention, coalitions, partnerships, program implementation

The dynamics of coalition functioning are important to understand because they enable multi-sector collaborative synergy and create implementation support infrastructure for the dissemination of evidence-based programs (EBPs; Butterfoss & Kegler, 2002; Lasker & Weiss, 2003; Wandersman et al., 2008). Community coalitions have been advanced as a means of providing support for the implementation of EBPs because they can build local knowledge and capacity, focus and coordinate efforts, reduce resistance to change, and enhance communication (Butterfoss, 2007; Durlak & DuPre, 2008; Kegler & Swan, 2011; Rhoades, Bumbarger, & Moore, 2012). Research suggests community coalitions that implement EBPs achieve improved outcomes for youth (Feinberg, Jones, Greenberg, Osgood, & Bontempo, 2010; Hawkins et al., 2012; Spoth et al., 2013), and weak or no effects when they do not use EBPs (Hallfors, Hyunsan, Livert, & Kadushin, 2002; Roussos & Fawcett, 2000).

Several theoretical models postulate relations between specific coalition characteristics and program implementation support (Brown, Feinberg, & Greenberg, 2012; Butterfoss, Goodman, & Wandersman, 1993; Butterfoss & Kegler, 2002). For example, in Community Coalition Action Theory, coalition processes such as cohesion and task focus influence collaborative synergy, which in turn enables program implementation (Butterfoss & Kegler, 2002; Kegler & Swan, 2011). However, there have been relatively few empirical investigations of how coalitions support the implementation of prevention. Existing studies are often underpowered and do not examine change over time, but identify associations between coalition functioning and program implementation (Brown, Feinberg, & Greenberg, 2010; Kegler & Swan, 2011; Spoth, Guyll, Lillehoj, Redmond, & Greenberg, 2007).

This study builds on existing literature by examining relations between specific coalition functioning characteristics and support for the implementation of prevention programs with three key methodological improvements. First, we use longitudinal cross-lagged panel models that capture reciprocal causation. Second, we examine a set of empirically distinct coalition characteristics that have the potential to predict unique variance in implementation support. Finally, we employ two methods in measuring coalition functioning – self-report ratings by coalition members and observer ratings from coalition technical assistance providers.

Implementation support

There are several ways in which coalitions can support the implementation of prevention programs. For example, coalitions can consolidate or secure resources for program implementation, thereby increasing the number of prevention programs in a community (Butterfoss, 2007). Coalitions can also guide program selection, helping to ensure programs address systematically observed community needs, are feasible, and have a strong evidence-base (Hardy et al., 2011). Finally, coalitions can support implementation quality, helping communities implement EBPs with fidelity to the way in which the program was designed and achieve the results found under testing conditions (Brown, et al., 2010).

Coalitions can provide or encourage the use of several fidelity supports that are known to influence outcomes, including the provision of initial training by certified trainers, ongoing coaching, and refresher trainings (Durlak & DuPre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). Such supports can help prevent program model drift and provide new staff with needed training (Fixsen, et al., 2005). Coalitions can also create structures and processes for monitoring fidelity, providing program implementers with performance feedback to improve areas where implementation is poor (Fixsen, et al., 2005; Jones, Wickstrom, & Friman, 1997).

Coalitions can use their resources to support many different programs, but coalitions that focus on implementing EBPs with fidelity may be more likely to have an impact on health and behavior outcomes (Durlak & DuPre, 2008; Flewelling et al., 2005; Spoth, Greenberg, & Turrisi, 2008). Thus, this paper explores the characteristics of coalitions related to both the support for implementation of prevention programs in general, and support for the implementation of EBPs with fidelity, specifically.

Coalition functioning characteristics

We conceptualize coalition functioning as a set of malleable characteristics that involve cooperation between coalition members to realize shared goals (Brown, et al., 2012). The coalition characteristics examined in this study derive from a model of coalition functioning that has emerged from and in turn guided several previous studies of Communities That Care (CTC) coalitions (Brown, et al., 2010; Brown, et al., 2012; Feinberg, Bontempo, & Greenberg, 2008; Feinberg, Gomez, Puddy, & Greenberg, 2008; Feinberg, Greenberg, Osgood, Anderson, & Babinski, 2002; Feinberg, et al., 2010; Feinberg, Ridenour, & Greenberg, 2008). This model specifies several coalition functioning characteristics that are likely to influence a coalition’s ability to support program implementation.

Our previous research has indicated that there are important reporter differences when examining coalition functioning. That is, coalition members and technical assistance providers view some domains of functioning in similar ways, but also provide unique factors that are a result of their perspective (Feinberg, Gomez, et al., 2008; O’Malley, Landon, & Guadagnoli, 2007). Thus, we examine the same domains for some aspects of coalition functioning and also examine unique features separately from the views of coalition members and technical assistance providers (Feinberg, Gomez, et al., 2008). We organize all coalition functioning characteristics, irrespective of reporter perspective, into three conceptual categories - collaborative processes, coalition capacities, and coalition activities. We use these categories because they are capable of parsimoniously describing a variety of coalition characteristics, including those used in this study.

Collaborative processes are the ways in which coalition members interact as a team to influence their objectives (Shapiro, Oesterle, Abbott, Arthur, & Hawkins, in press). Here we assess the collaborative processes of efficient orientation to tasks, team cohesion, and a participatory leadership style. Efficient functioning, where members are task focused and hardworking, is important because resources are limited (Zakocs & Edwards, 2006) and has been linked to coalition sustainability over time (Gomez, Greenberg, & Feinberg, 2005). Team cohesion is similarly viewed as a key ingredient to collaboration between diverse stakeholders because unity and friendly collaboration make participation more enjoyable and help to improve communication (Butterfoss, 2007; Foster-Fishman, Berkowitz, Lounsbury, Jacobson, & Allen, 2001; Shapiro, et al., in press; Zakocs & Edwards, 2006). A participatory style of leadership, where tasks and decision-making are shared, is also important because it helps coalitions engage diverse stakeholders (Foster-Fishman, et al., 2001; Zakocs & Guckenburg, 2007).

The second category of coalition functioning, coalition capacities, includes the attitudes, knowledge, and skills of the coalition members and the paid coordinator (henceforth called the mobilizer). Of interest in this study are the skills of the mobilizer, and the member’s knowledge of science-based prevention, both of which have been linked to coalition outcomes (Butterfoss, 2007; Feinberg, Bontempo, et al., 2008; Gomez, et al., 2005).

Coalition activities are the third category of coalition functioning, representing the ways in which coalitions direct their energy or spend their time. Whereas collaborative processes focus on teamwork, coalition activities are the actual tasks completed by the group. The coalition activity of interest in this study is sustainability planning, which involves planning for ongoing implementation and the attainment of continuation funds (Johnson, Hays, Center, & Daley, 2004). Such planning is likely to contribute to the continued success of coalition efforts (Brown, et al., 2010; Feinberg, Bontempo, et al., 2008; Perkins et al., 2011).

In addition to coalition functioning, we consider contextual characteristics that shape the way in which local coalitions organize their work, but are not potential targets for change through training or technical assistance. In this study, community poverty and coalition age are contextual characteristics of interest because previous research suggests they have a negative association with program implementation (Brown, et al., 2010). Community poverty may interfere with program implementation because of organizational barriers such as a lack of well-functioning local institutions, a demoralizing history of collaborative failure, and fewer local resources to support implementation (Greenberg, Feinberg, Meyer-Chilenski, Spoth, & Redmond, 2007). Coalition age may lead to mission drift away from a focus on EBP implementation, perhaps because of staff and stakeholder turnover (Brown, et al., 2010).

Communities that Care (CTC) coalitions in Pennsylvania

This study focuses on coalitions that use the Communities That Care (CTC) strategy to organize their prevention activities, which guides communities to adopt a science-based approach to community prevention planning and services (Hawkins, Catalano, & Arthur, 2002). The CTC strategy provides guidance through training and technical assistance to (1) organize members of communities into collaborative coalitions, (2) develop coalition capacities such as the knowledge and skills to use a science-based approach to prevention, (3) conduct an epidemiological assessment of adolescent risk and protective factors to identify community needs, (4) prioritize community needs and select EBPs to address those priorities, and (5) implement EBPs and monitor community prevention efforts and outcomes to ensure a high-quality implementation and the achievement of goals (Hawkins, et al., 2002). Coalition knowledge of the CTC model is a dimension of coalition capacity of particular interest in this study because the CTC model provides guidance on how to employ a science-based approach to prevention (Rhew, Brown, Hawkins, & Briney, 2013; Shapiro, Hawkins, & Oesterle, 2013). The extent to which communities employ a science-based approach to prevention fully mediates the impact of CTC coalitions on youth outcomes (Brown et al., 2013).

The current study is based on data collected from CTC coalitions in Pennsylvania. Since 1994, the Pennsylvania Commission for Crime and Delinquency (PCCD) has funded the development and training of over 125 CTC coalitions. Supporting the success of CTC coalitions are coalition mobilizers and technical assistance providers. Coalitions hire a mobilizer to help organize day-to-day operations. Technical assistance providers work with several coalitions under a separate contract with PCCD, helping support the use of best practices and troubleshooting implementation challenges. PCCD also provided funding for the implementation of EBPs, which CTC coalitions often successfully pursued. The population size of the counties where CTC coalitions operate ranged from 5,334 to 1,218,429 people with a median of 143,768 people; however, the CTC coalitions vary in the scope of their geographic focus, working within neighborhoods, towns, school districts, or counties.

Research aims

The goal of this study is to further our understanding of how coalition functioning characteristics relate to implementation support provided by coalitions for prevention programs generally, and for the implementation of EBPs with fidelity. The theoretical model of coalition functioning under investigation specifies a one-way relationship, where coalition functioning influences program implementation but program implementation does not influence coalition functioning (Brown et al., 2012). Based on this model, we hypothesize that (1) earlier measures of coalition functioning will predict later changes in coalition support for implementation, and (2) earlier measures of coalition support for implementation will not predict later changes in coalition functioning. Considering the importance of the CTC system in promoting the use of science-based prevention practices, we also hypothesize that CTC model knowledge will be a particularly salient predictor of implementation support for EBPs with fidelity. This analysis will have implications for coalitions and technical assistance providers who need to know which malleable coalition characteristics are most essential to develop in support of the implementation of prevention programs.

Method

Data for this study are drawn from an annual evaluation of CTC coalitions funded by PCCD. All coalition leaders from CTC coalitions in Pennsylvania were asked to provide the names and email addresses of coalition members. Every member was invited to participate in a web survey that assessed coalition functioning. Members who did not have access to the Internet could complete a paper-and-pencil version of the questionnaire. From 2004 to 2008, survey respondents were 68% female, 94% White, 5% Black, and 1% Hispanic. Respondents averaged 47 years old and had been involved in CTC for an average of 3.9 years.

Table 1 reports all survey response rates, which ranged from 46 to 62%. Although response rates are respectable for an online survey in an applied setting, requests to complete the survey could easily be ignored and we suspect highly involved coalition members were more likely to respond. One limitation of the study is that we do not know if there are statistical differences between responders and non-responders.

Table 1.

Response rates for each data collection instrument by year.

Instrument 2004 2005 2006 2007 2008
Self-reported coalition functioning (Respondents; Coalitions) 46% (867; 79) 50% (799; 75) 60% (929; 73) 62% (988; 72) 62% (946; 67)
Observed coalition functioning (Coalitions) 97% (77 of 79) 95% (71 of 75) 84% (61 of 73) 96% (69 of 72) 67% (45 of 67)
Self-reported implementation support (Coalitions) NA NA NA 86% (62 of 72) 90% (60 of 67)
Open in a new tab

Because the reports of coalition members are subject to self-report bias, coalition observers also provided information on coalition functioning. PCCD-funded technical assistance providers were asked to complete a survey about each coalition they supported. Additionally, in 2007 and 2008, coalition leaders (mobilizers or voluntary chairs) were asked to complete a supplemental survey section about the implementation support provided for prevention programs supported by the coalition.

Measures

Coalition functioning

Coalition functioning was measured through surveys (available from the first author) of coalition members and technical assistance providers. The items in these instruments have been refined from previous studies of coalitions (Arthur, Hawkins, Catalano, & Olson, 1998; Kegler, Steckler, McLeroy, & Malek, 1998; Lasker & Weiss, 2000; Moos, 2002). Table 2 reports some psychometric properties of the scales, including their internal reliabilities, which range from good to excellent.

Table 2.

Number of items, alpha, and correlations between coalition characteristics in 2006 (n = 58 – 74)

Coalition characteristics Num. Items Alpha (1) (2) (3) (4) (5) (6) (7) (8)
Collaborative Processes:
(1) Self-Report Coalition Efficiency 2 r = .78 -
(2) Observed Cohesion 3 .91 .14 -
(3) Self-Report Leadership Style 2 r = .80 .69* .34* -
Coalition Capacities:
(4) Self-Report Mobilizer Skill 4 .92 .66* .26* .74* -
(5) Observed Mobilizer Skill 4 .89 .34* .44* .40* .43* -
(6) Observed CTC Model Knowledge 3 .79 .36* .28* .35* .47* .57* -
Coalition Activities:
(7) Self-Report Sustainability Planning 3 .83 .73* .03 .55* .78* .26* .34* -
Contextual Characteristics:
(8) Community Poverty 1 NA -.25* .06 -.13 -.27* -.14 -.13 -.31* -
(9) Coalition Age 1 NA .02 .10 -.20 -.23* -.20 -.18 -.15 .14
Open in a new tab

Note:

*

p < .05

†

p < .10

Although the self-report and observer-report surveys of coalition functioning are similar, they are customized to the strengths of each perspective. Factor analytic work indicates the constructs that are empirically distinct for coalition members are not the same as the empirically distinct constructs for technical assistance providers. Results presented in Brown et al. (2012) indicate Efficiency, Leadership Style, Mobilizer Skill, and Sustainability Planning were empirically distinct in the self-report survey of members. Cohesion, Mobilizer Skill, and CTC Model Knowledge emerged as the empirically distinct factors in the observer-report survey of technical assistance providers.

We conducted the factor analytic work on the observer-reported scales for this study. Sample sizes by year were underpowered, so we combined observations across years and conducted an exploratory factor analysis using Mplus, which accounted for the clustering of observations within coalitions. After removing items that had cross loadings greater than .4, we identified the 3 factor solution used in this study, which had excellent model fit (CFI = .99; RMSEA = .05).

Following are the definitions of each construct along with sample items and Likert response option labels, which were all on a seven point scale. Efficiency items quantified the work ethic, efficiency, and task focus of the coalition (e.g. “This is a highly efficient, work-oriented team.” Response options: Strongly Disagree to Strongly Agree). Cohesion measured the extent to which there are feelings of unity, friendly collaboration, and appropriate conflict resolution (e.g. “There is a feeling of unity and cohesion on the board.” Response options: Poor to Excellent). Leadership Style is the extent to which leaders seek out members views and reach out for help (e.g. “The CTC leadership intentionally seeks out your views.” Response options: Strongly Disagree to Strongly Agree). Mobilizer Skill assessed perceptions of whether the lead staff person is knowledgeable and enthusiastic about the CTC model, and possesses strong organizational and interpersonal skills (e.g. “How skilled is your CTC Mobilizer or lead staff person in the following areas…Organizational skills.” Response options: Needs Work to Very Strong). CTC Model Knowledge captured coalition attendance at regional trainings, along with an understanding of the CTC philosophy and the risk and protective factor approach (e.g. “Understanding of CTC philosophy and practice.” Response options: Poor to Excellent). Sustainability Planning measured coalition exploration of funding strategies and the development of realistic, concrete plans for how to continue offering programs (e.g. “Has CTC explored potential funding sources for continuing similar programs?” Response options: Not At All to Yes).

Implementation support

We used survey responses from coalition leaders to construct two measures – Overall Implementation Support and EBP Implementation Support. Overall Implementation Support represents the quantity of implementation support irrespective of the evidence base of the program supported or whether the support is likely to impact health outcomes. More specifically, Overall Implementation Support is the number of programs supported (regardless of evidence-based status) multiplied by the average level of coalition involvement in supporting the programs (0 = not involved; 1 = provides consultation on selecting, setting up, or implementing the program; 2 = is a co-sponsor or helped obtain funding; 3 = is the primary sponsor).

EBP Implementation Support represents that amount of support provided that is likely to impact youth outcomes. More specifically, EBP Implementation Support considers only support for EBPs and is adjusted by the degree to which Fidelity Supports are in place. We defined a program as evidence-based if it was listed on SAMHSA’s national registry of EBPs (www.nrepp.samhsa.gov), the Blueprints for Healthy Youth Development program list (www.colorado.edu/cspv/blueprints), the Office of Juvenile Justice and Delinquency Prevention Model Programs Guide (www.ojjdp.gov/mpg), or in the case of Parents Who Care, because we identified a randomized trial indicating efficacy (Haggerty, Skinner, MacKenzie, & Catalano, 2007). Although inclusion criteria for the EBP lists are varied, complex, and somewhat subjective, in all cases a program had at least one well-conducted quasi-experimental evaluation demonstrating positive outcomes. Fidelity Support (α = .70) is the mean of three items that assess: (1) the degree to which fidelity is monitored in a systematic and objective manner; (2) the quality of the initial training for program delivery; and (3) the quality of ongoing training and technical assistance.

To compute EBP Implementation Support, we first multiplied Coalition Involvement (range = 0 – 3) by Fidelity Support (range = 0 – 3) for each EBP supported by a coalition. EBP Implementation Support (range = 0 – 30) is the sum of scores from all EBPs supported by a coalition. Although unconventional, the use of multiplication ensures coalitions receive a 0 when Fidelity Support is absent, thus ensuring EBP Implementation Support scores are low when coalitions support a large number of EBPs without attention to implementation fidelity. More traditional summary measures, such standardizing and taking the mean of component variables, produce similar results as those presented in this paper.

Correlations between the components of EBP Implementation Support are relatively small and not statistically significant. In 2007, r = .23 for Coalition Involvement and Fidelity Support; r = .04 for the Number of EBPs supported and Fidelity Support; and r = −.11 for the Number of EBPs supported and Coalition Involvement (df = 61). The small correlations may reflect a limited pool of implementation resources that each component draws from, even though the amount of resources varies across coalitions.

Contextual characteristics

We include the contextual characteristics Community Poverty and Coalition Age as covariates in all analyses because previous research suggests they are linked to implementation success, yet are not modifiable through training or technical assistance (Brown, et al., 2010). Community Poverty is measured as the percentage of families below the poverty level (National Center for Educational Statistics, 2012). For the sake of determining Community Poverty for the communities in which CTC coalitions are embedded, community was delineated by the school districts where coalition supported preventive interventions were in place. Coalition Age was measured in years from the date of the coalition’s initial funding award to implement CTC.

Plan of analysis

To analyze the relations between coalition functioning and implementation support, we used cross-lagged panel models to estimate reciprocal causation (Finkel, 1995). Figure 1 illustrates our analytic model. We examined relations between each coalition functioning characteristic and two measures of implementation support (overall and EBP). For each pair of variables, we ran four separate models, with each model using a different year of coalition functioning (2004, 2005, 2006, and 2007) to predict implementation support as measured in 2008. In all of these models we used 2007 implementation support to predict 2008 coalition functioning because we did not measure implementation support in earlier years. Across all models, the sample size was 74 coalitions, as this was the number of coalitions that provided implementation support data in 2007 or 2008. All analyses took place at the coalition level because implementation support and observer-reported coalition functioning only exist at the coalition level. We aggregated self-reported coalition functioning to the coalition level by computing the mean of respondent ratings on each item within a coalition. Models were estimated using Mplus version 6.0 via the method of full information maximum likelihood, which has the advantage of being able to estimate missing data (e.g., Wothke, 2000). In all models, we controlled for Community Poverty and Coalition Age by modeling the path from each covariate to coalition functioning and implementation support, as measured in 2008.

Figure 1.

Figure 1

Open in a new tab

Analytic model: relations between coalition functioning and implementation support.

Note: Models used arrow F1 (without F2) to predict 2008 implementation support with coalition characteristics in 2004, 2005, and 2006. When 2007 coalition characteristics predicted 2008 implementation support, models instead used arrow F2. Arrows A–E were used consistently across all models.

Results

In 2007, 62 CTC coalitions reported implementing 213 prevention programs, of which 180 (85%) were evidence-based. The following year, 60 coalitions reported supporting 277 programs, with 210 (76%) being EBPs. The number of programs supported by a single coalition ranged from 1 to 10 and the number of EBPs supported ranged from 0 to 7. Combining data from 2007 and 2008, coalitions served as the primary sponsor for 32% of their prevention programs; coalitions co-sponsored or helped with funding for 35% of their supported programs; and coalitions provided consultation on the remaining 33% of programs supported. For EBPs supported in 2007 and 2008, 62% had developer-certified training, 13% had training from someone familiar with the program, 7% had informal training, and the remaining 18% had no training. Ongoing training or technical assistance was provided through developer-certified mechanisms for 46% of the EBPs, through a consultant familiar with the program for 15% of the EBPs, and not provided for 39% of EBPs. Fidelity monitoring based on observation was used by 28% of the EBPs; structured self-report monitoring for 40% of EBPs; non-systematic monitoring for 11% of EBPs; and fidelity was not monitored for 21% of EBPs.

Table 3 presents the results from cross-lagged panel models of coalition functioning and implementation support. The table focuses specifically on the standardized path estimate from earlier years of coalition functioning to later years of implementation support, as illustrated by arrows F1 and F2 in Figure 1. Consistent with our first hypothesis, results from Table 3 indicate that all aspects of coalition functioning have a significant relation with implementation support in at least one model (df = 73). Self-reported Coalition Efficiency, Leadership Style, Mobilizer Skill, and Sustainability Planning all significantly predicted Overall Implementation Support in one or more years but never predicted EBP Implementation Support. Observer-reported Cohesion and Mobilizer Skill as measured in 2005 and 2007 respectively, were significant predictors of EBP Implementation Support, but did not predict Overall Implementation Support. The coalition capacity Observer-Reported CTC Model Knowledge was the only coalition characteristic that consistently predicted EBP Implementation Support, with a magnitude that ranged from .22 to 26. This finding was consistent with our hypothesis that CTC Model Knowledge would be a particularly salient predictor of EBP Implementation Support. CTC Model Knowledge also significantly predicted Overall Implementation Support in 3 out of 4 years, where the standarized path coefficient ranged from .23 to .36 when significant (df = 73).

Table 3.

Standardized path estimates (standard errors) for specific Coalition Functioning (CF) characteristics predicting Overall and Evidence-Based Program (EBP) Implementation Support (IS).

Source Construct name 2004 CF predicts Overall IS Est. (S.E.) 2005 CF predicts Overall IS Est. (S.E) 2006 CF predicts Overall IS Est. (S.E.) 2007 CF predicts Overall IS Est. (S.E.) 2004 CF predicts EBP IS Est. (S.E.) 2005 CF predicts EBP IS Est. (S.E.) 2006 CF predicts EBP IS Est. (S.E.) 2007 CF predicts EBP IS Est. (S.E.)
Collaborative processes:
Self-report Coalition Efficiency .31* (.12) .21 (.13) .28* (.12) .19 (.12) -.03 (.12) .11 (.11) -.01 (.10) .03 (.10)
Observer Cohesion .12 (.13) .13 (.13) -.01 (.16) -.04 (.13) .16 (.10) .22* (.10) .20† (.12) .08 (.11)
Self-report Leadership Style .31* (.13) .42* (.13) .16 (.15) .16 (.14) -.03 (.12) .20 (.13) -.07 (.12) -.02 (.12)
Coalition capacities:
Self-report Mobilizer Skill .26† (.14) .24† (.13) .32* (12) .15 (.13) -.08 (.12) .08 (.11) .18† (.10) .05 (.11)
Observer Mobilizer Skill .01 (.13) .05 (.13) .05 (.13) .10 (.11) .10 (.10) .17 (.11) .15 (.10) .21* (.10)
Observer CTC Model Knowledge .23* (.12) .22† (.12) .36* (12) .26* (.11) .25* (.10) .24* (.10) .26* (.11) .22* (.10)
Coalition activities:
Self-report Sustainability Planning .37* (.11) .22† (.12) .19 (.12) .06 (.13) -.02 (.10) .04 (.10) -.01 (.10) -.14 (.11)
Open in a new tab

Note: Est. = Estimate, S.E. = Standard Error,

*

p < .05,

†

p < .10, df = 73

Findings regarding the inverse relationship between implementation support in 2007 and coalition functioning in 2008 (labeled in Figure 1 as Arrow “E”) supported our second hypothesis, in that neither Overall Implementation Support nor EBP Implementation Support were significant predictors of coalition functioning. Standardized path coefficients ranged from −.17 to .14. All measures of coalition functioning were stable across time (labeled in Figure 1 as arrow “D”), with standardized path coefficients that ranged from .44 to .66. Both Overall Implementation Support and EBP Implementation Support also exhibited strong stability (Arrow “C” in Figure 1), with standardized path coefficients that ranged from .25 to .38 for Overall Implementation Support and .64 to .73 for EBP Implementation Support.

The contextual characteristic of Community Poverty consistently displayed a negative relation with Overall Implementation Support, with standardized path coefficients that ranged from −.20 to −.36. However, Community Poverty significantly predicted EBP Implementation Support in only two models, with standardized path coefficients that ranged from −.08 to −.22 (df = 73). Community Poverty maintained a negative relation with measures of coalition functioning that ranged from −.02 to −.33. Coalition Age also typically had a negative relation with coalition functioning, ranging from −.38 to .03. Standardized path coefficients from Coalition Age to Overall and EBP Implementation Support were never significant, ranging from −.08 to .18.

Discussion

Results were consistent with our study hypotheses that coalition functioning would predict changes in implementation support, but not vice versa. These findings are consistent with conceptual models suggesting coalition functioning has an important influence on a coalition’s ability to support program implementation (e.g., Brown, et al., 2012). Thus, our findings point to the importance of training and technical assistance efforts, which can improve coalition functioning (Buller et al., 2011; Feinberg, et al., 2002; Feinberg, Ridenour, et al., 2008; Riggs, Nakawatase, & Pentz, 2008). However, technical assistance may be most effective when it focuses on those aspects of coalition functioning that are most important for coalition outcomes, as not all coalition functioning characteristics equally predicted support for program implementation.

Coalition member knowledge of the CTC model was clearly the best predictor of implementation support, especially for EBPs. Thus, coalition member understanding of a science-based approach to prevention and the activities related to installing CTC likely enhances coalition efforts to support the implementation of EBPs with fidelity (Feinberg, Bontempo, et al., 2008; Gomez, et al., 2005). This finding builds on the evidence that a science-based approach to prevention may be a key variable that distinguishes between sustainable coalitions that impact population health outcomes and ineffective, unsustainable prevention coalitions (Brown et al., 2013; Feinberg, Bontempo, et al., 2008; Flewelling, et al., 2005). This finding is also encouraging because the provision of CTC training to enhance knowledge is relatively straightforward, rendering CTC model knowledge a highly modifiable predictor of high quality program implementation. The results of this study suggest the value of further innovation and testing of CTC training delivery models that build knowledge in coalition members. For example, online trainings modules and certification procedures could be developed and their use required by funding agencies to help ensure CTC coalition members have basic knowledge of prevention science principles and the CTC process.

It is more difficult to interpret findings for the other coalition functioning characteristics, which predicted support for program implementation in some years but not others. The variation across years could represent random fluctuations due to a small sample size. Without a larger sample size that can produce more precise estimates, it is difficult to know.

The self-reported coalition functioning characteristics frequently predicted the overall amount of support for program implementation, suggesting they may help to enhance general coalition productivity. Observer ratings of coalition cohesion, mobilizer skill, and especially CTC model knowledge by technical assistance providers predicted support for the implementation of EBPs with fidelity. Thus, self-report measures of coalition functioning may be better at capturing coalition functioning characteristics that contribute to general productivity whereas ratings by trained observers may be more capable of assessing characteristics that contribute to high fidelity EBP implementation. The distinction between overall and EBP implementation support is important because the implementation of EBPs with fidelity is more likely to lead to population level health outcomes (Durlak & DuPre, 2008; Wandersman, et al., 2008). Coalitions that are unaware of the value of EBPs may work well together to implement programs that subjectively appear productive, but do not lead to reductions in delinquency and youth substance use (Hallfors, et al., 2002).

Limitations and future directions

This study cannot rule out the possibility that the identified relations between coalition functioning and implementation support are caused by an unidentified third variable. If this were the case, the unidentified factor would have to cause changes in coalition functioning first and implementation support later in time. Thus, it seems likely that coalition functioning is a factor in the coalition’s provision of support for program implementation. The use of propensity scores with a larger sample of coalitions could improve causal inference (Stuart et al., 2009).

Conclusion

This study improves understanding of how CTC coalitions support the implementation of EBPs with fidelity, helping to refine a conceptual model explaining how CTC coalitions alter community-level health. Based on these analyses, attention to coalition functioning appears important. Efforts to improve malleable coalition characteristics through training and technical assistance may improve the efficacy of coalition efforts (Feinberg, et al., 2002; Riggs, et al., 2008). It seems particularly important to build coalition member understanding of the CTC strategy, which adopts a science-based approach to prevention, emphasizing high quality implementation by selecting EBPs that respond to observed community needs and by monitoring implementation fidelity (Fagan, Hanson, Hawkins, & Arthur, 2008; Hawkins, et al., 2002).

Acknowledgments

This research is supported by a grant from the National Institute of Drug Abuse (R03DA027942) and the Pennsylvania Commission for Crime and Delinquency (PCCD). Additionally, this article is supported in part by the National Cancer Institute through a Community Networks Program Center grant U54 CA153505. Findings and recommendations herein are not official statements of the NIDA, PCCD, or NCI. The authors acknowledge the enthusiastic support of Michael Pennington, Ruth Williams, Douglas Hoffman, Raymond Moneta, and Clay R. Yeager at PCCD. We are also grateful for the cooperation of the state-funded Communities That Care (CTC) technical assistance providers and the many CTC coalition members in Pennsylvania.

References

  1. Arthur MW, Hawkins DJ, Catalano RF, Olson JJ. Diffusion project: Fall 1998 community key informant interview. Seattle: University of Washington, Social Development Research Group; 1998. [Google Scholar]
  2. Brown EC, Hawkins JD, Rhew IC, Shapiro VB, Abbott RD, Oesterle S, Catalano RF. Prevention system mediation of Communities That Care effects on youth outcomes. Prevention Science, Online First. 2013 doi: 10.1007/s11121-013-0413-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Brown LD, Feinberg ME, Greenberg MT. Determinants of community coalition ability to support evidence-based programs. Prevention Science. 2010;11:287–297. doi: 10.1007/s11121-010-0173-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Brown LD, Feinberg ME, Greenberg MT. Measuring coalition functioning: Refining constructs through factor analysis. Health Education & Behavior. 2012;39:486–497. doi: 10.1177/1090198111419655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Buller DB, Young WF, Bettinghaus EP, Borland R, Walther JB, Helme D, Maloy JA. Continued benefits of a technical assistance web site to local tobacco control coalitions during a state budget shortfall. Journal of Public Health Management and Practice. 2011;17(2):E10–E19. doi: 10.1097/PHH.0b013e3181df7f6c. [DOI] [PubMed] [Google Scholar]
  6. Butterfoss FD. Coalitions and partnerships in community health. San Francisco: Jossey-Bass; 2007. [Google Scholar]
  7. Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health Education Research. 1993;8:315–330. doi: 10.1093/her/8.3.315. [DOI] [PubMed] [Google Scholar]
  8. Butterfoss FD, Kegler M. Toward a comprehensive understanding of community coalitions: Moving from practice to theory. In: DiClemente RJ, Crosby RA, Kegler M, editors. Emerging theories in health promotion practice and research: Strategies for improving health. San Francisco: Jossey-Bass; 2002. pp. 157–193. [Google Scholar]
  9. Durlak JA, DuPre EP. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology. 2008;41:327–350. doi: 10.1007/s10464-008-9165-0. [DOI] [PubMed] [Google Scholar]
  10. Fagan AA, Hanson K, Hawkins JD, Arthur MW. Bridging science to practice: Achieving prevention program implementation fidelity in the Community Youth Development Study. American Journal of Community Psychology. 2008;41:235–249. doi: 10.1007/s10464-008-9176-x. [DOI] [PubMed] [Google Scholar]
  11. Feinberg ME, Bontempo D, Greenberg MT. Predictors and level of sustainability of community prevention coalitions. American Journal of Preventive Medicine. 2008;34:495–501. doi: 10.1016/j.amepre.2008.01.030. [DOI] [PubMed] [Google Scholar]
  12. Feinberg ME, Gomez B, Puddy RW, Greenberg MT. Evaluation and community prevention coalitions: Validation of an integrated web-based/technical assistance consultant model. Health Education and Behavior. 2008;35:9–21. doi: 10.1177/1090198105282413. [DOI] [PubMed] [Google Scholar]
  13. Feinberg ME, Greenberg MT, Osgood DW, Anderson A, Babinski L. The effects of training community leaders in prevention science: Communities That Care in Pennsylvania. Evaluation and Program Planning. 2002;25:245–259. [Google Scholar]
  14. Feinberg ME, Jones D, Greenberg MT, Osgood DW, Bontempo D. Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science. 2010;11:163–171. doi: 10.1007/s11121-009-0161-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Feinberg ME, Ridenour TA, Greenberg MT. The longitudinal effect of technical assistance dosage on the functioning of Communities That Care prevention boards in Pennsylvania. Journal of Primary Prevention. 2008;29:145–165. doi: 10.1007/s10935-008-0130-3. [DOI] [PubMed] [Google Scholar]
  16. Finkel SE. Causal analysis with panel data. Thousand Oaks, CA: Sage; 1995. [Google Scholar]
  17. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida; 2005. [Google Scholar]
  18. Flewelling RL, Austin D, Hale K, LaPlante M, Liebig M, Piasecki L, Uerz L. Implementing research-based substance abuse prevention in communities: Effects of a coalition-based prevention initiative in Vermont. Journal of Community Psychology. 2005;33:333–353. doi: 10.1002/jcop.20052. [DOI] [Google Scholar]
  19. Foster-Fishman PG, Berkowitz SL, Lounsbury DW, Jacobson S, Allen NA. Building collaborative capacity in community coalitions: A review and integrative framework. American Journal of Community Psychology. 2001;29:241–261. doi: 10.1023/A:1010378613583. [DOI] [PubMed] [Google Scholar]
  20. Gomez B, Greenberg MT, Feinberg ME. Sustainability of community coalitions: An evaluation of communities that care. Prevention Science. 2005;6:199–202. doi: 10.1007/s11121-005-0003-4. [DOI] [PubMed] [Google Scholar]
  21. Greenberg MT, Feinberg ME, Meyer-Chilenski S, Spoth RL, Redmond C. Community and team member factors that influence the early phase functioning of community prevention teams: The PROSPER project. Journal of Primary Prevention. 2007;28:485–504. doi: 10.1007/s10935-007-0116-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Haggerty KP, Skinner ML, MacKenzie EP, Catalano RF. A randomized trial of Parents Who Care: Effects on key outcomes at 24-month follow-up. Prevention Science. 2007;8:249–260. doi: 10.1007/s11121-007-0077-2. [DOI] [PubMed] [Google Scholar]
  23. Hallfors D, Hyunsan C, Livert D, Kadushin C. Fighting back against substance abuse: Are community coalitions winning? American Journal of Preventive Medicine. 2002;23:237–245. doi: 10.1016/s0749-3797(02)00511-1. [DOI] [PubMed] [Google Scholar]
  24. Hardy KV, Moore M, Rose D, Bennett R, Jackson-Lane C, Gause M, Loewy R. Filling the implementation gap: A community–academic partnership approach to early intervention in psychosis. Early Intervention in Psychiatry. 2011;5:366–374. doi: 10.1111/j.1751-7893.2011.00310.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hawkins JD, Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addictive Behaviors. 2002;27:951–976. doi: 10.1016/s0306-4603(02)00298-8. [DOI] [PubMed] [Google Scholar]
  26. Hawkins JD, Oesterle S, Brown EC, Monahan KC, Abbott RD, Arthur MW, Catalano RF. Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial. Archives of Pediatrics & Adolescent Medicine. 2012;166:141–148. doi: 10.1001/archpediatrics.2011.183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning. 2004;27:135–149. doi: 10.1016/j.evalprogplan.2004.01.002. [DOI] [Google Scholar]
  28. Jones KM, Wickstrom KF, Friman PC. The effects of observational feedback on treatment integrity in school-based behavioral consultation. School Psychology Quarterly. 1997;12:316–326. [Google Scholar]
  29. Kegler MC, Steckler A, McLeroy K, Malek SH. Factors that contribute to effective community health promotion coalitions: A study of ten Project ASSIST coalitions in North Carolina. Health Education and Behavior. 1998;25:338–353. doi: 10.1177/109019819802500308. [DOI] [PubMed] [Google Scholar]
  30. Kegler MC, Swan DW. An initial attempt at operationalizing and testing the community coalition action theory. Health Education & Behavior. 2011;38:261–270. doi: 10.1177/1090198110372875. [DOI] [PubMed] [Google Scholar]
  31. Lasker RD, Weiss ES. An overview of the purpose, content, and psychometric properties of the Questionnaire for Individual Partners. New York: Center for the Advancement of Collaborative Strategies in Health; 2000. [Google Scholar]
  32. Lasker RD, Weiss ES. Broadening participation in community problem solving: A multidisciplinary model to support collaborative practice and research. Journal of Urban Health. 2003;80:14–47. doi: 10.1093/jurban/jtg014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Moos RH. Group environment scale manual: Development, applications, research. 3. Menlo Park, CA: Mind Garden; 2002. [Google Scholar]
  34. National Center for Educational Statistics. Poverty status of families by school district 2005 to 2009. 2012 Retreived from: http://nces.ed.gov/surveys/sdds/acs09/acsdownload.aspx.
  35. O’Malley AJ, Landon BE, Guadagnoli E. Analyzing multiple informant data from an evaluation of the Health Disparities Collaboratives. Health Services Research. 2007;42:146–164. doi: 10.1111/j.1475-6773.2006.00597.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Perkins DF, Feinberg ME, Greenberg MT, Johnson LE, Chilenski SM, Mincemoyer CC, Spoth RL. Team factors that predict to sustainability indicators for community-based prevention teams. Evaluation and Program Planning. 2011;34:283–291. doi: 10.1016/j.evalprogplan.2010.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Rhew IC, Brown EC, Hawkins JD, Briney JS. Sustained effects of Communities That Care on prevention service system transformation. American Journal of Public Health. 2013;103:529–535. doi: 10.2105/AJPH.2011.300567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Rhoades BL, Bumbarger BK, Moore JE. The role of a state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. American Journal of Community Psychology. 2012;50:386–401. doi: 10.1007/s10464-012-9502-1. [DOI] [PubMed] [Google Scholar]
  39. Riggs NR, Nakawatase M, Pentz MA. Promoting community coalition functioning: Effects of Project STEP. Prevention Science. 2008;9:63–72. doi: 10.1007/s11121-008-0088-7. [DOI] [PubMed] [Google Scholar]
  40. Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health. 2000;21:369–402. doi: 10.1146/annurev.publhealth.21.1.369. [DOI] [PubMed] [Google Scholar]
  41. Shapiro VB, Hawkins DJ, Oesterle S. The moderating effect of building coalition capacity on the relationship between Communities That Care and community adoption of a science-based approach to prevention. Under review 2013 [Google Scholar]
  42. Shapiro VB, Oesterle S, Abbott RD, Arthur MW, Hawkins JD. Measuring dimensions of coalition functioning for effective and participatory community practice. Social Work Research. doi: 10.1093/swr/svt028. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Spoth R, Redmond C, Shin C, Greenberg M, Feinberg M, Schainker L. Longitudinal effects of universal PROSPER community–university partnership delivery system effects on substance misuse through 6 1/2 years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine. 2013;56:190–196. doi: 10.1016/j.ypmed.2012.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Spoth RL, Greenberg MT, Turrisi R. Preventive interventions addressing underage drinking: State of the evidence and steps toward public health impact. Pediatrics. 2008;121:S311–S336. doi: 10.1542/peds.2007-2243E. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Spoth RL, Guyll M, Lillehoj CJ, Redmond C, Greenberg MT. Prosper study of evidence-based intervention implementation quality by community-university partnerships. Journal of Community Psychology. 2007;35:981–999. doi: 10.1002/jcop.20207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Stuart EA, Marcus SM, Horvitz-Lennon MV, Gibbons RD, Normand SLT, Brown CH. Using non-experimental data to estimate treatment effects. Psychiatric Annals. 2009;39:719–728. doi: 10.3928/00485713-20090625-07. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the gap between prevention research and practice: The interactive systems framework for dissemination and implementation. American Journal of Community Psychology. 2008;37:171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]
  48. Wothke W. Longitudinal and multi-group modeling with missing data. In: Little TD, Schnabel KU, Baumert J, editors. Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples. Mahwah, NJ: Lawrence Erlbaum Associates; 2000. [Google Scholar]
  49. Zakocs RC, Edwards EM. What explains community coalition effectiveness? A review of the literature. American Journal of Preventive Medicine. 2006;30:351–361. doi: 10.1016/j.amepre.2005.12.004. [DOI] [PubMed] [Google Scholar]
  50. Zakocs RC, Guckenburg S. What coalition factors foster community capacity? Lessons learned from the fighting back initiative. Health Education & Behavior. 2007;34:354–375. doi: 10.1177/1090198106288492. [DOI] [PubMed] [Google Scholar]

RESOURCES