Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Nov 1.
Published in final edited form as: Prev Sci. 2015 Nov;16(8):1136–1146. doi: 10.1007/s11121-015-0562-y

Building Local Infrastructure for Community Adoption of Science-Based Prevention: The Role of Coalition Functioning

Valerie B Shapiro 1,✉, J David Hawkins 2, Sabrina Oesterle 2
PMCID: PMC4623940  NIHMSID: NIHMS695321  PMID: 26017632

Abstract

The widespread adoption of science-based prevention requires local infrastructures for prevention service delivery. Communities That Care (CTC) is a tested prevention service delivery system that enables a local coalition of community stakeholders to use a science-based approach to prevention and improve the behavioral health of young people. This paper uses data from the Community Youth Development Study (CYDS), a community-randomized trial of CTC, to examine the extent to which better internal team functioning of CTC coalitions increases the community-wide adoption of science-based prevention within 12 communities, relative to 12 matched comparison communities. Specifically, this paper examines the potential of both a direct relationship between coalition functioning and the community-wide adoption of science-based prevention and a direct relationship between functioning and the coalition capacities that ultimately enable the adoption of science-based prevention. Findings indicate no evidence of a direct relationship between four dimensions of coalition functioning and the community-wide adoption of a science-based approach to prevention, but suggest a relationship between coalition functioning and coalition capacities (building new member skills and establishing external linkages with existing community organizations) that enable science-based prevention.

Keywords: Communities That Care, Coalition, Functioning, Capacities, Science-based prevention, Community-level intervention

A large body of scientific evidence has identified predictors of undesirable health and behavioral outcomes, has demonstrated the potential to reliably intervene with those predictors to reduce undesirable outcomes, and has shown that some of these preventive interventions save more public dollars than they cost (Kuklinski et al. 2012; O'Connell et al. 2009). Concerned about the insufficient impact of this research on the nation’s public health, the Society for Prevention Research (SPR) has called for bold action that would enable communities to conduct effective prevention practice for our countries’ mental, emotional, behavioral, and fiscal health (SPR 2009). Specifically, SPR calls for promoting the community-wide adoption of science-based prevention by building prevention infrastructure.

Adopting a science-based approach to prevention involves several activities, including the collection of local epidemiological data to obtain reliable and valid population-level estimates of the prevalence of preventable problems as well as estimates of the risk and protective factors shown by research to predict health and behavior problems. A science-based approach also involves using those data to prioritize risk and protective factors to target for change, and creating action plans for the implementation of programs and policies that have been tested and demonstrated to be effective in changing the targeted risk/protective factors and problem behaviors. Finally, science-based prevention includes regularly collecting data to monitor programs and policies and ensure they are well implemented and effective in the local context (Hawkins et al. 2002). Infrastructure that is needed to conduct science-based prevention includes a system of supports for research and innovation, training, and service delivery (O'Connell et al. 2009). Infrastructure for research and training is most efficiently developed at the state and federal levels. Infrastructure for service delivery is needed at the local level, where problems are defined and responses occur (Brown et al. 2014b; Wandersman et al. 2008). In this paper, we examine the extent to which a local infrastructure for service delivery (in the form of coalitions) enables science-based prevention.

The most clearly articulated and tested prevention service delivery models intended to enhance a community’s use of science-based prevention are the PROmoting School-community-university Partnerships to Enhance Resilience (PROSPER) model (Spoth et al. 2004, 2011) and the Communities That Care (CTC) system (Hawkins et al. 2002; Hawkins, Catalano, and Associates 1992). Both PROSPER and CTC build coalitions as an infrastructure for prevention services, attempting to change prevention practice, and ultimately prevent mental, emotional, and behavioral problems in young people. Coalitions, as local infrastructures, bring together diverse stakeholders (i.e., youth, parents, advocates, residents, business owners, religious leaders, educators, law enforcement) to advance a vision for youth development. Community coalitions “advocate for reform in structural arrangements for delivering and accessing healthcare, education, social welfare, and other human services” (Roberts-DeGennaro and Mizrahi 2005, p. 305). Although PROSPER and CTC are both coalition-based approaches, there are important differences between them. While PROSPER focuses on drug use prevention, CTC focuses on impacting shared risk and protective factors for multiple health and behavior problems (Haggerty and Shapiro 2013). CTC does not prescribe specific programs for parents and students, but trains the local coalition to choose from a menu of tested programs that best address the community’s profile of risk and protection. Also, CTC does not prescribe who leads the local prevention effort, but encourages a variety of organizations in the community to take leadership. We use data from the Community Youth Development Study (CYDS), a community-randomized controlled trial of CTC, to investigate how CTC coalitions enable community change toward the adoption of science-based prevention.

Determining when and how community change occurs as the result of a community-level intervention is challenging (Coulton 2005). Making a claim of intervention-induced change requires the dual determination that the community intervention has actually occurred and that the community’s approach to prevention has changed, ideally in the context of a randomized trial that can assert a causal link between the intervention and the outcome. Data from the CYDS provide this opportunity. Twenty-four communities were matched into pairs, and communities from each matched pair were randomly assigned to the CTC or control condition, providing 12 pairs of communities for comparisons (Arthur et al. 2005; Hawkins et al. 2008).

A comprehensive checklist of the major milestones, and the benchmarks needed to achieve each milestone, was used to assess whether each community progressed through the five stages of implementing CTC (Quinby et al. 2008). Each stage of CTC involves training and technical assistance to facilitate the completion of a series of tasks, some of which are described in Table 1. Within the first year of CTC implementation in the CYDS, ratings from coalition coordinators and external observers determined that CTC coalitions had achieved 89 to 100 % of the expected CTC milestones and benchmarks (Quinby et al. 2008). These data indicate that a local infrastructure for prevention service delivery was built in the CYDS CTC communities.

Table 1.

Tasks exemplifying each stage of Communities That Care (CTC)

Stage 1: Getting Started

  • Secure the support of key community leaders for CTC

  • Identify community stakeholders concerned with youth behavioral health problems

  • Decide the scope of the prevention effort

  • Ensure support for community collaboration and science-based Prevention

  • Address any community prevention readiness issues
Stage 2: Organizing, Introducing, and Involving

  • Orient and engage community leaders in CTC

  • Create or identify a coalition to facilitate system transformation with an articulated organizational structure and operational procedures

  • Develop a community vision for youth development

  • Create a plan for continuous community engagement
Stage 3: Developing a Community Profile

  • Collect data with the CTC Youth Survey (Arthur et al. 2007)

  • Prioritize risk and protective factors to target for change

  • Identify existing preventive interventions in the community that affect prioritized risk and protective factors as well as gaps in existing preventive services
Stage 4: Creating a Community Action Plan

  • Fill service gaps with programs, policies, and practices that have been tested and found to be effective at addressing the prioritized risk and protective factors and reducing health and behavior problems (see www.blueprintsprograms.com)

  • Develop a work plan, budget, timeline, and evaluation plan for each action item

  • Communicate the plan to community leaders and the community
Stage 5: Implementing and Evaluating the Community Action Plan

  • Implement the new programs, policies, and practices

  • Monitor programs, policies, and practices to ensure they are reaching the target population with requisite levels of fidelity

  • Adjust the programs, policies, and practices as necessary

  • Evaluate the outcomes of programs, policies, and practices

  • Conduct new community-level assessments at least every 2 years
Open in a new tab

Despite the high levels of CTC implementation observed in intervention communities, it was unclear whether CTC implementation would result in community change, as indicated by the extent to which intervention communities organized their delivery of prevention services in ways that were more consistent with science-based prevention relative to control communities. In other words, the availability of a coalition-based infrastructure for prevention services may or may not change actual community prevention services from business as usual. To assess whether a community-wide change in approach to prevention services has occurred, it is important to assess intervention communities over time and relative to control (prevention-as-usual) communities. Prior to the start of the CYDS, communities assigned to implement CTC were not statistically different from communities assigned to the control condition in their use of science-based prevention (Brown et al. 2007). At 1.5 years into the CYDS intervention period, a significant increase in, and greater levels of community-wide adoption of science-based prevention were observed in CTC than in control communities. Thus, the CYDS provides evidence that CTC implementation facilitates the community-wide adoption of a science-based approach to prevention. The adoption of science-based prevention has been shown to mediate the effects of CTC on youth outcomes (Brown et al. 2014a).

This paper specifically investigates how CTC coalitions facilitate the community-wide adoption of a science-based approach to prevention. Although one might expect that giving communities resources and ensuring their performance of the CTC milestones and benchmarks is sufficient to increase community adoption of science-based prevention (Quinby et al. 2008), analysis of the CYDS data has revealed that CTC communities varied significantly in the stage of science-based prevention they achieved. In other words, performance of the activities prescribed in the CTC model was uniformly high, yet community-wide adoption of science-based prevention was not (Shapiro et al. 2013a). Some CTC coalitions were more successful than others in achieving a community-wide adoption of a science-based approach to prevention. Florin et al. (2000) have suggested that a coalition’s success is determined not only by the activities of that coalition, but also by the way the coalition functions as a team and the extent to which the coalition has the necessary capacities to create community changes.

Florin et al. (1993) suggests that the internal functioning of a coalition (i.e., the extent to which coalitions are focused, efficient, participatory in their leadership style, and cohesive) impacts the success of the coalition by providing a context in which individual and organizational capacities (i.e., member skills, organizational linkages) develop. This theory (Fig. 1) argues that high coalition functioning is a facilitating context in which capacities are nurtured, presenting coalition functioning as a requisite condition for the nurturance of essential coalition capacities for change. For example, high levels of coalition functioning may sustain participation in coalitions in ways that retain and grow member skills (Roussos and Fawcett 2000; Wells et al. 2008). A participatory leadership style may affect coalition successes by influencing the individual capacities of coalition members (Kegler and Swan 2012) and the organizational capacities of the coalitions (Zakocs and Guckenburg 2007).

Fig. 1.

Fig. 1

Open in a new tab

Two alternative conceptual models depicting the relationship between coalition functioning and community change toward the community-wide adoption of science-based prevention

Alternatively, coalition functioning may be a coalition capacity in itself that has a direct influence on the coalition’s achievement of its objectives, such as the community-wide adoption of science-based prevention (Foster-Fishman et al. 2001). Support for this alternative hypothesis comes from studies that find that member perceptions of opportunities for participation in coalition decision making are directly related to the coalition chair’s account of coalition accomplishments (Allen 2005). In one study, a participatory orientation was cited by 80 % of coalition coordinators as the rationale for their coalition success (Mizrahi and Rosenthal 2001). In addition, member perceptions of coalition cohesion have been credited with coalition success (Kegler et al. 1998; Mizrahi and Rosenthal 2001; Zakocs and Edwards 2006). Some have argued that clarity and focus of the shared mission guiding the coalition’s work (the coalition’s directedness) are critical to coalition success (Brown et al. 2012; Himmelman et al. 2001; Shortell et al. 2002). Lastly, a general task orientation, or a coalition’s efficiency, has been related to the perceptions of coalition impact by leaders in the community (Florin et al. 2000). However, there is mixed evidence regarding the direct effect of coalition functioning on coalition achievement. When studies have used a global or second-order construct of coalition functioning to predict coalition impact, some have found that member-reported coalition functioning was directly related to member perceptions of coalition effectiveness (Brown et al. 2010, 2012; Feinberg et al. 2004; Hays et al. 2000). Others have found no direct relationship between internal functioning and coalition achievement (Crowley et al. 2000).

Previous analyses of data from the CYDS have provided some evidence of the importance of coalition capacities in the achievement of the coalition’s goals (Shapiro et al. 2015). Those analyses indicated that building the skills of coalition members predicted the degree of coalition success in fostering community-wide adoption of a science-based approach to prevention. These analyses also produced strong evidence that the involvement of diverse community sectors in the work of the coalition (organizational linkages) predicted the community-wide adoption of a science-based approach to prevention. Other capacities (such as coalition member knowledge, attitudes, and perceived influence on diverse community sectors) did not explain the variation in achievement of science-based prevention.

Given the competing theories specifying the relationship between coalition functioning and coalition achievement, and mixed empirical evidence supporting these theories, this paper explores the relations between coalition functioning, coalition capacities, and the community-wide adoption of science-based prevention in the context of a community-randomized trial. Specifically, we examine whether coalition functioning is directly related to the achievements of CTC coalitions, directly related to important coalition capacities that enable the adoption of science-based prevention, or unrelated to the adoption of science-based prevention. To do this, we pose three research questions. 1. To what extent do CTC coalitions differ in four dimensions of functioning, as perceived by coalition members, 1 year after CTC had begun? Although individual coalition member perceptions of functioning may vary within coalition, we expect a substantial proportion of the variation to reflect differences between coalitions. 2. To what extent do high levels of coalition functioning predict community-wide adoption of science-based prevention? 3. To what extent do high levels of coalition functioning predict high levels of the two coalition capacities previously found to influence community-wide adoption of science-based prevention: (a) coalition member skills and (b) linkages between the CTC coalition and other organizations in the community (“organizational linkages”). Understanding the association between coalition functioning and the adoption of science-based prevention may suggest ways to target and sequence training and technical assistance to increase coalition effectiveness.

Methods

Sample

The CYDS recruited communities from Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington (Arthur et al. 2005) and matched them within states into similar pairs by population size, racial and ethnic diversity, economic indicators, and crime rates. The communities that were randomized from each pair into the experimental and control conditions were small- to moderate-sized towns with population sizes that ranged from 1578 to 40,787 people (average 14,646) and percentages of White residents that ranged from 64 to 98 % (average, 89 %). Hispanic/Latino was the second most prevalent race/ethnic group (on average, 20 %), comprising 1 to 65 % of the population within the various communities. CTC communities were provided training and technical assistance in the implementation of CTC, funding for a full-time, local coordinator to oversee CTC activities, and $75,000 annually for selected prevention programs. Control communities received annual reports summarizing data provided from youth in their communities that was collected for the study, but no other training, technical assistance, or financial support from the CYDS (see Arthur et al. (2010) and Fagan et al. (2009) for information about the activities of coalitions).

CTC coalition members assessed their coalition’s functioning and capacities in the spring of 2004, 1 year after initiating CTC. Community leader reports of the adoption of science-based prevention occurred in the fall of 2004, 1.5 years after initiating CTC. This temporal ordering reflects CTC’s theory of change, which posits that training and technical assistance provided over the 1st year of implementation will enhance coalition functioning and capacities in ways that will subsequently enable the coalition to promote community changes. In the 12 CTC communities, coalition member respondents (maximum 20 per community) were randomly selected from coalition member rosters (Shapiro et al. 2013a). The percentage of interviews completed was 96.6 %. A total of 218 CTC coalition members participated. Race, class, and gender information was not collected from these respondents.

Community leaders reported on their community’s adoption of a science-based approach to prevention (Arthur et al. 2005). In each of the 24 CYDS communities, individuals in formal leadership positions (e.g., mayors, city managers, police chiefs, school superintendents, business leaders, and heads of health and social service agencies) across 11 different community sectors were interviewed by phone. Leaders were asked to identify two individuals who were the most knowledgeable about prevention efforts in the community. Five additional leaders per community were interviewed. This strategy yielded 335 complete interviews (96 % of eligible respondents) of key leaders across CTC and comparison communities (Brown et al. 2007), only 3.5 % of whom served on CTC coalitions. The samples of leaders did not differ across intervention condition in response rate, age, gender, level of education, length of time living in the community, or community sector that the leaders represented (Brown et al. 2007).

Measures

Information about coalition functioning and capacities was collected from CTC coalition members using a 40-min structured phone interview (Shapiro et al. 2013b). An 18-item instrument measured four theoretically derived, empirically validated dimensions of coalition functioning: goal directedness, efficiency, opportunities for participation, and cohesion. These data were collected in the context of an efficacy trial where technical assistance was provided to optimize functioning, so high levels and limited variance in functioning are expected and observed (Shapiro et al. 2013b). Items generally use a 4-point scale from strongly disagree to strongly agree, where higher scores indicate higher functioning. Goal direct-edness (α=.69) is a five-item scale that measures clarity of focus of coalition operations. Efficiency (α=.80) is a six-item scale that measures the task orientation of a coalition. Opportunities for participation (α=.65) is a four-item scale that measures the degree of inclusion fostered by coalition leadership. Cohesion (α=.64) is a three-item scale that measures the extent to which coalition members feel bonded to one another through coalition operations. Each dimension of functioning is represented by a latent factor score generated in Mplus (Muthén and Muthén 2010).

In a previous analysis of data from the CYDS, two measured components of coalition capacity were found to predict community-wide adoption of a science-based approach to prevention: members’ new skills and the coalition’s organizational linkages to diverse community sectors (Shapiro et al. 2015). Member skills is an average of three items asking coalition members if they have benefited (a great deal to not at all) from learning new skills in the areas of organization and communication, changing local politics, and designing and carrying out prevention programs. Organizational linkages is the average degree of involvement in the CTC coalition’s work (from very involved to not at all involved) of 13 different community sectors (elected leaders, parents, school teachers and staff, school district administrators, social service providers, students, business leaders, faith leaders, law enforcement, media representatives, recreation officials, community volunteers, and unelected leaders).

The adoption of a science-based approach to prevention was assessed using hour-long structured interviews with community leaders in both CTC and control communities (Arthur et al. 2005). The questions were not CTC-specific, but used generic language to assess the extent to which communities were using a science-based approach to prevention in their communities (e.g., “Did your community prioritize risk and protective factors that you wanted to address with prevention activities?”). The adoption score, on an ordinal scale from 0 to 5, was determined based on the highest stage of adoption reached (Brown et al. 2007). The adoption score indicates that the community had met the criteria for the indicated stage of adoption as well as all prior stages of adoption on the scale. An adoption score of 0 indicates little or no awareness of prevention science concepts such as risk and protective factors. A score of 1 indicates an awareness of risk and protection focused prevention, but no use of these concepts to guide prevention services. A score of 2 indicates that the community is seeking to reduce risks for problems, but not collecting epidemiologic data. A score of 3 indicates that the community is collecting epidemiologic risk and protective factor data, but is not using that data to select preventive interventions that have been tested and determined to be effective for addressing those risks. A score of 4 indicates that the community has collected epidemiological data and developed a plan to use tested and effective preventive interventions to address prioritized risk and protective factors based on those data. A score of 5 indicates that the community has met the criteria for all prior stages and is implementing tested and effective preventive interventions and conducting ongoing assessments to monitor implementation and effects of the interventions.

Previous analyses of CYDS data have determined that community leaders in CTC communities reported greater levels of adoption of science-based prevention 1.5 years after implementation of CTC relative to community leaders in control communities (Brown et al. 2007). Prior analyses of CYDS data have also determined that 1.5 years after the implementation of CTC began, community adoption scores in CTC communities ranged from 1.87 to 3.73 (M=2.80, SD=.55) on a scale from 0 to 5, indicating that leaders typically reported that their communities were collecting epidemiological risk and protective factor data but not consistently selecting tested and effective preventive interventions. Community adoption scores in control communities ranged from 0.62 to 3.29 (M= 1.69, SD=.79), indicating that leaders typically were aware of prevention science concepts, but on average were not yet using them to guide prevention practice (Shapiro et al. 2013a). Standardized mean differences in adoption scores between CTC and control communities within each of the 12 matched pairs were calculated by dividing the difference between average adoption scores within pairs by the pooled variance (Lipsey and Wilson 2001). Because of the small sample size, a Hedges Correction was applied to generate unbiased effect size estimates (Hedges 1981). Positive effect size values indicated that leaders in intervention communities reported higher levels of adoption of science-based prevention than their counterparts in control communities. Prior analyses of CYDS data determined that the overall standardized mean difference between CTC and control communities in adoption was d=.78, with 37 % of the total variation in adoption attributable to differences between community pairs (Shapiro et al. 2013a). The present study examined whether coalition functioning was significantly associated with this variance in community adoption of science-based prevention as reported by local leaders.

Analysis Strategy

Question 1 To understand the extent to which the level of four dimensions of coalition functioning varied across the 12 CTC coalitions, four unconditional multilevel models (one for each dimension of functioning) were estimated to determine the proportion of variance attributed to differences among members and/or differences among CTC coalitions (Muthén and Muthén 2010). The 2-level hierarchical linear model with random intercept nested coalition members (level 1) within coalitions (level 2): Yij=β0j+rij; whereas β0j =γ00+µ0j.
Question 2 Coalition functioning factor scores (based on coalition member reports) were then mean aggregated at the coalition level for a series of meta-regressions (Hedges and Olkin 1985) that could relate coalition functioning in the CTC communities to the effect size differences in the adoption of science-based prevention (based on community leader reports) calculated for each of the 12 matched community pairs. This meta-analytic approach, described elsewhere in detail (Shapiro et al. 2013a), permits examination of the impact of coalition functioning, which was observed only in the 12 intervention communities, on the effect size difference in community leader reports of adoption of science-based prevention between matched pairs of intervention and control communities (Shapiro et al. 2015). The outcome variable used in each meta-regression was the unbiased effect size estimate for each pair, and the predictor in each model was one of the dimensions of coalition functioning hypothesized to influence the size of the intervention effect. Fixed effect meta-regressions were estimated in SPSS (v.19) with the METAREG.SPS macro (Wilson 2005), assuming that the variability in the effect size differences between pairs of communities is systematic and exclusively due to variation in the dimension of coalition functioning under examination. In a meta-regression, the sum of squares for the regression model (QR) is analogous to an F test in traditional regression analysis. If significant, the predictor is significantly associated with heterogeneity in the effect size difference in adoption of science-based prevention. If the sum of squares of the residual (QE) is significant, variability in the effect size difference in adoption of science-based prevention remains to be explained. In cases where QR and QE for the fixed effect model are both significant, the model is re-specified as a mixed effect model to consider sources of variance in addition to the moderator (Lipsey and Wilson 2001; Raudenbush 1994).
Question 3 Eight multilevel models (one for each of the four dimensions of functioning as related to each of the two coalition capacities) were estimated to determine the extent to which each dimension of functioning predicted levels of capacities (new skills and organizational linkages). The four dimensions were modeled independently because high intercorrelations would obscure relationships (Allen 2005). These dimensions have moderate shared variance (average of 64 %), but modeling them independently speaks to the practice literature that seeks to enhance these dimensions of functioning separately (Shapiro et al. 2013b).
Open in a new tab

Results

Question 1 The intraclass correlations (ICCs) indicate the proportion of total variance that is due to between-coalition variance. Between 7 and 8 % of the total variance in goal directedness and opportunities for participation was attributable to coalition-level differences. Twelve percent of the total variance in efficiency and 17 % of the total variance in cohesion was attributable to differences among coalitions. This suggests that CTC coalitions varied in the levels of functioning and that the greatest variance among coalitions was in cohesion.
Question 2 Fixed effect meta-regressions showed that none of the four dimensions were significantly associated with the effect size differences in community leader reports of the adoption of science-based prevention (see Table 2). Each model generated a significant QE statistic, indicating that residual variance in adoption between matched pairs was left to be explained. Because no dimensions of coalition functioning directly explained the variance in the relationship between CTC implementation and the adoption of science-based prevention, no mixed effect models were specified. Mixed effect models tend to produce regression coefficients with larger confidence intervals and are thus insufficiently powered to detect effects that were not detected in fixed effect models (Berkey et al. 1995).
Question 3 Results of the multilevel models indicated strong relationships between coalition member-reported functioning and coalition member-reported capacities. As shown in Table 3, significant (p<.001) positive relationships were found between all of the dimensions of coalition functioning and the two capacities under study, suggesting that better functioning coalitions also had greater capacity in the form of acquired member skills and linkages with diverse community sectors. These particular capacities are associated with reaching higher stages of adoption of science-based prevention (Shapiro et al. 2015), which has in turn been shown to mediate the effect of CTC on the behavioral health of young people (Brown et al. 2014a).
Open in a new tab

Table 2.

Meta-regression of adoption of a science-based approach to prevention on dimensions of coalition functioning

Coalition functioning b SE z β QR QE r2
Goal directedness −.14 .40 −.35 −.06 .12 37.34*** .00
Efficiency .07 .34 .20 −.03 .04 37.43*** .00
Opportunities for participation −.09 .42 −.21 −.03 .04 37.42*** .00
Cohesion .08 .32 .24 −.04 .06 37.41*** .00
Open in a new tab

Each coalition functioning dimension was entered separately in the regression QR sum of squares for the regression, QE sum of squares for the residual

***

p<.001

Table 3.

Multi-level regression of coalition capacities on four dimensions of coalition functioning

Standardized regression coefficients
 Variance components
β SE Two-tailed p r2 Between coalition members Between coalitions ICC
Member capacities: new skills N=216 N=12
  Unconditional model – – – .502*** <.001 .000
  Goal directedness .363 .062 .000 .13** .436*** <.001
  Efficiency .392 .063 .000 .15** .425*** <.001
  Opportunities for participation .334 .079 .000 .11** .446*** <.001
  Cohesion .388 .067 .000 .15** .426*** <.001
Organizational capacities: linkages N=198 N=12
  Unconditional model – – – .165*** .026** .136
  Goal directedness .540 .048 .000 .29*** .118*** .031*
  Efficiency .577 .054 .000 .33*** .113*** .027
  Opportunities for participation .502 .054 .000 .25*** .124*** .030*
  Cohesion .535 .054 .000 .29*** .122*** .030*
Open in a new tab

Each coalition functioning dimension was entered separately in the regression ICC intraclass correlation =.06;

*

p<.05;

**

p<.01;

***

p<.001

This study finds that coalition members’ acquisition of new skills varied among coalition members, reliably explained by coalition efficiency and cohesion (r2=.15), goal directedness (r2=.13), and opportunities for participation (r2=.11). It also finds variance between coalitions in the coalition’s linkages with diverse sectors (ICC=.14). Variance was significantly explained by coalition functioning. Efficiency explained the most variance in linkages (r2=.33), followed by directedness and cohesion (r2=.29) and opportunities for participation (r2=.25).

Discussion

This study explored the relationship between coalition functioning and the community-wide adoption of a science-based approach to prevention. CTC coalitions differed in all four dimensions of functioning, as perceived by coalition members, 1 year after CTC implementation began. Coalitions differed the most in the dimension of cohesion. Although few studies of coalitions report intraclass correlations (ICCs), the ICC’s observed here are consistent with the range of .09–.19 reported in other studies (Allen 2005). Coalitions differed the least in the functioning dimension of participation. This result is similar to findings from previous studies that indicated small between-coalition variance in shared power and empowerment (Allen 2005; Allen et al. 2012). Interpreting the size of the ICCs in this study as large or small is difficult because so little is known about the typical size of ICCs in community-intervention research (Hedges and Hedberg 2007). However, knowing the typical size of community-level ICCs is important for making study design decisions, such as how many coalitions are required to detect coalition-level effects. This study provides information about the nature of ICCs that may be expected for coalition constructs (Brown et al. 2008; Murray and Blistein 2003). Since ICCs are generally higher for system constructs than for individual outcome variables, accounting for the ICCs of system variables is important when designing intervention research studies (Campbell et al. 2005; Taljaarda et al. 2008).

The dimensions of coalition functioning in this study did not directly explain the variance among CTC communities in the community-wide adoption of science-based prevention, relative to their matched control communities. A positive relationship was found, however, between each dimension of coalition functioning and the coalition capacities previously found to influence the adoption of science-based prevention: newly acquired coalition member skills and the coalition’s linkages with diverse sectors.We found a stronger relationship between the two characteristics of the coalition (functioning and linkages) than between a characteristic of the coalition (functioning) and a characteristic of the individual members (skill acquisition).

These findings collectively support Florin’s hypothesis that higher coalition functioning may increase the coalition capacities that lead to greater coalition achievements (in this case, community leader reports of science-based prevention). These results lend support to the importance of promoting the goal directedness, efficiency, participatory orientation, and cohesion of coalitions, but may also suggest that such efforts are only likely to lead to coalition achievements when they are leveraged to build member skills and external linkages to diverse community sectors. Furthermore, capacities for the adoption of science-based prevention may be more likely to exist where internal coalition functioning is high. This may imply a progression of coalition development that needs to be better understood in order to provide timely training and technical assistance when building local infrastructures for science-based prevention.

These results should be interpreted with caution. Although the recruited communities represent an unusual range of diversity in that they span seven states, replications should be conducted before generalizing the results to non-CTC coalitions, larger communities, or coalitions outside the USA (Steketee et al. 2013). Also, because race, gender, and class information was not collected from coalition members, we were unable to control for the ways in which these demographic characteristics may relate to perceptions of coalition functioning. Members affiliating with different identity groups may perceive or report different levels of cohesion or opportunities for participation. Furthermore, the coalition capacities measured in this study are perceptions of coalition members. Although coalition members are likely in the best position to make this assessment, there was no opportunity to scrutinize these ratings for validity. Finally, the relationship between coalition functioning and coalition capacities may be inflated because they were concurrently rated by the same individuals. Likewise, our analysis had substantially more power to detect relations between coalition functioning and coalition capacities (N≥198 coalition members) as compared to coalition functioning and community-wide adoption of science-based prevention (N=12 coalitions). However, the trivial effect size found (r2=0) for the relation between coalition functioning and community-wide adoption of science-based prevention suggests that the nonsignificant result was not due to limited power.

The findings of this study suggest areas for further investigation. (1) If coalition functioning sets the stage for capacity building and the subsequent adoption of science-based prevention, what contributes or serves as barriers to high coalition functioning (Brown et al. 2012)? (2) Exploring longitudinal data to determine the developmental progression of coalition processes, activities, and outcomes over the length of the CYDS would be informative for timing training and technical assistance supports to facilitate and sustain community change (Gloppen et al. 2012; Gomez et al. 2005).

Finally, these findings suggest that a formal test of mediation between coalition functioning, coalition capacities, and the adoption of science-based prevention may be useful; however, there are at least two challenges to conducting and interpreting a formal test of mediation for our purposes. A formal test of mediation traditionally requires that a significant direct effect be found between functioning and adoption before justifying the testing of an indirect effect (Baron and Kenny 1986). This traditional requirement restricts mediation analyses in cases where the outcome variable is quite distal relative to the predictor, such as in the study of coalition work and community change (Shrout and Bolger 2002). Conceivably, the relationship between coalition functioning and adoption is smaller and more difficult to detect than are the relationships between functioning and capacities or capacities and adoption. There is greater opportunity for random factors to influence relationships between constructs separated by long periods of time. Even when this traditional requirement is loosened, mediation testing has not been possible in a multilevel framework where outcomes are specified at the community level. New technologies, such as multilevel structural equation modeling (MSEM), may help overcome this limitation in future applications (Preacher et al. 2010). MSEM would not be appropriate in the present study, however, as the potential predictor and mediating variables were reports of coalition members in the CTC communities, whereas the outcome variable consisted of reports of community leaders in both intervention and control communities. Thus, we used the best tools available for answering the series of research questions presented in this paper.

In conclusion, this study suggests that coalition functioning may indirectly affect the degree of adoption of science-based prevention in communities through the effects of a well-functioning coalition on the development of new skills of coalition members and the building of greater engagement with the work of the coalition from diverse community sectors. Well-functioning coalitions can become an infrastructure for prevention service delivery that builds and utilizes new capacities to transform community systems (Spoth et al. 2013). In order to improve the results of community coalitions intending to prevent mental, emotional, and behavioral problems in young people, local infrastructures likely need to function well, build skills and linkages, and ultimately transform communities to adopt science-based prevention.

Acknowledgments

This work was supported by the National Institute on Drug Abuse (R01 DA015183-01A1) with co-funding from the National Cancer Institute, National Institute of Child Health and Human Development, National Institute of Mental Health (T32 MH20012), National Institute on Alcohol Abuse and Alcoholism, the Center for Substance Abuse Prevention, and the Society of Social Work & Research. The content of this paper does not represent the official views of these agencies. The authors acknowledge the CYDS communities and the thoughtful review of Peter J. Pecora.

Footnotes

Conflict of Interest The authors declare that they have no conflict of interest.

References

  1. Allen NE. A multi-level analysis of community coordinating councils. American Journal of Community Psychology. 2005;35:49–63. doi: 10.1007/s10464-005-1889-5. [DOI] [PubMed] [Google Scholar]
  2. Allen NE, Javdani S, Lehrner AL, Walden AL. ‘Changing the text’: Modeling council capacity to produce institutionalized change. American Journal of Community Psychology. 2012;49:317–331. doi: 10.1007/s10464-011-9460-z. [DOI] [PubMed] [Google Scholar]
  3. Arthur MW, Glaser RR, Hawkins JD. Steps towards community-level resilience: Community adoption of science-based prevention programming. In: Peters RD, Leadbeater B, McMahon RJ, editors. Resilience in children, families, and communities: Linking context to practice and policy. New York: Kluwer Academic/Plenum; 2005. pp. 177–194. [Google Scholar]
  4. Arthur MW, Briney JS, Hawkins JD, Abbott RD, Brooke-Weiss BL, Catalano RF. Measuring risk and protection in communities using the Communities That Care Youth Survey. Evaluation and Program Planning. 2007;30:197–211. doi: 10.1016/j.evalprogplan.2007.01.009. [DOI] [PubMed] [Google Scholar]
  5. Arthur MW, Hawkins JD, Brown EC, Briney JS, Oesterle S, Abbott RD. Implementation of the Communities That Care prevention system by coalitions in the Community Youth Development Study. Journal of Community Psychology. 2010;38:245–258. doi: 10.1002/jcop.20362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51:1173–1182. doi: 10.1037//0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  7. Berkey CS, Hoaglin DC, Mosteller F, Colditz GA. A random-effects regression model for meta-analysis. Statistics in Medicine. 1995;14:395–411. doi: 10.1002/sim.4780140406. [DOI] [PubMed] [Google Scholar]
  8. Brown CH, Wang W, Kellam SG, Muthén BO, Petras H, Toyinbo P, Windham A. Methods for testing theory and evaluating impact in randomized field trials: Intent-to-treat analyses for integrating the perspectives of person, place, and time. Drug and Alcohol Dependence. 2008;95(Suppl. 1):S74–S104. doi: 10.1016/j.drugalcdep.2007.11.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD. Effects of Communities That Care on prevention services systems: Outcomes from the Community Youth Development Study at 1.5 years. Prevention Science. 2007;8:180–191. doi: 10.1007/s11121-007-0068-3. [DOI] [PubMed] [Google Scholar]
  10. Brown EC, Hawkins JD, Rhew IC, Shapiro VB, Abbott RD, Oesterle S, Catalano RF. Prevention system mediation of Communities That Care effects on youth outcomes. Prevention Science. 2014a;15:623–632. doi: 10.1007/s11121-013-0413-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Brown LD, Feinberg ME, Greenberg MT. Determinants of community coalition ability to support evidence-based programs. Prevention Science. 2010;11:287–297. doi: 10.1007/s11121-010-0173-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Brown LD, Feinberg ME, Greenberg MT. Measuring coalition functioning: Refining constructs through factor analysis. Health Education & Behavior. 2012;39:486–497. doi: 10.1177/1090198111419655. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Brown LD, Feinberg ME, Shapiro VB, Greenberg MT. Reciprocal relations between coalition functioning and provision of implementation support. Prevention Science. 2014b;16(1):101–109. doi: 10.1007/s11121-013-0447-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Campbell MK, Fayers PM, Grimshaw JM. Determinants of the intracluster correlation coefficient in cluster randomized trials: The case of implementation research. Clinical Trials. 2005;2:99–107. doi: 10.1191/1740774505cn071oa. [DOI] [PubMed] [Google Scholar]
  15. Coulton C. The place of community in social work practice research: Conceptual and methodological developments. Social Work Research. 2005;29:73–86. [Google Scholar]
  16. Crowley KM, Yu P, Kaftarian SJ. Prevention actions and activities make a difference: A structural equation model of coalition building. Evaluation and Program Planning. 2000;23:381–388. [Google Scholar]
  17. Fagan AA, Hanson K, Hawkins JD, Arthur MW. Translational research in action: Implementation of the Communities That Care prevention system in 12 communities. Journal of Community Psychology. 2009;37:809–829. doi: 10.1002/jcop.20332. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Feinberg ME, Greenberg MT, Osgood D. Readiness, functioning, and perceived effectiveness in community prevention coalitions: A study of Communities That Care. American Journal of Community Psychology. 2004;33:163–176. doi: 10.1023/b:ajcp.0000027003.75394.2b. [DOI] [PubMed] [Google Scholar]
  19. Florin P, Mitchell R, Stevenson J. Identifying training and technical assistance needs in community coalitions: A developmental approach. Health Education Research. 1993;8:417–432. doi: 10.1093/her/8.3.417. [DOI] [PubMed] [Google Scholar]
  20. Florin P, Mitchell R, Stevenson J, Klein I. Predicting intermediate outcomes for prevention coalitions: A developmental perspective. Evaluation and Program Planning. 2000;23:341–346. [Google Scholar]
  21. Foster-Fishman PG, Berkowitz SL, Lounsbury DW, Jacobson S, Allen NA. Building collaborative capacity in community coalitions: A review and integrative framework. American Journal of Community Psychology. 2001;29:241–261. doi: 10.1023/A:1010378613583. [DOI] [PubMed] [Google Scholar]
  22. Gloppen KM, Arthur MW, Hawkins JD, Shapiro VB. Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health. 2012;51:259–264. doi: 10.1016/j.jadohealth.2011.12.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gomez BJ, Greenberg MT, Feinberg ME. Sustainability of community coalitions: An evaluation of Communities That Care. Prevention Science. 2005;6:199–202. doi: 10.1007/s11121-005-0003-4. [DOI] [PubMed] [Google Scholar]
  24. Haggerty KP, Shapiro VB. Science-based prevention through Communities That Care: A model of social work practice for public health. Social Work in Public Health. 2013;28:349–365. doi: 10.1080/19371918.2013.774812. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hawkins JD, Catalano RF & Associates. Communities That Care: Action for drug abuse prevention. 1st ed. San Francisco: Jossey-Bass; 1992. [Google Scholar]
  26. Hawkins JD, Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addictive Behaviors. 2002;27:951–976. doi: 10.1016/s0306-4603(02)00298-8. [DOI] [PubMed] [Google Scholar]
  27. Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM. Testing Communities That Care: The rationale, design and behavioral baseline equivalence of the Community Youth Development Study. Prevention Science. 2008;9:178–190. doi: 10.1007/s11121-008-0092-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hays CE, Hays SP, DeVille JO, Mulhall PF. Capacity for effectiveness: The relationship between coalition structure and community impact. Evaluation and Program Planning. 2000;23:373–379. [Google Scholar]
  29. Hedges LV. Distribution theory for Glass’s estimator of effect size and related estimators. Journal of Educational Statistics. 1981;6:107–128. [Google Scholar]
  30. Hedges LV, Hedberg EC. Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation and Policy Analysis. 2007;29:60–87. [Google Scholar]
  31. Hedges LV, Olkin I. Statistical methods for meta-analysis. Orlando: Academic Press; 1985. [Google Scholar]
  32. Himmelman AT, Johnson D, Kaye G, Salzman P, Wolff T. Roundtable discussion and final comments. American Journal of Community Psychology. 2001;29:205–2011. [Google Scholar]
  33. Kegler MC, Swan DW. Advancing coalition theory: The effect of coalition factors on community capacity mediated by member engagement. Health Education Research. 2012;27:572–584. doi: 10.1093/her/cyr083. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kegler MC, Steckler A, McLeroy K, Malek SH. Factors that contribute to effective community health promotion coalitions: A study of 10 Project ASSIST coalitions in North Carolina. Health Education and Behavior. 1998;25:338–353. doi: 10.1177/109019819802500308. [DOI] [PubMed] [Google Scholar]
  35. Kuklinski MR, Briney JS, Hawkins JD, Catalano RF. Cost-benefit analysis of Communities That Care outcomes at eighth grade. Prevention Science. 2012;13:150–161. doi: 10.1007/s11121-011-0259-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Lipsey MW, Wilson DB. Practical meta-analysis. Thousand Oaks: Sage; 2001. [Google Scholar]
  37. Mizrahi T, Rosenthal BB. Complexities of coalition building: Leaders’ successes, strategies, struggles, and solutions. Social Work. 2001;46:63–77. doi: 10.1093/sw/46.1.63. [DOI] [PubMed] [Google Scholar]
  38. Murray DM, Blistein JL. Methods to reduce the impact of intraclass correlation in group-randomized trials. Evaluation Review. 2003;27:79–103. doi: 10.1177/0193841X02239019. [DOI] [PubMed] [Google Scholar]
  39. Muthén LK, Muthén BO. Mplus 6.0. Los Angeles: Muthén & Muthén; 2010. [Google Scholar]
  40. O’Connell ME, Boat T, Warner KE, editors. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academies Press; 2009. [PubMed] [Google Scholar]
  41. Preacher KJ, Zyphur MJ, Zhang Z. A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods. 2010;15:209–233. doi: 10.1037/a0020141. [DOI] [PubMed] [Google Scholar]
  42. Quinby RK, Fagan AA, Hanson K, Brooke-Weiss B, Arthur MW, Hawkins JD. Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology. 2008;36:313–332. [Google Scholar]
  43. Raudenbush SW. Random effects models. In: Cooper H, Hedges LV, editors. The handbook of research synthesis. New York: Russell Sage Foundation; 1994. pp. 301–321. [Google Scholar]
  44. Roberts-DeGennaro M, Mizrahi T. Coalitions as social change agents. In: Weil M, editor. The handbook of community practice. Thousand Oaks, CA: Sage; 2005. pp. 305–318. [Google Scholar]
  45. Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health. 2000;21:369–402. doi: 10.1146/annurev.publhealth.21.1.369. [DOI] [PubMed] [Google Scholar]
  46. Shapiro VB, Hawkins JD, Oesterle S, Monahan KC, Brown EC, Arthur MW. Variation in the effect of Communities That Care on community adoption of a scientific approach to prevention. Journal of the Society for Social Work and Research. 2013a;4:154–164. doi: 10.5243/jsswr.2013.10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Shapiro VB, Oesterle S, Abbott RD, Arthur MW, Hawkins JD. Measuring dimensions of coalition functioning for effective and participatory community practice. Social Work Research. 2013b;37:349–359. doi: 10.1093/swr/svt028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Shapiro VB, Oesterle S, Hawkins JD. Relating coalition capacity to the adoption of science-based prevention in communities: Evidence from a randomized trial of Communities That Care. American Journal of Community Psychology. 2015;55:1–12. doi: 10.1007/s10464-014-9684-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Shortell S, Zukoski A, Alexander J, Bazzoli G, Conrad D, Hasnain-Wynia R, Margolin F. Evaluating partnerships for community health improvement: Tracking the footprints. Journal of Health Politics, Policy and Law. 2002;27:49–91. doi: 10.1215/03616878-27-1-49. [DOI] [PubMed] [Google Scholar]
  50. Shrout PE, Bolger N. Mediation in experimental and non-experimental studies: New procedures and recommendations. Psychological Methods. 2002;7:422–445. [PubMed] [Google Scholar]
  51. Society for Prevention Research. A call for bold action to support prevention programs and policies to achieve greater public health and economic impact. Position Statement of the Society for Prevention Research Task Force on Type 2 Translational Research. 2009 Retrieved from http://www.preventionresearch.org/SPR_T2_Task_Force_Position_Statement.pdf.
  52. Spoth R, Greenberg M, Bierman K, Redmond C. PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science. 2004;5:31–39. doi: 10.1023/b:prev.0000013979.52796.8b. [DOI] [PubMed] [Google Scholar]
  53. Spoth R, Redmond C, Clair S, Shin C, Greenberg M, Feinberg M. Preventing substance misuse through community-university partnerships: Randomized controlled trial outcomes 4(1/2) years past baseline. American Journal of Preventive Medicine. 2011;40:440–447. doi: 10.1016/j.amepre.2010.12.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Spoth R, Rohrbach LA, Greenberg M, Leaf P, Brown CH, Fagan A, Hawkins JD Society for Prevention Research Type 2 Translational Task Force Members and Contributing Authors. Addressing core challenges for the next generation of type 2 translation research and systems: The Translation Science to Population Impact (TSci Impact) Framework. Prevention Science. 2013;74:319–351. doi: 10.1007/s11121-012-0362-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Steketee M, Oesterle S, Jonkman H, Hawkins JD, Haggerty KP, Aussems C. Transforming prevention systems in the United States and the Netherlands using Communities That Care. European Journal on Criminal Policy and Research. 2013;19:99–116. doi: 10.1007/s10610-012-9194-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Taljaarda M, Donnerb A, Villarc J, Wojdylad D, Velazcof A, Batagliag V, Acosta A. Intracluster correlation coefficients from the 2005 WHO Global Survey on Maternal and Perinatal Health: Implications for implementation research. Paediatric and Perinatal Epidemiology. 2008;22:117–125. doi: 10.1111/j.1365-3016.2007.00901.x. [DOI] [PubMed] [Google Scholar]
  57. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Saul J. Bridging the gap between prevention research and practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology. 2008;41:171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]
  58. Wells R, Ward AJ, Feinberg M, Alexander JA. What motivates people to participate more in community-based coalitions? American Journal of Community Psychology. 2008;42:94–104. doi: 10.1007/s10464-008-9182-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Wilson DB. Meta-analysis macros for SAS, SPSS, and Strata. 2005 Retrieved from http://mason.gmu.edu/~dwilsonb/ma.html.
  60. Zakocs RC, Edwards EM. What explains community coalition effectiveness? A review of the literature. American Journal of Preventive Medicine. 2006;30:351–361. doi: 10.1016/j.amepre.2005.12.004. [DOI] [PubMed] [Google Scholar]
  61. Zakocs RC, Guckenburg S. What coalition factors foster community capacity? Lessons learned from the Fighting Back Initiative. Health Education and Behavior. 2007;34:354–375. doi: 10.1177/1090198106288492. [DOI] [PubMed] [Google Scholar]

RESOURCES