Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 Oct 1.
Published in final edited form as: Prev Sci. 2014 Oct;15(5):623–632. doi: 10.1007/s11121-013-0413-7

Prevention System Mediation of Communities That Care Effects on Youth Outcomes

Eric C Brown 1,✉, J David Hawkins 2, Isaac C Rhew 3, Valerie B Shapiro 4, Robert D Abbott 5, Sabrina Oesterle 6, Michael W Arthur 7, John S Briney 8, Richard F Catalano 9
PMCID: PMC3884024  NIHMSID: NIHMS502411  PMID: 23828448

Abstract

This study examined whether the significant intervention effects of the Communities That Care (CTC) prevention system on youth problem behaviors observed in a panel of eighth-grade students (Hawkins et al. Archives of Pediatrics and Adolescent Medicine 163:789–798 2009) were mediated by community-level prevention system constructs posited in the CTC theory of change. Potential prevention system constructs included the community’s degree of (a) adoption of a science-based approach to prevention, (b) collaboration on prevention activities, (c) support for prevention, and (d) norms against adolescent drug use as reported by key community leaders in 24 communities. Higher levels of community adoption of a science-based approach to prevention and support for prevention in 2004 predicted significantly lower levels of youth problem behaviors in 2007, and higher levels of community norms against adolescent drug use predicted lower levels of youth drug use in 2007. Effects of the CTC intervention on youth problem behaviors by the end of eighth grade were mediated fully by community adoption of a science-based approach to prevention. No other significant mediated effects were found. Results support CTC’s theory of change that encourages communities to adopt a science- based approach to prevention as a primary mechanism for improving youth outcomes.

Keywords: Community prevention system, Communities That Care, Adolescent drug use, Delinquency, Multilevel mediation, Underage drinking, Alcohol use, Tobacco use

Introduction

In recent years, researchers and practitioners seeking to promote adolescent well-being and prevent behavioral health problems have recognized that establishing community-wide change in the environments that affect adolescent behavioral health requires the participation of all stakeholders who influence the lives of young people. To this end, community coalitions have been advocated as a strategy to prevent health and behavior problems in young people (Chinman et al. 2005; Fagan et al. 2011; Feinberg, Greenberg, and Osgood 2004; Foster-Fishman and Behrens 2007; Spoth et al. 2004). Such place-based coalitions are composed of people living or working within a community, representing diverse interests, who collectively “advocate for reform in structural arrangements for delivering and accessing healthcare, education, social welfare, and other human services” (Roberts-DeGennaro and Mizrahi 2005, p. 305).

However, studies of the effectiveness of coalitions for impacting youth outcomes have produced mixed results. Many evaluations of coalition-based efforts to prevent adolescent health and behavior problems have failed to show desired effects (Hallfors et al. 2002; Kreuter et al. 2000; Merzel and D'Afflitti 2003). Two coalition-based strategies, PROSPER and Communities That Care (CTC), have shown positive effects in preventing adolescent substance use and delinquent behavior community wide (Feinberg et al. 2007; Feinberg et al. 2010; Hawkins et al. 2009; Hawkins et al. 2012; Spoth et al. 2004; 2013). These strategies consider the contextual influences and risk/protective factors that predict problem behaviors; rely on local ownership, focused goals, choice of evidence-based prevention programs based on local priorities, and high-quality program implementation; and incorporate ongoing evaluation and technical assistance to achieve and sustain intended youth outcomes (Spoth and Greenberg 2005).

This study investigated hypothesized mediators of the observed effects of a community-based prevention system, Communities That Care, on youth outcomes. CTC (Hawkins and Catalano 2002; Hawkins et al. 2002) is a manualized prevention system that relies largely on the work of a community prevention coalition to address adolescent health and behavior problems. By community-based prevention system, we refer to characteristics of a community that influence the community’s allocation of resources, both human and financial, toward the goal of preventing behavioral health problems of young people. After key community leaders are oriented to the CTC model, they appoint a CTC prevention coalition composed of diverse stakeholder groups (e.g., human services, law enforcement, juvenile justice, education, business, youth recreation, media, and religious). The coalition receives training and technical assistance to identify elevated risk factors and depressed protective factors experienced by the community’s youth population, select and implement prevention programs and policies that have been tested and found to be effective in a rigorous evaluation study that targets specific elevated risks, and monitor the implementation quality and the outcomes of their efforts. Implementation of evidence-based prevention programs and policies, through the work of the coalition, is hypothesized to lead to reductions in targeted community risks, improvements in community protection, and, ultimately, reductions in adolescent problem behaviors.

CTC targets five community-level constructs of a community’s prevention system in order to bring about population-based reductions in adolescent behavioral health problems (E. C. Brown et al. 2007). The first construct CTC seeks to change is adoption of a science-based approach to prevention (Arthur et al. 2005), which incorporates elements of public health and community mobilization models, and is theorized in CTC to be the primary mechanism leading to the use of tested and effective interventions and their implementation at scale with fidelity. The second construct is collaboration on prevention initiatives, which consists of inter-sector connections, information exchange, coordination, and resource sharing among community constituents involved in community prevention initiatives (Emshoff et al. 2007; Foster-Fishman et al. 2001; Roussos and Fawcett 2000). The third construct is community support for prevention as indicated by positive attitudes and beliefs about prevention and willingness to allocate resources and funding for prevention with regard to competing community needs (Beckhard and Harris 1987; Fawcett et al. 1993). The fourth construct, community norms against adolescent drug use, comes from social normative theory (Ajzen and Fishbein 1980) and refers to the general beliefs held by members of a community regarding the acceptability of drug use by adolescents. A fifth construct that CTC seeks to change is utilization of the social development strategy, which refers to the provision of opportunities for (a) prosocial engagement; (b) social, emotional, and cognitive skills; and (c) positive recognition and reinforcement as methods to strengthen bonds of attachment and commitment to prosocial others and the larger community. Changes in these five prevention system characteristics are theorized to be the mediators by which CTC achieves improvements in adolescent behavioral health.

Previous studies have shown significant effects of CTC on four of these constructs as measured by self-reports of key community leaders at 1.5, 4.5, and 6.5 years post CTC implementation (E. C. Brown et al. 2007; E. C. Brown et al. 2011; Rhew et al. 2013). Compared with control communities, communities implementing CTC exhibited significantly greater levels of adoption of a science-based approach to prevention at all three posttest time points, and greater pre-to-post increases in levels of collaboration around prevention activities by 1.5 years after implementation. Additionally, CTC communities demonstrated higher levels of desired funding for prevention activities at 6.5 years post CTC implementation, and greater growth in community norms against adolescent drug use during the course of the study, than did control communities. CTC also demonstrated impact on youth outcomes. Analysis of a panel of 4,407 youth followed from fifth grade in a community-randomized trial of 24 communities found significant effects of CTC by the end of Grade 8 on the incidence of alcohol, cigarette, and smokeless tobacco use and delinquent behavior; and on the prevalence of current alcohol use, binge drinking, and different delinquent acts in the past year (Hawkins et al. 2009).

In light of the observed intervention effects on the proximal community-level prevention system constructs and distal adolescent behavior outcomes, this study addressed two questions. First, were measures of (a) adoption of a science-based approach to prevention, (b) collaboration on prevention activities, (c) support and desired funding for prevention, and (d) norms against adolescent drug use predictive of subsequent adolescent drug use and delinquency? Second, did these prevention system constructs mediate the previously reported effects of the CTC intervention on youth outcomes in Grade 8? 1 To answer these questions, we used the longitudinal nested design of the Community Youth Development Study (CYDS; E. C. Brown et al. 2009; Hawkins et al. 2008a). Among those communities assigned randomly to receive CTC, implementation of the system began in 2003 and was implemented fully within an average of 11 months (Quinby et al. 2008). Ensuing changes in communities’ prevention systems were theorized to occur from 1 to 2 years post CTC implementation, with impact on youth outcomes realized in 4 or more years post CTC implementation.

As shown in Fig. 1, we assessed the mediated effects of CTC impact on youth outcomes using data on prevention system constructs obtained from key community leaders in fall 2004, approximately 1.5 years after CTC implementation began in intervention communities, and data on youth outcomes from the panel of students when they were in eighth grade in the spring of 2007, approximately 3 years after the prevention programs chosen by CTC communities were first implemented and 4 years after CTC began in intervention communities. We focused the present analyses on potential system-level mediators and student-level outcomes that were impacted significantly by the CTC intervention (i.e., when panel students were in the eighth grade).

Fig. 1.

Fig. 1

Open in a new tab

Mediation model of prevention system change using Communities That Care

Methods

Setting

Twenty-four communities across seven states (i.e., Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington) agreed to participate in the CYDS. Communities were selected from a larger pool of communities that participated in a naturalistic study of the diffusion of science-based prevention strategies (Arthur et al. 2005). CYDS communities were small- and medium-sized geographically distinct, incorporated towns with an average population of 14,646 according to the 2000 US Census (range=1,578 to 40,787). On average, 89 % of the population members were White/Caucasian (range=64 to 98 %), 3 % were African American (range=0 to 21 %), 10 % were of Hispanic origin (range=1 to 65 %), 12 % were between the ages of 10 and 17 years (range=9 to 16 %), and 38 % of students were from low-income families as indicated by eligibility for free or reduced-price school lunch (range=18 to 66 %).

Communities were matched into pairs within state according to community-level demographic indicators, and one community from each of the 12 matched pairs of communities was assigned randomly to receive CTC or remain as a “prevention-as-usual” control community. Schools participating in control communities were offered summary data on adolescent risk and protective factors, alcohol and drug use, and delinquent behaviors from the CTC Youth Survey for review but did not receive any information as part of the study in regard to a science-based approach to prevention or any other elements of the CTC intervention. See E. C. Brown et al. (2009) and Hawkins et al. (2008b) for more information on the design of the CYDS.

Participants

Key Community Leaders

Data on community prevention systems were obtained from 340 key community leaders in all 24 communities of the CYDS. Identification of key community leaders followed a two-stage approach in which up to 10 positional leaders who held positions of leadership within various sectors of the community (e.g., mayors, city managers, police chiefs, school superintendents, business leaders, and heads of social service agencies) were targeted to be interviewed. A sample of referred leaders was generated by asking each positional leader to identify two individuals in the community who they thought were the most knowledgeable about current prevention efforts in the community. Five referred leaders nominated frequently by the positional leaders in each community also were included in the target sample. The sample consisted of between 12 and 15 respondents per community (targeted positions did not exist in all communities). Participants averaged 49.0 years of age (SD=10.1) at time of the interview, 63 % of the sample were positional respondents, and 58 % were male. Participants had lived an average of 16.8 years in their respective communities (SD=17.3), 75 % of participants had lived in their community for more than 1 year, and 45 % of the sample had a bachelor’s degree or higher. The sample was balanced between CTC (50.5 %) and control (49.5 %) communities. Participants did not differ significantly between CTC and control groups on any demographic characteristic.

Students

Data on adolescent drug use and delinquent behavior were obtained from 4,181 students from the CYDS longitudinal panel of 4,407 consenting fifth-grade public school students in all 24 participating communities. Those students progressing normally in school were in the eighth grade during the 2006– 2007 school year. Using an intent-to-treat design, all youth in the panel who remained in participating communities for at least one semester beyond the baseline survey in fifth grade were tracked and surveyed annually regardless of their current grade in school or whether they still lived in the participating communities or attended public schools in those communities. Ninety-six percent of students in the longitudinal panel completed the survey in 2007. Details of student recruitment are available in Brown et al. (2009). Student participants were split evenly by gender; 54 % of students were from CTC communities; 68 % of students were European American, 9 % were Native American, 4 % were African American, and 19 % were of another racial/ethnic group. Twenty percent of students were of Hispanic origin. In 2007, students were an average of 14.1 years of age (SD=0.4). Rates of survey participation, demographic characteristics, and baseline levels of risk/protective factors and problem behaviors for youth in the panel sample were equivalent statistically across experimental and control conditions.

Measures

Prevention System Constructs

Measures of key community leaders’ perceptions of community prevention system constructs were obtained from the 2004 administration of the Community Key Informant Survey (Arthur et al. 2002), approximately 1.5 years after CTC implementation began in intervention communities. The survey included questions regarding characteristics of the community and its approach to prevention of adolescent problem behaviors. Trained interviewers administered the survey by telephone to key community leaders in both CTC and control communities using a computer-assisted interview, lasting, on average, about 1 h.

Five measures from the survey were analyzed (see E. C. Brown et al. 2007; 2011; and Rhew et al. 2013 for more information on the measurement of these constructs). First, adoption of a science-based approach to prevention was measured by responses to 22 questions drawn from content domains representing awareness and use of prevention science concepts, use of epidemiologic data, use of tested and effective prevention programming, and system monitoring (Arthur et al. 2005). Responses to these items were combined to indicate six discrete stages of adoption representing an ordinal progression from lower to higher levels of science-based prevention (stages 0 to 5, respectively). Items assessed respondents’ lack of awareness of prevention science terminology or concepts (stage 0), awareness of prevention science terminology and concepts such as risk and protective factors, but no use of these concepts to guide prevention programming (stage 1), use of a risk- and protection-focused prevention approach as a planning strategy, but no collection of epidemiologic data to guide the selection of prevention activities in the community or the use of tested-effective preventive interventions (stage 2), collection of epidemiologic risk and protective factor data but no use of tested-effective preventive interventions (stage 3), selection and use of tested-effective interventions to address prioritized risk and protective factors based on epidemiologic data collected in the community (stage 4), and collection and feedback of program processes and outcome data and adjustment of interventions based on the data (stage 5). Mean levels of this construct were 2.93 and 1.65 (SDs=.53 and .93) for CTC and control communities, respectively.

Second, community collaboration for prevention was constructed as a second-order factor consisting of sectorial collaboration, which measured the degree to which key community leaders reported collaborating across seven sectors of the community (i.e., civic, business, schools, law enforcement, community coalitions, human services, and religious; Cronbach’s alpha=.77), and prevention collaboration, which consisted of nine items that measured the extent to which key community leaders engaged in prevention-specific collaborative activities (e.g., sharing resources, coordinating activities, establishing networks; Cronbach’s alpha=.92). Mean levels of this construct on the factor score metric were 0.31 and −.025 (SDs=.11 and .12) for CTC and control communities, respectively; higher scores for collaboration indicated more collaboration in a community.

Third, community support for prevention was operationalized as a second-order factor identified by two first-order latent variables: community member support and community leader support, and an additional indicator representing key community leaders’ desired level of funding (in percentage terms) relative to desired funding for mental health treatment services and law enforcement. Four community member support items measured key community leaders’ perceptions of community members’ beliefs in prevention effectiveness, knowledge of prevention efforts, and willingness to pay for prevention programs. Three community leader support items assessed these same topics, but in reference to the key community leaders themselves, rather than their assessments of community members. Cronbach’s alpha for the seven items was .77. Mean levels of this construct on the factor score metric were 0.07 and −0.03 (SDs=.16 and .30) for CTC and control communities, respectively; higher scores indicated greater support in the community for prevention in that community.

Fourth, key community leaders were asked the question “If you were deciding how to spend money for reducing substance abuse, what percentage would you allocate to each of the following approaches? Law Enforcement, Treatment, and Prevention.” The percentage that key community leaders would allocate to prevention was used as an additional (separate) measure of community support for prevention.

Fifth, a measure of community norms against adolescent drug use was created using six items that assessed key community leaders’ perceptions of normative beliefs in the community regarding adolescent drug use, i.e., In this community, how wrong do most adults think it is for adolescents to (a) drink alcohol, (b) smoke cigarettes, and (c) use marijuana?, Adults in this community think that using (a) alcohol, (b) tobacco, and (c) marijuana, are a normal part of growing up (Van Horn et al. 2007). Cronbach’s alpha for the six items was .88. Mean levels of this construct were Ms=0.20 and 0.14 (SDs=.43 and .60) for CTC and control communities, respectively. Higher scores indicated stronger norms in the community against adolescent drug use.

Except for adoption, scale values represented factor scores derived from a confirmatory factor analysis of items for each construct. Key community leaders’ individual scores for each construct were averaged across all leaders in a community, with the resulting mean value representing the community’s aggregate level of the construct. Community-level averages for all constructs were approximately Gaussian distributed, with skewness and kurtosis values between −1.0 and 1.0.

Student Outcomes

Measures of student drug use and delinquency were taken from the CYDS Youth Development Survey (a modification of the CTC Youth Survey; Social Development Research Group 2005) administered in the spring of 2007 to students in the panel sample at the end of Grade 8, approximately 3 years after the prevention programs were first implemented and 4 years after CTC began in intervention communities. The Youth Development Survey is a self-administered, paper-and-pencil questionnaire designed to be completed in a 50-minute classroom period. The survey includes questions on student demographic characteristics; 30-day and lifetime use of alcohol, marijuana, cigarettes, and other drugs; heavy episodic (i.e., “binge”) drinking during the previous 2 weeks; past-year delinquency; and risk and protective factors in community, school, family, and peer/individual domains.

Analyses examined if the prevention system measures reported by key community leaders in 2004 mediated effects of the CTC intervention on 2007 prevalence rates of alcohol use and smokeless tobacco use during the previous 30 days, binge drinking (consuming five or more drinks in one occasion) within the past 2 weeks, and the number of delinquent behaviors during the past year among eighth-grade students in the panel sample (Hawkins et al. 2009). Self-reported frequency of 30-day alcohol use and 30-day smokeless tobacco use outcomes were dichotomized into 0=no use and 1=use categories, as was the frequency of the 2-week binge drinking outcome. The delinquent behavior index consisted of nine items that asked about the frequency of delinquent acts committed during the past year (i.e., Beat up someone so badly that they needed to see a doctor or nurse, Sold illegal drugs, Attacked someone with the idea of seriously hurting them, Stolen something worth more than $5, Carried a handgun, Purposely damaged or destroyed property that did not belong to you, Used a weapon to get money, Stolen or tried to steal a motor vehicle, Been arrested). Each delinquent behavior item was recoded as 0=never and 1=1 or more times. A summary measure of delinquent behavior was constructed as the number of different delinquent behaviors in which the respondent reported having engaged across all nine items. We combined the three drug use items and the delinquent behavior index into a composite problem behavior outcome measure using confirmatory factor analysis (CFA) with WLSMV estimation for the categorical indicators. Fit indices for the measurement model indicated good fit to the data, χ2 (2, N=4,181)=19.77, p<.01; RMSEA=.046; CFI=.99; and TLI=.98. Standardized factor loadings ranged from .69 for the delinquent behavior indicator to .92 for the 30-day alcohol use indicator (R2s=.48 to .93). Community norms against adolescent drug use were expected to be associated specifically with lower levels of youth drug use. To examine this relationship, we also created a composite measure of drug use outcomes without delinquent behaviors, constructed using a confirmatory factor analysis of the Grade 8 30-day alcohol use, 30-day smokeless tobacco use, and 2-week binge drinking items. Standardized factor loadings for the drug-use-only outcome were .92, .73, and .98 for the three respective items (R2s=.85, .54, and .95). Factor score means for the problem behavior measure and the drug-use-only measure were −0.14 and −0.20 (SDs=2.24 and 2.77, skewness=1.91 and 1.97, kurtosis=2.46 and 2.48), respectively, for students from CTC communities; and 0.18 and 0.25 (SDs=2.57 and 3.23, skewness=1.58 and 1.63, kurtosis=1.24 and 1.44), respectively, for students from control communities. Factor scores were saved and included in subsequent multilevel models that tested for possible mediation effects of prevention system constructs.

Student Characteristics

Student characteristics included in the analyses as covariates measured at baseline were: students’ age; gender (coded 0=male, 1=female); race/ethnicity (coded 0=Nonwhite, 1=White or Caucasian); whether the student was Hispanic (coded 1=yes, 0=no); parental education level (ranging from 1=grade school or less to 6=graduate or professional degree); attendance at religious services (How often do you attend religious services or activities?; coded 0=never to 4=about once a week or more); student rebelliousness (calculated as the average of three items: I like to see how much I can get away with, I ignore rules that get in my way, and I do the opposite of what people tell me, just to get them mad; alpha=.69); dichotomous 30-day alcohol use, dichotomous 30-day smokeless tobacco use, dichotomous 2-week binge drinking; and the number of four delinquent behaviors committed during the past year (i.e., Attached someone with the ideas of seriously hurting them, Stolen something worth more than $5, Purposely damaged or destroyed property that did not belong to you, and Used a weapon to get money). Intervention condition was coded 1 for CTC communities and 0 for control communities.

Statistical Analyses

Statistical analyses consisted of upper level (i.e., 2 → 2 → 1) meditational models (Bauer et al. 2006; MacKinnon 2008). As shown in Fig. 1, the student problem behavior outcome (at level 1) was regressed sequentially on each of the four community prevention system transformation mediators (at level 2), while controlling statistically for student characteristics (at level 1). In separate analyses, each prevention system mediator and the student problem behavior outcome were regressed simultaneously on intervention condition (at level 2). Direct effects of the intervention on youth outcomes and indirect effects mediated through the prevention system measures were assessed by dividing their regression coefficients by their respective delta-method standard errors with degrees of freedom equal to the number of communities minus community-level predictors (intervention condition and prevention system construct), minus one (i.e., df=21). All analyses were conducted using Mplus v6.1 (Muthén and Muthén 2010) using the TWOLEVEL option with restricted maximum likelihood estimation. Statistically significant effects were assessed using a t distribution with a two-tailed .05 Type I error rate.

Missing Data

Among the 4,407 students comprising the consented longitudinal panel, n=226 (5.2 %) were missing Grade 8 data on all substance use and delinquent behavior items and student demographic characteristics (i.e., model covariates). The analysis sample, therefore, consisted of N=4,181 students. Missing items among these students were addressed by using full information maximum likelihood estimation in the CFA measurement models, as was available in Mplus v6.11. The 4,181 students included in the analysis did not differ from the 226 students with missing data on gender, age, and baseline levels of parental education and religious service attendance (all ps>.05). There was no evidence of differential attrition by intervention condition (CTC vs. control community). Students with missing data, however, were more likely to be non-White, χ2 (1, N=4,407)=7.24, p<.05; Hispanic, χ2 (1, N=4,407)=7.47, p<.05; and have higher baseline levels of rebelliousness, F(1, 4,407)=7.15, p<.05.

Results

Effects of Intervention Condition on Student Problem Behavior

Results controlling for student characteristics and baseline covariates indicated a significant effect of intervention condition on the composite student problem behavior, b=−0.561, SE=.171, p<.01, with students from CTC communities exhibiting lower levels of problem behavior at the end of Grade 8 than students from control communities. For the drug-use-only composite outcome, however, results indicated a nonsignificant direct effect of the intervention, b=−0.403, SE=.216, p=.06. Although this effect failed to achieve statistical significance, we proceeded to test for mediation based on the recommendation of Shrout and Bolger (2002) that a significant main effect not be a requirement for tests of mediation when effects may be subject to limited statistical power.

Effects of Prevention System Constructs on Student Problem Behavior

The direct effect of community adoption of a science-based approach to prevention on student problem behavior was statistically significant, b=−0.050, SE=.020, p<.01; R2=.39. Higher levels of community adoption of a science-based approach to prevention predicted significantly lower levels of problem behavior reported by students at the end of Grade 8. Furthermore, a significant indirect effect for intervention condition through community adoption of a science-based approach to prevention was found, indicating mediation of the path from intervention condition to student problem behavior, b=−0.056, SE=.026, p<.05, with adoption accounting for 96 % of the variation between the CTC intervention and student problem behaviors. The direct effect of intervention status predicting student problem behavior was no longer significant, b=−0.036, SE=.040, p>.05, after accounting for the mediated pathway through community adoption of a science-based approach to prevention.

Community collaboration for prevention had a nonsignificant direct effect on student problem behavior, b=−0.093, SE=.138, p>.05, and a nonsignificant indirect effect of the mediated pathway from intervention condition to student problem behavior, b=−0.004, SE=.008, p>.05.

Results indicated a significant direct effect of community support for prevention on student problem behavior, b=−0.324, SE=.153, p<.05; R2=.38, with higher levels of community support for prevention predicting lower levels of student problem behavior 3 years later. However, community support for prevention did not mediate the path from intervention status to student problem behavior, b=−0.010, SE=.014, p>.05. The direct effect of intervention status predicting student problem behavior remained significant in the mediation model, b=−0.479, SE=.193, p<.05. Results for the single-item desired funding for prevention measure did not indicate a significant effect on student problem behavior, b=−0.258, SE=.232, p>.05; nor was there a significant indirect effect of the mediated pathway from intervention condition to student problem behavior through desired funding for prevention, b=−0.012, SE=.014, p>.05.

As hypothesized, community norms against adolescent drug use had a significant direct effect on student drug use, b=−0.122, SE=.032, p<.01; R2=.37, with higher levels of community norms against adolescent drug use predicting lower levels of student drug use at the end of eighth grade. Testing the indirect effect for the path from intervention status to student drug use through community norms indicated a nonsignificant indirect effect, b=−0.028, SE=.054, p>.05. The direct effect of intervention status predicting student drug use remained nonsignificant in the mediation model, b=−0.296, SE=.176, p=.09.

Discussion

Increasing attention has been given to the role of developing and implementing service delivery systems to improve the safety and well-being of youth populations (Aarons et al. 2011; Fixsen et al. 2010). Evidence now exists that demonstrates the effectiveness of community prevention coalitions as a mechanism to achieve these goals (Fagan et al. 2011; Feinberg et al. 2007). Although more information is being generated on internal coalition functioning (Shapiro et al. in press), how aspects of collation functioning relate to program implementation (Arthur et al. 2010; L. D. Brown et al. 2010), and how coalitions are sustained over time (Gloppen et al. 2012), empirical evidence identifying broad community-level prevention system characteristics that link prevention coalitions to youth outcomes has been lacking. The present study investigated key community leaders’ perceptions of community-level prevention system constructs, which are targets of CTC coalition work, reinforced by training and technical assistance to coalitions. Using the longitudinal and multi-source data from the CYDS, we assessed associations among experimental assignment of communities to CTC or control condition in 2003, levels of prevention system measures in 2004, and youth outcomes in 2007, when students were in the eighth grade. The study took advantage of the multilevel design of the CYDS to link community key informant data with youth survey outcome data in each of the self-contained and randomly assigned communities participating in CYDS. Internal validity was enhanced by using rigorously developed measures (Arthur et al. 2002; E. C. Brown et al. 2009; E. C. Brown et al. 2007; Van Horn et al. 2007).

Results of the study indicate that adopting a science-based approach to prevention, community support for prevention, and community norms against adolescent substance use are predictive of improved youth outcomes. Foremost among these constructs is the adoption of a science-based approach to prevention, which is viewed in the CTC theory of change as the key mechanism by which communities achieve better youth outcomes (Arthur et al. 2005). Results of multilevel mediation analysis indicated that adoption of a science-based approach to prevention as reported by key community leaders predicted lower levels of youth problem behaviors and mediated fully the impact of the CTC intervention on these youth problem behaviors.

These findings have important implications for prevention science. Communities That Care, as an example of a community-driven implementation approach (O'Connell et al. 2009), is a system for promoting the use of epidemiological data to assess community prevention needs and installing prevention programs that have been tested and found to be effective for targeted problems, thereby decreasing youth problem behaviors community wide. This is accomplished by providing coalition members with training and technical assistance to develop their capacities to change community prevention systems, such that a science-based approach to prevention is adopted community wide. Study findings indicate that, with regard to promoting science-based prevention programming in communities, CTC functioned as hypothesized. Training and technical assistance, provided to members of CTC communities but not control communities, ultimately created a difference in key community leaders’ reports of their prevention systems. In other words, the coalition-based CTC strategy ultimately changed the perceptions, attitudes, and behavior of community leaders who allocate resources and shape opinions. This study finds that coalition work that aims to improve the community-wide adoption of a science-based approach to prevention is an effective strategy for improving youth behavioral outcomes. In fact, coalition work that improves the community-wide adoption of a science-based approach to prevention may be the active ingredient that determines coalition success in achieving change in the prevalence rates of youth problem behavior.

While community collaboration has been identified by some as an important mechanism in transforming communities toward improved service delivery, our measure of community collaboration in 2004 was not related to youth problem behaviors in 2007. Despite significant improvement in prevention collaboration in CTC communities from 2001 to 2004 relative to control communities (E. C. Brown et al. 2007), average levels of collaboration overall were high and did not differ significantly between CTC and control communities in 2007 and 2009 (E. C. Brown et al. 2009; Rhew et al. 2013). These findings suggest that the CYDS sample of communities may have been at high levels of collaboration relative to the general population of communities, or more generally, stakeholders in many communities across the US already may understand the importance of collaboration across diverse community sectors in seeking to prevent adolescent problem behaviors, and that a high level of collaboration is, perhaps, a necessary, but not sufficient, condition to produce changes in youth outcomes. Future measures of collaboration might better be developed to include collaboration around specific science-based aspects of community prevention (e.g., inter-agency discussions and meetings regarding the use of a risk-and protective-factor targeted approach, sharing of community resources for epidemiologic data collection) to better distinguish these efforts from non-evidence-based collaborative efforts in communities.

Both support for prevention and norms against adolescent drug use are recognized as important mechanisms in prevention system transformation (Biglan et al. 1996; Fagan et al. 2007; Holder et al. 1997; Pentz 2000; Pentz et al. 1989). In line with this literature, the present study found that higher levels of support for prevention in communities, as measured by both our latent variable and manifest indicator of the construct, predicted lower levels of youth problem behaviors 3 years later among eighth-grade students, and higher levels of community norms against drug use predicted lower levels of drug use for students in the CYDS longitudinal panel as hypothesized. In contrast to the findings for adoption of a science-based approach to prevention, neither of the community support for prevention measures nor community norms against adolescent drug use mediated the effects of the CTC intervention on youth problem behavior outcomes in this study. Although previous research (E. C. Brown et al. 2007; Rhew et al. 2013) has found that CTC affected community norms in desired ways, and although community norms predicted adolescent drug use as expected, our results did not find evidence that changing community norms and strengthening community support for prevention were mechanisms through which CTC achieved better youth outcomes. In the CYDS, the primary community-level mechanism through which the CTC intervention achieved better youth outcomes was through increasing community adoption of a science-based approach to prevention.

The study is limited by the moderate number of communities in the CYDS. With only 24 communities in the study, tests of mediation may not have had sufficient statistical power to detect small indirect effects. These communities also did not represent a random selection of communities, and community leader respondents represented a select group of individuals in their communities, which may limit the generalizability of findings from this study. Another limitation is the exclusive reliance on self-report survey data, both from key community leaders and from the panel of students. We note, however, that key informant survey data have been used widely in community research (Shinn 1990), and the validity of youth self-report drug use data has been reported elsewhere (e.g., O'Malley et al. 1983; Smith et al. 1995). Moreover, the psychometric properties of the measures used in this study have been demonstrated previously (Arthur et al. 2005; E. C. Brown et al. 2007; Van Horn et al. 2007). Causal interpretations of this study’s findings regarding the effects of system-level mediators on youth outcomes are tempered by the non-randomization of mediators (Lynch et al. 2008). Our focus in this study was on characteristics of prevention systems at the community level as reported by key community leaders. The analyses did not investigate the characteristics of CTC prevention coalitions, themselves, as potential mechanisms through which CTC achieves effects on youth outcomes.

Despite these limitations, the CYDS, as a community-randomized trial of CTC, provides an excellent opportunity to test theorized mechanisms for developing and transforming community prevention systems toward improved youth outcomes. Findings from this study support efforts to mobilize and transform communities on a broad scale to achieve better outcomes for youth and underscore the importance of a science-based approach in doing so. Future research should extend these findings by understanding the coalition characteristics that are associated with prevention system change.

Acknowledgments

This research was supported by a grant from the National Institute on Drug Abuse (R01 DA015183-03), with co-funding from the National Institute of Mental Health, National Institute of Child Health and Human Development, National Cancer Institute, and the Substance Abuse and Mental Health Services Administration/Center for Substance Abuse Prevention. The content of this paper is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.

Footnotes

1

Data on the fifth prevention system construct, utilization of the social development strategy, were not available for the examined wave of data and were not included in this study.

Portions of this study were presented in May 2010 at the 18th annual meeting of the Society for Prevention Research in Washington, DC.

Richard F. Catalano is a board member of Channing Bete Company, distributor of Supporting School Success® and Guiding Good Choices®. These programs were used in some communities in the study that produced the data set used in this paper.

Contributor Information

Eric C. Brown, Email: ricbrown@uw.edu, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA.

J. David Hawkins, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA.

Isaac C. Rhew, Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA

Valerie B. Shapiro, School of Social Welfare, University of California at Berkeley, Berkeley, CA, USA

Robert D. Abbott, College of Education, University of Washington, Seattle, WA, USA

Sabrina Oesterle, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA.

Michael W. Arthur, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA

John S. Briney, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA

Richard F. Catalano, Social Development Research Group, School of Social Work, University of Washington, 9725 3rd Avenue NE, Suite 401, Seattle, WA 98115, USA

References

  1. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:4–23. doi: 10.1007/s10488-010-0327-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ajzen I, Fishbein M. Understanding attitudes and predicting social behavior. Englewood Cliffs: Prentice Hall; 1980. [Google Scholar]
  3. Arthur MW, Hawkins JD, Catalano RF, Olson JJ. Community Key Informant Survey. Seattle: Social Development Research Group, University of Washington; 2002. [Google Scholar]
  4. Arthur MW, Glaser RR, Hawkins JD. Steps towards community-level resilience: Community adoption of science-based prevention programming. In: Peters RD, Leadbeater B, McMahon RJ, editors. Resilience in children, families, and communities: Linking context to practice and policy. New York: Kluwer Academic/Plenum Publishers; 2005. pp. 177–194. [Google Scholar]
  5. Arthur MW, Hawkins JD, Brown EC, Briney JS, Oesterle S, Abbott RD. Implementation of the Communities That Care prevention system by coalitions in the Community Youth Development Study. Journal of Community Psychology. 2010;38:245–258. doi: 10.1002/jcop.20362. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bauer DJ, Preacher KJ, Gil KM. Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods. 2006;11:142–163. doi: 10.1037/1082-989X.11.2.142. [DOI] [PubMed] [Google Scholar]
  7. Beckhard R, Harris RT. Organizational transitions: Managing complex change. 2nd ed. Reading: Addison-Wesley; 1987. [Google Scholar]
  8. Biglan A, Ary DV, Koehn V, Levings D. Mobilizing positive enforcement in communities to reduce youth access to tobacco. American Journal of Community Psychology. 1996;24:625–638. doi: 10.1007/BF02509717. [DOI] [PubMed] [Google Scholar]
  9. Brown EC, Hawkins JD, Arthur MW, Briney JS, Abbott RD. Effects of Communities That Care on prevention services systems: Outcomes from the Community Youth Development Study at 1.5 years. Prevention Science. 2007;8:180–191. doi: 10.1007/s11121-007-0068-3. [DOI] [PubMed] [Google Scholar]
  10. Brown EC, Graham JW, Hawkins JD, Arthur MW, Baldwin MM, Oesterle S, et al. Design and analysis of the Community Youth Development Study longitudinal cohort sample. Evaluation Review. 2009;33:311–334. doi: 10.1177/0193841X09337356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Brown LD, Feinberg ME, Greenberg MT. Determinants of community coalition ability to support evidence-based programs. Prevention Science. 2010;11:287–297. doi: 10.1007/s11121-010-0173-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Brown EC, Hawkins JD, Arthur MW, Briney JS, Fagan AA. Prevention service system transformation using Communities That Care. Journal of Community Psychology. 2011;39:183–201. doi: 10.1002/jcop.20426. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chinman M, Hannah G, Wandersman A, Ebener P, Hunter SB, Imm P, et al. Developing a community science research agenda for building community capacity for effective preventive interventions. American Journal of Community Psychology. 2005;35:143–157. doi: 10.1007/s10464-005-3390-6. [DOI] [PubMed] [Google Scholar]
  14. Emshoff JG, Darnell AJ, Darnell DA, Erickson SW, Schneider S, Hudgins R. Systems change as an outcome and a process in the work of community collaboratives for health. American Journal of Community Psychology. 2007;39:255–267. doi: 10.1007/s10464-007-9110-7. [DOI] [PubMed] [Google Scholar]
  15. Fagan AA, Van Horn ML, Hawkins JD, Arthur MW. Using community and family risk and protective factors for community-based prevention planning. Journal of Community Psychology. 2007;35:535–555. [Google Scholar]
  16. Fagan AA, Hawkins JD, Catalano RF. Engaging communities to prevent underage drinking. Alcohol Research Health. 2011;34:167–174. [PMC free article] [PubMed] [Google Scholar]
  17. Fawcett SB, Paine AL, Francisco VT, Vliet M. Promoting health through community development. In: Glenwick DS, Jason LA, editors. Promoting health and mental health in children, youth and families. Binghamton: Springer Publishing Company; 1993. pp. 233–255. [Google Scholar]
  18. Feinberg ME, Greenberg MT, Osgood D. Readiness, functioning, and perceived effectiveness in community prevention coalitions: A study of Communities That Care. American Journal of Community Psychology. 2004;33:163–176. doi: 10.1023/b:ajcp.0000027003.75394.2b. [DOI] [PubMed] [Google Scholar]
  19. Feinberg ME, Greenberg MT, Osgood D, Sartorius J, Bontempo D. Effects of the Communities That Care model in Pennsylvania on youth risk and problem behaviors. Prevention Science. 2007;8:261–270. doi: 10.1007/s11121-007-0073-6. [DOI] [PubMed] [Google Scholar]
  20. Feinberg ME, Jones D, Greenberg MT, Osgood DW, Bontempo D. Effects of the Communities That Care model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science. 2010;11:163–171. doi: 10.1007/s11121-009-0161-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fixsen DL, Blase KA, Duda MA, Naoom SF, Van Dyke M. Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. In: Weisz JR, Kazdin AE, editors. Evidence-based psychotherapies for children and adolescents. 2nd ed. New York: Guilford Press; 2010. pp. 435–450. [Google Scholar]
  22. Foster-Fishman PG, Behrens TR. Systems change reborn: Rethinking our theories, methods, and efforts in human services reform and community-based change. American Journal of Community Psychology. 2007;39:191–196. doi: 10.1007/s10464-007-9104-5. [DOI] [PubMed] [Google Scholar]
  23. Foster-Fishman PG, Berkowitz SL, Lounsbury DW, Jacobson S, Allen NA. Building collaborative capacity in community coalitions: A review and integrative framework. American Journal of Community Psychology. 2001;29:241–261. doi: 10.1023/A:1010378613583. [DOI] [PubMed] [Google Scholar]
  24. Gloppen KM, Arthur MW, Hawkins JD, Shapiro VB. Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. Journal of Adolescent Health. 2012;51:259–264. doi: 10.1016/j.jadohealth.2011.12.018. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hallfors D, Cho H, Livert D, Kadushin C. Fighting back against substance abuse: Are community coalitions winning? American Journal of Preventive Medicine. 2002;23:237–245. doi: 10.1016/s0749-3797(02)00511-1. [DOI] [PubMed] [Google Scholar]
  26. Hawkins JD, Catalano RF. Investing in your community's youth: An introduction to the Communities That Care system. South Deerfield: Channing Bete; 2002. [Google Scholar]
  27. Hawkins JD, Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addictive Behaviors. 2002;27:951–976. doi: 10.1016/s0306-4603(02)00298-8. [DOI] [PubMed] [Google Scholar]
  28. Hawkins JD, Brown EC, Oesterle S, Arthur MW, Abbott RD, Catalano RF. Early effects of Communities That Care on targeted risks and initiation of delinquent behavior and substance use. Journal of Adolescent Health. 2008a;43:15–22. doi: 10.1016/j.jadohealth.2008.01.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, et al. Testing Communities That Care: The rationale, design and behavioral baseline equivalence of the Community Youth Development Study. Prevention Science. 2008b;9:178–190. doi: 10.1007/s11121-008-0092-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, et al. Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: A test of Communities That Care. Archives of Pediatrics Adolescent Medicine. 2009;163:789–798. doi: 10.1001/archpediatrics.2009.141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Hawkins JD, Oesterle S, Brown EC, Monahan KC, Abbott RD, Arthur MW, et al. Sustained decreases in risk exposure and youth problem behaviors after installation of the Communities That Care prevention system in a randomized trial. Archives of Pediatrics Adolescent Medicine. 2012;166:141–148. doi: 10.1001/archpediatrics.2011.183. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Holder HD, Saltz RF, Grube JW, Voas RB, Gruenewald PJ, Treno AJ. A community prevention trial to reduce alcohol-involved accidental injury and death: Overview. Addiction. 1997;92:S155–S171. [PubMed] [Google Scholar]
  33. Kreuter MW, Lezin NA, Young LA. Evaluating community-based collaborative mechanisms: Implications for practitioners. Health Promotion Practice. 2000;1:49–63. [Google Scholar]
  34. Lynch KG, Cary M, Gallop R, Ten Have TR. Causal mediation analyses for randomized trials. Health Services Outcomes Research Methodology. 2008;8:57–76. doi: 10.1007/s10742-008-0028-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. MacKinnon DP. Introduction to statistical mediation analysis. New York: Taylor & Francis Group/Lawrence Erlbaum Associates; 2008. [Google Scholar]
  36. Merzel C, D’Afflitti J. Reconsidering community-based health promotion: Promise, performance, and potential. American Journal of Public Health. 2003;93:557–574. doi: 10.2105/ajph.93.4.557. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Muthén LK, Muthén BO. Mplus Version 6.1. Los Angeles: Muthén and Muthén; 2010. [Google Scholar]
  38. O’Connell ME, Boat T, Warner KE, editors. Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington: National Academies Press; 2009. [PubMed] [Google Scholar]
  39. O’Malley PM, Bachman JG, Johnston LD. Reliability and consistency in self-reports of drug use. International Journal of the Addictions. 1983;18:805–824. doi: 10.3109/10826088309033049. [DOI] [PubMed] [Google Scholar]
  40. Pentz MA. Institutionalizing community-based prevention through policy change. Journal of Community Psychology. 2000;28:257–270. [Google Scholar]
  41. Pentz MA, Dwyer JH, MacKinnon DP, Flay BR, Hansen WB, Wang EYI, et al. A multi-community trial for primary prevention of adolescent drug abuse: Effects on drug use prevalence. Journal of the American Medical Association. 1989;261:3259–3266. [PubMed] [Google Scholar]
  42. Quinby RK, Fagan AA, Hanson K, Brooke-Weiss B, Arthur MW, Hawkins JD. Installing the Communities That Care prevention system: Implementation progress and fidelity in a randomized controlled trial. Journal of Community Psychology. 2008;36:313–332. [Google Scholar]
  43. Rhew IC, Brown EC, Hawkins JD, Briney JS. Sustained effects of Communities That Care on prevention service system transformation. American Journal of Public Health. 2013;103:529–535. doi: 10.2105/AJPH.2011.300567. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Roberts-DeGennaro M, Mizrahi T. Coalitions as social change agents. In: Weil MO, editor. Handbook of community practice. Thousand Oaks: Sage; 2005. pp. 305–318. [Google Scholar]
  45. Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annual Review of Public Health. 2000;21:369–402. doi: 10.1146/annurev.publhealth.21.1.369. [DOI] [PubMed] [Google Scholar]
  46. Shapiro VB, Oesterle S, Abbott RD, Arthur MW, Hawkins JD. Measuring dimensions of coalition functioning for effective and participatory community practice. Social Work Research. doi: 10.1093/swr/svt028. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Shinn M. Mixing, matching: Levels of conceptualization. measurement, and statistical analysis in community research. In: Tolan P, Keys C, Chertok F, Jason L, editors. Researching community psychology: Issues of theory and methods. Washington: American Psychological Association; 1990. pp. 111–126. [Google Scholar]
  48. Shrout PE, Bolger N. Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods. 2002;7:422–445. [PubMed] [Google Scholar]
  49. Smith GT, McCarthy DM, Goldman MS. Self-reported drinking and alcohol-related problems among early adolescents: Dimensionality and validity over 24 months. Journal of Studies on Alcohol. 1995;56:383–394. doi: 10.15288/jsa.1995.56.383. [DOI] [PubMed] [Google Scholar]
  50. Social Development Research Group. Community Youth Development Study, Youth Development Survey [Grades 5–7] Seattle: Social Development Research Group, School of Social Work, University of Washington; 2005. [Google Scholar]
  51. Spoth RL, Greenberg MT. Toward a comprehensive strategy for effective practitioner–scientist partnerships and larger-scale community health and well-being. American Journal of Community Psychology. 2005;35:107–126. doi: 10.1007/s10464-005-3388-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Spoth R, Greenberg M, Bierman K, Redmond C. PROSPER community–university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science. 2004;5:31–39. doi: 10.1023/b:prev.0000013979.52796.8b. [DOI] [PubMed] [Google Scholar]
  53. Spoth R, Redmond C, Shin C, Greenberg M, Feinberg M, Schainker L. PROSPER community-university partnership delivery system effects on substance misuse through 6 1/2 years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine. 2013;56:190–196. doi: 10.1016/j.ypmed.2012.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Van Horn ML, Hawkins JD, Arthur MW, Catalano RF. Assessing community effects on adolescent substance use and delinquency. Journal of Community Psychology. 2007;35:925–946. [Google Scholar]

RESOURCES