Prevention Service System Transformation Using Communities That Care

Abstract

This study examines prevention system transformation as part of a community-randomized controlled trial of Communities That Care (CTC). Using data from surveys of community leaders, we examine differences between CTC and control communities 4.5 years after CTC implementation. Significantly higher levels of adopting a science-based approach to prevention observed in CTC communities compared to controls in 2004 were maintained in 2007. Leaders in CTC communities expressed a willingness to contribute significantly more funds to prevention than did leaders in control communities in 2007. Significant differences in levels of community collaboration observed in 2004 were not maintained in 2007. Leaders in CTC communities with high poverty rates and large minority student populations reported higher levels of community norms against drug use and greater use of the social development strategy, respectively, than did leaders in control communities with similar characteristics.

A systems approach to improving communities has been the subject of much recent attention in community psychology (Kelly, 2007; Wandersman et al., 2008), systems theory (Ackoff & Rovin, 2003; Hirsch, Levine, & Miller, 2007), prevention science (Feinberg, Greenberg, Osgood, Sartorius, & Bontempo, 2007; Greenberg, 2004; Hawkins, 2006), children’s mental health (Stroul & Manteuffel, 2007), services research (Powell, Fixsen, Dunlap, Smith, & Fox, 2007), community safety (Ford, 2007), and public health (Suarez-Balcazar et al., 2007). System change or transformation has been defined as ‘‘efforts that strive to shift the underlying infrastructure within a community or targeted context to support a desired outcome, including shifting existing policies and practices, resource allocations, relational structures, community norms and values, and skills and attitudes” (Foster-Fishman & Behrens, 2007, p. 192). With regard to community prevention service systems, a systems approach suggests that successful system transformation should: (a) be guided by a clearly articulated purpose and theory of change; (b) be flexible with regard to leverage points for change, yet have defined boundaries and a formalized structure; (c) explicate the interdependencies among components; (d) be proactive; (e) use proven effective strategies, and (f) focus on measuring outcomes ((Behrens & Foster-Fishman, 2007; Fixsen, Naoom, Blase, & Friedman, 2005; Stevenson & Mitchell, 2003; Weiss, 1995).

Communities That Care

Communities That Care (CTC) is a manualized system for developing and transforming communities’ prevention systems to address adolescent health and behavior problems via community prevention coalitions (Hawkins & Catalano, 2002; Hawkins, Catalano, & Arthur, 2002). CTC takes a public health approach to community change based on the premise that a reduction in the prevalence of adolescent health and behavior problems in a community can be achieved by identifying elevated risk factors and depressed protective factors experienced by the community’s youth population, and then selecting and implementing preventive interventions that have been shown in experimental or quasi-experimental studies to affect those specific risk and protective factors and, in turn, adolescent health and behavior problems. Repeated assessments of epidemiologic risk and protective factor data in communities are used to evaluate the effects of the community’s prevention service system and guide future prevention planning (Arthur & Blitz, 2000; Hawkins et al., 2002). Details of the CTC implementation process are described in Fagan et al. (2008), Hawkins et al. (2008), and Quinby et al. (2006).

Prevention System Transformation Using Communities That Care

The theory of change underlying CTC posits five constructs through which CTC seeks to bring about prevention system transformation. The constructs that link CTC implementation to prevention system outcomes are shown in Figure 1. First, adoption of a science-based approach to prevention is theorized to be the primary mechanism by which CTC empowers communities to select appropriate prevention programs and implement them at scale with sufficient fidelity (Arthur, Glaser, & Hawkins, 2005). Adoption refers to community leaders’ understanding and use of a science-based approach to plan and implement prevention services for the prevention of mental, emotional, and behavioral disorders in youth populations (see Coie et al., 1993; O'Connell, Boat, & Warner, 2009). In brief, adoption includes understanding of prevention science concepts such as risk and protective factors, empirical assessment of adolescent exposure to risk and protective factors in communities, use of tested and effective prevention policies and programs, and ongoing monitoring of community prevention efforts and outcomes. Adoption of a science-based prevention approach is hypothesized to directly influence a community’s’ ability to base decisions regarding the choice of prevention policies and programs on evidence regarding the prevalence of exposure to empirically identified specific risk and protective factors in the community; to choose preventive interventions shown in previous scientifically valid studies to reduce risk, enhance protection, and reduce adolescent health and behavior problems; and to monitor prevention implementation processes and outcomes and adjust interventions based on results.

Figure 1.

Theoretical model of Communities That Care prevention system transformation.

Collaboration is a widely recognized construct for mobilizing communities for prevention system transformation (Emshoff et al., 2007; Foster-Fishman, Berkowitz, Lounsbury, Jacobson, & Allen, 2001; Roussos & Fawcett, 2000; Stevenson & Mitchell, 2003) and is viewed as a necessary condition for success in the CTC theory of change. Collaboration includes elements of networking, information exchange, coordination of activities, and sharing of resources within communities (Himmelman, 2001) and involves stakeholders concerned about adolescent development from diverse sectors of the community (Greenbaum & Dedrick, in press; Riggs, Feinberg, & Greenberg, 2002). Collaboration is theorized to result in broad support and shared responsibility for community-based initiatives, reduced duplication and fragmentation of community resources, more interagency cooperation, and improved implementation and sustainability of system change. With regard to prevention system transformation, we define community collaboration for prevention as the degree to which community members, representing different sectors of the community, engage in information exchange, coordination of activities, and sharing of resources to strengthen the prevention of adolescent health and behavior problems that are of concern to the community.

Community support for prevention refers to community members’ willingness to support prevention efforts and programs (Beckhard & Harris, 1987; Fawcett, Paine, Francisco, & Vliet, 1993). Community support has been shown to be a viable mechanism in community interventions for reducing tobacco sales to youth (Biglan, Ary, Koehn, & Levings, 1996). Pentz (2000) argued that community support for prevention is a key factor in promoting prevention policy initiatives. An important indicator of community support is the willingness to fund prevention programming in communities and to reallocate resources toward prevention. With regard to prevention system transformation, CTC implementation is expected to increase community support for prevention and, in turn, lead to increased installation of tested and effective prevention policies and programs. Ultimately, community support for prevention should help sustain prevention initiatives for the long term, which should increase the likelihood of sustained prevention system transformation.

Community norms are the “shared expectations of how people should behave within certain roles or situations” (Caughy, Brodsky, O'Campo, & Aronson, 2001, p. 682). In their examination of neighborhood parenting norms, Caughy et al. posited that community norms are (a) developed as a function of routinized behaviors over time, (b) dynamic, (c) often heterogeneous within communities, and (d) represent an important evaluative component for improving system outcomes. As a transformative mechanism leading toward positive youth development, community norms against adolescent drug use and other adolescent behavior problems should facilitate implementation of prevention policies and programs by encouraging community leaders to address perceived permissiveness toward adolescent drug use or other problem behaviors and should affect community levels of risk and protection by enhancing expectations of behaving in accordance with social norms (Pentz, 2000). The linkage to youth outcomes is supported by the effectiveness of community-based interventions that sought to change community norms regarding alcohol (Holder et al., 1997; Pentz et al., 1989; Wagenaar et al., 1999; Wagenaar & Perry, 1994), and etiological research showing both concurrent and prospective associations between levels of community norms regarding drug use and adolescent drug use (Beyers, Toumbourou, Catalano, Arthur, & Hawkins, 2004; Fagan, Van Horn, Hawkins, & Arthur, 2007; Van Horn, Hawkins, Arthur, & Catalano, 2007).

The social development strategy (Hawkins, 2006) refers to the provision of opportunities for prosocial engagement; social, emotional, and cognitive skills; and positive recognition and reinforcement as methods to strengthen bonds of attachment and commitment to prosocial others and the larger community, and, in turn, adoption of standards for healthy behavior. In CTC, the social development strategy is used both to facilitate coalition functioning by promoting bonding among coalition members and to promote positive behavior in communities’ youth populations through community members’ interactions with youth. Research on the social development strategy has shown it to predict adolescent delinquent behaviors, alcohol use, and other drug use (Brown et al., 2005; Catalano et al., 2005; Deng & Roosa, 2007; Fleming et al., 2008; Lonczak et al., 2001). The social development strategy hypothesizes that, to the degree that members of the community who interact with youth (a) provide youth with opportunities to participate in prosocial roles in the community, (b) help youth learn and practice the skills needed in these roles, and (c) recognize and reinforce youth for positive behaviors, youth will form strong positive bonds with these community members. In the presence of clear community norms against adolescent drug use and other adolescent problem behaviors, strong bonds to prosocial community members should reduce motivation to engage in adolescent drug use or other adolescent problem behaviors such as delinquency.

To summarize, the theory of change underlying CTC holds that (a) adoption of a science-based approach to prevention, (b) collaboration regarding prevention issues, (c) support within communities for prevention, (d) clear community norms against adolescent drug use, and (e) use of the social development strategy represent leverage points for system transformation that are activated and reinforced through CTC training sessions, technical assistance, and ongoing system monitoring. In turn, these constructs are hypothesized to have direct effects on the choice and implementation of specific prevention policies and programs and, in the case of community norms and the social development strategy, additional direct effects on community adolescent risk, protection, and health and behavior outcomes. Community prevention systems can be assessed in multiple ways, including through ethnographic research (e.g., Evans & Lambert, 2008), reviews of policy (e.g., Dershem, 1990), and analysis of administrative data (e.g., Armstrong et al., 2007). The present study used key informant survey methodology to measure the perceptions of diverse community leaders regarding the characteristics of the prevention systems of their communities.

The Community Youth Development Study

The Community Youth Development Study (CYDS) (Brown et al., 2009; Hawkins et al., 2008) is a community-randomized controlled trial of CTC currently being conducted in 12 matched pairs of communities across seven states: Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington. Communities in the CYDS were selected from a larger pool of communities that participated in a naturalistic study of the diffusion of science-based prevention strategies (Arthur et al., 2005). When CYDS was initiated, the 24 CYDS communities were small and medium-sized geographically distinct, incorporated towns with an average population of 14,646 (range = 1,578 to 40,787). On average, 89% of the population members were European American (range = 64% to 98%), 3% were African American (range = 0% to 21%), 10% were of Hispanic origin (range = 1% to 65%), 12% were between the ages of 10 and 17 years (range = 9% to 16%), and 38% of students were from low-income families as indicated by eligibility for free or reduced-price school lunch (range = 18% to 66%). One community from within each of the 12 matched pairs of communities was assigned randomly to either the CTC intervention or control condition involving prevention services as usual.

Aims of the CYDS include examining change in communities’ levels of adoption of science-based prevention and collaboration as mechanisms for prevention system transformation. Brown et al. (2007) reported significant differential change in levels of these constructs between CTC and control communities 1.5 years after CTC was initiated in intervention communities. The present paper examines the long-term effects of CTC on adoption and collaboration constructs using a third wave of data from community leaders, collected 4.5 years after CTC initiation. It also examines, at both 1.5 years and 4.5 years after CTC initiation, effects of the CTC intervention on measures of community support for prevention, community norms against adolescent drug use, and use of the social development strategy. Characteristics of the community leader respondents and communities were included as covariates in the multilevel models to help elucidate the factors that potentially influence prevention system transformation, as well as to rule out alternative explanations for hypothesized differences between CTC and control communities.

Methods

Participants

Participants for this study were 731 community leaders identified during administrations of the Community Key Informant Survey (Arthur, Hawkins, Catalano, & Olson, 2002) in 2001, 2004, and 2007. Sampling of community leaders followed a two-stage approach. First, positional leaders were identified in each community as individuals who held leadership positions (e.g., mayors, city managers, police chiefs, school superintendents, business leaders, or heads of social service agencies) in 11 different community sectors. Second, a list of referred leaders was generated by asking each positional leader to identify up to two individuals in the community who they thought were the most knowledgeable about current prevention efforts in the community. From this list, research staff interviewed the five referred leaders recommended most frequently by the positional leaders in each community. This procedure was repeated for each wave of data collection, resulting in samples of 354 community leaders for the 2001 survey, 340 for the 2004 survey, and 336 for the 2007 survey. Respondents who held the same leadership position in their community over time were interviewed across multiple waves. Thus, 10% of the total sample (n = 72) were interviewed in all three waves; 21% (n = 155) were interviewed in two of the three waves; and 69% (n = 504) were interviewed in one of the three waves. Participants averaged 49.0 years of age (SD = 10.1) at time of the interview; 63% of the sample were positional respondents; 58% were male. Participants had lived an average of 16.8 years in their respective communities (SD = 17.3), and 45% of the sample had a bachelor’s degree or higher. The sample was split between intervention (50.5%) and control (49.5%) communities. The percentage of community leaders in each of the 11 service sectors is shown in Table 1 by year and intervention status. No differences were observed between CTC and control communities in the percentages of community leaders within a service sector for each wave of data, except for the percentages of leaders that represented community coalitions in 2007 (i.e., 11.1% in CTC communities vs. 3.7% in control communities), χ² (1, N = 336) = 8.91, p < .01. Participants did not differ significantly between CTC and control groups on any other demographic characteristic within each survey year (all ps > .05).

Table 1.

Percentages of Community Leaders by Sector, Year, and Intervention Status

	Year
	2001		2004		2007
Sector	CTC	Control	CTC	Control	CTC	Control
Community coalitions	4.1	4.9	3.7	2.5	11.1a	3.7b
Human services	13.0	12.9	11.0	13.7	12.9	14.8
Schools	12.4	14.7	14.0	13.7	11.7	16.7
Health agencies	na	na	7.3	6.2	7.0	8.0
Business	7.1	6.7	6.7	6.8	7.0	8.6
Civic	13.0	12.3	8.5	14.3	9.4	9.3
Youth recreation	15.4	12.3	11.0	8.1	7.6	6.2
Law enforcement	11.2	11.7	12.2	10.6	8.2	8.6
Juvenile justice	10.1	9.2	9.8	9.9	9.4	11.1
Media	7.1	8.6	7.3	7.5	6.4	6.2
Religious	6.5	6.7	8.5	6.8	9.4	6.8

Note. Bolded sectors are included in analysis models as effect-coded dummy variables. Percentages with different superscripted letters are significantly different (p < .05). CTC = Communities That Care. na = not applicable (i.e., sector not included in that year).

Measures

Measures used in this study were taken from the Community Key Informant Survey (Arthur et al., 2002), which included questions regarding characteristics of the community and its approach to prevention services and activities. Trained interviewers administered the Community Key Informant Survey by telephone using a computer-assisted interview lasting, on average, about 1 hour.

System Transformation Constructs

Adoption of a science-based approach to prevention (adoption) was measured by responses to 21 closed-ended items drawn from content domains representing awareness and use of prevention science concepts, use of epidemiologic data, use of tested and effective prevention programming, and system monitoring (Arthur et al., 2005). Items assessed respondents’ awareness of a risk and protective focused prevention approach to prevention planning, knowledge of specific risk and protective factors, their perceptions of the community’s adoption of a prevention approach focused on reducing risk and enhancing protection, and the use of survey or archival data to guide prioritization of specific risk and protective factors as targets for prevention, prevention strategy selection, resource allocation, and prevention program evaluation. Each respondent’s rating of his or her community’s stage of adoption was calculated based on a set of decision rules, developed by CYDS investigators, for scoring responses to the 21 items. These rules were designed to categorize the community’s stage of adoption by assessing whether or not the respondent’s answers to specific questions met the criteria defining each stage. Based on the pattern of responses to the closed-ended items, each respondent was given an overall stage score representing the highest stage for which criteria were met. Stage scores ranged from 0, indicating little or no awareness of prevention science to 5, indicating the use of tested and effective prevention programs that were selected to address the community’s specific profile of risk and protection, with ongoing assessments to monitor program implementation and the effects of the programs on outcomes. Inter-rater reliability (defined as the ratio of true score variance to total variance; Raudenbush & Bryk, 2002) from a multilevel analysis of respondents nested within communities across all three waves of data was .81, indicating sufficient reliability in assessment of adoption of a science-based approach among community leaders within communities.

We measured community collaboration for prevention (collaboration) as a second-order factor consisting of two elemental components of collaboration regarding prevention activities in communities. The first component, sectoral collaboration, consisted of an index of the degree to which community leaders reported collaborating with other sectors of the community on prevention issues. Response options for this item were coded 0 = None, 1 = Some, 2 = A little, and 3 = A lot. Scores for this measure indicated the average response across seven community sectors (i.e., civic, business, schools, law enforcement, community coalitions, human services, and religious). The second element of collaboration consisted of a first-order factor measuring the extent to which community leaders engaged in prevention-specific collaborative activities (e.g., sharing resources, coordinating activities). This latent construct, which we have previously labeled prevention collaboration, was indicated by nine items indicating successful community collaboration (see Brown, Hawkins, Arthur, Abbott, & Van Horn, 2008). Items for this measure were coded using a 4-point scale (i.e., 1 = Strongly Disagree, 2 = Somewhat Disagree, 3 = Somewhat Agree, and 4 = Strongly Agree).

Community support for prevention was operationalized using two measures. First, “general support for prevention” was conceptualized as a second-order latent variable identified by two first-order latent variables: community member support and community leader support, developed in an earlier study (Arthur et al., 2005). The four community member support items measured community leaders’ assessments of community members’ beliefs in prevention effectiveness, knowledge of prevention efforts, and willingness to pay for prevention programs. The three community leader support items assessed these same topics, but in reference to community leaders themselves, rather than their assessments of community members (alphas = .77, .77, and .78, respectively). Response options for all items were coded as 1 = Strongly Disagree, 2 = Somewhat Disagree, 3 = Somewhat Agree, and 4 = Strongly Agree.

Additionally, community leaders were asked “If you were deciding how to spend money for reducing substance abuse, what percentage would you allocate to each of the following approaches? Law Enforcement, Treatment, and Prevention.” The percentage respondents would allocate to prevention was used as a measure of community support for prevention, which we labeled “Desired Prevention Funding.”

Community norms against adolescent drug use were measured by six items that indicated community leaders’ perceptions of community members’ normative beliefs about adolescent substance use. This construct was operationalized as a first-order factor indicated by responses to six items: In this community, how wrong do most adults think it is for adolescents to (a) drink alcohol, (b) smoke cigarettes, and (c) use marijuana? and Adults in [community] think that using (a) alcohol, (b) tobacco, and (c) marijuana, are a normal part of growing up. Responses to the first set of questions were coded 1 = Not wrong at all, 2 = A little wrong, 3 = Wrong, and 4 = Very wrong. Responses to the second set of statements were 1 = Strongly Disagree, 2 = Somewhat Disagree, 3 = Somewhat Agree, and 4 = Strongly Agree.

Use of the social development strategy was measured in the 2007 administration of the Community Key Informant Survey by five items (alpha = .91) that were modified from existing measures of the social development model (Catalano & Hawkins, 1996; Hawkins & Weis, 1985) for use with community leaders. Items asked about community leaders’ perceptions of the extent to which they work to (a) increase opportunities for prosocial youth activities, (b) help adolescents learn new skills, (c) recognize and compliment youth for positive accomplishments, (d) promote bonding between youth and prosocial members of the community, and (e) ensure clear and explicit standards for youth behavior. Response options were coded 1 = Strongly Disagree, 2 = Somewhat Disagree, 3 = Disagree Slightly, and 4 = Agree Slightly, 5 = Moderately Agree, and 6 = Strongly Agree.

Characteristics of Respondents and Communities

Respondent characteristics included: age, gender, positional versus referred status, education (dichotomized as baccalaureate degree or less versus master’s degree or more), how many years the community leader resided in the community, and the number of waves in which each community leader responded across the three survey administrations. The community sector from which the respondent was sampled also was included. Community sectors were effect-coded to represent the effects of being from that sector on system transformation.

Data on the characteristics of study communities were obtained from either the 2000 U.S. Census (i.e., total population of the community, percentage of the community population that was Nonwhite, and percentage of families under the 1999 federal poverty guidelines) or from the National Center for Education Statistics for the 2003–2004 school year (i.e., percentage of students in the community who were eligible to receive free or reduced-price lunch). Finally, a community-level dichotomous variable (intervention status), indicating whether the community was a CTC community (coded 1) or control community (coded 0), was included in the analysis.

Statistical Analyses

We used three-level hierarchical linear models (HLM; Raudenbush & Bryk, 2002) to examine differences in the six measures of system transformation by survey year (at Level 1), nested within community leaders (at Level 2), in turn, nested within communities (at Level 3). Outcome measures were modeled as normally distributed, except for the ordered-categorical adoption construct, which employed a cumulative probability model (Raudenbush, Bryk, Cheong, & Congdon, 2004) to model the discrete stages of adoption. We examined intervention and covariate effects within each survey year using a fully multivariate model specification (Snijders & Bosker, 1999). This specification incorporates three independent variables in the Level 1 equation—one for each wave of data—with the year of interest coded as 1 and other years coded as 0. All models explicitly partitioned the variability in outcomes across the three levels and assessed the intervention effects with communities as the unit of analysis.

Statistical analysis for each outcome proceeded by examining the characteristics of the community leaders and communities, including intervention status, as predictors in Level 2 and Level 3 equations, respectively. Interaction terms between community-level variables and intervention status were included sequentially to test for potential moderating effects of community characteristics on system transformation, with significant (p < .05) interaction terms retained in the final models.

Results

Means and proportions for the five prevention system transformation constructs are presented in Table 2 by intervention status for 2001, 2004, and 2007.

Table 2.

Means or Percentages (Standard Deviations) of Prevention System Transformation by Year and Intervention Status

	Year
	2001		2004		2007
Construct	CTC	Control	CTC	Control	CTC	Control
Adoption of a science-based approach to preventiona	27.5%	31.2%	49.2%	28.4%	51.7%	25.6%
Community collaboration for prevention	−.013 (.32)	.060 (.35)	−.001 (.34)	−.044 (.34)	.019 (.32)	−.003 (.36)
Community support for prevention	.034 (.50)	.043 (.51)	.018 (.46)	−.100 (.57)	.031 (.52)	−.071 (.57)
Desired funding for prevention	41.5%	40.9%	42.4%	40.3%	44.6%	40.8%
Community norms against drug abuse	.041 (.77)	.015 (.73)	.030 (.74)	−.087 (.76)	.067 (.71)	−.090 (.76)
Social development strategy	na	na	na	na	.105 (.77)	.009 (.86)

Bolded values represent significant differences (p < .05) within each year. CTC = Communities That Care. na = not applicable (i.e., construct not included in that year).

^aValues indicate average percentages of respondents at or above Stage 4.

Adoption of a Science-based Approach to Prevention

As reported elsewhere (Brown et al., 2007), CTC communities demonstrated significantly greater levels of adopting a science-based approach to prevention in 2004 than did control communities. This intervention effect was maintained in 2007, t(18) = 6.9, p < .001, with greater levels of adoption indicated by leaders from CTC communities than by leaders from control communities. Model-implied probabilities of being at the highest stage of adoption (Stage 5) in 2007 were .15 for control communities and .43 for CTC communities. The adjusted odds ratio associated with this effect indicated that community leaders from CTC communities were 5.37 times more likely to report a higher stage of adoption of a science-based approach to prevention than leaders from control communities.

Female community leaders and community leaders from the school sector reported significantly higher levels of adoption than their male and nonschool sector counterparts, ts(704) = −2.63 and −4.12, ps < .01, respectively. Community leaders from the business sector reported lower levels of adoption than leaders from other sectors, t(704) = 2.09, p < .05. No other covariates were related significantly to 2007 levels of adoption.

Community Collaboration for Prevention

CTC and control communities did not differ significantly in mean levels of collaboration at any of the three time points. Results of unconditional multilevel models assessing Community Collaboration for Prevention indicated significant covariate effects among respondent characteristics in the multivariate conditional models: (a) older community leaders reported higher levels of collaboration, ts(704) = 2.01, 3.72, and 2.71, ps < .05, for 2001, 2004, and 2007, respectively; (b) leaders from the business sector reported less collaboration in 2007, t(704) = 2.30; p < .05; and (c) leaders who had lived longer in the community reported greater collaboration in 2007, t(704) = 2.01, p < .05. Significant effects for community characteristics included: (a) community population was positively related to higher levels of collaboration in 2001 and 2004, ts(18) = 2.97, and 2.42, ps < .05, respectively; and (b) the percentage of families living in poverty was negatively related to levels of collaboration in 2007, t(18) = −2.41, p < .05.

Community Support for Prevention

General support for prevention

Full models with all respondent and community covariates found that levels of general support for prevention did not differ significantly between CTC and control communities at any of the three waves. The analyses indicated higher levels of perceived general support for prevention by community leaders from the business sector in 2001, t(704) = 2.55, p < .05, and the civic sector in 2007, t(704) = 2.22, p < .05, and lower levels of perceived general support reported by leaders from the human services sector in 2001, t(704) = 2.17, p < .05. Lower levels of general support for prevention also were reported in 2004 by younger respondents and female respondents, ts(704) = −4.43 and 2.17, ps < .01, respectively. No community-level covariates were related significantly to general support for prevention in any of the three waves of data.

Desired prevention funding

The percentages of funding that would be allocated to prevention by community leaders remained relatively constant in control communities across time from 2001 to 2007 (i.e., 40.9%, 40.3%, and 40.8% in 2001, 2004, and 2007, respectively). However, for CTC communities, these percentages increased over time (41.5%, 42.4%, and 44.7%, respectively). Results of the multilevel analysis modeling intervention status indicated that differences in these percentages were nonsignificant between CTC and control communities in 2001 and 2004; however, they were significantly different by 2007, t(19) = 2.16, p < .05, with higher levels of desired funding for prevention being reported by leaders in CTC communities than by leaders in control communities. Standardized effect sizes for the between-group difference in percentages of funding that would be allocated to prevention was d = .22 for respondent-level means and d = .78 for community-level means.

Results of the multilevel analysis including all model covariates indicated significantly higher levels of desired prevention funding among (a) referred respondents in 2001 and 2004, ts(704) = 2.36 and 2.11, ps < .05, respectively; (b) respondents from the human services sector at all three waves, ts(704) = 3.15, 2.36, and 3.95, ps < .05, respectively; and (c) respondents from the school sector in 2004, t(704) = 3.35, p < .01. Lower levels of desired prevention funding were reported by leaders from (a) the juvenile justice sector at all three waves, ts(704) = −8.59, 4.49, and −6.61, ps < .01, respectively; and (b) law enforcement sectors in 2001 and 2007, ts(704) = −4.66 and −2.82, ps < .05, respectively.

Community Norms Against Adolescent Drug Use

Results of full models with all respondent and community covariates indicated that CTC and control communities did not differ significantly in community norms against adolescent drug use in any of the three waves. However, a significant intervention status by poverty interaction was found in 2007, t(17) = −2.36, p < .05. Community leaders in CTC communities with higher percentages of families in poverty reported higher levels of community norms against adolescent drug use than did leaders in control communities with higher percentages of families in poverty. Figure 2 shows model-predicted mean factor scores for CTC and control communities at one standard deviation below (low poverty) and above (high poverty) the mean percentage of families in the communities below the federal poverty guidelines. Results also indicated that referred community leaders in 2001 and 2004, ts(704) = −2.82 and −2.35, ps < .05, respectively; and leaders from human services in 2001, t(704) = −2.26, p < .05), school in 2004, t(704) = −2.07, p < .05), law enforcement in 2004, t(704) = −2.93, p < .01), and juvenile justice in 2007, t(704) = −2.28, p < .05, reported lower levels of community norms against adolescent drug use than their respective counterparts. Conversely, leaders from the business sector in 2001, t(704) = 2.78, p < .01; and the civic sector in all three waves, ts(704) = 2.61, 2.02, and 3.53, ps < .05, respectively, reported higher levels of community norms against adolescent drug use. Among community-level covariates, community population was related to higher 2007 levels, with respondents from more populous communities reporting higher levels of community norms against adolescent drug use, t(17) = 2.17, p < .05.

Figure 2.

2007 model-implied factor scores for community norms against adolescent drug use for communities at high (1 SD above the mean) and low (1 SD below the mean) percentages of families in poverty by intervention status.

Social Development Strategy

Use of the social development strategy reported by community leaders in 2007 did not differ significantly between CTC and control communities. However, a significant interaction effect was found between the percentage Nonwhite in the community and intervention status, t(17) = 2.28, p < .05. This interaction effect is depicted in Figure 3, which shows model-predicted mean factor scores for CTC and control communities at one standard deviation above (high percentage Nonwhite) and below (low percentage Nonwhite) the mean percentage of Nonwhite students in communities. As shown in the figure, community leaders in CTC and control communities with low percentages of Nonwhite youths reported similar levels of use of the social development strategy. However, community leaders in CTC communities with high percentages of Nonwhite youths reported higher use of the social development strategy than did leaders in control communities with high percentages of Nonwhite youths. Results also showed significantly lower levels of social development strategy use among respondents from the human services sector, t(308) = −2.35, p < .05, and in communities with higher percentages of Nonwhite youth populations, generally, t(17) = −2.53, p < .05.

Figure 3.

Model-implied social development strategy factor scores (2007) for communities at high (1 SD above the mean) and low (1 SD below the mean) percentages of Nonwhite youth by intervention status.

Discussion

Increasing attention is being given to system transformation as an approach for improving community safety, health, and wellbeing. This study examined CTC, within the context of a community-randomized controlled trial, as a method for transforming community prevention systems toward reducing adolescent health and behavior problems and promoting positive youth development. As a model for prevention system transformation in communities, CTC’s theory of change holds that CTC implementation reduces community risk, enhances community protection, and ultimately promotes positive outcomes in communities’ youth populations through adoption of a science-based framework for prevention, enhancement of collaboration across community sectors on prevention initiatives, increased support for prevention in communities, promotion of community norms against adolescent drug use and other adolescent problem behaviors including delinquency, and use of the social development strategy.

Because the focus of this study was to examine CTC’s impact on community prevention system transformation, we used multilevel models to analyze key informant survey data collected from community leaders representing diverse sectors nested within each of the 24 CYDS communities. Although key informant survey methodology has been used widely in community research (Shinn, 1990), we recognize the limitation of using only one source of information to assess system change and note that this study focused on one aspect of a potentially larger systemic process (e.g., economic and political changes in communities).

Study data were collected in three waves, which allowed for examination of system transformation constructs over time. Results of the analyses indicated substantial variability across communities in almost all of the examined system transformation constructs, with derived ICCs being consistent with measures of community risk and protective factors (Van Horn et al., 2007), community coalition functioning (Allen, 2005), and other community processes (Wilkinson, 2007).

Examination of time-specific mean differences in system transformation constructs between CTC and control communities indicated no significant effects for any of the assessed measures at the pre-intervention time point, suggesting that CTC and control communities were comparable at baseline as would be expected by their randomization to condition in the CYDS. Several differences were found, however, in 2004 and 2007. Leaders in CTC communities reported higher levels of adopting of a science-based approach to prevention in both 2004 and 2007 than did leaders in control communities, with similar differences in levels of adoption between CTC and control communities for both waves. In CTC’s theory of change, adoption of a science-based prevention approach is a primary mechanism for prevention system transformation in that, without sufficient knowledge and utilization of a science-based approach to prevention efforts, other mechanisms for system change may not be as effective in helping communities achieve desired goals (Arthur et al., 2005). Second, leaders in CTC communities indicated a trend to be willing to provide greater funding to prevention versus law enforcement or treatment over time, with significant mean differences in this outcome by 2007. Because a lack of resources to fund community prevention initiatives is likely to stall meaningful efforts to change communities and inhibit sustainability of prevention efforts (George et al., 2008), this finding may help elucidate CTC’s effects in transforming communities’ prevention systems.

Other findings suggest that CTC affects prevention system transformation in the presence of community diversity, that is, in communities with high poverty and high minority adolescent populations. These two characteristics of communities appear to dampen prevention system transformation in control communities when compared to CTC communities. This suggests a potential role of CTC in helping disadvantaged communities achieve prevention goals and reduce adolescent health and behavior problems.

No effects relating to the CTC intervention were found for the constructs of collaboration or general support for prevention. Brown et al. (2007) found that rates of change (i.e., slopes) for prevention collaboration and sectoral collaboration decreased significantly in control communities relative to CTC communities from 2001 to 2004, though mean levels of collaboration did not differ significantly between CTC and control communities in 2004. The current study found that mean levels of collaboration in control communities did not differ significantly from those in CTC communities in either 2004 or 2007, and the trend of decreasing collaboration in control communities from 2001 to 2004 reversed from 2004 to 2007. This may reflect the strong emphasis on the importance of collaboration in federal and state programs that provide funding to communities for drug abuse prevention (Florin, Mitchell, Stevenson, & Klein, 2000; Foster-Fishman et al., 2001; Lindholm, Ryan, Kadushin, Saxe, & Brodsky, 2004). Moreover, mean levels of collaboration approached the maximum of the scale by 2007. It appears that by 2007, high levels of collaboration had become normative across all 24 CYDS communities.

Results of respondent-level covariate effects from full conditional models are interesting in that the system transformation constructs examined in this study were related to the examined characteristics of respondents and/or communities. Analysis of respondent characteristics indicated no differences between CTC and control communities except for the percentages of leaders that represented community coalitions in 2007, which were larger in CTC communities (11.1%) than in control communities (3.7%). This difference is a function of CTC increasing the number of referred respondents (i.e., those most knowledgeable about prevention activities in communities) from the coalition sector, which is expected as a result of CTC coalition member trainings. The various relationships between the community sectors represented by the community leaders and system transformation constructs demonstrate community leaders’ differing perspectives regarding system transformation and indicate community sector differences in prevention system functioning as reported by Riggs et al. (2002). However, our findings contrast with findings by Hays et al. (2000) and Emshoff et al. (2007) who reported nonsignificant (ps > .05) relationships between coalition sectoral representation and system transformation. Differences in construct measurement and statistical models may account for this; however, future studies of prevention system transformation would be well advised to attend to the characteristics of individual respondents in analyzing community-level constructs.

Among community-level covariate effects, it is noteworthy that high levels of poverty and community racial/ethnic diversity are related to lower levels of collaboration and perceived use of the social development strategy in communities. The poverty finding is similar to those reported by Greenberg, et al. (2007) and Feinberg, Chileski, et al. (2007), who found that greater poverty in communities was related to lower levels of community readiness, collaboration, and prevention coalition team cohesion and leadership quality. It is also noteworthy that the number of people living in a community was found to be associated positively with levels of collaboration and community norms against adolescent drug use. A few of the communities in the CYDS were very small towns of less than 5,000, and it may be difficult in such small towns to develop high levels of collaboration across organizations with very limited human resources.

The present findings suggest that the emphasis that is placed on providing education in prevention science to both key leaders and members of CTC boards in the CTC Key Leaders Orientation and the CTC Community Board Orientation training events achieves the desired goal of transforming prevention service systems in CTC communities to increase adoption of a science-based prevention approach community wide. It is possible that this transformation could be further enhanced by even broader community-wide training in prevention science for all community service providers and stakeholders interested in positive youth development.

In contrast, it does not appear that CTC training produced sustained increases in levels of community collaboration for prevention in CTC communities compared to control communities. This may reflect the fact that the importance of collaboration for prevention is widely understood and accepted across the United States. Clearly, by 2007, levels of collaboration reported in both CTC and control communities were quite high. It is noteworthy that collaboration alone, in the absence of adoption of a science-based approach, was not found to produce significant reductions in substance use community wide in the Robert Wood Johnson Fighting Back initiative (Hallfors, Cho, Livert, & Kadushin, 2002). It may be that collaboration is necessary, but not sufficient to produce significant effects on drug use outcomes at the community level.

Communities That Care (CTC) seeks to transform community-based service delivery and planning systems for the prevention of adolescent health and behavior problems. As a system transformation approach, CTC is consistent with principals of effective system change, including clear goals, local involvement, multiple perspectives, identified leverage points, and a clearly articulated theory of change that delineates the linkages between system change and ensuing outcomes. Results from this study suggest that CTC implementation could have long-term beneficial effects on community prevention system transformation, which should lead to reductions in adolescent health and behavior problems.