Testing the Universality of the Effects of the Communities That Care Prevention System for Preventing Adolescent Drug Use and Delinquency

Abstract

Universal community-oriented interventions are an important component in the prevention of youth health and behavior problems. Testing the universality of the effects of an intervention that was designed to be universal is important because it provides information about how the program operates and for whom and under what conditions it is most effective. The present study examined whether the previously established significant effects of the universal, community-based Communities That Care (CTC) prevention program on the prevalence of substance use and the variety of delinquent behaviors held equally for boys and girls and in risk-related subgroups defined by early substance use, early delinquency, and high levels of community-targeted risk at baseline. Interaction analyses of data from a panel of 4,407 students followed from Grade 5 to Grade 8 in the first randomized trial of CTC in 12 matched community pairs suggests that CTC reduced students' substance use and delinquency equally across risk-related subgroups and gender, with two exceptions: the effect of CTC on reducing substance use in 8^th grade was stronger for boys than girls and the impact of CTC on reducing 8th-grade delinquency was stronger for students who were nondelinquent at baseline.

Universal community-oriented interventions are an important component in the prevention of youth health and behavior problems, including drug use and delinquency (Spoth, Shin, Guyll, Redmond, & Azevedo, 2006). Several community-driven, community-wide approaches have been evaluated and found to have positive effects in reducing problem behaviors among community youth (Hawkins et al., 2008a; Hawkins et al., 2009; Pentz et al., 1989; Perry et al., 2002; Spoth et al., 2007; Wagenaar et al., 1999). Like universal interventions in other domains, community-based initiatives are usually designed to reach and affect a community's youth equally. While the effects of universal programs are not expected to differentially affect particular individuals or groups of individuals, preventive interventions have at times been demonstrated to be effective only for certain subgroups of the population, such as youth who are at risk of developing health and behavior problems (Kellam et al., 2008; Kellam, Ling, Merisca, Brown, & Ialongo, 1998; Van Horn et al., 2008). Differences in a universal prevention program's effectiveness may be overlooked, however, since evaluations of universal programs typically focus on demonstrating effectiveness using an intent-to-treat approach, analyzing the intervention's impact on all subjects in the intervention group compared to all those in the control group. As prevention science emphasizes the widespread dissemination of effective prevention programs (Elliott & Mihalic, 2004; Pentz, Jasuja, Rohrbach, Sussman, & Bardo, 2006; Rohrbach, Grana, Sussman, & Valente, 2006; Woolf, 2008), understanding the effectiveness of universal prevention programs becomes a priority. The present study is an examination of the universality of intervention effects found in the first randomized trial of Communities That Care (Hawkins et al., 2009), a universal, community-wide prevention system designed to prevent adolescent drug use and delinquency.

Establishing the generalizability of intervention findings is one of the standards of evidence for effective prevention programs and policies developed by the Society for Prevention Research (Flay et al., 2005). Understanding differential effectiveness of an intervention that was designed to be universal is important because it provides more complete information about how the program operates and for whom and under what conditions it is most effective (Brown et al., 2008; Spoth et al., 2006). Even more importantly, examination of differential effectiveness can reveal whether or not the program fails to produce change or if it leads to iatrogenic effects for certain individuals (Gottfredson & Wilson, 2003; Kumpfer, Smith, & Summerhays, 2008). If the program is not equally effective across individuals or groups, the results could be used to suggest changes to the program's content, methods, or intensity (Dawson-McClure, Sandler, Wolchik, & Millsap, 2004; Kraemer, Wilson, Fairburn, & Agras, 2002; Spoth et al., 2006).

Differential Effectiveness of Prevention Programs Across Subgroups of the Population

Despite recent emphasis of the need to examine differential effectiveness of preventive interventions, relatively few evaluations have done so, and there is little consensus regarding the degree to which the effects of universal interventions on reducing youth problem behaviors are expected to generalize across different subgroups of the population. Elliott and Mihalic (2004) have argued that universal programs work equally well for individuals of different backgrounds. Others have found that program effectiveness varies, especially when comparing at-risk youth with more universal populations; although evidence is mixed as to whether universal programs tend to be more or less effective for high-risk versus low-risk youth. Some investigations have found compensatory program effects, in which programs have been shown to be more effective for higher risk youth, and others have found leveraging effects, with stronger effects on lower risk youth (Spoth et al., 2006). Compensatory effects have been demonstrated for a range of community, school, and family programs (Allen & Philliber, 2001; Beach et al., 2009; Dawson-McClure et al., 2004; Gardner et al., 2009; Olds et al., 1997; Spoth et al., 2007; Stoolmiller, Eddy, & Reid, 2000; Tolan, Gorman-Smith, & Henry, 2004). These evaluations have found significant program effects in both high- and low-risk groups, but stronger effects in the higher risk group. In other cases, programs have been shown to work only in the highest risk groups (Bierman et al., 2007; Kellam et al., 2008; Khoo, 2001). Fewer studies report a leveraging effect with stronger program impact in low-risk groups, e.g., among nonusers of drugs at pretest (Eisen, Zellman, Massett, & Murray, 2002; Perry et al., 2002; Perry et al., 1996). It is worth noting that some drug prevention programs found iatrogenic effects where drug use among some students increased as a result of the intervention; sometimes among high-risk youths, e.g., students who already smoked (Ellickson & Bell, 1990), and in another case among lower risk youth, such as nonusers of drugs (Sloboda et al., 2009). Still other evaluations have found that program effectiveness did not significantly differ across subgroups (Botvin, Mihalic, & Grotpeter, 1998; Komro et al., 2008; Spoth et al., 2006).

Studies of the differential effectiveness of prevention programs have defined risk-related subgroups in different ways depending on the theoretical background and aims of the program (Van Horn et al., 2008). Interventions targeting adolescent delinquency and substance use often compare the effectiveness of the program among youth who had already initiated the behavior at the outset of the study and those who had not yet initiated (Allen & Philliber, 2001; Eisen et al., 2002; Ellickson & Bell, 1990). Other studies have examined differences in the impact of a prevention program based on baseline levels of specific risk factors for the targeted behavior, e.g., exposure to delinquent peers (Brody, Kogan, Chen, & Murry, 2008), family structure (Komro et al., 2008), family risk for social-emotional problems (Guyll, Spoth, Chao, Wickrama, & Russell, 2004), or cumulative risk defined broadly across several domains (Dawson-McClure et al., 2004).

Until recently, few evaluations of universal substance use and delinquency prevention programs have evaluated the degree to which results varied by gender (Blake, Amaro, Schwartz, & Flinchbaugh, 2001; Kumpfer et al., 2008; Zahn, Hawkins, Chiancone, & Whitworth, 2008). While boys still tend to report more substance use and delinquency than do girls (Chesney-Lind, 1997; Elliott & Mihalic, 2004; Snyder, 2008), girls appear to be catching up to boys in both their self-reported rates of drug use and in their representation as offenders in official arrest statistics (Kumpfer et al., 2008; Snyder, 2008; Steffensmeier, Schwartz, Zhong, & Ackerman, 2005). These increases have prompted agencies such as the Office of Juvenile Justice and Delinquency Prevention to call for more information regarding what works to reduce female offending and to better understand gender differences in the causes and prevention of delinquency and drug use (Zahn et al., 2008).

It is also important to examine potential gender differences in program effects because etiological studies have sometimes found that girls and boys report different levels of risk and protection in their peer groups, families, and neighborhoods, and that risk and protective factors are differentially associated with problem behaviors by gender (Amaro, Blake, Schwartz, & Flinchbaugh, 2001; Fagan, Van Horn, Hawkins, & Arthur, 2007; Foley, 2008). While these findings suggest that programs may vary in the degree to which they prevent substance use and delinquency, it is unclear whether or not programs benefit girls or boys more. Because boys are more likely to be involved in problem behaviors, intervention programs and policies may be more effective for boys; but if programs are more targeted towards risk or protective factors that are more salient for girls, then females may receive greater benefits from participation.

The few studies that have examined program effectiveness for males and females have produced mixed evidence. Some substance use and delinquency prevention programs have been demonstrated to be equally effective for boys and girls (Kulis, Yabiku, Marsiglia, Nieri, & Crossman, 2007; Zahn et al., 2008). Others have shown stronger effects for girls (Blake et al., 2001) or effects that are significant only for girls but not boys (Longshore, Ellickson, McCaffrey, & St. Clair, 2007; Mason et al., 2009). Still others have found that reductions in aggression and drug use were significant only for boys (Kellam et al., 2008; Kellam et al., 1998; Yin & Ware, 2000).

As Elliott and Mihalic (2004) and others (Flay et al., 2005; Kumpfer et al., 2008; Spoth et al., 2006) emphasize, prevention science researchers must evaluate the generalizability of their preventive interventions because differential effectiveness of programs has been demonstrated in past research. Analytically, it is not sufficient, however, to demonstrate that program effects found to be significant in the full sample are also significant for specific subgroups, such as boys and girls or baseline users and nonusers of drugs (Chou et al., 1998; Kulis, Nieri, Yabiku, Stromwall, & Marsiglia, 2007; Sloboda et al., 2009). Evaluations need to test whether between-group differences in program effects are statistically significant (Brown et al., 2008; Matthews & Altman, 1996).

Using interaction analyses, the present study examined whether the previously established significant effects of the universal, community-based Communities That Care prevention system on the prevalence of substance use and the variety of delinquent behaviors (as reported in Hawkins et al., 2009) held equally for boys and girls and in risk-related subgroups defined by early substance use, early delinquency, and high levels of community-targeted risk at baseline. Because several other evaluations have found that programs that did not have an overall effect worked only in a subgroup of the population or that an intervention had iatrogenic effects, but only in a specific subgroup of the population, the present study also examined differential effectiveness of CTC for previously examined outcomes where an overall impact of CTC was not found.

Communities That Care

Communities That Care (CTC -- Hawkins & Catalano, 2002; Hawkins, Catalano, & Arthur, 2002) is an operating system that mobilizes and empowers community stakeholders to collaborate on the development and implementation of a science-based community prevention system aimed at reducing risk, enhancing protection, and reducing the prevalence of adolescent health and behavior problems. The CTC system is expected to produce community-wide changes in prevention service system characteristics, including increased adoption of a science-based approach to prevention, increased collaboration among service providers, and increased use of tested, effective preventive interventions that address risk and protective factors prioritized by the community. These changes in prevention services are expected to produce changes in youths' exposure to the risk and protective factors targeted by the preventive interventions. These changes in risk and protective factors are expected, in turn, to produce changes in adolescent drug use and delinquent behaviors. CTC is installed in communities through a series of six training events delivered over the course of 6 to 12 months by certified CTC trainers. All CTC training materials are distributed by the Center for Substance Abuse Prevention of the federal Substance Abuse and Mental Health Services Administration and are available on the internet at http://preventionplatform.samhsa.gov.

Although communities using CTC select and implement different preventive programs to address their specific profiles of risk, the CTC system itself is a universal prevention system that is not intended to focus on specific populations; rather, communities select programs that reduce risk factors and promote protective factors known to predict a range of health-risking behaviors, including drug use and delinquency. The effects of the CTC prevention system are not expected to differentially affect particular individuals or groups of individuals. However, given that past research has found universal preventive interventions to be effective at times only for certain subgroups of the population, this study examined whether the intervention effects on youth substance use and delinquency found in the first randomized trial of CTC (Hawkins et al., 2009) were moderated by gender and baseline risk for substance use and delinquency.

Methods

The Community Youth Development Study (CYDS -- Hawkins et al., 2008b) is the first community-randomized trial of CTC. It was designed to investigate whether CTC reduces levels of risk, increases levels of protection, and reduces the incidence and prevalence of tobacco, alcohol, and other drug use and delinquency in early adolescence in communities. Communities in the CYDS were selected from 41 communities in the states of Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington that participated in an earlier naturalistic study of the diffusion of science-based prevention strategies, called the Diffusion Project (Arthur, Glaser, & Hawkins, 2005). The drug abuse prevention agencies in these states identified 20 of these communities that the agencies thought were trying to implement risk- and protection-focused prevention services. These 20 communities were then matched, within state, on population size, racial and ethnic diversity, economic indicators, and crime rates to comparison communities that were not thought to be using a risk and protection-focused approach, and the community pairs were recruited to participate in the Diffusion study. Following the prevention science framework for community prevention planning and Rogers' (1995) stages of innovation diffusion, each community's stage of adoption of a science-based approach to prevention was assessed. In Stage 0, the community showed little or no awareness of prevention science concepts and their relevance to prevention programming. At stage 1, the community showed awareness of prevention science terminology and concepts including risk and protective factors, but did not use these concepts to guide prevention programming. In Stage 2, the community had adopted a science-based approach in planning prevention initiatives, but did not collect epidemiologic data to guide the selection of prevention activities in the community or use tested and effective preventive interventions. A community at Stage 3 collected epidemiologic risk and protective factor data but did not use tested and effective preventive interventions. In Stage 4, the community used tested and effective preventive interventions to address prioritized risk and protective factors based on epidemiologic data collected in the community. Finally, if a community had reached Stage 5, it used tested and effective preventive interventions and engaged in ongoing assessments to monitor implementation and effects of the interventions (Arthur et al., 2005). Data for measuring community adoption of science-based prevention were obtained from telephone interviews conducted with 15 community leaders in each community across multiple sectors (including human services, schools, law enforcement, civic organizations, youth recreation, juvenile justice, health agencies, businesses, media, and religious organizations). In spite of states' initial assessments of these communities, neither community in 13 of the 20 pairs of communities was advanced in the use of science-based prevention to the point of Stage 4 where they selected and used tested, effective preventive interventions to address prioritized community risks during the 5 years of the Diffusion Project (Arthur et al., 2005). These 13 pairs of communities were deemed eligible for inclusion in the CYDS study. Recruitment of communities required securing letters from the superintendent of schools, the mayor or city manager, and the lead law enforcement officer, agreeing to all data collection activities required of the project. Twelve of the 13 pairs of matched communities (24 communities total) met all recruitment criteria and were successfully recruited for the CYDS. One community from within each matched pair was assigned randomly by a coin toss to either the intervention (CTC) or control condition (Hawkins et al., 2008b).

Implementation of the CTC System

CTC training and implementation began in the 12 intervention communities in the summer of 2003. Intervention communities received six CTC trainings delivered over the course of 6 to 12 months by certified CTC trainers. CYDS implementation staff provided technical assistance throughout the study via weekly phone calls, emails, and site visits to CTC communities at least once per year. The first stage of CTC began when community leaders were oriented to the CTC system and identified or created a community coalition of diverse stakeholders to implement CTC. Coalition members were trained to use data from surveys of community students collected every 2 years, beginning in 1998, to prioritize risk factors to be targeted by preventive actions in the community; to choose tested and effective prevention policies and programs that address the community's targeted risk factors; to implement these interventions with fidelity; and to monitor implementation and outcomes of newly installed prevention programs. Because the CYDS was initially funded by a 5-year grant, CTC communities in CYDS were asked to focus their prevention plans on programs for youths aged 10 to 14 years (Grades 5 through 9) and their families and schools so that possible effects on drug use and delinquency could be observed within the grant period.

Based on their unique profile of risk identified by the community-wide CTC student survey data, each CTC community in CYDS prioritized a different set of risk factors to be targeted by preventive programs. Each CTC community selected between 2 and 5 risk factors, for a total of 11 different risk factors across all 12 intervention communities (including community laws and norms favorable towards problem behavior, family management problems, family conflict, parental attitudes favorable to antisocial behavior, student attitudes favorable toward antisocial behavior, academic failure, low commitment to school, rebelliousness, antisocial peers, peer rewards for antisocial behavior, and low perceived risk of drug use among students).

To address their prioritized risk factors, CTC communities in CYDS chose programs from the CTC Prevention Strategies Guide (Substance Abuse and Mental Health Services Administration, 2005), which provides a menu of programs that have been found to be effective in well-controlled trials in preventing tobacco, alcohol, or other drug use or delinquent behavior. The menu also identifies the risk and protective factors addressed by each intervention. Chosen programs included school-based programs (All-Stars, Life Skills Training, Lion's Quest Skills for Adolescence, Project Alert, Olweus Bullying Prevention Program, and Program Development Evaluation Training) as well as community-based, youth-focused programs (Participate and Learn Skills, Big Brothers/Big Sisters, Stay Smart, and academic tutoring), and family-focused programs (Strengthening Families 10-14, Guiding Good Choices, Parents Who Care, Family Matters, and Parenting Wisely) (Fagan, Hanson, Hawkins, & Arthur, 2008b; Quinby et al., 2008). Most programs were universal in nature, designed to be implemented with all students in targeted grades, for example, or for parents of all middle school children in the community, regardless of family problems or youth involvement in problem behaviors. Tutoring programs and the Big Brothers/Big Sisters program, however, were selective interventions and targeted youth with low academic performance and those from single-parent families, respectively. During each of the 2004-2005, 2005-2006, and 2006-2007 school years, community coalitions implemented from one to five of these programs to address their prioritized risk factors, as identified through the student survey data. On average, three programs were implemented per community each year. Programs were implemented by local providers, including teachers for school programs; health and human service workers for community-based, youth-focused, and family-focused programs; and community volunteers for tutoring programs and Big Brothers/Big Sisters.

Previous analyses of CYDS data have found that the CTC system was successfully implemented with fidelity in intervention communities (Fagan, Hanson, Hawkins, & Arthur, 2009; Quinby et al., 2008) and that levels of adoption of science-based prevention and levels of community collaboration were significantly higher in CTC than control communities 1.5 years after initial implementation (Brown, Hawkins, Arthur, Briney, & Abbott, 2007). Prior analyses also found that tested and effective preventive programs were selected and well implemented in the CTC communities (Fagan, Hanson, Hawkins, & Arthur, 2008a). Hypothesized effects of CTC on risk factors targeted by the intervention communities and on the incidence of delinquent behavior among youth were observed 3 years after implementation of CTC (Hawkins et al., 2008a). Four years after implementation of CTC, the incidences of delinquent behavior, alcohol, cigarette, and smokeless tobacco initiation between Grades 5 and 8 were found to be significantly lower in CTC than in control communities. In addition, Grade 8 prevalences of alcohol and smokeless tobacco use in the last 30 days, binge drinking in the past 2 weeks, and the number of different delinquent behaviors committed in the past year were found to be significantly lower in CTC than in control communities (Hawkins et al., 2009).

Student Sample and Data Collection

Data on adolescent drug use and delinquent behavior were obtained from annual surveys of a panel of public school students who were in the fifth grade during the 2003-2004 school year in the 24 CYDS communities. The first wave of data, collected in the spring of 2004, was a pre-intervention baseline assessment. Tested prevention programs were implemented in CTC communities beginning in the summer and fall of 2004. The fourth annual wave of student data was collected in the spring of 2007 when panel students progressing normally were in Grade 8, about 2.67 years after the prevention programs chosen by CTC communities were first implemented.

Grade 6 (Wave 2) data collection included an effort to recruit students who were not surveyed in Grade 5. During Grades 5 and 6, parents of 4,420 students (76.4% of the eligible population) consented to their participation in the study. Final consent rates did not differ significantly by intervention condition. Consent rates were 76.2% for students in intervention communities and 76.7% for students in control communities. Thirteen of the 4,420 consented students were absent during scheduled dates of data collection and were not available for initial surveying. The final active longitudinal panel consisted of 4,407 students (2,194 girls, 2,213 boys; 55% from intervention communities). Students in the longitudinal panel who remained in intervention or control communities for at least one semester were tracked and surveyed at each of the following waves, even if they left the community. Ninety-six percent of students in the longitudinal panel completed the survey in Wave 4 (Grade 8).

Students completed the Youth Development Survey (YDS--Social Development Research Group, 2005-2007) a self-administered, paper-and-pencil questionnaire designed to be completed in a 50-minute classroom period. The YDS is based on the CTC Youth Survey which has been demonstrated to have good reliability and validity (Arthur, Hawkins, Pollard, Catalano, & Baglioni, 2002; Glaser, Van Horn, Arthur, Hawkins, & Catalano, 2005). To ensure confidentiality, identification numbers but no names or other identifying information were included on the surveys. Parents of panel students provided written informed consent for their children's participation in the study. Students read and signed assent statements indicating that they were informed fully of their rights as research participants and agreed to participate in the study. Upon completion of the survey, students received small incentive gifts worth approximately $5 to $8. The University of Washington's Human Subjects Review Committee has approved this protocol. Additional details on recruitment and data collection can be found in Brown et al. (2009) and Hawkins et al. (2009).

Measures

Measures of baseline risk, substance use and delinquency outcomes, and student characteristics were based on data collected with the YDS instrument. Community demographic characteristics were based on data from the National Center for Education Statistics.

Baseline risk

At-risk youth were identified at the baseline assessment according to three criteria: engagement in delinquent behavior, lifetime substance use, and high levels of risk factors targeted by intervention communities. The baseline measure of delinquent behavior was based on student reports of four different delinquent acts (stealing, property damage, shoplifting, and attacking someone) committed in the past year. If students had engaged in any of the four behaviors in the past year, they were coded as 1 (=delinquent), otherwise as 0 (=not delinquent). Dichotomous measures of lifetime alcohol and cigarette use at baseline (1 = use and 0 = no use) were created from student-reported use of both drugs (“Have you ever had more than just a sip or two of beer, wine, or hard liquor [for example, vodka, whiskey, or gin]?” and “Have you ever smoked a cigarette, even just a puff?”). High targeted risk was defined as at least one standard deviation above the sample mean (coded 1, otherwise 0) on the targeted risk factor scale at baseline. The targeted risk factor score was calculated by taking the average of the community-specific set of targeted risk factors in CTC communities. Since control communities did not prioritize and target risk and protective factors using the CTC process, each control community's risk factor score was calculated based on the set of targeted risk factors in its matched CTC community. Prior analyses showed that CTC and control communities had equivalent baseline levels of targeted risk factors (Hawkins et al., 2008a).

Outcome measures

In Grade 8, students reported whether they had engaged in nine different delinquent acts (stealing, property damage, shoplifting, attacking someone, carrying a gun to school, beating up someone, stealing a vehicle, selling drugs, and being arrested) in the past year. By summing across the nine behaviors, a measure of the variety of delinquent acts was constructed ranging from 0 to 9. The prevalences (with any use dichotomized as 1 and no use as 0) of the use of alcohol, cigarettes, smokeless tobacco, and marijuana in the past month and of binge drinking (consuming 5 or more drinks in one drinking occasion) during the past 2 weeks were also measured in Grade 8 (e.g., “On how many occasions (if any) have you had beer, wine, or hard liquor during the past 30 days?”).

Student and community characteristics

Variables measuring student characteristics used as covariates in analyses included: age at time of the Grade 6 survey; gender (coded 1 = male, 0 = female); race/ethnicity (coded 1 = White or Caucasian, 0 = other); whether the student was Hispanic (coded 1 = yes, 0 = no); parental education level (ranging from 1 = grade school or less to 6 = graduate or professional degree); attendance at religious services at baseline (coded 0 = never to 4 = about once a week or more); and rebelliousness at baseline, which consisted of the mean of three items (alpha = .69): I like to see how much I can get away with; I ignore rules that get in my way; and I do the opposite of what people tell me, just to get them mad (coded from 1 = very false to 4 = very true). Variables measuring community demographic characteristics included the total population of students in the community and the percentage of students who received free or reduced price school lunches. Intervention condition was coded 1 for CTC communities and 0 for control communities.

Analysis Sample and Missing Data Procedures

Among the 4,407 students comprising the consented longitudinal panel, 26.5% were recruited in Wave 2 (Grade 6 accretion sample) and consequently did not complete a questionnaire in Wave 1 (Grade 5). Overall, 96.7% of panel students participated in at least three of four waves of data collection. A few students' data (0.7% in Grade 5 and 1.4% in Grade 8) did not meet validity criteria because they reported being honest only “some of the time” or less, having used a fictitious drug included in the survey as a validity screen, or that they had used two of three drugs (marijuana, inhalants, or other drugs) on 40 or more occasions during the past month (Hawkins et al., 2009). If students met one or more of these validity screens in a given year, their data were deemed invalid in that year and were set to missing. Any valid information these students provided in any other year informed the imputation of the missing data (see below) and estimation of the analysis models. The proportion of students in the analysis sample who did not respond to the delinquency and drug use questions was small. Item nonresponse ranged from 0.6% (for smokeless tobacco use in Grade 5) to 2.7% (for specific delinquency items in Grade 8). Missing data were dealt with via multiple imputation (Schafer & Graham, 2002). Using NORM version 2.03 (Schafer, 2000), 40 separate data sets including data from all four waves were imputed separately by intervention condition (Graham, Taylor, Olchowski, & Cumsille, 2006). Imputation models included student and community characteristics, targeted risk factors, drug use and delinquent behavior outcomes, and community membership. Imputed data sets were combined subsequently to include both intervention and control groups for analysis. There was no systematic bias due to differential accretion or differential attrition in control and intervention conditions (analyses not shown). With regard to both accretion and attrition, the methods for imputing missing data used in this study have been shown in simulations by Collins, Schafer, and Kam (2001) and extensions by Graham (2009 – personal communication) to produce estimates of standard errors that differ little from population values.

Data Analyses

Intervention effects on eighth-grade drug use and delinquency and their moderation by baseline risk and gender were assessed using the same models as used in the previous study of the main effects of CTC. The Generalized Linear Mixed Model (GLMM -- Breslow & Clayton, 1993; Liang & Zeger, 1986; Murray, 1998) with the logit link function was used for the dichotomously coded prevalence of drug use outcomes and the Poisson link function for the count-based variety of delinquent behaviors outcome. Random-intercept models were estimated to account for variation within students, among students within communities, and communities within matched pairs of communities. All analyses were adjusted for the student- and community-level covariates (grand-mean centered) described above and were conducted using HLM version 6.0 (Raudenbush, Bryk, Cheong, & Congdon, 2004). Results were averaged across imputed data sets using Rubin's rules (Rubin, 1987). Approximate degrees of freedom across imputations were calculated using the formulas provided by Raudenbush et al. (Raudenbush & Bryk, 2002; Raudenbush et al., 2004).

To examine whether the effect of CTC on the prevalence of drug use and the variety of delinquent acts varied by baseline risk and gender, analyses included the community-level dichotomous indicator of intervention status (0 = control community, 1 = CTC community), the student-level dichotomous variable for baseline risk or gender, and the interaction of intervention status by baseline risk or gender. Analyses of Grade 8 alcohol use, binge drinking, and marijuana use included lifetime alcohol use at baseline as the risk variable; models predicting eighth-grade cigarette and smokeless tobacco use included lifetime cigarette use at baseline as the risk moderator. Because the 24 CYDS communities were matched in 12 pairs before randomization, the significance of the interaction effect was tested using a two-tailed critical t-value with p-1=11 degrees of freedom. Because power to detect interactions is inherently low (Brown et al., 2008; Leon & Heo, 2009), a Type-I error rate of .10 was used to assess the significance of the interaction effect.

Results

Table 1 shows the percent of boys and girls and students in each baseline risk group by intervention condition. The sample was about equally split by gender in both conditions. About a fifth of students in the panel had ever used alcohol at baseline and had engaged in delinquent behaviors in the past year. Less than 10% of students in both conditions reported ever having used cigarettes at baseline. About 15% of students in CTC and control communities had high targeted risk factor scores at baseline. CTC and control communities had equivalent baseline levels of targeted risk factors, delinquency, and lifetime substance use (Hawkins et al., 2008b; Hawkins et al., 2009).

Table 1.

Gender and Baseline Risk Groups by Intervention Condition

	Students in CTC Communities (N = 2,405)		Students in Control Communities (N = 2,002)
	%	N	%	N
Gender
Boys	49.1	1182	51.5	1031
Girls	50.9	1223	48.5	971
Lifetime Alcohol Use
Use	20.2	485	23.2	464
No Use	79.8	1920	76.8	1538
Lifetime Cigarette Use
Use	7.4	178	9.4	187
No Use	92.6	2227	90.6	1815
Past-year Delinquency
Yes	19.8	476	22.8	457
No	80.2	1929	77.2	1545
Targeted Risk Factors
High Risk	14.5	349	14.7	294

Baseline levels of lifetime substance use, delinquency, and targeted risk factor scores were equivalent in CTC and control communities.

Moderation by Baseline Substance Use and Delinquency

Hawkins et al. (2009) had previously reported significant intervention effects on Grade 8 prevalence of 30-day alcohol use (adjusted odds ratio [AOR] = 1.25), past-2-week binge drinking (AOR = 1.40), and 30-day smokeless tobacco use (AOR = 1.79), but not on past-month cigarette or marijuana use. The results of the current analyses found that the intervention effects on substance use did not significantly vary by baseline drug use status. Table 2 shows observed prevalence rates for Grade 8 substance use outcomes by intervention condition and substance use at baseline, AORs from separate GLMMs for each baseline subgroup, and the t-value and significance of the interaction effect estimated in the combined GLMM analyses. The prevalences and AORs illustrate the main effect of baseline substance use on Grade 8 use: students who had ever tried alcohol or cigarettes at baseline had higher rates of substance use in Grade 8 than those who had not used alcohol or cigarettes at baseline; however, in both groups, students in CTC communities had lower rates of current alcohol use, binge drinking, and smokeless tobacco use in eighth grade than students in control communities. None of the tested interaction effects were statistically (p ≤ .10) significant, indicating that intervention effects were similar regardless of students' baseline drug use status.

Table 2.

Observed Eighth-grade Substance Use and Delinquency by Intervention Condition and Outcome at Baseline

Grade 8 Substance Use	No Baseline Alcohol Use				Baseline Alcohol Use				Interactiona
	CTC	Control	AORa	[95% CI]	CTC	Control	AORa	[95% CI]	T-value	p-value
Past 30-day Use
Alcohol	12.8%	16.8%	0.77	[0.58-1.03]	30.8%	36.4%	0.90	[0.61-1.34]	0.84	0.42
Marijuana	3.1%	3.4%	1.04	[0.61-1.75]	10.8%	14.4%	0.75	[0.44-1.29]	-0.54	0.60
Past-2-week Use
Binge Drinking	3.6%	5.9%	0.70	[0.41-1.17]	13.8%	19.2%	0.85	[0.49-1.47]	0.72	0.49
	No Baseline Cigarette Use				Baseline Cigarette Use				Interactiona
	CTC	Control	AORa	[95% CI]	CTC	Control	AORa	[95% CI]	T-value	p-value
Past 30-day Use
Cigarettes	4.8%	5.7%	0.90	[0.62-1.31]	21.8%	29.7%	0.61	[0.31-1.18]	-0.64	0.52
Smokeless Tobacco	1.5%	3.6%	0.44	[0.26-0.76]	10.3%	11.2%	0.98	[0.34-2.83]	1.48	0.17
Grade 8 Delinquency	No Baseline Delinquent Acts				Baseline Delinquent Acts				Interactiona
	CTC	Control	AORa	[95% CI]	CTC	Control	AORa	[95% CI]	T-value	p-value
Past Year
Mean # of Delinquent Acts	0.56	0.85	0.66	[0.53-0.83]	1.67	2.07	0.84	[0.72-0.98]	2.10	0.06

^aBased on GLMMs, adjusted for student age, sex, race/ethnicity, parental education, religious attendance, rebelliousness, student population in the community, and percentage of students in the community receiving free/reduced-price school lunch.

AOR = Adjusted Odds Ratio from subgroup analysis. CI = Confidence Interval for the adjusted odds ratio. Bolded confidence intervals are statistically significant (p < .05).

Although the interaction analyses showed that CTC's impact on reducing substance use in eighth grade did not differ at statistically significant levels between early initiators and those who had not yet tried alcohol or cigarettes at baseline, separate subgroup analyses of baseline cigarette users and nonusers suggested that CTC may have had a somewhat stronger impact on reducing the use of smokeless tobacco among baseline nonsmokers than smokers. Because the prevalence of smokeless tobacco use was less than 1% at baseline, we examined whether the impact of CTC on smokeless tobacco use in eighth grade depended on whether youth had already tried cigarettes at baseline or not. This strategy seemed warranted since lifetime cigarette use at baseline was strongly associated with eighth-grade 30-day smokeless tobacco use (OR = 4.82) and cigarette use (OR = 6.31). Table 2 shows that the prevalence in eighth-grade smokeless tobacco use appeared to be significantly lower among students in CTC communities compared to those in control communities if they had not tried cigarettes at baseline (AOR = .44, p = .01); but this difference was not found among the 8% of students (between 3% - 16% depending on community) who had already smoked at baseline (AOR = .98, p = .97). However, because of the small size of this group, the treatment effect was estimated with much less precision in the baseline smoker group than in the nonsmoker group as the confidence intervals for the AORs indicate. The interaction effect may not have been significant in this case because power to detect it was limited due to restricted variance in the small group of baseline smokers as well as the number of communities in the CYDS. Power calculations showed that the study would have needed 36 communities per condition to detect the observed interaction effect for smokeless tobacco use (b = .75, se = .51) with .80 power and a Type I error rate of .10 (Murray, 1998).

As shown in the last row of Table 2, the interaction between intervention status and baseline delinquency was significant (t = 2.10, p = .06), suggesting that the previously reported significant difference in the eighth-grade variety of delinquent acts (AOR = 1.34) between students in CTC and those in control communities was somewhat greater for those students who had not yet engaged in any delinquent behavior at baseline than for those who had. While the results indicated that the intervention effect was somewhat greater for those who had not initiated delinquency at baseline, it is important to note that the intervention reduced delinquency for both at-risk and low-risk youth.

Moderation by Baseline Targeted Risk

There was no evidence that the effect of CTC on eighth-grade substance use and delinquency differed by baseline levels of targeted risk factors. Table 3 shows that students with higher targeted risk scores at baseline were more likely to use drugs and be involved in a greater number of delinquent behaviors in eighth grade, but there was no significant interaction between intervention condition and baseline levels of targeted risk.

Table 3.

Observed Eighth-grade Substance Use and Delinquency by Intervention Condition and Targeted Risk at Baseline

	Not High Risk				High Risk				Interactiona
	CTC	Control	AORa	[95% CI]	CTC	Control	AORa	[95% CI]	T-value	p-value
Grade 8 Substance Use
Past 30-day Use
Alcohol	14.3%	19.3%	0.79	[0.63-1.00]	28.8%	33.6%	0.92	[0.57-1.49]	0.60	0.56
Marijuana	3.5%	4.2%	1.04	[0.64-1.69]	11.7%	16.4%	0.73	[0.39-1.38]	-0.54	0.60
Past-2-week Use
Binge Drinking	4.5%	7.3%	0.75	[0.50-1.10]	12.9%	18.7%	0.82	[0.44-1.50]	0.33	0.75
Past 30-day Use
Cigarettes	4.9%	5.8%	0.98	[0.67-1.44]	12.8%	20.9%	0.61	[0.33-1.12]	-1.25	0.24
Smokeless Tobacco	1.5%	3.3%	0.49	[0.28-0.87]	6.3%	9.8%	0.68	[0.30-1.55]	0.76	0.46
Grade 8 Delinquency
Past Year
Mean # of Delinquent Acts	0.63	0.96	0.70	[0.60-0.81]	1.67	2.10	0.83	[0.62-1.12]	0.87	0.40

^aBased on GLMMs, adjusted for student age, sex, race/ethnicity, parental education, religious attendance, rebelliousness, student population in the community, and percentage of students in the community receiving free/reduced-price school lunch.

AOR = Adjusted Odds Ratio. CI = Confidence Interval for the adjusted odds ratio. Bolded confidence intervals are statistically significant (p < .05).

Moderation by Gender

Analyses examining the interaction between gender and intervention condition suggest that there were no significant differences in 30-day cigarette use between conditions for either gender, but the effects of CTC on current alcohol use, binge drinking, marijuana use, and smokeless tobacco use were significantly stronger for boys than for girls (Table 4). Subgroup analyses indicated that the differences in current alcohol use, binge drinking, and smokeless tobacco use between students in CTC and control communities were statistically significant only for boys; girls in the intervention communities had lower rates of substance use than girls in control communities, but not at statistically significant levels (odds ratios ranged from .88 to .92).

Table 4.

Observed Eighth-grade Substance Use and Delinquency by Intervention Condition and Gender

	Girls				Boys				Interactiona
	CTC	Control	AORa	[95% CI]	CTC	Control	AORa	[95% CI]	T-value	p-value
Grade 8 Substance Use
Past 30-day Use
Alcohol	19.2%	22.2%	0.91	[0.70-1.19]	13.2%	20.6%	0.69	[0.48-0.98]	-1.79	0.10
Marijuana	5.2%	5.0%	1.22	[0.69-2.13]	4.1%	6.9%	0.66	[0.40-1.09]	-2.18	0.05
Past-2-week Use
Binge Drinking	6.1%	8.3%	0.88	[0.58-1.36]	5.2%	9.7%	0.62	[0.40-0.95]	-1.81	0.10
Past 30-day Use
Cigarettes	7.0%	8.2%	0.92	[0.61-1.39]	5.1%	7.8%	0.71	[0.45-1.11]	-1.13	0.28
Smokeless Tobacco	1.7%	2.2%	0.90	[0.42-1.93]	2.7%	6.3%	0.44	[0.25-0.77]	-1.96	0.08
Grade 8 Delinquency
Past Year
Mean # of Delinquent Acts	0.95	1.34	0.76	[0.58-0.99]	1.67	2.07	0.71	[0.57-0.87]	-0.29	0.77

^aBased on GLMMs, adjusted for baseline level of outcome, student age, sex, race/ethnicity, parental education, religious attendance, rebelliousness, student population in the community, and percentage of students in the community receiving free/reduced-price school lunch.

AOR = Adjusted Odds Ratio from subgroup analysis. CI = Confidence Interval for the adjusted odds ratio. Bolded confidence intervals are statistically significant (p < .05)

While the previously reported analyses of the overall impact of CTC (Hawkins et al., 2009) did not find a significant difference by intervention condition in eighth-grade marijuana use, the present interaction analyses suggest that CTC may have reduced Grade 8 marijuana use for boys, but not for girls. The interaction between intervention condition and gender was statistically significant and subgroup analyses indicated that boys in CTC communities reported a lower prevalence of 30-day marijuana use than boys in control communities (AOR = .66, p = .09), while the prevalence of marijuana use was about the same for girls in both conditions (AOR = 1.22, p = .45).

Table 4 also shows that the effect of CTC on delinquent behavior was universal across gender groups. Boys and girls in CTC communities engaged in significantly fewer delinquent behaviors in eighth grade than boys and girls in control communities (Table 4).

Discussion

Communities That Care is a prevention system designed to prevent adolescent drug use and delinquency community-wide and universally across individuals and subgroups in the community. Consistent with the intent of CTC, the present analyses from the first randomized trial of CTC in 12 matched community pairs found that, for the most part, CTC reduced students' substance use and delinquency equally across genders and risk-related subgroups, with two exceptions: the effect of CTC on reducing adolescent substance use was somewhat stronger for boys than girls and the impact of CTC on reducing delinquency was stronger for students who were nondelinquent at baseline. It is noteworthy, however, that both boys and girls in CTC communities reported significantly fewer delinquent behaviors in eighth grade than boys and girls in control communities; and that both early initiators and those who had not yet tried alcohol and cigarettes at baseline in CTC communities reported lower prevalences of binge drinking in the past two weeks and alcohol and smokeless tobacco use in the past month when they were in eighth grade compared to early initiators and non-users at baseline in control communities. Further, the significant effect of CTC on both substance use and delinquency did not differ by baseline levels of community-targeted risk.

The somewhat greater effectiveness of CTC to reduce boys' substance use cannot be attributed to higher prevalences of drug use among boys compared to girls. Table 4 clearly shows that, with the exception of smokeless tobacco use, girls were equally, if not more likely, than boys to use alcohol, cigarettes, and marijuana in Grade 8. Fagan et al. (2007) found that boys in the Diffusion study reported higher levels than girls on most risk factors measured by the CTC Youth Survey and that about half the measured risk factors were more strongly associated with delinquency for boys than girls. It may be that the finding of differential effectiveness of CTC on substance use by gender is due to the fact that boys are more likely to be positively affected by programs that seek to reduce these risk factors.

While boys were more likely than girls to engage in delinquent behaviors, it is interesting that we did not find a significant moderating effect of gender for the impact of CTC on delinquent behaviors in eighth grade. However, CTC did have a greater effect on eighth-grade delinquency among students who had not engaged in any delinquent acts at baseline (by the end of Grade 5) than on those who had already initiated delinquent behavior by that point.

Although the interaction analyses showed that CTC's impact on reducing substance use in eighth grade did not differ at statistically significant levels between early initiators and those who had not yet tried alcohol or cigarettes at baseline, separate subgroup analyses of baseline cigarette users and nonusers suggested that CTC may have had a somewhat stronger impact on reducing the use of smokeless tobacco among baseline nonsmokers than smokers. Although the findings with respect to smokeless tobacco use are tenuous, they are notable because smokeless tobacco use has significant health risks that are similar to smoking, including cancer, cardiovascular disease, and addiction (Nelson et al., 2006). Furthermore, the prevention of smokeless tobacco use is particularly important in small communities, like the ones in the present study, where the prevalence of smokeless tobacco use is generally much higher than in urban areas (Nelson et al., 2006).

Power in interaction analyses is inherently low, and in group-randomized trials like CYDS power depends largely on the number of groups (Brown et al., 2008). It is possible that some subgroup differences in the effect of CTC on substance use and delinquency may not have been detected due to the limited power of the 12 matched pairs in the study. However, a few significant interactions were found for moderate differences in the effect of CTC by gender and baseline delinquency. For example, the gender interaction for alcohol use (b = -.35, se = .19, AOR = .71) indicated that the effect of CTC on reducing the prevalence of 30-day alcohol use was 30% less strong among girls than boys; and the effect of CTC among nondelinquents at baseline (b = .23, se = .11, AOR = 1.25) was about 25% stronger in decreasing the number of delinquent acts than among those who had already engaged in delinquent behaviors at baseline. Despite the risk of limited power for interaction analyses in group-randomized trials, Brown et al. (2008) assert the importance of reporting the effect of an intervention across risk-related subgroups so that the universality of the intervention's effect can be evaluated in subsequent meta-analyses.

Another limitation of this study is that the communities in CYDS are free-standing towns of 50,000 or fewer residents. The study does not include urban or suburban populations, and findings may not generalize to larger communities. Nonetheless, results from the present trial suggest the utility of testing CTC in larger and more complex settings.

Overall, the present analyses showed little evidence for differential effectiveness of CTC across risk-related subgroups. In the few instances where differential effectiveness was found, CTC appeared to have slightly stronger effects in lowering boys' substance use and in inhibiting delinquent behavior among those who had not yet initiated delinquency at baseline.