Sustained Decreases in Risk Exposure and Youth Problem Behaviors After Installation of the Communities That Care Prevention System in a Randomized Trial

Abstract

Objective

To test whether the Communities That Care (CTC) prevention system reduced levels of risk and adolescent problem behaviors community-wide 6 years after installation of CTC and 1 year after study-provided resources ended.

Design

A community-randomized trial.

Setting

24 small towns in 7 states, matched within state, randomly assigned to control or intervention condition in 2003.

Participants

A panel of 4407 fifth-grade students was surveyed annually through tenth grade from 2004-2009.

Intervention

A coalition of community stakeholders received training and technical assistance to install CTC, used epidemiologic data to identify elevated risk factors and depressed protective factors in the community, and implemented programs to address their community’s elevated risks from a menu of tested and effective programs for youths aged 10 to 14, their families, and schools.

Outcome Measures

Levels of risk and incidence and prevalence of tobacco, alcohol, and other drug use, delinquency, and violent behavior by grade 10.

Results

Mean levels of targeted risks increased less rapidly between grades 5 and 10 in CTC than in control communities and were significantly lower in CTC than control communities in grade 10. The incidence of delinquent behavior, alcohol use, and cigarette use, and the prevalence of current cigarette use and past-year delinquent and violent behavior were significantly lower in CTC than in control communities in grade 10.

Conclusions

Using the CTC system can produce enduring reductions in community-wide levels of risk factors and problem behaviors among adolescents beyond the years of supported implementation, potentially contributing to long-term public health benefits.

INTRODUCTION

Smoking, drinking, and violence are among the leading preventable causes of death in the United States and have large costs to society.^1-5 The prevention of delinquency, violence, and the use of alcohol, tobacco, and other drugs during early adolescence are important public health priorities because involvement in these behaviors often begins in childhood and adolescence,^6-8 and early initiation of these behaviors is associated with greater risk for later serious health and behavior problems.^9-15

Advances in prevention science over the past 2 decades have identified common risk factors for adolescent substance use and delinquent and violent behavior and have produced a growing list of tested and effective preventive interventions,^{16, 17} yet these advances are not systematically used by local decision makers.^18-20 Widespread installation of tested and effective preventive interventions could have significant benefits to public health.^{6, 21, 22} Systems for increasing use of tested and effective prevention programs in communities are needed. Communities That Care (CTC)²³ is a system for guiding communities to choose, install, and monitor tested and effective preventive interventions to address elevated risks and suppressed protective factors affecting youth.

CTC mobilizes community stakeholders to collaborate on the development and implementation of a science-based community prevention system. The CTC system is expected to produce community-wide changes in prevention system functioning, including increased adoption of a science-based approach to prevention and increased use of tested, effective preventive interventions that address risk and protective factors prioritized by the community. These changes in prevention system functioning are expected to produce community-wide changes in youths’ exposure to targeted risk and protective factors, which, in turn, result in decreases in adolescent substance use and delinquent behaviors. The CTC theory of change suggests that it take 2 to 5 years of implementing tested, effective programs for community-level impact on risk and protective factors, and 4 to 10 years for community-level impact on adolescent substance use, delinquency, and violence to be observed.²⁴

Results from the first randomized trial of CTC^{25, 26} found support for CTC’s theory of change,^27-31 and demonstrated that 4 years after the initial implementation of CTC the incidences of delinquent behavior, alcohol use, cigarette use, and smokeless tobacco use were significantly lower in CTC than in control communities by the end of eighth grade in a panel of students followed from fifth grade. In addition, grade 8 prevalence rates of alcohol and smokeless tobacco use in the past 30 days, binge drinking in the past 2 weeks, and the number of different delinquent behaviors committed in the past year were significantly lower in CTC than in control communities.³²

CTC has been placed in the public domain by the Substance Abuse and Mental Health Services Administration and is available at www.communitiesthatcare.net. Given the potential for widespread dissemination of the CTC system, it is important to understand the degree to which installation of CTC during the initial 5-year efficacy trial led to sustained differences in outcomes between CTC and control communities beyond the end of study-provided resources that supported the installation of CTC. ^33-36 The present study tested CTC’s effects on youth problem behaviors one year after study-provided resources ended, i.e., 6 years after initial implementation of CTC and 4.67 years after CTC communities began implementing prevention programs selected through the CTC process.

METHODS

The Community Youth Development Study (CYDS)²⁶ is the first community-randomized trial of CTC. Twenty-four communities in the states of Colorado, Illinois, Kansas, Maine, Oregon, Utah, and Washington were matched in pairs within state, on population size, racial and ethnic diversity, economic indicators, and crime rates. One community from within each matched pair was assigned randomly by a coin toss to either the intervention (CTC) or control condition.²⁶ CYDS communities are small- to moderate-sized incorporated towns with their own governmental, educational, and law enforcement structures ranging in population from 1500 to 50,000 residents.

Beginning in the summer of 2003, intervention communities received 6 CTC trainings delivered over the course of 6 to 12 months by certified CTC trainers. CTC coalition members were trained to use data from surveys of public school students in the community to prioritize risk factors to be targeted by tested and effective preventive actions in the community.^{38, 39} For the CYDS, CTC communities were asked to focus their prevention plans on programs for youths aged 10 to 14 years (grades 5 through 9) and their families and schools so that possible effects on drug use and delinquency could be observed within the initial 5-year grant period. Starting with the 2004-2005 school year and annually thereafter, community coalitions implemented between 1 and 5 preventive programs to address their prioritized risk factors. CYDS implementation staff provided technical assistance throughout the 5-year efficacy trial (2003-2008) via weekly phone calls, emails, and site visits to CTC communities at least once per year. Technical assistance ended after the fifth year of the study. Control communities received data from the CTC Youth Surveys administered in their schools every 2 years, but no other training or technical assistance from the study.

Sample and Data Collection

Data were drawn from a longitudinal panel of public school students in the 24 CYDS communities followed annually from grade 5 through grade 10 (N = 4407; 50% Male; 67% White; 20% Hispanic).²⁵ All students in fifth-grade classrooms during the 2003-2004 school year in these schools were eligible for participation in the CYDS. The first wave of data collection (fifth grade, 2004) was a pre-intervention baseline assessment. Recruitment continued in Wave 2 (grade 6) to increase the overall participation rate. This strategy resulted in a total of 4,420 students whose parents consented to their participation in the study (76.4% of the eligible population).The sixth annual wave of data was collected in the spring of 2009 when students were in grade 10, approximately 4.67 years after the prevention programs chosen by CTC communities were first implemented. At this point, CTC communities had not received any technical assistance for one year.

The active longitudinal panel consists of 4407 students who completed a Wave 1 or Wave 2 survey. Students in the longitudinal panel who remained in the intervention or control communities for at least one semester were tracked and surveyed annually, even if they left the community.²⁵ Ninety-four percent of students in the longitudinal panel completed the survey in Wave 6 (10th grade). Figure 1 shows the flow of communities and students through the study.

Figure 1.

Flow of communities and participants in the randomized trial.

Students completed the Youth Development Survey,⁴⁰ a self-administered paper-and- pencil questionnaire designed to be completed in a class period. Identification numbers but no names or other identifying information were included on the surveys. The University of Washington Human Subjects Review Committee approved this protocol.

Measures

Targeted Risk Factors

Based on anonymous baseline surveys of all assenting sixth- and eighth-grade students in each CTC community, CTC coalitions prioritized between 2 to 5 risk factors that were elevated in their community.^{31, 41} An average targeted risk factor score was calculated by standardizing and averaging the community-specific set of targeted risk factors in CTC communities. Because control communities did not prioritize and target risk and protective factors using the CTC process, each control community’s average risk factor score was calculated using the set of targeted risk factors identified in its matched CTC community.

Substance Use

Students self-reported the incidence of first use of alcohol, cigarettes, smokeless tobacco, marijuana, inhalants, and prescription drugs not prescribed by a doctor between grades 5 and 10. The prevalence of binge drinking during the last 2 weeks and use of alcohol, cigarettes, smokeless tobacco, marijuana, and inhalants in the last month were measured in grades 5 and 10. In grade 10, the prevalence rates of illicit use of prescription drugs and other illicit drugs (i.e., psychedelics, MDMA [Ecstasy], stimulants, and cocaine or crack) in the past month also were assessed.

Delinquent and Violent Behavior

The incidence of delinquent behavior was operationalized as the first self-reported occurrence of any of 4 delinquent acts (stealing, property damage, shoplifting, or attacking someone with intent to harm) between grades 5 and 10. Questions about more serious delinquent behavior (carrying a gun to school, beating someone up, stealing a vehicle, selling drugs, or being arrested) were added to the survey over time as developmentally appropriate. A measure of the variety of delinquent acts committed in the past year was calculated in the fifth grade (range from 0 to 4) and in the 10th grade (range from 0 to 9). A subset of the delinquency items was used to create a measure of violent behavior in the fifth grade (attacking someone with intent to harm; range 0 to 1) and 10^th grade (attacking someone with intent to harm, carrying a gun to school, beating somebody up; range from 0 to 3). There were no statistically significant differences in levels of average targeted risk factors or the prevalence or mean number of delinquent and violent acts by intervention status at baseline.^{25, 31}

Student and Community Characteristics

Characteristics of students and communities were used as covariates in all analyses. Student-level covariates included age at time of the grade 6 survey, race, Hispanic ethnicity, parental education, attendance at religious services during grade 5 (1=never to 4=about once a week or more), and rebelliousness in grade 5 (mean of 3 items; α = .69). Community-level covariates included total population of students in the community and the percentage of students eligible for free or reduced-price school lunch.

Analysis Sample and Missing Data Procedures

Of the 4407 students in the longitudinal panel, 26.5% were recruited at Wave 2 (in grade 6) and thus did not complete a questionnaire in Wave 1 (grade 5). Overall, 93.5% of the students participated in at least 5 of the 6 waves of data collection. There was no systematic bias from differential accretion or differential attrition in control and intervention conditions.²⁵ Students were excluded from the analysis if they reported being honest “some of the time” or less on the survey, reported having used a fictitious drug included in the survey as a validity screen, or reported that they had used 2 of 3 drugs (marijuana, inhalants, or other drugs) on 40 occasions or more during the past month.³² Based on these validity criteria, 66 students were excluded from analyses in grade 10, resulting in valid data from 4069 students in 10^th grade.

Item nonresponse was small (< 1%). Missing data were imputed using multiple imputations.⁴² Using NORM version 2.03,⁴³ 40 separate datasets including data from all 6 waves were imputed separately by intervention condition.⁴⁴ Analyses were conducted within each imputed dataset and results were averaged across all datasets using Rubin’s rules.⁴⁵

Analyses

Differential change in levels of average targeted risk by intervention condition from grade 5 to grade 10 was assessed using 3-level latent growth models^{46, 47} to account for nesting of time within students and students within communities. Characteristics of students and communities, and 11 dummy variables indicating the matching of communities into pairs, were included as model covariates. Analyses were conducted using HLM version 6.08.⁴⁸

Multilevel discrete time survival analysis^{49, 50} was used to examine the effect of CTC on the incidence of drug use and delinquency between grades 5 and 10. The risk of initiating drug use and delinquent behavior was assessed for students who had not yet initiated these behaviors before the grade 5 survey. To test if the effect of the intervention on incidence was constant across time, we included interaction effects between intervention condition and time. All analyses included student and community covariates and were analyzed using MLwiN version 2.02.⁵¹

The effect of CTC on grade 10 prevalence rates of substance use, delinquency, and violence was assessed using the Generalized Linear Mixed Model^{52, 53} and analyzed using HLM version 6.08⁴⁸ using a logit link transformation for the Bernoulli distributed drug use and delinquency outcomes and using a log link function for the Poisson distributed measures of the variety of delinquent and violent acts. In addition to student and community characteristics, the respective pre-intervention drug use, delinquency, or violence measure was included as a baseline covariate.

Intervention effects were estimated as the mean differences in adjusted community-level slopes, means, and rates between intervention and control communities as tested against the average variation among intervention condition-specific adjusted community-level slopes, means, and rates. Degrees of freedom for intervention effects were equal to the number of community-matched pairs (12) minus the number of community-level covariates, minus 1. Statistically significant effects were assessed at a .05 Type I error rate (2-tailed). To be consistent with prior reports of findings from this study,³² we report adjusted odds ratios (AOR), which were almost identical to corrected risk ratios.⁵⁴

RESULTS

Targeted Risk

Unadjusted mean levels of the average targeted risk factor score from grade 5 to grade 10 by intervention condition are shown in Figure 2. The latent growth model indicated that the increase in targeted risk between grades 5 and 10 was significantly smaller in CTC than in control communities (t (11) = −2.43, p < .05). The standardized effect size corresponding to this intervention effect was δ = −1.76 standard deviations.⁵⁵ Mean levels of targeted risk were equivalent in CTC and control communities at baseline in grade 5 (t (9) = 1.04, p > .05), but were significantly lower in CTC than in control communities at grade 10 (t (9) = −2.54, p < .05; Cohen’s d = −.12).

Figure 2.

Unadjusted average targeted risk factor score by intervention condition.

Incidence of Drug Use and Delinquency

Results from the multilevel discrete time survival analysis indicated a significant effect of CTC in reducing the incidence of the use of alcohol and cigarettes by 10th grade among students who had not yet initiated use by fifth grade. Figure 3 shows the unadjusted cumulative initiation rates for each drug by intervention condition. The AOR for the effect of CTC on alcohol use incidence in 10th grade was 0.62 (95% CI: 0.41, 0.94; t (11) = −2.53, p = .028), indicating that students in CTC communities had 38% lower odds of initiating the use of alcohol in grade 10 than students in control communities. The odds of initiating cigarette use were significantly different between CTC and control communities in 10th grade, with an AOR of 0.54 (95% CI: 0.36, 0.81; t (11) = −3.36, p = .006), suggesting that students in CTC communities had almost half the odds of beginning to smoke in 10th grade as students in control communities. Significant differences by intervention status in the incidence of smokeless tobacco, marijuana, inhalant, or prescription drug use were not observed by the spring of 10th grade (see Figure 3).

Figure 3.

Unadjusted cumulative initiation of substance use by intervention condition.

Figure 4 shows unadjusted cumulative initiation rates of delinquent behavior by intervention condition. Results indicated a significant effect of CTC on the incidence of delinquent behavior between grades 6 and 10. The effect of CTC on the incidence of delinquent behavior did not vary across time. The AOR for initiating any delinquent act between grades 6 and 10 was 0.79 (95% CI: 0.64; 0.98, t (9) = −2.52, p = .033), indicating that the odds of initiating delinquent behavior by 10th grade were 21% lower for students in CTC communities than students in control communities.

Figure 4.

Unadjusted cumulative initiation of delinquent behavior by intervention condition.

Prevalence of Drug Use

Unadjusted prevalence rates and adjusted odds ratios of current drug use in the 10th grade in CTC and control communities are presented in Table 1. Results from the multivariate analyses indicated significantly lower prevalence rates in the 10th grade in CTC communities compared with control communities for past-month cigarette use (t (9) = −2.38; p = .04; AOR = .79). The odds of smoking cigarettes in the past month were 21% lower for students in CTC communities than students in control communities. Tenth graders in control and CTC communities did not differ significantly in rates of binge drinking in the past 2 weeks, or in past-month alcohol, smokeless tobacco, marijuana, inhalant, prescription, or other illicit drug use.

Table 1.

Unadjusted Prevalence Rates of Current Drug Use and Delinquency in Grade 10 and Adjusted Odds Ratios Comparing Control and CTC Communities

	Control Communities	CTC Communities	Adjusted Odds Ratios (95% CI)a
Drug use, %
Last 30 days
Alcohol	29.5	28.7	1.10 (.82, 1.47)
Cigarettes	16.3	13.4	.79 (.64, .99)b
Smokeless tobacco	7.5	6.3	.85 (.63, 1.15)
Inhalants	2.0	3.1	1.50 (.88, 2.58)
Marijuana	15.0	14.5	.99 (.66, 1.49)
Prescription drugs	4.8	5.6	1.15 (.81, 1.65)
Other illicit drugs	5.5	6.5	1.25 (.90, 1.73)
Last 2 weeks
Binge drinking	13.6	11.0	.89 (.67, 1.19)
Delinquent behavior
Last year
No. delinquent behaviors (mean)	1.1	0.9	.89 (.77, 1.03)
Any delinquency (%)	42.6	36.1	.83 (.69, .99)b
Violent behavior
Last year
No. violent behaviors (mean)	0.3	0.2	.84 (.66, 1.05)
Any violence (%)	17.7	13.2	.75 (.58, .97)b

^aOdds ratios are adjusted for grade 5 prevalence, student age, sex, race, ethnicity, parental education, grade 5 religious attendance, grade 5 rebelliousness, student population of the community, and percentage of students in the community receiving free or reduced-price school lunch. Alcohol use in the last 30 days in grade 5 was used to adjust analyses of 10th-grade marijuana, prescription drug, and other illicit drug use because of very low baseline use of these drugs.

^bp < .05

Prevalence and Variety of Delinquent and Violent Behaviors

Table 1 shows that 10th-grade students in CTC communities had 17% lower odds of reporting any delinquent behavior in the past year (t (9) = −2.33; p = .04; AOR = .83), and 25% lower odds of reporting any violent behavior in the past year (t (9) = −2.51; p = .03; AOR = .75) compared to students in control communities. However, the variety of different delinquent or violent acts in which students engaged was not significantly lower among students in CTC communities compared to students in the control communities (delinquency: t (9) = −1.86; p = .10; AOR = .89; violence: t (9) = −1.75; p = .10; AOR = .84).

DISCUSSION AND CONCLUSIONS

The CTC prevention system previously has been found to decrease risk factors and youths’ involvement in adolescent substance use and delinquent behavior during a period of study-supported intervention.³² We have conducted analyses of the ratio of economic benefits of these outcomes to the costs of CTC implementation, which indicate that the benefits of CTC exceed its costs; these results are reported in a separate paper. The present study found that community-wide beneficial effects of the CTC prevention system on the incidence of adolescent delinquency, tobacco and alcohol use, and on the prevalence of delinquent behavior, violence, and tobacco use were observed one year beyond supported implementation of CTC. These are the most prevalent adolescent problem behaviors of those measured in this study and have the greatest costs to society. The use of CTC did not eliminate these adolescent problem behaviors in participating communities, but it reduced them significantly. It will be important to compare the economic value of these sustained benefits to the costs of implementing CTC. By the end of grade 10, the use of CTC did not have significant effects on the variety of different delinquent or violent acts and less prevalent adolescent problem behaviors including the use of marijuana or other illegal drugs. It is possible that effects on the use of these drugs may be observed later as their use becomes more prevalent in the adolescent population.

An important characteristic of the CTC system is that it guides coalitions of community stakeholders to assess the prevalence of exposure to empirically identified risk and protective factors among young people in the community and to choose tested and effective preventive interventions to reduce those risk factors that are most prevalent in each community. Each community in this study selected a different set of preventive interventions to address that community’s unique profile of risk and protection. This study found that using the CTC system to diagnose and address each community’s prevention needs in this way had community-wide effects in reducing alcohol and tobacco use and delinquent and violent behavior.

These findings from the randomized trial, coupled with quasi-experimental results from using CTC in Pennsylvania,^{56, 57} indicate that the use of CTC can contribute to long-term community-wide improvements in public health. We have developed a system for training new CTC trainers and technical assistance providers that can be offered to state agencies and other organizations that seek to build capacity to provide CTC to communities. This system ensures that new trainers have the opportunity to provide CTC training to communities under the supervision of certified CTC trainers and to become certified CTC trainers through this process. The availability of this system increases the likelihood that CTC can be disseminated widely and, if disseminated, will have widespread public health benefits.