Preventing Alcohol Use with a Voluntary After School Program for Middle School Students: Results from a Cluster Randomized Controlled Trial of Project CHOICE

Abstract

There are many mandated school-based programs to prevent adolescent alcohol and drug (AOD) use, but few are voluntary and take place outside of class time.

Objectives

This cluster randomized controlled trial evaluates CHOICE, a voluntary after school program for younger adolescents, which reduced both individual- and school-level alcohol use in a previous pilot study.

Methods

We evaluated CHOICE with 9,528 students from 16 middle schools. The sample was 51% female; 54% Hispanic, 17% Asian, 15% white, 9% multiethnic and 3% African American. Fifteen percent of students attended CHOICE. All students completed surveys on alcohol beliefs and use at baseline and 6–7 months later. We conducted intention-to-treat (ITT) school-level analyses and propensity-matched attender analyses.

Results

Lifetime alcohol use in the ITT analysis (i.e., school level) achieved statistical significance, with an odds ratio (OR) of 0.70 and a number needed to treat (NNT) of 14.8. The NNT suggests that in a school where PC was offered, 1 adolescent out of 15 was prevented from initiating alcohol use during this time period. Although not statistically significant (p=.20), results indicate that past month alcohol use was also lower in PC schools (OR = 0.81; NNT = 45). Comparisons of attenders versus matched controls yielded results for lifetime use similar to school-wide effects (OR = 0.74 and NNT = 17.6).

Conclusions

Initial results are promising and suggest that a voluntary after school program that focuses specifically on AOD may be effective in deterring alcohol use among early adolescents; however, further research is needed as program effects were modest.

Early adolescence is often associated with both initiation and escalation of alcohol use (D'Amico, et al., 2005; Ellickson, McCaffrey, Ghosh-Dastidar, & Longshore, 2003; Faden, 2006). School-based programs are one popular way to reach adolescents as these programs offer parents and adolescents a familiar environment that may be less stigmatizing than being referred to off-site services. This can promote “buy-in” and higher utilization of services in the school setting (Wagner, Tubman, & Gil, 2004). Most prevention programs offered in school settings, for example Project ALERT (Ellickson, McCaffrey, et al., 2003) or keepin’ it R.E.A.L. (Gosin, Marsiglia, & Hecht, 2003), are offered during class time and take at least 11 weeks to complete, with some programs requiring up to 40 sessions. These programs are typically part of the classroom curriculum and therefore required, with the majority of youth in the school receiving the program.

In contrast to the many mandated in-class programs for adolescent alcohol and drug (AOD) use, there are few voluntary programs available for youth that specifically target AOD, and even fewer that take place outside of class time (Little & Harris, 2003). Voluntary AOD programs may appeal more to youth because they can choose to attend sessions that interest them. For example, research has shown that providing youth with a choice can be motivational and increase the chance that they get further help (e.g., Baer, Garrett, Beadnell, Wells, & Peterson, 2007), make changes in their AOD use (D'Amico, Miles, Stern, & Meredith, 2008; Monti, et al., 2007; Spirito, et al., 2004), or remain abstinent (D'Amico & Edelen, 2007). Thus, adolescents who choose to attend AOD programs may be more motivated to utilize the information and make changes in their AOD use or remain abstinent (D'Amico, Osilla, & Stern, in press).

Although after-school programs are a good way to provide resources for youth because they do not require class time or class resources, it can be difficult to reach youth this way. For example, studies have shown that when structured after-school activities are available, youth from lower income families and from ethnic minority backgrounds are less likely to participate in these types of programs (Harvard Family Research Project, 2007). In addition, at-risk youth, such as those who have already initiated AOD use, may also be less likely to access these types of services (Sterling, Weisner, Hinman, & Parthasarathy, 2010; Wu, Hoven, Tiet, Kovalenko, & Wicks, 2002). Thus, after-school programs have to account for potential attendance barriers that would not be an issue in a classroom-based program so that youth from a variety of backgrounds feel comfortable attending the program, and so attendance at the program mirrors classroom composition. Perhaps because of these challenges, voluntary school-based AOD intervention programs are relatively rare.

To date, only three studies have evaluated the effects of a stand-alone AOD intervention that is brief, voluntary, and offered after school. Only one of these programs was tested with middle school youth. The Stay SMART program (St. Pierre, Kaltreider, Mark, & Aikin, 1992) involved high school aged youth who participated in Boys and Girls Clubs. Once youth voluntarily enrolled in the program, they were required to attend 9 of the 12 sessions. Thus, initial involvement was voluntary, but subsequent attendance was mandated. Youth (n=161) who continued participation were mainly male (75%) and ethnical diverse: 45% white, 14% Hispanic, and 42% African American. The youth who continued participation reported less involvement with marijuana compared to a control group (St. Pierre, et al., 1992). The second program involved a voluntary high school intervention targeting AOD self-change efforts (Brown, 2001). Participants were 45% female and mainly white (61%). Results based on a one-semester follow-up indicated that the heaviest drinking students who attended at least one session reported more attempts to cut down or quit drinking than non-attending heavy drinking adolescents. Results were not reported for actual rates of drinking (Brown, Anderson, Schulte, Sintov, & Frissell, 2005).

The third program, CHOICE, focused on middle school youth. CHOICE includes components utilized in other successful programs, such as enhancing protective factors and reducing risk factors (e.g., Henggeler, 1998; Morehouse and Tobler, 2000; Sussman et al., 1995), targeting multiple substances (e.g., D'Amico and Fromme, 2002; Ellickson et al., 2003b; Sussman et al., 2002), utilizing interactive techniques that allow active involvement in learning about problems from substance use, such as discussion of normative feedback or engaging in role plays (D'Amico & Fromme, 2002; Ellickson, McCaffrey, et al., 2003; Hansen & Graham, 1991), reinforcing skills (Botvin, 2000; Brown, 2001), and providing information in a nonjudgmental and nonconfrontational manner (Barnett, Monti, & Wood, 2001; Borsari & Carey, 2000). A pilot evaluation of CHOICE in one school showed that youth who participated in CHOICE were racially/ethnically diverse (D'Amico, et al., 2005). When CHOICE was compared to a control school, the program was associated with both individual- and school-level reductions in alcohol use (D'Amico & Edelen, 2007). In addition, a recent paper (Kilmer, Burgdorf, D’Amico, Tucker, & Miles, 2011) found that the per participant cost for CHOICE (based on average labor and space costs for the country) was lower than SAMHSA’s estimate for the “average effective school-based program” (SAMHSA, 2009).

The current study builds on the smaller pilot evaluation of CHOICE by evaluating this program in a large cluster-randomized controlled trial of 16 schools. A cluster randomized trial was employed because the intervention was offered at the school level. We focus on alcohol use in this paper because alcohol is the most frequently used substance in middle school (D'Amico & Edelen, 2007; Ellickson, McCaffrey, et al., 2003; Johnston, O'Malley, Bachman, & Schulenberg, 2011), particularly with younger students in 6^th and 7^th grade, and we would not have power to detect changes in other substances in this one year time period. Our primary outcome of interest was initiation of alcohol use and past month use of alcohol. Our secondary outcomes of interests were beliefs related to alcohol, such as intentions to drink and resistance self-efficacy. We hypothesized that the program would have two types of effects on alcohol use and alcohol beliefs based on our results from the smaller pilot study: first, we expected an effect of program availability, which we determined by examining differences between treatment and control schools (an intention-to-treat analysis); second, we expected an effect of program participation, which we determined by examining differences between attenders and propensity-matched controls (a treatment-received analysis).

Methods

Participants and Procedure

Sixth, seventh and eighth grade students in 16 middle schools across three school districts in southern California were recruited for the current study, which involved surveys and a voluntary after school program, CHOICE, that targeted substance use. Schools were selected and matched to their nearest neighbor school based on the squared Euclidean distance measure, estimated using publicly available information on ethnic diversity, approximate size and standardized test scores. One school of each pair was then randomized to intervention or control conditions using the MS Excel random number generator. JM, the third author, was responsible for matching and allocation of schools and had no direct contact with or knowledge of the schools other than the characteristics used for matching. Active parental permission was required for the survey and CHOICE. A total of 14,979 students across all sixteen schools received parental consent forms to participate in the study with approximately 7,271 students in the 8 control schools and 7,708 students in the 8 intervention schools; 92% of parents returned this form (n = 13,785). Approximately 71% of parents gave permission for their child to participate in the study (n = 9,828). Ninety-four percent of consented students completed the baseline survey (n = 8932; mean age = 12.6) and 89% completed the follow-up survey (n = 8522), which is higher or comparable to other school-based survey completion rates with this population (Johnson & Hoffmann, 2000; Johnston, O'Malley, Bachman, & Schulenberg, 2009; Kandel, Kiros, Schaffran, & Hu, 2004). There were no statistically significant differences in follow-up rates (intervention schools: 88.8%, control schools: 87.8%; p=0.59). The overall sample was ethnically diverse and comparable to the ethnic composition of the relevant school populations based on published demographic information for the schools. Rates of lifetime and past month substance use in our baseline sample of 7^th and 8^th graders were also comparable to national samples (SAMHSA, 2008). For example, in the 2007 National Survey on Drug Use and Health (SAMHSA, 2008), 28.2% of eighth graders reported lifetime alcohol use, compared with 29.2% in our sample of 8th graders. Finally, our matching of the intervention and control schools shows that the demographic information was similar; a slightly higher proportion of survey completers in control schools were 8th graders (see Table 1).

Table 1.

Demographic information for the baseline sample (N = 8932)

Variable	Control % N=4689	Intervention % N=4243
Gender
Female	50	51
Male	50	49
Race
Non-Hispanic White	14	17
Non-Hispanic African-American	3	4
Hispanic	56	52
Asian	16	17
Other	11	10
Grade
6th grade	31	34
7th grade	32	34
8th grade	37	32
Mother Education
No high school	11	9
High school	14	13
Some college	9	10
College	33	38
Don’t know	34	30
Father Education
No high school	11	9
High school	11	12
Some college	7	8
College	30	35
Don’t know	41	37
Family Living Situation
Mother and Father	57	61
Parent + stepparent	6	5
Female dominated household	22	21
Male dominated household	4	3
Other	11	9

Surveys

Baseline surveys were completed between late September and early October of 2008, with follow up surveys completed between late May and early June 2009. Surveys were administered on a pre-scheduled day during Physical Education (PE) class and took 45 minutes to complete. Trained staff described the survey to students, reviewed confidentiality, and answered questions. Spanish speaking staff members were available to answer student questions; survey booklets were available in Spanish and Korean¹. We used the Magnus method to translate our measures in which one individual translated the measure from English to Spanish/Korean. A second individual who had not seen the English version translated the Spanish/Korean version to English. The two English versions were then compared to each other and differences reconciled through deliberation by the two individuals. Survey responses are protected by a Certificate of Confidentiality from the National Institutes of Health. All materials and procedures are approved by the school districts, individual schools, and the institution’s Internal Review Board.

Project CHOICE

Eight schools were randomly assigned to receive CHOICE. CHOICE was delivered once per week after school on a set day when teens with permission could choose to attend. CHOICE consists of five distinct 30-minute sessions that rotate throughout the entire school year. The program is based on Social Learning Theory (Bandura, 1977), Decision-Making Theory (Kahneman & Tversky, 2000) and Self-Efficacy Theory (Bandura, 1997). For example, Social Learning Theory suggests that people make assumptions about their environment based on their perceptions of the behavior and attitudes of others (Bandura, 1986; Maisto, Carey, & Bradizza, 1999); however, these assumptions may not be accurate and may increase risk behavior. In fact, most teens overestimate the number of peers who drink, smoke cigarettes, or use marijuana, which can increase use of these substances (e.g., D'Amico & Fromme, 2002; Ellickson, Bird, Orlando, Klein, & McCaffrey, 2003; McNeal, Hansen, Harrington, & Giles, 2004). The CHOICE curriculum therefore provides normative information based on Monitoring the Future statistics (Johnston, et al., 2009), an approach which has been used successfully in several interventions with adolescents (Brown, et al., 2005; D'Amico & Fromme, 2002; Ellickson, McCaffrey, et al., 2003; Graham, Marks, & Hansen, 1991; Hansen & Graham, 1991). CHOICE also used a motivational interviewing (MI) approach (Miller & Rollnick, 2002; Rollnick, Miller, & Butler, 2008) to present the program curriculum. CHOICE was pilot tested with approximately 500 students to obtain feedback on the style and content of the program to ensure the program was developmentally relevant for this age group so that youth would choose to attend the program voluntarily. A thorough description of CHOICE is published elsewhere (D'Amico, et al., 2005). Briefly, sessions focus on providing normative feedback on alcohol and marijuana use among middle-school aged youth, challenging unrealistic beliefs about substances, resisting pressure to use substances through the use of role play, discussing potential benefits of both cutting down and stopping use and discussing risky situations and coping strategies (e.g., getting social support, learning how to avoid certain high-risk situations).

Facilitators advertised CHOICE through flyers and brief class presentations. When additional recruitment was needed, they promoted the program during lunch periods by providing information about the program and involving the students in games that offered small promotional items as prizes (e.g., a pencil that said “CHOICE”). Students could choose to attend as many of the five sessions as they wanted. Similar to other after schools programs (e.g., Ferrari, Lekies, & Arnett, 2009; Little, Wimer, & Weiss, 2008), youth who attended the program received a snack at each session. At the eight intervention schools, 15% (n = 703) of consented students voluntarily attended CHOICE during the year. The mean number of sessions attended was 3.0 (SD = 1.76). One-third of participants attended all five sessions and received a $5 gift certificate for completing all five sessions. There were some differences between CHOICE participants and non-participants in this larger trial. For example, African American and multiethnic students were more likely to attend. In addition, past month alcohol users were more likely to initially attend, and marijuana users were more likely to continue attendance (D'Amico, et al., in press).

CHOICE facilitators were eight Bachelor- and Masters-level project staff not affiliated with the schools. The first author (ED), a licensed clinical psychologist, was the lead study trainer. ED is also part of the Motivational Interviewing Network of Trainers (MINT). Training for the current study consisted of a workshop on MI and group and individual practice for each of the five sessions, with a total of approximately 30 hours of training for each of the facilitators over a four week period. To ensure fidelity of intervention delivery after training, facilitators were supervised weekly. In addition, trained observers watched each facilitator provide two different sessions over the year and coded them on adherence to MI and fidelity to the protocol.

Study Measures

Socio-demographic Characteristics

Socio-demographic covariates included age, gender, race/ethnicity, parental education, and current living situation (e.g., living with two parents, one parent, other relatives).

Alcohol Use

Lifetime and past month frequency of alcohol use were assessed with well-established measures from the California Healthy Kids Survey (WestEd, 2008), Monitoring the Future (Johnston, et al., 2009), and Project ALERT (Ellickson, McCaffrey, et al., 2003). Frequency of consumption was assessed by asking “During your life [or past month], how many times [or days] have you tried alcohol?” Respondents were also queried about heavy drinking in the past month by asking how frequently they had drunk “five or more drinks of alcohol in a row, that is, within a couple of hours.” A “drink” was defined as one whole drink of alcohol (not including a few sips of wine for religious purposes). Responses to the three alcohol items were dichotomized to indicate whether the student had engaged in any lifetime drinking, any past month drinking and any past month heavy drinking. The school level intraclass correlations (ICCs) for these measures at follow-up were 0.02 for lifetime alcohol consumption, 0.01 for past month alcohol consumption and 0.01 for heavy drinking.

Beliefs about alcohol

Intention to drink in the next six months was assessed with a single item rated on a 4-point Likert scale from 1 (definitely no) to 4 (definitely yes; ICC = 0.01.)

Resistance self-efficacy (RSE) was assessed using scales developed in Project ALERT (Ellickson, McCaffrey, et al., 2003). Students were asked: “Suppose you are offered alcohol and you do not want to use it. What would you do in these situations: A) your best friend is drinking alcohol; B) you are bored at a party; and C) all your friends at a party are drinking alcohol?” These three items were rated on a scale ranging from 1 = “I would definitely drink” to 4 = “I would definitely not drink” and scores were averaged to develop a single RSE scale (alpha = 0.93). Higher scores indicated higher resistance self-efficacy (ICC = 0.02). The mean resistant self-efficacy score in control schools was 3.57 (95% CIs 3.42, 3.63) and in intervention schools it was 3.63 (95% CIs 3.50, 3.76).

Perceived alcohol use among peers at school (WestEd, 2008) was measured by asking students to estimate how many students in their grade out of 100 (or approximately 3 classrooms of students) drink alcohol at least once a month. Responses were recorded on an 11-point scale where zero or no students out of 100 was coded as “1”, 10 students out of 100 was coded as “2”, and so forth (ICC = 0.02). The mean perceived alcohol use in control schools was 1.98 (95% CIs 1.63, 3.22) and in intervention schools it was 1.76 (95% CIs 1.44, 2.08).

Fidelity and acceptability measures

We used the Motivational Interviewing Treatment Integrity scale (MITI 3.1) (Moyers, Martin, Manuel, Miller, & Ernst, 2010) to measure MI adherence. The MITI comprises both global scores and behavioral counts to assess overall adherence. CHOICE attendees were invited to complete a short, anonymous questionnaire at the end of each session that assessed satisfaction with the session (1=not at all to 5=completely).

Analysis Plan

Data that are not missing completely at random are likely to cause bias in parameter estimates (Allison, 2001). Data were cleaned and logical imputation was carried out to ensure consistency. We used a multiple imputation approach using a sequence of regression models to ensure that parameters were unbiased in the presence of data missing at random (Schafer, 1999; Schafer & Graham, 2002). We used the IVEware software package (Raghunathan, Solenberger, & van Hoewyk, 2002). This package uses a multiple imputation approach similar to SAS PROC MI, but IVEware also allows for the incorporation of clustered data (at the school level) in the imputation process. Thus, we imputed and analyzed 30 data sets separated by 10 iterations (sufficient to achieve stable results). Imputations were carried out separately for males and females and for intervention and control groups (Allison, 2001). Where appropriate, we treated variables as categorical for the purposes of imputation. These 30 imputed datasets were analyzed separately, and results were combined following the formulae provided by Little and Rubin (1987) and implemented in the SAS 9.2 (SAS Institute, 2008) MIAnalyze procedure. This approach gives unbiased results when the data are missing at random or missing completely at random (Schafer & Graham, 2002). We imputed all variables with missing values, with the exception of age, gender and race/ethnicity.

Data were analyzed using two approaches. First, we used an intention-to-treat approach (ITT) in which adolescents were analyzed in the groups to which they were assigned; that is, all respondents in control schools were compared to all respondents in intervention schools, regardless of attendance at CHOICE. Second, we used a propensity matching approach, using twang (Techniques for Weighting and Analysis of Non-equivalent Groups; Ridgeway, McCaffrey, & Morral, 2006) to create a sample of individuals from control schools equivalent to those in intervention schools (so-called Average Treatment for the Treated or ATT weights). twang is a package for the R statistical programming environment (R Development Core Team, 2009) that uses boosted regression to calculate sample weights which balance the two groups on predictors. It is one of the most efficient and valid propensity methods to use with this type of data set (Harder, Stuart, & Anthony, 2010). Fifty-six variables were incorporated as potential covariates for balance (e.g., age, gender, grade, academic performance, parental education, past year and past month substance use). We estimated effects of the intervention for “attenders” versus propensity matched controls.

For continuous and dichotomous outcome variables we used generalized estimating equation (GEE) regression models to account for the clustering of individuals within schools. For the ITT analyses, we estimated unadjusted outcomes in a bivariate analysis without baseline covariates, and we also estimated the outcomes in an adjusted multivariate analysis in which we included the following covariates: baseline level of the variable and current school grade, sex and race.

We calculated effect sizes based on parameter estimates from the regressions; we used the pooled standard deviation at baseline to calculate Cohen’s d for continuous measures and the control group probability of behavior to calculate the number needed to treat (NNT) for dichotomous measures. To explore the effect of attendance at CHOICE, we regressed outcomes against the number of sessions attended, incorporated the same covariates as in the previous analysis and included school as a fixed effect.

Results

Fidelity and Acceptability of Project CHOICE

The mean spirit rating on the MITI across all facilitators was 4 (competent) and MI adherence averaged 93%. Adherence to protocol content across sessions was 90% (SD = 10%). Satisfaction scores were high (1=completely satisfied, 5= not at all satisfied), indicating that students liked the style of the meeting (M = 1.3), thought the facilitator was helpful (M = 1.2) and thought the discussion was helpful (M = 1.3).

Reliability Checks

We examined the data for inconsistent responses. Of those individuals who said that they had not consumed alcohol in the past month at wave 1, 0.3% said that they had engaged in heavy drinking in the past month; at wave 2, the value was 0.4%. Similarly, of those who said that they had not consumed alcohol in their lifetime, 1.6% said that they had consumed alcohol in the past month at wave 1, and 1.1% said that they had consumed in the past month at wave 2. Thus, our reliability checks show that the proportion of inconsistent responses was very low.

Effects of Project Choice

Descriptive statistics for outcome variables are based on the imputed and combined datasets (Table 2). After controlling for the grade difference, there were no significant baseline differences on the measured variables between intervention and control schools. Attenders were matched to controls through propensity weighting, which eliminated differences in covariates.

Table 2.

Descriptive statistics for outcomes at baseline and follow-up, N=9528^*

	Outcome	Control		Intervention
		Mean (SD) / %		Mean (SD) / %
	Time	Baseline	Follow up	Baseline	Follow up
Intervention School vs Control School (ITT)	Lifetime Alcohol Use	19.1%	29.0%	16.7%	22.2%
	Past Month Alcohol Use	8.1%	12.9%	6.8%	9.7%
	Heavy Drinking in Past Month	3.3%	6.1%	3.9%	4.5%
	Perceived Alcohol Use	1.98 (1.80)	2.50 (2.27)	1.76 (1.64)	2.37 (2.16)
	Alcohol Intentions	8.7%	9.4%	6.9%	7.6%
	Resistance Self-Efficacy (Alcohol)	3.57 (0.77)	3.46 (0.82)	3.63 (0.70)	3.48 (0.79)
Attenders vs Propensity Matched Controls	Lifetime Alcohol Use	17.5%	28.3%	16.9%	22.7%
	Past Month Alcohol Use	7.1%	11.8%	6.8%	10.5%
	Heavy Drinking in Past Month	2.2%	5.4%	1.9%	4.4%
	Perceived Alcohol Use	1.69 (1.69)	2.33 (2.14)	1.64 (1.54)	2.39 (2.27)
	Alcohol Intentions	17.7%	18.6%	15.0%	19.0%
	Resistance Self-Efficacy (Alcohol)	3.55 (0.24)	3.47 (0.25)	3.58 (0.80)	3.46 (0.80)

Note: Perceived alcohol use responses were recorded on an 11-point scale where zero or no students out of 100 were coded as “1”, 10 students out of 100 was coded as “2”, and so forth. Alcohol intentions were rated on a 4-point Likert scale from 1 (definitely no) to 4 (definitely yes). RSE items were rated on a scale ranging from 1 = “I would definitely drink” to 4 = “I would definitely not drink.

^*this number differs from the baseline sample of 8932 as this sample includes both wave 1 and wave 2 students.

From baseline to follow up, consumption and intention measures all increased (as expected of this age group); however, rates of consumption were lower among CHOICE attenders compared to a matched sample of controls after accounting for covariates. In particular, lifetime alcohol use increased to 28% among weighted controls, whereas the increase was only to 23% in the attender group. Examination of school-wide effects indicated that rates of consumption and intentions to use alcohol were also significantly lower in CHOICE schools compared to control schools.

Overall, results were substantively equivalent regardless of adjustment for covariates in the ITT analysis, thus we report results for which the covariates were adjusted. Results from the regression models (Table 3) show that most effects are in the predicted direction, with the exception of perceived alcohol use, which is 0.03 units higher (on an 11-point scale) in the intervention group. Lifetime alcohol use in the ITT analysis (i.e., at the school level) achieved statistical significance, with an odds ratio (OR) of 0.70 and a number needed to treat (NNT) of 14.8. The NNT suggests that in a school where CHOICE was offered, 1 adolescent out of 15 was prevented from initiating alcohol use during this time period. Although not statistically significant (p=.20), results indicate that past month alcohol use was also lower in CHOICE schools (OR = 0.81; NNT = 45). Comparisons of attenders versus matched controls yielded results for lifetime use similar to school-wide effects (OR = 0.74 and NNT = 17.6), as shown in the bottom of Table 2. This difference did not reach statistical significance at the conventional p=0.05 level given the reduced sample size for this analysis and the resulting larger standard errors. Statistically significant differences were not found for intentions, perceived alcohol prevalence or RSE at the individual- or school-level.

Table 3.

Parameter Estimates from Generalized Estimating Equation Regression

Intervention School versus Control School
Unadjusted Estimates
Outcome	Estimate	SE	OR	NNT	D
Lifetime Alcohol Use	−0.36**	0.12	0.70	14.7
Past Month Alcohol Use	−0.32+	0.17	0.73	31.3
Heavy Drinking in Past Month	−0.32	0.23	0.78	62.5
Perceived alcohol use	−0.13	0.10			−0.06
Alcohol Intentions	−0.23	0.18	0.79	55.6
Resistance Self-Efficacy (Alcohol)	0.02	0.04			0.02
Estimates Adjusted for Baseline Outcome and Covariates
Outcome	Estimate	SE	OR	NNT	D
Lifetime Alcohol Use	−0.36*	0.16	0.70	14.8
Past Month Alcohol Use	−0.21	0.17	0.81	45.0
Heavy Drinking in Past Month	−0.25	0.30	0.78	78.6
Perceived alcohol use	0.03	0.09			0.02
Alcohol Intentions	−0.08	0.07	0.92	77.2
Resistance Self-Efficacy (Alcohol)	0.02	0.03			0.03
Attenders vs Matched Controls
Outcome	Estimate	SE	OR	NNT	D
Lifetime Alcohol Use	−0.30	0.22	0.74	17.6
Past Month Alcohol Use	−0.13	0.27	0.88	76.5
Heavy Drinking in Past Month	−0.21	0.45	0.81	100.7
Perceived alcohol use	0.09	0.15			0.05
Alcohol Intentions	0.10	0.14	1.11	67.4
Resistance Self-Efficacy (Alcohol)	0.01	0.05			0.02

Note: D = standardized difference between groups; NNT = number needed to treat.

⁺p < 0.10,

^*p < 0.05,

^**p < 0.01.

To explore the effects of CHOICE attendance further, we investigated the association between the number of CHOICE sessions individuals attended and lifetime, past month and heavy drinking, intentions to drink, perceived prevalence and RSE, controlling for covariates as in the prior models. Controlling for baseline levels, youth who attended a greater number of CHOICE sessions also reported higher resistance self-efficacy at follow-up (p<.05). No other effects reached a statistically significant magnitude (at the 0.05 level).

Discussion

This cluster randomized trial of a voluntary alcohol and drug prevention program for middle school youth replicates school-level results found in a pilot trial of CHOICE. We found a school-wide effect on alcohol use for all students at the intervention schools, whether or not they attended CHOICE. Specifically, students at the eight schools who received the CHOICE program were less likely to initiate alcohol use during the academic year compared to students at the eight control schools. In addition, although not statistically significant (possibly due to smaller sample sizes), there was a trend for individuals who attended CHOICE to be less likely to initiate and drink alcohol in the past month compared to a matched control group that did not attend CHOICE. The odds ratio (OR=0.74) and number needed to treat (NNT=17.6) for these individual-level analyses were comparable to those found for school-wide effects (OR=0.70, NNT=14.8), which emphasizes the clinical significance of both the school-level and individual-level findings. That is, only 15–18 people would need to be exposed to this brief, voluntary 30-minute program to see benefits for one individual. In terms of facilitator time for the benefit obtained, this appears to be an efficient and promising approach to reduce alcohol use among youth; however, further research is needed as effects were modest.

Program effectiveness is dependent upon the fidelity of implementation (Wagner, et al., 2004). Findings from the trained observers indicate that facilitators adhered to the CHOICE protocol and consistently used MI during the session. Thus, across all eight schools, facilitators discussed the same content and also provided content to students using motivational interviewing. In addition, student reports indicated that they liked the style of the meeting and that the facilitator and information were helpful.

In contrast to other after-school programs, this voluntary after-school program reached a diverse and at-risk population of youth (D'Amico, et al., in press) who typically do not obtain services (Harvard Family Research Project, 2007; Sterling, et al., 2010; Wu, et al., 2002). In addition, given funding cuts and the limited resources available to schools, CHOICE is an attractive alternative to providing programming to youth in the school setting as the per participant cost of CHOICE is less expensive than many other school-based prevention programs (Kilmer, Burgdorf, D’Amico, Tucker, & Miles, 2011). Thus, this program can offer some significant benefits over other prevention programs for this age group.

A 2009 report from the Afterschool Alliance on a nationally representative sample of youth found that 15% of youth attend after school programs, which includes programs focused on academics, sports, mentoring, etc. (Afterschool Alliance, 2009). In the only other study reporting on a voluntary after school program for middle school youth that actually focused on alcohol and drug use, but was also part of a more comprehensive after school program that included tutoring, academics, and leisure activities, participation averaged 14.5% across five different schools (Gottfredson, Cross, Wilson, Rorie, & Connell, 2010). Thus, the participation rate for CHOICE is the same as other after school programs, which tend to address a variety of behaviors. Furthermore, a 2006 review of the effects of after school programs on student outcomes (Zief, Lauver, & Maynard, 2006) found that only 20% of after school programs targeted middle school youth. Therefore, the current study addresses an important prevention and intervention gap by focusing on this population.

Despite the effects of CHOICE on alcohol use, we did not find individual- or school-level changes in RSE, perceived prevalence of drinking, or intentions to drink. One exception involved the dosage of the intervention: individuals who attended more sessions of CHOICE reported increases in resistance self-efficacy for alcohol, suggesting that students may need to attend more sessions so that they can practice these skills several times and feel confident in using them. It is interesting to note that results from substance use intervention studies examining whether changes occur in these types of mediating variables have been mixed. Some intervention studies report changes in both the hypothesized mediating variables and substance use (D'Amico, et al., 2008; Kelly & Lapworth, 2006; Neighbors, et al., 2010; Orlando, Ellickson, McCaffrey, & Longshore, 2005). Other studies report changes in substance use in the absence of changes in the hypothesized mediating variables (Agostinelli & Grube, 2005; Colby, et al., 2005; Peterson, Baer, Wells, Ginzler, & Garrett, 2006), similar to the present study. Still other studies only report on substance use outcomes and do not discuss potential mediating mechanisms (Baer, et al., 2007; Hollis, et al., 2005; Horn, Dino, Hamilton, & Noerachmanto, 2007; Mason & Posner, 2009; McCambridge, Slym, & Strang, 2008). Jones and colleagues reviewed this issue for one mediator, alcohol beliefs, by examining interventions that targeted both alcohol beliefs and drinking behavior. They concluded from these studies that there was no evidence that changing beliefs was an effective means of changing drinking (Jones, Corbin, & Fromme, 2001a; Jones, Corbin, & Fromme, 2001b). Thus, while we cannot draw conclusions from this study about the mechanisms through which CHOICE has an effect on alcohol use among middle school students, the absence of mediating effects for alcohol-related beliefs is certainly not without precedence in the literature.

Schools are increasingly faced with challenges that limit their ability to provide prevention and intervention services. They often lack resources (e.g., tools or funding) or capacity (e.g., knowledge, attitudes, skills) to adapt and implement strategies that have been developed in resource-intense research settings (D'Amico, Chinman, Stern, & Wandersman, 2009). Many schools are currently operating in an environment of budget cuts and increased teacher workload, combined with pressures to complete more intensive mandated curricula and obtain high test scores. These factors can challenge a school’s ability to provide alcohol and drug prevention programming and to implement programming as it was intended (D'Amico, et al., 2009). Thus, there is often a large gap between research and practice (Green, 2001; Wandersman & Florin, 2003). Building CHOICE with community input and collaborating on program development decreased this gap, creating an inexpensive, easily implemented, and accessible program. In addition, this larger trial replicates the pilot study of CHOICE and demonstrates that 1) adolescents will voluntarily attend an after school program that specifically provides information on AOD, and 2) this type of programming can reduce alcohol use at the school level. This study is the next step in understanding how voluntary after-school AOD programs may positively affect adolescent AOD use; further research is needed in this area to continue to move the field forward.

As with most research of this nature, we relied on self-report from adolescents, the limitations of which are well-known, although possibly exaggerated (Chan, 2008). In fact, much research has shown that self-report among youth is valid when procedures, such as those used in the current study are implemented, for example, discussing confidentiality, using Scantron forms for survey answers and having study staff not affiliated with the school collect information (D'Amico & McCarthy, 2006; Dennis, et al., 2002; Shillington & Clapp, 2000).

Although our effect sizes suggest that CHOICE attendees had clinically meaningful decreases in initiation and past month alcohol use, higher attendance at CHOICE would have allowed us a greater chance of detecting statistically significant effects among the individual participants. In addition, although our sample size is large, the power of the study to detect effects is not as large as it appears due to the use of cluster randomization. For example, the effective sample size for a sample of 9,000 for an outcome with an ICC of 0.02 is 801 (Murray, 1998), giving an effective sample size of approximately 400 per group. When considering attenders only, this effective sample size is reduced further. Thus, although the results of the study for several outcome measures failed to achieve statistical significance at the 0.05 level, this should not necessarily be taken as evidence of the ineffectiveness of this intervention. This drawback can be avoided through the use of individual randomization; however, such an approach is not feasible if the aim is to examine school level effects. In addition, CHOICE involved a number of components utilized in other successful prevention and intervention approaches (e.g., normative feedback, skills training). Further research could dismantle these different components to determine if certain features were associated with greater change compared to others.

Despite limitations, results emphasize that if prevention researchers build a program with developmentally relevant content and provide this content in an engaging and non-judgmental way, we can reach many adolescents who will voluntarily access this information. This is key as voluntary after school programs must be delivered in a way that holds adolescents’ interest or they will not attend regularly (Gottfredson, et al., 2010). After school programs can benefit youth by encouraging positive social development and better school behavior (Durlak & Weissberg, 2007; Lauver, Little, & Weiss, 2004). This cluster randomized trial demonstrates that structured, voluntary after school programs that specifically focus on AOD can also be an effective way to reach a diverse group of youth and potentially decrease alcohol use during this important developmental period.

Figure I.