Alcohol Interventions for Mandated College Students: A Meta-Analytic Review

Abstract

Objective

When college students violate campus alcohol policies, they typically receive disciplinary sanctions that include alcohol education or counseling. This meta-analysis evaluated the efficacy of these “mandated interventions” to prevent future alcohol misuse.

Methods

Studies were included if they evaluated an individual- or group-level intervention, sampled students mandated to an alcohol program, used a pretest-posttest design, and assessed alcohol use as an outcome. Thirty-one studies with 68 separate interventions (N = 8,621 participants; 35% women; 85% White) were coded by independent raters with respect to sample, design, methodological features, and intervention content; the raters also calculated weighted mean effect sizes, using random-effects models. A priori predictors were examined to explain variability in effect sizes.

Results

In the five studies that used assessment-only control groups, mandated students reported significantly less drinking relative to controls (between-group contrasts), d₊ ranged from 0.13-0.20 for quantity and intoxication outcomes. In the 31 studies that provided within-group contrasts, significant effects were observed for all outcomes in the short-term (i.e., ≤ 3 months post-intervention), with d₊ ranging from 0.14-0.27; however, fewer significant effects appeared at longer follow-ups. Four commercially-available intervention protocols (i.e., BASICS, e-CHUG, Alcohol 101, and Alcohol Skills Training Program) were associated with risk reduction.

Conclusions

Providing mandated interventions to students who violate campus alcohol policies is an effective short-term risk reduction strategy. Continued research is needed to maintain initial gains, identify the most useful intervention components, and determine the cost-effectiveness of delivery modes.

College is a time when emerging adults explore adult behaviors, roles, and relationships, including alcohol and other drug use (Arnett, 2005). This youthful exploration can result in two related but distinct consequences: drinking by underage students, and excessive drinking regardless of age. Although more than half of college students are under the minimum legal drinking age of 21 (American College Health Association, 2012), 81% report lifetime alcohol use (L. D. Johnson, O'Malley, Bachman, & Schulenberg, 2012). Relative to students over 21, underage students are more likely to drink to excess (Wechsler, Lee, Nelson, & Kuo, 2002). A substantial proportion of college students endanger themselves and their fellow students by excessive consumption of alcohol. Young adults attending college are more likely to engage in heavy episodic drinking (5 [4] or more drinks on occasion for men [women] in the last month) than their peers not in college (40% vs. 35%) (Substance Abuse and Mental Health Services Administration, 2013), and this increased risk for heavy drinking holds for both underage students and students of legal age (Hingson, Zha, & Weitzman, 2009). Nationally representative samples indicate that 40% of college students admit getting drunk in the past 30 days and 1 out of 7 college students reported consuming 10 or more drinks in a single day (L. D. Johnson et al., 2012). The college environment appears to promote a hazardous style of drinking.

More than half of students who drink report one or more negative alcohol-related problems. Some of the most frequent involve doing something later regretted, partial or full blackouts, unprotected sex, and alcohol-related injury (American College Health Association, 2012). Alcohol-related problems are associated with poorer academic functioning (Read, Merrill, Kahler, & Strong, 2007). Adverse consequences of college student drinking also affect other members of the campus community (Perkins, 2002). Nondrinking students experience collateral effects of others' drinking, such as interrupted studying, aggression and assault, and having to tend to the safety of drunken peers (Hingson et al., 2009; Wechsler et al., 2002). Institutions incur expenses related to property damage, security and emergency services, and nearby communities deal with noise and other drinking-related violations (Carey, McClurg, et al., 2009; Gebhardt, Kaphingst, & DeJong, 2000; Perkins, 2002). When underage drinking or problematic alcohol use is detected, college administrators are required by law to implement disciplinary sanctions (U.S. Department of Education, 2006). This legal obligation is accompanied by a fundamental duty to promote student development and to ensure a healthy and safe campus environment. Thus, for both legal and educational reasons, campus administrators often require both underage and of-age students who violate campus alcohol policies to receive alcohol-specific education or counseling to learn to use alcohol more safely and responsibly.

A national survey of nearly 200 institutions of higher education reveals a variety of sanctions in use, including participation in an alcohol education group (74%), parental notification (54%), individual motivational interviews (46%), and computer-delivered alcohol education (40%) (Anderson & Gadaleto, 2006). Although a growing number of empirical studies have evaluated the efficacy of mandated interventions, these studies vary on a range of methodological dimensions (e.g., mode of intervention administration, intervention dose, and targeted outcome measures), making interpretation of this growing literature challenging. Administrators and health care professionals responsible for choosing among a wide variety of intervention options often have little guidance with respect to their efficacy.

Both qualitative (Cronce & Larimer, 2011; Larimer & Cronce, 2007) and quantitative (Carey, Scott-Sheldon, Carey, & DeMartini, 2007; Carey, Scott-Sheldon, Elliott, Garey, & Carey, 2012; Fachini, Aliane, Martinez, & Furtado, 2012; Scott-Sheldon, Carey, Elliott, Garey, & Carey, 2014)reviews of alcohol interventions targeting college students support their efficacy both relative to control conditions (between-group contrasts) and over time (within-group contrasts). Meta-analyses often find small but statistically significant effect sizes; however, the distribution of the effect sizes across a number of alcohol use outcomes is usually heterogeneous. Differences among weighted mean effect sizes (between-groups) have been associated with student characteristics such as heavy drinking and mandated status (Carey et al., 2012) as well as specific intervention components (Scott-Sheldon et al., 2014). The results of extant reviews suggest that it may be fruitful to evaluate the extent to which mandated alcohol interventions produce changes in alcohol use and related problems.

Mandated students have been characterized as a special population at high risk for negative outcomes (Cronce & Larimer, 2011). On average, mandated students drink more than other students (Barnett et al., 2004; Fromme & Corbin, 2004; Merrill, Carey, Lust, Kalichman, & Carey, 2014). Mandated students are often more defensive and less ready to change than volunteers participating in the same intervention (Barnett et al., 2008; Dimeff, Baer, Kivlahan, & Marlatt, 1999; Palmer, Kilmer, Ball, & Larimer, 2010). Furthermore, males are represented disproportionately in mandated samples (Fromme & Corbin, 2004; Merrill et al., 2014), male students drink more than female students (American College Health Association, 2012), and males are more resistant to changing their alcohol use relative to females (Carey & DeMartini, 2010; Henson, Pearson, & Carey, 2015). These behavioral, contextual, and motivational factors may influence outcomes of mandated interventions and/or maintenance of risk reduction over time. A focused review of intervention efficacy offered in the context of sanctions for campus alcohol violations is needed.

We conducted a meta-analytic review with two over-arching goals, namely, (a) to evaluate the effects of interventions for mandated students across multiple outcomes, and (b) to identify student or intervention characteristics associated with changes in alcohol consumption and problems after mandated interventions. We addressed five research questions. First, do students mandated to alcohol interventions reduce their alcohol use and related problems compared to controls? This question focuses on between-group effects to address whether interventions (vs. wait-list controls) improve alcohol outcomes. Because most schools require students who violate campus alcohol policy to complete an intervention, our ability to test for the efficacy of delivering an intervention is limited to the few studies that randomized mandated students to wait-list control conditions. Second, over time, do students mandated to alcohol interventions drink less and report fewer related problems? Due to the requirement that all mandated students receive an appropriate intervention, we evaluate the magnitude and duration of within-group effects. We assess pre- to post-test change because most studies evaluating alcohol interventions with mandated students compare two active intervention conditions. Active comparison conditions with alcohol content are likely to produce some risk reduction among college drinkers (Carey, Scott-Sheldon, Elliott, Bolles, & Carey, 2009; Scott-Sheldon et al., 2014) and so between-group differences between two active interventions can be small and difficult to interpret. Consistent with the broader college alcohol intervention literature, we hypothesize that mandated students who receive any alcohol intervention will report lower alcohol consumption and fewer alcohol-related problems relative to baseline. Effect sizes representing within-group changes over time are subject to alternative explanations for change (such as maturation), however this interpretive limitation is addressed in part by the next set of questions regarding factors that predict magnitude of change.

Third, we examine whether the magnitude of drinking reduction varied by student gender. We predict that samples with more women would be associated with larger effect sizes, based on prior research (Carey et al., 2007). Fourth, we determine what types of interventions are associated with larger drinking reductions. To do this we coded structural and content features of interventions to evaluate their association with outcomes. Hypothesized predictors of change over time included (a) mode of administration (i.e., individual vs. group; face-to-face vs. computer-delivered interventions; length of intervention), and (b) intervention components (e.g., education, personalized feedback, moderation strategies, and goal-setting). We hypothesized that interventions will be more effective when they were longer and used delivery formats (i.e., individual and face-to-face interventions) that have been associated with stronger effects in non-mandated student samples (Carey et al., 2007; Larimer & Cronce, 2007). We also hypothesized that interventions will be more effective when they used intervention components that are personalized (i.e., drinking feedback, moderation strategies, challenges to alcohol expectancies, goal-setting, identification of high-risk situations) consistent with results obtained with non-mandated students (Scott-Sheldon et al., 2014). Finally, we characterize the efficacy of commercially-available intervention protocols (e.g., BASICS, Alcohol 101) that have been marketed and widely disseminated to campuses, in order to evaluate the evidence base for these products.

Methods

Sample of Studies and Selection Criteria

Following established guidelines (Reed & Baxter, 2009; Rosenthal, 1991), studies were retrieved from (1) electronic databases, (2) reference sections of relevant papers, (3) professional journals, and (4) author responses to requests. First, we searched electronic databases (PubMed, PsycInfo, ERIC, CINAHL, Dissertation Abstracts, The Cochrane Library) using a Boolean search strategy that included the following broad search terms: (alcohol OR drink* OR binge) AND (college OR university) AND (intervention OR prevention). Search terms were adjusted based on the specific requirements of each electronic database. We conducted a broad search of alcohol interventions targeting college students rather than restricting the search to mandated students only. Second, we reviewed the references of papers obtained through the database searches. Third, we examined the tables of contents and/or abstracts available from relevant electronic journals (e.g., Addiction, Addictive Behaviors, Journal of Studies on Alcohol and Drugs). Finally, we searched the National Institutes of Health Research Portfolio Online Reporting Tools (NIH RePORTER) database for relevant awards, sent requests for papers to individual researchers, and posted messages to several electronic mailing lists. We continued to review manuscripts received from researchers and obtained through journals for the 6-month period following the last database search.

Both published and unpublished studies fulfilling the selection criteria and available by the end of December 2012 were included if they (a) examined an individual- or group-level intervention to reduce alcohol use, (b) sampled students mandated to an alcohol intervention, (c) used either a single-group or an independent-groups pretest-posttest design, (d) measured alcohol use, and (e) provided statistical information needed to calculate effect sizes. Studies were excluded if they (a) did not focus on alcohol use (e.g., combined substance use interventions), (b) exclusively sampled non-mandated college students, or (c) included a mass media or structural-level intervention component. If a study reported an intervention delivered to non-mandated comparison sample, that condition was not included in the analyses. No-intervention/wait-list control conditions were also excluded from the within-groups analyses. Thirty-one studies (68 interventions) were included (Figure 1). We refer to all active alcohol-focused interventions (whether listed as experimental or control) as “interventions” [k] throughout this manuscript.

Figure 1. Selection of Studies.

Coding and Reliability

Two trained coders independently rated the study descriptive information, sample characteristics (e.g., sex), design and measurement specifics (e.g., number of follow-ups), and length and content of intervention (e.g., number of total minutes, skills-training). Study quality was assessed using 14 items (e.g., manualized treatment, retention, reactivity of assessment procedures) adapted from validated measures (Jadad et al., 1996; Miller et al., 1995). A random selection of 20 studies was used to assess inter-rater reliability. For the categorical variables, raters agreed on 84% of the judgments (mean Cohen's kappa = 0.65, signifying substantial agreement) (Landis & Koch, 1977). Reliability for the continuous variables (using the intraclass correlation coefficient; ρ) yielded an average ρ of 0.82 (median = 0.97). Discrepancies between coders were resolved through discussion.

Study Outcomes

Effect size (ES) estimates were calculated for alcohol consumption and related problems. Consumption outcomes included: quantity consumed (a) over a period of time (e.g., week or month) and (b) during specific drinking periods (e.g., weekends, spring break); (c) frequency of drinking days over a period of time (e.g., week or month); (d) frequency of heavy drinking, usually defined as 5 or more drinks for men and 4 or more drinks for women (Wechsler, Dowdall, Davenport, & Rimm, 1995); (e) maximum amount of alcohol consumed on a single occasion; and (f) peak (maximum) and (g) typical (average) estimated blood alcohol concentration (BAC, or the percentage of alcohol contained in the blood, measured as weight/volume). Alcohol-related problems (e.g., hangover, blackouts, missed work or classes) were typically operationalized using multi-item scales.

Effect Size Calculations

Two methods were used for calculating ES: (a) the standardized mean difference (between-group contrasts) and (b) the standardized mean gain (within-group contrasts). ES were calculated as the mean differences between the pre- and post-test divided by the pre-test standard deviation for each condition (Becker, 1988; Morris & DeShon, 2002). For the between-group contrasts, the standardized mean gain for each wait-list control group was subtracted from the standardized mean gain for each intervention group. Thus, all ES were adjusted for baseline. This is the preferred method for repeated-measures designs according to the Agency for Healthcare Research and Quality (AHRQ) Methods Research Report (B. T. Johnson & Huedo-Medina, 2013). When means and standard deviations were not provided, other measures (e.g., t- or F-test) were used (Lipsey & Wilson, 2001). All ES were corrected for sample size bias (Hedges, 1981).

Multiple ES were calculated from individual studies when they reported more than one outcome variable, when outcomes were reported separately by sample characteristics (i.e., gender), or studies reported outcomes across multiple follow-ups. We grouped follow-up intervals into short-term (≤ 13 weeks, equivalent to ≤3 months), intermediate (14-51 weeks) and longer-term (≥ 52 weeks, equivalent to one year or more). If a study conducted multiple assessments within one of these intervals, we report the ES for the longest follow-up within the interval. When a study reported multiple measures of a single outcome at the same assessment interval, the ES were averaged. Two coders independently calculated ES; ES were examined for consistency and discrepancies were corrected. Positive ES indicate that mandated students receiving an intervention reported the intended effects (lower alcohol consumption and fewer alcohol-related problems relative to wait-list controls or at post-test).

Statistical Analysis

Weighted mean ES (d₊) were calculated using a random-effects model following maximum likelihood assumptions (Lipsey & Wilson, 2001). Consistent with procedures described by Lipsey and Wilson (2001; pp. 107-108) we examined each dependent variable for outliers prior to conducting statistical analyses. Because few outliers were detected (see table notes, Table 2 and 3), we eliminated outliers from subsequent analyses. ES calculated for a single study by sample characteristic (i.e., men, women) were analyzed as separate studies because the subgroups are independent from each other (Borenstein, Hedges, Higgins, & Rothstein, 2009; Lipsey & Wilson, 2001). To assess the null hypothesis of homogeneity (i.e., that all studies share a common ES), the Q statistic and associate p-value for each outcome were calculated. The proportion of the observed dispersion was assessed using the I² index and its corresponding 95% uncertainty intervals (Higgins & Thompson, 2002; Huedo-Medina, Sanchez-Meca, Marin-Martinez, & Botella, 2006; Ioannidis, Patsopoulos, & Evangelou, 2007). The I² index ranges from 0 to 100% with percentages of 25%, 50%, and 75%, considered low, moderate, and high levels of observed variance reflecting true differences in ES (Higgins, Thompson, Deeks, & Altman, 2003). Asymmetries in the distributions of ES, indicating a possible reporting bias (Rosenthal, 1979), were examined by inspecting funnel plots (Sterne & Egger, 2001) and assessing the degree of funnel plot asymmetry using recommended methods (Begg & Mazumdar, 1994). Consistent with meta-analytic recommendations, tests for funnel plot asymmetry were conducted only for variables with at least 10 studies (Higgins & Green, 2011).

Table 2.

Weighted mean difference (between-groups) effect sizes and heterogeneity tests for interventions versus wait-list controls used to reduce alcohol consumption and alcohol-related problems among mandated college students.

Outcome	k	d+ (95% CI)	Heterogeneity
		Random effects	Q	p	I2(95% UI)
Short-term assessment (≤13 weeks)
Alcohol consumption
Quantity, per week/month	10	0.13 (0.02, 0.25)	9.70	.375	7 (0, 45)
Quantity, specific intervals/drinking day	8	0.08 (-0.05, 0.21)	6.85	.445	0 (0, 00)
Peak consumption	2	0.28 (-0.15, 0.71)	1.18	.277	15 (0, 54)
Frequency of drinking days	3	0.09 (-0.13, 0.31)	2.13	.345	6 (0, 35)
Frequency of heavy drinking	7	0.10 (-0.02, 0.23)	5.86	.439	0 (0, 00)
Peak BAC	7	0.20 (0.06, 0.33)	9.68	.139	38 (0, 74)
Typical BAC	5	0.16 (0.01, 0.31)	0.85	.932	0 (0, 51)
Alcohol-related problems	11	-0.02 (-0.15, 0.12)	7.82	.646	0 (0, 61)

Note. All dependent variables were examined for outliers (Emerson & Strenio, 1983). Of the 53 effect sizes, one outlier was detected for typical BAC (short-term). The unadjusted and adjusted (i.e., trimmed) weighted mean effect size-differed in magnitude for typical BAC (d_+unadjusted = 0.10, 95% CI = -0.03, 0.23, k = 6; d_+adjusted = 0.16, 95% CI = 0.01, 0.31, k = 5). The adjusted effect size is reported above. Positive weighted mean effect sizes (d₊) indicate that the treatment group reduced consumption or problems relative to the control group. k, number of interventions; d₊, weighted mean effect size; Q, heterogeneity statistic; I², consistency of effect sizes; CI, confidence interval; UI, uncertainty interval; BAC, blood alcohol concentration. Bold font indicates significant weighted mean difference effects.

Table 3.

Weighted mean gain (within-group) effect sizes and heterogeneity tests for interventions used to reduce alcohol consumption and alcohol-related problems among mandated college students.

Outcome	k	d+ (95% CI)	Heterogeneity
		Random effects	Q	p	I2(95% UI)
Short-term assessment (≤13 weeks)
Alcohol consumption
Quantity, per week/month	42	0.22 (0.14, 0.31)	58.59	.037	30 (0, 52)
Quantity, specific intervals/drinking day	28	0.14 (0.06, 0.23)	41.72	.035	35 (0, 59)
Peak consumption	19	0.27 (0.13, 0.41)	32.21	.009	49 (13, 70)
Frequency of drinking days	23	0.17 (0.04, 0.30)	88.66	<.001	75 (63, 83)
Frequency of heavy drinking	22	0.23 (0.13, 0.33)	20.58	.485	0 (0, 0)
Peak BAC	23	0.26 (0.17, 0.36)	43.80	.004	50 (19, 69)
Typical BAC	22	0.17 (0.08, 0.26)	31.13	.071	33 (0, 60)
Alcohol-related problems	50	0.25 (0.19, 0.31)	84.08	<.001	42 (18, 58)
Intermediate assessment (14-51 weeks)
Alcohol consumption
Quantity, per week/month	18	0.08 (-0.03, 0.18)	40.60	.001	58 (29, 75)
Quantity, specific intervals/drinking day	15	0.07 (-0.03, 0.18)	27.47	.017	49 (7, 72)
Peak consumption	9	0.06 (-0.14, 0.27)	14.06	.080	43 (0, 74)
Frequency of drinking days	5	0.12 (-0.11, 0.36)	56.90	<.001	93 (87, 96)
Frequency of heavy drinking	18	0.14 (0.04, 0.23)	15.39	.567	0 (0, 59)
Peak BAC	16	0.25 (0.14, 0.36)	43.59	<.001	66 (42, 80)
Typical BAC	10	0.17 (0.04, 0.29)	33.02	<.001	73 (48, 86)
Alcohol-related problems	25	0.13 (0.06, 0.21)	113.68	<.001	79 (69, 85)
Long-term assessment (≥52 weeks)
Alcohol consumption
Quantity, per week/month	10	-0.06 (-0.25, 0.12)	21.09	.012	57 (13, 79)
Quantity, specific intervals/drinking day	11	0.03 (-0.11, 0.17)	9.87	.452	0 (0, 0)
Peak consumption	2	-0.13 (-0.78, 0.52)	0.01	.904	0 (0, 100)
Frequency of drinking days	6	-0.15 (-0.42, 0.13)	14.34	.014	65 (16, 85)
Frequency of heavy drinking	14	-0.02 (-0.15, 0.11)	16.20	.239	20 (0, 57)
Peak BAC	10	0.11 (-0.03, 0.25)	54.38	<.001	83 (71, 91)
Typical BAC	10	0.12 (0.01, 0.25)	9.83	.364	8 (0, 48)
Alcohol-related problems	10	0.05 (-0.09, 0.20)	29.72	<.001	70 (42, 84)

Note. All dependent variables were examined for outliers (Emerson & Strenio, 1983). Of the 428 effect sizes, 10 outliers were detected (2% of the total number of effect size estimates). The weighted mean effect sizes for each dependent variable were consistent in magnitude and direction for unadjusted and adjusted (i.e., trimmed) effect sizes, so only the adjusted effect sizes are reported above. Positive weighted mean effect sizes (d₊) indicate reduced consumption or problems at post-test. k, number of interventions; d₊, weighted mean effect size; Q, heterogeneity statistic; I², consistency of effect sizes; CI, confidence interval; UI, uncertainty interval; BAC, blood alcohol concentration. Bold font indicates significant weighted mean gain effects.

Predictors of Outcome

Meta-regression was used to assess the association between sample, methodological, or intervention characteristics and the magnitude of within-group effects. Predictors were selected a priori and included proportion of women in the sample, intervention delivery mode (individual vs. group, face-to-face vs. computer-delivered, length of intervention), and content (e.g., feedback on consumption) (Harbord & Higgins, 2008; Hedges, 1981; Lipsey & Wilson, 2001). Mixed-effect models were used such that the inverse variance for each ES included error associated with (a) within-study level sampling error and (b) additional between-study population variance. We selected this model because it is more conservative than purely fixed-effects models, minimizing the Type I error rate (Lipsey & Wilson, 2001). Significant univariate predictors were simultaneously entered into multiple regression models to evaluate whether they explained unique variance. Permutation tests were conducted to adjust the p-values for multiple testing (Harbord & Higgins, 2008; Higgins & Thompson, 2004). All analyses were conducted in Stata 12.1 (StataCorp, 2012).

Results

Study, Sample, and Intervention Characteristics

Study, sample, and intervention characteristics of the included studies are provided in Table 1. Thirty-one studies (from 30 publications) met the inclusion criteria and were included in the meta-analysis. (Details for the included studies are provided in Table A, Supplemental Digital Content.) Most studies (29 out of 31) were published in journals (2 studies were unpublished dissertations) with a median publication date of 2007 (range = 1991 to 2015).¹

Table 1.

Study, Sample, and Treatment Characteristics of the 31 Studies (68 treatments) included in the Meta-Analysis.

Study Characteristics
Publication year
Mdn	2007
Range	1991-2015
Data collection year
Mdn	2004
Range	1989-2010
Published in journals, %	94
Funded study, %	68
Sample Characteristics
Sample size, Mdn/M	151/287
Retention, M% (SD)	77 (21)
Age, M (SD) (n = 24)	19 (0.51)
Sex, % women (n = 30)	35
Ethnicity, % White (n = 28)	85
Risk Characteristics
Baseline alcohol use, % (n = 29)	100
Quantity, M (SD), n = 23	13 (6)
Problems, M (SD), n = 16	7 (4)
Year in school, % (n = 24)
First	58
Second	27
Third	12
Fourth	5
Greek members, % (n = 9)	22
Institution Characteristics
Region, no. of studies
U.S. Northeast	15
U.S. Southeast	4
U.S. Midwest	3
U.S. Northwest	6
U.S. Southwest	3
Institution(s), %
Public university	61
Private university	36
Private university/liberal arts college	3
Size of institution, %
Small (<5,000)	9
Medium (5,000-15,000)	19
Large (>15,000)	72
Design and Measurement
Randomized controlled trial, %	68
Provided incentives, %	61
Follow-up length, %
<3 months	29
3 to 5 months	26
≥ 6 months	45
Treatment Characteristics
Dose, Mdn (Range)
Sessions	1 (1 - 4)
Minutes	60 (5 - 450)
Delivery method, %
FTF	74
CDI	19
Computer-facilitated FTF	4
FTF and CDI	1
Written (facilitator provided)	1
Type of delivery, %
Individual	59
Group	39
Both individual and group	1
Group size, Mdn (Range)	10 (2-19)
Group composition, % mixed-sex	74
Facilitators, Mdn (Range)	1 (0 – 6)
Type of Facilitators, %
Peers	12
Paraprofessionals	18
Professionals-in-training	31
Professionals	0
Multiple	16
None	24
Treatment Components
Personalized feedback, %	82
Normative comparisons, %	90
Alcohol/BAC education, %	85
Moderation strategies, %	79
Goal-setting, %	62
Challenges to expectancies, %	60
Focus on high-risk situations, %	44
Skills training, %	29
Decisional balance exercise, %	26
Values clarification, %	10
Writing/journaling, %	9
Provided materials, %
Generic	34
Tailored	41
Boosters	6

Note. n, number of studies. FTF, face-to-face. CDI, computer-delivered intervention.

Studies were typically conducted at large public universities in the northeast U.S. Of the 8,621 college students mandated to an alcohol program, 65% were men and most were White (85%); the average age was 19 years (range = 18.6 to 20.4). Participants were typically first (58%) or second (27%) year students. Most studies randomized participants to two or more conditions (68%), and reported providing incentives to participants for completing the assessments (61%). On average, participants completed 2 (SD = 2) post-intervention assessments, typically occurring at 13 weeks after intervention but ranging from 0 to 65 weeks.

Interventions were typically delivered during a single one-hour session. Most of the sessions were delivered face-to-face (74%) but some (19%) were delivered via computer or included a computer-based component and/or session (4%). When interventions involved facilitators, 12% used peer facilitators; non-peer facilitators included paraprofessionals (18%), professionals-in-training (31%), or a combination of paraprofessionals, professionals-in training, and professionals (16%). Most interventions were delivered to individuals (59%) but some were delivered in small groups (39%; Mdn = 10 participants, range = 2 to 19); one study used a combination of individual and group sessions. All of the group interventions, but only 59% of the individual interventions, were delivered face-to-face. Less than half (44%) of the interventions used a publicly available intervention protocol (video, computer, or published manual). Content included normative comparisons (90%), alcohol education (85%), personalized feedback on alcohol consumption, alcohol-related problems, or alcohol-related risks (82%), strategies to modify alcohol consumption (79%), goal setting (62%), or expectancy challenge exercises (60%).

Methodological Quality

The studies satisfied an average of 57% (SD = 17%; range = 20% to 85%) of the methodological quality criteria.² Ratings for the 14 methodological quality items are as follows: (1) 68% of the studies randomized participants to treatment and/or control conditions, (2) 100% of the studies used a pre/post-test design, (3) 100% controlled the quality of the study by standardizing intervention delivery (using a manual and/or systematic training, or delivery by computer), (4) 45% had post-intervention assessments of 12 months or longer, (5) 50% of the studies reported ≥85% of their samples completed follow-ups, (6) 3% reported that the study was anonymous, (7) 87% used methods to minimize reactivity of measure completion (e.g., using different personnel for intervention and measurement), (8) 3% used objective measures (e.g., medical records, breathalyzer) in their assessments, (9) 3% used collateral verification, (10) 48% used assessment methods (e.g., interviewers, computerized questionnaires) that were blind to the group assignment, (11) 75% of studies reported on the students who withdrew from the study, (12) 55% included those lost to follow-up in outcome reporting (e.g., intent-to-treat analyses), (13) 100% of the studies reported acceptable statistical analyses for the study design (e.g., assessed groups for differences at baseline), and (14) 3% reported conducting the study at more than one site.

Efficacy of the Alcohol Interventions Compared to Wait-List Controls

Weighted mean ES (d₊) for alcohol consumption and alcohol-related problems between the intervention and wait-list controls are presented in Table 2. Mandated students who received an active alcohol intervention reduced their quantity of drinking (d₊ = 0.13, 95% CI = 0.02, 0.25), peak BAC (d₊ = 0.20, 95% CI = 0.06, 0.33), and typical BAC (d₊ = 0.16, 95% CI = 0.01, 0.31) at the short-term assessment compared to wait-list controls. There were no differences between the intervention and wait-list controls on quantity of drinking during specific intervals/drinking days, peak consumption, frequency of drinking days or heavy drinking, and alcohol-related problems. The null hypothesis of homogeneity was confirmed for all outcomes (i.e., non-significant Q). I² ranged from 0% to 38% (low to moderate); however uncertainty limits were wide for many outcomes and exceeded the 50% threshold. These analyses suggest that some heterogeneity between ES estimates may exist but our ability to test for heterogeneity is limited due to the small number of included studies (Ioannidis et al., 2007).

Efficacy of the Alcohol Interventions over Time

Table 3 displays the weighted mean ES (d₊) for the change in alcohol consumption and alcohol-related problems at the short-term, intermediate, and long-term assessment intervals. Overall, students who participated in a mandated intervention reduced their alcohol consumption (drinks per week/month, drinks per drinking interval, peak consumption, frequency of drinking days and heavy drinking, typical and peak BAC) and alcohol-related problems at the short-term assessment (d₊ = 0.14 to 0.27). The hypothesis of homogeneity was rejected for all outcomes except for the frequency of heavy drinking (Q [22] = 20.58, p = .485, I² = 0, 95% UI = 0, 0).I² ranged from 30% to 75% (low to high), however uncertainty limits were wide for many outcomes and exceeded the 50% threshold. We examined whether a priori determined sample or intervention characteristics explained the variability in the ES estimates (see below).

Changes in alcohol consumption (frequency of heavy drinking, typical and peak BAC) and alcohol-related problems were also found at the intermediate assessment (d₊ = 0.13 to 0.25) but only a single outcome (i.e., typical BAC) was found to be significant at the long-term assessment (d₊ = 0.12, 95% CI = 0.01, 0.25, k = 10). The hypothesis of homogeneity was rejected for all of these outcomes except for the frequency of heavy drinking at the intermediate assessment (Q [18] = 15.39, p = .567, I² = 0, 95% UI = 0, 59) and typical BAC at the long-term assessment (Q [10] = 9.83, p = .364, I² = 8, 95% UI = 42, 84). I² ranged from 0% to 79% (low to high) but the uncertainty limits were wide for the frequency of heavy drinking (intermediate assessment) and typical BAC (long-term assessment), exceeding the 50% threshold.

Predictors of the Pre- to Post-test Changes in Alcohol Outcomes at Short-Term Assessment

Univariate analyses, using mixed-effects models, were conducted to examine a priori selected predictors of change in alcohol consumption and alcohol-related problems at the short-term assessment only (within-group contrasts). We restricted our analyses to the short-term assessment because this interval contained the largest number of studies available for multiple predictor tests. Table 4 presents the 10 predictors, with results from the univariate regression analyses; these are organized below by outcome. P-values were adjusted based on permutation tests with 5,000 permutations (Harbord & Higgins, 2008). We controlled for methodological quality in testing predictors of change in the frequency of heavy drinking and alcohol-related problems.

Table 4.

Predictors of alcohol outcomes at the short-term assessment (≤13 weeks).

	Quantity, per week/month		Quantity, Specific Interval		Peak Consumption		Drinking Days		Heavy Drinking		Peak BAC		Typical BAC		Alcohol Problems
Sample Characteristics	B (SE)	p	B (SE)	p	B (SE)	p	B (SE)	p	B (SE)	p	B (SE)	p	B (SE)	p	B (SE)	p
Sex	.00 (.09)	.98	-.11 (.09)	.25	.13 (.15)	.40	-.13 (.16)	.43	-.09 (.08)	.28	-.15 (.12)	.21	-.10 (.11)	.39	.05 (.08)	.51
Intervention Delivery
Individual vs. group	.04 (.07)	.54	-.09 (.06)	.17	-.04 (.11)	.75	-.09 (.11)	.41	--	--	-.01 (.15)	.96	-.04 (.17)	.80	-.03 (.05)	.57
FTF vs. CDI	.04 (.07)	.50	.14 (.06)	.03	.03 (.12)	.78	-.11 (.14)	.45	-.10 (.06)	.11	.10 (.08)	.26	.08 (.08)	.29	.04 (.06)	.53
Duration in minutes	-.00 (.00)	.50	.00 (.00)	.89	.00 (.00)	.50	-.00 (.00)	.97	.00 (.00)	.83	.00 (.00)	.14	.00 (.00)	.10	.00 (.00)	.56
Intervention Components
Alcohol education	-.31 (.12)	.01	-.42 (.10)	<.01	-.03 (.14)	.84	-.43 (.16)	.01	--	--	.31 (.19)	.12	.18 (.19)	.35	-.22 (.09)	.02
Personalized feedback	.06 (.09)	.49	.14 (.11)	.24	.24 (.16)	.16	.18 (.24)	.47	.05 (.20)	.79	-.02 (.24)	.94	.12 (.20)	.57	.16 (.07)	.03
Moderation strategies	-.09 (.09)	.33	-.11 (.10)	.30	.00 (.12)	.99	.13 (.17)	.46	--	--	.31 (.19)	.12	.18 (.19)	.35	-.01 (.07)	.94
Goal-setting	-.01 (.07)	.84	.14 (.07)	.06	.18 (.11)	.12	.25 (.12)	.04	.01 (.06)	.90	.14 (.09)	.14	.13 (.13)	.34	.13 (.05)	.02
Expectancy challenges	-.07 (.06)	.25	-.00 (.07)	.96	.14 (.11)	.24	.26 (.12)	.04	.01 (.06)	.87	-.02 (.09)	.82	-.06 (.09)	.48	.03 (.06)	.57
Identifying risk situations	-.01 (.06)	.91	.10 (.06)	.10	.14 (.10)	.20	.18 (.10)	.09	.03 (.05)	.53	-.05 (.08)	.54	-.02 (.08)	.80	.06 (.05)	.24

Note. FTF = face-to-face. CDI = computer delivered interventions. Reported coefficients are unstandardized. Predictors that were significant (mixed-effects models) are in bold. Predictors that remained significant using the adjusted p-values obtained from the permutation tests (5,000 permutations) are indicated by underline.

Quantity of alcohol consumed

Less change in the quantity of alcohol consumed was observed for interventions providing alcohol education (B = -0.31, SE = .12, p = .01), accounting for 44% of the between-study variance, but this variable did not retain significance when adjusted for multiple univariate tests (adjusted p-value = .231). All other tests of predictors were non-significant (adjusted ps >.994).

Quantity of alcohol consumed during specific intervals

Alcohol consumption during specific intervals (e.g., drinking days) reduced significantly if the intervention was delivered face-to-face (vs. computer; B = 0.14 SE = 0.06, p=.03) and the intervention content did not emphasize alcohol education (B = -0.42, SE = .10, p<.01), but only alcohol education retained significance when adjusting for multiple univariate tests (adjusted p-value = .005). All other tests for predictors were non-significant (ps >.398). Delivery mode and alcohol education were simultaneously entered into a regression analysis; only (lack of) alcohol education (B = -0.39, SE = 0.10, p< .001; I²_Residual = 0%) remained a significant predictor of change in the quantity of alcohol consumed during specific intervals, accounting for 100% of the between-study variance. Alcohol education retained significance when adjusting for multiple meta-regression (adjusted p-value = .003).

Peak consumption

No significant predictors of change in peak consumption were found (adjusted p-values >.736).

Drinking days

Interventions were associated with reduced drinking frequency when they did not provide alcohol education (B = -.43, SE = .16, p = .01), they encouraged goal setting (B = .25, SE = .12, p = .04),and they challenged expectancies (B = 0.25, SE = 0.12, p=.04); none of these pre reached significance when they were adjusted for multiple univariate testing (ps = .185, .071, and .431, respectively). These variables were simultaneously entered into a multiple regression analysis. Only alcohol education (B =-0.36, p<.02; I²_Residual = 54%) remained a significant predictor of change in the frequency of drinking days, and accounted for 34% of the between-study variance (model testing alcohol education alone; adjusted R² = 56% for the full model). Alcohol education did not reach statistical significance when the p-value was adjusted for multiple meta-regression (adjusted p-value = .085).

Heavy drinking frequency

No significant predictors of change in heavy drinking frequency were found (adjusted p-values >.444).

Peak and Typical BAC

No significant predictors of change in peak BAC (adjusted p-values >.717) or typical BAC (adjusted p-values >.607) were found.

Alcohol-related problems

Reductions in students' alcohol-related problems were greater when interventions included personalized feedback (B = 0.16, SE = 0.09, p=.03) and goal-setting (B = 0.13, SE = 0.05, p=.02), and when they did not include alcohol education (B = -0.22, SE = 0.09, p = .02) but these variables did not retain significance when adjusting for multiple univariate testing (adjusted p-values = .954, .714, and .142, respectively). These variables were simultaneously entered into a regression analysis with goal-setting remaining a significant predictor of the change in alcohol-related problems (B = 0.13, SE = .06, p = .03, I² _Residual = 10%), and accounted for 65% of the between-study variance (adjusted R² = 80% for the full model). Goal-setting did not reach statistical significance when the p-value was adjusted for multiple meta-regression (adjusted p-value = .090).

Exploratory Analyses: Intervention Protocols

Interventions protocols that have published manuals for use by campus staff or that are commercially available as intact interventions are particularly useful, as these features facilitate dissemination. These can be contrasted to interventions that are available primarily in research contexts and are not easily disseminated in practice settings. Four intervention protocols meet these criteria (a published manual or a commercial available product) and have been evaluated in at least two studies, which allows for ES estimation (i.e., BASICS, e-CHUG, Alcohol 101, and Alcohol Skills Training Program). Stratified analyses examined the weighted mean ES for quantity of alcohol consumed and alcohol-related problems (the two most commonly reported outcomes) by specific intervention protocols at the short-term assessment. Mixed-effects models, using maximum likelihood estimates, were used. Interventions based on BASICS (d₊ = 0.26, 95% CI = 0.17,0.50; k = 14), Alcohol 101 (d₊ = 0.15, 95% CI = 0.06,0.24; k = 11), and e-CHUG (d₊ = 0.41, 95% CI = 0.05,0.77; k = 2) achieved significant reductions in alcohol consumption (Figure 2), Q_B (3) = 13.90, p<.001. Significant short-term improvements were associated with all four of the intervention protocols for alcohol-related problems (Figure 2): BASICS (d₊ = 0.31, 95% CI = 0.21,0.41; k = 14), Alcohol 101 (d₊ = 0.14, 95% CI = 0.03,0.25; k = 11), e-CHUG (d₊ = 0.39, 95% CI = 0.13,0.65; k = 3), and the Alcohol Skills Training Program (d₊ = 0.36, 95% CI = 0.16,0.55; k = 3), Q_B (3) = 12.89, p =.005.

Figure 2.

Weighted mean gain effect sizes at short-term follow-up by intervention protocols.

Publication Bias

Both published and unpublished manuscripts were included in this meta-analysis to reduce the possibility of publication bias; nonetheless, we also examined our data using graphical and statistical tools to test for the possibility of publication bias. (Funnel plots and results of the statistical tests are available in the Supplemental Online Figures A1-A27 and Table B.) Inspection of the funnel plots revealed some asymmetries that might be interpreted as publication bias (e.g., positive skew of data for quantity of alcohol consumed). Results for Begg's test (Begg & Mazumdar, 1994) were non-significant for all outcomes except for alcohol-related problems at the short-term interval (p = .02). Trim-and-fill procedures (Duval & Tweedie, 2000) were used to estimate and correct for the possibility of missing studies. These analyses found that six studies measuring the quantity of alcohol consumption, four studies measuring the quantity of alcohol consumed at specific intervals, four studies measuring the frequency of heavy drinking, six studies measuring peak BAC, and fourteen studies measuring alcohol-related problems are estimated to be missing at the short-term assessment. The random-effect weighted mean ES would be similar in magnitude to the estimate mean ES from the original dataset for all of the listed outcomes. The findings from the trim-and-fill measures should be interpreted with caution as performance of this method is poor when significant heterogeneity is present (Lau, Ioannidis, Terrin, Schmid, & Olkin, 2006; Sutton, 2009; Terrin, Schmid, Lau, & Olkin, 2003).

Discussion

This meta-analysis summarizes the efficacy of alcohol interventions for college students who were mandated to participate after violating campus alcohol policy. Our analyses showed that mandated interventions are associated with reductions in alcohol use and related problems across a range of measured variables. Reductions across all reported outcomes were observed at follow-ups up to 3 months, however, the magnitude and number of significant reductions diminished over longer assessment intervals.

Only five studies compared outcomes of active alcohol interventions to a pure (i.e., no intervention or wait list) control; evidence from these trials support the efficacy of providing an alcohol intervention even when it is mandated. Finding small effects on selected outcomes when comparing alcohol interventions to wait-list controls is consistent with observations that the process of being sanctioned can prompt self-initiated change even in the absence of an active intervention (e.g.,(Carey, Henson, Carey, & Maisto, 2009; Hustad et al., 2011; Morgan, White, & Mun, 2008). However, even if students waiting for an intervention reduce how much they drink, the active intervention produced larger reductions in standard drinks consumed and average and peak BACs. Therefore, these findings confirm the additional benefit of offering an alcohol intervention to students who have been sanctioned for alcohol violations.

These data provide empirical validation for use of alcohol risk reduction interventions as part of institutional sanctions for those who violate campus alcohol policies (U.S. Department of Education, 2006). Finding that mandated alcohol prevention programs reduce alcohol use among college students corroborates observations from community-based treatment studies that mandatory treatment results in positive outcomes (Kelly, Finney, & Moos, 2005; Yeterian, Greene, Bergman, & Kelly, 2013). Thus, enforcement of campus alcohol policy is an opportunity to reach students who can benefit from an alcohol risk reduction interventions.

Given the large number of within-group intervention effects, many of them coming from randomized controlled trials that evaluated multiple active interventions, we supplemented our analyses with tests of change over time. The observed effects of these mandated interventions are small but are consistent with other meta-analyses evaluating within-group changes in alcohol-related outcomes for college drinkers (Carey, Scott-Sheldon, et al., 2009; Samson & Tanner-Smith, 2015; Scott-Sheldon, Terry, Carey, Garey, & Carey, 2012).Given that most campuses deliver some intervention to students violating alcohol policy, if such evidence-based interventions were to be applied widely, they could have significant public health impact. It is notable that most within-group effects were heterogeneous, meaning that there was variability across active interventions in the amount of change observed. Variability in outcomes was explained, in part, using characteristics of the interventions.

Our predictor analyses identified intervention components associated with stronger effects. Most consistently, we found that an explicit focus on alcohol education may be counterproductive. The presence of this component was associated with less change on several outcomes, consistent with prior reviews of that have concluded that information or educational interventions do not change drinking behavior (e.g., Larimer & Cronce, 2007). This finding is notable given that alcohol education groups are the most commonly employed sanction (Anderson & Gadaleto, 2006).

In contrast, several of the other intervention components were associated with larger reductions in drinking. Interventions containing goal setting and expectancy challenges were associated with greater reduction in drinking frequency. Interventions containing personalized feedback and goal setting were associated with fewer alcohol problems. Goal setting is an established self-management strategy known to facilitate behavior change (Hester, 1995; Locke & Latham, 2002). Personalized feedback can raise awareness for students who do not realize that they are accumulating alcohol-related problems, consistent with the Transtheoretical Model (Prochaska, DiClemente, & Norcross, 1992). In addition, expectancy challenge interventions have been shown to reduce drinking behavior in the short-term in general college samples (Labbe & Maisto, 2011; Scott-Sheldon et al., 2012).

Several of the hypothesized predictors of change were not supported. With regard to intervention length, all but one intervention was delivered in 1 or 2 sessions. Although alcohol risk reduction may be achieved by relatively short interventions (cf.(Samson & Tanner-Smith, 2015), this review could not fully evaluate the potential effects of more intensive multi-session interventions. Evaluations of multiple-session interventions are needed to test whether longer interventions could optimize outcomes for mandated students.

We also did not find strong evidence for the superiority of face-to-face interventions. Because mandated students are more likely to drink more and express defensiveness regarding the circumstances of their referral (Barnett et al., 2004; Palmer et al., 2010), an intervention delivered by a skilled facilitator may be more engaging than a computer-delivered intervention. However, delivery mode was not consistently associated with intervention effects. Furthermore, few trials have evaluated some of the promising computer-delivered interventions (e.g., e-CHUG), indicating the need for additional investigation of the relative value of face-to-face versus computer delivery with mandated students.

This quantitative review updates the qualitative review of Barnett and Read (2005), which summarized 16 studies of mandated alcohol interventions. Our larger sample of 31 studies reflects the growth of the field with more controlled outcome studies, larger samples, more evaluations of different intervention modalities, more detailed descriptions of intervention components, and longer follow-ups. The intervention literature now contains many examples of comparative efficacy of different interventions. Despite these improvements, few comparisons have been replicated, limiting our ability to compare one specific intervention to another. A growing number of studies now support the efficacy of commercially-available intervention protocols, including both face-to-face and computer-delivered modalities. The absence of cost-effectiveness analyses remains a glaring gap in the literature.

To advance the field, we make the following suggestions for future research. First, given the evidence that mandated interventions help to change behavior, systematic exploration is needed to determine which interventions optimize outcomes and for whom (Kraemer, Frank, & Kupfer, 2006). Heterogeneity in ES suggests that differences in efficacy exist among interventions or in the response of certain individuals to a given intervention, and these predictors of response need to be identified. Second, systematic exploration of the efficacy of intervention components is warranted. The predictors of change that were identified in this meta-analysis can guide the design and evaluation of comparative efficacy trials that test optimal combinations of intervention components. Third, given the decay of ES over time, future studies should evaluate booster sessions or other strategies to enhance maintenance, and trials need to incorporate both short-term and longer-term follow-ups to evaluate the maintenance of intervention gains.

We acknowledge several limitations of this meta-analysis. First, the small number of no-treatment or waitlist control groups in studies of mandated students constrained between-groups analyses. Thus, our conclusions are limited to establishing that any intervention produces better outcomes than no intervention; as the literature matures, it will be important to test which interventions are more efficacious. Second, alternative interpretations of the observed within-group change cannot be ruled out; these include short-term self-initiated change after the sanction event (Carey, Henson, et al., 2009; Hustad et al., 2011; Morgan et al., 2008), maturation, or regression to the mean (Campbell & Stanley, 1963). We note, however, that regression to the mean cannot account for the observed associations of ES with intervention content (e.g., goal setting). Therefore, not all of the observed changes can be explained by temporal or statistical artifact. Third, intervention components are often delivered in combination, making it difficult to discern which components or combination might explain outcomes. We encourage the use of dismantling designs to confirm if the components identified by this review are the active ingredients of change. Fourth, our analyses of predictors of change should not be interpreted as moderator analyses; we identified correlates of the degree of change associated with participation in an alcohol intervention, not differential responsiveness to one intervention or another. Fifth, missing data, a common problem in meta-analysis (Cooper & Hedges, 2009), limited our ability to fully test predictors of interest. Finally, we acknowledge the limitations of the primary level studies included in this meta-analysis; these limitations include not providing the reasons for mandate, predominance of brief interventions, limited descriptions of intervention content, brief follow-ups, reliance on self-report assessment, and limited reporting of non-drinking outcomes (e.g., retention or additional alcohol violations).

In summary, mandated alcohol interventions for students who violate campus alcohol policy reduce alcohol-related risk relative to no intervention. Participation in a mandated intervention is associated with lower alcohol consumption and fewer alcohol-related problems. Change over time was seen across most consumption and problems measures, indicating that the process of participating in mandated interventions has a broad impact on drinking. However, intervention-related gains weakened over time, with almost no benefit apparent one year later. This meta-analysis revealed that certain components were associated with better outcomes (i.e., goal setting, personalized feedback, expectancy challenge) whereas alcohol education was associated with poorer outcomes. Future intervention research might identify which interventions and components optimize outcomes for mandated students, develop strategies to maintain initial gains, evaluate interventions of varying intensity, and provide information regarding cost-effectiveness.

Public Health Significance.

Providing mandated interventions to college students who have violated alcohol policy has significant but modest effects on reducing drinking and alcohol-related problems.