Brief motivational interventions for college student drinking may not be as powerful as we think: An individual participant-level data meta-analysis

Abstract

Background

For over two decades, brief motivational interventions (BMIs) have been implemented on college campuses to reduce heavy drinking and related negative consequences. Such interventions include in-person motivational interviews (MIs), often incorporating personalized feedback (PF), and stand-alone PF interventions delivered via mail, computer, or the Web. Both narrative and meta-analytic reviews using aggregate data from published studies suggest at least short-term efficacy of BMIs, although overall effect sizes have been small.

Method

The present study was an individual participant-level data (IPD) meta-analysis of 17 randomized clinical trials evaluating BMIs. Unlike typical meta-analysis based on summary data, IPD meta-analysis allows for an analysis that correctly accommodates the sampling, sample characteristics, and distributions of the pooled data. In particular, highly skewed distributions with many zeroes are typical for drinking outcomes, but have not been adequately accounted for in existing studies. Data are from Project INTEGRATE, one of the largest IPD meta-analysis projects to date in alcohol intervention research, representing 6,713 individuals each with two to five repeated measures up to 12 months post-baseline.

Results

We used Bayesian multilevel over-dispersed Poisson hurdle models to estimate intervention effects on drinks per week and peak drinking, and Gaussian models for alcohol problems. Estimates of overall intervention effects were very small and not statistically significant for any of the outcomes. We further conducted post hoc comparisons of three intervention types (Individual MI with PF, PF only, and Group MI) vs. control. There was a small, statistically significant reduction in alcohol problems among participants who received an individual MI with PF. Short-term and long-term results were similar.

Conclusions

The present study questions the efficacy and magnitude of effects of BMIs for college drinking prevention and intervention and suggests a need for the development of more effective intervention strategies.

INTRODUCTION

For over two decades, brief motivational interventions (BMIs) have been implemented on college campuses to reduce heavy drinking and related consequences among students (for reviews, see Carey et al., 2007; Cronce and Larimer, 2011; Larimer and Cronce, 2007). BMIs aim to increase students’ motivation to reduce harmful drinking patterns through increased awareness and salience of personal patterns of use, expectancies regarding alcohol’s effects, peer-use normative beliefs, alcohol-specific risk factors, experience of use-related consequences, and protective behavioral strategies to limit harm. The Brief Alcohol Screening and Intervention for College Students (BASICS) program (Dimeff et al., 1999; Marlatt et al., 1998) is the prototypical BMI for college drinking. Based on motivational interviewing (Miller and Rollnick, 1991) and Marlatt’s cognitive behavioral relapse prevention model (Marlatt and Gordon, 1980), BASICS originally targeted high-risk drinkers and consisted of two in-person, individual 45- to 60-minute sessions, delivered by doctoral-level clinical psychologists or graduate students. In session one, students completed an assessment interview to identify relevant topics for discussion in session two, establish rapport, and introduce participants to the task of self-monitoring their alcohol use between sessions. Participants also completed additional assessments via paper questionnaires. In session two, participants reviewed personalized feedback (PF) generated from their responses on the previous assessment within the framework of a motivational interviewing therapeutic style (Miller and Rollnick, 2013).

BMIs have been adapted in a variety of ways, including 1) reducing in-person sessions from two to one, 2) targeting other student subpopulations including fraternity and sorority members, athletes, and first-year students, 3) provision of stand-alone PF by mail, Web, or on-site computer without an in-person motivational interview (MI), 4) broader drinking inclusion criteria (e.g., light or nondrinkers) for prevention purposes, 5) MIs delivered in small group format, without PF (GMI), and 6) using peers rather than professionals to facilitate in-person sessions. Given the flexibility of BMIs along with their designation as a Tier 1 prevention strategy by the National Institute on Alcohol Abuse and Alcoholism (NIAAA), signaling evidence of effectiveness (NIAAA, 2002), BMIs have become a popular choice among college administrators nationwide. In fact, according to Nelson et al. (2010), BMIs are used at 62% of schools that utilize an empirically-supported prevention program targeting alcohol.

Despite the increasing acceptance of BMIs on college campuses, the basic efficacy of BMIs has varied across studies. Notably, systematic reviews have indicated that effects on any given outcome varied across studies and assessment time points, with limited consistency across studies for which outcomes were significant at which time points (Cronce and Larimer, 2011; Larimer and Cronce, 2007). In a quantitative meta-analysis, the estimated effect sizes of any individually-focused interventions that included various components (e.g., MI, skills training, alcohol expectancy challenge, alcohol education and/or exercise) were generally small (Carey et al., 2007). With the exception of peak blood alcohol concentration immediately post-intervention (d = .36), the average effect size ranged from d = .11 to .22 across all outcomes. In addition, a recent meta-analysis (Foxcroft et al., 2014), which focused on MI vs. no MI for alcohol misuse among adolescents and emerging adults between the ages of 15 and 25, similarly concluded that the effect sizes were quite small. Thus, these existing meta-analyses and narrative reviews collectively suggest that if effects of college alcohol interventions are statistically significant, they are generally small, even in the short-term.

The research syntheses described above, however, leave room for a more fine-grained investigation of the efficacy of BMIs in reducing alcohol use and problems for college students. The narrative reviews by Cronce and Larimer (2011) and Larimer and Cronce (2007), although systematic and comprehensive, were not designed to provide overall effect size estimates and associated ranges of precision. Carey et al. (2007) combined effect size estimates from individually-focused interventions based on a number of theoretical orientations, not solely MI, while excluding data from group-based MIs or unpublished studies. Furthermore, more recent BMIs were not included in their meta-analysis, although about a third of individual-focused interventions during the period from 1984 to early 2010 were published between 2007 and 2010 (Cronce and Larimer, 2011). The analysis by Foxcroft et al. (2014) exclusively focused on MIs delivered for adolescents and emerging adults in various settings. Many important BMI adaptations for college students were either excluded or analyzed as alternative controls for MIs in Foxcroft et al.

College students are a special population associated with unique developmental and situational challenges. It is important to examine the efficacy of BMIs that have been specially adapted for this population. A meta-analysis using individual participant-level data (IPD) represents an ideal approach for examining the efficacy of BMIs for college students in a controlled statistical analysis that accounts for study heterogeneity and the unique characteristics of alcohol use data.

Meta-analysis using Individual Participant-level Data

IPD meta-analysis (also called Integrative Data Analysis [IDA; Curran and Hussong, 2009]) is a newly emerging large-scale research synthesis method in the field of behavioral research. IPD meta-analysis differs from meta-analysis using aggregated data (AD; e.g., effect size estimates) in terms of its challenges and capabilities. Of the many advantages of IPD meta-analysis over AD meta-analysis (see Cooper and Patall, 2009; Curran and Hussong, 2009; Simmonds et al., 2005), the opportunity to utilize more appropriate, flexible analytic techniques is perhaps most significant for the current article. Alcohol outcome measures (e.g., drinks per week) are often highly skewed with many zeroes that are more appropriately modeled using count regression methods such as zero-inflated or hurdle regressions (Atkins et al., 2013). With such highly skewed outcomes, effect sizes based on means and standard deviations may yield biased estimates, whereas IPD meta-analysis can fit regression models that are appropriate for the outcome distribution.

Furthermore, IPD meta-analysis using pooled data from multiple trials provides an ideal means for clarifying whether the efficacy of BMIs differs by demographic subgroups (e.g., gender) or type of BMIs (Brown et al., 2013; Cooper and Patall, 2009). These questions have remained largely unanswered due to the limitations of individual studies, which may be powered to assess efficacy, but not moderation effects (e.g., Mun et al., 2009). A typical AD meta-analysis utilizes study-level, not individual-level, information, and therefore individual-level variables cannot be evaluated as moderators. The present study included gender and baseline levels of alcohol use and problems, which have been frequently cited as potential individual-level characteristics that moderate BMI efficacy. However, there have not been any definitive, consistent findings from either single studies or meta-analyses.

The current study evaluates the overall intervention effect size of BMIs for reducing weekly drinking quantity, peak drinking quantity, and alcohol-related negative consequences using IPD meta-analysis. The three specific goals are to: 1) evaluate the overall efficacy of BMIs, 2) determine if efficacy differs by BMI type (i.e., individual MIs with PF, PF only, and GMI), and 3) examine whether intervention effects are moderated by gender or baseline alcohol use or problems.

METHODS

Studies and Sample

Data come from Project INTEGRATE, one of the largest IPD meta-analysis projects in behavioral treatment research to date, and the first of its kind for college alcohol interventions. Specifically, the Project INTEGRATE data set includes IPD (N = 12,630; 42% men; 58% first-year or incoming students at baseline) from 24 independent BMI trials conducted over the past two decades that aimed to reduce college student alcohol use and related harm. For more details on original sample demographics and individual study design features, see Mun et al. (2014).

To be included in the current study, original studies had to include a BMI condition (i.e., sessions were facilitated based on principles of MI [Miller and Rollnick, 2013], and/or PF was delivered to participants) and a control condition¹, and participants had to be randomly assigned to conditions. Seventeen studies met these criteria (see Table 1).² Non-BMI alcohol intervention conditions (e.g., alcohol education, alcohol expectancy challenge) or any other unique conditions included in these 17 studies were excluded as they were highly heterogeneous and not meaningfully grouped as a single category.

Table 1.

BMI Characteristics across Studies Included in the Present Study (N = 6,713 from 17 Studies)

Study	Reference(s)	Randomized Group (n)	BMI Type	Follow-up schedulea (in months)	Reported Results from Original Studies for Typical Drinks per Week, Peak Number of Drinks, and Alcohol-related Problems
2	White et al. (2008)	92	PF	2	Drinks per week and peak drinks - not availableb Alcohol-related problems (nsd = 0.07)
		102	Control
7.1	Fromme and Corbin (2004)	81	GMI	1	Drinks per week (ns) Peak drinks - not available Alcohol-related problems (ns)
		23	Control
7.2	Fromme and Corbin (2004)	218	GMI	1, 6
		111	Control
8a	Larimer et al. (2007)	551	PF	12	Drinks per week (sig) Peak drinks - not available Alcohol-related problems (ns)
		551	Control
8b	Larimer et al. (2007)	781	PF	12	No published results are currently available.
		806	Control
8c	Larimer et al. (2007)	133	PF	12	No published results are currently available.
		165	Control
9	Lee et al. (2009)	86	GMI	3, 6	No published results are currently available.
		87	MI + PF
		92	PF
		91	Control
10.1	Baer et al. (2001)	157	MI + PF	12	Drinks per week and peak drinks – not available Alcohol-related problems – not available
		164	Control
11	Walters et al. (2007)	150	PF	2, 3	Drinks per week (sig at 2 mos.; ns at 3 mos.) Peak drinks – not available Alcohol-related problems (ns)
		160	Control
12	Wood et al. (2007)	75	MI + PF	1, 3, 6	Drinks per week and peak drinks – not available Alcohol-related problems (sig, ds = 0.32 at 3 mos. and 0.29 at 6 mos.)
		80	Control
13/14	Murphy et al. (2004) Murphy et al. (2001)	54	MI + PF	3, 6, 12	Study 13: MI + PF vs. PF Drinks per week (ns) Peak drinks – not available Alcohol-related problems (ns) Study 14: MI + PF vs. control Drinks per week (ns) Peak drinks – not available Alcohol-related problems (ns)
		27	PF
		24	Control
15	LaBrie et al. (2008)	139	GMI	1, 2, 3	Drinks per week (sig) Peak drinks (ns) Alcohol-related problems (ns)
		98	Control
16	LaBrie et al. (2009)	156	GMI	1, 2, 3, 6	Drinks per week (sig) Peak drinks (sig) Alcohol-related problems - not available.
		126	Control
18	Martens et al. (2010)	94	PF	1, 6	Drinks per week (ns) Peak drinks - not available Alcohol-related problems (ns)
		100	Control
20	Larimer et al. (2001)	217	MI + PF	12	Drinks per week (sig, d = 0.42) Peak drinks - not available Alcohol-related problems (ns)
		244	Control
21	Walters et al. (2009)	74	MI + PF	3, 6, 12	MI + PF vs. Control: Drinks per week (sig at 3 and 6 mos.) Peak drinks – not available Alcohol-related problems (ns at 3 mos., sig at 6 mos.) PF vs. Control: Drinks per week (ns) Peak drinks – not available Alcohol-related problems (ns) MI + PF vs. PF: Drinks per week (sig at 3 and 6 mos.) Peak drinks – not available Alcohol-related problems (ns)
		63	PF
		70	Control
22	Wood et al. (2010)	229	MI + PF	12	Drinks per week and peak drinks - not available Alcohol-related problems – not available
		242	Control

MI + PF = Individually-delivered Motivational Interview with Personalized Feedback, PF = Stand-alone Personalized Feedback. GMI = Group Motivational Interview.

^aThis reflects time points utilized in the current study design, and may not reflect the total number of assessment points in the original study design, either short-term or long-term.

^bUnavailable because a particular outcome measure was either not assessed in the original study or unreported in the publication. In the case of study 10.1, there were statistically significant intervention effects on changes in peak drinks and problems over two years, which exceeds the follow-up period targeted in the present study.

The resulting intervention conditions were 1) individually-delivered MI + PF, 2) stand-alone PF, or 3) GMI. Ultimately, there were 21 BMI conditions (and 17 controls) across 17 studies (N = 8,275 at baseline), including seven MI + PF conditions, nine PF conditions, and five GMI conditions. These three treatment conditions were derived and labeled based on detailed coding of intervention materials and characteristics (see Ray et al., 2014 for information on BMI characteristics and fidelity checks). With the exception of one study (study 7), GMIs did not provide a PF. Facilitators delivering MI + PF sessions or GMIs either received MI training or had previous experience delivering BMIs.

At the participant level, data from a total of 6,713 students (38.5% men) who had at least one follow-up were analyzed in the analysis of the efficacy of BMIs. The majority of the sample identified as White/Caucasian (75.1%), with 12.9% Asian, 5.4% Hispanic, 2.2% Black/African American, and 3.9% mixed race or other. Approximately two-thirds of participants (64.6%) were first-year or incoming students³ and 4.4% were mandated to participate due to violations of their university's substance use policies.

Both number and timing of follow-up assessments within 12 months post-intervention varied across studies. Forty-seven percent (n = 8) of studies included a single post-baseline assessment, 24% (n = 4) two assessments, 24% (n = 4) three assessments, and 6% four assessments (n = 1). In terms of timing, 24% (n = 4) of studies conducted only short-term follow-ups (1–3 months post baseline), 35% (n = 6) only long-term assessments (6–12 months post baseline), and 41% (n = 7) both short- and long-term follow-up assessments.

Measures

As this study utilized IPD across multiple studies that were independently conducted, we first ensured that measures were comparable across studies (Mun et al., 2014). Harmonization was used for the two measures of alcohol use quantity analyzed in the present study because they were almost identical across studies (see below). For alcohol-related problems, we utilized hierarchical, two-parameter logistic item response theory (IRT) models. Technical details of the IRT models are fully reported in Huo et al. (2014). In addition, we conducted an additional IRT analysis that accommodated differential item functioning (DIF) across studies, and compared the results. Results suggested that the derived latent trait scores from the original IRT analysis were essentially invariant to DIF items across studies (see Mun et al., 2014 for detail). The rank orders of individuals within and across studies were preserved (rs ≥ 0.95), and the rank order of the studies in terms of their mean scores were largely the same.

Alcohol use quantity

Two indices of alcohol use quantity were utilized: typical drinking and peak drinking. Typical drinking was the total number of drinks consumed in a typical week, and peak drinking was the maximum number of drinks consumed on a given occasion, both in the past month (typical and peak drinking in the last three months for studies 8a, 8b, and 8c). A more detailed description of these two measures is provided in the Supporting Information.

Alcohol-related problems

Across the original studies, six different alcohol problems scales were used (see Supporting Information). We used latent trait scale scores estimated from hierarchical, two-parameter logistic IRT models for multiple groups to establish comparable alcohol problems trait scores for all participants across studies and time (Huo et al., 2014; Mun et al., 2014).

Demographic variables

Demographic variables included gender (men vs. women), race (non-White vs. White), first-year student status (first-year vs. non first-year), and mandated status (mandated vs. volunteer). We used these demographic variables as well as baseline alcohol use measures as covariates in all analyses.

Data Analyses

The present analyses focus on estimating intervention effect sizes using IPD in a single integrated analysis per outcome. The Project INTEGRATE data have repeated measures nested within individuals who are nested within studies; thus, multilevel models (MLMs) are a natural analytic framework (Gelman and Hill, 2006; also called mixed effect or hierarchical linear models). In IPD meta-analysis using multilevel modeling, study-level random effects can be used to derive study-specific treatment effect sizes.

One important challenge with the current data set is that some studies had multiple, active intervention conditions (e.g., Study 21; Walters et al., 2009), whereas others did not. To fit the combined data within a standard MLM framework, it would be necessary to either pool active intervention conditions into a single condition within a study or remove one or more conditions, effectively reducing each study to a two-arm randomized clinical trial and resulting in information loss. This issue has been noted in the wider meta-analysis literature (see, e.g., Gleser and Olkin, 2009). To tackle this challenge, the present analyses used Bayesian MLM estimation via Markov chain Monte Carlo (MCMC) sampling (see Gelman and Hill, 2006, for an introduction on Bayesian MCMC methods with multilevel data) to derive effect sizes for all original intervention conditions without collapsing multi-arm interventions into a single treatment group. Study-specific effect sizes were estimated by specifying study by intervention condition (i.e., randomized groups) as the highest level in the MLMs. The intervention effect size estimate and its confidence interval were then calculated by utilizing the mean and highest probability density interval of the posterior distribution of the difference in random effects between the estimate of each intervention condition and its corresponding control within study. These study-specific intervention effects were then pooled to calculate overall intervention effects and BMI condition-specific effects (see the Supporting Information for details).

Two outcomes of interest (i.e., drinks per week and peak drinking) were count variables with notably skewed distributions and many zeroes. For these two outcomes we used a hurdle model (Atkins et al., 2013), a two-part regression model, which fits two sub-models simultaneously: a) a logistic regression for zeroes vs. not zeroes, and b) a zero-truncated over-dispersed Poisson regression for the distribution of nonzero values. Thus, for these two outcomes there were two sets of results corresponding to treatment impact on likelihood of any drinking (i.e., logit model) and mean drinking given any drinking (i.e., zero-truncated count model). We used Gaussian models for alcohol problems latent trait scores, which were reasonably normally distributed. Intervention effects were estimated in three separate models, one for each outcome.

Moderators of intervention effects were also considered, including baseline values of the outcome (i.e., drinks per week, peak drinking, or alcohol problems) and gender. A binary intervention indicator (intervention vs. control) and its interaction with (1) baseline outcome values and (2) gender were included in three moderation models, one for each of the outcomes. These moderation analyses are analogous to meta-regressions in meta-analysis using AD (van Houwelingen et al., 2002). All analyses were conducted in R v3.1.0 (R Core Team, 2014), and we used the MCMCglmm package (Hadfield, 2010) for Bayesian generalized linear mixed models.

RESULTS

Descriptive Statistics

Figure 1 displays frequency distributions of number of drinks per week pooling across all post-baseline assessments. Across all studies, an average of 7.7 drinks per week was reported with no drinking reported 30% of the time. With the exception of studies 12, 13/14, and 21, the modal number of drinks per week was zero. Figure 1 also illustrates substantial between-study variability. The mean number of drinks per week varied from 3.2 to 21.6 drinks across studies, and the percentage of no drinking varied from 0 to 66%.

Figure 1.

Frequency Distributions of Post-Baseline Drinks per Week by Study.

Figure 2 depicts the average weekly number of drinks when drinking⁴ by intervention condition at each assessment point for each study. Although there is notable variability across studies in the mean number of drinks when drinking, very little evidence for intervention vs. control differences appears to exist within studies. With the exception of study 16, mean drinks per week for intervention and control conditions were generally similar over time with overlapping 95% confidence intervals (CIs). Plots for other outcomes and plots for proportion of individuals reporting any drinking – that approximates the logistic submodel of the hurdle model – generally showed the same pattern, suggesting little to no intervention effects in IPD.

Figure 2.

Mean Number of Drinks per Week when Drinking by Study and Intervention vs. Control. Multiple intervention groups within the same study are combined for illustrative purposes.

Intervention Outcomes

As described in the Data Analyses section, we examined study-specific effects, as well as overall effects of intervention. Figure 3 shows a forest plot of study-specific intervention effects on the probability of drinking (i.e., any drinking vs. no drinking) and number of drinks (i.e., mean drinks given some drinking) in a typical week. Odds ratios (ORs) and rate ratios (RRs) below 1.0 correspond with lower likelihood of any drinking and lower levels of drinking (given any drinking), respectively, for the intervention (compared to the control) condition. Intervention effects on the probability of any drinking varied by study from an OR of 0.37 to 1.37, with 16 of 21 intervention conditions having point estimates of reduced likelihood of any drinking relative to control. Intervention effects on the number of drinks when drinking also varied by study from a RR of 0.84 to 1.09, with 13 of 21 intervention conditions with point estimates of reduced quantity of drinking (when drinking) relative to control. However, none of these intervention effects was statistically significant, with one exception (study 16, GMI) in the probability, but not quantity, of drinking. Overall, BMIs were associated with statistically nonsignificant, small reductions in the probability of any drinking (OR = 0.79, 95% CI = [0.61, 1.10]) and drinking quantity when drinking (RR = 0.96, 95% CI = [0.91, 1.00]). As seen in Table 2 (top row), similar null findings were found for peak drinking and alcohol problems.

Figure 3.

Forest Plot of Intervention Effects by Study for Any Drinking and Quantity of Drinks when Drinking. MI + PF = Individually-delivered Motivational Interview with Personalized Feedback, PF = Stand-alone Personalized Feedback. GMI = Group Motivational Interview. No. of drinks = Number of drinks when drinking.

Table 2.

Intervention Effects Aggregated across Follow-ups 1 to 12 months Post-Baseline

	Drinks per Week						Peak Drinks						Alcohol Problems
	Logit			Count			Logit			Count
	OR	95% CI		RR	95% CI		OR	95% CI		RR	95% CI		B	95% CI
BMIs vs. Control (Overall Effect)	0.79	0.61	1.10	0.96	0.91	1.00	0.82	0.60	1.15	0.98	0.94	1.01	−0.02	−0.05	0.02
Specific BMI Condition
– MI + PF vs. Control	0.75	0.47	1.20	0.94	0.89	1.02	0.76	0.41	1.44	0.96	0.90	1.01	−0.06	−0.12	−0.01
– PF vs. Control	0.84	0.57	1.19	0.98	0.91	1.04	0.93	0.66	1.36	0.97	0.93	1.02	0.02	−0.03	0.07
– GMI vs. Control	0.78	0.49	1.31	0.96	0.87	1.03	0.73	0.43	1.25	0.99	0.93	1.06	−0.01	−0.09	0.05

MI + PF = Individually-delivered Motivational Interview with Personalized Feedback, PF = Stand-alone Personalized Feedback, GMI = Group Motivational Interview, OR = Odds Ratio, RR = Rate Ratio, BMI = Brief Motivational Intervention. Results highlighted in bold are statistically significant.

We conducted post hoc contrasts of the three specific BMI conditions (MI + PF, PF, and GMI) compared to control (Table 2; three bottom rows). There were no statistically significant intervention effects for the drinks per week and peak drinking outcomes. However, there was a statistically significant, small reduction in alcohol problems among participants who received a combination of individual MI + PF, compared to their control counterparts (B = −0.06, 95% CI = [-0.12, −0.01]).

Covariate estimates for all three outcomes are provided in a table in the Supporting Information. No baseline drinking was associated with lower probability of any drinking and lower mean number of drinks. Baseline alcohol use quantity and being white were associated with higher probability of any drinking, and higher mean number of drinks both during their typical week and during their peak drinking occasion. Baseline alcohol problems were associated with higher levels of alcohol problems. First-year student status was associated with lower probability of drinking during a typical week, but a greater mean number of typical weekly drinking and peak drinks and higher levels of alcohol problems. Men, compared to women, drank more when drinking both during their typical week and during their peak drinking occasion, and had higher levels of alcohol problems. Volunteer students drank more when drinking during their typical week and reported higher levels of alcohol-related problems than mandated students.

Intervention Moderation Analyses

Table 3 summarizes moderation effects by gender and baseline alcohol measures for the post-baseline alcohol outcomes. Moderation analyses by gender indicated no evidence that the interventions were differentially effective for men vs. women for any of the alcohol outcomes when controlling for baseline alcohol measures. Moderation analyses by baseline alcohol measures also indicated no evidence that the interventions were differentially effective for drinkers vs. nondrinkers and across different levels of alcohol use and problems.

Table 3.

Intervention Effect Moderation Models of Post-Baseline Alcohol Outcomes

	Drinks per Week						Peak Drinks						Alcohol Problems
	Logit			Count			Logit			Count
	OR	95% CI		RR	95% CI		OR	95% CI		RR	95% CI		B	95% CI
Moderation by Gender
Intervention × Men vs. Women	0.82	0.50	1.34	1.03	0.94	1.12	0.69	0.39	1.18	1.00	0.93	1.06	−0.02	−0.09	0.04
Moderation by BL severity
Intervention × No BL drinking	0.86	0.50	1.55	0.98	0.86	1.14	0.79	0.42	1.62	1.04	0.92	1.15	–	–	–
Intervention × BL alcohol quantity	0.99	064	1.56	0.97	0.93	1.01	0.82	0.53	1.27	1.01	0.98	1.05	–	–	–
Intervention × BL alcohol problems	–	–	–	–	–	–	–	–	–	–	–	–	−0.02	−0.05	0.01

BL = Baseline, OR = Odds Ratio, RR = Rate Ratio. Covariate estimates for BL alcohol quantity represent the effect of a 1 SD increase in BL quantity. BL alcohol quantity measures were drinks per week and peak drinks at baseline for their respective outcome model. None of the moderated effects were statistically significant at p < .05.

Sensitivity Analyses

As seen in Figure 2, studies included different follow-up assessments, and, therefore, we also conducted a set of analyses that were stratified by short-term (up to 3 months) and long-term (6 to 12 months) assessments. Substantive results were identical to those reported above (and are available from the first author). In addition, studies 8a (n = 1,102) and 8b (n = 1,587) were the two largest studies (at the participant level), contributing a combined 40% of the analyzed sample. Sensitivity analyses were conducted to compare overall results when studies 8a, 8b, or both were excluded from the estimation of the overall intervention effect. The estimates of the overall effects of BMIs and the three specific intervention types on all outcomes were comparable to the reported results with either or both of the studies excluded from the models.

DISCUSSION

Findings from the current, IPD meta-analysis suggest that the efficacy of BMIs for reducing harmful drinking on college campuses is much less robust and smaller than believed. Results indicated no significant overall effect of BMIs on likelihood of any drinking at follow-up, nor on amount of alcohol consumed per typical week or per peak occasion for those who drank. There was also no overall intervention effect on alcohol-related problems, though evaluation of BMI types indicated in-person MI with PF had a small but statistically significant effect on reducing problems. Moreover, across all outcomes there was no evidence that overall BMI efficacy was moderated by either gender or baseline alcohol severity.

The lack of an overall significant intervention effect may seem surprising given the wealth of individual studies demonstrating BMI efficacy on at least some outcomes at some assessment time points (Cronce and Larimer, 2011; Larimer and Cronce, 2007). However, our results are relatively consistent with the results of Carey and colleagues’ (2007) traditional meta-analysis, which showed that evidence of the efficacy of college alcohol interventions is mixed with small overall effect sizes. However, the current study is different from Carey et al. in several aspects. First, we accounted for distributional properties of the data, notably the zero-inflated distributions, which cannot be incorporated into traditional meta-analyses using AD; this may partly account for the fact that, in contrast to Carey et al., our approach did not yield statistically significant overall intervention effects. In addition, we report adjusted mean differences controlling for the effects of individual-level covariates on outcomes, whereas typical AD meta-analysis report effect size estimates based on unadjusted mean differences.

Second, Carey and colleagues (2007) included published or in press studies, whereas the current analyses included data not incorporated in previous publications. Given that failed trials are less likely to be published (i.e., publication bias or the file drawer problem) and that studies with bigger effect sizes from small studies tend to be published more quickly (Tanner-Smith and Polanin, 2014), this difference may help to explain our findings. Third, the 38 intervention groups from 17 trials analyzed in this study were limited to BMIs for college students, whereas Carey et al. included some BMIs with other interventions; and Foxcroft et al. (2014) included heterogeneous interventions conducted in different settings and more heterogeneous samples (e.g., adolescents and noncollege emerging adults).

It is important to note that outcome measures tend to vary slightly across studies, and one cannot rule out the possibility that some of the significant findings in individual studies may have been chance findings due to multiple null hypothesis significance tests in the original studies. This observation can also be seen in Table 1. It is also relatively well known in the meta-analysis literature that it is easier to find significant effects in small samples than in a large, controlled study (Borenstein et al., 2009). These are ongoing issues in the clinical trials literature that have spurred the development of clinical trial registries (e.g., ClinicalTrials.gov) where researchers pre-specify outcomes and hypotheses to reduce chance results due to changes in methods and reporting (De Angelis et al., 2004). The IPD meta-analytic approach allowed us to evaluate a consistent set of outcomes using the same analytical model across all follow-ups across studies, yielding a clearer pattern of very small and mostly non-significant effects on three major outcomes.

Of three BMI types, only in-person MI with PF demonstrated a significant but small intervention effect on reducing alcohol-related problems. Although the magnitude of this effect (0.06) is quite small, the finding is consistent with results of the original BASICS research (Baer et al., 2001; Marlatt et al., 1998), which demonstrated a sustained effect of the BASICS in-person MI with PF on alcohol-related negative consequences.

The literature has been mixed regarding whether in-person MI with PF interventions yield advantages on drinking outcomes compared to stand-alone PF, as many trials of stand-alone PF have reported similar effect sizes to in-person studies (Doumas and Hannah, 2008; Doumas et al., 2009; see Walters and Neighbors, 2005 for a review). Relatively few individual studies have directly compared in-person to stand-alone PF, and those that have have often found no differences at short-term follow-up (Butler and Correia, 2009; Doumas and Hannah, 2008; White et al., 2007), although advantages of in-person MIs with PF have been found at longer-term follow-ups (Walters et al., 2009; White et al., 2007). Some of this variability may be the result of differences in therapist adherence to and competence in MI principles (Mastroleo et al., 2008; Tollison et al., 2008) as well as therapist background and training (Fromme and Corbin, 2004; Larimer et al., 2001), which may impact the effects of in-person MI. The current findings suggest that in-person MI with PF, similar to the original BASICS, may be necessary to produce significant effects on negative consequences.

The literature has also been mixed regarding gender as a moderator of intervention effects. In this study, gender was not a significant moderator of intervention effects, with men and women in BMIs reporting comparable reductions in typical drinks per week, peak drinking, and negative consequences. The current IPD meta-analysis included both prevention and intervention studies, with participants ranging from abstainers to heavy drinkers, with and without negative consequences of drinking at baseline. Interestingly, BMI efficacy was not moderated by baseline likelihood or severity of alcohol use or negative consequences. These findings suggest that while positive effects of BMIs in these analyses were quite small, these interventions are unlikely to have harmful effects even for light drinkers or abstainers. From a public health perspective, interventions with very small effects may nonetheless have population-level impact if they can be implemented broadly, inexpensively, and with little risk of harmful effects.

Limitations

There are several limitations that must be considered. First, the original studies reported in the current paper were not randomly selected for inclusion. Rather, only studies for which the original investigator was approached and willing to provide data to Project INTEGRATE were included. Therefore, although the included studies reflect a wide range of BMIs implemented on college campuses across the nation, results might not generalize to the broader pool of BMI studies. Also, all included studies were published in or before 2010, with several studies conducted more than a decade ago; thus, recent variations in BMIs, such as single-component BMIs or BMIs with new content topics, are not represented.

It is also important to note that unlike AD meta-analysis where methodological and clinical heterogeneity across studies in designs, measures, intervention groups or comparisons, and samples may or may not be captured by statistical tests of heterogeneity, and is typically glossed over in the name of standardized effect sizes, we explicitly tackled these differences at the operational level one study at a time. The end result is an alternative approach to large-scale research synthesis that is not bound by what is available in published reports; not subject to the limitations of conventional, study-level analyses; not thwarted by the complexity and challenges of establishing measurement invariance across a number of studies; and not daunted by the computational demands of analyzing individual assessments. Therefore, while the present study does not feature all available BMIs for college students in the field or a random sample of BMIs, it is as close an approximation as possible to the population-based effect sizes in a highly controlled and well-executed IDA.

Another consideration is that the current analyses collapsed across follow-ups to accommodate differences in study design rather than analyzing each time point separately as is typical for individual studies. However, an evaluation of short-term (1–3 months) and longer-term (6–12 months) outcomes separately yielded similar results. The current analyses also retained some of the limitations of the original trials, such as reliance on self-report of outcomes. The original studies also varied in intervention fidelity and therapist competence in (and adherence to) MI principles, as well as intervention content (see Ray et al., 2014), completion rates, and retention at follow-up (see Mun et al., 2014). These differences across BMIs can potentially be important moderators of intervention efficacy. Due to the relatively modest sample size for each intervention condition (e.g., seven MIs + PF, nine PFs, and five GMIs for typical weekly drinking) and also because of the fact that these potential moderators were confounded with one another, we did not examine them empirically. Note, however, that the effect sizes from all analyses were generally consistent across studies. Figure 3, for example, shows fairly consistent null findings across conditions and studies.

Conclusions and Recommendations

Despite limitations, the current findings are an important addition to the literature on efficacy of BMIs to reduce harmful drinking among college populations. Results suggest a need for caution in implementing BMIs on college campuses, particularly when adapting the original, two-session, in-person BASICS (Dimeff et al., 1999; Marlatt et al., 1998) to stand-alone PF interventions or GMI formats. During the last two decades, reliance on these altered interventions, which have reported small and typically only short-term effects on drinking in individual studies, may have posed an unintentional barrier to researchers’ and clinicians’ development of better intervention approaches. Even in-person, individualized MIs were associated with very small effects only on alcohol consequences in this study, indicating a need to carefully consider methods to enhance efficacy of this approach. While enthusiasm for lower-cost adaptations, including stand-alone Web, print, and text-message interventions, is high as campuses struggle to do more with less resources, the current findings suggest a need to cautiously temper this enthusiasm.

Given the wide variation in study design elements, particularly length of follow-up and the multiplicity of consumption and consequence outcomes, the field of college drinking prevention research would benefit from adherence to a consistent set of standards for randomized clinical trials to enable more effective comparison and interpretation of findings. Greater agreement on key outcomes relevant to BMI efficacy would be especially helpful in aiding the comparison of findings. It is also critical to stringently replicate findings regarding efficacy of BMIs, particularly adapted interventions, over longer periods of time. The field would also benefit from greater consideration of both theoretical mediators and moderators of intervention efficacy, as well as research to understand mechanisms through which moderators exert their impact on response to interventions. Relatedly, a clearer distinction between universal prevention trials (to prevent escalation in drinking) and indicated prevention (to reduce risky drinking among at-risk individuals) as well as delineation of appropriate inclusion criteria for these different intervention types would improve interpretation of study findings.

In sum, the current IPD meta-analysis represents a new way of evaluating the literature on efficacy of BMIs for college students. Results suggest it is imperative that we not be satisfied with demonstrating BMI efficacy on some outcomes at some time points for some students in some studies, but rather give concerted attention to understanding our failures as well as our successes, and developing the next generation of college alcohol prevention programs.