Do Brief Motivational Interventions increase motivation for change in drinking among college students? A 2-step meta-analysis of individual participant data

Abstract

Background:

Brief Motivational Interventions (BMIs) are one of the most effective individually-focused alcohol intervention strategies for college students. Despite the central theoretical role of motivation for change in BMIs, it is less clear whether BMIs actually increase motivation to change drinking behavior. We conducted a two-step meta-analysis of individual participant data (IPD; 15 trials, N = 5,903; 59% women, 72% White) from Project INTEGRATE to examine whether BMIs increased motivation for change. The BMIs included individually delivered motivational interviewing with personalized feedback (MI+PF), stand-alone personalized feedback (PF), and group-based motivational interviewing (GMI).

Methods:

Different measures and responses used in the original trials were harmonized. Effect size estimates were derived from a model that adjusted for baseline motivation and demographic variables for each trial (step 1) and subsequently combined in a random-effects meta-analysis (step 2).

Results:

The overall intervention effect of BMIs on motivation for change was not statistically significant (standard mean difference [SMD]: 0.026, 95% CI: [−0.001, 0.053], p = .06, k = 19 comparisons). Of the three subtypes of BMIs, GMI, which tended to provide motivation-targeted content, had a statistically significant intervention effect on motivation, compared to controls (SMD: 0.055, 95% CI: [0.007, 0.103], p = .025, k = 5). In contrast, there was no evidence that MI + PF (SMD = 0.04, 95% CI: [−0.02, 0.10], k = 6, p = 0.20) nor PF increased motivation (SMD = 0.005, 95% CI: [−0.028, 0.039], k = 8, p = 0.75), compared to controls. Post hoc meta-regression analysis suggested that motivation sharply decreased each month within the first three months post-intervention (b = −0.050, z = −2.80, p = .005 for k = 14).

Conclusions:

Although BMIs provide motivational content and normative feedback and are assumed to motivate behavior change, the results do not wholly support the hypothesis that BMIs improve motivation for change. Changing motivation is difficult to assess during and following interventions, but it is still a theoretically important clinical endpoint. Further, the evidence cautiously suggests that changing motivation may be achievable, especially if motivation-targeted content components are provided.

Introduction

In the United States (US), over half (54.3%) of 18- to 25-year-olds report consuming alcohol in the past month (Substance Abuse and Mental Health Services Administration [SAMHSA], 2020). Within this age group, 52.5% of full-time college students ages 18 to 22 report drinking alcohol in the past month, and 33.0% report engaging in heavy episodic drinking (SAMHSA, 2020). Heavy alcohol use—including heavy episodic drinking— is associated with a wide range of negative consequences, including greater risk for academic problems, assault, alcohol use disorder, injuries, hospitalization, and death (Hingson et al., 2017a; Hingson et al., 2017b). To reduce alcohol use, several individually focused intervention strategies have been developed and deployed on college campuses (Larimer et al., 2022). These interventions are aimed at reducing alcohol consumption and alcohol-related negative consequences by changing students’ alcohol-related knowledge, attitudes, and behaviors.

Among these individually focused interventions, brief motivational interventions (BMIs) have shown effectiveness in reducing young adults’ alcohol use (Cronce et al., 2018; Mun et al., 2022b), alcohol-related problems (Huh et al., 2015, 2022; Jiao et al., 2020), and driving after drinking (Mun et al., 2022a). Many BMIs have been modeled after the Brief Alcohol Screening and Intervention for College Students (BASICS; Dimeff et al., 1999), which incorporates personalized feedback with motivational interviewing (Miller & Rollnick, 2002). The body of large-scale evidence from Project INTEGRATE indicates that the effect size of BMIs on typical alcohol consumption per week is an 8% difference (vs. controls) in the overall mean number of drinks through 6 months post intervention (Mun et al., 2022b); a 19% difference (vs. controls) in odds for driving after four+/five+ drinks (an approximately ˗0.12 standardized mean difference; Mun et al., 2022a); and a ˗0.031 reduction per month in latent trait scores for alcohol-related problems through 12 months post intervention for those who received in-person motivational intervention with personalized normative feedback profile (MI + PF vs. controls; Jiao et al., 2020). The reduction of 0.031 per month is 0.372 at 12 months, which can be understood as changing an average person from the 50^th percentile at baseline to the 35.49^th percentile at 12 months after the intervention. Furthermore, Project INTEGRATE has reported a ˗0.015 difference between stand-alone personalized feedback intervention (PF vs. controls) in the indirect effect on alcohol-related problems via protective behavioral strategies (Huh et al., 2022). Therefore, there is consistent evidence that BMIs change alcohol consumption and alcohol-related consequences.

How behavior change occurs in BMIs, however, has been relatively unclear in the literature. College students often have low motivation to change their drinking behavior; compared to other populations, they are less likely to recognize their own drinking problems while simultaneously overestimating their peers’ pro-drinking norms (Baer, 2002; Cox et al., 2019; Perkins et al., 2005). Through contrasting (a) students’ perceptions with the actual drinking patterns of peers and (b) students’ current drinking behaviors with desired life values and goals, it has been theorized that BMIs develop discrepancies – normative discrepancy and self-ideal discrepancy, respectively – to increase students’ awareness of alcohol-related negative consequences (Borsari & Carey, 2001; Miller & Rollnick, 2013; Neighbors et al., 2004), and motivation for change (Larimer et al., 2021; Miller & Rollnick, 2013). Despite being a critical mechanistic link theorized to explain the efficacy of BMIs, there is limited evidence on the effects of BMIs on young adults’ motivation to change their drinking behavior.

Findings from a few prior studies regarding the effects on motivation for change have been sparse and mixed (Barnett et al., 2007; Borsari et al., 2009; Murphy et al., 2010; Ostafin & Palfai, 2012). For instance, in a study evaluating the efficacy of a counselor-delivered BMI and a computer-delivered intervention with 225 mandated college students, Barnett et al. (2007) reported increased motivation from pre- to post-intervention but did not find significant differences in motivation between the BMI and the computer-based alcohol education program. In another study that analyzed data from three randomized trials implementing BMIs, Borsari et al. (2009) found no evidence of BMI effects on motivation compared to an assessment-only control in one trial (Carey et al., 2006) or to a feedback-only condition in another trial (Murphy et al., 2004). The BMI in the third trial did have significant beneficial effects on student motivation that persisted up to 6 weeks post-intervention compared to an assessment-only control (Borsari & Carey, 2000). In two other studies, Murphy et al. (2010) and Ostafin and Palfai (2012) reported increased motivation for change among college students after receiving a BMI (vs. a computer-based alcohol education program and an assessment-only control, respectively). However, the sample sizes of both studies were relatively small (n = 74 in study 1 and n = 133 in study 2 in Murphy et al. [2010], and n = 87 in Ostafin and Palfai [2012]). As Mun et al. (2015) have noted, individual studies with small sample sizes are more prone to biased findings than those from larger sample sizes studies or meta-analyses because it is common to find large effects in small studies, which often fail to replicate. In sum, it is unclear whether BMIs actually increase motivation for change among young adults, despite this being a purported mechanism of action. The sparse empirical evidence on BMI effects on young adults’ motivation for change may be partially due to “selective reporting” in the literature, where results are selectively withheld from publication due to statistical non-significance (Chan et al., 2004; Dwan et al., 2013). A meta-analysis of individual participant data (IPD) can effectively overcome this barrier and examine whether BMIs affect motivation in a large pool of research trials conducted on college student populations. The current study aimed to address this knowledge gap by utilizing a large pool of IPD from multiple primary BMI trials with a variety of measures of motivation designed to reflect stages of behavior change from a transtheoretical model perspective (Prochaska et al., 1992).

The Current Study

We conducted an IPD meta-analysis using data from Project INTEGRATE (Mun et al., 2015) – the largest IPD meta-analysis project in alcohol intervention research for college students. The IPD included in 24 studies of Project INTEGRATE are reasonably representative of the trials conducted between 1998 and 2009 (see Mun et al., 2015 for a detailed description). Of the studies, multiple independent trials assessed motivation using different measures among samples with diverse participant characteristics. Heterogeneous outcome measures and samples from different trials are considered strengths as well as challenges of IPD meta-analysis. The use of various measures of motivation for change across the included trials allows for a more nuanced examination of whether an overall intervention effect exists and how robust the effect is. Similarly, the diversity of samples and intervention/comparison groups in an IPD meta-analysis permits the examination of the robustness of intervention effects across different participants and settings by directly modeling the sources of heterogeneity in IPD or by examining variation in aggregate estimates in a meta-analysis with many studies. Further, IPD meta-analysis provides an opportunity to assess whether effects are replicated across trials. We hypothesized that BMIs (vs. control) would be associated with greater motivation for change in drinking behavior among college students.

The current study utilized a two-step or two-stage meta-analysis approach as opposed to a one-step or one-stage meta-analysis (Simmonds et al., 2005, 2015). The “one-step” approach typically involves a simultaneous analysis of all IPD in a single multilevel analysis. For example, Huh et al. (2015, 2019, 2022) show examples of a one-step approach to probe intervention effects and mediation effects. This one-step approach, which is closely related to an integrative data analysis (IDA) approach proposed by Curran and Hussong (2009), allows researchers to explicitly address the multilevel data structure, such as clustering within studies and participant-level missing data. However, the one-step IPD meta-analysis approach assumes that all studies have measures and designs that can be made comparable. This can be a prohibitive requirement, especially in alcohol intervention research, given the heterogeneity in the measurement of motivation to change outcomes and other sources of heterogeneity across trials. Even if this requirement is met or circumvented, it can be computationally intensive and challenging to implement (Burke et al., 2017).

A second approach to IPD meta-analysis is a “two-step” approach (Simmonds et al., 2015), in which participant-level data are first analyzed and aggregated at the study level (e.g., study-specific estimates of the treatment effect). These study-level data derived from IPD are then combined in the second step using traditional meta-analytic methods to estimate the overall effect and between-study heterogeneity. This two-step approach is more common (e.g., Jiao et al., 2020; Mun et al., 2022a, 2022b; White et al., 2015) and has certain advantages over the one-step approach, especially for complex models, given increased flexibility in model specification and decreased computational burden. We, therefore, used the two-step IPD meta-analysis approach in the current study, accommodating between-study heterogeneity in a random-effects meta-analysis model. We subsequently examined BMI subtype differences as well as short-term vs. long-term follow-up differences in effect sizes. Finally, we conducted a meta-regression analysis to examine to what extent the overall BMI effect on motivation decreased over time.

Materials and Methods

Participants

Of the 24 studies with available IPD from Project INTEGRATE (Mun et al., 2015; N = 12,630 participants at baseline), 15 trials met the inclusion criteria of the current study: (a) a randomized trial with at least two arms – a BMI arm and a control group arm, and (b) at least one outcome measure representing participants’ motivation for change in drinking behavior, measured at baseline and a follow-up within 12 months post-intervention (see Table 1). At baseline, IPD from 7,433 participants across 15 trials were available. Of those, 1,530 participants did not have outcome data at the first follow-up within 12 months post-intervention, resulting in a final sample of 5,903 participants (41% men, 72% White, and 59% first-year student) across the 15 trials (see Figure 1).

Table 1.

Description of the Individual Participant Data at Baseline from the 15 Studies Included in the Current Study (N = 8,067; 35 Randomized Groups and 15 Studies)

Study	Reference	BMI Type	Randomized Group n	Follow-up (Months)	%White	% Male	% First Year
2	White et al. (2008)	PF Control	111 119	2	67	71	63
7.1	Fromme et al. (2004)	GMI Control	100 24	1	73	76	57
7.2	Fromme et al. (2004)	GMI Control	317 135	1	58	59	37
8a	Larimer et al. (2007)	PF Control	736 750	12	86	33	49
8b	Larimer et al. (2007)	PF Control	1094 1061	12	61	41	47
8c	Larimer et al. (2007)	PF Control	303 297	12	82	38	36
9	Lee et al. (2009)	GMI MI + PF PF Control	97 101 100 101	3	72	38	100
10.1	Baer et al. (2001)	MI + PF Control	174 174	12	84	46	100
11	Walters et al. (2007)	PF Control	185 198	2	63	59	100
12	Wood et al. (2007)	MI+PF Control	84 83	1	93	47	3
14	Murphy et al. (2001)	MI+PF Control	30 25	3	96	44	38
15	Labrie et al. (2008)	GMI Control	155 108	1	51	0	100
16	LaBrie et al. (2009)	GMI Control	161 126	1	56	0	100
18	Martens et al. (2010)	PF Control	102 113	1	89	25	33
20	Larimer et al. (2001)	MI + PF Control	318 369	12	82	54	74
21	Walters et al. (2009)	MI + PF PF Control	76 68 72	3	85	36	42

Notes. Study 7 is a single study with two distinct subsamples. Study 8 is a multi-site study (Studies 8a, 8b, 8c). MI + PF = individually delivered in-person motivational interviewing intervention with personalized feedback, GMI = group motivational interviewing intervention, BMI = brief motivational intervention, PF = stand-alone personalized feedback intervention.

Figure 1.

PRISMA IPD Flow Diagram

Notes. IPD come from Project INTEGRATE (Mun et al., 2015). Data flow at the identification, screening, and eligibility stages is not applicable (unknown).

Intervention and Control Groups

BMIs included in Project INTEGRATE were grouped into three categories: individually delivered motivational interviewing with personalized feedback (MI+PF), stand-alone personalized feedback (PF), and group-based motivational interviewing (GMI). All BMIs were delivered individually in person, in group, via mail, or via computer/online. Stand-alone personalized feedback was printed and handed to participants (Study 2), mailed to participants (Study 8a, 8b, and 8c), or emailed or shown on a computer to participants (Studies 9, 11, 18, and 21). Although the BMIs differed slightly in content topics and personalization levels, they had considerable content overlap across studies (see Ray et al., 2014; Mun & Ray, 2018). The content in each intervention group per study was checked and coded by two independent coders. Of the 20 identified intervention components in BMIs (see Ray et al., 2014), we defined motivation-targeted intervention content as content that included decisional balance information on weighing the pros and cons of alcohol use or change (henceforth referred to as decisional balance) or content related to changing one’s alcohol use, such as readiness to change or personal goal setting (referred to as change discussion). These components were coded based on how they were provided (0 = did not include; 1 = general content; 2 = personalized content; 3 = both general and personalized content; see Figure 2).

Figure 2.

Intervention Exposure to Motivation-related Content by Group per Study

Notes. Decisional balance: Information on weighing the pros and cons of alcohol use or barriers to change. Personal content includes feedback on a participant’s pros and cons or barriers to change, whereas general content includes information on general pros and cons or typical barriers that students face. Change discussion: content that relates to changing one’s alcohol use behaviors, including readiness to change or goal-setting. Personal content addresses a participant’s readiness to change or personal goals about changing; general content includes information such as the importance of goal setting.

Controls in the current study were mostly assessment-only controls (Studies 2, 7–12, 14, 20, 21). Studies 15 and 16 provided a handout to control participants that contained information on alcohol use and alcohol-related consequences. Study 18 provided controls with a handout containing brief educational information about alcohol-related negative consequences and calories (i.e., stating that alcohol has no nutritional value). Study 20 provided a single-page handout with tips to reduce risk from alcohol. These “control” groups were similarly classified with assessment-only controls in an analysis based on their content components (Mun & Ray, 2018).

Measures

Motivation for change.

The primary outcome of interest was motivation for change in drinking behavior. Studies used three different measures: (1) the 12-item Readiness to Change Questionnaire (RCQ; Heather, Rollnick, & Bell, 1993; see Supplemental Table 1 for items, (2) the 13-item University of Rhode Island Change Assessment (URICA; McConnaughy et al., 1983; see Supplemental Table 2 for items, and (3) the Contemplation Ladder (Biener & Abrams, 1991; see Supplemental Table 3 for the item). Table 2 summarizes all measures of motivation for change used in the 15 trials. No trials used more than one measure of motivation for change, so no items were common in all trials.

Table 2.

Variation in the Measurement of Motivation for Change in Drinking Behavior Across Studies

Study	Questionnaire	Number of items	Original response categories (values in original studies)		Possible range of total scale scores
2, 8a, 8b, 8c, 9, 11, 18, 21	Readiness to Change Questionnaire (RCQ)	12	Strongly Disagree Disagree Undecided Agree Strongly Agree	(−2) (Study 18: 1) (−1) (Study 18: 2) (0) (Study 18: 3) (1) (Study 18: 4) (2) (Study 18: 5)	−24–24 (Study 18: 12–60)
7.1, 7.2, 10.1, 14, 20	University of Rhode Island Change Assessment (URICA)	13 (Study 14: 5)	Strongly Disagree Disagree Undecided Agree Strongly Agree	(1) (2) (3) (4) (5)	13–65 (Study 14: 5–25)
12, 15, 16	Contemplation Ladder	1	I’ve never needed to change my drinking (Study 12: No thought of changing) Sometimes I think about drinking less (Study 12: Think I need to consider someday) I have decided to drink less (Study 12: Think I should change, but not quite ready) I am already trying to cut back on my drinking (Study 12: Start to think about how to change my drinking patterns) My drinking has changed; I now drink less than before (Study 12: Take action to change [e.g., cutting down])	(1) (3) (Study 12: 2) (5) (Study 12: 3) (7) (Study 12: 4) (10) (Study 12: 5)	1–10 (Study 12: 1–5)

Follow-up schedule.

Most trials had 1 or 2 follow-ups. The first follow-up assessment occurred 1–3 months post intervention in 10 studies (67%). Five trials (33%) had only one assessment at 12 months post intervention.

Demographic information.

Demographic variables included sex at birth (1 = male vs. 0 = female), race (1 = White vs. 0 = non-White), and first-year student status (1 = first-year vs. 0 = non-first-year).

Data Preparation and Analysis

Before data analysis, all IPD were checked for errors, outliers, and statistical assumption violations. First, we checked whether any raw items needed to be reverse-coded to create a scale score for motivation for change. We checked codebooks and also examined item-to-total correlations and descriptive statistics. Second, four different scales and different response options used in the primary trials were harmonized across trials (Table 2, see also Supplemental Tables 1–4 for scale details, and Supplemental Table 6 for Cronbach’s α for the motivation measure for each trial).

Next, we calculated Cohen’s d, standardized mean difference (SMD), a standardized effect size measure that reflects the difference in means on a continuous outcome variable between intervention and control groups at the first follow-up. Due to the various measures of motivation across the trials that ranged in measurement scales (see Figure 3 and Supplemental Table 5), it was necessary to use the SMD so that effect sizes from different trials could be combined meaningfully.

Figure 3.

Motivation for Change in Original Scale by Group per Study at Baseline (N = 7,433) and Follow-up (N = 5,903)

Note. Supplemental Table 5 reports all values shown in this figure. RCQ = The Readiness to Change Questionnaire, URICA = The University of Rhode Island Change Assessment, URICA* = A 5-item URICA, LADDER = The Contemplation Ladder. X-axis = study. Empty round symbols indicate control groups. Filled triangle symbols = PF, filled square symbols = GMI, and filled diamonds = MI + PF. The symbols connected with a grey line indicate baseline and follow-up data.

Because some trials showed baseline differences in motivation for change and demographic characteristics across randomized groups, in the first step of the IPD meta-analysis, we estimated intervention effect sizes within each study while accommodating baseline motivation scores and demographic covariates. This IPD approach is advantageous over aggregate data meta-analysis because derived estimates from the same model have the same interpretation and also because estimates are covariate-adjusted. The full model at the study level can be shown as:

$$\text{POST\_MO}{\text{T}}_i={\text{b}}_0+{\text{b}}_1{\left(\text{MI}+\text{PF}\right)}_i+{\text{b}}_2\text{P}{\text{F}}_i+{\text{b}}_3\text{GM}{\text{I}}_i+{\text{b}}_4\text{BL\_MO}{\text{T}}_i+{\text{b}}_5\text{WHIT}{\text{E}}_i+{\text{b}}_6\text{MAL}{\text{E}}_i+{\text{b}}_7\text{FIRSTY}{\text{R}}_i+{\text{e}}_i,$$

where $i$ indexes participant, b₀, b₁, … b₇ are regression coefficients, and e_i is a participant-specific residual error term. For the intervention groups MI + PF, PF, or GMI compared to controls, (MI+PF)_i, PF_i, and GMI_i are dummy-coded variables that indicate random allocation to MI + PF, PF, or GMI, respectively (each coded 1), compared to controls (coded 0). The regression coefficients b₁, b₂, and b₃ quantify the covariate-adjusted average difference between participants who received (1) MI + PF, (2) stand-alone PF, or (3) GMI, respectively, compared to control participants within each study. The covariate BL_MOT_i refers to baseline motivation scores, and the covariates WHITE_i, MALE_i, and FIRSTYR_i refer to the baseline demographic characteristics: sex (1 = man vs. 0 = woman), race (1 = White vs. 0 = non-White), and first-year student status (1 = first-year vs. 0 = non-first-year). POST_MOT_i refers to the post-intervention motivation score for participant i. In the second step of the IPD meta-analysis, we used random-effects meta-analysis models to obtain the pooled overall effect size and meta-regression to examine whether the intervention effects on motivation differed across BMI subtypes and which BMI subtype had stronger intervention effects on motivation in short-term vs. long-term follow-ups.

The analysis in the current study was limited to complete cases. The participants included did not differ statistically at baseline from those excluded due to missing data at follow-ups. We included key covariates, such as baseline covariates, in the analysis so that the resulting inferences account for these variables, assuming that missing responses were related to observed data but not unobserved data (i.e., missing at random [MAR]). In alcohol intervention research, MAR has been assumed reasonable in prior synthesis studies (Huh et al., 2022; Mun et al., 2009). Data preparation was conducted using SAS 9.4 (SAS Institute Inc., Cary, NC), IBM SPSS Statistics (Version 26), and Microsoft Excel (Version 16); meta-analyses were conducted using the package ‘metafor’ version 3.0–2 (Viechtbauer, 2010) for R version 4.1.2 (R Core Team, 2021). Statistical significance was set at p < .05, and analyses were two-tailed. Annotated R code and data can be accessed in the online repository at doi:10.17632/ndzhdv86dg.1(Tan et al., 2023b).

Results

Measure Harmonization

Motivation for change was assessed with different questionnaires and/or items across the included trials. Even when trials used the same questionnaire, they sometimes used different response options. For example, at the item level, items were sometimes worded slightly differently (“No thought of changing” vs. “I’ve never needed to change my drinking” in the Contemplation Ladder). And at the questionnaire level, trials sometimes used different versions of the same questionnaire. For instance, Study 14 used a subset (Items 1, 3, 5, 6, 8) of the items in the full URICA. For item response values, all RCQ responses were assigned scores ranging from −2 to 2, except that in Study 18, the responses ranged from 1 to 5. In Study 10, pre-contemplation items (Items 1, 2, 5, 10) had already been reversely coded in the original primary study (see Supplemental Tables 1–4). Both the RCQ and the URICA used a 5-point Likert scale ranging from “Strongly Disagree” to “Strongly Agree.” The Contemplation Ladder used a 1-to-10 scale from “No thought of changing” to “Take action to change,” except that a 1-to-5 scale was used in Study 12.

We carefully reviewed codebooks for item content and response item scores to harmonize these measures. All raw item variables were checked, and if necessary, some of the items were recoded. Examples included reverse-coding Items 1, 5, 10, and 12 for studies that used the RCQ (except Study 10, which was already reverse-coded). In addition, response options for Study 18 were recoded to make them consistent with other studies that used the RCQ. Responses to the Contemplation Ladder in Study 12 were also appropriately changed (see details in Supplemental Tables 1–4).

Checking Data and Descriptive Analysis

Figure 3 (see also Supplemental Table 5) shows the means and standard deviations of motivation for change scores for intervention and control groups for each included trial at baseline and follow-up. Figure 3 shows the between-study heterogeneity across different measures and baseline imbalance across groups in Study 7.2. We also examined funnel plots to detect any potential publication bias due to the suppression of non-significant findings and found no concerning evidence of possible publication bias (Supplemental Figure 1).

Overall BMI on Motivation for Change

Figure 4 shows a forest plot summarizing the study-level effect of BMIs on motivation for change and the overall effect of BMIs on motivation for change in drinking behavior. The overall effect from the model adjusted for baseline motivation and demographic characteristics was not statistically significant, SMD = 0.026, 95% CI: [−0.001, 0.053], p = 0.06, across k = 19 comparisons (study-specific results of model-based intervention effects are available upon request). The estimated amount of total heterogeneity ${\tau }^2$ = 0.001, with a ratio of total heterogeneity to total variability, I² statistic = 28.1%, indicating a small to moderate between-study heterogeneity. Figure 4 presents the different degrees of efficacy across BMI types and trials, with 13 out of 19 effect sizes in the direction of BMIs increasing motivation for change compared to the control groups.

Figure 4.

Forest Plot of Motivation for Change in Drinking from the Model Adjusted for Baseline Motivation for Change and Demographic Variables

Note. MI + PF = individually delivered in-person motivational interviewing intervention with personalized feedback, GMI = group motivational interviewing intervention, PF = stand-alone personalized feedback intervention, SMD = standardized mean difference.

BMI subtypes and motivation for change.

We subsequently examined whether BMI subtypes differed in their effects on motivation for change. Although there was no statistically significant subgroup difference in SMDs, GMI statistically significantly increased motivation for change compared to controls (SMD = 0.055, 95% CI: [0.007, 0.103], k = 5, ${\tau }^2$ = 0.000, I² = 0.00%, p = 0.02). In contrast, there was no evidence that MI + PF (SMD = 0.04, 95% CI: [−0.02, 0.10], k = 6, ${\tau }^2$ = 0.002, I² = 37.26%, p = 0.20) nor PF increased motivation (SMD = 0.005, 95% CI: [−0.028, 0.039], k = 8, ${\tau }^2$ = 0.001, I² = 20.22%, p = 0.75), compared to controls.

Supplemental Figure 2 shows all effect sizes from the unadjusted models in the first stage of the IPD meta-analysis in a forest plot. All statistical conclusions were the same, except that confidence intervals tended to be wider, which is expected. In addition, the combined effect size of GMI from the unadjusted models was larger (SMD = 0.19, p = 0.01), which was somewhat expected due to the model differences.

Intervention content components targeted for motivation.

Upon examining the intervention content provided in each of the intervention groups, we found that GMIs tended to provide both decisional balance and change discussion (see Figure 2). This motivation-targeted content included both general information (such as typical barriers students might face) and content underscoring the importance of goal setting and personalized information (e.g., barriers to change specifically related to the student). Most interventions (12 out of 19 intervention groups) did not use a decisional balance. Four GMIs (Studies 7.1, 7.2, 15, and 16) included decisional balance activities that were both personalized and general. Change discussion was more common as it was provided in most studies with two exceptions (Study 9: PF and Study 18).

Time since Intervention and Motivation

Finally, we examined whether the effects of BMIs on student motivation varied according to the post-intervention follow-up time. The results showed that studies with short-term (within the first three months) and long-term (12 months) follow-ups did not have statistically significantly different effect sizes, b = 0.002, z = 0.05, k = 19, ${\tau }^2$ = 0.001, I² = 32.08%, p = 0.96. However, meta-regression analysis among the studies with a short-term follow-up indicated that there was a statistically significant decrease in the intervention effects over time within the first three months post-intervention, b = −0.050, z = −2.80, k = 14, ${\tau }^2$ = 0.000, I² = 0.000%, p = 0.01.

Discussion

The current study is the first large-scale meta-analysis evaluating the intervention effect of BMIs on motivation for change in drinking among college students. Facilitated by IPD, we verified raw data, checked baseline balance between randomized groups, and made appropriate adjustments for variation in participant characteristics across 19 comparisons in 15 trials. In this way, we ensured that the necessary assumptions were met for proper estimation and inference and that the same interpretation of effect sizes could be made across heterogeneous trials in a meta-analysis.

The findings from the current study did not support the primary hypothesis that BMIs increase motivation for change in drinking among college students. However, a subsequent subgroup analysis suggests that GMIs led to significantly improved motivation compared to control groups. Although MI + PF interventions yielded effects on motivation that were of similar magnitude to that observed for GMIs, the effects for those types of BMIs were not statistically significant, which could be in part due to the limited statistical power in these subgrouped analyses. Nevertheless, the evidence suggests that if there is an effect on participant motivation, it is likely to be small, equivalent to a 0.06 standard deviation improvement. Using a standard deviation of ~8.5 for the RCQ scale scores in Supplemental Table 5, this improvement of SMD = 0.06 translates to about a half unit of the response scale. For example, it would represent an improvement of 0.5 from Unsure (0) toward Agree (1) in response to all RCQ questions, such as “I enjoy my drinking, but sometimes I drink too much.” This small effect of BMIs on motivation for change among college students is consistent with the findings in a large meta-analysis of treatment trials (Magill et al., 2018). Further, this small effect size may explain the previously reported inconsistencies among study findings (Borsari & Carey, 2000; Carey et al., 2006; Murphy et al., 2004; Murphy et al., 2010; Ostafin & Palfai, 2012).

Given the importance of motivation in behavior change theories (DiClemente and Velasquez, 2002; Miller and Rollnick, 2013) and BMIs (Larimer et al., 2021), an open challenge to the field is how best to strengthen BMIs, so that young adults’ motivation for change is increased while simultaneously reducing drinking motives. Young adults, including college students, are often unmotivated to participate in alcohol-focused interventions (see Murphy et al., 2022, for a scoping review). This is likely because they have established drinking motives that are stable in their 20s and 30s (Windle et al., 2018). Recent studies suggest that drinking motives distinguish high-intensity from non-high-intensity drinking at both the event level (Patrick & Terry-McElrath, 2021) and the individual level (White et al., 2016).

Our post hoc findings suggest that decisional balance and change discussion are potential components to increase motivation. However, these components would be easier to implement during in-person sessions (vs. computerized or online sessions), which require considerable funding, time, and human resources. Where an individually delivered in-person MI session is not practical for motivation-focused content, a GMI that incorporates these exercises may be viable as a more cost-efficient yet effective alternative.

Although BMIs tend to share many of the same intervention components (Ray et al., 2014), BMI subtypes may operate through different mechanisms and thus vary in their effects on different outcomes. For example, stand-alone PF interventions may be efficacious through different therapeutic paths rather than changing motivation. It may be because, with minimal in-person contact, PF focuses more on providing personalized feedback on drinking norms and other helpful feedback, such as protective behavioral strategies, rather than addressing motivation, which may require more interaction and processing. In prior research from Project INTEGRATE, we found that PF was more effective in improving use of protective behavioral strategies, which then helps reduce alcohol-related problems (Huh et al., 2022). PF interventions often require fewer resources to deliver and thus may be easier to scale up in their implementation, especially with newer technological adaptations. Thus, PF interventions are often more practical than in-person interventions (Cronce et al., 2018). How to improve motivation when feedback is delivered through the Web or an app may require additional considerations for better engagement and outcome. Recent studies that provided synchronous teleconferencing (e.g., Zoom) or text-based intervention have demonstrated feasibility and preliminary effectiveness (e.g., Gex et al., 2022). With the COVID-19 pandemic, no-contact in-person meetings have proliferated and have been generally accepted (Mark et al., 2022; Oliveira et al., 2021). Therefore, future trials may explore supplementing PF with an additional “as-needed exposure” to improve motivation for college students in a stepped-care approach (Murphy et al., 2022; Ray et al., 2023).

In previous Project INTEGRATE studies, intervention effects on alcohol consumption (Mun et al., 2022b) and driving after drinking (Mun et al., 2022a) wane over time following intervention completion, which is common among behavioral interventions (Tanner-Smith and Lipsey, 2015). Thus, we expected that motivation might present similar decreasing intervention effects over time within the first 12 months post-intervention. Surprisingly, the findings from the subgroup analysis suggest no evidence of different effects across time. However, this should be cautiously interpreted because the follow-up schedule and study membership were confounded due to the limited sample size at the study level. Nevertheless, there was a significant average decrease of 0.05 in SMD every month in the first three-month post-intervention. Given the small effect size of BMIs (i.e., an SMD = 0.026), this precipitous drop suggests that the motivation level decreased quickly. Hence, there may be a sensitive window of opportunity in which participants may be receptive to intervention and motivated to initiate behavior changes. This “motivated” time window may help intervention researchers determine the timing for additional personalized support for independently sustainable behavior changes. This also suggests that periodic booster sessions, including digitally delivered boosters, may be worth considering at key follow-up periods to promote more sustained effects over time (Barnett et al., 2004; Braitman et al., 2020).

Limitations and Future Directions

The current study had some limitations. First, the IPD sample was not systematically or probabilistically sampled from the entire body of literature. It is, therefore, uncertain how representative the findings are in the broader BMI and brief alcohol intervention literature. Obtaining IPD representative of the entire population is difficult, if not impossible, given numerous structural and procedural barriers to data sharing (Levis et al., 2021; Ventresca et al., 2020). Furthermore, simple random sampling is rare, even in original primary studies. On a related point, even with this large, pooled IPD sample, we could not disentangle all potential sources of confounding due to the limited number of studies across design variations, such as delivery modes of PF and different measures of motivation. However, given the effect sizes across studies, unaccounted systematic influences on effect sizes are expected to be small.

Second, the current analyses retained some of the limitations of the original trials, specifically, the limitation of original measures for motivation for change. The URICA and the RCQ were originally designed to reflect different stages of motivation. Hence, they are inherently multi-unidimensional and may not be ideal for assessing motivation quantitatively. We reverse-coded item responses to approximate a unidimensional, trait-level motivation. More empirical work would be needed to better assess motivation and examine whether measures are invariant across major racial/ethnic groups and sexes and also across time. For example, a recent exploratory factor analysis study to examine invariance suggests that the URICA may not be invariant across three months (Richards et al., 2022). A thorough investigation of the measures of motivation is beyond the scope of the current study. Nonetheless, the development of new measures informed by emerging theories of behavior change and the refinement of the existing measures may help fill the existing knowledge gap about the nature of motivation and change following an intervention.

Third, the frequency and timing of assessing motivation may not have been ideal for capturing motivation that is fluid. Kaysen et al. (2009) showed that motivation does considerably fluctuate week to week among a sample of female college students, accounting for 28% of the total variability. They also showed that although people in the intervention group were more motivated to change than controls, drinking the week before negatively affected peoples’ motivation to change. More broadly, change in motivation may occur relatively quickly within a single session or immediately following the presentation of feedback. Furthermore, once drinking behavior changes have occurred, the individual may no longer feel additional changes are needed. Thus, motivation may appropriately decrease over a follow-up period, or an individual may experience an unsuccessful change attempt and a concomitant reduction in motivation due to discouragement. For example, a recent effectiveness trial among adults experiencing homelessness showed that a motivation-guided, just-in-time intervention delivered via ecological momentary assessment reduced drinking (Walters et al., 2022), suggesting how changing motivation could be utilized to provide a tailored, repeated intervention. Assessing and responding to motivation as a dynamic variable rather than a static indicator of individual differences may help our understanding of how interventions change motivation and, ultimately, drinking behaviors.

Fourth, in the current IPD analysis context, some studies used a single-item measure, while others used the URICA and the RCQ. To overcome the heterogeneity in measures, we used harmonization procedures. Harmonization is an essential preparation step for IPD meta-analysis to make measures comparable across different studies or assessment time points for subsequent synthesis (Huo et al., 2015; Mun et al., 2015, 2016, 2019). In the current study, no items were shared across all trials, which is needed for item equating. Each study used only one of the measures to assess motivation, with none of the measures sharing similar response ranges or anchor points. This makes the heuristic or quantitative harmonization of the measures difficult, especially with single-item measures. This data condition makes it impossible to analyze data in a one-step meta-analysis or IDA. We overcame this challenge by conducting a two-step IPD meta-analysis, which is a prevailing approach to IPD meta-analysis; recent simulation studies indicate that the results from one- vs. two-step IPD meta-analysis generally converge (e.g., Huh et al., 2023).

With sufficient item overlap across primary studies, it may be possible to conduct an item-based, more granular analysis in the future. Compared to traditional scale score-based approaches, novel item-level analysis, such as the cognitive diagnosis modeling (CDM) method used in recent studies (Tan et al., 2023a; Liang et al., 2023), may be feasible to calibrate item parameters better and characterize participants cross-sectionally and longitudinally by taking into account other related behaviors of interest under certain data situations (e.g., large samples). The CDM approach may be a promising future tool for brief alcohol interventions to classify participants at the individual level and examine intervention effects across a broad range of related health behaviors because alcohol misuse commonly co-occurs with other mental health problems. The CDM-informed intervention approach may help develop “precision” alcohol intervention programs.

Finally, we acknowledge that some comparisons were nested within the trial when primary trials had multiple intervention arms. Though it is common in meta-analysis to analyze the multiple arms from the same trial separately, it violates the independence assumption when the analysis shares the same control group for the multiple arms within the trial. However, its impact on the reported results is expected to be limited because only two contributing trials had multiple intervention arms.

Conclusions

Although more work is necessary to fully evaluate the mechanisms of change of BMIs and brief alcohol interventions more broadly, the current study suggests that not all BMIs may motivate change unless specific exercises are included to address motivation. While this finding may be disappointing given that BMIs are associated with reduced drinking (Mun et al., 2022b) and motivation is the stated pathway (Dimeff et al., 1999), motivation is likely a multifaceted and dynamic construct that involves various aspects of salience, efficacy, and timing (e.g., “ready,” “willing,” and “able”). Measures with a better “bandwidth,” which cover these aspects of motivation, may be needed. The current study also highlights the difficulty of assessing motivation over time and disentangling temporal associations. When asking about changing motivation among college students, the timing of measurement may be critical. It is also possible that motivation does not play the same role among young drinkers as it does among older drinkers or those in treatment. Finally, changing motivation among young adults may be possible when carefully developed intervention content components are provided. Although BMIs share commonly assumed mechanisms of behavior change and critical intervention components, there still remains a great deal of heterogeneity in this family of brief alcohol interventions. The current study utilized a fine-grained approach to data synthesis to highlight how BMIs may help improve motivation, particularly for young adults who are motivated to drink.