Brief Motivational Interventions don’t actually change motivation? A commentary on Tan et al.

Tan and colleagues (2023) conducted a two-stage, Individual Patient Data (IPD) Meta-Analysis that examined the effect of Brief Motivational Interventions (BMIs), relative to control conditions, on changes in motivation among four-year college students engaged in heavy alcohol consumption. The emphasis of this meta-analysis was not on the primary outcomes of these 15 clinical trials (19 total comparisons in studies conducted between 2001 and 2010), but on a central purported mechanism of this class of interventions. BMIs typically combine the relational and technical principles of Motivational Interviewing (MI; Miller & Rollnick, 2013) with the social comparator exercises of Motivational Enhancement Therapy (i.e., Personalized Normative Feedback [PNF]; Miller et al., 1992) to impact motivation to change a specified behavioral outcome. These interventions can be heterogeneous in approach, and in fact, the group of BMIs studied here included remote-delivered PNF interventions, MI plus PNF, and group-delivered MI. The authors should be commended for moving beyond the standards of this field that consider consumption and consequences as the most important outcome metrics for meta-analytic review. Instead, a central theorized change mechanism was examined, and spoiler alert, the answer to the question of impact is no or at least, mostly no. BMIs do not consistently influence college student motivation to change their drinking and/or reduce related harms. In the commentary that follows, I will consider three themes that arose for me when reading this study.

First, a certain personal disclosure is needed; I am a Project Integrate super fan. Project Integrate is a National Institute on Alcohol Abuse and Alcoholism funded meta-analysis that began in 2010 and has included several follow-ups and for certain iterations, renewals. The original study included data from 24 brief intervention trials, a sample of 12,630 participants (i.e., 42% men; 74% white, and 58% first-year students), and as of February of 2023, has yielded 64 publications related to intervention efficacy and effectiveness and meta-analytic methodology (Mun et al., 2015; https://www.unthsc.edu/school-of-public-health/integrate/). The current publication is another output of the project’s IPD approach. IPD is similar to Curran and Hussong’s (2009) Integrative Data Analysis with the key distinction that in two-stage IPD, data are aggregated to the study-level prior to meta-analytic synthesis. This has implications for the current study where the number of data-points available to the authors was 19 and this was coupled with clinical heterogeneity in the sample (PNF k = 8; PNF plus MI k = 6; GMI k = 5; Tan et al., 2023). This is a common limitation in meta-analyses of clinical trials where heterogeneity can be extensive and sample sizes can be relatively small. With that said, the two-stage IPD will maximize the available number of studies because it is not reliant on the published data and rather on the kindness of the study authors to provide the study dataset (or descriptive data on a specific set of variables/measures). The result is an increased sample size as well as reduced bias due to publication status (i.e., the increased likelihood that a given datapoint is available in the published literature when that datapoint shows efficacy for the experimental condition). Therefore, the current study has not solved this meta-analytic conundrum but is a meaningful improvement on the status quo.

Next, there is motivation as a construct that has shown mixed empirical support in addictions literature. In a recent meta-analysis across populations, health behaviors, and measures (e.g., stages of change, processes of change, and readiness to change), the overall pooled effect size across 76 studies was d = .41 and heterogeneous (Krebs et al., 2018). The measures examined were consistent with those studied in Tan and colleagues (2023) and these authors provide several helpful pieces of documentation for interpreting their data. For example, Table 2 describes measurement harmonization procedures, Figure 3 shows boxplots of effect size by measure, and the Appendix describes all measurement items. A close view of Figure 3 is a real challenge for any brain keen on pattern recognition. In other words, there is clear variability in baseline to follow-up changes with motivation sometimes increasing and sometimes decreasing in the experimental as well as control groups. One descriptive conclusion that can be drawn is that the magnitude of these shifts are small and this is similarly shown in the near-zero effect sizes across the three intervention delivery categories (as the authors note, a change in roughly half of a unit of the response scale on average; Tan et al., 2023). The other observation that can be made is that these students started and stayed low in their motivation, but understandably so when the item upper bounds of these measures describe the action stage of change (e.g., “I am actively working on changing my drinking habits”; McConnaughy et al., 1983). As such, how much should we expect motivation to change among primarily freshman college students engaging in essentially normative, yet risky alcohol consumption?

Finally, as a reader of this manuscript and viewing the overall null result, the immediate thought was – blame it on the control groups! However, different than clinical trials of more comprehensive interventions such as cognitive behavioral therapy (see e.g., Magill et al, 2019), the control groups in brief intervention trials are often homogenous and theoretically inert. In Tan and colleagues (2023) review, roughly 70% of trials used an assessment only control group while the remainder included assessment plus a pamphlet of information and maybe, resources. Therefore, we cannot blame it on the control groups (as can also be observed in Figures 3 and 4). Measurement reactivity is a methodological concern for substance use clinical trials (Clifford et al., 2007), but it would appear in this study, not in relation to measures of motivation among non-treatment seeking young adults. The authors also encoded the “motivational content” within each BMI intervention type (i.e., PNF, MI plus PNF, GMI) with a primary focus on decisional balance exercises and change discussions. However, all elements of brief motivational interventions, from the verbal style of MI (i.e., OARS) to the social comparator methods of a mailed PNF, are intended to impact motivation and as a result, we can infer these elements were not acting as intended on their purported mechanism of change.

There are implications for the field that arise from these three themes. With respect to bias due to publication status, it is heartening to say that rigor and transparency are a force in motion. Journals have increasingly requested data availability statements where authors provide direct access to their data or statements on how to gain access. This ideal was not the norm when Project Integrate began in 2010. Specific institutes at the National Institutes of Health (i.e., National Institute on Alcohol Abuse and Alcoholism) now require periodic data sharing to a public-facing archive for their funded research. Preregistration of studies requires authors to publish the study they planned at outset and be explicit when exploratory analyses were undertaken. These shifts in research ideology and practice will result in a larger number of available datapoints for future meta-analyses and likely, increasing meta-analytic reports such as this one that show null effects. With respect to motivation as a construct, it may be important to consider how our existing measures of motivation operate within linear statistical models. A linear model asks for constant change along units of a continuum but a shift from not thinking about changing to thinking about changing or at the other extreme, from preparing for change to actively changing might not correlate with linear changes in behavior from baseline to a particular follow-up. I not sure I have the answer here, but we may need to truncate the lower and upper bounds of motivation measures, or we at least need to consider where a person begins in relation to where we think they can go when planning analyses. Finally, the Tan and colleagues (2023) study and especially Figure 3 show us that treatment groups got a little better or worse on average and so did the control groups. The homogeneity of controls in this meta-analysis was a real strength with respect to the capacity to draw conclusions about the nature of the effect. Unfortunately, this was coupled with heterogeneity in the experimental groups and Tan and colleagues provided some outstanding sensitivity analyses to help us make sense of their results. The recommendation here is for other meta-analysts to follow this model.

Overall, the work of Tan and colleagues (2023) suggests that BMIs as a class of interventions may not consistently change motivation among college students who engage in alcohol consumption. The exception is the studies involving group-based MI that might have some advantage, but the magnitude of these effects was still quite small. The results shown here are potentially liberal as publication bias cannot be ruled out. These are also a portion of BMI trials from a certain timeframe. The motivation measures were heterogeneous, but all defined the upper bound of the construct as action to change drinking and this may not be a nuanced enough metric for this population. Finally, the control groups were homogenous and the results do not suggest measurement reactivity with respect to changes in student motivation.