When Less is More and More is Less in Brief Motivational Interventions: Characteristics of Intervention Content and their Associations with Drinking Outcomes

Abstract

Brief motivational interventions (BMIs) that aim to reduce alcohol use and related problems have been widely implemented in college settings. BMIs share common principles, but vary in specific content. Thus far, the variation in content has not been thoroughly understood in relation to intervention outcomes. The present study addressed this gap by examining variation in breadth of BMI content (i.e., total number of components covered), the extent to which content was personalized to participants, and the interaction between breadth and personalization in relation to treatment outcomes. Data (N = 6,047 participants across 31 separate BMI conditions) came from an integrative data analysis (IDA) study featuring individual-level data from a broad sample of 24 BMI studies. Participants were assessed at baseline and at least one follow-up point, conducted up to 12 months post baseline. Structural equation modeling revealed a significant interaction effect between breadth and personalization of BMI content on alcohol use and related problems at the long-term follow-up (6-12 months) but not at the short-term follow-up (1-3 months). Results indicated that “more is better” for reducing both alcohol use and related problems when BMIs were highly personalized to participants. For less personalized BMIs, coverage of more components was associated with increases in both alcohol use and problems. Findings point to the importance of strategically designing BMIs to maximize their impact on drinking outcomes in college students.

Risky alcohol use continues to be a major public health issue for colleges and universities throughout the United States. Recent epidemiological research indicates that 40% of college students nationwide reported being drunk in the previous month, and 37% reported at least one heavy drinking episode (defined as drinking five or more drinks in a row) in the last two weeks (Johnston, O’Malley, Bachman, & Schulenberg, 2013). College drinking is associated with serious negative consequences including sexual assaults, accidents and injuries, academic problems, brain impairment, and death (Hingson, Zha, & Weitzman, 2009; White & Rabiner, 2012). Thus, there is a clear need for effective programs to mitigate hazardous drinking and related harm. Brief motivational interventions (BMIs; e.g., the Brief Alcohol Screening and Intervention for College Students [BASICS; Dimeff, Baer, Kivlahan, & Marlatt, 1999]) have been widely implemented in college settings, and have demonstrated small but significant reductions in heavy drinking and/or related problems (for reviews, see Carey, Scott-Sheldon, Carey, & DeMartini, 2007; Carey, Scott-Sheldon, Elliott, Garey, & Carey, 2012; Cronce & Larimer, 2011). Nonetheless, important characteristics of BMI content that contribute to BMI efficacy have not been fully elucidated (Cronce & Larimer, 2011; Larimer & Cronce, 2007; Martens, Smith, & Murphy, 2013). The present study addressed this gap by examining how characteristics of BMI content, including the number of components covered and the extent to which content was personalized, related to drinking outcomes.

Variability in BMI Implementation and Content

BMIs are brief motivational interventions based on a harm reduction approach that are typically facilitated based on principles of Motivational Interviewing (Miller & Rollnick, 2013), and are often guided by the use of personalized feedback to motivate students to change their drinking behavior (National Institute on Alcohol Abuse and Alcoholism, 2002). The feedback profile and/or conversations that form the crux of this approach center around components such as personal patterns of alcohol use, normative beliefs about other students’ drinking, alcohol-specific consequences, alcohol expectancies, and protective behavioral strategies to limit drinking and related harm (e.g., BASICS; Dimeff et al., 1999). However, actual characteristics of the implementation of BMIs over the past two decades have been quite heterogeneous. BASICS, for example, was originally designed as a two session intervention, delivered to heavy drinking students in person via professional clinicians (Baer, Kivlahan, Blume, McKnight, & Marlatt, 2001; Dimeff et al., 1999; Marlatt et al., 1998). Over the years BMIs were adapted to be delivered via mail or computer, or other similar means (Carey et al., 2007; Walters & Neighbors, 2005; White, 2006), to extend reach and improve cost-effectiveness. Researchers have also examined ways to more efficiently implement in-person BMIs, by testing the efficacy of a group BMI format (LaBrie, Huchting, et al., 2008; LaBrie et al., 2009), as well as peer-delivered BMI sessions (Fromme & Corbin, 2004; Larimer et al., 2001; Mastroleo, Turrisi, Carney, Ray, & Larimer, 2010). Although some support exists for these variations, more research is needed to determine what characteristics of BMIs lead to the best outcomes. One step towards this goal is to systematically examine the intervention content, to see whether variation in BMI content leads to changes in drinking outcomes, after controlling for study design features.

In comparison to the large body of research on implementation characteristics, research on the content of implemented BMIs is surprisingly limited, and thus it is not clear what characteristics of BMI content may be responsible for changes in drinking outcomes. A few recent studies have tried to identify specific components related to BMI efficacy (e.g., Carey et al., 2007; Carey et al., 2012; Martens et al., 2013; Miller et al., 2013), however, it is difficult to isolate the effect of one component (e.g., protective behavioral strategies) when it is typically delivered in the context of several others (e.g., alcohol use patterns, normative beliefs, consequences, expectancies, etc.). Moreover, the number and type of components covered are not always consistent from one study to the next (see Carey et al., 2007; Carey et al., 2012; Miller et al., 2013). For example, in a recent review of 43 different feedback profiles (Miller et al., 2013), some components were consistently covered (i.e., alcohol use, normative beliefs), whereas inclusion of other components ranged widely across interventions, from approximately 30% (e.g., decisional balance, alcohol expectancies, local resources) to 70% or greater (e.g., level of intoxication, risk factors, didactic information). Similarly, Carey et al. (2007, 2012) reported considerable variability in component coverage in two separate meta-analytic studies. Accordingly, in this study, we chose to focus on previously unexplored characteristics of BMI content in relation to study outcomes, including total number of components covered and the extent to which components were personalized to participants.

Breadth and Personalization of BMI Content

In terms of the total number of BMI components covered (breadth of content), little is known about the optimal number of components to include. Acknowledging there is little empirical evidence addressing this issue, Walters and Neighbors (2005) posited several possibilities, including that the effect of components may be additive, suggestive of a “more is better” approach. This view is consistent with the broad prevention literature that highlights comprehensiveness when designing prevention programs (Borkowski, Smith, & Akai, 2007). Moreover, several studies suggest that adding other components to a BMI, such as a parent-based intervention or session focused on increasing substance-free activities and academic engagement, can improve BMI outcomes (Murphy et al., 2012; Turrisi et al., 2009; Wood et al., 2010).

In contrast, some research has suggested that, in the context of BMIs, “less is more” (e.g., Kulesza, Apperson, Larimer, & Copeland, 2010, 2013). In other words, simpler, more straightforward single- or focused-component interventions may be preferred, and there may be no advantage or even a disadvantage of covering additional components. In fact, a closer look at recent reviews and meta-analyses suggests that both comprehensive BASICS-type feedback and single-component personalized normative feedback interventions have similar effect sizes (Carey et al., 2012; Walters & Neighbors, 2005), with no advantage of the more comprehensive approaches. Relatedly, Kulesza et al. (2010, 2013) reported that a 10-minute BASICS session focused primarily on alcohol use, perceived norms, and protective behavioral strategies did not differ from a 50-minute session, in which a wide range of content was discussed, in producing drinking reductions at 1-month follow-up, and both were significantly better than control. Taken together, these studies suggest that adding additional components or length to brief interventions may not have an additive intervention effect. However, the majority of studies do not directly address the issue of breadth of content in relation to outcomes, because the number of intervention variations that can be tested is limited in single intervention studies, and also because the goal of these studies is to establish the efficacy of an intervention as a whole rather than based on specific parts. Thus, there is a need to evaluate the extent to which the number of feedback components is related to behavioral outcomes.

There is also little research on the extent to which BMI content is personalized to participants, and, in turn, how personalization impacts drinking outcomes. For example, although personalized feedback is a key design feature of many individual-level BMIs, there is often educational, or general, information included as well (see Carey et al., 2007; Carey et al., 2012; Miller et al., 2013). It seems intuitive that group BMIs, which target multiple individuals at the same time, would include more general content compared to individual-level feedback interventions. Yet, within group BMIs, it is also plausible that some interventions include more personalized content than others. Thus, personalization of content not only varies by intervention type (e.g., individual vs. group efforts), but likely varies within intervention type as well.

In addition to additive effects of the number of components and personalization, we can reasonably explore the possibility that the relationship between breadth of intervention components and intervention outcomes may differ depending on levels of personalization. Although there is some research to suggest that coverage of fewer components is better than implementation of a more expansive BMI, it is plausible that more components may actually be more beneficial for participants when BMIs are highly personalized. That is, in a highly personalized BMI, provision of a wide range of content may be more salient to participants, and thus more memorable, possibly leading to changes in drinking behaviors over time. Conversely, in less personalized interventions, a more concise, focused message could be easier to remember and more effective. However, there has not been any research, to our knowledge, in which the interaction between breadth and personalization of BMI content has been examined.

Current Study

The major aim of the current study was to investigate BMI content in relation to drinking outcomes reported post-intervention. Specifically, we examined the relationship between the total number of intervention components covered and percent of personalized intervention content in relation to both short- and long-term alcohol use and alcohol-related problems using data from a large, integrative data analysis (IDA) study of multiple BMIs (Project INTEGRATE; Mun et al., 2011). Given the many variations in the way BMIs were implemented, we controlled for implementation characteristics, including whether or not a BMI included personalized feedback and was delivered in person or in an alternative format. In addition, we statistically controlled for student demographic characteristics, including gender, status as a first-year student, and whether or not the student was mandated to receive the BMI.

Method

Studies and Sample

Data and intervention materials were a subset of a larger collection of 24 independent BMI trials included in Project INTEGRATE. Project INTEGRATE utilizes IDA of individual-level data from a broad sample of BMI studies conducted over the past two decades that aimed to reduce college student alcohol use and related harm. The combined data set features a large sample of students (N = 12,630; 42% men; 58% first-year or incoming students) who were assessed at baseline and at least one additional follow-up time point, conducted up to 12 months post baseline. All original studies included at least one BMI condition, although the majority also included a comparison condition (e.g., assessment-only control condition), and many included multiple BMI conditions, or other non-BMI alcohol intervention conditions (e.g., alcohol education, alcohol expectancy challenge, parent-based intervention). A more detailed description of sample and study design features of the original studies is presented in Mun et al. (2011).

The sample was limited to individuals who received a BMI, meaning that sessions were facilitated in the spirit of Motivational Interviewing (Miller & Rollnick, 2013) and/or intervention content was considered to be motivational in nature (i.e., personalized feedback). Any participants in a control condition, a non-BMI intervention condition (e.g., alcohol expectancy challenge), or a BMI in combination with a non-BMI intervention condition (e.g., BMI in combination with a parent-based intervention) were excluded. This yielded 33 BMI intervention conditions across 24 studies (see Table 1 for details). In terms of key design features, the majority of BMI conditions were either (1) in-person motivational interviews delivered with personalized feedback (MI + PF) targeting individuals (n = 11); (2) non in-person personalized feedback only interventions (PF) targeting individuals (n = 11); or (3) in-person motivational interviews delivered without personalized feedback targeting groups (GMI; n = 9). The remaining two BMI conditions had unique designs: an in-person MI + PF intervention delivered in groups (study 7) and an in-person MI only (no PF) targeting individuals (study 21). Given that these latter two BMI designs were represented by single studies, both conditions were excluded from analysis. Thus, the current study included 31 BMI conditions across 23 studies (study 7 only had one BMI condition and, upon its elimination, was not represented in the analysis) with 6,047 participants; 41.0% of whom were men. Almost three-fourths of the sample (71.6%) identified as White/Caucasian, with 12.5% Asian/Native Hawaiian/Other Pacific Islander, 6.4% Hispanic, 2.2% Black/African American, 0.5% American Indian/Alaska Native, and 5.4 % Mixed Race or Other. Most of the participants were first-year students (58.9%), and 17.9% were mandated by their university to attend an intervention program because of their infractions of alcohol and/or drug policies.

Table 1. BMI Characteristics across Studies.

Study	Reference	n	Personalized Feedbacka	BMI Formatb	BMI Deliveryc	Total Components	Personalized Components	Percent of Personalized Content (%)	BMI Length (in min)	Short-term Follow-upd (in months)	Long-term Follow-upd (in months)
1	White, Mun, Pugh, & Morgan (2007)	180	PF	Ind	IP	13	11	85	30-60
		168	PF	Ind	NIP	12	10	83	N/A	3	12
2	White, Mun, & Morgan (2008)	111	PF	Ind	NIP	15	14	93	N/A	2	6
3	Barnett, Murphy, Colby, & Monti (2007)	113	PF	Ind	IP	15	11	73	30-60	3	12
4	Cimini et al. (2009)	228	No PF	Grp	IP	14	4	29	120-180	N/A	12
5	LaBrie, Lamb, Pedersen, & Quinlan (2006)	167	No PF	Grp	IP	12	5	42	60-90	3	6
6	LaBrie, Thompson, Huchting, Lac, & Buckley (2007)	115	No PF	Grp	IP	13	5	38	120-180	3	N/A
7	Fromme & Corbin (2004)	417e	PF	Grp	IP	17	7	41	180+	1	6
8a		736	PF	Ind	NIP	14	11	79	N/A
8b	Larimer et al. (2007)	1094	PF	Ind	NIP	14	11	79	N/A	N/A	12
8c		303	PF	Ind	NIP	14	11	79	N/A
9	Lee, Kaysen, Neighbors, Kilmer, & Larimer (2009)	97	No PF	Grp	IP	14	6	43	180+
		103	No PF	Grp	IP	10	8	80	90-120
		101	PF	Ind	IP	14	12	86	60-90	3	6
		100	PF	Ind	NIP	13	11	85	N/A
10	Marlatt et al. (1998)	174	PF	Ind	IP	12	8	67	30-60	N/A	12
11	Walters, Vader, & Harris (2007)	185	PF	Ind	NIP	15	9	60	N/A	3	N/A
12	Wood, Capone, Laforge, Erickson, & Brand (2007)	84	PF	Ind	IP	9	9	100	30-60	3	6
13	Murphy, Benson, & Vuchinich (2004)	26	PF	Ind	IP	11	9	82	30-60	N/A	6
		28	PF	Ind	NIP	10	7	70	N/A
14	Murphy et al. (2001)	30	PF	Ind	IP	10	8	80	30-60	3	12
15	LaBrie, Huchting, et al. (2008)	155	No PF	Grp	IP	12	6	50	120-180	3	N/A
16	LaBrie et al. (2009)	161	No PF	Grp	IP	12	6	50	120-180	3	6
17	LaBrie, Pedersen, Lamb, & Quinlan (2007)	120	No PF	Grp	IP	12	5	42	60-90	3	6
18	Martens, Kilmer, Beck, & Zamboanga. (2010)	114	PF	Ind	NIP	9	8	89	N/A	1	6
		102	PF	Ind	NIP	8	7	88	N/A
19	LaBrie, Hummer, Neighbors, & Pedersen (2008)	537	No PF	Grp	IP	5	5	100	60-90	2	N/A
20	Larimer et al. (2001)	318	PF	Ind	IP	15	11	73	120-180	N/A	12
21	Walters, Vader, Harris, Field, & Jouriles (2009)	76	PF	Ind	IP	16	10	63	30-60
		68	PF	Ind	NIP	15	9	60	N/A	3	12
		72e	No PF	Ind	IP	5	4	80	30-60
22	Wood et al. (2010)	187	PF (drinker)	Ind	IP	9	9	100	30-60	N/A	12
		66	PF (abstainer)	Ind	IP	7	6	86	30-60

Note.

^aPF = personalized feedback delivered.

^bInd = individually delivered intervention; Grp = group delivered intervention.

^cIP = in-person delivered intervention; NIP = non in-person delivered intervention.

^dThis reflects time points utilized in the current study design, and may not reflect the total number of assessment points in the original study design, either short-term or long-term.

^eDenotes intervention groups that were removed from further analysis (see Methods for detailed description).

For this paper, we utilized drinking outcome data at both short- and (intermediate-to) long-term follow-up assessment points. Short-term outcomes fell within a range of 1 to 3 months post baseline, whereas long-term outcomes were assessed between 6 to 12 months post baseline. The exact timing of the assessment from which follow-up data were utilized is noted in Table 1.

Coding of Intervention Content Components and Personalization

Coding was an iterative, interactive process consisting of two primary phases: 1) identification of an overall list of components, and 2) determination of whether content specific to each component was general in nature, personalized to the participant, or both. Prior to the first phase, all original study investigators provided copies of the intervention materials used, when available, including personalized feedback, session outlines, handouts, etc. In addition, we reviewed the Method sections of published articles describing the interventions. Then, two raters from the authorship team independently reviewed all materials in an effort to determine a list of components (e.g., alcohol quantity and frequency, use-related consequences). Consensus meetings were held between both raters, as well as two additional team members who are knowledgeable about BMIs for college students. The list was narrowed to a total of 20 components, and definitions were established by the research team to indicate what type of information should be included in order to be classified as having content specific to each component (see the Appendix for definitions of each component).

For the second phase of coding to determine the extent of personalization for each component, the two raters independently coded each component for all study conditions. For instance, for the component on alcohol-related problems, personalized content would include a list of consequences experienced by the participant, whereas general content would include a statement or discussion that the experience of negative consequences is a risk factor for future alcohol problems. Any discrepancies were discussed between raters, and in a few cases, discussed with the entire research team until there was full agreement across all team members. Finally, the original study investigators reviewed the code sheets for their respective studies, and noted any discrepancies, which were discussed and corrected when applicable.

Information on fidelity of each BMI can be found in the published papers for each original study (see Table 1 for a list). A closer examination of these studies reveals that most fidelity checks were specific to whether or not in-person sessions were adherent to MI principles. Less attention was given to whether or not BMIs were adherent to the provision of actual content. For example, it is possible that components intended to be covered in in-person sessions were not covered. However, for the purposes of this study, it is critical to note that for MI + PF and PF only conditions, the components coded were derived primarily from the personalized feedback and other handouts given to all participants. Thus, in the event a facilitator did not get a chance to cover a certain component in the context of an in-person MI + PF session, the participant would still have received the content. Fidelity was assessed less frequently in GMI conditions; however, a review of intervention protocols for those studies revealed that the majority were scripted, guided by a workbook, or highly structured with a list of a specific components (and corresponding information) to cover. Taken together, it is likely that the components coded were delivered to participants.

Measures

Total number of components

For the 31 intervention conditions we noted whether each of the 20 identified components was covered in some capacity (0 = did not include information, 1 = included information). Values were then summed to reflect the total number of components covered by that specific intervention. For the structural equation modeling (SEM) analyses (see below), the number of components was grand mean centered across the entire data set for the interpretation of interactions as well as main effects.

Percent of personalized content

As noted previously, content within each component was first coded as general, personalized, or both. We then created a variable to reflect whether or not each component included personalized content (whether alone or delivered with general information on that same component; 0 = no, 1 = yes). Thus, components with content tailored to participants were considered to be personalized regardless of whether general content was also included. Scores were then summed for each intervention condition to reflect the total number of personalized components. The total number of personalized components was divided by the total number of components covered by that intervention, with values reflecting the percent of intervention content that was personalized to the participant. This variable was also grand mean centered for the SEM analyses.

Alcohol use

Alcohol use measures differed across studies. However, most studies included a version of the Daily Drinking Questionnaire (DDQ; Collins, Parks, & Marlatt, 1985), in which participants were asked to report the number of drinks they consume each day of a typical week. We first calculated measures of alcohol use frequency (number of drinking days per week) and quantity (average number of drinks on drinking days). A single item measure of peak drinking was available in most studies (e.g., the maximum number of drinks consumed on a given occasion within the past month). One study capped responses for this item at 25 drinks; thus, in order to have consistency across studies, we capped any responses that exceeded this value in all other studies with a measure of peak drinking.

A few studies utilized different but conceptually similar alcohol measures.¹ In total, we were able to derive a measure of typical drinking quantity for all studies, whereas four studies did not have a comparable measure for maximum number of drinks on an occasion, and one study did not have a measure of typical drinking days per week. We then adopted a latent variable modeling approach to derive a latent trait alcohol use variable that consisted of these three indicators.² This variable was estimated at baseline as well as at both the short- and long-term follow-ups, and factor loadings for each indicator were significant at all time points (p < .001). All standardized factor loadings were over .65, which indicates adequate convergent validity.

Alcohol-related problems

Studies varied in their measures of alcohol problems, including the Rutgers Alcohol Problem Index (RAPI; White & Labouvie, 1989), the Young Adult Alcohol Problems Screening Test (YAAPST; Hurlbut & Sher, 1992), the Brief Young Adult Alcohol Consequences Questionnaire (BAACQ; Kahler, Strong, & Read, 2005), the Alcohol Use Disorders Identification Test (AUDIT; Saunders et al., 1993), the Positive and Negative Consequences Experienced questionnaire (PNCE; D’Amico & Fromme, 1997), and the Alcohol Dependence Scale (ADS; Skinner & Allen, 1982; Skinner & Horn, 1984). To establish commensurate alcohol problems trait scores for participants across different studies, we conducted a unidimensional item response theory (IRT) analysis for multiple groups. The latent trait scores derived from the IRT analysis were validated with the original scale scores within each study (for details see White et al., 2012). Latent trait scores were calculated at baseline and follow-up.

Covariates

Demographic covariates included gender (0 = female, 1 = male), mandated status (0 = no, 1 = yes), and first-year student status (0 = no, 1 = yes). We controlled for two intervention implementation characteristics, including whether or not personalized feedback was provided (0 = no, 1 = yes) and whether or not the intervention was delivered in person (0 = no, 1 = yes). We also intended to control for whether or not an intervention was delivered to an individual or group; however this was completely confounded with provision of personalized feedback (r = −1.00), so it was automatically dropped from the analysis.

Analytic Plan

First, descriptive analyses were conducted examining frequencies and percentages of components, number of components, and personalization of content across studies and types of interventions. Then, to examine the relationships between the predictors (the total number of components, the percentage of personalized components, and their interaction) and drinking outcomes (alcohol use and alcohol-related problems at follow-up), two structural equation models (SEM) were estimated via maximum likelihood (ML) estimation using Mplus 7 (Muthén & Muthén, 1998-2012), in order to evaluate both short- and long-term follow-up outcomes. For both models, grand mean-centered components and personalization, and the components × personalization interaction were the main predictors of drinking outcomes at follow-up while controlling for baseline alcohol outcomes, and the five pre-specified covariates: personalized feedback, in-person delivery, male, mandated, and first-year (see Figure 1). To assess the magnitude of the interaction effect between components and personalization, the SEMs were fit first without and then with the interaction term in the model.

Figure 1.

A path diagram of the structural equation model for drinking outcomes. Personalized feedback (1; 0 = no); in-person delivery (1; 0 = no); male (1; 0 = female); mandated (1; 0 = no); and first-year (1; 0 = no).

A sandwich-type estimator was used (Froot, 1989; Huber, 1967; White, 1980) to compute unbiased standard errors that adjust for the nested data structure (i.e., individual participants nested within studies). This estimator prevents standard errors from being underestimated, especially with a large n, which, otherwise, could produce false positive findings (i.e., incorrect rejection of a true null hypothesis). We also weighted the data to account for differences in sample sizes by using an inverse of the square root of each original study’s sample size (see Clarke, Kim, White, Jiao, & Mun, 2013 for more detailed information).

To assess the robustness of the findings to study heterogeneity, we examined the influence of each individual study on regression coefficients in the SEMs. Specifically, a “leave-one-out” strategy was adopted, in which SEM models were sequentially fit while removing one individual study at a time, and DFBETAs (Storer & Crowley, 1985) were calculated. DFBETA is a measure assessing the extent that each “observation” influences a regression coefficient. In the current analysis, the DFBETA for a particular study was calculated as the difference between the regression coefficients with and without the particular study divided by the respective standard error from the model without the particular study. This estimate is indicative of the influence of each individual study on each coefficient in a multi-group study. The recommended cut-off values are 1 or $2∕\sqrt{n}$ for small or large samples, respectively (Nelson, 2007). We considered n to represent the number of studies, as opposed to the number of participants included in the combined sample, and therefore used 1 as the cut-off value.³

Results

Breadth and Personalization of Intervention Content

The number of components covered in each BMI ranged from 5 to 16, with BMIs targeting about 12 components on average (M = 12.06, SD = 2.70). The percentage of interventions that included content within each of the 20 identified components is documented in Table 2. All intervention conditions included content specific to alcohol quantity and frequency, alcohol-related consequences, and descriptive alcohol use norms, with nearly 75% or more covering dependency factors, protective behaviors, normative discrepancies, alcohol expectancies, discussion of change, alcohol-specific sexual behavior, and DWI. Content on family history, special groups (i.e., women, athletes, Greeks), financial costs, caloric intake, and biphasic effects was quite varied, as results indicated that these components were included in 45% to 65% of BMIs. Less than 30% of all intervention conditions covered illicit drug use, alcohol and drug interactions, tobacco use, and information on alcohol and mental health.

Table 2. BMI Content by Individual Components Across Conditions (N = 31).

	Any		G Only		P Only		G + P
	n	%	n	%	n	%	n	%
Alcohol quantity and frequency	31	100.0	0	0.0	0	0.0	31	100.0
Alcohol-related problems	31	100.0	6	19.4	5	16.1	20	64.5
Descriptive alcohol use norms	31	100.0	5	16.1	3	9.7	23	74.2
Dependency factors	26	83.9	9	34.6	2	7.7	15	57.7
Protective behaviors / skills	26	83.9	12	46.2	0	0.0	14	53.8
Norm discrepancy	25	80.6	1	4.0	10	40.0	14	56.0
Discussion of change	25	80.6	4	16.0	8	32.0	13	52.0
Expectancies / motives	24	77.4	1	4.2	3	12.5	20	83.3
Alcohol-related sexual behavior	23	74.2	12	52.2	2	8.7	9	39.1
DWI	23	74.2	20	87.0	0	0.0	3	13.0
Family history	20	64.5	0	0.0	2	10.0	18	90.0
Special group information	17	54.8	16	94.1	0	0.0	1	5.9
Financial costs	16	51.6	3	18.8	7	43.8	6	37.5
Calorie content	15	48.4	0	0.0	6	40.0	9	60.0
Biphasic effects	14	45.2	13	92.9	0	0.0	1	7.1
Decisional balance	9	29.0	3	33.3	1	11.1	5	55.6
Drug use	7	22.6	3	42.9	0	0.0	4	57.1
Alcohol and drug interactions	4	12.9	4	100.0	0	0.0	0	0.0
Tobacco use	4	12.9	0	0.0	0	0.0	4	100.0
Mental health	3	9.7	0	0.0	0	0.0	3	100.0

Note. G Only = general content; P Only = personalized content; G + P = both general and personalized content. Percentages of type of content for each component (i.e., G, P, or G + P) were calculated relative to the number of groups that included any information in that area.

In terms of personalization, an average of 72% (SD =20%) of BMI components covered by a given intervention were tailored to the individual participants, meaning content specific to those components included either solely personalized information or a blend of both personalized and general content. A closer examination of the results in Table 2 suggests this was driven primarily by the latter, as the combination of general and personalized information was most common across all components. However, the extent to which content was personalized to a participant varied within each content area. Within the most commonly included intervention components (i.e., alcohol quantity and frequency, alcohol-related problems, and descriptive alcohol use norms), the majority of content was personalized to the participant, with less than 20% of content coded as solely general. Other components also varied in terms of personalization; for example, just over one-third of BMIs that covered dependency factors and nearly one-half of BMIs that covered protective behaviors included content that was void of any personalization. Content specific to alcohol-related sexual behavior and DWI was primarily general in nature (see Table 2).

Breadth and Personalization in Relation to Drinking Outcomes

Short-term alcohol use and alcohol-related problems

Across participants in the short-term outcome analysis (n = 2,851), means for components and personalization were 11.13 (SD = 3.47) and 74% (SD = 22%), respectively. The regression coefficients and information criteria for comparing models (e.g., Akaike Information Criterion [AIC] and Bayesian Information Criterion [BIC]; the smaller the better) are provided in Table 3.⁴ Given that we controlled for baseline drinking measures, the regression coefficients reflect change in alcohol use and related problems from baseline to follow-up.

Table 3. Effects of Components and Personalization on Alcohol Outcomes at Short-Term Follow-Up (n =2,851).

	Without Interaction		With Interaction
Predictors	Est (SE)	Std Est	Est (SE)	Std Est
Alcohol Use
Baseline Alcohol Use	0.75 (.06)***	0.82	0.75 (.06)***	0.81
Personalized Feedback	0.32 (.38)	0.07	0.43 (.42)	0.09
In-person Delivery	0.56 (.26)*	0.11	0.56 (.28)*	0.11
Male	0.18 (.12)	0.08	0.19 (.12)	0.08
Mandated	0.02 (.28)	0.01	0.14 (.27)	0.06
First-year	−0.29 (.17)	−0.12	−0.18 (.18)	−0.08
Components	−0.02 (.05)	−0.03	0.02 (.05)	0.02
Personalization	0.71 (.73)	0.07	0.87 (.82)	0.08
Components × Personalization			−0.40 (.24)	−0.18
R 2	0.69		0.69
Alcohol Problems
Baseline Alcohol Problems	0.65 (.06)***	0.67	0.65 (.06)***	0.67
Personalized Feedback	−0.04 (.21)	−0.02	−0.04 (.22)	−0.02
In-person Delivery	0.11 (.11)	0.05	0.11 (.11)	0.05
Male	−0.01 (.05)	−0.01	−0.01 (.05)	−0.01
Mandated	−0.32 (.13)*	−0.33	−0.31 (.14)*	−0.32
First-year	−0.03 (.09)	−0.03	−0.02 (.09)	−0.02
Components	0.01 (.03)	0.02	0.01 (.03)	0.03
Personalization	0.06 (.43)	0.01	0.08 (.42)	0.02
Components × Personalization			−0.04 (.10)	−0.04
R 2	0.45		0.45
Information Criteria
AIC	85,288		85,274
BIC	85,592		85,590

Note. Est = Estimate; Std Est = Standardized Estimate.

^*p < .05;

^**p < .01;

^***p < .001.

Personalized feedback (1; 0 = no); in-person delivery (1; 0 = no); male (1; 0 = female); mandated (1; 0 = no); first-year (1; 0 = no); components (number of content areas); personalization (percent of personalized content).

The results revealed that neither components nor personalization were significantly associated with alcohol use and alcohol problems at the short-term follow up. Further, the interaction term was not significantly associated with outcomes. Students who attended in-person BMIs, compared to other formats, reported greater alcohol use (p < .05) at the short-term follow-up. Individuals mandated to attend a BMI, compared to volunteers, reported lower levels of alcohol problems (p < .05). In addition, as expected, baseline alcohol use and alcohol problems were positively associated with follow-up alcohol use (p < .001) and alcohol problems (p < .001), respectively.⁵

Of the DFBETA values calculated for components, personalization, and the interaction term for each individual study, only four out of 114 coefficients exceeded 1 in their absolute values. Overall, this implies that the reported findings are quite robust to study heterogeneity and that no particular study exerted overbearing influence on the reported estimates.

Long-term alcohol use and alcohol-related problems

Across participants in the long-term outcome analysis (n = 5,018), means for components and personalization were 13.05 (SD = 1.94) and 74% (SD = 17%), respectively. Regression coefficient estimates and the information criteria are provided in Table 4. Similar to the short-term results, personalization was unrelated to either outcome. Students in BMIs that covered a greater number of components reported lower levels of alcohol use at follow up (p < .05), and those who attended in-person BMIs, compared to other formats, reported higher levels of use (p < .05). Men, compared to women, reported greater alcohol use (p < .001) and alcohol-related problems (p < .01), and mandated, compared to volunteer, students reported lower levels of alcohol-related problems (p < .001). In addition, we found a statistically significant interaction between components and personalization for both alcohol use (p < .001) and alcohol problems (p < .05). The interaction model was also supported based on the two information criteria, AIC and BIC (AIC = 136,842 and BIC = 137,187 for the interaction model vs. AIC = 136,911 and BIC = 137,243 for the model without the interaction term; smaller AIC/BIC values indicate better fit). Notably, number of components and in-person delivery were not significantly associated with alcohol use when the interaction term was included in the model. The calculated DFBETAs for the main regression coefficients indicated that no particular study overly influenced the results.⁶

Table 4. Effects of Components and Personalization on Alcohol Outcomes at Long-Term Follow-Up (n =5,018).

	Without Interaction		With Interaction
Predictors	Est (SE)	Std Est	Est (SE)	Std Est
Alcohol Use
Baseline Alcohol Use	0.73 (.03)***	0.81	0.73 (.03)***	0.81
Personalized Feedback	0.44 (.38)	0.08	0.25 (.32)	0.05
In-person Delivery	0.34 (.15)*	0.07	0.17 (.14)	0.04
Male	0.42 (.11)***	0.18	0.43 (.10)***	0.19
Mandated	−0.21 (.21)	−0.09	−0.16 (.13)	−0.07
First-year	−0.16 (.12)	−0.07	−0.07 (.11)	−0.03
Components	−0.10 (.04)*	−0.09	−0.02 (.06)	−0.02
Personalization	−0.87 (1.22)	−0.07	−0.96 (.81)	−0.08
Components × Personalization			−0.84 (.24)***	−0.37
R 2	0.68		0.69
Alcohol Problems
Baseline Alcohol Problems	0.63 (.03)***	0.67	0.63 (.03)***	0.66
Personalized Feedback	0.12 (.22)	0.05	0.07 (.19)	0.03
In-person Delivery	0.02 (.06)	0.01	−0.03 (.05)	−0.02
Male	0.11 (.04)**	0.12	0.12 (.03)***	0.12
Mandated	−0.29 (.08)***	−0.31	−0.28 (.07)***	−0.29
First-year	−0.05 (.05)	−0.05	−0.02 (.05)	−0.02
Components	−0.01 (.02)	−0.03	0.01 (.02)	0.02
Personalization	−0.59 (.51)	−0.12	−0.67 (.45)	−0.13
Components × Personalization			−0.26 (.11)*	−0.27
R 2	0.46		0.47
Information Criteria
AIC	136,911		136,842
BIC	137,243		137,187

Note. Est = Estimate; Std Est = Standardized Estimate.

^*p < .05;

^**p < .01;

^***p < .001.

Personalized feedback (1; 0 = no); in-person delivery (1; 0 = no); male (1; 0 = female); mandated (1; 0 = no); first-year (1; 0 = no); components (number of content areas); personalization (percent of personalized content).

The significant interaction effects found in the long-term follow-up analysis were plotted for both alcohol use and alcohol problems (see Figures 2 and 3, respectively). The main effects and interaction effects used for Figures 2 and 3 are presented in Table 4. Since both components and personalization were continuous response variables, three values equivalent to −1 SD, the mean, and +1 SD were selected to illustrate the interaction between these variables. Overall, Figures 2 and 3 show that the relationships between components and alcohol use and related problems differed across different levels of personalization, with the strongest intervention effects on both drinking outcomes when interventions were highly personalized with broader coverage of components. Participants in these BMIs drank less and reported fewer problems when more components were covered. In less personalized BMIs, coverage of fewer components was associated with lower levels of alcohol use. Further, the beneficial effect of less comprehensive, less personalized BMI content appeared to be more important for alcohol use than problems. That is, when fewer components were covered, there was little meaningful difference in the experience of alcohol problems across levels of personalization, whereas there was more clear differentiation in alcohol use, indicating less personalized BMIs benefit students more.⁷

Figure 2.

Components × personalization interaction effect on alcohol use at long-term follow-up. For the purpose of the graph, three points (± 1 SD and the mean) of personalization were selected (−0.17 for low, 0 for average, and 0.17 for high). Data points were calculated for mean values of baseline alcohol use and all other control variables set to zero.

Figure 3.

Components × personalization interaction effect on alcohol problems at long-term follow-up. For the purpose of the graph, three points (± 1 SD and the mean) of personalization were selected (−0.17 for low, 0 for average, and 0.17 for high). Data points were calculated for mean values of baseline alcohol problems and all other control variables set to zero.

Discussion

The current study adds to the extant body of literature on BMIs to reduce college student drinking through the examination of important characteristics of BMI content in relation to student drinking outcomes. To date, the majority of studies on BMI efficacy have examined variations in implementation characteristics (e.g., mode of delivery) with little focus on elements of BMI content that may drive change. To our knowledge, this was the first study to examine both breadth and personalization of BMI content, and their interaction, in relation to drinking outcomes, including both alcohol use and related problems assessed post intervention. In addition, this study utilized a unique methodology to address these questions by drawing its sample from an integrative study of BMIs in which individual participant-level data were pooled across studies. Because individual studies only examine at most a few different BMI variations, they have been limited in their ability to probe the questions raised in the present study. Furthermore, although previous meta-analyses have focused on isolating specific components related to intervention efficacy (Carey et al., 2007; Carey et al., 2012; Miller et al., 2013), it is less clear how BMIs may be delivered more efficiently as it relates to number of components and personalization of content.

Consistent with other recent meta-analysis studies (e.g., Carey et al., 2007; Carey et al., 2012; Miller et al., 2013), descriptive results indicated that the actual components covered by BMIs varied considerably. Only content specific to alcohol use quantity and frequency, alcohol problems and descriptive alcohol use norms was addressed by all BMIs in the current analysis, whereas the remaining components ranged in inclusion from 10% to just over 80%. Such variation in BMI content makes it difficult to determine the components that are critical to BMI efficacy along with what exact material should be provided for a given component. Accordingly, the current study addressed broader characteristics of BMI content, including number of components covered as well as the extent to which they were personalized, in relation to their success.

The primary finding of this study was the significant interaction between breadth and personalization of BMI content in relation to long-term drinking outcomes (i.e., alcohol use and problems assessed 6 to 12 months post baseline). Specifically, results suggested that across multiple drinking outcomes, “more is better” in terms of number of BMI components covered, but only when the BMI is highly personalized to the participant. For a more concrete example using our data (i.e., using the criteria of 1 SD above and below the mean as demonstrated in Figures 2 and 3), when 90% or more of BMI content is personalized, it is optimal to cover approximately 15 or more components. It is possible that a highly personalized BMI is perceived by students as more relevant, such that they can better relate, and in turn, attend to a message that is tailored to their own behaviors. Then, the nature of the comprehensive approach (i.e., coverage of multiple, personalized components) keeps students engaged, leading to a more memorable experience for participants, and, in turn, an enduring impact on drinking outcomes.

On the other hand, it appears that “less is more” for BMIs that include more general, and less personalized, content. That is, coverage of fewer components was associated with less consumption when BMIs were less personalized to participants. Again using Figures 2 and 3 as a guide, our data suggested that if the ability to tailor the intervention is limited, such that 60% or less of components are personalized to the recipient(s), it is optimal to cover no more than 11 components. It is possible that students exposed to a more general BMI are more receptive to an overall message that is more concise, and may become less attentive to an intervention as it becomes more expansive. Specifically, it may take more effort for participants in generalized BMIs to relate the information that they are receiving to their own behaviors and experiences. A focus on fewer components may allow for more in-depth coverage so that students are able to personalize the message for themselves, whereas presentation of too many general topics may be overwhelming in a short amount of time.

It is also important to clarify that personalization of a component area, as defined in the current study, did not necessarily equate to an absence of general information on that component. As described earlier, components were considered to be personalized as long as there was some content specific to the recipient, regardless of whether or not the tailored content was coupled with general information. In fact, descriptive results (see Table 2) showed that for most components, the majority of studies included content that was a blend of both personalized and general messages. Thus, for those interested in delivering a highly personalized BMI in the future, findings support that general content on a given component area can be included, as long as there is an element of personalization specific to the recipient(s).

Notably, breadth and personalization of content did not distinguish levels of drinking and related harm students reported in the first few months post intervention (i.e., for short-term results). One plausible explanation for the lack of association at this assessment interval is that the novel experience of any message in the context of a BMI, regardless of the extent to which it is tailored or how many components are covered, is beneficial initially. Then, over time, the initial benefits of the BMI continue for those individuals who received a more salient message, and subside for those individuals who had a less relevant, or memorable experience. The absence of an effect of BMI content characteristics in the short-term and corresponding emergence of their intricate roles on limiting drinking outcomes assessed 6 to 12 months post intervention also underscores the importance of including long-term assessments when examining whether design characteristics of BMIs are associated with differences in alcohol use and related problems.

Beyond the BMI content variables that were of primary interest, implementation characteristics and demographic variables found to be important in previous studies were controlled in the current analysis. Although these variables were not the main focus of the current investigation, results revealed some significant findings worthy of discussion. With regards to implementation characteristics, the short-term analysis (i.e., defined as 1 to 3 months post baseline) revealed that non in-person delivered BMIs were associated with decreased alcohol use, relative to interventions delivered in person. However, this effect should be interpreted with caution given the existing confounds (e.g., in-person BMIs included both group and individual formats). Further, in their recent meta-analysis, Carey et al. (2012) compared face-to-face interventions with computer-delivered interventions and found inconsistent effects across outcomes and follow-up assessment periods. Given the lack of consistency across drinking outcomes and assessment points in both our study and Carey et al.’s (2012) meta-analysis, there appears to be little evidence of a meaningful difference between in-person and non in-person programs at this point, although this is worthy of exploration in future studies.

With respect to demographic control variables, students mandated to receive treatment reported significantly fewer alcohol-related problems at both the short- and long-term follow-up assessments relative to students who participated voluntarily. Notably, there were no differences in reported levels of alcohol use between these two groups, suggesting that although mandated students did not reduce their alcohol use post intervention (or post sanction), they consumed alcohol in a manner that allowed them to avoid related harm. It is plausible that these students were motivated to be more cautious with their drinking as a result of the infraction itself, regardless of BMI characteristics. Although mandated students were represented in multiple BMI types in the current study (e.g., in-person vs. non in-person), we were unable to include an interaction term for these variables, as the nature of the pooled data would have led to an unbalanced design. Consistent with other recent reviews and meta-analytic studies (e.g., Larimer & Cronce, 2007; Carey et al., 2007), results also indicated that women exhibited lower levels of alcohol use and problems compared to men over time. Whereas there were no gender differences in drinking outcomes at short-term follow-up, men reported significantly greater levels of alcohol use and more alcohol-related problems than women at the long-term follow-up.

Results should be interpreted with appropriate caution in light of a few limitations of this study. First, the reported data in the current study were not a random sample of college alcohol BMI studies, as the sample was limited by the design of the parent study. Specifically, Project INTEGRATE was designed to examine the magnitude and boundaries of BMI efficacy and to address questions that have not been thoroughly understood in the literature (i.e., mediators and moderators) using emerging analytic approaches and actual participant-level data from multiple studies. Nevertheless, the studies included represent a broad range of BMI efforts conducted over the past few decades, including both individual and group-based efforts, BMIs delivered with and without personalized feedback, and those delivered in person or via other means (e.g., written, mailed, or computerized feedback). In addition, although we attempted to account for differences in implementation characteristics across studies, it is important to acknowledge that due to the nature of the pooled data, several important BMI characteristics overlapped and were confounded, making it difficult to determine their unique contributions, as well as interaction effects. Given that the majority (i.e., 72%) of the sample was White/Caucasian, it is unclear whether results would generalize to BMIs implemented at colleges with greater proportions of minority students.

Although considerable effort was taken in the coding of BMI content, it is possible that there were nuances in information presented to students that may be important to consider, but were beyond the scope of this study. For example, although we are confident that intervention content coded was delivered to each participant, it is unclear how much emphasis was given to each component. For example, for BMIs delivered in person, it is possible that certain components were discussed more in-depth than others, based on the concerns or interests of the participant. Whereas individual variation is consistent with the motivational style that underlies this approach (e.g., Motivational Interviewing; Miller & Rollnick, 2013), and speaks to the flexibility of BMIs, it presents a challenge for researchers and practitioners alike, when trying to pinpoint key elements of intervention content that drive change. It is also important to clarify that BMIs, as defined in the current study, included both interventions that were delivered in the context of a motivational interview delivered by trained facilitators, as well as interventions in which the materials or messages were designed to be motivational in nature.

Beyond the components themselves, there are other elements of BMI content yet to be explored in the literature that may exert some degree of influence on outcomes. For example, beyond how many or what components to include, there has been little attention given to the order, or sequence, in which components are presented. Related to the idea that participants’ interests should be of primary focus, it may be beneficial to assess what is most relevant to them, and adjust the ordering of content accordingly. In addition, it is unknown whether the visual presentation of feedback or other intervention materials makes any difference. For example, in the current study, the visual presentation of the BMI content varied quite considerably from one study to the next, including formatting of the information, inclusion and type of graphs used to underscore key intervention messages (e.g., normative discrepancies), as well as other visual enhancements designed to make content look more appealing. For BMIs that are delivered via means other than an in-person facilitator, it is possible that those designed to be more visually appealing may be more successful in drawing the attention of the participant, and in turn, lead to more promising outcomes.

Finally, the existing variability in BMI content and methodological heterogeneity noted above also points to a need for continued research in this area, and more specifically, analytic frameworks that are suited to address questions related to what BMI components are most critical. Dismantling studies may initially seem like an intuitive approach, but when considering all possible combinations of BMI components (e.g., 20 components were identified in this study), it is likely not a practical one. For example, in the current study, some topics (e.g., alcohol-related problems, descriptive drinking norms) were covered by all studies. Since there were no studies that did not include these topics to serve as a comparison, there was no way to determine if they were critical to BMI efficacy. Further, the variability in coverage yielded too many different combinations to allow for any type of meaningful dismantling, as the majority of BMIs had a different make-up of components. In fact, Collins, Murphy, Nair, and Strecher (2005) suggest that when intervention components are tightly integrated, it might not make sense to try to break them down into separate parts. One option may be to focus research efforts on a few key components based on the theories underlying BMIs as well as empirical evidence from existing literature. For example, one approach would be to formally test a general BMI in which content is limited in scope against a more expansive, tailored BMI. Such research also has implications for refinement of existing theories as they relate to BMIs, as well as cost-effectiveness of BMI implementation, which could be particularly useful to college administrators in identifying an evidence-based option best suited for their campus.

Overall, the current study highlights some new considerations for BMIs that aim to reduce drinking outcomes among college student populations. Researchers and practitioners may want to consider investing resources in the implementation of highly personalized BMIs that cover many components, given our finding that this type of intervention exerted the strongest impact on both alcohol use and problems over time. When the ability to personalize BMIs is limited, a more streamlined approach, with a few, key educational messages may be a reasonable option with some overall benefit on alcohol use.

Appendix: Definitions of Content by Component

Component	Definition
Alcohol quantity and frequency	Content specific to quantity and frequency of alcohol use, including patterns of drinking behavior (e.g., binge drinking), or blood alcohol level/content (BAL/BAC).
Alcohol-related problems	Content specific to consequences, or problems, reported by college students that occur as a result of drinking.
Descriptive alcohol use norms	Content related to descriptive drinking norms of college student peers (i.e., how much students drink). The actual reference group of peers may vary (e.g., typical students, typical students of one’s same gender, etc.).
Dependency factors	Content on factors indicative of future dependence (not listed as one of the other components on this table, e.g., family history) including AUDIT scores or tolerance, or content related to addiction, abuse, or dependence.
Protective behaviors / skills	Content on protective behavioral strategies, or other harm reduction techniques, coping strategies, or resistance skills students can employ to reduce alcohol use and/or harm.
Discrepancies in normative perceptions	Content on comparisons between how much alcohol students think others drink, relative to how much they actually drink.
Discussion of change	Content that relates to changing one’s alcohol use behaviors including readiness to change or personal goal setting.
Expectancies / motives	Content related to motives or reasons for drinking, and/or expectancies about the effects of alcohol consumption.
Alcohol-related sexual behavior	Content specific to sexual behavior consequences or risks that can result from alcohol use (separate from a general list of consequences provided to participants).
Driving while intoxicated (DWI)	Content specific to DWI as a behavior (apart from its inclusion on a general list of consequences provided to participants).
Family history	Content specific to family history of alcoholism.
Special group information	Special group information contains content on alcohol use that is relevant to specific groups (e.g., content specific to women and alcohol use, such as higher BACs or risk of sexual assault, content on how alcohol can affect athletic performance, or content tailored to fraternity and sorority members).
Financial costs	Content on the financial costs associated with alcohol use, i.e., the amount of money one spends when drinking.
Calorie content	Content on calories associated with alcohol consumption.
Biphasic effects	Content on the biphasic effects of alcohol such as maximizing the positive effects of alcohol (e.g., a buzz) and minimizing the negative effects (e.g., getting drunk).
Decisional balance	Content specific to weighing the pros and cons of alcohol use or barriers to change.
Drug use	Content related to illicit drug use (i.e., illegal drugs and prescription drugs used for non- medical reasons) including use, consequences, norms, financial costs, etc.
Alcohol and drug interactions	Content related to drug (legal or illegal) and alcohol interactions.
Tobacco use	Content related to tobacco use including use, consequences, norms, financial costs, etc.
Mental health	Content specific to mental health issues and drinking (e.g., depression, anxiety, etc.)