Individual and Situational Factors that Influence the Efficacy of Personalized Feedback Substance Use Interventions for Mandated College Students

Abstract

Little is known about individual and situational factors that moderate the efficacy of Personalized Feedback Interventions (PFIs). Mandated college students (N = 348) were randomly assigned to either a PFI delivered in the context of a brief motivational interview (BMI; n = 180) or a written PFI only (WF) condition and followed up at 4 months and 15 months post-intervention. We empirically identified heterogeneous subgroups utilizing mixture modeling analysis based on heavy episodic drinking and alcohol-related problems. The four identified groups were dichotomized into an improved (53.4%) and a non-improved (46.6%) group. Logistic regression results indicated that the BMI was no more efficacious than the WF across all mandated students. However, mandated students who experienced a serious incident requiring medical or police attention and those with higher levels of alcohol-related problems at baseline benefited more from the BMI than from the WF. It may be an efficacious and cost-effective approach to provide a written PFI for low-risk mandated students and an enhanced PFI with a BMI for those who experience a serious incident or with higher baseline alcohol-related problems.

Over 40% of college students report heavy episodic drinking (HED) at least once in the past two weeks and over 20% report HED three or more times in the past two weeks (Wechsler, Lee, Kuo, Seibring, Nelson, & Lee, 2002). Consequences of excessive drinking among college students include injuries, motor vehicle accidents, unprotected sex, sexual victimization, academic problems, health problems, suicide attempts, destructive behavior, and police involvement (Engs, Diebold, & Hansen, 1994; Hingson, Heeren, Zakocs, Kopstein, & Wechsler, 2002; Presley, Meilman, & Cashin, 1996; Wechsler, Lee, Nelson, & Lee, 2001). In 2001, more than 1,700 U.S. college student deaths and over 500,000 unintentional injuries were alcohol-related (Hingson, Heeren, Winter, & Wechsler, 2005). In response, a number of preventive interventions have been implemented to help college students move safely through this risky transitional developmental period between adolescence and young adulthood (i.e., emerging adulthood) (Arnett, 2000, 2007; Dimeff, Baer, Kivlahan, & Marlatt, 1999). The massive growth in college prevention programs seen over the last decade (Anderson & Milgram, 1996, 2001; Wechsler et al., 2002) reflects efforts to provide universal and selective preventive interventions to college students.

Personalized Feedback Interventions

The available evidence suggests that individually-oriented, multi-component interventions that enhance cognitive-behavioral skills, enhance motivation to change, provide accurate peer norms for alcohol use and drug use on campus, and challenge any inaccurate alcohol expectancies are efficacious for college students (Larimer & Cronce, 2002; National Institute on Alcohol Abuse and Alcoholism [NIAAA], 2002). In particular, personalized feedback interventions (PFIs) often delivered within the context of a brief motivational interview (BMI) have been shown to be efficacious with heavy drinking volunteer students (e.g., Baer, Kivlahan, Blume, McKnight, & Marlatt, 2001; Baer, Marlatt, Kivlahan, Fromme, Larimer, & Williams, 1992; Borsari & Carey, 2000; Carey, Carey, Misto, & Henson, 2006; Larimer et al., 2001; Marlatt et al., 1998; Murphy et al., 2001) and mandated students (Borsari & Carey, 2005; White, Mun, Pugh, & Morgan, 2007). The theoretical rationale behind PFIs is that personalized feedback will increase a student's readiness to change his or her drinking behaviors (Miller & Rollnick, 2002). Also, students will alter their perceptions about risk and peer use norms, as well as alcohol/drug expectancies (Dimeff et al., 1999). These changes will lead to reduced drinking, which, should reduce negative consequences of alcohol use. Therefore, when PFIs are presented within the context of a BMI, in which the counselor provides feedback in an empathetic, non-threatening, and nonjudgmental manner, it is expected that they will increase students’ readiness to change and help guide them through the change process. Recent reviews of individual-focused interventions have found that in-person interventions using motivational interviewing and personalized normative feedback are more efficacious than other types such as education-focused programs (see Carey, Scott-Sheldon, Carey, & DeMartini, 2007; Larimer & Cronce, 2007). White et al. (2007) also found that, over a long-term follow-up a PFI delivered with a BMI proved to be more efficacious in reducing risky drinking and related problems for mandated college students compared to a written PFI without a BMI.

Few studies, however, have empirically examined moderating factors of PFIs on drinking outcomes. Thus, it is not well understood under which conditions or for whom PFIs work best (for reviews, see Carey, Scott-Sheldon, et al., 2007; Larimer & Cronce, 2007; Neighbors, Larimer, Lostutter, & Woods, 2006; Walters & Neighbors, 2005; White, 2006). It is critical that we begin to determine for whom PFIs work best and for whom we need different types of interventions. The present study attempts to fill this gap and assesses whether there are individual and situational factors that moderate the efficacy of brief PFIs for mandated students over a long term.

Individual and Situational Factors that Influence PFI Efficacy

Pre-intervention drinking levels

It has been suggested that PFIs may have a greater effect for heavier drinkers than for lighter drinkers because feedback for the former group is more extreme (Walters & Neighbors, 2005). However, studies have been inconsistent in their findings among non-mandated students (Larimer et al., 2007; Murphy et al., 2001). Murphy et al. compared the efficacy of a PFI within the context of a BMI and an educational intervention to an assessment-only control on weekly alcohol consumption and binge drinking among 84 volunteer high-risk students. They found that the PFI contributed to greater reductions in alcohol use and heavy drinking at the 3-month and 9-month follow-ups among those students who were heavier drinkers at baseline. This finding needs to be interpreted with caution because Murphy et al. did not formerly test the interaction between baseline drinking and PFI conditions, and used α = .15 as the Type I error rate due to a small sample size. In contrast, in a large sample of volunteer students, Larimer et al. did not find that severity of baseline drinking moderated the efficacy of a mailed PFI at the 1-year follow-up. Interestingly, Larimer et al. reported that abstainers benefited more from the feedback, compared to drinkers. A recent meta-analysis of 62 studies (Carey, Scott-Sheldon, et al., 2007) reported that individual-level interventions were less successful when heavy drinkers or other at-risk groups (e.g., Greeks, athletes, first-year students) were targeted.

This apparent inconsistency regarding whether pre-intervention alcohol use levels play a role in PFI efficacy may be attributed to several methodological issues. First, some of the previous studies may have lacked the necessary power to detect moderation effects due to insufficient overall sample size. Whereas many clinical trial studies are designed to have enough power to detect treatment (i.e., main) effects, few have enough power to detect differential efficacy across subgroups (i.e., moderation effects) (see Pocock, Assmann, Enos, & Kasten, 2002 for a review). The power to detect moderation is also affected by subgroup sample sizes, restriction in predictor variable range, magnitude of the moderating effect (Aguinis & Stone-Romero, 1997), and measurement error (Sackett, Harris, & Orr, 1986). Therefore, some existing studies may have inadvertently restricted the range of observations by screening out those whose baseline drinking levels were at lower ends of the spectrum, which may have resulted in lowered power. Second, previously reported findings are often based on univariate/bivariate analysis although pre-intervention drinking levels are generally known to be confounded with other individual and situational factors (e.g., gender). Treatment groups are typically balanced using random assignment on measured and unmeasured variables. However, any covariates that are strongly related to outcomes need to be adjusted in examining treatment effects (Pocock et al., 2002). This recommendation is also applicable for examining moderation effects. Thus, adjusting for individual and situational factors related to treatment outcomes may help clarify whether pre-intervention drinking levels affect the efficacy of a PFI above and beyond the influences of these confounding factors. Third, some of the existing studies categorized students based on an a priori definition (e.g., those with five or more drinks in a row in the past 2 weeks, or those in the upper half of a sample based on drinks). However, this heuristic dichotomization approach may be arbitrary. In recent studies of natural trajectories of alcohol use among adolescents and college students, heterogeneous subgroups are empirically identified based on their trajectories over time (e.g., Sher, Gotham, & Watson, 2004). The same methodology may also be adopted for evaluation studies to document subgroups with distinctive profiles of change over time post intervention and to examine predictors and moderators of change.

Incident seriousness

Existing studies on mandated students have not paid much attention to the possibility that mandated students may initiate the self-regulatory, self-recovery process due to getting caught and sanctioned, and that PFIs may facilitate rather than cause this self-recovery process. A few recent studies of mandated students suggest that the alcohol-related violation itself prior to any intervention contributes to reductions in alcohol use (Morgan, White, & Mun, 2008), and that perceived aversiveness of the incident is positively related to students’ motivation to change their drinking (Barnett, Goldstein, Murphy, Colby, & Monti, 2006). Barnett and colleagues hypothesized that salient alcohol-related events such as hospitalization or medical problems would bring about self-evaluation and greater motivation to change especially among those with less prior experience with alcohol and fewer prior alcohol problems. Barnett et al. found, as expected, that prior alcohol use was negatively linked to incident aversiveness and prior alcohol-related problems (AP) were also negatively associated with personal attribution of the incident. In addition, greater perceived incident aversiveness was linked with greater motivation to change alcohol use. Morgan et al. (2008) provided some empirical evidence that mandated students, who were involved in an incident requiring medical or police attention, actually reduced their drinking prior to the intervention more than those with a non-serious incident.¹ Therefore, it is critical that we look at the nature of the incident when examining the efficacy and moderated efficacy of PFIs to better understand changes among mandated students.

Readiness to change

The findings that an incident itself (or self-regulation following it) has an effect on behavior change (Morgan et al., 2008) underscore the need to examine students’ readiness to change or motivation to change following the incident as a potential explanation for differential intervention efficacy across different individuals. The existing literature is inconclusive regarding whether readiness to change moderates the efficacy of PFIs. While Carey, Henson, Carey, and Maisto (2007) did not find a significant moderation effect between BMI and readiness to change among volunteer students, there is limited evidence that it does. For example, Fromme and Corbin (2004) found that, at baseline, mandated participants reported higher levels of readiness to change compared to volunteer students. When they tested readiness to change as a potential moderator of intervention efficacy, results showed a trend toward greater reductions in heavy alcohol consumption following the intervention, compared to the control condition, among the volunteer but not mandated students with greater readiness to change at baseline.

Positive alcohol expectancies

Alcohol expectancies are defined as “structures in long-term memory that have impact on cognitive processes governing current and future consumption” (Jones, Corbin, & Fromme, 2001, p. 59). It is hypothesized that a PFI can alter positive alcohol expectancies and thus reduce motivations to use and advance one's movement across the stages of change (Dimeff et al., 1999). Limited evidence exists that those who drink to enhance their social functions (positive alcohol expectancies) may benefit more from a PFI, at least in a college volunteer sample, because they may be more sensitive to peer norms (Neighbors, Larimer, & Lewis, 2004). However, little is known as to whether positive alcohol expectancies are related to differential efficacy of PFIs among mandated students.

Gender

A number of studies have also looked at gender as a potential moderator of PFI efficacy among volunteer college samples and the results have been equivocal. Murphy et al. (2004) found that women in both PFI conditions with and without a motivational interview lowered their weekly drinking at the 6-month follow-up, while men did not reduce their drinking in either condition. Similarly, Chiauzzi, Green, Lord, Thum, and Goldstein (2005) reported that, although volunteer students who received a PFI were not statistically different from students in the control group overall, a subset of heavy drinking women in the PFI condition reduced their total drinks and HED during special occasions more than their heavy drinking counterpart in the educational control condition. In contrast, there were no such group differences among men. However, several other studies have found no gender differences in response to PFIs (e.g., Carey, Henson, et al., 2007; Marlatt et al., 1998). For example, Marlatt and colleagues (1998) reported that, although women overall reported significantly more declines in AP than men, men and women responded similarly to a PFI. Thus, it is generally unclear whether a relative advantage for women exists following a PFI compared to men.

First-year student in college and other drug use

First-year students in college are generally considered to be at risk for excessive alcohol use. Although evidence of the efficacy of PFIs exists for first-year college students (see Larimer & Cronce, 2007), PFIs may be less beneficial for first-year students according to Carey, Scott-Sheldon et al. (2007). However, it is unclear whether the efficacy of PFIs works differently for first-year students, compared to non first-year students, when their different patterns of alcohol use are controlled. In addition, many mandated students get caught for drug use. The current study also investigates whether other drug use at baseline moderates PFI efficacy among mandated students.

The Current Study

The current study sought to examine whether some mandated students reduce alcohol use more following a PFI than other students, and whether some students respond better to a PFI delivered in the context of a BMI than to a written PFI only. This study aimed to extend the earlier study with the same sample (White et al., 2007) by empirically identifying heterogeneous subgroups of mandated students who differentially respond to a PFI. To achieve this goal, we first analyzed HED and AP based on their change patterns, as well as their overall levels, using the latent change score approach proposed in a recent study (Mun, von Eye, & White, 2009) but with an extension of mixture modeling analysis.² HED and AP were chosen because reductions in these alcohol use behaviors reflect self-regulated harm reduction better than other alcohol use measures. We then used empirically identified groups as the outcome variable in subsequent logistic regression analyses. We formally tested the following six individual and situational factors as predictors of change in the context of a PFI: incident seriousness, readiness to change, positive alcohol expectancies, gender, first-year student, and other drug use. We then examined whether the efficacy of a PFI delivered in the context of a BMI and a written PFI differed depending on individual and situational factors, as well as baseline HED and AP (i.e., differential efficacy of the PFI types by individual and situational factors). Thus, we tested for moderation effects by examining the interaction between PFI condition and each of the predictors.

Method

Participants

Participants were students mandated to a university Alcohol and Other Drug Assistance Program (ADAPS) due to infractions of university rules about alcohol and drug use in residence halls. The sample was recruited during the fall semester 2003 and spring and fall semesters 2004. Of the 390 mandated students, 24 (6.2%) were ineligible for the study based on the following exclusion criteria: prior substance abuse treatment, a score greater than 13 on the Beck Depression Inventory (Beck & Steer, 1984), a .24% Blood Alcohol Concentration (BAC) or higher in a typical week, more than 10 occasions of HED (five or more drinks on one occasion for men and four or more for women) in the past month, nine or more alcohol/drug-related negative consequences, near daily marijuana use, or abstinence from alcohol and drugs (i.e., they were caught in a room with alcohol or drugs but had never used them themselves). Because this was a randomized study and there was no prior research to support the efficacy of written feedback alone for mandated students, the highest risk students were excluded for ethical and clinical reasons. All of these high-risk students received an in-person intervention. In addition, only first offenders were eligible for the study.³ Another 18 students (4.9%) declined to participate in the research study leaving a final sample of 348 students (see Figure 1 for participant flow). The resulting sample was 60.1% male and most students were in their first (61.6%) or second year (29.9%) of college. The sample was 79% Caucasian, 15.5% Asian American, 2.2% African American, and 3.4% of other or mixed ethnicity. Over 90% were caught violating residence life rules while in a group, and 88.6% were referred for alcohol-related violations (for greater detail on sample characteristics, see also White, Morgan, Pugh, Celinska, Labouvie, & Pandina, 2006; White et al., 2007).

Figure 1.

A flowchart of recruitment, participation, and follow-up rates.

Procedures and Interventions

All students referred to ADAPS completed a baseline assessment questionnaire. Using data from the initial assessment, eligibility was determined and an individualized profile was created for each eligible student. The personal profile included information on peer norms for alcohol and drug use, typical peak BAC, alcohol- and drug-related problems, alcohol expectancies, high-risk behaviors (e.g., driving under the influence, unplanned sex after using alcohol or drugs), and personal risk factors (e.g., depression, family history of alcoholism). In addition, the profile contained general information about the effects of various BAC levels and tolerance to alcohol.

Students returned approximately a week later and were randomly assigned (by a flip of a coin) to either a BMI condition (n = 180; 51.7%) or a written feedback only condition (WF, n = 168; 48.3%). Students in the BMI condition met individually with a counselor and discussed their written personal profile, which they were given to take home. The counselor provided feedback in an empathic, non-confrontational, and nonjudgmental style based on the principles of motivational interviewing (Miller & Rollnick, 2002). Students in the WF condition were handed their written profile and left without discussing it with their counselor. Intervention fidelity was assured in several ways. First, counselors were trained specifically in motivational interviewing techniques and received weekly supervision from the third author, a clinical psychologist with expertise in motivational interviewing techniques. Second, five BMI and two WF sessions for each counselor were audio-taped and were listened to by the supervising clinical psychologist, and feedback was provided back to the counselor. Third, the counselors completed a therapist checklist after each BMI session. The checklist consisted of the therapeutic tasks during the session, as well as a self-evaluation for the counselor in terms of being empathic and nonjudgmental, and providing support to the student. The clinical supervisor reviewed the checklists to ensure that the counselors adhered to the protocol.

Students were followed up approximately 4 months after the second session (n = 319, 91.7%) and again 15 months post-baseline (n = 220; 63.2%). There were no significant differences between those followed up and those who dropped out on demographic or baseline alcohol use characteristics (see White et al., 2007 for means and SDs).

Measures

Alcohol use variables

Students reported the number of HED occasions in the past month (defined as five or more standard drinks for men and four or more for women; Wechsler et al., 2002). The number of AP was obtained from the 18-item short version of the Rutgers Alcohol Problem Index (RAPI; White & Labouvie, 1989, 2000). The RAPI has demonstrated reliability and discriminant construct validity in both general population and clinical samples of adolescents and young adults (White, Filstead, Labouvie, Conlin, & Pandina, 1988; White & Labouvie, 1989, 2000) and the 18-item version correlates above .9 with the 23-item version (White & Labouvie, 2000). Students reported on the total number of AP experienced in the last three months (α = .73 - .80 across the three assessments). The distributions of the HED and AP were positively skewed and leptokurtic. They were subsequently log-transformed after adding a constant of one to normalize skewed distributions.⁴ The self-report alcohol use measures used in the current study are widely used in the literature on college drinking and have been found to be reliable when corroborated by collateral reports (Borsari & Carey, 2005; Marlatt et al., 1998). Other studies of college student drinking and related problems have shown that use of collateral reports does not improve validity of the data (Carey et al., 2006; Marlatt et al., 1998).

Incident seriousness and demographic variables

The incident for which the student was mandated was coded as “serious” (coded 1) if the referral was made by EMS/hospital (15.3%) or law enforcement personnel (2.3%) and non-serious (coded 0) if the student was referred by a residence hall advisor (83.4%). Gender was coded 1 for men and 0 for women. First-year students were coded 1 and all others were coded 0. Other drug use (cigarette, marijuana, and other substances) at baseline was coded 1 for those with any use of any substance and 0 for those without any use in the past month. Existing studies have found that students provide valid self-report drug use data (e.g., Johnston, O'Malley, Bachman, & Schulenberg, 2007).

Readiness to change

Readiness to change was measured at baseline by the Readiness to Change Questionnaire (RCQ; Heather, Rollnick, & Bell, 1993). The RCQ is a 12-item self-report measure designed to provide a single stage of change assignment (precontemplation, contemplation, or action) as well as a continuous score for each of the three stages of change. Items were presented on a 5-point Likert scale that ranged from “strongly disagree” to “strongly agree” (e.g., “I am trying to drink less than I used to,” “I enjoy my drinking, but sometimes I drink too much”). In the present study, four items capturing the precontemplation stage were reverse coded, and averaged with the other items to create a continuous scale score (α = .88 at baseline). Higher scores reflect a person's greater readiness to start to change or to actually be changing his or her drinking habits.⁵

Positive alcohol expectancies

Alcohol expectancies were measured at baseline by the Comprehensive Effects of Alcohol Questionnaire (CEOA; Fromme, Stroot, & Kaplan, 1993). The CEOA consists of 20 positive and 18 negative expectancy items. Positive alcohol expectancies included items related to tension reduction, sexuality, liquid courage, and sociability factors. Example items from each factor, respectively, are “I would feel calm”, “I would be a better lover”, “I would be courageous”, and “I would act sociable.” Students responded on a 4-point Likert-type scale ranging from “disagree” to “agree.” We administered only eight positive expectancy items out of the original 20 positive items in order to lessen the burden of students filling out a lengthy questionnaire (the two items with the highest factor loadings from each of the four factors; Fromme et al., 1993). The eight items were averaged to create a positive alcohol expectancy score. Higher positive expectancy scores reflect stronger beliefs that consuming alcohol would result in positive effects for the participant (α = .73 at baseline).

Social desirability

We included a 13-item shortened version (MC-C; Reynolds, 1982) of the original Marlowe-Crowne Social Desirability Scale (MC; Crowne & Marlowe, 1960) that assesses a person's tendency to present himself or herself in a socially desirable way. This short version has been found to discriminate criminal and non-criminal groups and been known to have acceptable test-retest reliability and internal consistency (Andrews & Meyer, 2003). We included this social desirability scale in the baseline assessment to control for potential demand characteristics among mandated students in reporting substance use. Example items are “I'm always willing to admit it when I make a mistake”, “I have never deliberately said something that hurt someone's feelings”, or “I have never been annoyed when people expressed ideas very different from my own.” Responses were coded 1 for True and 0 for False responses. The scale score was created by summing responses (α = .66 at baseline). High scores indicate higher levels of social desirability. Mandated students may be more motivated to underreport alcohol use levels than volunteer students. Previously we reported from a different sample that mandated students with high demand characteristics tended to report lower levels of alcohol and drug use (White et al., 2008). Therefore, although there was no difference in social desirability between two PFI conditions at baseline with the present sample (White et al., 2007), we controlled for social desirability mean levels (and variances) by constraining them to be equal across classes in mixture analysis.⁶

Missing Data

We used the expectation maximization (EM) algorithm for maximum likelihood (ML) estimation for missing data imputation using SAS (SAS Institute, 2002-2006), after the Little's chi-square test of Missing Completely At Random (MCAR test; Little, 1988) resulted in a non-significant chi-square of 8078.96 (df = 8020), p > .05, indicating that missing values were a random subset of the complete data. Thus, we deemed that the imputed data were unbiased (Little & Rubin, 1987; Schafer, 1997).

Results

We utilized a latent change score approach using latent curve models. The previous study (White et al., 2007) showed that overall substance use decreased between baseline and the 4-month follow-up assessment and increased between the 4-month and 15-month follow-up assessments. Instead of analyzing this change pattern using typical nonlinear latent curve models, we examined latent changes between baseline and the first follow-up at 4 months (i.e., latent change variable, initial change [IC] in Figure 2), and between 4 months and 15 months post intervention (i.e., latent change variable, subsequent change [SC] in Figure 2).⁷ We specified the outcome levels at 15 months post intervention as the intercept level (i.e., latent variable, level in Figure 2) because participants in the present study were randomly assigned to a treatment condition and there were no group differences between the BMI and WF groups at baseline. Thus, we focused on the long-term outcome levels rather than baseline levels. This intercept selection approach is equivalent to centering a time metric variable at 15 months in latent curve models (see also the Appendix). All latent variable analyses were conducted using Mplus version 5.0 (Muthén & Muthén, 1998-2007) and subsequent logistic regressions were conducted using SPSS version 16 (SPSS Inc., 1989-2007).

Figure 2.

Analyzed mixture models using latent change variables. Solid lines indicate directly estimated parameters and dotted lines indicate either fixed parameters (i.e., factor loadings) or a mixture part of the analyzed model (i.e., class to latent variables). Social desirability was constrained to be equal in mean, variance, and its paths across classes. Level = Outcome levels at 15 months post intervention, IC = Initial change from baseline to 4 months post intervention, SC = Subsequent change from 4 months to 15 months post intervention. Figure 3 shows the results for each alcohol use measure. T1 = baseline, T2 = 4 months post intervention, T3 = 15 months post intervention.

Latent Change Score Analysis with Mixture Modeling Analysis and Outcome Groups

We analyzed both the number of HED and AP over time simultaneously using mixture analysis.⁸ We added social desirability as a covariate to ensure that derived groups were equivalent in social desirability. Results indicated that, based on the Bayesian Information Criterion (BIC), the model with four latent classes was the best fitting, most parsimonious model (BIC₂ = 5654.72, BIC₃ = 5604.54, BIC₄ = 5432.33 and BIC₅ = 5465.26 for two-, three-, four-, and five-latent class models, respectively). The four latent classes were very well separated (Entropy = .99) and the average posterior probability for the most likely class exceeded .98. Entropy values approaching one are considered to indicate well-separated classes (Celeux & Soromenho, 1996). Figure 3 clearly illustrates that a considerable number of students (classes 1 through 3) continued to engage in HED (Figure 3a) and to report AP (Figure 3b) throughout the observed period.⁹ In the present study, we decided to focus on those who improved versus those who did not, primarily to increase power and improve accuracy of parameter estimates in detecting predictors and moderators in subsequent logistic regression analyses. For instance, cross-tabulating incident seriousness with the four classes resulted in a few cells with a limited number of observations, especially for the smallest class (class 1). Previous studies in the literature have combined empirically identified groups into a smaller number of groups based on other practical and conceptual considerations (e.g., Bongers, Koot, van der Ende, & Verhulst, 2004), or rejected alternate solutions from consideration based on additional criteria (e.g., minimum cluster size [5% or more], distinctively shaped trajectories, or large bivariate residuals). Therefore, in all subsequent analyses, classes 1 through 3 were combined into a single non-improved group (n = 162, 46.6%). The remaining group was labeled as an improved group (n = 186, 53.4%).¹⁰

Figure 3.

Estimated mean growth trajectories of heavy episodic drinking (HED; Figure 3a) and alcohol-related problems (AP; Figure 3b) for the four-class models specified as shown in Figure 2. In all subsequent analysis, classes 1 through 3 were combined into a single non-improved group, and class 4 was classified as an improved group. pp = the average posterior probability for the most likely class.

Table 1 shows the means and standard deviations of the alcohol use outcome variables at the three time points for these two groups. The improved group reported significantly lower levels of HED and AP at baseline as well as the two follow-up assessments.¹¹ Table 2 shows the within-person changes (mean changes, t values, and effect sizes) from baseline to 4 months, from 4 months to 15 months, and from baseline to 15 months post-intervention from paired t-tests. The improved group, compared to the non-improved group, showed reductions in the range of medium to large effect sizes (see Table 2; d = .68 - .77; d = .2, .5, and .8 for small, medium, and large effects; Cohen, 1988) from baseline to 4 months, followed by significant upward swings in the range of small to moderate effect sizes (d = .41 - .61). The initial reduction in AP was maintained over time for the improved group; however, the initial reduction in HED was not maintained over the long-term. When followed-up at 15 months post intervention, the improved group reported lower levels of AP, but not HED, compared to their baseline levels. The non-improved group did not improve in HED or AP over the long-term. They had higher levels of HED and AP than the improved group at all times, and the only positive outcome for this group was the initial reduction in AP from baseline to the 4-month follow up.

Table 1.

Means and Standard Deviations of Heavy Episodic Drinking (HED) and Alcohol-related Problems (AP) for the Improved and Non-improved Groups

	Improved (n = 186)	Non-improved (n = 162)	F (1, 346)	η2
HED at baseline	0.45 (0.57)	1.03 (0.71)	73.29**	0.18
HED at 4 months	0.01 (0.04)	1.16 (0.53)	865.44**	0.71
HED at 15 months	0.34 (0.56)	1.21 (0.71)	162.99**	0.32
AP at baseline	0.75 (0.74)	1.02 (0.72)	11.54**	0.03
AP at 4 months	0.23 (0.46)	0.66 (0.69)	48.21**	0.12
AP at 15 months	0.50 (0.63)	0.92 (0.78)	30.39**	0.08

Note. Values in parenthesis indicate standard deviations.

* p < .05

^**p < .01.

Table 2.

Within-individual Change in Heavy Episodic Drinking (HED) and Alcohol-related Problems (AP) Following Brief Interventions (N = 348)

	Improved (n = 186)			Non-improved (n = 162)
	Mean	t (185)	d	Mean	t (161)	d
From Baseline to 4 mo.
HED	0.44	10.46**	0.77	−0.13	−2.16*	0.17
AP	0.52	9.29**	0.68	0.36	5.19**	0.41
From 4 mo. to 15 mo.
HED	−0.34	−8.27**	0.61	−0.05	−0.95	0.07
AP	−0.26	−5.55**	0.41	−0.25	−3.82**	0.30
From baseline to 15 mo.
HED	0.10	1.80	0.13	−0.18	−2.58*	0.20
AP	0.26	3.94**	0.29	0.11	1.46	0.11

Note. Positive mean changes indicate decreases; negative mean changes indicate increases. Mean = Mean change.

^*p < .05

^**p < .01.

d = Cohen's effect size; calculated by mean change divided by standard deviation.

Predictors of Change in the Context of the PFI

We first investigated whether each of the individual and situational factors, as well as intervention condition and baseline HED and AP, significantly predicted improved group membership using univariate logistic regression without controlling for any covariates. As expected, all individual and situational factors, with the exception of readiness to change, significantly predicted improved group membership when we examined these factors separately in univariate logistic regression analysis without adjusting for any other individual and situational variables or baseline HED and AP (see the first column in Table 3). Also, as expected, baseline HED and AP levels were significantly associated with improved group membership.

Table 3.

Individual and Situational Factors as Predictors of Change in the Context of the PFI: The Improved Group's (n = 186) Odds Ratios Compared to the Non-improved Group (n = 162)

	Unadjusted OR			Adjusted ORa			Adjusted ORb
	Logit	OR	95% CI	Logit	OR	95% CI	Logit	OR	95% CI
BMI (1; 0 = WF)	0.08	1.09	0.71 – 1.66	0.11	1.12	0.70 – 1.78	0.11	1.11	0.68 – 1.82
HED at baseline	−1.36**	0.26**	0.18 – 0.37	−1.36**	0.26**	0.17 – 0.38	−1.27**	0.28**	0.19 – 0.43
AP at baseline	−0.49**	0.61**	0.46 – 0.82	0.00	1.00	0.71 – 1.41	−0.24	0.79	0.53 – 1.17
Incident seriousness (1; 0 = non-serious)	0.87**	2.39**	1.32 – 4.34	1.04**	2.83**	1.46 – 5.47	0.72*	2.05*	1.02 – 4.12
Readiness to change	0.24	1.27	0.92 – 1.75	0.48*	1.62*	1.11 – 2.36	0.36	1.44	0.96 – 2.16
Positive alcohol expectancies	−0.61**	0.55**	0.35 – 0.84	−0.12	0.89	0.54 – 1.46	0.12	1.13	0.66 – 1.93
Female (1; 0 = male)	0.67**	1.95**	1.26 – 3.03	0.61*	1.85*	1.14 – 3.01	0.62*	1.86*	1.10 – 3.13
First-year student (1; 0 = other)	−0.61**	0.54**	0.35 – 0.84	−0.51*	0.60*	0.37 – 0.98	−0.49	0.62	0.37 – 1.03
Other drug use (1; 0 = no use)	−0.72**	0.49**	0.31 – 0.76	−0.54*	0.59*	0.36 – 0.95	−0.37	0.69	0.41 – 1.15

Note.

^aAdjusted for heavy episodic drinking (HED) and alcohol-related problems (AP) at baseline

^bAdjusted odds ratios from the model that included all variables at the same time

Logit (Log Odds Ratio) = Coefficient from logistic regression; OR = Odds Ratio; 95% CI = 95% Confidence Interval.

^*p < .05

^**p < .01.

Next, we added baseline HED and AP levels to each model to statistically control for their effects, in order to examine whether the individual and situational factors uniquely contributed to improved group membership above and beyond the influences of baseline HED and AP. Thus, including these covariates enabled us to infer predictors of change that were not confounded with pre-intervention drinking levels. In other words, we examined, given the same levels of HED and AP at baseline, whether the individual and situational factors predicted improved group membership. When we adjusted for baseline HED and AP (see the middle column in Table 3), we found that experiencing a serious incident, reporting greater readiness to change, being female or a non first-year student, and reporting no other drug use significantly predicted improved group membership. Baseline AP and positive alcohol expectancies no longer significantly predicted improved group membership. Positive alcohol expectancies and AP at baseline may largely be accounted for by baseline HED. Interestingly, greater readiness to change was a significant predictor of improved group membership only after baseline HED and AP levels were taken into consideration, suggesting that given the same levels of HED and AP, those with greater readiness to change were more likely to be in the improved group.

When all variables were examined simultaneously in a single multivariate model (see the last column in Table 3), results indicated that reporting lower levels of HED at baseline, experiencing a serious incident, and being female were the only significant predictors of improved group membership. Other individual and situational factors including reporting greater readiness to change and positive alcohol expectancies, being a first-year student, and reporting other drug use did not uniquely predict improved group membership when statistical adjustment was made to remove confounding influences. These findings indicate that because many individual and situational factors are somewhat related, their unique contributions to intervention outcomes cannot be comprehended fully using univariate analysis alone. For example, greater readiness to change no longer significantly predicted the outcome, in part, because it was related to incident seriousness (r = .32, p < .01) and its effects were confounded with incident seriousness. In contrast, the advantage for female students and for those who experienced a serious incident persisted above and beyond their other co-occurring individual and situational factors and different pre-intervention drinking levels. Note that in all three analyses, we found that having received the BMI did not predict improved group membership.

Moderators of the PFI Efficacy across the BMI and WF Conditions

We then examined whether PFI efficacy was different across the BMI and WF conditions depending on individual and situational factors. In other words, we tested for moderation effects (i.e., differential efficacy of the PFI types by individual and situational factors). In addition to the six individual and situational factors, we examined baseline levels of HED and AP as potential moderators. All continuous variables were centered in order to avoid potential multicollinearity problems, and interaction terms were created with centered variables. We added each interaction term one at a time to the final multivariate model shown in Table 3.

Incident seriousness and AP at baseline were linked to the differential PFI efficacy across the BMI and WF conditions. Of the mandated students who experienced a serious incident, those who were assigned to the BMI were more likely to be in the improved than non-improved group, logit = 1.56, odds ratio (OR) = 4.76, 95% Confidence Interval (CI) = 1.21 – 18.66, p < .05 (see Figure 4 for simple slopes). In addition, of the mandated students who reported higher levels of AP at baseline, those who were assigned to the BMI were more likely, compared to those in the WF, to be in the improved group, logit = 1.43, OR = 4.18, 95% CI = 2.02 – 8.64, p < .01 (see Figure 5 for simple slopes). All other individual and situational factors were not statistically significant moderators.¹²

Figure 4.

The interaction of PFI condition with incident seriousness. The BMI is more efficacious than the WF for mandated students who were referred after a serious incident. BMI = Brief Motivational Interview Intervention; WF = Written Feedback Only Intervention.

Figure 5.

The interaction of PFI condition with baseline level of alcohol-related problems (AP). The BMI is more efficacious than the WF for mandated students with higher levels of AP at baseline. BMI = Brief Motivational Interview Intervention; WF = Written Feedback Only Intervention.

Discussion

This study examined whether subgroups exist in their response to a PFI, and whether different PFIs are differentially efficacious for mandated college students. We found heterogeneous subgroups with distinctively different outcome trajectories. Overall, we found that the majority of the mandated students (53.4%) improved in both HED and AP after the PFI regardless of whether they were assigned to the BMI or the WF condition. The non-improved group consisted of 46.6% of the mandated students who improved neither in HED nor AP over the long-term. This group may represent a group who have chronic drinking problems and resist changes in their drinking. However, it is noteworthy that the mandated students in the current study were relatively low-risk individuals compared to other studies that screened and selected high-risk volunteer students (e.g., Chiauzzi et al., 2005; Murphy et al., 2001; Walters, Roudsari, Vader, & Harris, 2007) or other studies of mandated samples (e.g., Barnett et al., 2006) because of the study's clinical exclusion criteria. For example, many of the students in Barnett et al. had been mandated for more serious infractions than ours (e.g., 82% were referred for acute intoxication or an alcohol-related injury compared to only 15% of our students whose incident required police or medical attention).

Predictors of Change, Moderators of the PFI Efficacy, and Clinical Implications

The findings from the present study may provide some answers to the inconsistent findings in the literature. Our findings indicated that it was lighter drinking individuals who improved more following the PFI over a long-term follow-up. This finding is consistent with a recent conclusion from a large meta-analysis that PFIs are more beneficial for lighter-drinking individuals (Carey, Scott-Sheldon, et al., 2007). We also found that there was no overall difference in the efficacy between the BMI and WF groups. Therefore, for mandated students whose baseline levels of HED or AP are low, written or web-based personalized feedback may be a cost-effective way to deliver a PFI as a selective intervention. The present study also demonstrated that the advantage for those who are female, who experienced a serious incident, and who engaged in less frequent HED at baseline was maintained even after taking into account other individual and situational factors as well as pre-intervention AP levels. In previous studies, it has been difficult to assess to what extent ensuing reductions are due to unique effects of individual and situational factors, above and beyond other confounding variables. In the present study, findings for the effects of being a female, experiencing a serious incident and being a less heavy drinker cannot be considered an artifact of omitted baseline confounding factors because these confounded effects were statistically adjusted. There may be other factors that were not considered in the present study. However, the individual and situational factors considered in the present study, as well as the alcohol use controls, represent most of the factors that have been discussed in the literature.

The present study also suggests that it may still be a valuable goal for interventionists to improve readiness to change, as well as to reduce other drug use. In particular, greater readiness to change, being a non first-year student, and no other drug use at baseline predicted a better intervention outcome when baseline HED and AP levels were statistically controlled but not when other individual and situational factors were also examined simultaneously. Although this indicates that there were no unique effects of these variables above and beyond other co-existing individual and situational factors, targeting these co-existing factors might improve intervention outcomes. Whereas experiencing a serious incident, and certainly being a first year student, cannot be subject to change by interventions, early preventive interventions with incoming students might help them reduce problematic behaviors that may lead to serious incidents.

In our analysis of the moderated PFI efficacy, we found that experiencing a serious incident prior to the PFI and reporting high levels of AP at baseline were statistically significant moderators of the PFI efficacy favoring the BMI over the WF. These findings indicate that there is an additional benefit of an in-person, face-to-face motivational interview for students who experienced a serious incident or who reported higher levels of AP at baseline. Other individual and situational factors - gender, first-year in college, other drug use, readiness to change, and positive alcohol expectancies – did not moderate the efficacy of the BMI. In addition, the BMI efficacy did not differ across different levels of HED; that is, no interaction effect was found between BMI and baseline HED.

In a previous study, we reported that there were no group differences in intermediate-term (4 months post intervention) alcohol reductions across the BMI and WF conditions, and discussed that given the cost of administering an in-person motivational interview in terms of time and staffing, providing a written feedback alone may be a cost effective way of reducing alcohol use among mandated college students (White et al., 2006). However, the more long-term follow-up study from the same sample demonstrated that additive benefits of providing an in-person BMI exist for AP above and beyond the benefit from a normative written feedback alone for the mandated students (White et al., 2007). Based on findings from the present study, we conclude that not all mandated students benefit additionally from an in-person BMI. Mandated students with lower levels of AP and HED may benefit just as much from a written personalized feedback alone as from an in-person BMI. The current finding that the BMI was no more efficacious than the WF appears to differ from the previous finding (White et al., 2007). The difference in the findings may be understood in the context of differences in our approaches. First, in the present study, the PFI efficacy was assessed in terms of whether mandated students could be considered as an improved case. In contrast, in the previous study we measured the PFI efficacy using quantitative increments in each outcome variable unit. Therefore, statistically significant treatment group differences from the previous study may not sufficiently translate into a case of improvement (i.e., qualitative distinction) as we examined in the current study. Second, in the present study we analyzed HED and AP simultaneously when identifying heterogeneous subgroups. In contrast, previously we examined each behavioral outcome separately. Taken together, perhaps the best way to understand the findings from these two studies is that incrementally the BMI was more efficacious than the WF especially for AP. However, there was no clear advantage of the BMI over the WF across all individuals when we defined the efficacy outcome as a qualitatively distinct, categorical improvement. It is important to highlight that the BMI was more efficacious than the WF selectively for certain mandated students in the current study. That is, for those mandated students who experienced a serious incident or whose levels of AP at baseline were high, an in-person brief motivational interview was particularly more efficacious than a written feedback alone. These findings underscore the importance of better understanding of the goodness of fit between necessary components of evidence-based treatments and different groups of students with different needs.

Limitations and Contributions

The current findings must be interpreted with caution in light of several limitations. First, we studied mandated students and did not have a true no-treatment control group (because of ethical considerations and program requirements). This restriction most likely decreased our power to detect stronger intervention effects, although our effect sizes were comparable to other studies on mandated and volunteer students (see Carey, Scott-Sheldon, et al., 2007; Larimer & Cronce, 2007) (see also Barnett & Read, 2005 for likely reasons). In addition, the absence of a true control group prohibited us from attributing change to a PFI. Thus, we interpreted individual and situational factors as the predictors of change in the context of a PFI.

Second, findings from studies of mandated students, including the current study, may need to be understood in the context of being mandated. Two recent studies from a different sample that compared a WF with a delayed treatment control reported that mandated students reduced alcohol use on their own prior to the PFI (Morgan et al., 2008), and there were no differences between students who received the WF and those who did not at 2 months post baseline (White et al., 2008). Therefore, reductions in alcohol use among mandated students post intervention may be attributed partially to cognitive and affective reactions to the incident for which they were mandated and subsequent self-regulation. In the present study, we did not have sufficient data on students’ alcohol use following the incident but prior to the PFI. Thus, it is unclear to what extent that students had self-regulated their drinking behaviors on their own prior to the PFI. Nonetheless, the findings from the present study suggest that mandated students following a serious incident tended to reduce their HED and AP over the long term especially if they received the BMI. In addition, mandated students with high levels of AP before the incident were more likely to be one of the improved students, if they were assigned to the BMI.

Third, on a related issue, we did not measure how intoxicated students were when caught, or how aversely or seriously students perceived the incident that they were mandated for. More detailed information regarding the nature and subjective evaluation of the incident would facilitate better understanding of the efficacy of PFIs among mandated students. Fourth, the sample consisted of primarily white and Asian American students and the findings may not generalize to other ethnic/racial groups. In addition, the findings may not be generalized to other mandated student populations with different university policies on alcohol and other drugs and policy enforcement practices (see Barnett et al., 2008).

Fifth, we found from mixture analysis that heterogeneity existed even among the non-improved group. Factors that differentiate the three subgroups and their long-term trajectories may be of interest in order to develop more intensive treatment models for these high-risk groups. The present study did not have sufficient sample size to conduct comparative analysis on these groups. A larger scale study of mandated students will allow for researchers to examine whether the three distinctive groups, identified based on statistical considerations such as the BIC and entropy statistics, can be useful in practice (Everitt, Landau, & Leese, 2001; Muthén & Muthén, 2000).

Finally, we examined the potential moderators one by one in the current study because little is known about predictors and moderators of the PFI efficacy in the literature. In addition, power to detect moderation effects is well known to be low (McClelland & Judd, 1993; Sackett et al., 1986). Given that identifying different subgroups is critical for screening, triaging, and implementing cost-effective interventions for those students in need, the current study adopted an exploratory approach. With a larger scale study designed to test moderation effects, it would be preferable to examine potential moderators simultaneously to understand their unique contributions.

Despite these limitations, the present study contributes to prevention research for alcohol use and AP among emerging adults and more broadly to evidence-based treatment research. First, the present study sheds new light on predictors of change in the context of a PFI and the efficacy of an in-person PFI delivered within a BMI among mandated students. Based on these findings, we suggest that it may be more cost-effective to deliver a written or web-based PFI for low-risk mandated students while providing an enhanced PFI with an in-person BMI for those who experienced a serious incident or those with higher levels of AP at baseline. A two-session intervention utilizing an in-person motivational interview with personalized normative feedback presents a relatively low-cost psychological intervention. However, even at low cost, findings from the current study suggest that an in-person BMI does not provide an additional benefit over a written PFI for many low-risk mandated students. Therefore, more research that could further identify other important moderators of PFIs among mandated and volunteer students is sorely needed to identify which students require which types of interventions.

Second, this study also draws attention to the utility of a person-oriented approach (Bergman & Magnusson, 1997; von Eye & Bergman, 2003) for evaluation research, and of the integrative strategy between person-oriented and variable-oriented approaches (Bates, 2000) to clinical research more broadly. As Foster, Dodge, and Jones (2003) recently discussed, many prevention and treatment studies are conducted from a variable-oriented perspective. Foster and colleagues illustrated that while studies utilizing a variable-oriented approach allow one to measure cost-effectiveness per one unit improvement in a single outcome measure, it is difficult to answer whether the cost of interventions outweighs benefits when the emphasis lies not on persons, but on variables. It is especially challenging when outcomes co-occur. Foster and colleagues, therefore, suggested that a person-oriented outcome may be used as a global measure of cost-effectiveness for prevention research. Identifying heterogeneous subgroups based on two related outcome measures (e.g., AP and HED) may be more insightful than those from an isolated single outcome when assessing clinical significance at the individual level or global cost-effectiveness. Recent advances in longitudinal research methodology (see Foster & Kalil, 2008) provide attractive analytic options for evidence-based intervention research. The refined focus on subgroup analysis utilized in the present study may be beneficial in the future for tailoring necessary intervention components to those who need them the most.

Appendix

With three latent variables, the long-term follow-up outcome level (Level), initial change (IC) from baseline to 4 months post intervention, and subsequent change (SC) from 4 months to 15 months post intervention, the factor loading matrix for each repeated measures outcome shown in Figure 2 was specified

$$\Lambda =\left(\begin{array}{ccc}\hfill 1\hfill & \hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill & \hfill 0\hfill \end{array}\right).$$

Thus, the observation y for individual i at Time 1, Time 2, and Time 3 can be expressed as

$$\begin{array}{c}\hfill {\mathit{y}}_{\mathit{i}1}={\mathit{Level}}_{\mathit{i}}+{\left(\mathit{IC}\right)}_{\mathit{i}}+{\left(\mathit{SC}\right)}_{\mathit{i}}+{∊}_{\mathit{i}1}\hfill \\ \hfill {\mathit{y}}_{i2}={\mathit{Level}}_{\mathit{i}}+{\left(\mathit{SC}\right)}_{\mathit{i}}+{∊}_{i2}\hfill \\ \hfill {\mathit{y}}_{\mathit{i}3}={\mathit{Level}}_{\mathit{i}}+{∊}_{\mathit{i}3}.\hfill \end{array}$$

Therefore, the expected average at Time 3 indicates the long-term follow-up outcome level. The expected average changes from baseline and to the 4-month, and from the 4-month to the 15-month assessment are indicated by IC and SC, respectively (for greater detail, see Mun et al., 2009).