Efficacy of Web-Based Personalized Normative Feedback: A Two-Year Randomized Controlled Trial

Abstract

Objective

Web-based brief alcohol interventions have the potential to reach a large number of individuals at low cost; however, few controlled evaluations have been conducted to date. The present study was designed to evaluate the efficacy of gender-specific versus gender-nonspecific personalized normative feedback (PNF) with single versus biannual administration in a 2-year randomized controlled trial targeting a large sample of heavy-drinking college students.

Method

Participants included 818 freshmen (57.6% women; 42% non-Caucasian) who reported 1 or more heavy-drinking episodes in the previous month at baseline. Participants were randomly assigned in a 2 (gender-specific vs. gender-nonspecific PNF) × 2 (single vs. biannual administration of PNF) + 1 (attention control) design. Assessments occurred every 6 months for a 2-year period.

Results

Results from hierarchical generalized linear models provided modest effects on weekly drinking and alcohol-related problems but not on heavy episodic drinking. Relative to control, gender-specific biannual PNF was associated with reductions over time in weekly drinking (d = −0.16, 95% CI [−0.02, −0.31]), and this effect was partially mediated by changes in perceived norms. For women, but not men, gender-specific biannual PNF was associated with reductions over time in alcohol-related problems relative to control (d =−0.29, 95% CI [−0.15,−0.58]). Few other effects were evident.

Conclusions

The present research provides modest support for the use of biannually administered web-based gender-specific PNF as an alternative to more costly indicated prevention strategies.

Web-based alcohol interventions have the potential to reach a large number of people at relatively low cost. Despite detailed advantages and optimistic reviews of web-based interventions (Cunningham, 2007), a surprisingly small number of controlled evaluations have been conducted to date. The majority of computer-delivered interventions use gender-nonspecific normative referents and have few and/or short follow-up assessments (Carey, Scott-Sheldon, Elliott, Bolles, & Carey, 2010). Moreover, most interventions consist of a single administration. The present study was designed to evaluate the efficacy of gender-specific versus gender-nonspecific personalized normative feedback (PNF) with single versus biannual administration in a 2-year randomized controlled trial targeting a large sample of heavy-drinking college students.

Previous Randomized Controlled Trials Evaluating Web-Based Alcohol Interventions

Research examining the efficacy of web-based interventions has produced mixed findings, with some studies reporting greater reductions in alcohol consumption among web-based intervention participants relative to control participants (Bewick, Trusler, Mulhern, Barkham, & Hill, 2008; Chiauzzi, Green, Lord, Thum, & Goldstein, 2005; Neighbors, Lee, Lewis, Fossos, & Walter, 2009; Riper et al., 2008) and other studies finding no differences (Moore, Soderquist, & Werch, 2005; Saitz et al., 2007; Walters, Vader, Harris, Field, & Jouriles, 2009). Findings from a recent meta-analysis of computer-delivered alcohol interventions (including web-based interventions) suggest that these interventions had stronger effects in earlier publications, in studies with fewer participants, and when a commercial product was not utilized (Carey et al., 2010). The authors suggested that the likelihood of detecting trends diminishes as evaluation trials become larger and achieve greater dissemination—which they indicated is similar to the transition from efficacy to effectiveness.

Additionally, few of the studies evaluated in this meta-analysis had more than one follow-up assessment or follow-up assessments that extended beyond 6 weeks, limiting the ability to determine whether improvements persisted over time or rather represented more immediate change. Results from existing studies suggest that the ability of web-delivered interventions to induce change may be short lived. For example, some studies have found immediate effects of web-based interventions on drinking that were no longer evident at the terminal follow-up (Chiauzzi et al., 2005; Walters, Vader, & Harris, 2007). Although these studies have contributed to the understanding and advancement of web-based alcohol interventions, additional studies with larger samples and more extensive follow-up assessments are clearly needed. Moreover, results from these studies may suggest the need for biannual administration or booster follow-ups for web-based interventions.

PNF

Prior research has demonstrated that peer drinking norms are among the strongest influences on students' personal drinking behaviors compared with the influence of parents, resident advisors, and faculty (Perkins, 2002). Perceived prevalence of drinking by other students (i.e., descriptive drinking norms) has been found to account for more variance in alcohol consumption relative to gender, Greek membership, alcohol expectancies, subjective evaluation of alcohol effects, drinking motives, and perceived approval of drinking by friends and parents (Neighbors, Lee, Lewis, Fossos, & Larimer, 2007). These results are consistent with numerous other studies documenting the strong association between perceptions of other students' drinking and personal alcohol consumption (for a meta-analysis, see Borsari & Carey, 2003).

In addition to research demonstrating the strong relationship between descriptive drinking norms and alcohol consumption, research also consistently has indicated that students overestimate heavy drinking among their peers (Borsari & Carey, 2001, 2003; Carey, Borsari, Carey, & Maisto, 2006; Lewis & Neighbors, 2004; Lewis, Neighbors, Oster-Aaland, Kirkeby, & Larimer, 2007; Neighbors, Dillard, Lewis, Bergstrom, & Neil, 2006). For example, Neighbors, Dillard, et al. (2006) demonstrated that students' estimates of other students' drinking were approximately double the actual rates for both drinking quantity and drinking frequency.

One promising strategy in addressing heavy drinking among college students is PNF, which is designed to reduce normative misperceptions and thereby utilize social influence as an intervention modality. Given findings that students overestimate the drinking of other students and that these perceptions are strongly predictive of drinking behavior, correction of overestimated normative perceptions has become a prominent focus of many college drinking intervention studies (for reviews, see Carey, Scott-Sheldon, Carey, & DeMartini, 2007; Larimer & Cronce, 2002, 2007; Lewis & Neighbors, 2006a; Walters & Neighbors, 2005).

Normative feedback aimed at reducing overestimated normative perceptions has been incorporated in several multicomponent interventions (e.g., Baer et al., 1992; Borsari & Carey, 2000; Dimeff, Baer, Kivlahan, & Marlatt, 1999; Larimer et al., 2001, 2007; Marlatt et al., 1998; Walters et al., 2007; Walters, Vader, Harris, Field, & Jouriles, 2009). When examined as a single-component intervention, findings from four randomized controlled trials indicate that computer-delivered PNF in a laboratory setting is effective in reducing normative misperceptions and alcohol consumption among heavy-drinking students (Lewis & Neighbors, 2007; Lewis et al., 2007; Neighbors, Larimer, & Lewis, 2004; Neighbors, Lewis, Bergstrom, & Larimer, 2006). Between-subjects effect sizes for PNF across these studies ranged from 0.61 to 0.96 on normative perceptions and from 0.35 to 0.97 on drinking behavior relative to assessment only, with follow-up periods ranging from 1 to 6 months postbaseline.

Previous Research Using PNF

Although the research examining the efficacy of PNF as a stand-alone intervention has been computer-delivered (Lewis & Neighbors, 2007; Lewis et al., 2007; Neighbors et al., 2004; Neighbors, Lewis, et al., 2006), it is important to note that all of these studies were conducted in a laboratory setting and thus involved some form of face-to-face contact with a representative of the study. In contrast, web-based interventions are completely self-guided and can be accessed and completed by participants on demand, offsite, and in an unsupervised context. Web-based interventions that do not require face-to-face contact are a natural extension of computer-based interventions administered in person.

For researchers, web-based interventions provide cost efficiency, uniformity of delivery, and the potential for widespread dissemination that is not possible with more labor-intensive, in-person interventions (Portnoy, Scott-Sheldon, Johnson, & Carey, 2008; Zisserson, Palfai, & Saitz, 2007). For college students, web-based interventions allow for anonymity, economy of time, and convenience of access (Koski-Jannes & Cunningham, 2001; Kypri, Saunders, & Gallagher, 2003). When deciding between reading materials, health education seminars, web-based assessments, or assessments by a professional, the majority of college problem drinkers selected web-based assessments as the most appealing intervention (Kypri et al., 2003). Research has yet to examine the efficacy of PNF as a stand-alone intervention delivered via the web, outside of the laboratory, with no face-to-face contact with study researchers. Thus, the present study was designed to fill this gap in the literature.

Additionally, previous studies examining the efficacy of computer-delivered PNF (Lewis & Neighbors, 2007; Lewis et al., 2007; Neighbors et al., 2004; Neighbors, Lewis, et al., 2006) have all consisted of short follow-up periods ranging from 1 to 6 months. Moreover, the PNF intervention has consisted of a single administration. On the basis of research suggesting that the reduction in drinking from web-delivered interventions may be short-lived (Chiauzzi et al., 2005; Walters et al., 2007), the present study was designed to evaluate the efficacy of web-based PNF with single versus biannual administration in a 2-year randomized controlled trial.

Gender-Specific Versus Gender-Nonspecific Normative Comparisons

In a recent meta-analysis of computer-delivered alcohol interventions for college students (which included web-based interventions), 77% of interventions provided generic normative comparisons, and only 25% of interventions provided normative comparisons matched to participant characteristics (Carey et al., 2010). However, previous research has suggested that using gender-specific normative referents may be more effective. Previous research has demonstrated that normative perceptions for same-sex normative referents are more strongly associated with personal drinking quantity and drinking frequency than opposite-sex normative referents or gender-nonspecific normative referents (i.e., typical student; Lewis & Neighbors, 2004; Lewis et al., 2007). In addition, Neighbors et al. (in press) found that the relationship between perceived descriptive drinking norms and alcohol consumption was moderated by level of identification with the normative referent (i.e., typical same-sex student, typical same-race student, and typical same-Greek-status student). Findings indicated that when examining these relationships for all three normative referent groups, descriptive normative perceptions for the normative referent were more strongly associated with alcohol consumption when participants reported stronger identification to the normative referent. These findings are consistent with several theories (i.e., social comparison, Festinger, 1954; social identity, Tajfel, 1982; self-categorization, Turner, Hogg, Oaks, Reicher, & Wetherell, 1987) that suggest that socially proximal normative referents (i.e., same-sex student) should have greater influence on one's behavior in comparison with socially distal normative referents (i.e., opposite-sex student; typical student). When comparing the short-term efficacy of gender-nonspecific and gender-specific PNF, there were no significant differences (Lewis & Neighbors, 2007; Lewis et al., 2007). However, Lewis and Neighbors (2007) found that same-sex, gender-specific PNF was especially effective for women who more strongly identified with their gender. In addition, the follow-up periods for these two studies were relatively short (i.e., 1 month and 5 months), thus it is unclear whether gender-specific PNF would be more efficacious than gender-nonspecific PNF when evaluated with a longer follow-up assessment. In the present study, we aimed to examine the long-term efficacy of same-sex gender-specific PNF in comparison with gender-nonspecific PNF.

Present Study

The present study extends previous research by (a) being entirely web-based, (b) extending follow-up assessments to 2 years, (c) evaluating the utility of biannual administration, and (d) further examining the impact of providing gender-specific versus gender-nonspecific normative information. On the basis of the above considerations, we expected that gender-specific PNF would be more effective than both gender-nonspecific PNF and attention control at reducing perceived drinking and self-reported drinking behavior. Moreover, we expected that biannual delivery of gender-specific PNF would result in the largest reductions in perceived drinking and self-reported drinking. Finally, we expected that changes in perceived drinking for gender-specific and gender-nonspecific peers would mediate intervention effects.

Method

Participants

Participant flow throughout this study is presented in Figure 1. The invited sample consisted of 4,103 freshmen students at a large public northwestern university. The average age of invited students was 18.7 years (SD = 0.5). Gender and ethnic representation of the invited sample was 47.45% male, 51.04% Caucasian, 28.05% Asian, and 20.91% other ethnicities (none higher than 5%) or not indicated.

Figure 1.

Participant flow chart. GSF = gender-specific feedback; bac = blood alcohol content; GNSF = gender-nonspecific feedback; mo = months.

Just over half (51.1%) of the invited students completed the screening survey (N = 2,095). Students who completed the screening survey had an average age of 18.16 years (SD = 0.6). Gender and ethnic representation of those who completed the screening survey was 42.20% male, 58.04% Caucasian, 31.12% Asian, and 10.84% other ethnicities. Those who completed the screening survey were younger than those who did not, t(4102) = 32.10, p < .001. Caucasian and Asian/Pacific Islander participants were more likely to complete the screen relative to other students (ps < .001).

Of the 2,095 students who completed the screening questionnaire, 898 (56.68% female) met the drinking eligibility criteria of at least five/four drinks for men/women, respectively, on one or more occasions during the past month and were invited to complete the baseline assessment. Eligible participants did not differ from noneligible participants with respect to age or ethnicity with the exception that they were less likely to be Asian/Pacific Islander (p < .01).

Of the 898 eligible participants, 818 (91.09%; 57.58% female) completed the baseline assessment and were included as participants in the longitudinal study. Baseline completers did not significantly differ from noncompleters in age or ethnicity. Ethnicity/racial representation was 65.28% Caucasian, 24.21% Asian/Pacific Islander, 4.16% Hispanic/Latino, 1.47% African American, 0.49% Native American/American Indian, and 4.40% other.

Procedures

In the fall of 2005, all incoming freshmen students not already participating in a similar ongoing study evaluating marijuana use in the transition to college were invited to complete a 20-min, web-based screening survey. Invitations for the screening survey were sent by e-mail and U.S. post and included a brief description of the survey. Participants were informed that the survey would ask about their personal characteristics, drinking patterns, alcohol-related consequences, and perceptions of other students' drinking on their campus. Participants were also informed that if the study was right for them, they would be invited to complete a 50-min survey immediately following the 20-min screening survey (or within 2 weeks) and four additional 50-min surveys at 6-month intervals. They were notified that after completing the first 50-min survey, they would be randomly assigned (like flipping a coin) to receive or not to receive information comparing their drinking practices with other students' drinking practices at their university. Information statements and decline postcards were sent to the parents of students who were not yet 18 years of age (n = 164). Consent documents indicated the study was designed to examine relationships among social norms, motivation, and drinking among college students and to consider the influence of normative information on perceived norms and drinking behavior. A Federal Certificate of Confidentiality (AA-79-2005) was obtained to help ensure privacy of research participants. All procedures were approved by the university's Institutional Review Board. No adverse events were reported.

A priori power analyses indicated that 800 participants would provide adequate power for detecting effect sizes in the small to medium range. Thus, we aimed to invite 4,000 students, with the expectation that 50% would complete screening (N = 2,000), and 40% of those would meet the eligibility criteria and would complete baseline (n = 800). Eligibility criteria included consumption of at least five/four drinks for men/women, respectively, on one or more occasions during the past month at screening.

Students who met screening criteria were given the option to complete the baseline survey immediately or, if they preferred, to return to complete it within 2 weeks. Those who chose to complete the baseline survey immediately were seamlessly routed to the baseline survey. Those who elected not to complete the baseline survey immediately were sent two e-mails. One e-mail contained a link to the survey. A second e-mail contained their unique pin for logging-in to the baseline survey. Participants received up to three e-mail reminders, one phone call reminder, and one postcard reminder before being removed from the invitation list.

All participants were randomly assigned to one of five conditions immediately after completing the baseline assessment. All measures and interventions were completed entirely via the Internet. Incentives for participation were $10 for completing the screening survey, $25 for completing the baseline survey, and $25 for completing each of the follow-up assessments at 6, 12, 18, and 24 months postbaseline. Participants were carefully monitored at each assessment point for consumption of potentially lethal amounts of alcohol (blood alcohol content of .35 or greater). Those reporting consumption at this potentially lethal level were sent information regarding the risks of drinking at their reported level (baseline n = 134, 6-month n = 85, 12-month n = 74, 18-month n = 71, 24-month n = 69). The proportion of participants contacted did not vary significantly across conditions at any assessment point (see Figure 1).

Randomization

In the study, we utilized a 2-year longitudinal randomized 2 (feedback interval) × 2 (feedback gender specificity) + 1 (assessment only control) experimental design. Participants were randomly assigned to (a) attention control (no PNF); (b) single exposure to PNF following the baseline assessment; and (c) biannual exposure of PNF delivered following baseline and after the 6-, 12-, and 18-month assessments. Participants receiving feedback were also randomly assigned to receive either (a) gender-specific or (b) gender-nonspecific normative feedback. Random assignment was administered automatically using a computer algorithm and occurred in blocks of five to keep cell sizes equal.

Intervention

The PNF interventions were modeled after the gender-nonspecific intervention from Neighbors et al. (2004) and the gender-specific intervention from Lewis and Neighbors (2007). These interventions were developed on the basis of the normative feedback component of the Brief Alcohol Screening and Intervention for College Students (BASICS) intervention (Dimeff et al., 1999). Following the conceptualization of PNF as personalized information designed to correct overestimated normative perceptions, this intervention was extremely brief and contained only three required elements, which included information regarding (a) one's own drinking behavior, (b) one's perceptions of other students' drinking behavior on the participating campus, and (c) other students' self-reported drinking behavior in text and bar graph formats. Together, these three pieces of information explicitly illustrated that participants overestimated the prevalence of drinking among their peers and, for participants who reported heavy drinking, that most students drank less than the participant did. Bar graphs were provided for weekly frequency and number of drinks consumed per week. Each graph included three bars representing the campus norm, the participants' reported perception of the campus norm, and the participants' reported behavior. Normative feedback about episodic heavy drinking was not provided. Participants randomized to the feedback conditions were given feedback regardless of whether they overestimated the campus norm. The structures of the bar graphs were individually tailored to the participants' data so that, for each graph, the scale on the y-axis was dependent on the maximum of these three values for each participant. Participants were also provided with their percentile rank comparing them with other students (e.g., “Your percentile rank is 96%, which suggests that you drink more than 96% of other college students”). Participants were notified at each time-point that the information contained in the feedback came from a random sample of 2,548 freshmen students at their university.

Gender-nonspecific feedback

Gender-nonspecific feedback was identical to the feedback described above, thus perceived and actual norms presented in text and graphs were based on the “average” student (without reference to the gender of the average student) at his/her university.

Gender-specific feedback

Gender-specific feedback presented feedback regarding the students' own drinking behavior, the students' reported perception of typical drinking by the average same-sex student at his/her university, and actual typical drinking by same-sex students at his/her university. Thus, gender-specific feedback followed the same format described above with the exceptions that perceived and actual norms were based on the “average same-sex” student.

Attention control participants

Attention control participants received facts about students at the university that were generated from a recent large survey. For example, students were told that 49% of students at the university play a musical instrument and that 65% work during the school year. The layout of the attention control information mirrored the layout of the normative feedback, with text on the left and two graphs on the right. However, none of the information presented directly related to alcohol, and it was not personalized to the participant.

Single versus biannual administration of feedback

Students randomized to the biannual administration of feedback received the same feedback (either gender-specific or gender-nonspecific, depending on initial randomization) at the completion of each follow-up survey. Normative information was the same as initially presented in the first PNF, though participants' own drinking and perceived norms utilized in the PNF were based on information provided in the most recent assessment. Students randomized to the single administration of gender-specific or gender-nonspecific PNF received attention control information following all assessments except baseline.

Measures

All assessments were completed over the Internet and included measures of demographics, alcohol consumption, alcohol-related problems, and perceived norms. Details regarding measures utilized in this evaluation are provided below. Additional constructs assessed at the 20-min screening and/or one or more of the 50-min surveys included drinking motives, alcohol expectancies, readiness to change, social desirability, self-determination, relationship conflicts, sexual risk behaviors, social identity, social adjustment, and self-esteem.

Alcohol consumption

Typical weekly drinking was assessed using the Daily Drinking Questionnaire (DDQ; Collins, Parks, & Marlatt, 1985; Dimeff et al., 1999), which asks the participant to report the average number of drinks consumed on each day of a typical week over the previous 3 months. It has been extensively used in the college-drinking literature and has demonstrated good construct validity and test-retest reliability (Neighbors, Lewis, et al., 2006). The DDQ was scored by summing the responses for each day of the week. Thus, scores reflect average number of drinks per week over the previous 3 months. Frequency of heavy episodic drinking was assessed with a composite of four items from the Alcohol Consumption Index (Knee & Neighbors, 2002). Items asked participants the number of times they had consumed five or more drinks at one sitting in the past week, in an average week, in the past month, and in an average month (alphas at all time-points were >.90).

Alcohol-related problems

The Rutgers Alcohol Problem Index (RAPI; White & Labouvie, 1989) was used to assess alcohol-related problems. The scale includes 23 items assessing the number of times participants experienced each alcohol-related problem in the previous 3 months. Sample items include the following: “Caused shame or embarrassment to someone?” and “Missed a day (or part of a day) of school or work?” Two items were added to the scale to examine the frequency of driving after consuming two and four or more drinks. Responses ranged from 1 (never) to 5 (more than 10 times). Items were summed to create a composite. Alphas at all time-points were >.85.

Perceived gender-nonspecific drinking norms were measured using the Drinking Norms Rating Form (DNRF; Baer, Stacy, & Larimer, 1991; Dimeff et al., 1999). The DNRF mirrors the DDQ and asks participants to estimate the number of drinks they think the typical student has on each day of the week. Perceived weekly descriptive drinking norms were calculated by summing the participants' estimations for each day of the week. This measure has demonstrated good test–retest reliability and convergent validity (Baer et al., 1991; Borsari & Carey, 2000; Neighbors, Lewis, et al., 2006).

Perceived gender-specific drinking norms were measured using a modified version of the DNRF (Lewis & Neighbors, 2004, 2007). This measure was identical to the DNRF with the exemption that participants were asked to estimate the typical drinking of other students of the same sex.

Statistical Methods

Primary outcomes were weekly drinks, alcohol-related problems (i.e., RAPI), and heavy episodic drinking. Gender-specific norms and gender-nonspecific norms were evaluated as mediators. All of these variables are count variables. Count variables are nonnegative integers and tend to have positively skewed distributions that are more appropriately modeled by the Poisson (or negative binomial) distribution, as opposed to regression methods that assume normality of the residuals (for a review of count regression models, see Atkins & Gallop, 2007). The primary analytic model was a hierarchical generalized linear model (HGLM; Raudenbush & Bryk, 2002) assuming a Poisson distribution for the Level 1 outcomes. The following system of equations describes the basic model that was used in most analyses:

$$\begin{array}{cc}\hfill \log \left(DV\right)& ={\pi }_{0i}+{\pi }_{1i}{\mathit{Time}}_{ti},\hfill \\ \hfill {\pi }_{0i}& ={\beta }_{00}+{\beta }_{01-04}Tx+{\beta }_{05}D{V}_{\mathit{pre}}+{\beta }_{06}\mathit{Gender}+r_{0i},\hfill \\ \hfill {\pi }_{1i}& ={\beta }_{10}+{\beta }_{11-14}Tx+r_{1i},\hfill \end{array}$$

where t indexes time, and i indexes individuals. Time measures months since postintervention. Tx is a set of four dummy-variables comparing each active treatment with the no treatment control (note that we do not represent each individual dummy-variable above, but do note that there are four coefficients, i.e., β_01–04). The baseline value of the dependent variable is included as a Level 2 covariate (DV_pre) so that the outcome is reserved for values that could be affected by the treatment. Thus, the intercept represents the dependent value at 6-month follow-up controlling for the baseline dependent value, and main effects of condition represent differences at the first follow-up. Gender (0 = women, 1 = men) was included as a main effect in all models, and exploratory models examined gender interactions by treatment. Exponentiated coefficients are interpreted as percentage of change in rate (Atkins & Gallop, 2007; Long, 1997).

Mediation analyses are complicated somewhat by the multilevel and nonnormal distribution of the data. Kenny et al. (2004) described mediation and provided an example using multilevel models for longitudinal treatment data, much like the present data. Following this approach, we compared treatment main effects (β_01–04) and treatment cross-level interactions with time (β_11–14) for the model presented above with one in which norms (either gender-specific or gender-nonspecific) were included as a Level 1 time-varying covariate. We used this approach and considered the percentage of reduction in effects after the mediator was included in the model. Effect sizes were calculated using the equation d = 2Z/√N (Rosenthal, Rosnow, & Rubin, 2000). All analyses were conducted in R v2.8.1 (R Development Core Team, 2008) and made use of the lme4 (Bates, Maechler, & Dai, 2008) package of functions for generalized linear mixed models and the MCMCglmm package for Bayesian ZIP mixed models.

Results

Descriptive Analyses

Figure 2 presents unadjusted means and 95% confidence intervals for drinking outcomes and perceived norms by assessment point and intervention group. Consistent with past research, student's perceptions of weekly drinking were notably higher than actual drinking and were quite similar between gender-specific and general norms. Most outcomes improved (i.e., decreased) after intervention and over 2 years following intervention. In general, this is consistent across treatment groups, though it is also true of the control group. One notable exception to this general pattern is the RAPI, in which the gender-nonspecific biannual group appears to worsen over time.

Figure 2.

Unadjusted means and standard deviations by condition for perceived norms and drinking outcomes. GNS = gender-nonspecific; GS = gender-specific; RAPI = Rutgers Alcohol Problem Index (used to assess alcohol-related problems); mos = months; DDQ = Daily Drinking Questionnaire (used to assess drinks per week); NSNORM = gender-nonspecific perceived norms; SSNORM = gender-specific perceived norms; HED = heavy episodic drinking.

Factorial Design: Gender and Timing of Interventions

With the current 2 (gender-specific vs. gender-nonspecific) × 2 (single intervention vs. biannual intervention) + 1 (control) design, there are two logical sets of comparisons. One set of comparisons is between each intervention group and the control, which were tested via dummy-variables, whereas the second set of contrasts is the two design factors (i.e., gender-specific vs. gender-nonspecific, averaging over levels of single vs. biannual, and vice versa). These were tested via planned comparisons on the output from the HGLM models.

There were two notable effects considering the factorial design. First, there was a strong three-way interaction between participant's gender, gender-specific conditions, and time for the RAPI (Z = −2.75, p < .01, d = −0.19). Simple slopes revealed that women who received a gender-specific intervention decreased their alcohol-related problems significantly over time (rate ratio [RR] = 0.95, Z = −4.2, p < .01, d = −0.29). The RR suggests a 5% reduction per month, or a 63% total reduction from 6 months (predicted mean of 4.7) to 24 months postbaseline (predicted mean of 1.7). There were no significant effects for the two design factors across other outcomes.

Individual Treatment Comparisons

Baseline treatment differences

The five individual treatment groups were compared at baseline for each outcome using negative binomial regressions. There were no significant differences between treatment groups on the basis of omnibus tests for differences in group means (all ps > .12). Figure 3 presents effect sizes (ds) and 95% confidence intervals representing differences between each treatment and the control group for all outcomes.

Figure 3.

Effect sizes (ds) and 95% confidence intervals for Treatment versus Control × Time interactions for drinks per week, alcohol-related problems, and heavy episodic drinking. GNS = gender-nonspecific; GS = gender-specific.

Weekly drinking

Table 1 has results from an HGLM analysis using a Poisson model for the Level 1 errors. Similar to past literature, results show that men reported more drinking than women (approximately 29% more), and those with a greater baseline drinking reported increased drinking over follow-up. Examining treatment differences, there were no significant effects at the first follow-up assessment. However, across all follow-up assessments, there was a significant interaction with time for the biannual gender-specific intervention relative to control group but not for other intervention groups. The biannual gender-specific intervention group reported approximately 1.5% fewer average weekly drinks. From the 6-month follow-up assessment point to the 24-month follow-up assessment point, this difference translates to 22% less drinking relative to the control group (i.e., RR = e^{[18 × −0.014]} = 0.78). Predicted mean drinking for the biannual gender-specific group goes from 10.5 at postintervention (i.e., 6-month assessment) to 8.2 at 24-month follow-up.¹

Table 1.

Changes in Weekly Drinking by Intervention Condition Using HGLM With a Poisson Model for Level 1 Errors

Variable	B	SE B	Z	d	P	eB	Lower 95% eB	Upper 95% eB
Intercept	1.87	0.07	27.74	1.94	.00	6.48	5.67	7.41
Baseline Drinks per Week	0.04	0.00	15.90	1.11	.00	1.04	1.04	1.04
Gender	0.25	0.06	4.62	0.32	.00	1.29	1.16	1.44
GNS One Time	0.16	0.09	1.78	0.12	.08	1.18	0.98	1.41
GNS Biannual	−0.07	0.09	−0.73	−0.05	.47	0.94	0.78	1.12
GS One Time	0.02	0.09	0.20	0.01	.84	1.02	0.85	1.22
GS Biannual	0.11	0.09	1.18	0.08	.24	1.11	0.93	1.34
Time	0.00	0.00	0.07	0.00	.95	1.00	0.99	1.01
Time × GNS One Time	−0.01	0.01	−1.20	−0.08	.23	0.99	0.98	1.01
Time × GNS Biannual	−0.01	0.01	−1.00	−0.07	.32	0.99	0.98	1.01
Time × GS One Time	−0.01	0.01	−1.01	−0.07	.31	0.99	0.98	1.01
Time × GS Biannual	−0.01	0.01	−2.29	−0.16	.02	0.99	0.97	1.00

Note. HGLM = hierarchical generalized linear model; GNS = gender-nonspecific; GS = gender-specific.

Alcohol-related problems

Table 2 has results from an HGLM analysis using a Poisson model for the Level 1 errors. The results of the HGLM applied to the RAPI revealed no significant effects at the first follow-up assessment. There was significant improvement for all groups over time (RR = 0.975, Z = −2.8, p < .01, d = −0.20), but there were no significant effects due to individual treatment conditions. However, there was a significant gender interaction such that women who received the biannual gender-specific intervention significantly improved over and above the control over follow-up (RR = 0.972, Z = −2.2, p = .03, d = −0.15), but men did not show this effect (RR = 1.03, Z = 1.4, p = .17, d = 0.10). There was a similar pattern of coefficients across the two genders in the single gender-specific conditions, but these coefficients did not reach significance.

Table 2.

Changes in Alcohol-Related Problems by Intervention Condition Using HGLM With a Poisson Model for Level 1 Errors

Variable	B	SE B	Z	d	P	eB	Lower 95% eB	Upper 95% eB
Intercept	0.99	0.10	10.22	0.71	.00	2.69	2.21	3.26
Baseline Alcohol-Related Problems	0.07	0.01	14.36	1.00	.00	1.07	1.06	1.08
Gender	0.31	0.08	3.90	0.27	.00	1.36	1.16	1.59
GNS One Time	0.35	0.13	2.71	0.18	.01	1.42	1.10	1.85
GNS Biannual	0.17	0.13	1.34	0.09	.18	1.19	0.92	1.54
GS One Time	0.30	0.13	2.27	0.16	.02	1.34	1.04	1.74
GS Biannual	0.24	0.13	1.79	0.13	.07	1.27	0.97	1.64
Time	−0.02	0.01	−2.87	−0.20	.00	0.98	0.97	0.99
Time × GNS One Time	0.00	0.01	0.27	0.02	.79	1.00	0.98	1.02
Time × GNS Biannual	−0.01	0.01	−0.88	−0.06	.38	0.99	0.97	1.01
Time × GS One Time	−0.02	0.01	−1.60	−0.11	.11	0.98	0.97	1.00
Time × GS Biannual	−0.01	0.01	−1.31	−0.09	.19	0.99	0.97	1.01

Note. HGLM = hierarchical generalized linear model; GNS = gender-nonspecific; GS = gender-specific.

Heavy episodic drinking

Table 3 has results from an HGLM analysis using a Poisson model for the Level 1 errors. The HGLM results revealed no significant group differences at postintervention. All groups were reducing heavy episodic drinking over time (RR = 0.98, Z = −3.4, p < .01, d = −0.24). However, no treatment groups were different from control.

Table 3.

Changes in Heavy Episodic Drinking by Intervention Condition Using HGLM With a Poisson Model for Level 1 Errors

Variable	B	SE B	z	d	P	eB	Lower 95% eB	Upper 95% eB
Intercept	1.28	0.08	15.87	1.11	.00	3.59	3.06	4.23
Baseline Heavy Episodic	0.09	0.01	14.23	1.00	.00	1.10	1.08	1.11
Gender	0.10	0.07	1.43	0.10	.15	1.10	0.96	1.26
GNS One Time	0.09	0.11	0.82	0.06	.41	1.09	0.88	1.36
GNS Biannual	0.00	0.11	0.01	0.00	1.00	1.00	0.80	1.25
GS One Time	0.10	0.11	0.92	0.06	.36	1.11	0.89	1.37
GS Biannual	0.09	0.11	0.81	0.06	.42	1.09	0.88	1.36
Time	−0.02	0.01	−3.37	−0.24	.00	0.98	0.97	0.99
Time × GNS One Time	0.00	0.01	0.35	0.02	.73	1.00	0.99	1.02
Time × GNS Biannual	0.00	0.01	0.46	0.03	.64	1.00	0.99	1.02
Time × GS One Time	−0.01	0.01	−0.88	−0.06	.38	0.99	0.98	1.01
Time × GS Biannual	−0.01	0.01	−1.09	−0.08	.28	0.99	0.98	1.01

Note. HGLM = hierarchical generalized linear model; GNS = gender-nonspecific; GS = gender-specific.

Mediation

Gender-specific perceived norms

Table 4 contains the results from an HGLM with Poisson Level 1 errors. Results show that treatments had a strong impact on perceived norms. Similar to previous analyses, men reported higher perceived norms, and baseline norms were related to postintervention norms. Each treatment condition reported lower, postintervention norms relative to the control group (approximately 6%–8% reduced frequency relative to control participants). In addition, results show that the control group's perceptions of weekly drinking were dropping over time, though two intervention conditions show significant reductions over and above the drop by the control group. Gender-nonspecific one-time intervention and the biannual gender-specific intervention showed significant reductions relative to the control group over time. Over 18 months of follow-up, this leads to a 30% reduction {e^{[18 × (−0.012 −0.008)]} = 0.70} in perceptions of normative drinking behavior (for biannual gender-specific group). Figure 4 presents effect sizes (ds) and 95% confidence intervals representing differences between each treatment and the control group for gender-specific and gender-nonspecific norms.

Table 4.

Changes in Gender-Specific (GS) Perceived Norms by Intervention Condition Using HGLM With Poisson Level 1 Errors

Variable	B	SE B	z	d	P	eB	Lower 95% eB	Upper 95% eB
Intercept	2.75	0.02	116.00	8.11	.00	15.61	14.88	16.38
Baseline GS Norms	0.03	0.00	41.33	2.89	.00	1.03	1.03	1.04
Gender	0.17	0.02	8.41	0.59	.00	1.18	1.14	1.23
GNS One Time	−0.09	0.03	−2.86	−0.20	.00	0.91	0.86	0.97
GNS Biannual	−0.08	0.03	−2.34	−0.16	.02	0.93	0.87	0.99
GS One Time	−0.07	0.03	−2.04	−0.14	.04	0.94	0.88	1.00
GS Biannual	−0.09	0.03	−2.72	−0.19	.01	0.92	0.86	0.98
Time	−0.01	0.00	−5.80	−0.41	.00	0.99	0.98	0.99
Time × GNS One Time	−0.01	0.00	−2.39	−0.17	.02	0.99	0.99	1.00
Time × GNS Biannual	−0.00	0.00	−1.44	−0.10	.15	1.00	0.99	1.00
Time × GS One Time	−0.00	0.00	−1.20	−0.08	.23	1.00	0.99	1.00
Time × GS Biannual	−0.01	0.00	−2.81	−0.20	.01	0.99	0.99	1.00

Note. HGLM = hierarchical generalized linear model; GNS = gender-nonspecific.

Figure 4.

Effect sizes (ds) and 95% confidence intervals for Treatment versus Control × Time interactions for gender-specific (GS) perceived norms and gender-nonspecific (GNS) perceived norms.

Gender-nonspecific perceived norms

Table 5 contains results of HGLM with Poisson errors. Results are similar to those found with gender-specific perceived norms. Moreover, immediate changes in gender-nonspecific norms relative to control at 6-month follow-up were evident for participants in the gender-specific one-time condition, the gender-nonspecific one-time condition, and the gender-nonspecific biannual condition. There was, however, no significant immediate reduction relative to control among participants in the gender-specific biannual condition, and there was not a main effect for gender. Evaluations of changes over time were identical to those for gender-specific perceived norms.

Table 5.

Changes in Gender-Nonspecific (GNS) Perceived Norms by Intervention Condition Using HGLM With Poisson Level 1 Errors

Variable	B	SE B	z	d	P	eB	Lower 95% eB	Upper 95% eB
Intercept	2.86	0.02	129.07	9.03	.00	17.53	16.78	18.32
Baseline GNS Norms	0.03	0.00	44.39	3.10	.00	1.03	1.03	1.03
Gender	0.01	0.02	0.50	0.03	.62	1.01	0.98	1.04
GNS One Time	−0.08	0.03	−2.73	−0.19	.01	0.92	0.87	0.98
GNS Biannual	−0.06	0.30	−1.92	−0.13	.06	0.94	0.89	1.00
GS One Time	−0.06	0.03	−2.03	−0.14	.04	0.94	0.89	1.00
GS Biannual	−0.05	0.03	−1.62	−0.11	.11	0.95	0.90	1.01
Time	−0.01	0.00	−5.85	−0.41	.00	0.99	0.98	0.99
Time × GNS One Time	−0.01	0.00	−2.79	−0.20	.01	0.99	0.99	1.00
Time × GNS Biannual	−0.00	0.00	−1.33	−0.09	.18	1.00	0.99	1.00
Time × GS One Time	−0.01	0.00	−1.55	−0.11	.12	1.00	0.99	1.00
Time × GS Biannual	−0.01	0.00	−2.54	−0.18	.01	0.99	0.99	1.00

Note. HGLM = hierarchical generalized linear model; GS = gender-specific.

Only the biannual gender-specific condition showed a significant effect on drinking over time. Thus, our mediation analyses focused on this pathway. Following Kenny et al. (2004), we included norms as a time-varying covariate. The cross-level interaction of biannual gender-specific treatment condition and time was still significant (B = −0.012, SE = 0.006, Z = −2.05, p = .04, d = −0.14) but was reduced by approximately 11%. When gender-nonspecific norms were included as a mediator, the interaction of biannual gender-specific treatment condition and time was marginally significant (B = −0.011, SE = 0.006, Z = −1.82, p = .07, d = −0.13), and the coefficient dropped 22%. Both of these results suggest that the effect of this treatment condition was working, in part, through its impact on norms. Similar analyses examining the significant treatment effects for the RAPI failed to reveal mediation, as coefficients and significance values were virtually identical across original models and models with mediators included.

Discussion

In this study, we evaluated the efficacy of web-based PNF in reducing college drinking over an extended period of time. This research represents among the largest and longest entirely web-based intervention trials targeting college student drinking to date of which we are aware. There were few significant overall differences between gender-specific and gender-nonspecific feedback or between single versus biannual administrations of feedback. Few effects were also evident for specific comparisons between PNF conditions and control. Biannual administration of gender-specific web-based PNF was associated with reduced drinks per week over time relative to an attention control group. There were no intervention effects on frequency of heavy episodic drinking, and there were virtually no significant effects on drinking at the 6-month follow-up. The strongest support in changing behavior was with biannual administration of gender-specific PNF with women. In comparison with gender-nonspecific PNF, gender-specific PNF was associated with reduced alcohol-related consequences over time for women. For women, but not for men, biannual administration of gender-specific web-based PNF was also associated with reduced alcohol-related problems over time. Overall, the results are modest, and the effect sizes are small.

Assessment reactivity may explain why participants in all intervention groups, including the control group, showed improvement in drinking outcomes. For example, prior research has found reactivity to alcohol assessment measures among college students (Walters, Vader, Harris, & Jouriles, 2009), such that students who complete multiple assessments were more likely to report lower peak blood alcohol content and Alcohol Use Disorders Identification Test (AUDIT) scores in comparison with students who completed minimal assessments. Results from Walters, Vader, Harris, and Jouriles's (2009) study did not show assessment reactivity for overall volume of drinking.

Intervention effects on changes in perceived norms were more evident. Both gender-specific and gender-nonspecific PNF were associated with reductions in perceived norms. Gender-specific norms were reduced at the first follow-up point in all four intervention conditions relative to control and over the entire follow-up period among those receiving biannual administration of gender-specific PNF as well as those who received a single exposure to gender-nonspecific PNF. Gender-nonspecific norms were reduced at the first follow-up point in three of the four intervention conditions relative to control and over the entire follow-up period among those receiving biannual administration of gender-specific PNF and among participants who received a single exposure to gender-nonspecific PNF. Moreover, results provide support for changes in norms as a mediator of the effect of biannual administration of gender-specific feedback on drinks per week over time. Mediation results do not clearly distinguish gender-specific versus gender-nonspecific norms as mediators of this effect. Rather, both types of norms were affected relatively similarly. Because the present findings provide support for perceived norms as a partial mediator of intervention efficacy, this suggests that there may be other active ingredients to PNF. PNF presents two discrepancies. The first discrepancy illustrates a difference between the participant's perceptions of student drinking and the actual drinking norm—that is, normative misperception. The second discrepancy highlights the difference between the participant's drinking and the actual drinking norm—that is, that the participant's drinking is deviant from the actual drinking norm. In the present study, we found that reducing overestimated normative perceptions (i.e., the first discrepancy) partially mediated the effect of biannual gender-specific PNF on weekly drinking. Thus, exposure to the second discrepancy may also have influenced this effect such that repeatedly informing participants that their drinking is heavier than the actual drinking norm of their same-sex peers may, in turn, influence drinking.

In the context of previous literature, this research provides several important contributions. The significant findings for biannual administration of web-based gender-specific PNF in reducing weekly drinking over time and, for women, alcohol-related consequences over time, is encouraging but also suggests mitigating enthusiasm for web-based versus in-person interventions. Previous research evaluating in-person computer delivered PNF (gender-nonspecific and gender-specific) using a similar format and similar populations has found efficacy in reducing drinking in multiple randomized trials with follow-up assessments ranging from 1 to 6 months (Lewis & Neighbors, 2007; Lewis et al., 2007; Neighbors et al., 2004; Neighbors, Lewis, et al., 2006). In the present research, there were virtually no intervention effects on drinking at 6-month follow-up, and only gender-specific feedback had a significant impact on weekly drinking, and then only with biannual administration. Normative feedback about heavy episodic drinking was not provided, and there were not any intervention effects on this outcome.

The primary difference between the present study and previous studies is that participants in the present study completed assessments and received interventions offsite, whereas participants in previous studies were required to come onsite to complete assessments and to receive interventions. This suggests that interventions that are completely web-based in this population may be less effective than those in which students are required to come onsite. Attentional and motivational factors may contribute to this variation in on- and offsite intervention efficacy (i.e., the elaboration likelihood model; Petty & Cacioppo, 1986). Additional research is needed to carefully evaluate the location of administration for computerized interventions to better understand the potential limitations of offsite Internet interventions. Specific limitations to consider might include distraction. Outside of a laboratory setting, participants may be completing procedures and reviewing feedback while simultaneously attending to many possible distracters (e.g., television, phone calls, chat programs, simultaneous e-mail checking). Future studies should evaluate whether increasing participant interest in the initial intervention content, degree of tailoring of feedback, length of the intervention, loading time of the web page, professional appearance of the website, ease of navigational structure, and conciseness of the text have an affect on participant attention (Brouwer et al., 2009).

Similarly, there may be limitations related to motivation. Students may be more motivated to process the feedback, print it out, and think about it in a controlled laboratory setting where their participation is being monitored. Outside of the laboratory, they may take the procedure less seriously and feel less accountable for the quality of their participation. Selection bias related to motivation to come into the laboratory may also contribute to differences between web-based and in-person computer delivered interventions. Participants who come into the laboratory may be more motivated to attend to feedback and/or to consider changing their drinking relative to students who are willing to participate from home but unwilling to attend a laboratory session.

Although we tentatively conclude that web-based interventions may be less effective than the same interventions delivered in laboratory settings, the present findings provide some support for web-based interventions. Previous research has provided some support for gender specificity in the provision of norms feedback, relative to gender-nonspecific feedback, at least for women who are stronger in gender identity (Lewis & Neighbors, 2007). These findings were echoed here with biannual gender-specific feedback being associated with greater reductions in alcohol-related problems relative to control for women but not men. Greater specificity of feedback may translate to greater relevance, and this may be particularly important when interventions are delivered in the context of multiple potential distractions that cannot be easily controlled outside of a laboratory session. Biannual administration may similarly increase the likelihood that students will process the information. The need for additional research to evaluate these speculations is likely to increase as the dissemination of web-based interventions continues to outpace careful evaluation of their effectiveness. In the present research, neither of these features alone was sufficient to produce significant changes in drinking. Rather, greater specificity in combination with repeated administration (i.e., biannual gender-specific PNF) resulted in modest reductions in drinks per week and, for women, alcohol-related consequences.

The present findings add to the literature regarding the importance of specification of the reference group with respect to social-norms-based interventions. It is important to note that there was considerable overlap in the impact of PNF interventions on perceived norms. Gender-specific PNF impacted gender-specific perceived norms, but it also impacted gender-nonspecific norms. Similarly, gender-nonspecific feedback impacted gender-nonspecific norms, but it also impacted gender-specific norms. This may relate to how students conceptualize the reference groups. For example, when asked to estimate the drinking of “the typical student on campus,” participants may automatically apply gender and/or other characteristics to the prototypical student (Lewis & Neighbors, 2006b). Nevertheless, significant changes in weekly drinking were only evident with biannual administration of gender-specific PNF.

Clinical Significance

Although this research provides modest support for the efficacy of gender-specific PNF in reducing weekly drinking and alcohol-related problems among college women over time, the effects are relatively small and inconsistent with respect to whether delivered once versus every 6 months. The effect sizes in this study are smaller than many interventions for this population, most of which have been administered in person (Carey et al., 2010). There were no effects on heavy episodic drinking. Moreover, these data combined with other evaluations of web-based interventions, few though they may currently be, provide only modest support for the Internet as an intervention medium for college student drinking. The probable offset in efficacy between onsite and offsite computerized interventions should be considered from a public health perspective in which impact is a function of the reach and efficacy per unit cost (Abrams & Clayton, 2001). There is little question that offsite Internet interventions have the potential for reaching large numbers of students at relatively low cost in comparison with interventions that require students to come into a laboratory setting, classroom, counseling center, or clinic. Future research summarizing and comparing effect size differences between in-person and offsite computerized interventions would allow for estimating differences in cost effectiveness as campuses prioritize the allocation of limited prevention and treatment resources.

Limitations/Future Directions

The strengths of the present study should be considered in the context of several limitations. The sample was from a single university, and questions remain regarding how well the present results might generalize to other universities with different characteristics. For example, norms information presented on smaller and/or more cohesive campuses might have more influence on students in comparison with more diverse and/or commuter campuses where students may not identify as strongly with their campus. The norms presented in the interventions remained constant and were based on freshmen drinking. In hindsight, it may have been preferable to present norms that were updated and class specific (i.e., freshman or sophomore) at each administration in the biannual PNF conditions. It is also important to consider the limited generalizability associated with compensating assessments $25 each, which would not be possible in most university settings. Future research is needed to evaluate the efficacy of PNF with booster sessions without these incentives. All assessments were based on self-report. Although self-report has been shown to be valid in assessing drinking behavior among college students relative to collateral reports (e.g., Laforge, Borsari, & Baer, 2005), it is not clear whether or how self-reported behavior might be influenced in the context of receiving information regarding peer drinking. Moreover, although intervention studies aimed at correcting normative misperceptions have moved forward, fundamental questions remain with respect to the relationships between perceived norms and behavior and the underlying causes of normative misperceptions (Prentice, 2008). Although interventions rely on the causal influence of perceived norms on subsequent behavior, some evidence suggests that the relationship between these two variables is bidirectional (Neighbors, Lewis, et al., 2006). Thus, reporting one's own drinking behavior may influence estimates of peer drinking and vice versa.

This research was limited to focus exclusively on descriptive norms. Compelling arguments have been made with respect to the potential for integrating injunctive norms in social-norms-based interventions (Blanton & Burkley, 2008; Prentice, 2008), but relatively little work has empirically tested injunctive norms feed back in the context of college student drinking (e.g., Schroeder & Prentice, 1998). This is an important direction for future research, and some evidence suggests that specificity of the reference group is more important for injunctive drinking norms than it is for descriptive drinking norms (Chawla, Neighbors, Lewis, Lee, & Larimer, 2007; Neighbors et al., 2008).

Conclusions

In summary, despite limitations, this research provides a substantial contribution to the existing literature. To date, it is among the largest and longest evaluations of a randomized trial of a web-based intervention for college student drinking. Results provide modest support for the use of web-based PNF, provided it is gender-specific and biannually administered. Moreover, web-based interventions are likely to be less effective than in-person interventions, but these relatively smaller effects may be offset by the potential to reach large numbers of individuals at relatively low cost.