In Pursuit of a Self-Sustaining College Alcohol Intervention: Deploying Gamified PNF in the Real World

Abstract

Our recent work (Boyle, Earle, LaBrie, & Smith, 2017) showed that the efficacy of personalized normative feedback-based (PNF) college alcohol interventions can be improved through the addition of gamified elements including points, chance, competition, and personal avatars. However, participants in that study were compensated with subject pool credit. In the current study, we piloted an upgraded, smartphone-based version of the game, which was designed to be truly self-sustaining (i.e., engaging enough that students play voluntarily without the presence of external motivators). First-year students were invited to play the game weekly for six rounds, with participants submitting and voting on their own questions each week and receiving a novel type of feedback in addition to standard descriptive PNF: opposite peers’ judgments of participants’ self-reported drinking behavior, or reflective norms. With no play-based incentives, 222 first-year college students voluntarily played the game, CampusGANDR. ANCOVA models revealed that, relative to participants randomized to receive feedback on control topics during the three intervention rounds, those who received both descriptive and reflective feedback on peer alcohol use had significantly reduced normative perceptions and reduced alcohol use two months post intervention. This was especially true among heavy drinkers. The results suggest that our gamified “GANDR” approach shows promise as a self-sustaining intervention approach and, further, that high-risk drinkers may benefit disproportionately from this methodology. Thus, self-sustaining interventions represent an encouraging avenue for future research and development and may hold the potential to impact risky college drinking on a large scale.

1. Introduction

College students’ perceptions of how much and how often their peers drink, known as peer drinking norms, are among the strongest predictors of students’ own future alcohol consumption (Neighbors, Lee, Lewis, Fossos, & Larimer, 2007; Perkins, 2003). Problematically, students consistently get these norms wrong, significantly overestimating the drinking behaviors of their peers (Borsari & Carey, 2003; Larimer, Turner, Mallett, & Geisner, 2004; Lewis & Neighbors, 2004). Because of this, many successful university-based alcohol interventions have featured a component designed to correct misperceived peer drinking norms (Cronce & Larimer, 2011; Larimer & Cronce, 2002). This component increasingly takes the form of web-based personalized normative feedback (PNF), a brief intervention modality in which students first answer survey questions about their perceptions of peer drinking and about their own alcohol use and then receive an individualized report that employs a combination of charts and text to highlight misperceptions (Berkowitz, 2005; Lewis & Neighbors, 2007; Miller & Prentice, 2016). The popularity of PNF has grown rapidly because studies have consistently found this approach to be effective at reducing alcohol use among college students (Berkowitz, 2005; Lewis & Neighbors, 2007; Miller & Prentice, 2016). However, the size of these effects has generally been small/modest—especially among heavy drinkers (Dotson, Dunn, & Bowers, 2015; Walters & Neighbors, 2005). Researchers have speculated these effects might be improved if PNF feedback could be made more believable (Hummer & Davison, 2016; LaBrie, Hummer, Neighbors, & Larimer, 2010), students could be enticed to pay more attention to the feedback (Lewis & Neighbors, 2015), and if the intervention could be formatted to produce less defensive reactions in students (Leffingwell, Neumann, Leedy, & Babitzke, 2007; Steers et al., 2016).

1.1 Gamification in the PNF context

Researchers outside of the college alcohol sphere have recently begun experimenting with a technique known as gamification to improve attention, motivation, adherence, and learning in web-based interventions (Breuer & Bente, 2010; Brown et al., 2016; Comello et al., 2016; Girard, Ecalle, & Magnan, 2013; Wouters, Van Nimwegen, Van Oostendorp, & Van Der Spek, 2013). Gamification is the use of game design elements (e.g., points, levels, leader boards, personal avatars) in non-game contexts (Deterding, Dixon, Khaled, & Nacke, 2011). Similarly, the term “serious games” refers to the practice of creating games that serve a purpose other than pure entertainment—generally learning and training (Cugelman, 2013; Gabarron, Schopf, Serrano, Fernández Luque, & Dorronzoro Zubiete, 2013; Landers, Bauer, & Callan, 2015; Lewis, Swartz, & Lyons, 2016; Mekler, Brühlmann, Opwis, & Tuch, 2013). Studies involving serious games have consistently found that gamified elements, when added to typical intervention protocols, lead to larger changes in behavior, whether the behavior in question is physical activity among arthritis patients (Allam et al., 2015), self-management of chronic illness (AlMarshedi, Wills, & Ranchhod, 2016), or learning among college students (Buckley & Doyle, 2016).

Inspired by the gamification and serious games literature, we developed a novel PNF format called GANDR (Gamified Alcohol Norm Discovery & Readjustment), which included gamified elements and integrated participants’ Facebook photos. GANDR was specifically designed to address the potential limitations to web-based PNF suggested in previous work. The pilot version of CampusGANDR (v1) introduced by Boyle, Earle, LaBrie, & Smith (2017) included 3 gamified elements supported by the broader gamification literature. First, consistent with studies suggesting that point-based reward systems increase motivation and strengthen the learning of facts, procedures, and strategies in educational games (Kap, 2012; Johnson, Deterding, Kuhn, Staneva, Stoyanov, et al., 2016), CampusGANDR featured a system of points. Participants won and lost based points each round based on the accuracy of their normative perceptions. Second, the placement of chance elements before intervention content has been found to give the content a positive, serendipitous feel (Rao, 2016), decreasing psychological reactance among heavy drinkers (Boyle, Earle, McCabe, & LaBrie, 2017). Thus, the questions and feedback topics in CampusGANDR appeared to be selected at random by a slot machine-style spinner. Finally, in typical PNF interventions heavy drinkers may question the validity of the normative statistics presented (Campo & Cameron, 2006; Granfield, 2005; Polonec, Major, & Atwood, 2006). Identity Process Theory suggests that resistance to change is elevated when the prospect of such change constitutes a threat to an individual’s self-identity (Murtagh, Gatersleben, & Uzzell, 2012). This has proven true in alcohol feedback studies, with research suggesting that heavy drinkers tend to react more defensively (Leffingwell, Neumann, Leedy, & Babitzke, 2007) and negatively (Butler, Silvestri, & Correia, 2014; Miller, & Leffingwell, 2013) to feedback on their alcohol-related perceptions and behaviors. Previous work by our lab suggests this issue can be remedied when the statistics are derived from a visible group of students who complete the intervention together (LaBrie et al., 2008; 2009). Thus, to induce a feeling of co-presence, or being together online (Campos-Castillo, 2012; Lee, 2004), participants in CampusGANDR were led to believe they were playing live against 132 other students across the country and Facebook profile pictures were used to represent these 132 other students.

In that pilot study, undergraduate students were randomized to either participate in a standard survey-based PNF intervention or to play CampusGANDR. Notably, each condition required students to answer the same questions and culminated in the presentation of identical feedback screens. Students assigned to the CampusGANDR condition merely answered these questions and received this feedback in the context of a social game about college life while control participants did so in the context of a typical web-based college alcohol intervention. Two weeks post intervention, participants in the CampusGANDR condition reported significantly reduced perceptions of peer drinking norms and significantly reduced alcohol consumption, relative to those in the standard survey condition. These findings support the use of gamification in the PNF context. However, this pilot study had some real limitations. First, it employed a short follow-up period. Second, it utilized deception to ensure that the statistics presented in both conditions were truly identical; CampusGANDR participants were led to believe they were connected live with other students across the country but this was untrue. Finally, students were recruited through the psychology department subject pool and received course credit for their participation. Thus, despite the promising results of the pilot study, it is still unclear whether a gamified alcohol intervention could be successfully delivered in a non-laboratory setting. Is it possible to make a gamified intervention that is appealing enough that students will play voluntarily? The current research seeks to answer this question while testing a second iteration of CampusGANDR (v2).

1.2 CampusGANDR v2: A Real-World Test of the GANDR Approach

Building on our previous work, CampusGANDR v2 took the form of a more elaborate smartphone-based game testing students’ perceptions of various college life topics. In addition to the features supported in CampusGANDR v1 (points, chance, and visual representation of other players), v2 also included three new features informed by longstanding social and cognitive psychological theories. These features were included in an attempt to make CampusGANDR sufficiently appealing and impactful that students would play voluntarily and that effects could be observed in the absence of participant stipends, course credit, or mandated participation.

Self-determination theory (Deci & Ryan, 2000, 2010) divides motivation into two separate components—intrinsic and extrinsic. Intrinsic motivation involves participating in an activity because the activity itself is pleasurable, while extrinsic motivation involves the promise of rewards and/or punishments. Nearly every alcohol intervention study to date has relied exclusively on extrinsic motivators to encourage student participation and retention. Specifically, participants have either been awarded course credit, paid directly in cash or gift cards, entered into a drawing for prizes or privileges, or required to participate either as part of a mandatory first-year alcohol education program or to fulfill a judicial sanction (e.g., Barnett & Read, 2005; Geisner, Larimer, & Neighbors, 2004; Larimer et al., 2001; Murphy et al., 2001; Neighbors, Lewis, Bergstrom, & Larimer, 2006; Wood, Capone, Laforge, Erickson, & Brand, 2007). However, if students could be persuaded to take part in an intervention based on intrinsic, rather than extrinsic, factors, the intervention may be more effective for two main reasons. First, a growing body of literature on self-determination theory suggests that intrinsically-motivating activities are more impactful than those which rely solely on extrinsic motivation (Deterding, 2011; Kuvaas, Buch, Weibel, Dysvik, & Nerstad, 2017; Lemos & Veríssimo, 2014; Ross, Perkins, & Bodey, 2016). Intrinsically-motivated students may pay closer attention to their feedback and be more accepting of the information relative to those who are compensated for their participation or forced to participate. Second, and importantly, paying participants to take part in interventions is simply not a generalizable large-scale dissemination strategy from a public health standpoint. If an intervention is to have a significant impact at a national level it is imperative that participation be self-sustaining. That is, the content or framing of the program must be appealing enough that students will take part without being paid. Thus, in contrast to previous intervention studies, including CampusGANDR V1, no incentives were offered for individual participation in V2. Instead, this version of CampusGANDR minimized external rewards, inviting hundreds of students to compete for just three grand prizes. Thus, CampusGANDR V2 incentivized performance, not merely participation.

The following sub-sections provide theoretical background for the three new features added to increase intrinsic motivation in CampusGANDR V2. We also provide more depth on the implementation of these features. First, rather than delivering feedback on three different alcohol use items at one time, as v1 did, v2 was designed to present the feedback in three separate modules released gradually during a 6-week period. Second, instead of presenting participants with questions that were pre-selected by the research team, CampusGANDR v2 invited participants to play a role in determining the content by submitting their own questions and voting on their favorites. Finally, to further improve results among heavy-drinking students, a novel type of feedback was added in addition to the descriptive PNF.

1.2.1 Gradual PNF Delivery

According to the distributed practice or spacing effect, a longstanding phenomenon dating back to the early days of cognitive psychology (Ebbinghaus, 1885), learning is most efficient when practice is spaced out in time rather than occurring all at once (Dempster & Farris, 1990; Shaughnessy, 1976). Thus, the same volume of feedback delivered gradually over time with space in between should result in significantly better learning and retention. Similarly, cognitive research suggests there may be retention benefits associated with delayed feedback presentation (Pashler, Rohrer, Cepeda, & Carpenter, 2007) and giving feedback gradually over the course of multiple shorter messages rather than one single message (Beck, Lakkaraju, & Rai, 2017). However, all PNF studies we are aware of to date have focused on delivering feedback for multiple questions all in a single sitting and have tended to present feedback immediately after participants report their perceptions and behavior. This feedback schedule may be ineficient from a cognitive standpoint.

Based on the research outlined above, CampusGANDR v2 was designed to present feedback to participants in six separate modules delivered once per week over a 6-week period, with a 4-day gap between students’ answering of questions and delivery of feedback. That is, in CampusGANDR v2 new questions were posed at the beginning of each week (Monday), and players were given four days to answer these questions (Monday-Thursday). On Friday, PNF was delivered. During three of the six weeks the feedback centered around and alcohol-related topic with filler topics used during the other 3 weeks.

1.2.2 User-Generated Content

An offshoot of cognitive dissonance theory called effort justification theory (Rosenfeld, Giacalone, & Tedeschi, 1984), and a newer iteration dubbed the “Ikea effect” (Mochon, Norton, & Ariely, 2012; Norton, Mochon, & Ariely, 2011), posit that people tend to value things significantly more when we play a role in their creation. For example, in one study, participants who used computer software to precisely replicate a simple design on a t-shirt valued the shirt much more than participants who received a pre-printed shirt featuring the same design (Franke, Schreier, & Kaiser, 2010). The Ikea effect suggests participants in an alcohol education intervention will be more attentive and interested in the content if they are allowed to take part in its creation. Consistent with this theoretical paradigm and literature participants in CampusGANDR v2 were invited to submit their own questions on student attitudes/behaviors each week and to vote on the questions submitted by their peers. The user-submitted questions that received the most votes were then featured in the next round of play.

1.2.3 A Novel Injunctive Feedback Component

Web-based PNF interventions for college students have long focused on correcting descriptive norms, or perceptions about the quantity and frequency of peer alcohol use (Dotson et al., 2015). However, injunctive norms, or perceptions about how approving peers are of drinking, have also been found to be independently predictive of alcohol consumption (Lee, Geisner, Lewis, Neighbors, & Larimer, 2007; Pedersen et al., 2017). Thus, interest has grown regarding how injunctive feedback might also be incorporated in PNF interventions to increase effect sizes (Neighbors et al., 2008; Padon, Rimal, Jernigan, Siegel, & Dejong, 2016; Steers et al., 2016). The current study introduces a new form of injunctive feedback that may be particularly interesting and impactful for college students while feeling organic within the playful, gamified intervention format: opposite-sex students’ reflective judgments of participants’ reported behaviors.

Reflective norms are perceptions about the extent to which members of the opposite sex find a given behavior desirable (LaBrie, Cail, Hummer, Lac, & Neighbors, 2009). Importantly, as with descriptive norms, heterosexual college students overestimate the degree to which students of the opposite sex value/desire heavy drinking and these perceptions uniquely influence alcohol consumption (Hummer, LaBrie, Lac, & Louie, 2013; Hummer, LaBrie, Lac, Sessoms, & Cail, 2012). Thus, CampusGANDR v2 was designed to employ a third condition in which standard descriptive norm-based PNF was supplemented with feedback on a novel type of reflective norms. This reflective feedback consisted of a graphical display through which participants learned how players of the opposite sex evaluated the behaviors they reported (e.g. a male student who reported consuming 7 drinks over the weekend would learn how female students evaluated males who consumed 7 drinks over the weekend). To our knowledge, this is the first intervention seeking to utilize this unique source of normative influence. The feedback designed for CampusGANDR v2 was presented in a graphically appealing way by positioning Facebook’s popular “thumbs up” and “thumbs down” icons on continuums of perceived “attractiveness” and “togetherness” (See Figure 1). Students who were rated favorable received a “thumbs up” while those who were rated unfavorable received a “thumbs down”.

Figure 1.

Example reflective judgments from opposite-sex peers, which supplemented PNF focused on descriptive same-sex drinking norms in one CampusGandr intervention condition.

1.4 The Current Study

The current study sought to expand on our previous work (Boyle et al., 2017) by assessing the feasibility of enrolling students in a gamified intervention outside of the controlled laboratory setting and by examining the degree to which the two types of drinking feedback delivered within the game were effective in reducing normative beliefs and alcohol use. Specifically, this study had three main Aims:

• Aim 1a) Investigate the degree to which un-incentivized first-year students will self-select into the 6-week disguised intervention after minimal campus advertising. We anticipated enough students would participate voluntarily that an intervention effect on alcohol consumption would be detected at the .05 significance level. Thus, AIM 1a was closely related to AIM 2, which examined main effects of the intervention.

• Aim 1b) Determine whether the proportion of alcohol and other drug related questions submitted by students during the game is high enough for this intervention approach to be feasible in the real world. Based on focus groups conducted with students about the types of questions they would be interested in receiving feedback on during CampusGANDR, we predicted that substance use-related questions would be among the most popular, being submitted in a high proportion and receiving a high average number of votes, relative to other question types.

• Aim 2a) Measure the impact of PNF on descriptive same-sex drinking norms in this gamified format on students’ perceived drinking norms and alcohol use 2 months later, relative to PNF on control topics only. We anticipated a small to moderate effect on norms and alcohol use, as this is consistent with previous remotely-delivered PNF interventions and because our previous work using the GANDR approach found that effects for gamified PNF were larger than effects for standard PNF.

• Aim 2b) Examine whether supplementing this descriptive PNF with reflective opposite-sex feedback on participants’ own behavior will promote greater reductions in alcohol use 2 months later. Based on our previous work suggesting the influential role that reflective norms play in shaping students’ behavior, we predicted this additional feedback module would significantly improve the effects of the intervention, leading to greater reductions on alcohol consumption than standard PNF alone.

• Aim 3) Test students’ baseline drinking behavior as a moderator of the conditional effects of the intervention. We hypothesized that the playful context in which both descriptive PNF and reflective feedback were delivered in CampusGANDR would make the intervention conditions particularly impactful among students who were heavier drinkers.

2. Method

2.1 Participants

Participants were 276 students who were still playing the game at the end of week 2 of play (from 356 who signed up for the app; 77.5%), and were randomized to one of three conditions: Control, PNF, and PNF + Reflective feedback. Play continued for a total of six weeks, with students playing as many or as few of these six rounds as desired. Then, 2 months after the final round of play, participants were texted a link to view the final results of the game. Before they were allowed to view the results, participants were asked to answer a final set of questions about their perceptions of drinking norms and alcohol consumption (2-month follow-up assessment), ostensibly as a “sudden death” round because there were multiple ties in point totals within the game. A flow chart detailing recruitment and condition assignment is provided in Figure 2. The final sample consisted of the 222 students (80.4%) who were randomized to a condition, played at least one round that included alcohol feedback, and completed the follow-up. The majority of participants were female (55%) and Caucasian (47%). Twenty percent of participants described themselves as Hispanic/Latino, 14% were Asian, 12% were African American, and 7% were Multiracial or Other.

Figure 2.

Participant Flow Diagram.

2.2 Design and Procedure

Recruitment occurred during September 2016, at a private, mid-sized university on the west coast of the United States. All procedures and measures were approved by the host university’s IRB. First-year students received an email during the first month of school about a new web-based app that was being beta-tested/studied on campus. The app was described as a social game that would allow students to find out information about their new classmates’ attitudes and behaviors and to win prizes for making accurate guesses about other students (including a top dorm pick and two $250 cash prizes). Recruitment remained open for one week, during which 356 students visited the app’s website, consented to participate in the study, created a profile, and played the first round. Upon opening the app, students were presented with CampusGANDR informed consent information (“Terms and Conditions”). After providing consent and indicating that they were 18 years of age or older, participants were invited to create a profile by providing their student email address, first name, and uploading a photo. After their email address was verified, participants immediately played the first round.

Each round of play consisted of four phases. First, each week from Monday through Thursday, participants were able to answer the two weekly “GANDR Questions”. Each question contained two parts. First, participants estimated how the average same-sex student in their class would answer. Next, students reported their own answer. For instance, during Round 1 female students were asked “How many hookup partners does the typical freshman female plan to have this semester” and “How many hookup partners do you plan to have this semester?” After responding to the questions, participants were given the option to take part in the second phase of gameplay by voting on the question for the following week. Participants earned two points, or “Gandollars”, each time they voted on a question by giving it either a “love it”, “hate it”, or “meh”. After voting on their favorite questions, participants were given the option of “judging” other students. During this phase of gameplay, participants were first asked to report the maximum response they would find acceptable by a member of the opposite sex. For instance, during Round 1, female students were asked, “What is the maximum number of hookup partners you think a male student should have in a semester?” Next, participants were shown a few random values for each question and were asked to rate an opposite sex student on two sliding scales anchored from “attractive” to “unattractive” and from “together” to “hot mess.” Three Gandollars were awarded each time a participant made one of these “judgments.” Finally, participants were given the option to submit a question of their own to be voted on by other students during the following week. Students whose questions were selected in subsequent weeks were awarded 50 Gandollars.

At the end of each weekly question round, on Fridays, participants received text messages informing them that results were ready for viewing and a new round was beginning. The message included a unique link and clicking on the link brought participants to a “loading” page. This page informed students of the number of their peers who answered the current question (eg., “Loading responses from 341 freshmen who played this week”), and displayed the profile photos of all students who answered in an animated grid. Next, participants viewed their feedback in two separate modules. During the first feedback module, representative of typical PNF, participants’ misperceptions were corrected and they were shown how their behavior compared to that of their peers. First, they viewed a bar chart comparing their estimate of typical student behavior to the actual average reported by other same-sex first-year students. This screen also informed students how many points they had won during the round; 50 Gandollars for a perfect guess, 25 if they were one off, 13 if they were two off, and so on. The following screen showed students a second bar chart comparing their own self-reported behavior to the actual norm of the group. The first bar had the participant’s profile photo next to it and was labelled “You” while the second bar had the university logo next to it and was labelled “Guys at Your School” or “Girls at Your School”. At the bottom of this screen were two statistics labelled “Largest Group” and “Consensus”, meant to trigger conformity. These statistics revealed the most frequently endorsed answer or range of answers as well as the range in which the vast majority of students (~90%) fell. After viewing these “How You Compare” results, participants then progressed to the second feedback module, during which they received reflective feedback showing how they were judged by other students. First, participants saw how opposite-sex students responded on the “attractive” and “hot mess” scales. This screen read “Based on your answer of [participant’s response from the previous week], male students judged you as…” and contained two sliding scales that mirrored the ones participants had used to make their own judgments during the previous week. A small graphic was positioned on the slider at the point that represented the average rating made by opposite-sex students of a given participant’s reported behavior. Finally, participants viewed one final screen of feedback. This revealed the answer most frequently endorsed by opposite-sex students as the maximum acceptable response. For instance, it might read, “73% of LMU guys thought 6 drinks was the maximum a female student should consume on a weekend.” Examples of these feedback screens are provided in Figure 2. After viewing both feedback modules, participants began playing the next round including answering the current questions, voting on questions for the following week, judging other students, and submitting a question of their own. The entire process of gameplay for a single round lasted approximately six minutes.

2.3 Condition Assignment and Manipulation

During the first round, all participants answered and received feedback on a question about hookup partners. No alcohol-related content was presented during the first round. The question was “open” for a week, during which time participants were able to respond. Then, at the end of the week, the true norms for male and female students were calculated and participants were all texted a link to view graphical feedback based on these actual group norms. After viewing the feedback for Round 1, participants began Round 2 and were informed that there were two questions in this round (all subsequent rounds contained two questions). Participants answered both questions and, thus, had the ability to win twice as many Gandollars as they had during Round 1. Importantly, during Rounds 2, 3, and 4, one question was alcohol-related and the other focused on a non-alcohol-related aspect of college life. Randomization occurred as participants logged in to view their Round 2 feedback and play Round 3. An automated randomization algorithm assigned participants to either the control, PNF, or PNF + Reflective condition.

All elements of gameplay were completely identical across conditions, as were the questions answered by students each week. The only aspect of CampusGANDR that was manipulated was which question participants received feedback on during each of the two feedback modules. Thus, intervention dosage was constant across all conditions; participants in the control and in each of the active conditions saw the same exact number of feedback screens. In order to organically manipulate the proportion of these screens that showed alcohol- (vs. non-alcohol) related feedback, CampusGANDR included a “chance spinner” prior to each of the two feedback modules. Participants pressed a button to spin a “random” spinner with two arrows that selected one of the two weekly questions on which they would receive feedback and on which of the two they would see reflective evaluations. In the control condition, the spinners always pointed to the number that corresponded with the non-alcohol-related question. In the PNF condition, the first spinner pointed to the number representing the alcohol-related question while the second spinner pointed to the number signifying the non-alcohol-related question. And in the PNF + Reflective condition, both spinners pointed toward the number corresponding with the alcohol-related question. Thus, control participants saw no alcohol-related feedback at all, PNF participants saw alcohol-related feedback during the first module (telling them how accurate their guess was and how they compared to other students) but not during the second module (telling them how other students judged them), and PNF + Reflective participants saw alcohol-related feedback during both modules. Like the descriptive PNF feedback, the reflective feedback was also calculated from actual data collected from participants’ classmates during the “judge” module, described above.

2.4 Follow-Up Survey

Eight weeks after the end of the final round of gameplay, students received a text message informing them that the final results were now available for viewing. Clicking on a link in the message brought students to a landing page explaining that there were multiple ties in point totals within the game and asking them to participate in a brief “sudden death round” before viewing the results and seeing how their final score ranked in comparison to the other student-players. The sudden death round consisted of answering the same three alcohol-related questions that students had received feedback on during gameplay. For each question, students gave their perception of typical student behavior in addition to reporting their own behavior, just as before. Following the sudden death round students were shown their final score in Gandollars, their final rank within the app, and a “leader board” of the top 10 highest scorers, which actually displayed the profile photo, first name, and point totals of the 10 students who scored the highest during the 6 weeks of gameplay.

2.5 Measures

2.5.1 Perceived Descriptive Drinking Norms and Drinking Behavior

Three questions assessed perceptions of classmates’ alcohol consumption and partying and three parallel questions assessed participants’ own drinking and partying. These questions asked participants to report the maximum drinks consumed on a single night so far during the semester (Round 2), number of times participants had partied during the past week (Round 3), and number of drinks consumed during the previous weekend (Round 4). Importantly, these questions were submitted by students playing the game rather than being selected by the research team. Thus, the wording is not based on previously published scales. Prior to launching each of the three intervention rounds, the researchers looked over the questions submitted by students during the week and selected one alcohol-related and one non-alcohol related question with the highest number of votes. At the 2-month follow-up, participants re-answered these same questions. As answers to respective sets of norm and behavior questions at each assessment were highly correlated (r > .52), responses were first standardized then averaged to create composite drinking norm and behavior variables at initial assessment and follow-up.

2.5.2 Feedback Weeks Viewed

Gameplay occurred once a week for six weeks but not all participants played all six rounds. This means that some participants assigned to the intervention conditions viewed alcohol feedback on more occasions than others. To control for these differences, a variable was computed to indicate the number of feedback modules each participant viewed during the 3 consecutive intervention weeks. This resulted in a number between 1 and 3.

2.6 Data Analytic Plan

Our first Aim (1a) was to investigate whether we could attract sufficient student interest and participation in CampusGANDR with only minimal advertising and without the standard incentives associated with web-based alcohol intervention studies. To evaluate this, descriptive statistics examined the number of students who visited the CampusGANDR website, created a profile, played the first round, played at least 1 of the intervention rounds, and completed the follow-up to reveal their score and rank. As the success of CampusGANDR as a risk-reduction intervention relied on students freely submitting their own questions about alcohol and other drug use in the game (AIM 1b), analyses examined the proportion of all questions submitted by students that pertained to drinking, drug use, and partying. AIM 2a and AIM 2b pertained to whether condition assignment was differentially associated with perceived drinking norms or alcohol consumption at the 2-month follow-up. As composite norm and alcohol consumption variables were normally distributed, these questions were evaluated by one-way ANCOVAs. Respective models for norms and consumption outcomes featured the intervention condition (Control, PNF only, PNF + Reflective) as the between subjects’ factor with participants’ sex, and the baseline measure of the outcome variable included as covariates. In the presence of significant omnibus F tests, pairwise comparisons between the 3 conditions were evaluated. Finally, as we were particularly interested in the effectiveness of the two intervention conditions among heavy-drinking students (AIM 3), multiple regression analyses also examined alcohol use at the initial assessment as a moderator of conditional effects. This analysis was conducted using the Process Macro (Hayes, 2012). As there were 3 study conditions, the independent variable was defined as multi-categorical and was represented by contrasts that compared each intervention condition (coded 1) to control (coded 0).

3. Results

3.1 AIM 1a: Descriptive Statistics and Missing Data

Figure 3 details student participation in CampusGANDR over the course of the study. Emails announcing CampusGANDR were sent to the university email accounts of approximately 1,300 first-year students. Of these 1,300 students, only those who were 18 years old at the time of the study (approximately 60%, or 780 students) were eligible to participate. While we have no way to estimate the number of these students who opened the email to learn about the game, 356 (45.6%) students visited the website, created a profile and played the first round. Of these students, 276 played through week 3 where they were randomized to a study condition and received the first piece of control or alcohol feedback during the 3 intervention weeks. Eight weeks later, 222 (80%) of these intervention students answered the follow-up questions. Beyond the students lost to attrition there was no additional missing data.

Figure 3.

Drinking behavior at follow-up as a function of study condition and drinking behavior at initial assessment.

3.2 AIM 1b: Alcohol, Drug and Party Related Questions Submitted by Students

A total of 418 questions were submitted by students playing CampusGANDR. In each of the 5 weeks that questions could be submitted, roughly 22% of the 83 questions submitted on average pertained to drinking, drug use, or partying. Notably, questions focused on these topics received the largest numbers of votes by classmates. The two other most frequent and popular question topics pertained to romantic relationships or sexual behavior (20% of the questions submitted per week) and academic behavior/activities (19% of the questions submitted per week).

3.3 AIM 2: Conditional Effects on Norms and Behavior

Table 2 presents descriptive statistics by study condition for individual items and composite measures for both perceived descriptive norms and behaviors. Randomization resulted in no conditional differences in perceptions of norms or reported behaviors at the time they were initially assessed. Further, the average number of intervention round PNFs viewed (which could range from 1 to 3) was 2.66 (SD=.59), with no conditional differences in the number of feedbacks received, F (2,219) = 1.08, p = .34.

Table 2.

ANCOVA Results testing the effect of condition assignment on perceived norms and alcohol use behavior at follow-up (N=222).

	Omnibus Test for Condition				Conditional Pairwise Comparisons M(SE)
Dependent Variable	df	F	p	Partial n2	Control (N=71)	Alcohol PNF (N=79)	Alcohol PNF + Judgment (N=72)
Drinking Norms (z)	2, 217	13.79***	< .001		.11.32 (.08)*** A,B	−.04 (.08) *** A; * C	−.27 (.08) *** B; * C
Alcohol Use (z)	2, 217	3.82*	.02	.03	.14 (.07)** B	.00 (.07)	−.14 (.07) ** B

Note. ANCOVA models for Drinking Norms and Alcohol Use controlled for the baseline measure of the outcome variable in addition to participants’ sex, and alcohol weeks played. For pair-wise comparisons,

^Aindicates a significant difference between the Control and Alcohol PNF only condition;

^Bindicates a significant difference between the Control and Alcohol PNF +Judgment condition;

^Cindicates a significant difference between the Alcohol PNF and Alcohol PNF + Judgment condition.

^*p<.05

^**p<.01;

^***p<.001.

Results for respective ANCOVA models are presented in Table 3. Omnibus F tests revealed significant conditional effects on both drinking norms and behavior at follow-up. For norms, pairwise comparisons revealed that both the Alcohol PNF only and the Alcohol PNF + Reflective conditions reduced perceptions of descriptive norms significantly better than control. Also, unexpectedly, the Alcohol PNF + Reflective condition reduced perceptions of descriptive norms better than did the Alcohol PNF alone. For drinking behavior at follow-up, relative to control, pairwise comparisons revealed that this behavior was only reduced significantly in the Alcohol PNF + Reflective condition. Concerned about observed statistical power, which for the behavioral model was only 68%, Cohen’s d effect size statistics (Rosenthal, Cooper, & Hedges, 1994) were calculated for the difference between each intervention condition and control. For the Alcohol PNF only condition d=.24, which is slightly better than the effect size typically observed in web-based PNF interventions correcting descriptive drinking norms (Dotson et al., 2015). For the Alcohol PNF + Reflective condition d=.46, a far larger effect on drinking behavior than typically observed in web-based PNF interventions.

Table 3.

Drinking at follow-up as a function of intervention condition, drinking/parting at initial assessment, and interactions between condition and drinking at initial assessment (N=222).

Predictors	B	SE	R2	F	df
Sex	.02	.08
Intervention PNFs Viewed	.09	.07
Initial Assessment z-Drinking Behavior	.94***	.08
PNF only (ref=control)	−.15	.10
PNF + Reflective Judgements (ref=control)	−.28**	.10
Initial Drinking* PNF only	−.27*	.11
Initial Drinking* PNF + Reflective Judgements	−.41***	.12
Total model			.56***	38.51	7, 214
Change due to interactions			.03**	6.31	2, 214

^*p<.01

^**p<.01;

^***p<.001.

3.4 AIM 3: Drinking at Initial Assessment as a Moderator of Conditional Effects

The regression results presented in Table 3 indicated that conditional effects on drinking behavior at the 2-month follow-up were indeed moderated by students’ drinking at initial assessment. Interactions were plotted and the PROCESS macro tested whether each intervention condition differed from control among students who were light, moderate, and heavy drinkers at the initial assessment. For the purposes of this analysis, and to be consistent with standard practices in assessing moderation (Tabachnick & Fidell, 2012), these three drinking levels were defined relative to the overall mean for baseline alcohol use. Specifically, light drinking was defined as baseline consumption of 1SD below the mean, moderate drinking was defined as a mean level of alcohol consumption at baseline, and heavy drinking was defined as 1SD above the mean. As visible in Figure 3, conditional differences increased as drinking status at the initial assessment increased. Specifically, among the light drinkers neither intervention condition significantly reduced behavior relative to control, presumably because there was little room for behavior to decrease. Among moderate drinkers, the PNF + Reflective condition was significantly more effective in reducing drinking behavior than was control (B= −.28, p=.005) while the PNF only condition did not significantly differ from control (B= −.15, p=.12). The largest drinking reduction effects associated with the intervention conditions were observed among the heavier drinking students. Among these students, both the PNF only (B= −.38, p=.004) and PNF + Reflective (B= −.63, p <.001.) conditions significantly reduced drinking behavior relative to control.

4. Discussion

This study sought to create the first fully-gamified self-sustaining web-based college alcohol intervention. The goal was to incorporate personalized normative feedback, the main component of drinking-reduction interventions, into a social game that students would play for fun across multiple weeks without being individually incentivized to do so. It was anticipated that providing feedback in the context of a game, which was not explicitly focused on alcohol, might improve efficacy among heavier drinkers by reducing defensive reactions to the intervention content. Another objective was to introduce an additional type of feedback in addition to the standard descriptive norms feedback that is generally used in PNF studies. It was predicted that students, and especially heavy drinkers, who received feedback on how acceptable opposite-sex students found their self-reported drinking behavior (reflective norms) would be more influenced by the intervention content than those who only received feedback on descriptive norms and those who did not receive either type of alcohol feedback.

With only 3 recruitment emails and no campus advertising or university support, approximately 45.6% of the eligible first-year students created a profile and played the first round of CampusGANDR. Further, students were given freedom to submit and vote on the questions themselves and alcohol- and drug-related questions were the most frequent submissions and received the highest number of votes, on average. Results revealed that CampusGANDR successfully reduced both participants’ normative perceptions of typical same-sex student behavior and their own alcohol use two months following the final round of play. Further, this effect trended strongest among those who received both types of feedback compared to those who saw just the typical descriptive norms feedback. In fact, the effect size for drinking reductions in the PNF + Reflective condition was .46, double what is typically found for remotely-delivered PNF, suggesting this additional feedback coupled with the gamified elements may prove to be a leap forward for brief alcohol interventions with this population.

Importantly, results showed that this gamified intervention format was especially impactful among heavy-drinking students, the most at-risk and consistently difficult-to-influence cohort of college alcohol users. It appears that heavy drinkers may benefit the most from the gamified features described in this paper. As suggested by our recent research (Boyle, Earle, McCabe, & LaBrie, 2017), this may be because heavy drinkers are more likely to react defensively to the feedback presented in a typical PNF intervention. However, this may simply be a dosage effect (i.e., participants in the reflective condition received more alcohol feedback). Similarly, the opposite-sex reflective feedback in the present study was not tested against same-sex feedback. Thus, it is impossible to tell from the present findings whether same-sex feedback or opposite-sex feedback would be more effective and for whom. However, it appears that gamified interventions, like CampusGANDR, may reduce defensiveness among heavy drinking students, leading to larger effects. Further, the addition of feedback on reflective judgments in the third condition of the present study also appears to have been most beneficial among heavy-drinking students. This may have happened because heavy-drinking students simply have more opportunity to reduce their drinking due to starting off at a higher level of consumption. Or this might have occurred because heavy drinkers somehow uniquely benefited from the reinforcement provided by supplementing standard descriptive norms feedback with additional feedback detailing the attitudes of opposite-sex students. More research is needed to determine which is the case.

These promising results were generated with minimal recruitment efforts and zero incentives for individual participation. This provides support for the premise that, in some situations, students may not need to be mandated to participate in risk-reduction interventions but can be recruited into these programs voluntarily when they are packaged as interactive games rather than as alcohol education programs. Of course, mandatory programs generally have considerably higher completion rates and, thus, are a better option under certain circumstances. Further, another aim was to allow students to take ownership of the content themselves by submitting and voting on their own questions. The majority of the participants in this study submitted at least one question (which was entirely optional), and approximately a quarter of all submitted questions focused on drug/alcohol use and partying. This suggests that allowing participants to determine the content of social norms programs themselves, rather than predetermining all of the questions in advance, appears to be a viable possibility; risk-reduction topics will emerge organically because students are naturally interested in these issues.

The GANDR approach has the potential to be utilized by universities nationwide as an alternative and/or addendum to traditional remotely-delivered alcohol education and prevention programs. The game could be sponsored by universities and marketed to incoming first-year students just prior to their arrival on campus as a fun way to connect with and get to know their soon-to-be classmates. Future research should investigate methods for increasing participation including starting recruitment earlier (prior to students’ arrival on campus), promoting the game at first-year orientation, marketing it in the dorms, targeting students using Facebook ads, and offering more enticing university-based prizes.

4.1 Implications

The study was relatively small and more research is clearly needed before strong conclusions can be drawn. However, the results suggest that this approach shows promise as a novel risk-reduction strategy for college students. Most importantly, this second iteration of the CampusGANDR intervention appears to be self-sustaining. Students played for fun, not because they were compensated for their participation. Thus, this study represents an important contribution not just in terms of intervention content but also with regard to dissemination. If PNF is to have a significant public health impact on a national level, researchers must find ways to distribute normative feedback-based interventions to vast groups of students across the country for minimal cost. The GANDR approach, in which PNF is delivered within an interactive game that leverages social media connectivity, elements of chance, points, user-generated content, and virtual co-presence appears to be one potential solution. Participants in the current study played for 6 weeks, voluntarily viewing three separate modules of alcohol-related feedback, purely because they enjoyed the game and were interested in the way the content was presented.

Further, when viewed in the context of our other recent work on gamified alcohol interventions, this small investigation takes on more significance (Boyle et al., 2017). It appears that interventions to reduce high-risk drinking do not necessarily have to be framed to students as studies about alcohol use or programs to educate them about drinking. In fact, effects may be larger, especially among heavy drinkers, when the explicit alcohol focus is removed and feedback is instead incorporated into a game. Additionally, this study was unique in the literature in that students did not participate because they were required to or because they were compensated but because they actually wanted to. The success of this approach suggests that voluntary student participation is viable and that if interventions are framed in more appealing ways students will participate without the presence of extrinsic motivators. Finally, to our knowledge, this was the first PNF study in which participants were asked to help choose the content themselves by submitting and voting on their own questions each week. Our findings revealed that risk-reduction topics emerged organically and that these student-submitted questions were often nearly identical to items traditionally used by researchers in alcohol use interventions. This finding both confirms that these issues are relevant to students and suggests that it may not be necessary for researchers to dictate all of the intervention content themselves as has previously been done in PNF interventions.

4.2 Limitations and Directions for Future Research

As a small preliminary trial, the current study has limitations that future work will want to address. First, the observed reductions in drinking were statistically significant when the PNF + Reflective condition was compared directly to Control, but the other two comparisons, between PNF and Control and between PNF and PNF + Reflective, did not reach significance—though a clear trend toward significance was observed. It appears that the relatively small sample size used (222 students in the final longitudinal sample) may not have been large enough to observe these effects at the .05 significance level. Given the positive findings reported here, a larger study is needed with more students, a longer follow-up period, and multiple sites. Second, the current investigation did not allow for an examination of dosage effects. Further, our smaller sample when broken across three conditions did not give us enough power to test gender differences between the conditions. Future studies could look at whether the addition of reflective feedback is important for both male and female students. Next, it is unclear if the current feedback delivery schedule is optimal for best effects on drinking. In this study, students played for six weeks and received alcohol feedback during three of these weeks. It is an open research question whether delivering alcohol-related feedback less often would make it more effective—as predicted by the scarcity principle—or whether delivering it more often would make it more effective—as predicted by reinforcement theory. Future studies should examine different ways of structuring gameplay and delivering alcohol feedback within the game. For instance, gameplay might be extended over an entire semester or even the entire first year of college.

Additional limitations pertain to the follow-up period, the addition of multiple new features across all conditions, and the inability to provide reflective feedback that was specific to a participant’s sexual identity. First, the follow-up period in the present study was two months. While this is a considerable improvement compared to the 2-week follow-up used in the initial CampusGANDR study, it is still relatively short. In future research it will be important to examine the long-term effects of the CampusGANDR approach across the entire first year of college and beyond. Second, the design of this study allowed us to test only the cumulative impact of these gamified features; not the impact of any single feature on its own. This design was consistent with our goal of testing a fully-gamified smartphone-based intervention in the real world. However, these findings leave the question of how each specific feature and interaction between features (ie., gradual delivery of feedback across several weeks of play) contributed to the reported effects unanswered. Finally, the sample size in the current study did not allow for the providing of reflective feedback that was specific to students’ sexual identity. The manner in which the reflective norms were calculated in CampusGANDR requires a minimum of approximately 200 participants per week in order to generate meaningful averages for each response option. Based on the proportion of sexual minority students in the typical university’s student body, this means that providing separate reflective norms to sexual minority students would require an overall sample size of well over 1000 participants per week. This is a limitation of the approach. Investigating methods for providing more relevant feedback to sexual minority students using the CampusGANDR approach is an important area for future research. Nonetheless, sexual minority students may be influenced by opposite-sex peers’ evaluations of their drinking.

4.3 Conclusion

In this study, first-year college students played a smartphone-based social game for 6-weeks without being forced to participate or offered individual incentives for participation. Approximately 45% of eligible students played with very minimal recruitment. During the game participants answered three questions about their own and their classmates’ alcohol use and were randomly assigned to receive either no alcohol-related feedback (control) standard feedback, or supplemented feedback on their perceptions and behaviors. Results revealed that participants who received the supplemented feedback, and especially those who were heavy drinkers before taking place in the intervention, reduced their drinking significantly during the two months post-intervention, relative to control participants. This suggests that alcohol risk-reduction interventions can potentially be self-sustaining if they are packaged as games, involve user-generated content, and include multiple types of feedback.

Table 1.

Descriptive Statistics by condition for norm and behavior items at initial assessment and follow-up.

	Control N=71		Alcohol PNF N=79		Alcohol PNF + Judgments N=72
	DNorm M (SD)	Behavior M (SD)	DNorm M (SD)	Behavior M (SD)	DNorm M (SD)	Behavior M (SD)
Initial Assessment
Wk2 Peak Drinks	4.99 (2.87)	3.80 (4.09)	4.89 (2.73)	3.87 (4.07)	5.09 (2.72)	4.23 (4.14)
Wk3 Weekly Party Nights	1.93 (1.28)	1.09 (1.28)	2.09 (.73)	1.39 (1.63)	1.91 (.71)	1.12 (.90)
Wk4 Weekend Drinks	4.54 (3.37)	3.32 (4.81)	3.70 (2.17)	2.65 (3.74)	3.83 (2.06)	3.08 (4.10)
Z-Composite	.05 (.91)	−.004 (.93)	−.01 (.55)	.02 (.88)	−.04 (.56)	−.004 (.77)
2 Month Follow-Up
Wk2 Peak Drinks	5.62 (3.43)	3.82 (4.28)	4.80 (2.84)	3.53 (3.38)	4.19 (2.31)**	2.97 (3.25)**
Wk3 Weekly Party Nights	2.25 (1.28)	1.14 (1.32)	1.66 (.65)**	1.06 (.91)*	1.53 (.67)***	.82 (.84)**
Wk4 Weekend Drinks	5.77 (4.07)	3.21 (4.70)	4.34 (3.05)	2.26(3.28)*	3.63 (1.85)	1.94 (2.67)*
Z-Composite	.35 (1.04)	.13 (1.11)	−.07 (.66)	.01 (.79)	−.26 (.50)***	−.15 (.69)***

Note. At initial assessment (Weeks 2, 3, and 4) there were no significant differences on the norms or behaviors by study condition. Significant reductions in norms and behaviors from initial assessment to follow-up are flagged at follow-up

^*p<.05

^**p<.01;

^***p<.001.

Highlights.

• GANDR, a gamified alcohol preventative intervention for college students, reduced normative perceptions and drinking.

• GANDR produced reductions in norms and drinking in a self-sustaining way through game play.

• GANDR employs intrinsic motivation as students play a role in normative content to be presented and the content is spread out over time.

• Students are interested in understanding what their peers are doing with respect to drinking as nearly ¼ of questions submitted in GANDR for subsequent feedback dealt with partying and alcohol use.