Real-Time Demonstration of a mHealth App Designed to Reduce College Students Hazardous Drinking

Abstract

Heavy alcohol use is a serious health issue in the United States with consequences such as illness, injury, and death. College students are among the most vulnerable to problems associated with risky drinking. This demographic is known as the Net Generation because members have grown up with digital technologies such as smartphones and apps. Thus, mobile health (mHealth) applications, successful in the delivery of health information and interventions to tech-savvy individuals, are a promising means of reaching them. To that end, we developed a smartphone application (SmarTrek) that targets college students and aims to reduce risky alcohol use. SmarTrek features are easy to use and have interactive components including text messages that incorporate motivational interviewing and ecological momentary interventions. We conducted iterative theater testing, field testing, and focus groups to evaluate the acceptability of SmarTrek with college students. We identified salient issues that might arise from SmarTrek use and modified the app based on feedback from participants. Participants were assigned to 2 groups (Group 1 [n = 4] and Group 2 [n = 6]). At baseline, participants completed 2 standardized surveys. Following their field testing, each group of participants was then invited to a focus group session of the app in which they provided feedback. The majority of participants (90%) agreed that SmarTrek was easy to use and that the information provided was useful and had a positive effect on decreasing their drinking.

College students drink heavily, engage in binge drinking, and often do so at considerable risk to themselves and others (Hingson, Zha, Simons-Morton, & White, 2016). In 2017, the American College Health Association conducted their annual National College Health Assessment of 52 U.S. postsecondary institutions to gather representative data on students’ health behaviors and trends. The National College Health Assessment survey of 37,638 college students found that in the past 12 months 67% reported consuming alcohol. In addition, as a result of drinking, 12% of college students were physically injured, 35% did something that they later regretted, 29% had memory loss, 3% seriously considered suicide, and 22% engaged in unprotected sex (American College Health Association, 2017). The data evidently support the observation that interventions to reduce risky drinking and related consequences in college students should be examined.

Because of the high costs of implementation, effective in-person interventions for reducing alcohol use are unavailable to a vast number of individuals (Zarkin, Bray, Hinde, & Saitz, 2015). However, mobile-based interventions do not face similar constraints and can reach a broader population (Milward, Day, Wadsworth, Strang, & Lynskey, 2015). Mobile-based interventions, such as smartphone apps, show promise in promoting behavioral change in alcohol use and in a cost-effective way (Gulliver et al., 2015). These apps have features, such as goal setting and feedback, which allow users to tailor the app to meet their individual needs (Attwood, Parke, Larsen, & Morton, 2017; Perski, Blandford, Ubhi, West, & Michie, 2017).

Today’s college students are known as the Net Generation because they have grown up with digital technologies including mobile apps, text messaging, and online chat (Oblinger & Oblinger, 2006). Their comfort level with technology makes them a prime demographic for mobile health (mHealth) apps that deliver health services including prevention, treatment, and education. Health service professionals deliver mHealth services to patients, clients, and research participants using portable communication devices such as smartphones, smart-books, and iPads (Free et al., 2010; Kazemi, Cochran, Kelly, Cornelius, & Belk, 2014). Newer technologies, such as smartphones and Fitbits (that track heart rate, exercise, sleep patterns, and weight) may provide an even more efficient means of intervention delivery (Berman, Gajecki, Sinadinovic, & Andersson, 2016; Cohn, Hunter-Reel, Hagman, & Mitchell, 2011; Haug et al., 2015). To test this theory, we created SmarTrek, a smartphone app that delivers real-time interventions to college students with risky drinking behaviors.

The app was designed in Swift for the Apple iOS framework by a collaboration of university-affiliated researchers. It was built in the Smart Mobile Innovation Lab at a large public university in the southeastern United States. The server side of the app was designed, developed, and hosted by the Smart Mobile Innovation Lab team. SmarTrek is informed by current theories of motivational interviewing (MI; Miller & Rollnick, 2013) and ecological momentary interventions (EMIs; Heron & Smyth, 2010). MI is an interpersonal communication style that uses a directive, evocative, and collaborative approach to strengthen intrinsic motivation for behavior change. EMIs are mHealth technologies that deliver interventions to individuals in their natural physical environments in real time (Heron et al., 2010). EMIs have been accepted as effective for alcohol interventions delivered by smartphones and applied in real time in natural environments (Heron et al., 2010). It is important to note that EMI presents the content in real time; however, it does not describe the content. Given the emerging trend to use mHealth with young adults, we combined the communication style of MI with the real-time application strategies of an EMI for use in an intervention to reduce hazardous drinking among college students.

SmarTrek was designed to collect data, track behavior, provide education, and offer incentives for behavioral changes. SmarTrek addresses alcohol use through eight functions. Interactive Games enhance knowledge of the potential harm from drinking. Know Your BAC (blood alcohol content) and Daily Log are tracking features that provide information relevant to changing drinking behaviors. The virtual Coach is fully automated and sends daily text messages, which help users explore the pros and cons of their drinking behaviors. The Coach text messages are designed to engage participants in using the app’s other features. Through daily text messages and alerts, the Coach encourages participants to complete logs on their drinking and offers alternative strategies to drinking. For example, “Would you like to calculate your blood alcohol content?” is a prompt for the participant to use the BAC calculator. Consistent with MI, the My Strategies and Personalized Feedback features assist users to develop a plan to change their drinking behaviors. My Strategies provides a list of options users may choose to help them change drinking behaviors. Personalized Feedback provides a profile of drinking behaviors and their consequences with comparisons to peer drinking behaviors. Where to Go provides access to local resources. Finally, Learn More has educational information regarding the harmful effects of drinking. These features were used to enhance intervention appeal, to provide positive nonconfrontational feedback, and to engage students in reducing their risky drinking behaviors.

The research objective was to obtain information regarding SmarTrek feasibility and efficacy through iterative phases of theater testing, field testing, and focus groups with college students using SmarTrek to enhance the clarity, consistency, and usability of the app. The study’s objectives were to (a) test the acceptability of SmarTrek, (b) identify salient issues that might arise from SmarTrek use, and (c) make modifications based on feedback from participants.

Method

Participants

Participants were eligible if they: (a) were enrolled as an undergraduate at the university, (b) owned an iPhone, (c) were able to communicate in English, and (d) had consumed alcohol on at least one occasion in the past month. Participants received a $40 gift card for each theater testing, field testing, or focus group session—up to $120 for all three sessions. The mean age of participant (N = 10) was 22.70 years (SD = 7.66). Six were freshmen (60%), two were sophomores, and two were juniors. Institutional review board approval and informed consent were obtained.

Measures

Participants were asked to provide basic demographic information and fill out surveys at the baseline session. Their responses were used to create user profiles. These, in turn, were used to generate each user’s personalized feedback and text messaging transcripts. The Readiness Ruler, which is a simple tool used to determine an individual’s readiness to change drinking behaviors, was also administered at baseline (Moyers, Martin, Houck, Christopher, & Tonigan, 2009). The Ruler has a scale of 1–10, with 1 being not at all important and 10 being extremely important. Participants are asked, “On a scale of 1 to 10, how important is it for you to make a change in drinking behavior?” Responses to the Readiness Ruler were used to provide individualized text messages aimed at helping individuals understand their drinking patterns. The Daily Drinking Questionnaire (DDQ; Collins, Parks, & Marlatt, 1985) is a brief questionnaire that takes about 5 minutes to complete. The DDQ measures drinking patterns, including quantity and frequency of use, and daily peak drinking events over a typical week and averaged over the past month. The DDQ has demonstrated moderate to high reliability, with Cronbach’s alphas ranging between .66 and .75 (Allen & Wilson, 2003).

Postfield testing assessment.

The purpose of the two surveys at this phase of research was to gather data on participants’ views of the feasibility and acceptability of SmarTrek. First, the Usefulness, Satisfaction, and Ease of Use survey (USE; Lund, 2001) is a 26-item questionnaire widely used for measuring the effectiveness of apps. USE was designed as a 7-point Likert scale questionnaire, where 1 = strongly disagree and 7 = strongly agree. Usability is measured in three dimensions: usefulness (four items), satisfaction (seven items), and ease of use (15 items). Sample prompts include, “I am satisfied with it,” “It is useful,” “It meets my needs,” “It is a pleasure to use,” and “I learned to use it quickly.” Second, the Satisfaction Survey is a 22-item measure developed by the research team to assess the participants’ response to SmarTrek. Survey questions elicited participants’ opinions about SmarTrek features. For example, “Which SmarTrek feature(s) did you like best and why? Which SmarTrek feature(s) did you like least and why? On average, how much time did you spend on SmarTrek each day?” The frequency of each feature’s selection was tallied for each question.

Procedures

The purpose of the theater testing, field testing, and focus groups was to gather data on participants’ views on the feasibility and acceptability of SmarTrek. Groups were run sequentially through theater testing, field testing, and focus groups to permit modifications suggested by Group 1 to be made to SmarTrek before Group 2 began. To obtain preliminary data comparing the optimal length of time for app usage, we had Group 1 use the app for 1 week and Group 2 for 2 weeks.

Theater testing.

Theater testing is a method used to gather data from a large group of people at the same time (National Cancer Institute, 2002). It allows individuals to interact with products or audiovisual presentations to simulate real-world experiences (National Cancer Institute, 2002). The 10 participants were randomly assigned to two groups (Group 1: n = 4; Group 2: n =6). Each group attended a demonstration session moderated by facilitators at a central location. After receiving a brief overview of the study, participants received instructions on the functionalities of SmarTrek and information on the day-to-day use of the app. Then the app was downloaded on each participant’s iPhone. Facilitators made sure the app functioned and answered participants’ questions. Participants were provided with facilitator contact information and were told to phone or e-mail if there were any issues. These sessions lasted for approximately 1 hr. At the end of the theater testing, participants field tested the SmarTrek app.

Field testing.

The participants used the SmarTrek app for 1 week (Group 1) or 2 weeks (Group 2). Participants were encouraged to use the app for a minimum of 5 min each day. At the end of field testing, participants completed the USE and Satisfaction surveys and a focus group session.

Focus groups.

At the conclusion of field testing, a focus group was conducted by two skilled facilitators to assess the acceptability and satisfaction of SmarTrek. The facilitators were research assistants who had been trained in focus group methods. A moderator directed the hour-long discussion, and an assistant facilitator operated the tape recorder, kept notes, and helped moderate the discussion. The research team developed seven questions to explore the participants’ views of the clarity, consistency, and usability of SmarTrek. Following Krueger’s (1995) recommended methods, the moderator followed the same sequence of questions each session. The questions were designed to elicit data regarding the participants’ overall impression of the SmarTrek app. Sample questions included: “Which of the features did you like best? Why? Least favorite feature? Were the texts too long?” “Did you experience any technical difficulties while using the app?” and “Did you like SmarTrek appearance, colors, name, and font size?” Recordings of the focus group sessions were transcribed verbatim. Thematic analysis of the focus group content in the transcriptions was used to analyze the participant responses using a codebook developed following three steps. First, concurrent reading of transcribed interviews and listening to audio recordings were done. Second, the data were then categorized according to a thematic topic guide developed for this project (key components, text messaging dose, and delivery protocol). Third, three researchers compared their themes to arrive at a consensus. The coding yielded a 90% intercoder agreement, in the excellent range (Landis & Koch, 1977). The responses guided the final components of SmarTrek.

Results

Alcohol Use

All 10 participants had consumed alcohol 9.60 times (SD =7.97) in the past 30 days, a value at the 95th percentile of the drinking frequency of the university students. On average, the typical BAC of all participants was 0.061 (SD = 0.042), with a range of 0.016–0.128. Six of 10 participants reported that the SmarTrek app had a positive effect on their drinking less. One of the 10 participants reported that the SmarTrek did not have a positive effect and three were undecided (see Table 1).

Table 1.

Tallied Responses From the Satisfaction Survey About SmarTrek’s Effect on Drinking

SmarTrek app had a positive effect on my drinking less?
Strongly agree	2 (20%)
Agree	4 (40%)
Undecided	3 (30%)
Disagree	1 (10%)
Total	10 (100%)

Note. N = 10.

Satisfaction

Results from the USE (Lund, 2001; n = 7) indicate good usability of SmarTrek. The average overall usability score was 5.97 of 7 (SD =1.20, α = .916). The three subscales and their average scores were usefulness (M = 5.68, SD = 1.06, α = .78), ease of use (M = 6.03, SD = 1.24, α = .951), and satisfaction (M = 6.02, SD = 1.20, α = .65). The Satisfaction survey (N = 10) revealed that among the eight features (Games, Know Your BAC, Daily Log, Coach, My Strategies, Personalized Feedback, Where to Go, and Learn More), Games was the most frequently selected best feature (chosen by eight of the 10 participants), with Know Your BAC selected second. Participants reported that Know Your BAC and My Strategies were most useful in monitoring alcohol intake, creating behavioral change plans, and reminding them of their goals. Participants also said that Daily Log, Coach, and Personalized Feedback to be the most useful. Half of the participants reported that Where to Go was the least useful feature of the app.

Focus Group Results

Most of the participants (90%) believed that it was not a burden to report drinking and to keep a daily log. The Coach feature connected well with participants. Half of the participants enjoyed interactions with the Coach but also reported that they sometimes ignored information from the Coach. Half of the participants agreed/strongly agreed that they drank less while using SmarTrek and six agreed that SmarTrek had a positive effect on reducing their drinking. Participants also reported that it was fun to interact with SmarTrek, to receive different types of messages, to track drinking with a personalized drinking profile, and to play the competitive Trivia Game, which enhanced their learning (see Table 2).

Table 2.

Focus Group Participant Comments

Group 1	Group 2
Positive comments
Easier to figure out and use than most apps	App was very user friendly!
I liked it because you could track how you drink and your BAC	Loved the layout, self-explanatory tabs
	Easy to learn
Easy to navigate with the categories on the left side	I often played with the app throughout the day
Foolproof	Entertaining, not bland
Self-explanatory	Not a hassle to use every day
Some were really cool and I had no idea those were myths or real [about the texts]	Liked how the Coach was never negative; it was always positive
The calculator was so cool—could lead to fewer accidents and more understanding of BAC and timing of drinking	Informative, very in-depth information
These strategies raised my awareness and I noticed which strategies do correlate with drinking less	I did have fun with the trivia…pulled out the trivia game with my friends; it was a learning experience, even with them
I think that you could help a lot of students	Good facts in the trivia game; I learned a lot
I liked that you could customize them—weight gender, type of drink, timing [about BAC calculator]	I liked the quotes—They were unexpected
I think it’s beneficial because it’s not in your face, like “your drinking habits are bad”	I planned on using it 10–15 min every night, but I ended up using it more often when I was bored
It helps you help yourself
Negative comments
Would like to be able to add drinks at a later date	Sometimes I missed the messages because they were coming in fast The appearance was a little too bright
Overwhelming at first—It didn’t stop. I like to get rid of my notifications [about the texts]
Too many topics back to back in a short period of time	I would like larger text size

Note. BAC = blood alcohol content.

Themes and Recommendations

Both focus groups responded favorably to SmarTrek. Seven major themes were identified; SmarTrek was described as very informative, user friendly, and easy to navigate. Participants also suggested many additional features to add to SmarTrek’s daily log, settings, games, and login. For example, participants suggested adding a calorie count and money spent on alcohol to the daily log. Other suggestions included adding an option to record zero drinks when the participant did not drink that day, the ability to edit the log (if something was entered incorrectly), and the ability to add drinks from different days (instead of having only today or yesterday as options). Participants requested campus-specific rules for the trivia game, a bonus-point option, and more questions/a greater variety of questions such as harm reduction, scenario based, and relevant topics. Regarding the settings, participants would like the option to use SmarTrek in different languages and color schemes, and the ability to log in with a fingerprint. Participants also would like the Coach to be more interactive.

Based on feedback from the surveys and focus groups, SmarTrek’s usability and functionality were enhanced. Technical issues identified by Group 1 were corrected before Group 2 began. Text messages were sent at a faster rate, the text font size was enlarged, the length of text messages was reduced, and delays in the Coach feedback were fixed. Further enhancements were made based on feedback from Group 2 participants.

Discussion

This study was among the first to identify salient issues that might arise from SmarTrek use and to create an intervention pairing mHealth technologies with MI and EMI theoretical constructs. During the iterative testing, we identified and updated SmarTrek to address the resolutions to enhance the app usability and functionality. We developed SmarTrek with a suite of features to enable effective communication and interactions with college students to address high-risk drinking. SmarTrek has a wide variety of purposes, including data collection, intervention, behavior tracking, incentive offers, and education.

Using variable methods (e.g., quantitative surveys, qualitative focus groups), we found that the majority of participants (90%) agreed that SmarTrek was easy to use, and the information provided was useful and had a positive effect on decreasing their drinking. The major themes that emerged endorsed SmarTrek’s ability to motivate, encourage, and support reflection on drinking behaviors. Features included a choice of personal strategies to reduce drinking, self-monitoring on alcohol intake, and choice of coach. Participants’ suggestions were used to modify the app.

Several limitations should be noted. The sample size was small and drawn from a specific public university in the southeastern United States, which limits the generalizability of the information. The findings may also be biased for SmarTrek usability because only seven of the 10 participants completed the USE questionnaire. Furthermore, it should be noted that although we encouraged the participants to use the app 5 min a day, we did not tract their daily usage time. Future investigations at other geographic locations and in private and public schools in urban and rural settings will enhance generalizability. Despite these limitations, the participants were willing to inform research on SmarTrek, a mobile-based intervention designed to reduce hazardous drinking.

Widespread implementation of in-person, evidence-based interventions in the substance abuse field has been slow. This is due, in part, to feasibility issues in training staff, providing in-person services, and funding such interventions. Therefore, we created the SmarTrek app to take advantage of students’ smartphones as a platform for delivering a real-time alcohol intervention for college students with high-risk drinking behaviors. Despite the positive response from our project, further examination of the efficacy of the mobile app technology is needed. The modifications made in response to user feedback need to be tested with larger sample sizes. Furthermore, a randomized trial is warranted to test the efficacy of SmarTrek as an intervention among college students. SmarTrek has the potential to be a stand-alone intervention as a university orientation tool for first-year students or a complementary intervention in conjunction with in-person counseling. The mHealth smartphone app also has the potential to address other important health issues, such as risky sexual behavior, HIV risk reduction, smoking cessation, and obesity.