Skip to main content
PMC home page
Journal of Studies on Alcohol and Drugs logoLink to Journal of Studies on Alcohol and Drugs
. 2009 Sep;70(5):771–775. doi: 10.15288/jsad.2009.70.771

Alcohol Assessment Among College Students Using Wireless Mobile Technology*

Jay M Bernhardt 1,†, Stuart Usdan 1,†, Darren Mays 1,†, Ryan Martin 1,†, Jennifer Cremeens 1,†, Kimberly Jacob Arriola 1,†
PMCID: PMC2741555  PMID: 19737502

Abstract

Objective:

This study used a two-group randomized design to assess the validity of measuring self-reported alcohol consumption among college students using the Handheld Assisted Network Diary (HAND), a daily diary assessment administered using wireless mobile devices.

Method:

A convenience sample of college students was recruited at a large, public university in the southeastern United States and randomized into two groups. A randomly assigned group of 86 students completed the daily HAND assessment during the 30-day study and a Timeline Followback (TLFB) at 30-day follow-up. A randomly assigned group of 82 students completed the paper-and-pencil Daily Social Diary (DSD) over the same study period. Data from the daily HAND assessment were compared with the TLFB completed at follow-up by participants who completed the HAND using 95% limits of agreement analysis. Furthermore, individual growth models were used to examine differences between the HAND and DSD by comparing the total drinks, drinking days, and drinks per drinking day captured by the two assessments over the study period.

Results:

Results suggest that the HAND captured similar levels of alcohol use compared with the TLFB completed at follow-up by the same participants. In addition, comparisons of the two study groups suggest that, controlling for baseline alcohol use and demographics, the HAND assessment captured similar levels of total drinks, drinking days, and drinks per drinking day as the paper-and-pencil DSD.

Conclusions:

The study findings support the validity of wireless mobile devices as a daily assessment of alcohol use among college students.

Heavy alcohol use among college students is a public health problem in the United States (Substance Abuse and Mental Health Services Administration, 2004; Wechsler et al., 2000). Despite significant advances in alcohol surveillance, prevention, and intervention research, the measurement of alcohol use among college students remains a challenge for alcohol researchers because students' drinking behaviors are characterized by infrequent, heavy drinking episodes (Dawson, 2003; Greenfield, 2000; Substance Abuse and Mental Health Services Administration, 2004).

In part to overcome some of the methodological limitations of existing assessments of alcohol use, the National Institute on Alcohol Abuse and Alcoholism identified the need to determine the most effective uses of computer-based technologies to help prevent dangerous alcohol use in college settings (Task Force for the National Advisory Council on Alcohol Abuse and Alcoholism, 2002). For instance, wireless mobile devices (MDs) such as handheld computers, cellular phones, and smart phones provide a number of advantages over traditional paper-and-pencil methods of measurement and thus may be useful in improving the assessment of alcohol use among college students (Bernhardt et al., 2001).

Previous research has demonstrated that the use of MDs, such as handheld computers, for collection of sensitive behavioral data (e.g., drug and alcohol use, sexual behavior) from students resulted in substantially fewer missing data points compared with paper-and-pencil questionnaires and face-to-face interviews (van Griensven et al., 2006). Research also suggests that electronic daily diaries may be a useful method for alcohol assessment; however, only a few prior studies have used the wireless capabilities of MDs for alcohol assessment among college students (Bernhardt et al., 2007; LaBrie et al., 2006; Weitzel et al., 2007). By using existing daily reporting methods of alcohol assessment and incorporating innovative wireless mobile technology, it may be possible to create an accurate assessment of drinking behaviors among college students and provide opportunities to apply this technology for tailored interventions (Bernhardt et al., 2005, 2007; Weitzel et al., 2007).

This study sought to evaluate the validity of measuring self-reported alcohol consumption among college students using daily Handheld Assisted Network Diary (HAND) assessments for alcohol use administered on wireless MDs by comparing data collected using the HAND with two validated methods of alcohol assessment. First, reported alcohol use on the HAND was compared with a paper-and-pencil, self-completed Timeline Followback (TLFB; Sobell et al., 1986) assessment completed at follow-up by the same participants. Second, data collected using the HAND assessment were compared with data collected among a randomly assigned comparison group of participants who completed a paper-and-pencil Daily Social Diary (DSD) over the same 30-day period.

Method

Design and procedures

The study used a pre-test, post-test design with randomized group assignment to examine the validity of the HAND for measuring alcohol use among college students at a large, public university in the southeastern United States. The study procedures were approved by the institutional review board at the participating institution, and all participants provided signed informed consent before taking part in the study.

The methods of the study are described in detail elsewhere (Arriola et al., 2009). Briefly, a convenience sample of college students (n = 170) was randomly assigned to complete a daily assessment of their drinking using either the HAND (Bernhardt et al., 2007; Weitzel et al., 2007) or the paper-and-pencil DSD each day for 30 days. Participants were asked to complete a baseline TLFB, the daily HAND or DSD for 30 days, and a follow-up TLFB after 30 days.

Measures

Daily drinking was measured on the TLFB, HAND, and DSD by asking participants whether they drank alcohol on the previous day and, if so, how many standard drinks they consumed. Standard drinks were consistently defined across the three assessments as 12 oz of beer, 5 oz of wine or champagne, 3 oz of fortified wine, or 1.5 oz of distilled spirits. Conversion factors were also provided to estimate the number of standard drinks consumed based on state-specific alcohol distribution laws at that time that required drinking establishments to serve distilled spirits in single-serving bottles rather than using conventional serving methods.

The TLFB administered at baseline and follow-up was self-completed (i.e., paper-and-pencil). Similar to prior research (Maisto et al., 2008), participants were provided with the TLFB calendar and instructions and were asked to complete information about their alcohol use each day for the previous 30 days. The reliability of the TLFB has been previously established by comparing it with electronic forms of measurement, including handheld computers, interactive voice response programs, computer-based assessments, and telephone interviews (Carney et al., 1998; Searles et al., 2000; Sobell et al., 1996).

The questions examining drinking on the daily HAND and DSD assessments were identical; the only difference between the two assessments was the mode of administration. The DSD was based on the drinking self-monitoring log, a validated daily diary-type assessment for alcohol use (Sobell et al., 1989), and was administered in paper-and-pencil format. At baseline, DSD group participants received a packet containing a one-page assessment for each day of the study period to be completed over the course of the study, which they then turned in when completing the follow-up TLFB.

The HAND was designed using EntryWare Designer software (Techneos Systems, Inc., Vancouver, BC) and administered using Palm Tungsten W smart phones (Palm Inc., Sunnyvale, CA), which were enabled to wirelessly transmit data to a secure web-based server after each assessment was completed. Participants who did not complete the HAND assessment by 12:00 pm each day were sent an automated email from the researchers reminding them to complete the HAND. Participants who still had not completed the assessment by 5:00 pm each day received an automated phone call from the researchers reminding them to complete the HAND. Between administrations of the daily HAND assessments, the MDs were locked at a waiting screen until the next day's assessment was available, which prevented participants from completing multiple HAND assessments in 1 day.

Statistical analyses

Data were analyzed using SAS 9.2 (SAS Institute, Cary, NC). Variables created for the HAND, DSD, and TLFB assessments included total drinks consumed, number of drinking days, and drinks per drinking day. The 95% limits of agreement method (Bland and Altman, 2003) using a correction factor for sample sizes of less than 100 (Ludbrook, 2002) was used to assess agreement between the HAND and the follow-up TLFB.

Three individual growth models were created using PROC MIXED (Singer, 1998) to compare differences in total drinks, drinking days, and drinks per drinking day between the HAND and DSD. Each outcome variable was aggregated at weekly intervals (Weeks 1 and 2 with 7 days, Weeks 3 and 4 with 8 days), totaling four time points for each individual growth model. The primary independent variable of interest was a group variable coded as “1” for the HAND group and “0” for the DSD group. Individual growth models also included age, gender, and the corresponding baseline TLFB drinking-variable controls. The total drinks variable from the baseline TLFB was transformed using a natural logarithm (natural log + 1) because it was strongly kurtotic (Emerson and Soto, 1983). Two participants were excluded from the analyses reported because they had extreme outlier values on all assessments of alcohol use.

Results

Participants

Of the participants included in the analyses (n = 168), 50.3% were female, the majority were white (77.7%), and the mean (SD) age was 19.96 (1.35) years. Demographic characteristics did not differ significantly between the two study groups. Among the participants in the DSD group (n = 82), the mean number of completed daily assessments on the DSD was 29.29 (0.91), with an average completion rate of 97.6%. Among participants in the HAND group (n = 86), the mean number of completed assessments on the HAND was 25.75 (5.90), with an average completion rate of 85.8%.

Comparison of the HAND with the TLFB

Overall, the mean total drinks (p = .21), drinking days (p = .56), and drinks per drinking day (p = .54) captured on the HAND and TLFB, respectively, did not differ significantly. The mean difference in drinks per drinking day between the HAND and TLFB was 0.07. The adjusted 95% limits of agreement using the small sample size (n < 100) correction factor (Ludbrook, 2002) were −3.68 to 3.83 standard drinks. In total, 94.2% of data (81/86 respondents) fell within this agreement interval.

Comparison of the HAND with the DSD

When controlling for the respective alcohol-use variables on the baseline TLFB, age, and gender, there were no statistically significant differences between the HAND and DSD in total drinks, drinking days, or drinks per drinking day captured each week during the 30-day study period (Table 1).

Table 1.

Unstandardized parameter estimates and 95% confidence intervals (CIs) comparing the Handheld Assisted Network Diary (HAND) to the Daily Social Diary (DSD) for alcohol-related variables

Variable Total drinks b (95% CI) Drinking days b (95% CI) Drinks/drinking day b (95% CI)
Assessment (1 = HAND, 0 = DSD) −0.15 (−2.52, 2.21) −0.09 (−0.37, 0.19) 0.50 (−0.41, 1.41)
Baseline TLFB
 Log (1 + total drinks)a 7.45† (6.06, 8.90) — —
 Drinking days — 0.18† (0.14, 0.23) —
 Drinks/drinking day — — 0.66† (0.49, 0.83)
Gender (1 = male, 0 = female) −2.97* (−5.41,−0.54) −0.25 (−0.53, 0.04) −0.58 (−1.52, 0.37)
Age 0.42 (−0.38, 1.35) 0.14* (0.03, 0.24) 0.29 (−1.52, 0.37)
Null model χ2 = 90.9, χ2 = 105.41 χ2 = 50.07
 likelihood 9 df, 9 df, 9 df,
  ratio test p < .001 p ≤ .001 p ≤ .001
Open in a new tab

Notes: TLFB = Timeline Followback.

a

Natural log transformation was applied because the baseline total drinks variable was strongly kurtotic.

Discussion

This study sought to evaluate the validity of measuring self-reported quantity and frequency of alcohol use among college students using the MD-based HAND. Our analyses suggest that the HAND assessment captured similar levels of alcohol use over the course of the study compared with two valid paper-and-pencil assessments of alcohol use, the TLFB and DSD. This study contributes to a growing body of literature on the assessment of alcohol consumption using a number of different communication technologies (Dawson, 2003). Specifically, this study suggests that wireless MDs may represent a valid method of assessment of daily drinking among college students.

Technical aspects of the design of the two daily assessments may have affected the comparisons reported. For example, the average completion rate among DSD participants (98%) was substantially higher than among HAND participants (86%). However, the design of the HAND prevented participants from completing assessments that were missed, and HAND group participants received reminders to complete daily assessments. Neither of these features was incorporated into the design of the DSD, which likely contributed to the observed differences in completion rates in particular. These technical aspects notwithstanding, the completion rate observed for the HAND indicates that, despite the potential burden for participants of completing the daily HAND assessment, the HAND can be completed daily with little deviation.

Although research suggests that technology such as wireless MDs offers many benefits for behavioral assessment (Bernhardt et al., 2001, 2005), the evolving technology of wireless MDs, including enhanced multimedia and improved wireless capabilities of more recent MDs, may expand the benefits of applying such technology for alcohol assessment. In particular, although preliminary evidence suggests that wireless MDs may be an effective tool for delivering individually tailored messages to reduce drinking among college students (Weitzel et al., 2007), future research is needed to examine both the assessment and intervention capabilities of more recent, more advanced wireless MDs over longer time periods.

The results of this study should be considered in the context of several limitations. The study used a convenience sample of college students; therefore, generalizations to broader populations should be made with caution. All drinking behaviors are based on participant self-reports. Although research suggests that self-report assessments are reliable (Del Boca and Darkes, 2003), the results should be interpreted with this limitation in mind. Because of the nature of the DSD, it could not be determined when the daily assessment was completed, thereby limiting the comparisons that can be drawn between the DSD and the HAND. Finally, participants' reported alcohol use on the follow-up TLFB may have been affected by the fact that they completed a daily assessment of alcohol use over the preceding 30 days. The comparisons of the HAND with the follow-up TLFB in particular should be interpreted with this limitation in mind.

Two primary research implications should be considered based on the present study. First, future research is needed to examine both the alcohol assessment and intervention capabilities of MDs, in particular investigating their use over longer study periods. In addition, research is needed to examine how advanced multimedia and communication capabilities of more recent wireless MDs can be incorporated into assessment and intervention methods for alcohol use and potentially other behaviors.

Acknowledgments

The authors are indebted to Richard E. Dixon and James B. Weaver III for their thoughtful feedback on this manuscript.

Footnotes

*

This research was supported by National Institute on Alcohol Abuse and Alcoholism grant 5R21AA013969-03. The preparation of this manuscript was supported in part by an appointment to the Research Participation Program for the Centers for Disease Control and Prevention (CDC) administered by the Oak Ridge Institute for Science and Education through an agreement between the Department of Energy and the CDC (Darren Mays). The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the U.S. Department of Health and Human Services or the Centers for Disease Control and Prevention.

References

  1. Arriola KD, Usdan S, Mays D, Weitzel JA, Cremeens J, Martin RJ, Borba C, Bernhardt JM. Reliability and validity of the Alcohol Consequences Expectations Scale. Amer. J. Hlth Behav. 2009;33:504–512. doi: 10.5993/ajhb.33.5.3. [DOI] [PubMed] [Google Scholar]
  2. Bernhardt JM, Strecher VJ, Bishop KR, Potts P, Madison EM, Throp J. Handheld computer-assisted self-interviews: User comfort level and preferences. Amer. J. Hlth Behav. 2001;25:557–563. doi: 10.5993/ajhb.25.6.5. [DOI] [PubMed] [Google Scholar]
  3. Bernhardt JM, Usdan SL, Burnett A. Using handheld computers for daily alcohol assessment: Results from a pilot study. J. Subst. Use. 2005;10:347–353. [Google Scholar]
  4. Bernhardt JM, Usdan S, Mays D, Arriola KJ, Martin RJ, Cremeens J, McGill T, Weitzel JA. Alcohol assessment using wireless handheld computers: A pilot study. Addict. Behav. 2007;32:3065–3070. doi: 10.1016/j.addbeh.2007.04.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bland JM, Altman DG. Applying the right statistics: Analyses of measurement studies. Ultrasound Obstet. Gynecol. 2003;22:85–93. doi: 10.1002/uog.122. [DOI] [PubMed] [Google Scholar]
  6. Carney MA, Tennen H, Affleck G, Del Boca FK, Kranzler HR. Levels and patterns of alcohol consumption using the Timeline Follow-Back, daily diaries, and real-time “electronic interviews.”. J. Stud. Alcohol. 1998;59:447–454. doi: 10.15288/jsa.1998.59.447. [DOI] [PubMed] [Google Scholar]
  7. Dawson DA. Methodological issues in measuring alcohol use. Alcohol Res. Hlth. 2003;27:18–29. [PMC free article] [PubMed] [Google Scholar]
  8. Del Boca FK, Darkes J. The validity of self-reports of alcohol consumption: State of the science and challenges for research. Addiction. 2003;98(Suppl. No. 2):1–12. doi: 10.1046/j.1359-6357.2003.00586.x. [DOI] [PubMed] [Google Scholar]
  9. Emerson JD, Soto MA. Transforming data. In: Hoaglin DC, Mosteller F, Tukey JW, editors. Understanding Robust and Exploratory Data Analysis. Hoboken, NJ: John Wiley & Sons; 1983. pp. 97–128. [Google Scholar]
  10. Greenfield TK. Ways of measuring drinking patterns and the difference they make: Experience with graduated frequencies. J. Subst. Abuse. 2000;12:33–49. doi: 10.1016/s0899-3289(00)00039-0. [DOI] [PubMed] [Google Scholar]
  11. LaBrie J, Earlywine M, Lamb T, Shelesky K. Comparing electronic-keypad responses to paper-and-pencil questionnaires in group assessments of alcohol consumption and related attitudes. Addict. Behav. 2006;31:2334–2338. doi: 10.1016/j.addbeh.2006.03.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Ludbrook J. Statistical techniques for comparing measures and methods of measurement: A critical review. Clin. Exp. Pharmacol. Physiol. 2002;29:526–536. doi: 10.1046/j.1440-1681.2002.03686.x. [DOI] [PubMed] [Google Scholar]
  13. Maisto SA, Conigliaro JC, Gordon AJ, McGinnis KA, Justice AC. An experimental study of the agreement of self-administration and telephone administration of the Timeline Followback interview. J. Stud. Alcohol Drugs. 2008;69:468–471. doi: 10.15288/jsad.2008.69.468. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Searles JS, Helzer JE, Walter DE. Comparison of drinking patterns measured by daily reports and Timeline Follow Back. Psychol. Addict. Behav. 2000;14:277–286. doi: 10.1037//0893-164x.14.3.277. [DOI] [PubMed] [Google Scholar]
  15. Singer JD. Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. J. Educ. Behav. Stat. 1998;24:323–355. [Google Scholar]
  16. Sobell LC, Brown J, Leo GI, Sobell MB. The reliability of the Alcohol Timeline Followback when administered by telephone and by computer. Drug Alcohol Depend. 1996;42:49–54. doi: 10.1016/0376-8716(96)01263-x. [DOI] [PubMed] [Google Scholar]
  17. Sobell MB, Borgardis J, Shuller R, Leo GI, Sobell LC. Is self-monitoring of alcohol consumption reactive? Behav. Assess. 1989;11:447–458. [Google Scholar]
  18. Sobell MB, Sobell LC, Klajner F, Pavan D, Basian E. The reliability of a timeline method for assessing normal drinker college students' recent drinking history: Utility for alcohol research. Addict. Behav. 1986;11:149–161. doi: 10.1016/0306-4603(86)90040-7. [DOI] [PubMed] [Google Scholar]
  19. Substance Abuse and Mental Health Services Administration (Office of Applied Studies) Overview of Findings From the 2003 National Survey on Drug Use and Health, DHHS Publication No. 04-3963. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2004. [Google Scholar]
  20. Task Force of the National Advisory Council on Alcohol Abuse and Alcoholism. A Call to Action: Changing the Culture of Drinking at U.S. Colleges, NIH Publication No. 02-5010. Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism; 2002. [Google Scholar]
  21. van Griensven F, Naorat S, Kilmarx PH, Jeeyapant S, Manopaiboon C, Chaikummao S, Jenkins RA, Uthaivoravit W, Wasinrapee P, Mock PA, Tappero JW. Palmtop-assisted self-interviewing for the collection of sensitive behavioral data: Randomized trial with drug use urine testing. Amer. J. Epidemiol. 2006;163:271–278. doi: 10.1093/aje/kwj038. [DOI] [PubMed] [Google Scholar]
  22. Wechsler H, Lee JE, Kuo M, Lee H. College binge drinking in the 1990s: A continuing problem. J. Amer. Coll. Hlth. 2000;48:199–210. doi: 10.1080/07448480009599305. [DOI] [PubMed] [Google Scholar]
  23. Weitzel JA, Bernhardt JM, Usdan S, Mays D, Glanz K. Using wireless handheld computers and tailored text messaging to reduce the negative consequences of drinking alcohol. J. Stud. Alcohol Drugs. 2007;68:534–537. doi: 10.15288/jsad.2007.68.534. [DOI] [PubMed] [Google Scholar]

Articles from Journal of Studies on Alcohol and Drugs are provided here courtesy of Rutgers University. Center of Alcohol Studies

RESOURCES