Comparing Two Alcohol Screening Measures on Rates of Risky Alcohol Use in a University Health Clinic

Abstract

College drinking is a serious health concern. Few studies have examined screening measures and methods of administration. This study compares two alcohol screens (NIAAA 5/4 binge drinking question or Alcohol Use Disorders Identification Test (AUDIT)) in a college student health clinic waiting room and two modes of administration (self-administered either on a computer kiosk or on a tablet computer). Participants were 259 undergraduates from the University of Miami. Most (78–98%) students completed screening. More students were identified with risky alcohol use with the 5/4 (49%) than AUDIT (14%). On the 5/4, administration method was not linked to completion, 93% kiosk vs. 95% tablet, p = .554, but was related to identification as a risky alcohol user, 42% kiosk vs. 56% tablet, p = .033. On the AUDIT, administration method was significantly related to completion, 73% kiosk vs. 98% tablet, p < .001, and identification, 8% kiosk vs. 23% tablet, p = .003. Method of administration of the single item 5/4 binge drinking question was related to the a higher proportion of students identified with risky alcohol use when screened by a computer tablet, but not completion rates; the AUDIT method of administration was related to both completion and identification rates (higher rates with the tablet in both cases). Education of student health providers who make decisions about what screening tools to use in their centers and who interpret the results of alcohol screening in college health centers should consider the potential influence of administration method. Future research should examine the reasons that method of administration might influence screening results.

College student health centers, the primary care providers for college students, present a unique opportunity to reduce alcohol use and/or prevent more serious consequences. College students feel that health center staff is a highly believable source of health information (American College Health Association [ACHA] 2011). Almost half of college students (49% private, 43% public) utilize student health centers (McBride et al. 2010). However, providers in college student health centers, as in most primary care settings, often lack the time and training to screen patients for alcohol and/or to make referrals for alcohol use disorders (e.g., Miller et al. 2006; Saitz et al. 2003), even though college students are more likely to drink, binge drink, and drink heavily than their non-college peers. Estimates show that 39% of young adults in college binge drink (Substance Abuse and Mental Health Services Administration [SAMHSA] 2013). A significant proportion (14–15%) have alcohol use disorders (SAMHSA 2013), which peak between 18 and 24 years of age (SAMHSA 2010). However, few college students with alcohol use disorders receive treatment (SAMHSA 2010). Alcohol has been linked to many serious problems, including physical injuries (11%), unprotected sex (9%), and blackouts (21%) (ACHA 2011).

Previous interventions with students have screened for alcohol in the emergency room (e.g., Cunningham et al. 2009) or judicial system (e.g., Borsari et al. 2012). Screening for alcohol in primary care has the potential to identify risky drinkers (i.e., heavy and/or binge drinkers) and prevent serious physical injury and/or legal problems by intervening before students are in the emergency room or have been arrested. There is substantial evidence that brief alcohol screening and interventions in primary care are effective for reducing alcohol frequency and are efficient in terms of costs (e.g., Bertholet et al. 2005; Fleming et al., 1997; Grossberg, Brown, & Fleming, 2004; Kaner et al. 2018; Solberg et al. 2008). As such, the US Preventive Services Task Force (Curry et al. 2018) recommended that screening for alcohol should be part of routine, primary care for adults.

Despite this recommendation, few student health centers have evidence-based alcohol screening (Winters et al. 2011; Lenk et al. 2012). In the USA, estimates show between 44 and 50% of college health centers screened for alcohol problems using a standardized, evidence-based instrument (Winters et al. 2011; Lenk et al. 2012), and as few as 24% at 2-year colleges (Lenk et al. 2015). When sites do screen, less than half use recommended screeners for this population, e.g., the Alcohol Use Disorders Identification Test (AUDIT), CAGE (Cut down, Annoyed, Guilt, Eye-opener), or College Alcohol Problems Scale (CAPS) (Winters et al. 2011), and small proportions (e.g., 22%) of students have reported having been screened in some surveys (Angelini et al. 2017). Even fewer include alcohol use disorder prevention interventions, though evidence supports the effectiveness of brief screening and/or feedback interventions (e.g., Babor et al. 2007); Seigers and Carey 2010. Most institutions use alcohol screening after a student has some type of incident related to alcohol use, which is often too late to have the most benefit to students and the larger community (Lenk et al. 2012).

Several alcohol screening measures have been examined with college students. One widely used measure is a single-item binge drinking assessment, the 5/4 question (Wechsler et al. 1994). This measure was designed to identify when an average person has a blood alcohol concentration = 0.08%, the US legal driving limit. This measure is recommended by the National Institutes of Health/National Institute on Alcohol Abuse and Alcoholism (NIH/NIAAA) (NIAAA 2005) for use in primary care settings. The 5/4 question has high sensitivity for identifying college students with the potential to experience alcohol-related problems but may over-estimate the numbers of students with high-risk drinking.

Another widely used screening measure is the Alcohol Use Disorders Identification Test (AUDIT; Babor et al. 2001). The AUDIT was developed by the World Health Organization (WHO) as a brief screen to detect problematic use of alcohol in adults (Babor et al. 2001). The NIAAA (2005) also recommends this measure for use in primary care. Reviews suggest that the AUDIT demonstrates sensitivities and specificities that are often better than other screening tools (Reinert and Allen 2007). With college students, DeMartini and Carey (2012) found sensitivity of 0.80 and specific of 0.74 with the AUDIT for at-risk drinking. These results were similar to Kokotailo et al. (2004), who found sensitivity = 0.82 and specificity = 0.78 for the AUDIT in a college student sample. However, compared with the 5/4 measure, this measure requires greater effort and time (about 2–3 min) to administer and score when interviewer administered.

There have been recommendations for computer-based administration of alcohol screening to increase efficiency in settings with limited resources to pay for screening (Schaus et al. 2009). Bowling (2005) identified potential ways that method of administration might influence responses. Cognitive demands vary with presentation (e.g., auditory, oral, visual), e.g., literacy is required with visual text administration. Privacy and anonymity also vary, which may affect the veracity of responses, especially with stigmatized behavior. Traditionally, more alcohol screens were administered by a staff member interview (85%) than were patient self-administered (15%), but self-administration has increased with the availability and affordability of computers (Schaus et al. 2009). Comparisons between computer and paper/pencil administration of alcohol screens showed equivalence (e.g., Bonevski et al. 2010). Although both the AUDIT and 5/4 question have been examined extensively with college students with multiple administration methods, there is little research that has examined how the method of administration (e.g., tablet vs. kiosk, self-administered vs. interview) of computerized screening influences rates of screener completion and of alcohol abuse detection.

The purpose of this study is to report on the rates of screener completion and identification of risky alcohol use of college students. We compared two alcohol screens (5/4 question or AUDIT) in a student health clinic and two modes of administration (self-administered on a computer kiosk in the clinic waiting room at the time of patient registration with the results going to the primary care provider at the student health service visit that day vs. entered by the student on a tablet computer after a request by study staff with the results not going to the primary care provider). Research questions were as follows: Is the screening measure (5/4 question or AUDIT) related to screening results (completion and identification)? Is method of administration (self-administered at kiosk or on tablet) related to screening results (completion and identification)? Does the method of administration change the relationship between measure and results? To our knowledge, this study is the first to test whether both alcohol screening measure and method of administration influence rates of completion, i.e., how students use alcohol screens, and rates of identification of risky alcohol users, i.e., how university health centers use alcohol screens.

Methods

Participants

Undergraduates at least 18 years old who visited the student health clinic at the University of Miami on 17 days (November 2013 to February 2014) were eligible. To be eligible, students also had to be enrolled at the university and have paid their annual student health fee (to use the clinic). Students enrolled for six or more credits pay a health fee, which is similar to insurance and are eligible for care at this university. All students are required to have some form insurance, either university-sponsored or outside. Graduate students were not eligible to participate. As shown in Fig. 1, a convenience sample of 512 students were approached in the clinic waiting room and invited to participate. Of these, 119 (23%) refused to participate, and 134 (26%) were ineligible (e.g., graduate status), with 259 (51%) randomized into one of the two conditions using cluster randomization. No data were collected from non-participants. The two conditions did not differ on rates of refusal to participate, χ² = 0.25, p = .617, or on age, gender, or school year (Table 1). Power analyses during the study planning period indicated that there would need to be more students randomized to the AUDIT first condition because there were potentially more response levels requiring more time with the 10-item AUDIT than the single-item 5/4 question, so 10 days were randomized to the AUDIT first condition, and 7 days to the 5/4 first condition.

Fig. 1.

Number of students in each condition. Clinic days, not individuals, were randomized, so all students who visited the clinic on a particular day were enrolled in the same condition. Power analyses during the study planning period indicated that there would need to be more students randomized to the AUDIT first condition because there were potentially more response levels with the 10-item AUDIT than the single-item 5/4 question

Table 1.

Participant characteristics of college students in a student health service in each condition (receipt of the NIAAA 5/4 screener first vs. receipt of the AUDIT screener first)

Characteristic	5/4 first (n = 105)		AUDIT first (n = 154)		p
	M (SD) or N (%)		M (SD) or N (%)
Age (years)	21.08	(2.96)	20.97	(2.82)	.492
Women	70	(67%)	91	(59%)	.217
Year in college					.562
First	25	(24%)	31	(21%)
Second	14	(13%)	29	(29%)
Third	22	(21%)	37	(24%)
Fourth	28	(27%)	41	(27%)
Other	16	(15%)	19	(12%)

Design

This study is a secondary analysis that reports intake data from a randomized trial comparing an experimental screening brief intervention to standard care (i.e., screening with a single binge drinking question) in a university health clinic. The data from this trial provided an opportunity to examine method of computer-based administration of alcohol screening tools in a university health center. The clinic provides primary care and some specialty services (e.g., women’s health, travel visits, and allergy injections). Most primary care services are free. Providers at the center are primary care nurse practitioners or physicians.

Conditions

The order of the screening instruments and mode of administration varied by condition (either 5/4 first or AUDIT first). In the 5/4 first condition, students who visited the clinic first self-completed the 5/4 question on the kiosk computer in the waiting room as part of the clinic’s check-in registration process. Then, if the student consented to participate in the study, study staff asked the student to complete the AUDIT using a tablet computer in the waiting room. In the AUDIT first condition, students who visited the clinic self-completed the AUDIT on a kiosk computer during the clinic’s intake process. Then, students who consented were asked by study staff to answer the 5/4 question using a tablet computer in the waiting room (Fig. 2) Providers only saw the result from the screen that was self-administered on the kiosk computer because the second screening was for research. The research team had access to all screening results from enrolled participants.

Fig. 2.

Administration of screening measures in each condition. In the 5/4 first condition, the 5/4 measure was self-administered on a kiosk computer, and then the AUDIT was interviewer-administered on a tablet computer. In the AUDIT first condition, the AUDIT was self-administered on a kiosk computer, and then the 5/4 was self-administered-administered on a tablet computer

The health clinic uses the Medicat v10 (medicat.com) Commission for Healthcare Information Technology-certified ambulatory HIPAA-compliant EHR system, which is designed for college health systems. The system allows students to self-complete screening instruments at a computer kiosk during check-in in the waiting room. Screening measures can be adjusted daily. Providers have immediate access to the scores from screens.

Measures

Student Characteristics

Three characteristics were from clinic records: age, gender, and college class.

5/4 Measure

This measure is a single gender-normed question about binge drinking (NIAAA 2005): “On any single occasion during the past 3 months, have you had more than 4 drinks containing alcohol if you’re female, or more than 5 drinks containing alcohol if you’re male?” Students could answer positively or negatively. It takes a minute or less to administer in college settings and may be administered on paper or computer (Centers for Disease Control and Prevention [CDC] 2014). Previous studies have shown high sensitivity (0.80) but lower specificity (0.55) for risky alcohol use in college settings (e.g., Wechsler and Nelson 2006).

AUDIT

This measure has 10 items, eight on a 5-point Likert scale and two on a 3-point Likert scale. Items are summed to a total score. It has been used extensively with adults, including college students (Reinert and Allen 2007). Although the WHO recommended cutoffs at four levels of risky use, this study used three cutoffs recommended by more recent research (McPherson and Fischer 2013): 0–7 no/low risk, 8–19 moderate risk, and 20 or more high risk. Studies with men and women in college have shown high sensitivity (0.80–0.82) and specificity (0.74–0.82) (e.g., DeMartini and Carey 2012). It takes 2 or 3 min to administer in college settings and may be administered on paper or computer (CDC 2014).

Method of Administration

Two methods of administration of the screeners were tested. Students were assigned to complete alcohol screening by self-administration on a clinic kiosk computer at the time of clinic registration (with results going to the provider) and self-administration on a tablet computer following a study staff request (with results not going to the provider). For the kiosk, the students answered questions about alcohol use during self-check-in at a computer near the entrance to the clinic in the waiting room, and results went to the provider (Fig. 2). All students were required to use university credentials to login and then answered several screening questions (including the alcohol screener) and stated the visit purpose. For the tablet computer mode of administration, students who consented to participate in the study were asked by study staff to complete the self-administered alcohol screening using a tablet computer. Qualtrics, an online survey software system, was used to store answers. Answers on the tablet computer were not seen by providers.

Screening Results

There are two results considered in this study: completion and identification. Completion of the alcohol use screen was defined as completing all items. Completion on the 5/4 questions was answering one question. With the AUDIT, students could complete the measure by either answering never to the first question, “How often do you have a drink containing alcohol?” or by answering all 10 items. Identified was defined as having risky alcohol use on the screening instrument, defined as a positive answer (yes) on the 5/4 question, or a score in either the moderate (8–19) or high (20+) risk categories on the AUDIT.

Procedure

Student characteristics (age, gender, and college class) were extracted from the clinic health record by two of the authors. Students could, depending on condition, complete the alcohol screening measures (5/4 binge drinking question, AUDIT) either by self-administration on a clinic kiosk computer at the time of clinic registration or by self-administration on a tablet computer following a study staff request. The primary author combined measures of the method of administration, completion rates, and screening results from the kiosk computer/health clinic records and table computer into a single database for analyses. The University of Miami IRB approved all study procedures. Participants received no compensation for intake assessments but could receive $10–15 for the follow-up study assessments of the clinical trial.

Analysis Plan

Chi-square tests of difference were used to test relationships that compared proportions between groups in a series of three steps. Chi-square was used to test for differences in 2 × 2 tables. In the first step, the number of students who completed a screen and who were identified as risky alcohol users were compared for each screening measure. In the second step, rates of completion rates and identification of risky alcohol users were compared for each method of administration. The third step compared rates of completion and identification of risky alcohol users for each screening measure by method of administration.

Results

Screening Measure

This section compares rates (screener completion and identification of risky alcohol use) for each screening measure regardless of method of administration. There was no statistically significant difference in the proportions of students who completed the AUDIT (83%) compared to the 5/4 question (95%), χ² = 0.06, p = .811. Of the students who completed a screen, there was a significant difference in rate identified as risky alcohol users on the AUDIT (14%) and the 5/4 question (49%), χ² = 17.74, p < .001.

Method of Administration

This section compares rates (screener completion and identification of risky alcohol use) for each method of administration regardless of screening measure. Students were more likely to complete screening with the tablet (97%) compared to the kiosk computer (81%), χ² = 13.07, p < .001. Of students who completed the screening measures, there was a significant difference in rate identified as risky alcohol users with tablet (44%) vs. kiosk administration (26%), χ² = 7.12, p = .007.

Screening Measure and Method of Administration

This section compares rates (screener completion and identification of risky alcohol use) for each screening measure by method of administration. With the 5/4 question, completion rate did not differ by administration method, 95% tablet vs. 93% kiosk, χ² = 0.35, p = .554. Of students who completed the 5/4 question, there was a significant difference in rate identified as having risky alcohol use by method of administration, 56% tablet vs. 42% kiosk, χ² = 3.92, p = .033. On the AUDIT, completion rate did differ significantly by administration method, 98% tablet vs. 73% kiosk, χ² = 25.21, p < .001. Of students who completed the AUDIT, there was a significant difference in rate identified as having risky alcohol use by administration method, 23% tablet vs. 8% kiosk, χ² = 8.59, p = .003.

Discussion

The goal of this study was to examine how students in a university health clinic—a primary care clinic on campus—used two commonly used alcohol screens (AUDIT and 5/4 question), in terms of completion rates and rates of identification of risky alcohol use and how these rates varied by method of administration. In summary, the majority of students completed screening (73–98%) regardless of method of administration or type of screen. Method of administration did not affect completion rates on the 5/4 question (93% kiosk, 95% tablet) but was related to higher completion rates on the AUDIT (73% kiosk, 98% tablet). More students were identified having risky alcohol use by the 5/4 question (49%) than the AUDIT (14%), regardless of the method of administration. It should be noted that these rates of identification refer to all students, not only students who reported any alcohol use. Method of administration was related to the rate of students identified as having risky alcohol use on the 5/4 question (42% kiosk, 56% tablet) and on the AUDIT (8% kiosk, 23% tablet). These results are different than the findings of equivalence of mode of administration between computer and paper/pencil administration (Bonevski et al. 2010).

We can speculate as to why completion of the screen was better, and rates of identification of risky alcohol use were higher, on the tablet following a request from a study staff member with the results not going to the provider, than on a kiosk computer as part of routine clinic registration with the results going to the provider. The kiosk may have been less comfortable because the student was standing while answering questions but sat in the waiting area while using the tablet. There could be a function of “coolness” with a tablet vs. the older desktop-type computers on kiosks. The kiosk self-administration was in the context of the full clinic intake process, so students may have wanted to only complete the reasons for their visit. Another possibility is that because students knew that their providers would not see the results of the screen administered on the tablet, the students may have felt more comfortable revealing information about drinking. That is, students reported higher amounts of drinking in a research context than in a medical context. This explanation is consistent with Bowling’s (2005) suggestion about the possible influence of privacy or anonymity on alcohol screening.

Although it is not possible to disentangle all the possible reasons for higher completion rates and risky alcohol use rates on the tablet computers, we believe that the personal engagement of the study staff with the college student may have been a potent influence on completion. Students had consented to be part of the study after hearing about the reasons for the study and the confidentiality protections before being asked to complete the screen on the tablet. If the personal engagement including discussion of confidentiality protections was a meaningful influence, it implies that a very brief conversation about the rationale and privacy of screening is sufficient to increase screening rates. This also suggests that the provider or other administrator of the screen is a major component of successful alcohol screening, so training is paramount. It could also be that the students were less likely to complete screeners related to sensitive topics like risky alcohol use if they worried about their results being given to their provider in the clinic’s kiosk and that worry was not present for the tablet computer mode of administration.

Limitations

Non-random sampling restricted representativeness of the findings, and generalization of the findings was limited to one student health clinic in a large private university. Data were not collected from non-participants, so we could not examine how similar participants were to the general population. This study did not assess other environmental characteristics that may have influenced results. The lack of a clinician rating of alcohol use disorder and/or outside ratings of alcohol-related consequences prevented conclusions about sensitivity and specificity of either screen. However, it is likely that using a single-item binge drinking question (i.e., the 5/4) as a screening measure leads to higher false positive rates and overidentifies the number of students who would benefit from intervention, based on the lower specificity in past studies. Future research should examine sensitivity and specificity of these screens with multiple administration methods in college health centers and the duration of screening for each screening measure which was not available in this dataset.

Recommendations for Providers

Many student health centers do not provide alcohol screening, or appropriate treatment and referral for alcohol problems (Winters et al. 2011). This is likely due to several factors, including limited time and resources for provider training and implementation. Both screeners were quick to complete via self-administration on a computer (≤1 min, 5/4 question; ~ 5 min, AUDIT). Providers or other decision-makers in clinics considering implementing alcohol screening could use study findings to inform their decisions. The trade-offs between the greater specificity of a longer, more comprehensive assessment (AUDIT) vs. a shorter, less demanding tool (5/4 question) should be considered. Without brief personal engagement, a clinic might see many fewer patients complete the 10-item AUDIT than a single-item screen. Further, anonymous self-administered screening on computers in the waiting room without the results going to the provider at first might increase rates of completion and identification in student health clinics, with a later option of reporting it to their provider (e.g., AUTHOR). A single-item screen appears to identify many more students as needing an intervention, which may be a challenge for clinics to address because of limited provider time.

Recommendations for Researchers

Future research should examine shorter screening measures, e.g., the three-item AUDIT-C, that are brief but may be more specific to alcohol use and thus identify fewer students that are misclassified with risky alcohol use than the 5/4 question. Another option is to use a two-step screening process, following the recommendations of the NIAAA (2005). In the two-step screening process, patients complete between one and three questions, and then those who are identified as having risky alcohol use on a brief screen (e.g., 5/4 question) would be asked to complete a longer screen (e.g., AUDIT). This process identifies those who would benefit from low-intensity intervention and those who require more intense intervention. Either the first or the second screening could be part of a computerized intervention, if provider time is limited in the clinic for conducting the screening (Schaus et al. 2009). We suggest that future studies explore the influence of different methods of administration and measures of alcohol screening to identify the most efficient and effective routes for screening in multiple contexts, including a variety of primary care and health clinics. Future research should also include a larger and broader sample of students, including those attending large and small public colleges and universities, and investigate other factors that could influence screening such as gender, ethnicity, or culture.