Optimizing the US-AUDIT for Alcohol Screening in U.S. College Students

Abstract

This study examined cutoff scores on the new (2014) US-AUDIT (Alcohol Use Disorders Identification Test), adapted for U.S. standard drinks. No studies have examined optimal cutoff scores on the US-AUDIT for college students. 250 undergraduates (65% men) completed the US-AUDIT. At-risk drinkers reported at least four binge drinking episodes per week. Likely alcohol use disorder was assessed with a self-report diagnostic measure. Using the Youden method, the ideal cutoff to identify at-risk drinkers for the US-AUDIT was 5 for men (sensitivity = .93, specificity = .96) and 6 for women (sensitivity = .77, specificity = .86); and to identify likely alcohol use disorder was 13 for men (sensitivity = .69, specificity = .81) and 8 for women (sensitivity = .83, specificity = .80). Cutoffs were lower than the original AUDIT. Different US-AUDIT cutoffs for men and women should be used for likely alcohol use disorder, which may reflect differences in drinking quantity and frequency. Empirical guidelines for alcohol screening with the new US-AUDIT may be used to enhance research or identification of at-risk drinkers in college settings, or for college students in primary care or other health care settings.

Introduction

Risky drinking is a serious public health problem in college. Estimates for 2014 (Substance Abuse and Mental Health Services Administration [SAMHSA], 2016) show adults 18-22 years old have higher rates of binge drinking (5 standard drinks/occasion for men, 4 for women; Wechsler & Austin, 1998) than any other group, and college students have higher rates of binge drinking (38%) than non-college, same-age peers (33%). Alcohol use disorder rates during this life stage are about double other stages, 13% vs. 6% (SAMHSA, 2016). Binge drinking increases risks for many adverse consequences, including blackouts, physical injuries, and unprotected sex (American College Health Association, 2011). Further, about 1800 people in this age group die annually from alcohol (White & Hingson, 2013). Estimates show underage drinking costs, including car accidents, violence, crimes, risky sex, treatment, and other costs, as much as $62 billion/year (Miller et al., 2006).

Students with the highest risk for alcohol-related problems are often not identified and treated. Screening and brief interventions for alcohol consumption have been recommended in multiple healthcare settings such as primary care facilities, prenatal care settings, emergency departments, and criminal justice systems (e.g., National Institute on Alcohol Abuse and Alcoholism [NIAAA], 2009; Siegers & Carey, 2010). Though similar recommendations have been made for college health centers, recent estimates showed that about a third (32%) of four-year colleges and university health centers routinely screened students for alcohol use (Foote et al., 2004; Siegers & Carey, 2010). In addition, of those university health centers who do routinely screen, only 12% report using a standardized instrument (Foote et al., 2004; Siegers & Carey, 2010). Less than a third (28-30%) of college students with an alcohol use disorder received any treatment (SAMHSA, 2016). Low treatment rates may be partly due to inefficiencies in screening practices in college settings with high rates of alcohol use.

The Alcohol Use Disorders Identification Test (AUDIT; Babor et al., 2001) is a widely-used brief alcohol screening measure. Recently, the AUDIT was revised for the U.S.—this new version is called the US-AUDIT (Centers for Disease Control [CDC], 2014; Babor et al., 2016). This new version changed the wording of some items for the U.S. population, e.g., using five drinks (for men) and four (for women) to correspond with the definition of binge drinking using standard drink sizes (14g vs. 10g used in original the international version). in the U.S. (Wechsler et al., 2002; CDC, 2014). The response scale of the first three items was increased from five responses to seven responses. As with the original AUDIT, the first three items can be used as a briefer screening, the US-AUDIT-C.

Research on the original AUDIT and/or AUDIT-C had mixed findings regarding cutoff points for college students. Kokotailo and colleagues (2004) recommended a cutoff of six (sensitivity = .91, specificity = .60) for high-risk drinking defined as four or more instances of binge drinking (Wechsler et al., 2002; CDC, 2014) in a 28-day period. Results showed that a cutoff of nine would maximize sensitivity and specificity for alcohol use disorder (Kokotailo et al., 2004). Adewuya and colleagues (2005) reported that a cutoff of five maximized sensitivity (.94) and specificity (.92) for hazardous use defined as drinking more than 280g (men) or 168g (women) in a week (World Health Organization [WHO], 2000); a cutoff of seven for harmful use defined by the ICD-10 (WHO, 1992); and a cutoff of nine for ICD-10 alcohol dependence (WHO, 1992); DeMartini and Carey (2012) examined cutoffs to identify at-risk drinkers, defined as at least four binge drinking (Wechsler et al., 2002; CDC, 2014) episodes in a month, separately for men and women. For women, a cutoff of five maximized sensitivity (.82) and specificity (.82) on the AUDIT-C, and a cutoff of seven (sensitivity = .80, specificity = .74) on the AUDIT. For men, a cutoff of seven maximized sensitivity (.80) and specificity (.88) on the AUDIT-C, and a cutoff of eight (sensitivity = .89, specificity = .73) on the AUDIT.

Recent work (Babor et al., 2016) recommended the use of the same cutoff scores for the new US-AUDIT as the original AUDIT (Babor et al., 2001), but there have been no studies that assessed the optimal US-AUDIT cutoff (for sensitivity, specificity) to identify college students with 1) at-risk drinking or 2) likely alcohol use disorder. This study will compare multiple potential cutoffs to identify two optimal cutoff points for college-enrolled men and women on the US-AUDIT and the US-AUDIT-C.

Materials and Methods

Participants and Design

Undergraduate students (N = 250) age 18 years or older were recruited from a recreational facility at a private Southeastern university on nine consecutive Monday evenings during the Fall semester of 2014. Students completed a brief (M = 23 min, SD = 6.34) battery of questionnaires using a tablet computer. Compensation was $20 cash for participation and entry in a raffle for a tablet computer or the cash equivalent (about $200). All study procedures were approved by the University of Miami IRB. Students were, on average, 19.58 (SD = 1.89) years old; 162 (65%) were men; 124 (50%) identified as non-Hispanic White, 55 (22%) as Hispanic, 22 (9%) as Black/African American, 47 (19%) as Asian/Pacific Islander/other, and 2 (1%) did not identify as any ethnic group. Most (101; 40%) were 1^st year students, 44 (18%) 2^nd year, 54 (22%) 3^rd year, 45 (18%) 4^th year.

Measures

US-AUDIT

The US-AUDIT has ten items, three questions on alcohol consumption, and seven questions on alcohol-related harm and alcohol use disorder symptoms (CDC, 2014; Babor et al., 2016). This measure typically can be completed in two to three minutes. For adults, including young adults, the recommended cutoffs are eight or more for the 10-item US-AUDIT, and seven for women/eight for men with the US-AUDIT-C (the first three consumption items). In this sample, the 10-item US-AUDIT and US-AUDIT-C had acceptable internal consistency, Cronbach’s α = .76 and .82, respectively. On average, students in this sample completed the 10-item US-AUDIT in about sixty seconds (M = 62.86s, SD = 27.43).

At-risk Drinking

was defined as reporting at least four binge drinking episodes per week. Binge drinking episodes were defined as having four or more standard drinks for women, or five or more standard drinks for men (NIAAA, 2009; Wechsler et al., 2002; CDC, 2014). We chose to use this definition for at-risk drinking because it was similar to past research on the AUDIT (e.g., DeMartini & Carey, 2012). Using this definition, 37% of students (43% of men, 25% of women) in this sample were at-risk drinkers.

Likely Alcohol Use Disorder

was assessed with eleven self-report questions mirroring DSM-5 symptoms of alcohol use disorder over a 12-month period; likely alcohol use disorder was defined as at least two positive responses. Using this measure, 50% of students in this sample had a likely alcohol use disorder (52% of men, 45% of women). This approach has been used in previous survey research when a clinical diagnostic interview is not feasible (e.g., Clements, 1998; Kokotailo et al., 2004; Hagman et al., 2014). In the current sample, this scale had acceptable internal consistency, Cronbach’s α = .72.

Analysis Plan

We used Receiver Operating Curve (ROC) analysis to estimate the discrimination of the US-AUDIT and the US-AUDIT-C for identifying students with (a) at-risk drinking and (b) likely alcohol use disorder. We examined each scale in the full sample, and then separately for men and women. ROC analysis plots the sensitivity against (1-specificity). For each ROC analysis, we estimated the area under the curve (AUC) with a 95% confidence interval (CI). When the CI for the AUC excludes 0.50, the discrimination is significantly different from chance. We then tested for the equality of the AUC across gender using the method proposed by DeLong et al. (1988). Youden’s Index J (Youden, 1950), which maximizes sensitivity and specificity using the formula J = sensitivity + specificity − 1, to determine ideal cutoff scores for the US-AUDIT and the US-US-AUDIT-C. Ideal cutoffs were identified separately for each outcome in the full sample, and by gender. For each cutoff point, we calculated sensitivity as the proportion of participants with a likely alcohol use disorder or reported binge drinking who had positive scores on the US-AUDIT or US-AUDIT-C. Specificity was defined as the proportion of participants who did not report binge drinking or have a likely alcohol disorder with negative scores on the US-AUDIT or US-AUDIT-C. All analyses used R 3.3.2.

Results

At-risk Drinking

Discrimination

Using at-risk drinking as the criteria for the US-AUDIT, the AUC in the entire sample was 0.96 (95% CI: 0.93, 0.98); in men, AUC = 0.98 (95% CI: 0.97, 1.00); and in women, AUC = 0.89 (95% CI: 0.83, 0.96). AUC values were significantly different for men and women (p = 0.01). For the US-AUDIT-C in the entire sample, AUC = 0.96 (95% CI: 0.94, 0.99); for men, AUC = 0.98 (95% CI: 0.96, 1.00); and for women, AUC = 0.92 (95% CI: 0.87, 0.98). AUC values differed for men and women, but these differences were not statistically significant (p = .06). Figure 1 shows the ROC for at-risk drinking on the US-AUDIT and US-AUDIT-C.

Figure 1.

ROC curves with at-risk drinkers on the US-AUDIT and US-AUDIT-C for women (solid lines) and men (dashed lines).

Ideal Cutoff Values

Table 1 shows sensitivity and specificity estimates for at-risk drinkers at potential cutoff scores of the US-AUDIT and the US-AUDIT-C separately in 1) the entire sample, 2) men, and 3) women. The US-AUDIT cutoff score yielding the best discrimination for classifying at-risk drinkers in the entire sample was six (sensitivity = 88%; specificity = 91%, J = .79). For men, the ideal US-AUDIT cutoff score was five, with sensitivity = 93% and specificity = 96%, J = .89. For women, the ideal US-AUDIT cutoff score was six, with sensitivity = 77% and specificity = 86%, J = .63. The US-AUDIT-C cutoff score with the best discrimination for classifying at-risk drinking in the entire sample was four, with sensitivity = 93% and specificity 89%, J = .96. For men, the ideal US-AUDIT-C score was four, with sensitivity = 96% and specificity = 96%, J = .88. For women the ideal US-AUDIT-C cutoff score was four, with sensitivity = 88% and specificity = 83%, J = .71.

Table 1.

Sensitivity, specificity, and Youden’s Index (J) for at-risk drinking using the US-AUDIT and US-AUDIT-C in the total sample, men and women.

	Total Sample			Men			Women
Score	Sensitivity	Specificity	J	Sensitivity	Specificity	J	Sensitivity	Specificity	J
US-AUDIT
4	.94	.78	.72	.96	.88	.84	.89	.68	.57
5	.90	.85	.75	.93	.96	.89	.82	.75	.57
6	.88	.91	.79	.81	.98	.79	.77	.86	.63
7	.84	.94	.78	.73	1.00	.73	.71	.89	.61
8	.79	.96	.75	.64	1.00	.64	.66	.93	.59
9	.74	1.00	.74	.55	1.00	.55	.61	1.00	.61
10	.66	1.00	.66	.45	1.00	.45	.54	1.00	.54
11	.60	1.00	.60	.31	1.00	.31	.46	1.00	.46
12	.54	1.00	.54	.24	1.00	.24	.41	1.00	.41
13	.47	1.00	.47	.14	1.00	.14	.32	1.00	.32
14	.42	1.00	.42	.10	1.00	.10	.29	1.00	.29
US-AUDIT-C
3	.95	.78	.73	.96	.88	.85	.93	.68	.61
4	.93	.89	.82	.96	.96	.92	.88	.83	.71
5	.87	.94	.81	.92	.96	.88	.75	.93	.68
6	.81	.98	.79	.86	.96	.82	.68	1.00	.68
7	.73	1.00	.73	.78	1.00	.78	.61	1.00	.61
8	.64	1.00	.64	.71	1.00	.71	.46	1.00	.46
9	.55	1.00	.55	.62	1.00	.62	.36	1.00	.36
10	.45	1.00	.45	.51	1.00	.51	.29	1.00	.29
11	.31	1.00	.31	.36	1.00	.36	.16	1.00	.16

Note. Bold shows cutoff determined by J. Higher scores for women could not be estimated due to empty cells in the cross-tabulation.

Likely Alcohol Use Disorder

Discrimination

With likely alcohol use disorder as the criteria, for the US-AUDIT across the entire sample, AUC = 0.80 (95% CI: 0.75, 0.86); in men, AUC = 0.79 (95% CI: 0.71, 0.86); and in women, AUC = 0.85 (95% CI: 0.77, 0.94). AUC values differed for men and women, but these differences were not statistically significant, p = .24. For the US-AUDIT-C in the entire sample, AUC = 0.75 (95% CI: 0.68, 0.81); for men, AUC = 0.72 (95% CI: 0.64, 0.80); and for women, AUC = 0.83 (95% CI: 0.73, 0.93). AUC values differed for men and women, but these differences were not statistically significant, p = .10. Figure 2 shows the ROC for at-risk drinking on the US-AUDIT and US-AUDIT-C.

Figure 2.

ROC curves with likely alcohol use disorder on the US-AUDIT and US-AUDIT-C for women (solid lines) and men (dashed lines).

Ideal Cutoff Values

Table 2 shows sensitivity and specificity estimates for likely alcohol use disorder at potential cutoff scores of the US-AUDIT and the US-AUDIT-C in 1) the entire sample, 2) men, and 3) women. The US-AUDIT cutoff score yielding the best discrimination for classifying likely alcohol use disorder in the entire sample was 13. The cutoff of 13 had sensitivity = 61% and specificity = 86%, J = .14. For men, the ideal cutoff score was 13, with sensitivity = 69% and specificity = 81%, J = .50. For women, the ideal cutoff score was eight, with sensitivity = 83% and specificity = 80%, J = .63. The US-AUDIT-C cutoff score with the best discrimination for classifying likely alcohol use disorder in the entire sample was seven, with sensitivity = 79% and specificity =57%, J = .36. For men, the ideal score was ten, with sensitivity = 61% and specificity = 71%, J = .32. For women the ideal cutoff score was five, with sensitivity = 88% and specificity = 71%, J = .59.

Table 2.

Sensitivity, Specificity, and Youden’s Index (J) for the US-AUDIT and the US-AUDIT-C in the total sample, men, and women for Likely Alcohol Use Disorder.

	Total Sample			Men			Women
Score	Sensitivity	Specificity	J	Sensitivity	Specificity	J	Sensitivity	Specificity	J
US-AUDIT
4	.96	.32	.28	.99	.21	.20	.95	.37	.32
5	.95	.35	.30	.96	.25	.21	.93	.51	.44
6	.94	.41	.35	.95	.27	.22	.90	.66	.56
7	.90	.48	.38	.93	.35	.28	.85	.71	.56
8	.87	.53	.40	.88	.38	.26	.83	.80	.63
9	.80	.57	.37	.85	.44	.29	.70	.80	.50
10	.76	.68	.44	.82	.59	.41	.65	.88	.53
11	.70	.71	.41	.76	.63	.39	.55	.86	.41
12	.65	.77	.42	.71	.70	.41	.50	.89	.39
13	.61	.86	.47	.69	.81	.50	.43	.94	.37
14	.55	.88	.43	.61	.83	.44	.40	.97	.37
US-AUDIT-C
3	.98	.24	.22	.99	.19	.18	.98	.34	.32
4	.96	.32	.28	.98	.22	.20	.93	.49	.42
5	.93	.43	.36	.95	.27	.22	.88	.71	.59
6	.87	.49	.36	.90	.33	.23	.78	.77	.55
7	.79	.57	.36	.82	.43	.25	.73	.83	.56
8	.69	.61	.30	.75	.48	.23	.55	.86	.41
9	.61	.69	.30	.70	.60	.31	.40	.86	.26
10	.53	.80	.33	.61	.71	.32	.35	.94	.29
11	.36	.86	.22	.44	.81	.25	.18	.94	.12

Note. Bold shows cutoff determined by J. Higher scores for women could not be estimated due to empty cells in the cross-tabulation.

Discussion

This study reports the first examination of ideal cutoffs, sensitivity, and specificity to identify at-risk drinking and likely alcohol use disorder on the US-AUDIT, and the brief US-AUDIT-C, in a sample of college undergraduates. Youden’s J, which maximizes sensitivity and specificity, was used to determine the optimum cutoff scores for each criterion, at-risk drinking and likely alcohol use disorder, separately for the US-AUDIT and US-AUDIT-C. The ideal cutoff on the US-AUDIT to identify at-risk drinkers was five for men and six for women. These cutoffs had a sensitivity of .93 for men and .77 for women, and specificity of .96 for men and .86 for women. The ideal cutoff to on the US-AUDIT for likely alcohol use disorder was thirteen in men (sensitivity = .69, specificity = .81) and eight in women (sensitivity = .83, specificity = .80). These cutoffs had a sensitivity of .69 for men and .83 for women, and specificity of .81 for men and .81 for women. The ideal cutoff on the three-item US-AUDIT-C to identify at-risk drinkers was four for men and women. This cutoff had a sensitivity of .93 and specificity of .89. The ideal cutoff to on the US-AUDIT-C for likely alcohol use disorder was ten for men and five for women. These cutoffs had a sensitivity of .61 for men and .88 for women, and specificity of .71 for men and for women.

These empirically-based guidelines for cutoffs on the new US-AUDIT and US-AUDIT-C may improve research and clinical identification in college settings, or with college students in primary care or other health care settings. Professionals in the field can use this empirical evidence to make decisions about ideal cutoffs on the US-AUDIT, similar to how the results of empirical studies (Kokotailo et al, 2004; Adewuya, 2005; DeMartini & Carey, 2012) of cutoffs with the original AUDIT have been used in the past. We recommend that gender-specific cutoffs be used for at-risk drinking and likely alcohol use disorder on the US-AUDIT, which is similar to recommendations for the original AUDIT with college students (Rienert & Allen, 2007) although not recommendations for general use (Babor et al., 2001; Babor et al., 2016). With the US-AUDIT-C, we recommend using gender-based cutoff for likely alcohol use disorder, but not for at-risk drinking. The differences in these cutoffs may reflect differences in drinking quantity and frequency, or culturally-rooted patterns of acceptable drinking behavior. Further examination of the factors that influence cutoffs or drinking behavior that may differ for men and women is warranted.

The sensitivity and specificity of the US-AUDIT and US-AUDIT-C are in the same range (i.e., for at-risk drinking, sensitivity .80 - .94; specificity .60 - .92) as work with college students that examined the original AUDIT (Kokotailo et al, 2004; Adewuya et al., 2005; DeMartini & Carey, 2012). It is important to note that Youden’s (1950) method for identifying ideal cutoffs is designed to maximize both sensitivity and specificity, so does not have a preference for either. Generally, increasing the cutoff score reduces specificity, but increases sensitivity. Decreasing the cutoff has the opposite effect. In clinical practice, providers may not place equal importance on sensitivity and specificity, depending on the meaning of a false positive or false negative. Given the high rates of drinking on campus, a college health center may wish to reduce the number of false positives, i.e., increase specificity, during in-person screening to identify only those with highest risk to maximize efficient use of resources. However, if identification of at-risk drinkers requires less intensive resources, as with online-based services, then false positives may be less concerning. If reducing the false positive rate is important, then users of the US-AUDIT may want to maximize specificity.

This study has several limitations to consider. The current study was not designed to explain differences in cutoffs, but these differences are likely related to changes in items, i.e., binge drinking using U.S. standard drink sizes and a larger response range for the first three questions. The sample was not randomly selected. However, sensitivity and specificity are usually understood as properties of the measure not the sample, suggesting that cutoffs are not strongly biased by sample characteristics. Although the self-report analog measure of alcohol use disorder has been widely used in past research, it is not as rigorous as a clinician interview to diagnose an alcohol use disorder. A diagnostic interview may have resulted in fewer students identified as having a disorder, but it is difficult to predict how having fewer students would influence ideal cutoffs, with changes in the denominator for both sensitivity and specificity. We recommend that follow-up studies use interview-based diagnostic measures of alcohol use disorder, as well as other criteria such as alcohol-related consequences. Past work with the AUDIT used DSM-IV/ICD 10 criteria for alcohol dependence, which required a great number of symptoms than the DSM-5, which may explain differences in cutoff scores. The ideal cutoff scores for men and women that were identified empirically with college students are lower than the cutoffs of eight (zone II: at-risk drinking),16 (zone III: harmful use use), 25 (zone IV: dependent use) for the AUDIT suggested by Babor and colleagues (2016). Based on these results, using the higher cutoff scores would likely decrease specificity and increase sensitivity of both 10-item and 3-items screens. Findings from our study only apply to college undergraduates, so this data is not able to determine cutoffs for other adults. We suggest further empirical research with samples of adults from a wider age-range, as well as non-college young adults.

In summary, these findings expand on past examinations of the original AUDIT with college student samples. The re-design for standard U.S. drink sizes is comparable for typical definitions of binge drinking or heavy episodic drinking, which may be useful in college student samples. Results suggest that the US-AUDIT and US-AUDIT-C have acceptable performance for identifying at-risk drinking and likely alcohol use disorders of college students. Findings have two important implications for potential users of the new US-AUDIT with college students. First, different cutoff scores should be used for men and women, with the exception of the US-AUDIT-C for at-risk drinking. Second, the optimal cutoff scores that were identified are lower than those recommended for US-AUDIT in the general adult population. Further research is needed to determine the most efficient ways to use the US-AUDIT-C or the US-AUDIT in more research and/or clinical practice on college and university campuses in the U.S.