Screening for alcohol use disorders in college student drinkers with the AUDIT and the USAUDIT: a receiver operating characteristic curve analysis

Abstract

Background:

The Alcohol Use Disorders Identification Test (AUDIT) and its consumption subscale (AUDIT-C) are international gold standard screeners for identifying at-risk drinkers. Items have been modified to reflect United States low-risk drinking guidelines in the USAUDIT and USAUDIT-C, which also perform well in identifying at-risk drinkers. The USAUDIT may also be used to screen for potential AUD, an important first step to identify individuals needing diagnostic testing and treatment referrals.

Objectives:

The present study sought to evaluate the sensitivity and specificity of each measure in predicting potential AUDs via diagnostic criteria from the Diagnostic and Statistical Manual of Mental Disorders, 5^th edition in a college sample.

Methods:

Participants were 382 college student drinkers (M_age = 20.2, SD = 1.5; 68.7% female) who completed online surveys assessing alcohol use, at-risk drinking, and AUD symptom endorsement. Receiver operating curves provide optimal cutoff scores for each measure in overall, male, and female samples.

Results:

Results indicated the AUDIT and USAUDIT are equally superior in detecting potential AUD in the current sample. Recommended cutoff scores for detecting likely AUD with the USAUDIT are 12 for males (sensitivity = 62.0%, specificity = 86.6%) and 8 for females (sensitivity = 65.3%, specificity = 87.7%).

Conclusions:

Whereas prior work supports the USAUDIT-C in detecting at-risk drinking, the current study supports the AUDIT and USAUDIT in detecting potential AUD. Based on prior work, and in an effort to be consistent with standard US drinking guidelines, we recommend using the USAUDIT in screening and brief interventions across college campuses.

Introduction

Alcohol use during college has been a public health concern for over 30 years given that most college students drink and roughly half engage in heavy episodic drinking (i.e., 4/5 drinks in one sitting for women/men (1)) (2). In fact, traditional age college students (18–25 years) endorse the highest rates of heavy drinking as compared to all other age groups and to their noncollege peers (2). Excessive drinking is particularly concerning given the alarming rates of associated academic, personal, and social consequences (3), the low rates of treatment utilization (~5%) (4), and that roughly 30% of students meet criteria for an alcohol use disorder (AUD) each year (5). Identifying those students who engage in risky drinking patterns (i.e., engaging in excessive drinking, experiencing associated consequences, and/or demonstrating symptoms of AUD) has been a major initiative across college campuses. To this end, evidence-based alcohol screening tools and brief behavioral interventions have proliferated across college campuses. Brief motivational interventions are recommended Tier 1 interventions that effectively reduce alcohol-related harm by, in part, providing personalized normative feedback to students based on their responses to an alcohol-screening tool (6). The Alcohol Use Disorder Identification Test (AUDIT) (7) is the gold standard alcohol screening tool with demonstrated accuracy in detecting risky drinking (8,9).

Originally designed to be a brief, easy-to-understand screener for individuals in healthcare settings, the AUDIT is a 10-item self-report measure of past-year harmful and hazardous alcohol use (7). The AUDIT addresses three conceptual domains: alcohol consumption (three items), alcohol-related problems (four items), and adverse psychological reactions (three items). The research foundation with the AUDIT is strong as it has been validated in several populations, including college students (9,10). Despite its strong psychometric support, there are two primary concerns with how the AUDIT has been validated in the literature. First, countries differ in how they define a standard drink and risky drinking limits, generating concerns about the accuracy of identifying at-risk individuals in the country in which the AUDIT is used (11). Second, prior studies tested the predictive validity of the AUDIT in detecting active AUD, a criterion that was not consistent with the tool’s original purpose (12). In fact, a recent meta-analysis concluded that the AUDIT did not perform well as a screening tool for identifying AUD among women and countries with low AUD prevalence (13). Given these concerns, researchers have recently adapted the AUDIT to clarify its purpose and accuracy of screening risky drinking patterns for individuals in the United States.

The Center for Disease Control and Prevention recently outlined adaptations to the AUDIT to be compliant with the definition of a standard drink (14 grams) and the recommended low-risk drinking limits in the United States (i.e. USAUDIT) (14,15). The gender-specific low-risk drinking limits are defined as no more than four/five drinks in one sitting or seven/14 drinks in one week for women/men (16, 17). Thus, differences between the AUDIT and the USAUDIT revolve around the consumption subscale, such that the response options for these items were changed to reflect a standard US drink and the content of item three was changed to assess gender-specific heavy episodic drinking. Additionally, the purpose of the USAUDIT was clarified such that it screens for individuals along the spectrum of risky drinking patterns (see Table 1), including potential not active AUD (14,15). Given the changes in how AUD is diagnosed in the Diagnostic and Statistical Manual of Mental Disorders, 5^th edition (18), from separate abuse and dependence categories to AUD diagnosis with varying severity levels, researchers have offered guidance to detect an individual’s risk level and its correspondence with possible AUD (see Table 1). Establishing cutoff scores on the USAUDIT to detect potential AUD can help clinicians shape the discussion during a brief intervention in accordance with the individual’s drinking patterns, as well as inform appropriate diagnostic and treatment referrals. Despite similar cutoff score recommendations for low-risk and hazardous drinking on the AUDIT and the USAUDIT in primary care settings (15), there has been minimal research on optimal cutoff scores of the USAUDIT to identify at-risk drinking and potential AUD in college students.

Table 1.

Definitions of risky drinking patterns and guidance for interpreting the USAUDIT.

Risky Drinking Patterns	Definition	Risk Level	Intervention	USAUDIT Score	Potential AUD (DSM-5, ICD-10)
Low Risk	No more than 3/4 drinks on one occasion and no more than 7/14 drinks per week for women/men	Zone 1	Feedback	0 – 6/7 (Women/Men)	None
Hazardous use	Drinking pattern that increases risk for future alcohol problems	Zone II	Feedback/brief intervention	7/8 – 15 (Women/Men)	Mild AUD, hazardous use
Harmful use	Drinking pattern that is affecting physical or mental health	Zone III	Feedback/monitoring/brief outpatient treatment	16 – 24	Moderate AUD, harmful use
Dependent use	Drinking pattern that reflects a reliance on alcohol to function	Zone IV	Referral to evaluation and treatment	25+	Moderate/severe AUD, alcohol dependence

Adapted from Babor, Higgins-Biddle, & Robaina, 2017.

Preliminary work with the AUDIT based on NIAAA drinking standards has focused largely on identifying optimal cutoff scores of at-risk drinking among college men and women. For example, DeMartini and Carey used receiver operating characteristic (ROC) curve analysis to identify optimal cutoff scores for at-risking drinking on the AUDIT and AUDIT-C, and then compared these scores to determine the ideal screener for college students (19). They concluded that the AUDIT-C outperformed the AUDIT, and recommended a cutoff score of 5 for women and 7 for men. Expanding on their work, Madson and colleagues evaluated and compared ROC curve analyses of the AUDIT, AUDIT-C, USAUDIT, and USAUDIT-C (20). They found that the USAUDIT-C was superior in minimizing false negatives across gender and identified 4 as the optimal cutoff score. Whereas McCabe, Brincks, Halstead, Munoz-Rojas, and Falcon also identified 4 as the optimal cutoff score across gender when using the USAUDIT-C, they found the USAUDIT was superior in detecting at-risk drinkers, recommending a cutoff score of 5 for men and 6 for women (21). While the preliminary evidence supports the USAUDIT in screening for at-risk drinking in college students, less is known about using the USAUDIT to detect potential AUD.

To our knowledge, only one prior study has evaluated optimal cutoff scores on the USAUDIT for detecting potential AUD. McCabe and colleagues compared the USAUDIT and USAUDIT-C using ROC analyses and found the USAUDIT was superior at classifying students with potential AUD (21). The authors recommended a cutoff score of 13 for men (sensitivity = .69, specificity = .81, J = .50) and 8 (sensitivity = .83, specificity = .80, J = .63) for women on the USAUDIT and 10 for men (sensitivity = .61, specificity = .71, J = .32) and 5 for women (sensitivity = .88, specificity = .71, J = .59) on the USAUDIT-C (21). While an important first step, it is also important to evaluate how the USAUDIT compares to the AUDIT in detecting potential AUD in college students. Separate studies reveal that the AUDIT and USAUDIT were superior to the consumption subscales in detecting potential AUD and the optimal cutoff scores were similar for detecting potential AUD in college women (21,22). However, cutoff scores differed for college men (21,22). Although the cutoff scores found in these two studies were similar to the recommended scores to detect potential AUD in primary care (15), the riskier drinking patterns of college students (2) suggest the need for additional research comparing the AUDIT and USAUDIT with a college student sample to accurately determine optimal cutoff scores for detecting potential AUD.

Establishing standard cutoff scores for the USAUDIT is critical to its purpose as (a) a screener to identify those students with a potential AUD, (b) a feedback tool to shape brief intervention discussions in accordance with personal drinking patterns, and (c) a helpful reference to inform referrals for further assessment and treatment following brief intervention. Given suggestions that the USAUDIT may be used to screen for potential AUD (14,15), the current study sought to replicate findings by McCabe et al. in a larger sample of college students, as well as compare the sensitivity and specificity of the USAUDIT and AUDIT in detecting potential AUD. As a secondary analysis of data derived by Madson and colleagues (20), the present study conducted ROC curve analyses on the total and consumption subscale of the AUDIT and USAUDIT to identify and compare optimal cutoff scores for college students with a potential AUD using the DSM-5 diagnostic criteria.

Method

Participants and design

Undergraduate students (N = 382) from a public university in the Southern region of the US were recruited using a psychology participant pool and through direct or listserv e-mails. Those who participated in the psychology classes received credit to be used as partial fulfillment of a research requirement while non-psychology participants were entered into a drawing to receive one of 10 ($10) university gift cards. Students completed an online questionnaire using Qualtrics. All study procedures were approved by the University of Southern Mississippi IRB. Students were, on average, 20.2 (SD = 1.5) years old; 248 (64.9%) were White; 108 (28.3%) were African American; 8 (2.1%) were Hispanic; 10 (2.6%) were Asian; 8 (2.1%) were Other Race; 263 (69.2%) were females; 117 (30.8%) were males; 2 (0.5%) did not indicate their gender; 156 (40.8%) were freshmen; 70 (18.3%) were sophomores; 82 (21.5%) were juniors; 73 (19.0%) were seniors; 1 (0.3%) did not indicate their classification.

Measures

DSM-5 alcohol use disorder checklist

An 11-item checklist using DSM-5 (18) criteria for AUD was created to assess the potential and severity of AUD. Participants responded to each criterion with “yes” (1) or “no” (0) as to whether they experienced a symptom or not over the past 12 months. We followed Hagman’s guidelines (23) for using the DSM-5 to classify potential AUD. Participants who endorsed 0–1 symptoms were classified as “No DSM-5 AUD;” participants who endorsed 2+ symptoms were classified as “potential DSM-5 AUD.” Using this measure, 152 students (39.8%) in this sample were classified as potential DSM-5 AUD (32.9% of men, 66.4% of women, 0.7% missing) Table 5. McCabe et al. noted that this approach has been used in previous survey research when a clinical diagnostic interview is not feasible (21). In this sample, this scale had acceptable internal consistency (α = .81). Invariance testing by gender was performed on the DSM-5 AUD checklist and we found support for configural invariance (CLI = .92, TLI = .88, RMSEA = .054 [90% CI: .042, .065]) after covarying three pairs of items (items 2 and 8, items 8 and 9, items 4 and 11). We also found support for metric (Δχ² = 15.208, Δdf = 10, p = .125) and scalar (Δχ² = 16.093, Δdf = 11, p = .138) invariance. Although our test of strict invariance was significant such that covariances and residuals differed by gender, prior researchers note that strict invariance is rarely achieved given it represents a highly constrained model (24).

Table 5.

USAUDIT and USAUDIT-C cutoff score analysis in the overall sample and by gender.

	Overall Sample					Males					Females
Score	Sensitivity	Specificity	J	PPV	NPV	Sensitivity	Specificity	J	PPV	NPV	Sensitivity	Specificity	J	PPV	NPV
USAUDIT
1	1.000	.004	.004	.399	1.000	1.000	.015	.015	.421	1.000
2	.987	.087	.074	.417	.909	.980	.060	.040	.427	.807	.990	.099	.089	.407	.941
3	.974	.196	.170	.445	.918	.980	.134	.114	.447	.904	.970	.222	.192	.437	.922
4	.961	.326	.287	.485	.926	.980	.269	.249	.490	.950	.950	.352	.302	.478	.919
5	.914	.461	.375	.529	.891	.940	.433	.373	.543	.910	.901	.475	.376	.517	.885
6	.829	.565	.394	.558	.833	.940	.507	.447	.577	.922	.772	.586	.358	.538	.805
7	.776	.717	.493	.645	.829	.880	.567	.447	.592	.869	.723	.778	.501	.670	.818
8	.717	.796	.513	.699	.810	.840	.597	.437	.599	.839	.653	.877	.530	.768	.802
9	.645	.839	.484	.726	.781	.740	.687	.427	.628	.787	.594	.901	.495	.789	.781
10	.566	.874	.440	.748	.753	.720	.761	.481	.683	.792	.485	.920	.405	.791	.741
11	.487	.904	.391	.771	.727	.640	.806	.446	.702	.758	.406	.944	.350	.819	.718
12	.428	.935	.363	.813	.712	.620	.866	.486	.768	.761	.327	.963	.290	.846	.697
13	.375	.948	.323	.826	.696	.560	.881	.441	.771	.737	.287	.975	.262	.877	.687
14	.309	.961	.270	.839	.678	.500	.896	.396	.775	.715	.218	.988	.206	.919	.670
15	.283	.978	.261	.896	.674	.440	.955	.395	.875	.705	.208	.988	.196	.915	.667
USAUDIT-C
1	1.000	.009	.009	.400	1.000	1.000	.015	.015	.421	1.000	1.000	.006	.006	.385	1.000
2	.954	.104	.058	.413	.774	.960	.090	.050	.430	.759	.950	.111	.061	.400	.781
3	.914	.243	.157	.444	.810	.920	.194	.114	.449	.772	.911	.265	.176	.436	.827
4	.875	.391	.266	.487	.826	.900	.343	.243	.495	.827	.861	.414	.275	.478	.827
5	.829	.509	.338	.527	.818	.880	.463	.343	.540	.844	.802	.531	.333	.516	.811
6	.697	.678	.375	.589	.772	.860	.567	.427	.587	.850	.614	.722	.336	.579	.750
7	.612	.787	.399	.655	.754	.760	.627	.387	.593	.785	.535	.852	.387	.693	.746
8	.454	.865	.319	.690	.706	.600	.687	.287	.578	.706	.386	.938	.324	.795	.710
9	.342	.917	.259	.731	.678	.480	.806	.286	.639	.684	.277	.963	.240	.824	.681
10	.250	.952	.202	.775	.658	.420	.896	.316	.743	.684	.168	.975	.143	.807	.653

Optimal cutoff scores are shaded in gray and interpreted as “greater than or equal to” a given value.

Alcohol use disorders identification test (AUDIT)

The 10-item AUDIT (7) assessed harmful and hazardous drinking patterns (e.g., “How often do you have six or more drinks on one occasion?”). The AUDIT is the leading instrument for detecting early-phase risky drinking patterns across different cultures and age groups (25–27). In this sample, the AUDIT and AUDIT-C had acceptable internal consistency (α = .80 [AUDIT] and .77]AUDIT-C]).

Alcohol use disorder identification test – US (USAUDIT)

The 10-item USAUDIT is a modified version of the AUDIT with the goal of better reflecting the NIAAA guidelines (16) for low risk drinking (14). The response options for items 1–3 on the USAUDIT were modified while items 4–10 remained the same. Specifically, participants responded on a 7-point (rather than 5-point) scale for the first three items and the wording of item 3 has changed to reflect the US definition of heavy episodic drinking for men and women (i.e. “How often do you have 5 drinks (male) 4 drinks (female) or more on one occasion?”). In this sample, the USAUDIT and USAUDIT-C had acceptable internal consistency (α = .79 [USAUDIT] and .78 [USAUDIT-C]).

At-risk drinking

At-risk drinkers were identified using three questions based on the NIAAA guidelines for safe drinking (16,18). For females, these questions were: “In the past year, how many times have you had (a) 4 or more drinks in 2 hours, (b) 3 or more drinks in a day, and (c) 7 or more drinks in a week? For males, the questions were: “In the past year, how many times have you had (a) 5 or more drinks in 2 hours, (b) 4 or more drinks in a day, and (c) 14 or more drinks in a week?” At-risk dinking was defined as engaging in any of these drinking behaviors at least once. Prior work has demonstrated the most support for the heavy drinking indicator of at-risk drinking (i.e., 4/5 or more drinks in 2 hours for women/men; 28–30) and preliminary support for the other two at-risk indicators (31,32). Given prior concerns with the predictive validity of using a single indicator of at-risk drinking (33), we chose to ask the three risky drinking questions derived by NIAAA and used in the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC; 34). Using this definition, 287 (75.1%) of participants (31.0% of men; 69.0% of women) in this sample were at-risk drinkers.

Analysis plan

We used Receiver Operating Characteristic (ROC) curve analysis to estimate the discrimination of the AUDIT, AUDIT-C, USAUDIT, and USAUDIT-C for identifying students with potential DSM-5 AUD. We examined each version in the full sample, and then separately for males and females. ROC curves plot sensitivity (true positive rate) against 1–specificity (false positive rate) for diagnostic status. For each ROC analysis, we estimated the area under the curve (AUC) with a 95% confidence interval. When the confidence interval excludes 0.50, the discrimination is statistically different from chance. Higher AUC scores indicate better overall classification performance (35). The AUC statistic represented the probability that a person identified as potential DSM-5 AUD has a greater score on a particular scale than a person identified as not having a potential DSM-5 AUD. We compared the ROC curves across screeners using Hanley and McNeil’s z test (36).

An optimal cutoff score ideally considers prevalence in the sample and the costs and benefits of wrong decisions (19). For this study, we made no distinction between the importance of sensitivity and specificity. Youden’s J index (37), which maximizes both sensitivity and specificity, was used to determine the ideal cutoff scores for the various scales. We identified ideal cutoffs separately for each scale in the full sample, and by gender. For each cutoff point, we calculated sensitivity as the proportion of correct classifications among those identified as likely AUD. Specificity was the proportion of correct classifications among those identified as not likely AUD. Positive predictive value (PPV) was the proportion of positive results (likely AUD) that are true positives. Negative predictive value (NPV) was the proportion of negative results (i.e., no potential DSM-5 AUD) that are true negatives. All analyses used SPSS (version 25).

Results

Alcohol use characteristics

On average, the sample reported endorsing 1.71 potential DSM-5 AUD symptoms (SD = 2.25) Table 2. The proportionality of potential DSM-5 AUD classification was not statistically different across gender, χ² = .64, p = .43. Of the 287 participants identified as at-risk drinkers, 152 (53.0%) were classified as no potential DSM-5 AUD, and 135 (47.0%) were classified as potential DSM-5 AUD Table 3. At-risk classification and potential DSM-5 AUD were moderately correlated, r = .27, p < .001. Participants reported consuming an average of 10.29 drinks per week (SD = 11.93) and drinking on 5.30 days in the past month (SD = 4.92). Regarding low-risk drinking standards in the past year, women reported having: 4+ drinks within a two-hour period 6.56 times (SD = 11.43); 3+ drinks in a day 10.85 times (SD = 17.27); and 7+ drinks in a week 8.95 times (SD = 17.71). Men reported having: 5+ drinks within a two-hour period 12.45 times (SD = 20.31); 4 + drinks in a day 16.56 times (SD = 22.71); and 14+ drinks in a week 12.33 times (SD = 23.58).

Table 2.

Descriptive statistics for the DSM 5 Likely AUD, AUDIT, and USAUDIT measures in the overall sample.

Item	Frequency	Mean	SD
DSM1 (Yes)	37.4%
DSM2 (Yes)	9.7%
DSM3 (Yes)	20.9%
DSM4 (Yes)	27.8%
DSM5 (Yes)	11.0%
DSM6 (Yes)	7.6%
DSM7 (Yes)	3.7%
DSM8 (Yes)	9.9%
DSM9 (Yes)	10.7%
DSM10 (Yes)	13.4%
DSM11 (Yes)	19.3%
AUDIT1		1.72	0.79
Never	2.9%
Monthly or less	39.5%
2–4 times/month	42.3%
2–3 times/week	13.8%
4+ times/week	1.6%
USAUDIT1		2.17	1.13
Never	1.0%
Less than monthly	33.0%
Monthly	31.2%
Weekly	20.5%
2–3 times/week	11.7%
4–6 times/week	2.3%
Daily	0.3%
AUDIT2		0.88	0.97
1–2	42.6%
3–4	35.6%
5–6	14.0%
7–9	6.5%
10+	1.3%
USAUDIT2		2.10	1.52
1	15.8%
2	23.1%
3	25.2%
4	16.4%
5–6	11.4%
7–9	6.5%
10+	1.6%
AUDIT3		0.99	1.07
Never	39.5%
Less than monthly	34.8%
Monthly	16.6%
Weekly	5.2%
Daily/almost daily	3.9%
USAUDIT3		1.51	1.29
Never	22.6%
Less than monthly	33.8%
Monthly	25.7%
Weekly	9.6%
2–3 times/week	4.9%
4–6 times/week	2.6%
Daily	0.8%
AUDIT4		0.25	0.63
Never	84.0%
Less than monthly	9.2%
Monthly	5.0%
Weekly	1.8%
Daily/almost daily	0.0%
AUDIT5		0.28	0.60
Never	79.3%
Less than monthly	14.7%
Monthly	5.0%
Weekly	1.0%
Daily/almost daily	0.0%
AUDIT6		0.12	0.45
Never	91.6%
Less than monthly	5.0%
Monthly	2.9%
Weekly	0.5%
Daily/almost daily	0.0%
AUDIT7		0.48	0.76
Never	65.9%
Less than monthly	23.1%
Monthly	8.9%
Weekly	1.8%
Daily/almost daily	0.3%
AUDIT8		0.39	0.70
Never	70.4%
Less than monthly	22.3%
Monthly	6.0%
Weekly	0.5%
Daily/almost daily	0.8%
AUDIT9		0.41	1.07
No	85.3%
Yes, not in last year	8.7%
Yes, during last year	6.0%
AUDIT10		0.20	0.82
No	94.0%
Yes, not in last year	2.1%
Yes, during last year	3.9%

Table 3.

Descriptive statistics for the AUDIT, and USAUDIT measures among those identified as likely AUD vs. non likely AUD.

	Frequency		Mean		SD
Item	Not Likely AUD	Likely AUD	Not Likely AUD	Likely AUD	Not Likely AUD	Likely AUD
AUDIT1			1.47	2.07	0.69	0.79
Never	3.9%	1.3%
Monthly or less	52.2%	20.4%
2–4 times/month	36.5%	51.3%
v2–3 times/week	7.4%	23.7%
4+ times/week	0.0%	3.3%
USAUDIT1			1.84	2.64	0.97	1.15
Never	1.3%	0.7%
Less than monthly	43.5%	17.1%
Monthly	33.0%	28.9%
Weekly	15.2%	28.9%
2–3 times/week	6.1%	19.7%
4–6 times/week	0.9%	3.9%
Daily	0.0%	0.7%
AUDIT2			0.63	1.23	0.82	1.05
1–2	53.5%	27.0%
3–4	33.5%	39.5%
5–6	10.0%	19.1%
7–9	2.2%	12.5%
10+	0.9%	2.0%
USAUDIT2			1.67	2.72	1.37	1.52
1	20.9%	8.6%
2	30.4%	12.5%
3	26.1%	24.3%
4	11.3%	24.3%
5–6	7.8%	15.1%
7–9	2.2%	13.2%
10+	1.3%	2.0%
AUDIT3			0.70	1.40	0.90	1.14
Never	51.3%	22.4%
Less than monthly	33.9%	36.8%
Monthly	10.0%	26.3%
Weekly	3.0%	7.9%
Daily/almost daily	1.7%	6.6%
USAUDIT3			1.14	2.03	1.16	1.27
Never	31.7%	9.2%
Less than monthly	39.6%	25.7%
Monthly	19.1%	36.2%
Weekly	4.8%	15.8%
2–3 times/week	2.6%	8.6%
4–6 times/week	1.3%	3.9%
Daily	0.9%	0.7%
AUDIT4			0.07	0.51	0.35	0.84
Never	95.2%	66.9%
Less than monthly	2.2%	19.9%
Monthly	2.6%	8.6%
Weekly	0.0%	4.6%
Daily/almost daily	0.0%	0.0%
AUDIT5			0.07	0.59	0.29	0.79
Never	93.4%	57.6%
Less than monthly	5.7%	28.5%
Monthly	0.9%	11.3%
Weekly	0.0%	2.6%
Daily/almost daily	0.0%	0.0%
AUDIT6			0.04	0.25	0.24	0.62
Never	96.5%	84.1%
Less than monthly	2.6%	8.6%
Monthly	0.9%	6.0%
Weekly	0.0%	1.3%
Daily/almost daily	0.0%	0.0%
AUDIT7			0.23	0.85	0.47	0.94
Never	79.4%	45.7%
Less than monthly	18.4%	29.8%
Monthly	2.2%	19.2%
Weekly	0.0%	4.6%
Daily/almost daily	0.0%	0.7%
AUDIT8			0.14	0.77	0.37	0.88
Never	86.9%	45.7%
Less than monthly	12.2%	37.1%
Monthly	0.9%	13.9%
Weekly	0.0%	1.3%
Daily/almost daily	0.0%	2.0%
AUDIT9			0.21	0.72	0.77	1.35
No	92.1%	75.5%
Yes, not in last year	5.3%	13.2%
Yes, during last year	2.6%	11.3%
AUDIT10			0.03	0.45	0.37	1.19
No	99.1%	86.1%
Yes, not in last year	0.0%	5.3%
Yes, during last year	0.9%	8.6%

The mean scores across the scales were: AUDIT (5.70, SD = 4.84), USAUDIT (7.89, SD = 5.60), AUDIT–C (3.60, SD = 2.36), USAUDIT–C (5.76, SD = 3.29). For men, mean scores were: AUDIT (7.53, SD = 5.86), USAUDIT (9.91, SD = 6.67), AUDIT–C (4.68, SD = 2.90), USAUDIT–C (7.03, SD = 3.93). For women, mean scores were: AUDIT (4.86, SD = 4.06), USAUDIT (6.97, SD = 6.67), AUDIT–C (4.68, SD = 2.90), USAUDIT–C (7.03, SD = 3.93). Men were significantly higher on all versions than women, Fs (1, 381) = 24.01–40.68, ps < .001.

Discrimination

AUDIT and AUDIT-C

Using potential DSM-5 AUD as the criteria, for the AUDIT in the entire sample, AUC = .820 (95% CI: .777, 864) Figure 1; in males, AUC = .831 (95% CI: .757, .905) Figure 2; and in females, AUC = .820 (95% CI: .767, .873) Figure 3. For the AUDIT-C in the entire sample, AUC = .736 (95% CI: .685, .787); in males, AUC = .742 (95% CI: .652, .833); and in females, AUC = .734 (95% CI: .673, .796). AUC values differed across gender, but these differences were not statistically significant. AUC values also differed between AUDIT and AUDIT-C, and these differences were statistically significant, z = 3.222, p < .001.

Figure 1.

ROC curves of the overall sample with likely AUD on the several AUDIT versions. Points nearest the northwest corner (0,1) of the curve are considered optimal using the J statistic as they maximize sensitivity and minimize 1-specificity. The reference line represents the probability of chance; therefore, curves above the reference line are considered better than chance. The AUC statistic for each curve is calculated as the area between the reference line and the curve itself. Higher AUC statistics indicate better overall tests.

Figure 2.

ROC curves of males with likely AUD on the several AUDIT versions. ROC Curves of the overall sample with likely AUD on the several AUDIT versions. Points nearest the northwest corner (0,1) of the curve are considered optimal using the J statistic as they maximize sensitivity and minimize 1-specificity. The reference line represents the probability of chance; therefore, curves above the reference line are considered better than chance. The AUC statistic for each curve is calculated as the area between the reference line and the curve itself. Higher AUC statistics indicate better overall tests.

Figure 3.

ROC curves of females with likely AUD on the several AUDIT versions. ROC Curves of the overall sample with likely AUD on the several AUDIT versions. Points nearest the northwest corner (0,1) of the curve are considered optimal using the J statistic as they maximize sensitivity and minimize 1-specificity. The reference line represents the probability of chance; therefore, curves above the reference line are considered better than chance. The AUC statistic for each curve is calculated as the area between the reference line and the curve itself. Higher AUC statistics indicate better overall tests.

USAUDIT and USAUDIT-C

For the USAUDIT in the entire sample, AUC = .817 (95% CI: .773, 860) Figure 1; in males, AUC = .822 (95% CI: .746, .898) Figure 2; and in females, AUC = .814 (95% CI: .760, .868) Figure 3. For the USAUDIT-C in the entire sample, AUC = .744 (95% CI: .692, .795); in males, AUC = .734 (95% CI: .641, .826); and in females, AUC = .745 (95% CI: .682, .808). AUC values differed across gender, but these differences were not statistically significant. AUC values also differed between USAUDIT and USAUDIT-C, and these differences were statistically significant, z = 2.800, p = .003. We also compared the AUC values between both versions (AUDIT vs. USAUDIT) of the full scale and consumption subscales, and those differences were not statistically significant.

Ideal cutoff scores

AUDIT and AUDIT-C

Table 4 shows sensitivity and specificity estimates for potential DSM-5 AUD at potential cutoff scores of the AUDIT and AUDIT-C in the entire sample and by gender. The AUDIT score yielding the best discrimination for classifying potential DSM-5 AUD in the entire sample was 6. The cutoff of 6 had sensitivity = 68.4%, specificity = 82.6%, and J = .51. For males, the ideal cutoff was 7, with sensitivity = 76.0%, specificity = 74.6%, and J = .51. For females, the ideal cutoff was 6, with sensitivity = 62.4%, specificity = 88.3%, and J = .51. The AUDIT-C score with the best discrimination for classifying potential DSM-5 AUD in the entire sample was 3, with sensitivity = 82.2%, specificity = 53.0%, and J = .35. For males, the ideal cutoff was 4, with sensitivity = 80.0%, specificity = 61.2%, and J = .41. For females, the ideal cutoff was 3, with sensitivity = 78.2%, specificity = 58.6%, and J = .37.

Table 4.

AUDIT and AUDIT-C cutoff score analysis in the overall sample and by gender.

	Overall Sample					Males					Females
Score	Sensitivity	Specificity	J	PPV	NPV	Sensitivity	Specificity	J	PPV	NPV	Sensitivity	Specificity	J	PPV	NPV
AUDIT
1	1.000	.017	.017	.402	1.000	1.000	.015	.015	.421	1.000	1.000	.019	.019	.389	1.000
2	.967	.226	.193	.452	.912	.980	.149	.129	.452	.912	.960	.259	.219	.447	.912
3	.934	.422	.356	.517	.906	.980	.299	.279	.500	.954	.911	.475	.386	.520	.895
4	.836	.613	.449	.588	.850	.900	.522	.422	.574	.879	.802	.648	.450	.587	.840
5	.750	.752	.502	.667	.820	.860	.627	.487	.623	.862	.693	.802	.495	.686	.807
6	.684	.826	.510	.722	.798	.800	.687	.487	.646	.828	.624	.883	.507	.769	.790
7	.592	.870	.462	.751	.763	.760	.746	.506	.682	.813	.505	.920	.425	.797	.749
8	.493	.900	.393	.765	.729	.660	.806	.466	.709	.768	.406	.938	.344	.803	.717
9	.441	.922	.363	.789	.714	.620	.836	.456	.730	.755	.347	.957	.304	.834	.702
10	.388	.948	.336	.831	.701	.560	.881	.441	.771	.737	.307	.975	.282	.884	.693
11	.342	.965	.307	.866	.689	.540	.925	.465	.837	.738	.248	.981	.229	.891	.677
12	.270	.978	.248	.890	.670	.420	.940	.360	.834	.694	.198	.994	.192	.954	.665
13	.217	.983	.200	.894	.655	.360	.955	.315	.851	.676	.149	.994	.143	.939	.652
14	.178	.991	.169	.929	.646	.300	.985	.285	.935	.663	.119	.994	.113	.925	.644
15	.151	.991	.142	.917	.638	.280	.985	.265	.930	.657	.089	.994	.083	.902	.636
AUDIT-C
1	1.000	.017	.017	.402	1.000						1.000	.025	.025	.390	1.000
2	.921	.278	.199	.458	.842	.940	.209	.149	.459	.830	.911	.309	.220	.451	.848
3	.822	.530	.352	.536	.818	.900	.403	.303	.519	.849	.782	.586	.368	.541	.812
4	.632	.717	.349	.596	.747	.800	.612	.412	.596	.811	.545	.759	.304	.585	.728
5	.461	.835	.296	.649	.701	.640	.701	.341	.605	.731	.376	.889	.265	.679	.696
6	.342	.913	.255	.722	.677	.520	.791	.311	.640	.697	.257	.963	.220	.812	.675
7	.230	.948	.178	.745	.651	.420	.866	.286	.692	.676	.139	.981	.120	.820	.646
8	.125	.974	.099	.761	.627	.260	.910	.170	.674	.632	.059	1.000	.059	1.000	.630
9	.086	.978	.064	.721	.618	.220	.925	.145	.677	.624	.020	1.000	.020	1.000	.621
10	.053	.996	.049	.898	.614	.160	.985	.145	.884	.621

Note: Optimal cutoff scores are shaded in gray and interpreted as “greater than or equal to” a given value.

USAUDIT and USAUDIT-C

Table 5 shows sensitivity and specificity estimates for potential DSM-5 AUD at potential cutoff scores of the USAUDIT and USAUDIT-C in the entire sample and by gender. The USAUDIT score yielding the best discrimination for classifying potential DSM-5 AUD in the entire sample was 8. The cutoff of 8 had sensitivity = 71.7%, specificity = 79.6%, and J = .51. For males, the ideal cutoff was 12, with sensitivity = 62.0%, specificity = 86.6%, and J = .49. For females, the ideal cutoff was 8, with sensitivity = 65.3%, specificity = 87.7%, and J = .53. The USAUDIT-C score with the best discrimination for classifying potential DSM-5 AUD in the entire sample was 7, with sensitivity = 61.2%, specificity = 78.7%, and J = .40. For males, the ideal cutoff was 6, with sensitivity = 86.0%, specificity = 56.7%, and J = .43. For females, the ideal cutoff was 7, with sensitivity = 53.5%, specificity = 85.2%, and J = .39.

Exploratory analyses

Optimal scores across the AUD continuum

An exploratory analysis was performed on three subsamples of the entire dataset to determine optimal AUDIT and USAUDIT scores that correspond to each level of the AUD continuum. The first subsample (n = 88) was chosen to differentiate the “mild” (2–3 endorsements) and “no” levels (0 or 1 endorsements). The second subsample (n = 35) was chosen to differentiate the “mild” and “moderate” levels (4–5 endorsements) among those classified with some form of likely DSM-5 AUD. The third subsample (n = 29) was chosen to differentiate the “severe” level (6 or more endorsements) from the mild or moderate levels (2–5 endorsements) among those classified with some form of likely DSM-5 AUD. The AUC statistics again demonstrated that the AUDIT and USAUDIT full scales outperformed the consumption subscales in classification accuracy. The full scale versions of the AUDIT are the only two versions with interpretable classification accuracy. The AUDIT-C optimal scores go in the opposite direction as expected for increasing severity, while the USAUDIT-C optimal scores are constant across all continuum levels. Table 6 presents the optimal scores of each version of the AUDIT for each level of the AUD continuum.

Table 6.

Optimal scores and classification performance of the AUDIT versions for likely AUD.

Version	Mild	Moderate	Severe
AUDIT	5	8	12
USAUDIT	7	11	14
AUDIT-C	7	6	4
USAUDIT-C	8	8	8

Thresholds are from the entire sample and are not disaggregated by gender.

Discussion

This is the first study, to our knowledge, to examine the diagnostic performance of both versions of the AUDIT and their respective consumption subscales in detecting potential DSM-5 AUD among college students. We report the ideal cutoffs, sensitivity, specificity, J and predictive values to identify likely AUD. Youden’s J (37), which maximizes sensitivity and specificity, was used to determine the optimum cutoff scores for likely AUD separately for each AUDIT version. The overall performance of the AUDIT and USAUDIT scales were high (AUCs > .80). This finding is consistent with prior research, which found high AUCs for the AUDIT (19) and USAUDIT (20) in samples of college students. However, these prior studies examined detection of at-risk drinking among college student samples, not detection of potential AUD based on DSM-5 criteria. This is an important distinction given that prior work supports the use of the AUDIT-C or the USAUDIT-C in detecting at-risk drinking (20), whereas the current study supports the use of the AUDIT or the USAUDIT in detecting potential DSM-5 AUD. Three other studies evaluated optimal cutoff scores to detect likely DSM-5 AUD in college students but only used the original AUDIT (22), AUDIT-C (38), or the USAUDIT and USAUDIT-C (21). Table 7 displays the optimal cutoff scores, sensitivity, specificity, and J statistics of those studies and includes the results from this study.

Table 7.

Optimal scores and classification performance of the AUDIT versions for likely AUD.

Scale	Optimal Score	Sensitivity	Specificity	J	Study
Males
AUDIT	9	.82	.63	.45	Hagman (2016)
	7	.76	.75	.51	*This study
AUDIT-C	5	.59	.83	.42	Hagman (2015)
	4	.80	.61	.41	*This study
USAUDIT	13	.69	.81	.50	McCabe et al. (2019)
	12	.62	.87	.49	*This study
USAUDIT-C	10	.61	.71	.32	McCabe et al. (2019)
	6	.86	.57	.43	*This study
Females
AUDIT	8	.59	.75	.34	Hagman (2016)
	6	.62	.88	.51	*This study
AUDIT-C	3	.73	.67	.40	Hagman (2015)
	3	.78	.59	.37	*This study
USAUDIT	8	.83	.80	.63	McCabe et al. (2019)
	8	.65	.88	.53	*This study
USAUDIT-C	5	.88	.71	.59	McCabe et al. (2019)
	7	.54	.85	.39	*This study

N for each study was McCabe (162 males, 88 females); Hagman 2015 (472 males, 1225 females); Hagman 2016 (133 males, 118 females); This study (120 males, 263 females).

Broadly, our findings suggest the AUDIT and USAUDIT are appropriate to screen for potential AUD, and if indicated, warrants referral for further assessment to confirm an AUD diagnosis. Additionally, offering feedback to students during a brief intervention on how their cutoff scores compare to others can aid in determining perceived severity of drinking patterns and the need for additional behavioral services. Importantly, our finding that the AUDIT is capable of detecting potential AUD is contrary to recent meta-analytic findings (13) that the AUDIT did not perform well as a screening tool for identifying likely AUD among women and countries with low AUD prevalence. In addition to the sample and geographic differences in our study, Lange et al asserted that the poor performance of identifying women with a likely AUD was “likely due to the low prevalence of AUDs among women (compared to men), as the false positive rate is dependent on prevalence” (p. 4) (13). In fact, it is the PPV and NPV that are influenced by prevalence, and not the sensitivity, specificity, and the false positive and negative rates (39). The authors make no mention of the PPV or NPV in their argument. As such, the multiple sources of accuracy used in the current study, in addition to our restriction to college students in the United States, may explain the contradictory findings of the AUDIT.

As observed by McCabe et al. (21), increasing the cutoff score generally reduces sensitivity and increases specificity. At the same token, that reduction in sensitivity typically increases the PPV and decreases the NPV. This has ramifications on error and misclassification, i.e., the false negative and false positive classification rates. In practice, providers may not place equal importance on sensitivity and specificity, depending on the meaning and costs associated with a false positive decision. In this context, a false positive reflects an incorrect diagnosis of potential DSM-5 AUD. Generally, reducing sensitivity (by increasing the cutoff score) also reduces the false positive rate. By extension, a false negative reflects an incorrect diagnosis of not having a likely AUD. Generally, reducing specificity (by decreasing the cutoff score) also reduces the false negative rate. Conceptually, a false negative is perhaps more harmful, as incorrectly identifying an individual as not likely to have a potential DSM-5 AUD may result in no further evaluation and intervention (when such an intervention would have been warranted). If a health center wished to prioritize the correct identification of those with the highest risk of likelihood to have a potential DSM-5 AUD in order to maximize their resources, they should focus on cutoff scores that maximize the positive predictive value (PPV) and typically err on the side of higher cutoff scores. However, if identification is less costly, then PPV might be of lesser concern and a desire to better identify those who likely do not have a potential DSM-5 AUD (i.e., maximize the negative predictive value by erring on the side of lower cutoff scores) to avoid delivering an unnecessary intervention.

Distinct from Hagman (22), we compared the AUDIT and the USAUDIT and found that they performed equally well in detecting DSM-5 AUDs based on overall AUC statistics and J values at the respective optimal cutoff scores. We take this to suggest that either version could be used with accuracy, but we advocate for the USAUDIT for several reasons. First, the original AUDIT was developed as an international screener (7) but there is not a consensus across countries on definitions of risky drinking or a standard drink (40). The NIAAA established clear definitions of these terms in the US to better identify individuals who engage in excessive alcohol use and experience alcohol-related injuries, which is currently the third leading cause of mortality (41). As such, the USAUDIT is more accurate in identifying such at-risk individuals (12,42) and can be incorporated into brief counseling interventions to offer accurate normative feedback on an individual’s drinking behavior in comparison to similar individuals.

Second, a more in-depth review of the sources of accuracy for the AUDIT and USAUDIT lend support for the USAUDIT in its classification accuracy. In the AUDIT, males have far different correct classification rates (PPV = 68.2%, NPV = 81.3%) compared to females (PPV = 76.9%, NPV = 79.0%). This suggests that the use of the AUDIT with males is potentially subject to higher levels of false positives. With the USAUDIT, there is greater balance in PPV and NPV across gender (Males: PPV = 76.8%, NPV = 76.1%; Females: PPV = 76.8%, NPV = 80.2%). In this sample, the USAUDIT does a better job balancing correct and incorrect classifications across gender compared the AUDIT.

Finally, prior work has supported using the NIAAA criteria to assess at-risk drinking in college students (19). In the Madson et al. study, the USAUDIT-C subscale outperformed the AUDIT-C and their full-scale versions to identify at-risk drinkers (20). Although the full-scale version of the USAUDIT significantly outperformed the consumption subscale when using the DSM-5 potential AUD classification scheme, the better performance of the USAUDIT across these studies suggests its utility as the new “gold standard” screener in the United States. Although replication of the current study in a national sample of college students is needed, distinguishing those students who are considered risky drinkers versus those with a potential AUD can be used to (a) determine if additional diagnostic testing is warranted to confirm an AUD diagnosis, (b) inform appropriate treatment referrals (e.g., brief intervention versus formal treatment), and (c) offer personalized normative feedback in a brief intervention on how an individual’s drinking patterns compare to their same-gender peers.

An interesting finding of this study was the optimal cutoffs using the USAUDIT-C with males and females. Using the J statistic, we found an optimal score of 7 for women, and 6 for men. This runs counter to the previous literature, which suggests males typically have higher scores than females. Such was the case between males and females on the three other versions of the AUDIT in this study. It is possible that this anomaly is attributed to the stark difference in sensitivity and specificity estimates. With our male sample, the model was far better at correctly identifying those with a potential DSM-5 AUD (sensitivity = 86.0%) compared to correctly identifying those who do not have a potential DSM-5 AUD (specificity = 56.7%). The PPV (58.7%) and NPV (85.0%) in males revealed that a significant number of the positive results may be false positives (i.e., incorrectly identifying a male student as potentially having a DSM-5 AUD), whereas most of the non-AUD identifications are likely true negative results. In our female sample, the model was far better at identifying those who potentially do not have a DSM-5 AUD (specificity = 85.2%) compared to those who potentially have a DSM-5 AUD (sensitivity = 53.5%). An examination of the PPV (69.3%) and NPV (74.6%) in females revealed a lesser degree of false positives, but a somewhat higher degree of false negatives (i.e., incorrectly identifying a female student as no potential DSM-5 AUD). The false negative rate could be reduced by selecting a lower cutoff score, thereby conforming to the pre-existing literature that place male cutoff scores typically higher than female cutoff scores.

Although optimal cutoff scores differed between the current study and those recommended by Babor and colleagues (15), the procedure for screening for risky drinking patterns in primary care settings may be appropriate in college settings. Babor et al recommends a step-wise process for alcohol screening such that clients complete the first three items of the USAUDIT to identify at-risk drinkers, and if indicated, complete the remaining USAUDIT items to determine severity of drinking patterns, including potential AUD (15). Implementing this procedure in college health centers may be ideal given that many students endorse risky drinking but do not meet criteria for a DSM-5 AUD (19). In the current sample, approximately 53% of students were classified as risky drinkers but did not meet criteria for potential DSM-5 AUD. As such, it may be beneficial to administer the USAUDIT-C to all college students, and then administer the full USAUDIT to those students who meet or exceed the clinically-indicated cutoff score (7: men; 5: women (19)) to determine risk for AUD. Clinically-indicated scores on the USAUDIT could then be used to determine if additional diagnostic testing is needed and to inform treatment referrals (e.g., brief intervention vs traditional treatment).

The current findings provide several avenues for future research. First, an emerging body of evidence is forming around the stability of cutoff score findings. Continued research is needed with a national sample of college students. Second, recruiting a larger college sample can permit a more accurate examination of cutoff scores on the four AUDIT scales with respect to DSM-5 AUD severity groups (mild, moderate, severe), as well as differences across race and/or class year. We concur with Hagman (22) that a more complete picture of the scale performances by those categories and demographics may have implications for practice and allow for the tailoring of screening initiatives within specific subgroups on college campuses. Finally, despite finding different cutoff scores on the USAUDIT with college students as compared those recommended for use in primary care, it is unclear whether location (i.e., primary care vs education setting) or type of service available affects participants responses on the USAUDIT. Future research on the impact of screening location and health services sought/offered on participant responses to the USAUDIT can clarify whether cutoff scores translate to various settings for college students.

Limitations

This study had several limitations. First, data were based on self-reports, which can be subjected to recall bias and social desirability. Assurances of anonymity and the use of psychometrically-sound measures were employed in this study design to enhance response accuracy and reduce threats to internal/external validity. AUD was not determined via a clinical interview, which is the “gold standard” for collecting diagnostic information. Future research using a structured clinical interview to diagnose AUD using either an online version (43) or completed in-person by a trained clinician will increase confidence in current findings. Relatedly, while the current study administered items to assess at-risk drinking that are consistent with national agencies (i.e., NIAAA; CDC), further validation of the at-risk drinking criteria is warranted. Additionally, we offered preliminary cutoff scores on the AUDIT and USAUDIT for students who endorsed symptoms consistent with mild, moderate, and severe potential AUD. The scores in the current study were much lower than those recommended by Babor et al (18) which may be, in part, due to the small sample sizes across these three groups. Therefore, in addition to employing a clinical interview to accurately diagnose AUD, future research would benefit by obtaining a larger sample of college students to investigate optimal cutoff scores across the AUD spectrum. Lastly, we used a convenience sample of college students comprising primarily White, non-Hispanic women. This type of sampling design potentially reduces the generalizability of study findings. While the drinking frequencies reported in the current study are consistent with other studies of alcohol use among college students (44), the two prior studies that used the AUDIT (22) or the USAUDIT (21) had an equivalent or higher, respectively, number of men in their sample. Thus, a multi-site, experimental design with a gender-balanced, diverse sample can provide clarity on optimal cutoff scores with the USAUDIT.

Conclusions

This is one of the first studies to evaluate and compare the diagnostic performance of the AUDIT and USAUDIT in detecting DSM-5 AUDs among college students. This study provides further support for both scales as valid screening tools in detecting DSM-5 AUDs among college students. Findings indicated that the AUDIT and USAUDIT have a high degree of diagnostic accuracy in screening for DSM-5 AUDs in males and females. Also, this study suggests that different gender-specific AUDIT and USAUDIT cutoff scores should be used to detect AUD. Given the problem of risky drinking in college students, it is vital to identify and validate accurate screening measures. Considering prior support with the USAUDIT-C in detecting risky drinking (20), we expect school administrators to benefit most from using the USAUDIT to identify those students engaging in risky drinking (USAUDIT-C) as well as those with a potential AUD (USAUDIT), permitting early prevention for those students most in need.