Screening for underage drinking and DSM-5 alcohol use disorder in rural primary care practice

Abstract

Objective

To examine the NIAAA Youth Guide alcohol frequency screening thresholds when applied to DSM-5 diagnostic criteria, and to describe alcohol use patterns and AUD characteristics in rural youth from primary care settings.

Study design

Adolescents (n=1193; ages 12 through 20) visiting their PCP for outpatient visits in six rural primary care clinics were assessed prior to their practitioner visit. A tablet computer collected youth self-report of past-year frequency and quantity of alcohol use and DSM-5 AUD symptoms. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were determined

Results

; For early adolescents (ages 12 through 14), 1.9% met DSM-5 criteria for past-year AUD and ≥3 days with alcohol use in the past year yielded a screen for DSM-5 with optimal psychometric properties (sensitivity: 89%; specificity: 95%; PPV: 37%; NPV: 100%). For middle adolescents (ages 15 through 17), 9.5% met DSM-5 AUD criteria, and ≥3 past year drinking days showed optimal screening results (sensitivity: 91%; specificity: 89%; PPV: 50%; NPV: 99%). For late adolescents (ages 18 through 20), 10.0% met DSM-5 AUD criteria, and ≥12 past year drinking days showed optimal screening results (sensitivity: 92%; specificity: 75%; PPV: 31%; NPV: 99%). The age stratified NIAAA frequency thresholds also produced effective results.

Conclusion

In rural primary care clinics, 10% of youth over age 14 had a past-year DSM-5 AUD. These at-risk adolescents can be identified with a single question on alcohol use frequency.

Given the relatively high prevalence of alcohol-related harm among youth, medical organizations recommend routine screening for underage drinking in clinical practice. ^1,2,3,4,5 However, most adolescents visiting primary care practitioners (PCPs) do not receive alcohol screening and related services, with PCPs citing barriers including time constraints and inadequate training.^6,7,8 Advances in computer administered self-assessment methods and the availability of brief screening tools may combine to provide feasible and effective methods for PCPs to optimize their efforts to efficiently identify underage drinking in their adolescent patients.

Approaches to facilitate screening to identify adolescents with alcohol-related problems are particularly needed for PCPs in rural settings, given higher rates of alcohol use among rural youth.^1,9 Alcohol use patterns dramatically change across adolescent development. Average ages of onset for drinking milestones provide some guidelines regarding the development of alcohol use behavior, and suggest the potential utility of age-specific alcohol screening thresholds.

The purpose of a screening tool is to efficiently and effectively identify adolescents likely to have an alcohol use disorder (AUD). For adults, such screening has utilized items referencing alcohol related problems (eg, CAGE [Cut down, Annoyed, Guilty, Eye opener]). With CAGE showing poor psychometric properties in teens, attempts have been made to devise other problem-based screens.¹¹ CRAFFT¹² (Car, Relax, Alone, Forget, Friends, Trouble) has been the most extensively studied, and is comprised of three substance use questions and six substance problem questions. Although found to have acceptable psychometric properties in some studies,¹² the specificity of the CRAFFT has been unacceptably low in some clinical settings¹³ and inferior to screening based on substance use frequency.¹⁴ CRAFFT and similar approaches also require substantial administration time.

Among teens, alcohol use frequency has been observed to be highly correlated with AUD.¹⁵ In a national sample of 12 to 18 year olds,¹⁶ past year alcohol use frequency has been found to be a screen for AUD with excellent psychometric properties. The use of an alcohol frequency item to screen youth for problematic drinking was adopted for the National Institute on Alcohol Abuse and Alcoholism (NIAAA) Alcohol Screening and Brief Intervention for Youth: A Practitioner’s Guide (NIAAA Youth Guide).⁵

When adolescents are asked to report their alcohol use in a primary care setting, assessment may have several advantages.^{17, 14,18,19} This method could facilitate routine alcohol screening of adolescents seen in rural primary care settings.

This study used a computer-administered assessment to examine alcohol involvement in a large sample of adolescents seen in rural primary care settings, and examined the psychometric characteristics of alcohol use patterns as screening for DSM-5 AUD. We also tested the DSM-5 AUD performance of NIAAA-recommended age stratified alcohol use frequency cut-offs,⁵ which were developed for DSM-IV AUD. The AUD diagnostic criteria in DSM-5 represent the current consensus AUD definition. Compared with DSM-IV AUD, DSM-5 AUD defines a single diagnosis (vs. abuse and dependence in DSM-IV) and requires a minimum of two endorsed items to meet diagnostic criteria (vs. one for abuse in DSM-IV).

METHODS

The region served by the rural primary care practitioners involved in this study has been designated “Pennsylvania Wilds” by the Pennsylvania (PA) Tourism Office. The PA Wilds region was selected for this study because this area has the lowest population density in Pennsylvania and was the rural area most proximal to the academic centers conducting the study (ie, University of Pittsburgh, Pittsburgh and Bradford campuses). The study was conducted from September 1, 2008, through June 3, 2015.

The subjects were 1193 adolescents (ages 12 through 20 years) sequentially recruited from the practices described above. The subjects and methods reported here were distinct from those described in Gordon et al.⁷

Subjects were first seen by a practice staff member, who provided a brief description of the project, and asked the adolescent whether he or she was interested in receiving a more detailed description of the study by research staff. No information was collected about patients who declined participation. Informed consent was then conducted with the subject [18 or older] or parental consent with subject assent [if under 18 years old]. Subjects received $25 for research participation. Participating youth were provided with a tablet computer that collected data on past year alcohol use and alcohol-related symptoms. Survey completion time was three to eight minutes. Participation in the study did not impede patient flow in the practices. The study was approved by the University of Pittsburgh Institutional Review Board.

Measures

Alcohol use patterns

Subjects were provided with a “standard drink” definition⁵ as part of the computer administered assessment. With the exception of an initial question on any lifetime alcohol use, all questions were defined as referencing at least one standard drink. The alcohol use items were: age of first drink; alcohol use frequency for the past 30 days and past 12 months; typical number of drinks per occasion (ie, quantity); lifetime greatest number of drinks in 24 hours; age of first binge [traditional definition: ≥5 male/ ≥4 female or more drinks within two hours]; age of first incident of intoxication [“drunk”]; frequency of binge drinking in the past 30 days. Although the traditional definition of a drinking binge has typically been applied across development, binge definitions based on estimated blood alcohol concentrations (eBAC) have been developed that are more appropriate for younger teens.²⁰ Using the “lifetime greatest number of drinks” response, the eBAC²⁰ binge thresholds were calculated as follows: ages 9 to 13 years: ≥3 drinks; 14 or 15 years: ≥4 for males, ≥3 for females; 16 or 17 years: ≥5 for males, ≥3 for females.

AUD symptoms and diagnoses

The NSDUH computer administered structured diagnostic assessment for determining DSM-IV AUD symptoms and diagnoses^21,22,23 for the past 12 months was expanded to cover the 11 DSM-5²⁴ AUD symptoms (i.e., the DSM-5 “craving” symptom was added to the DSM-IV²⁵ AUD symptoms queried in NSDUH). The DSM-IV “legal problems” item was assessed but was not used to determine DSM-5 AUD diagnosis.

Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV)

Relative to disorder status (i.e., DSM-5 AUD), screening test results may be true positive (TP: screen+, disorder+); false positive (FP: screen+, disorder-), true negative (TN: screen-, disorder) and false negative (FN: screen-, disorder+). The psychometric characteristics of a screening test are indicated by four statistics: sensitivity (SE=TP/TP+FN); specificity (SP=TN/TN+FP); positive predictive value (PPV: TP/TP+FP); negative predictive value (NPV: TN/TN+FN). The selection of threshold values involves optimizing these values while taking into consideration the consequences of inaccurate results. We examined the screening performance of past-year frequency, average quantity per occasion, and a quantity x frequency (QxF) product estimate of the total number of drinks consumed in the past year to identify DSM-5 AUD.

We also examined the screening performance of the alcohol use frequency cut-points recommended in the NIAAA Youth Guide⁵ to identify youth with DSM-5 AUD symptoms, given that these were developed using DSM-IV AUD. The NIAAA Youth Guide⁵ age-specific alcohol frequency cut-points to identify levels of AUD risk by age as follows: “Moderate Risk” were age 12–15: 1 or more days per year; age 16–17: 3 or more days; age 18: 12 or more days. For “Highest Risk”, the alcohol frequency cut-points were age 12–15: 3 or more days per year; age 16: 12 or more days; age 17: 24 or more days; age 18: 52 or more days. In line with the NIAAA Guide’s detection of “Moderate” and “Highest” levels of alcohol use risk, the performance of the NIAAA-recommended alcohol use frequency cut-points was examined against any DSM-5 AUD symptom, any DSM-5 AUD diagnosis (≥2 symptoms), AUD–Moderate (≥4 symptoms); and AUD–Severe (≥6 symptoms). These DSM-5 AUD severity definitions were included to correspond with DSM-IV AUD severity levels described in the NIAAA Youth Guide.⁵ In addition, the NIAAA Youth Guide DSM-IV thresholds and criteria were also examined.

RESULTS

The six practices involved in the present study included three pediatric practices (P1: 5 practitioners; 412 patients; P2: 3 practitioners; 131 patients; P3: 4 practitioners, 311 patients), and three general medical practices serving adolescents (P4: 5 practitioners, 157 patients; P5: 1 practitioner, 99 patients; P6: 1 practitioner, 83 patients). Subjects from the Pediatric practices (12 practitioners; 854 subjects) were, on average, younger than subjects from the General (7 practitioners; 339 subjects) practices (pediatric: 14.9±2.1 years; other: 16.4±2.4 years; t=11.0, d.f.=1191, p<.001). General practices (females: 226, 67%; males: 113, 33%) had a higher proportion of female patients than did the Pediatric (females: 449, 53%; males: 405, 47%) practices (χ²=19.6, d.f.=1, p<.001). Unadjusted for age and sex, pediatric practice patients had significantly lower rates of past month drinking (105/854: 12%) than did patients from General practices (70/339: 21%) (Wald χ²=13.3, d.f.=1, p<.001). However, after controlling for age and sex, patients from pediatric and general practices did not significantly differ on past month drinking rates (Wald χ²=0.2, d.f.=1, p=.7).

Overall, the subject ages were distributed as follows: ages 12–14: n=479, 40%; ages 15–17: n=463, 39%; ages 18–20: n=251, 21%. The sex distribution was 57% female (n=675) and 43% male (n=518). Race identification was primarily White (n=1114, 93.4%), with other represented groups including Native American (n=53, 4.4%), Black (n=15, 1.3%), and other groups less the 1% (n=11, .9%). Hispanic ethnicity was indicated by 4.5% (n=54).

Alcohol involvement characteristics

Alcohol use and AUD characteristics were examined by age and sex (Table I). All alcohol involvement variables were strongly associated with age. There was a significant sex difference in the rates of ever having tried alcohol, with females having higher rates than males. For all other alcohol involvement characteristics, males and females were not significantly different.

Table 1.

Patient alcohol involvement characteristics by age group and gender

Characteristic		Age group
		12–14 n=479	15–17 n=463	18–20 n=251	Age			Gender
		n (%)	n (%)	n (%)	χ 2	d.f.	p	χ 2	d.f.	p
Ever tried alcohol	female	118(45)	183(74)	217(87)	158	1	<.001	5.1	1	.02
	male	86(40)	145(67)	69(82)
Past month (≥ 1 standard drink)	female	8(3)	42(17)	53(32)	114	1	<.001	0.4	1	.5
	male	5(2)	32(15)	35(42)
ever binge (eBAC)	female	13(5)	73(32)	--	79	1	<.001	3.4	1	.06
	male	9(4)	47(23)	--
ever binge (5/4)	female	12(5)	54(22)	60(36)	139	1	<.001	3.5	1	.06
	male	8(4)	48(22)	45(54)
Past month binge (5/4)	female	4(2)	23(9)	31(19)	57	1	<.001	.01	1	.9
	male	4(2)	16(7)	16(19)
AUD by DSM-5 (past year)	female	6(2)	24(10)	18(11)	25	1	<.001	.2	1	.7
	male	3(1)	20(9)	7(8)
Past year frequency (≥ 1 standard drink)
None	female	221(84)	143(58)	63(38)	244	5	<.001	2.7	5	.8
	male	190(88)	127(58)	31(37)
1 or 2	female	32(12)	53(22)	32(19)
	male	21(10)	47(22)	15(18)
3–11	female	2(1)	27(11)	27(16)
	male	3(1)	26(12)	15(18)
Monthly (12–23 days)	female	1(.4)	4(2)	9(5)
	male	1(.5)	4(2)	7(8)
2 or 3 per month (24–51 days)	female	3(1)	7(3)	20(12)
	male	2(1)	8(4)	7(8)
Weekly (52 or more days)	female	3(1)	12(5)	16(10)
	male	0(0)	5(2)	9(11)

Note: Test statistics are Wald chi-square statistics from logistic regression analyses including age and sex as independent variable. AUD=alcohol use disorder; eBAC= estimated blood alcohol concentration

Screen thresholds and performance

During early adolescence, 9 of 479 subjects (1.9%) met DSM-5 criteria for AUD in the prior year (Table I). A threshold of ≥3 days with alcohol use in the past year (Table II) yielded optimal psychometric performance. A typical quantity of ≥2 drinks per occasion performed similarly, as did an estimated overall quantity of ≥3 drinks consumed in the past year.

Table 2.

Using past year alcohol use patterns to screen for DSM-5 alcohol use disorder

Age 12–14years: Early Adolescence
Test	threshold	sensitivity	specificity	PPV	NPV
Frequency	1 or more days	1.00	0.83	0.15	1.00
	3 or more	0.89	0.95	0.37	1.00
	12 or more	0.58	0.99	0.55	0.99
	24 or more	0.42	0.99	0.50	0.98
Quantity	2 or more drinks	1.00	0.98	0.46	1.00
	3 or more	1.00	0.95	0.36	1.00
QxF	3 or more drinks	1.00	0.95	0.36	1.00
	12 or more	0.69	0.98	0.48	0.99
	52 or more	0.50	0.99	0.53	0.99
Age 15–17 years: Mid-Adolescence
Test	threshold	sensitivity	specificity	PPV	NPV
Frequency	1 or more days	1.00	0.63	0.25	1.00
	3 or more	0.91	0.89	0.50	0.99
	12 or more	0.65	0.97	0.73	0.96
	24 or more	0.50	0.97	0.71	0.94
Quantity	2 or more drinks	0.94	0.88	0.44	0.99
	3 or more	1.00	0.86	0.48	1.00
QxF	3 or more drinks	1.00	0.86	0.48	1.00
	12 or more	0.81	0.95	0.67	0.98
	52 or more	0.78	0.97	0.78	0.97
Age 18–20 years: Late Adolescence
Test	threshold	sensitivity	specificity	PPV	NPV
Frequency	1 or more days	1.00	0.42	0.16	1.00
	3 or more	0.96	0.62	0.22	0.99
	12 or more	0.88	0.80	0.32	0.98
	24 or more	0.72	0.85	0.35	0.96
Quantity	2 or more drinks	0.90	0.66	0.26	0.98
	3 or more	0.81	0.76	0.31	0.95
QxF	3 or more drinks	0.96	0.59	0.22	0.99
	12 or more	0.92	0.75	0.31	0.99

Note: Bolded values indicate optimal cut-points. Alcohol use items are asked for “past year” QxF= Quantity x Frequency; PPV= positive predictive value; NPV= negative predictive value

For middle adolescence, 44 of 463 subjects (9.5%) met DSM-5 criteria for AUD. Among the frequency options, a threshold of ≥3 days with alcohol use in the past year yielded optimal psychometric performance. Similar screening performance properties were observed with typical quantity of ≥2 drinks per occasion and with an estimated overall quantity of >3 drinks consumed. Note that, for the combined early and middle adolescent groups, the frequency threshold of ≥3 days with alcohol use in the past year showed the following psychometric properties: sensitivity: 91%; specificity: 93%; PPV: 44%; NPV: 99%.

For late adolescence, 25 of 251 subjects (10.0%) met DSM-5 criteria for AUD. Among the frequency options, a threshold of ≥12 days with alcohol use in the past year yielded optimal psychometric performance in this age group. Acceptable screening performance properties were also observed with an estimated overall quantity of ≥12 drinks consumed in the past year.

The NIAAA Youth Guide (2011) age-specific alcohol frequency thresholds to identify “Moderate Risk” and “Highest Risk” described above were examined among patients 12 through 18 years old. For the DSM-IV thresholds specified for the NIAAA Youth Guide, the NIAAA Moderate Risk threshold yielded acceptable performance for identifying those likely to have one or more DSM-IV AUD symptom (sensitivity: 84%; specificity: 87%; PPV: 42%; NPV: 98%). The NIAAA Highest Risk threshold yielded acceptable performance for identifying those likely to have DSM-IV alcohol dependence (sensitivity: 63%; specificity: 94%; PPV: 20%; NPV: 99%).

For DSM-5, the NIAAA Moderate Risk threshold performance characteristics for identifying those likely to have one or more DSM-5 AUD symptom or an AUD diagnosis are presented in Table III. The NIAAA Highest Risk threshold yielded acceptable performance for identifying those likely to have a severe DSM-5 AUD diagnosis.

Table 3.

Performance of NIAAA Youth Guide recommended cut-points in identifying youth with DSM-5 alcohol use disorder symptoms: 12–18 years old

NIAAA-recommended Alcohol Frequency Cut-Point for Moderate Risk
DSM-5 Outcome	sensitivity	specificity	PPV	NPV
Any symptom	0.85	0.87	0.45	0.98
AUD - Mild (≥2 symptoms)	0.92	0.84	0.28	0.99
AUD - Moderate (≥4 symptoms)	1.00	0.81	0.10	1.00
AUD - Severe (≥6 symptoms)	1.00	.80	0.05	1.00
NIAAA-recommended Alcohol Frequency Cut-Point for Highest Risk
DSM-5 Outcome	sensitivity	specificity	PPV	NPV
Any symptom	0.47	0.97	0.64	0.94
AUD - Mild (≥2 symptoms)	0.65	0.96	0.51	0.98
AUD - Moderate (≥4 symptoms)	0.57	0.94	0.15	0.99
AUD - Severe (≥6 symptoms)	0.91	0.93	0.12	1.00

Note: NIAAA= National Institute on Alcohol Abuse and Alcoholism; AUD= Alcohol Use Disorder; PPV= positive predictive value; NPV= negative predictive value

DISCUSSION

Adolescent alcohol involvement characteristics were strongly related to age, such that few in the early adolescence group had a significant alcohol involvement history. Among those in the middle adolescence group, 9.5% met DSM-5 criteria for a past year AUD. The alcohol involvement rates among rural youth seen in primary care noted here were similar to those reported in representative community samples with urban and rural populations.^2,16 For example, among those in the middle adolescence age group, 42% reported past year alcohol use. This rate is similar to the 44% of 10^th grade students who reported past year alcohol use in the 2014 Monitoring the Future survey.¹⁶ The lack of sex differences noted here for alcohol involvement characteristics was also similar to findings observed in recent years with representative community samples^2,16. In the past 10–15 years, male and female adolescents have become increasingly similar on their alcohol involvement patterns. ¹⁶ The substantial proportion of rural youth already meeting DSM-5 criteria for AUD in the 15–17 year old age period reinforces the importance of primary preventive efforts beginning in early adolescence.

Prior studies¹⁶ have been interpreted as indicating that past year alcohol use frequency has acceptable psychometric properties as a brief initial screen for DSM-IV AUD among adolescents. This approach has been recommended for routine adolescent alcohol screening.⁵ The acceptability of a screening test, traditionally judged through examining validity statistics including sensitivity, specificity, PPV and NPV, depends on the clinical context.²⁶ In this study, for example, screening for DSM-5 AUD among adolescents ages 12 through 17 with a threshold of ≥3 days with alcohol use in the past year resulted in 99% NPV and 44% PPV. Adolescents reporting fewer than 3 days with alcohol use very rarely meet AUD criteria. Among those adolescents with 3 or more days with alcohol use, 44% had AUD. We anticipate that a positive screen would be followed by a diagnostic assessment. This threshold is highly efficient in identifying adolescents who do not need the additional AUD assessment, but less efficient in the identification of those with AUD. However, even though the subsequent AUD assessment is a low cost and low risk diagnostic test, we interpret the overall result to be acceptable. Applying this reasoning, the NIAAA recommended thresholds as well as the optimal thresholds noted here produced acceptable results.

The present study confirmed that alcohol use frequency demonstrates acceptable psychometric properties as a screen for DSM-5 AUD among this rural primary care adolescent sample with thresholds and utility indicators similar to those previously reported. Specifically, when applied to the same age range (i.e., ages 12 through 18), the NIAAA⁵ Moderate Risk thresholds showed acceptable overall sensitivity (85%) and specificity (87%) as a screen for any DSM-5 AUD symptom. Similarly, the NIAAA⁵ Highest Risk thresholds showed acceptable overall sensitivity (91%) and specificity (93%) as a screen for severe DSM-5 AUD. Despite methodological differences, the results of the present study, which assessed alcohol use and AUD by tablet computer in the context of clinical care, were comparable with those obtained using a national sample.¹⁶

Several previously unaddressed issues pertinent to clinical practice were examined here. With the relatively complex age and severity stratification of the NIAAA Guide, some clinicians may be interested in a simplified approach to alcohol screening. These results indicate that, for adolescents age 17 and younger, a threshold of 3 or more days consuming at least one standard drink produces acceptable screening results for DSM-5 AUD. In clinical practice, these acceptable sensitivity and specificity results need to be considered in the context of PPV and NPV. The above noted 3 or more days threshold for DSM-5 AUD in ages 12 through 17 produced a PPV of 44% and NPV of 99%. In other words, among those screening positive, 44% would be expected to have AUD, and, among those screening negative, 99% would be expected to not have AUD. An alcohol use frequency screen followed by an AUD evaluation among those who screen positive would constitute a simple, brief and cost-effective clinical assessment procedure. The present study extended prior findings to ages 19 and 20 years. Regular alcohol use becomes more common at ages 19 and 20 years, and, despite being considered underage, often occurs without AUD. This is reflected by diminishing specificity and lower PPV for frequency thresholds among the late adolescent group. Age adjusted thresholds are clearly necessary for this older adolescent period.

Some obstacles to alcohol related screening and AUD assessment in primary care settings may be addressed through computer administration. Computer administered assessment does not require PCP or staff training or expertise. The uniform presentation of items and response options enhances measurement reliability. Adolescents are generally more willing to divulge sensitive information through computer administration of items, improving validity. Computer administered assessment may be completed prior to the visit, thereby reducing PCP visit time. The approach can be readily disseminated. A disadvantage is that, when presented without explanations or discussion, adolescents may misinterpret diagnostic items.²⁷ Most practices are not currently using tablet computers to collect information, and integration of this approach into standard practice presents surmountable cost and logistical challenges.

Although the number of subjects included in the present study (n=1193) was larger than some similar studies conducted in clinical settings (e.g., n=219¹⁵; n=216¹⁴), the sample size provided limited statistical power for the examination of screening thresholds by subgroups within ages, such as age by sex. Note that a study of alcohol use frequency thresholds by AUD in a large sample of adolescents (n=166,165) concluded that these relationships were similar for males and females, and that sex based adjustments were not warranted. By design, the study focused on the understudied group of rural adolescents, such that results may have limited generalizability to other populations. To comply with the HIPAA Privacy Rule, information on patients not participating was not collected. Consequently, the extent to which patients participating in the study represented the practice population could not be determined. The assessment was limited to alcohol involvement. Other research with adolescents suggests that querying past year substance use frequency provides an efficient screen for SUD (e.g., cannabis use disorder).^14,28,29 More comprehensive, computer administered risk assessment needs to be developed and tested for clinical applications.