Abstract
BACKGROUND:
It is important to improve primary care providers’ capability to identify youth at risk for alcohol and other drug use. To our knowledge, this is the first study to use Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria to compare screeners for youth for both alcohol and marijuana, given that these are the most frequently used substances by this age group.
METHODS:
We compared the psychometric performance of 4 screeners: the National Institute on Alcohol Abuse and Alcoholism Screening Guide (NIAAA SG), the Alcohol Use Disorders Identification Test, the Car–Relax–Alone–Forget–Family and Friends–Trouble (CRAFFT) screener, and the Personal Experience Screening Questionnaire Problem Severity Scale (PESQ-PS) in identifying alcohol and marijuana use outcomes. Youth age 12 through 18 (N = 1573; 27% black, 51% Hispanic) were screened with the NIAAA SG, followed by a Web survey that included the other screeners and outcomes.
RESULTS:
Sensitivity for alcohol outcomes indicated that the NIAAA SG (0.87) did not perform as well as the CRAFFT (0.97) or PESQ-PS (0.97) screeners but performed better than the Alcohol Use Disorders Identification Test (0.70). The pattern for sensitivity across screeners for marijuana outcomes was similar.
CONCLUSIONS:
An important tradeoff in primary care settings is precision versus practicality. Because of brevity and focus on frequency of drinking, the NIAAA SG offers ease of administration and is good at identifying youth with probably problematic drinking levels. The PESQ-PS and the CRAFFT correctly identify more at-risk youth for alcohol and marijuana than the NIAAA SG. Future work is needed to elucidate how to efficiently and accurately identify at-risk youth in the primary care setting, including determining the best cutoff points to use to increase sensitivity.
What’s Known on This Subject:
Most adolescents are not screened for substance use in primary care because of time constraints or insufficient training. Providers need a screener that can be easily incorporated into an appointment; however, there is a tradeoff between precision and practicality.
What This Study Adds:
We examined sensitivity, specificity, and positive and negative predictive value for the new National Institute on Alcohol Abuse and Alcoholism screener and 3 widely used adolescent screeners for various levels of alcohol and marijuana use and impairment in a large, racially and ethnically diverse sample of adolescents.
It is important to improve the capability of primary care providers and associated health care staff to identify youth at risk for alcohol and other drug (AOD) use because use during adolescence is associated with academic problems, poorer mental health, future use of other illicit drugs (including heroin and cocaine), and a higher likelihood of abuse or dependence in adulthood.1–3 The primary care setting provides a unique opportunity to screen significant numbers of adolescents4 and identify those at risk for problematic use. Guidelines propose that providers screen for AOD use and provide brief counseling and referrals where appropriate5 and that doctors screen all patients for alcohol use starting in middle school.6
Unfortunately, most adolescents are not screened for AOD use in primary care settings,7–11 and significant numbers of at-risk youth remain unidentified and never receive appropriate preventive or treatment services.12–14 Lack of screening and preventive services is even more profound among younger adolescents age 11 to 14 and socioeconomically disadvantaged youth.15,16 Lack of primary care screening typically is caused by provider time constraints, discomfort discussing AOD use, insufficient training, or lack of referral options.9,10,17,18
Given these concerns, providers need an easy-to-administer screener that takes little time or training, can be incorporated into a primary care appointment to determine an adolescent’s risk level, and will facilitate appropriate referral or treatment. There is a difficult tradeoff between precision and practicality in the primary care setting. To increase screening in pediatric settings, the National Institute on Alcohol Abuse and Alcoholism (NIAAA) developed a brief 2-item screening guide (SG) with 2 age-specific screening questions about friend and self-drinking.19 Age-sensitive cutoff points can assist providers in determining whether brief advice, counseling, or referral is appropriate. For example, any report of drinking if the youth is ≤15 years old warrants brief advice and counseling. For youth ≥16 years old, the threshold is a bit higher, and drinking ≥6 days in the past year is considered moderate or high risk.
In 2014, Kelly et al11 evaluated the NIAAA SG in a sample of 525 youth age 12 to 17 years (54% female) who were mainly black (92.8%). Their focus was on expanding the NIAAA SG to include drug and tobacco questions for a total of 10 questions and determining cutoff points for their Brief Screener for Tobacco, Alcohol and Other Drugs. They established cutoff points for this new screener for each substance and compared the Brief Screener for Tobacco, Alcohol and Other Drugs cutoff points with the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) criteria.11 In 2016, Clark et al20 examined cutoff points for the NIAAA SG and found that ≤3 days of alcohol use in the past year yielded optimal psychometric performance.
Several studies21–25 have reviewed adolescent screeners to determine performance against DSM-4 and DSM-5 criteria. Some screeners focus only on alcohol, such as the NIAAA SG, whereas others address AOD more broadly. A comparison of the Alcohol Use Disorders Identification Test (AUDIT; 10 items; alcohol only), Car–Relax–Alone–Forget–Family and Friends–Trouble (CRAFFT; 6 items; AOD), Cut Down, Annoyed, Guilty, Eye-Opener (CAGE; 4 items; alcohol only), and Tolerance, Worry, Eye-Opener, Cutoff (TWEAK; 5 items; alcohol only) in a sample of youth recruited in an emergency department found that Cut Down, Annoyed, Guilty, Eye-Opener items were reported infrequently by adolescents and that the AUDIT performed the best across the range of cutoff scores.21 The Problem Severity Scale on the Personal Experience Screening Questionnaire (PESQ-PS; 18 items; AOD)23,26,27 had the highest reliability estimates when compared with several other screening measures that address behavior and consequences for youth, such as the Rutgers Alcohol Problem Index.28 Furthermore, the 2011 American Academy of Pediatrics guidelines recommended routine use of the CRAFFT, which measures AOD use,29 has been validated in diverse populations,30,31 and has good sensitivity and specificity with new DSM-5 criteria.32
Thus, across diverse studies, the AUDIT, CRAFFT, and PESQ-PS appear to be the most sensitive and reliable screeners with adolescents. They are short, taking ∼5 to 10 minutes to complete and score. The NIAAA SG19 could also easily be integrated into primary care practice given its brevity,11 and studies have shown that asking youth about alcohol use frequency is often the best predictor of their alcohol use over time.33,34 We therefore compared these 4 screeners in identifying both alcohol and marijuana use among adolescents because these substances are the most frequently reported substances for this age group.35
This study moves the field forward by comparing 4 screeners in a large racially and ethnically diverse sample of adolescents, ages 12 through 18, recruited across 4 primary care clinics in Los Angeles, California and Pittsburgh, Pennsylvania. The study’s purpose was to inform primary care providers seeking to screen adolescents for alcohol and marijuana use about the 4 screeners’ strengths and weaknesses; we compared screeners with respect to sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for various levels of alcohol and marijuana use and impairment.
Methods
Procedures
This study involved 4 clinics (1 in Los Angeles, 3 in Pittsburgh). We obtained a certificate of confidentiality; procedures were approved by both the RAND institutional review board and the 4 clinics. Every youth age 12 through 18 who came for an appointment during the 2.5-year study period (April 2013–November 2015) was asked to be in the project. We obtained parental consent and youth assent (<18 years old) or consent (18 years old). Youth were screened with the NIAAA SG, completed a survey via the Web, and paid $25.
Setting, Participants, and Surveys
The Pittsburgh and Los Angeles area clinics are family-based community health clinics that provide care for ethnically and racially diverse and underserved populations of youth. Sites offer both longitudinal, continuity-based care and episode-based urgent care to their patients. Clinics in both cities have a large percentage of minority patients and serve a high proportion of low-income patients. Approximately 3309 youth were approached to be in the project. Of these youth, 27% (n = 892) were ineligible because of age, lack of English proficiency, being present for an appointment other than their own, or disability status; 18.5% (n = 614) declined to participate, mostly because of time constraints or youth being at the clinic for family planning and not wanting their parents to know they were there. This process yielded a total sample of 1803 youth who enrolled or provided consent to contact. Of the 1803 youth, 230 did not complete the baseline within the field period or had unreliable contact information. The final enrolled sample included 1573 youth. Screening and surveys were completed in a private clinic space without a parent present. RAND staff first screened youth in person by using the NIAAA SG; youth then completed a Web-based survey comprising other screeners and outcome variables on a laptop immediately after completion of the NIAAA SG (see Supplemental Information for screener questions).
Screeners
NIAAA SG
Two screening questions were asked in a different order depending on age19: “In the past year, on how many days have you had more than a few sips of beer, wine, or any drink containing alcohol?” and “Do any of your friends drink alcohol?” Youth age 12 to 14 years were first asked about friend drinking and then self-drinking as a less threatening way to gauge use, whereas youth age ≥15 years (and 14-year-olds in high school) were first asked about self-drinking and then friend drinking. Adolescents were categorized based solely on the days of use question according to the published NIAAA risk assessment guide, which results in 4 risk categories: no risk, lower risk, moderate risk, and highest risk. For example, youth age 12 to 15 years were categorized as moderate risk if they reported 1 to 5 days of use; adolescents 16 years old were categorized as moderate risk if they reported 6 to 11 days of use.
CRAFFT
The 6-item CRAFFT29 addresses both alcohol and other drugs (eg, “Do you ever use AOD to Relax, feel better about yourself, or fit in?”). Response options are “yes” or “no,” and a “yes” response to ≥2 questions indicates risk.
PESQ-PS
The 18-question PESQ-PS26 assesses AOD use rated on a 4-point response scale (never, once or twice, sometimes, often). The summed score categorizes adolescents into 3 groups, no AOD problem (“Green Flag”), mild or moderate AOD problem (“Yellow Flag”), and severe AOD problem (“Red Flag”), by using established thresholds. Thresholds vary depending on sex and age.
AUDIT
The AUDIT36 focuses on frequency and consequences of drinking. It was modified slightly to be developmentally appropriate for youth.21 Youth are categorized into Zone I (alcohol education), Zone II (simple advice), Zone III (simple advice plus brief intervention and follow-up), or Zone IV (referral to specialist).
Outcomes
Youth with an alcohol use disorder (AUD) or cannabis use disorder (CUD) were identified via the Diagnostic Interview Schedule for Children Version IV (DISC-IV)37 computerized version, valid and reliable in adolescent populations.38–40 We used DSM-5 criteria available in this version (eg, the craving item was included; see Supplemental Figure) to identify subjects with AUD or CUD. Adolescents were classified as having an AUD or CUD if they reported ≥2 of the 11 criteria for AUD or CUD. Past-year alcohol and marijuana use was assessed via well-established measures with adolescents.41 We asked, “During the past year, how many times did you [drink at least one full drink of alcohol] [use marijuana]?” Responses ranged from 1 = “0 times” to 6 = “11–20 times” and were dichotomized (1 = “any use” versus 0 = “no use”). For past-year heavy alcohol use, respondents were asked, “During the past year, how many times have you tried five or more drinks of alcohol in a row, that is, within a couple of hours?” with the same response options and dichotomization. For past-year heavy marijuana use, respondents were asked, “On days you use marijuana, how often do you use it?” Responses were “Once,” “Twice,” and “3 or more times,” and responses of “twice” and “3 or more” were considered heavy marijuana use.
Statistical Analysis
We examined the proportion of youth identified as at risk for each screener. We estimated sensitivity, specificity, PPV, and NPV against DSM-5 diagnoses of AUD and CUD, past-year any use, and past-year heavy alcohol and marijuana use. Sensitivity is the probability that the screener correctly identifies at-risk youth (as at risk), whereas specificity is the probability that the screener correctly identifies no-risk youth (as low or no risk). PPV is the probability of a case screened as positive actually being positive; this depends on specificity of the test and prevalence of the condition. NPV is the probability of a negative diagnosis indicating a true negative. For screeners with multiple risk categories, we used the lowest established risk threshold to dichotomize adolescents into low- and high-risk categories. Specifically, for the AUDIT, youth categorized in Zone II or above were considered at risk, for the PESQ-PS, youth categorized in the yellow or red flag categories were considered at risk, and for the NIAAA SG screener, youth categorized as moderate or high risk were considered at risk. We had <2.5% of data missing for any 1 variable; pairwise deletion was used to handle missingness. We performed analyses via R version 3.2.4 (R Foundation for Statistical Computing, Vienna, Austria).
Results
More than half the sample was female; youth were racially and ethnically diverse, with a mean age of 15.5 years (Table 1). More than 40% of adolescents reported drinking >1 time in the past year, with almost a quarter reporting heavy use. About 37% reported using marijuana >1 time in the past year, with 1 in 5 reporting heavy marijuana use. According to DSM-5 criteria, 3.6% of youth were identified with an AUD; 13.6% were identified with a CUD.
TABLE 1.
Sample Characteristics by Age Group
| Overall, N = 1573, Mean (SD) or N (%) | Age 12–14, N = 498, Mean (SD) or N (%) | Age 15–18, N = 1075, Mean (SD) or N (%) | |
|---|---|---|---|
| Age | 15.5 (1.9) | 13.2 (0.8) | 16.6 (1.1) |
| Sex | |||
| Male | 662 (42.5%) | 235 (48%) | 427 (40.1%) |
| Female | 894 (57.5%) | 255 (52%) | 639 (59.9%) |
| Race or ethnicity | |||
| White | 232 (14.7%) | 78 (15.7%) | 154 (14.3%) |
| Black | 420 (26.7%) | 166 (33.3%) | 254 (23.6%) |
| Hispanic | 808 (51.4%) | 209 (42%) | 599 (55.7%) |
| Other or multiracial | 113 (7.2%) | 45 (9%) | 68 (6.3%) |
| Prevalence: past-year use | |||
| Alcohol use | 655 (41.7%) | 66 (13.3%) | 589 (54.9%) |
| Heavy alcohol use | 347 (22.1%) | 24 (4.8%) | 323 (30.1%) |
| Marijuana use | 575 (36.6%) | 77 (15.5%) | 498 (46.4%) |
| Heavy marijuana use | 302 (19.3%) | 38 (7.7%) | 264 (24.7%) |
| Prevalence: DSM-5 diagnosis | |||
| AUD | 61 (3.9%) | 4 (0.8%) | 57 (5.4%) |
| CUD | 211 (13.6%) | 23 (4.7%) | 188 (17.8%) |
Percentages are among nonmissing values; 17 missing sex, 10 missing past-year alcohol or marijuana use responses, 35 missing responses needed to determine DSM-5 Diagnostic Interview Schedule for Children diagnosis category.
Table 2 shows number and proportion of adolescents in each risk category by screener, stratified by age. Older youth were more likely to be identified as at risk. Overall, the NIAAA SG, CRAFFT, PESQ-PS, and AUDIT identified 19%, 30%, 31%, and 8% of adolescents as at risk, respectively.
TABLE 2.
Number and Percentage of Youth Identified at Risk for Alcohol or Drug Use by Screener
| Overall, N = 1573, N (%) | Age 12–14, N = 498, N (%) | Age 15–18, N = 1075, N (%) | |
|---|---|---|---|
| NIAAA | |||
| No risk | 992 (63.1%) | 445 (89.4%) | 547 (50.9%) |
| Lower risk | 287 (18.2%) | 0 (0%) | 287 (26.7%) |
| Moderate risk | 199 (12.7%) | 46 (9.2%) | 153 (14.2%) |
| Highest risk | 95 (6%) | 7 (1.4%) | 88 (8.2%) |
| CRAFFT | |||
| <2 | 1100 (70.2%) | 445 (89.7%) | 655 (61.2%) |
| ≥2 | 467 (29.8%) | 51 (10.3%) | 416 (38.8%) |
| PESQ-PS | |||
| Green Flag | 1064 (69%) | 432 (89.8%) | 632 (59.6%) |
| Yellow Flag | 227 (14.7%) | 28 (5.8%) | 199 (18.8%) |
| Red Flag | 250 (16.2%) | 21 (4.4%) | 229 (21.6%) |
| AUDIT | |||
| Zone I: Education | 1440 (91.8%) | 490 (98.6%) | 950 (88.6%) |
| Zone II: Simple Advice | 106 (6.8%) | 4 (0.8%) | 102 (9.5%) |
| Zone III: Advice and Counseling | 13 (0.8%) | 2 (0.4%) | 11 (1%) |
| Zone IV: Referral | 10 (0.6%) | 1 (0.2%) | 9 (0.8%) |
Percentages are among nonmissing values; 6 youth missing ≥1 response needed to calculate CRAFFT score, 32 youth missing ≥1 response needed to calculate PESQ-PS score, 4 youth missing ≥1 response needed to calculate AUDIT score.
Table 3 presents sensitivity, specificity, PPV, and NPV and associated 95% confidence intervals for screeners by AUD, past-year alcohol use, and past-year heavy alcohol use. For sensitivity, for all alcohol outcomes, the NIAAA SG did not perform as well as the CRAFFT or PESQ-PS screeners but performed better than the AUDIT. For AUD, the sensitivity was 0.87 for the NIAAA SG, 0.98 for the CRAFFT, 0.97 for the PESQ-PS, and 0.70 for the AUDIT. Specificity tended to be high, but it was lowest for the CRAFFT and PESQ-PS when AUD was used as the outcome. For PPV, among adolescents identified as at risk on the NIAAA SG, 19% had an AUD, 90% reported past-year alcohol use, and 66% reported heavy past-year alcohol use.
TABLE 3.
Sensitivity, Specificity, PPV, and NPV for Each Screener, With 95% Confidence Intervals, for 3 Different Outcomes: DSM-5 Diagnosis of AUD, Past-Year Alcohol Use, and Past-Year Heavy Alcohol Use
| Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|
| AUD | ||||
| NIAAA | 0.87 (0.76–0.94) | 0.84 (0.82–0.86) | 0.19 (0.14–0.24) | 0.99 (0.99–1.00) |
| CRAFFT | 0.98 (0.91–1.00) | 0.73 (0.71–0.76) | 0.13 (0.10–0.17) | 1.00 (0.99–1.00) |
| PESQ-PS | 0.97 (0.88–1.00) | 0.72 (0.70–0.74) | 0.12 (0.09–0.15) | 1.00 (0.99–1.00) |
| AUDIT | 0.70 (0.57–0.81) | 0.94 (0.93–0.96) | 0.34 (0.26–0.43) | 0.99 (0.98–0.99) |
| Past-year alcohol use | ||||
| NIAAA | 0.40 (0.37–0.44) | 0.97 (0.95–0.98) | 0.90 (0.86–0.93) | 0.69 (0.67–0.72) |
| CRAFFT | 0.61 (0.57–0.65) | 0.93 (0.91–0.94) | 0.85 (0.82–0.88) | 0.77 (0.74–0.79) |
| PESQ-PS | 0.64 (0.60–0.67) | 0.93 (0.91–0.94) | 0.86 (0.83–0.89) | 0.78 (0.75–0.80) |
| AUDIT | 0.19 (0.16–0.22) | 0.99 (0.98–1.00) | 0.94 (0.88–0.97) | 0.63 (0.61–0.66) |
| Past-year heavy alcohol use | ||||
| NIAAA | 0.56 (0.51–0.61) | 0.92 (0.90–0.93) | 0.66 (0.60–0.71) | 0.88 (0.86–0.90) |
| CRAFFT | 0.81 (0.76–0.85) | 0.84 (0.82–0.86) | 0.59 (0.55–0.64) | 0.94 (0.92–0.95) |
| PESQ-PS | 0.85 (0.80–0.88) | 0.84 (0.82–0.86) | 0.61 (0.56–0.65) | 0.95 (0.94–0.96) |
| AUDIT | 0.33 (0.28–0.39) | 0.99 (0.98–0.99) | 0.89 (0.82–0.94) | 0.84 (0.82–0.86) |
95% confidence intervals obtained by using exact binomial confidence limits.42
Table 4 parallels Table 3 for marijuana outcomes. Similar to alcohol, the NIAAA SG performed better than the AUDIT but did not perform as well as the CRAFFT or PESQ-PS in terms of sensitivity. For example, among adolescents with CUD, the CRAFFT, PESQ-PS and NIAAA SG correctly identified 88%, 91%, and 54% as at risk, whereas the AUDIT correctly identified 32%. For PPV, among adolescents identified as at risk on the NIAAA SG, 40% had a CUD, 77% reported past-year marijuana use, and 50% reported past-year heavy marijuana use. For both PPV and NPV, the NIAAA SG performed similarly to the CRAFFT and PESQ-PS when CUD and heavy marijuana use were examined.
TABLE 4.
Sensitivity, Specificity, PPV, and NPV for Each Screener, With 95% Confidence Intervals, for 3 Different Outcomes: DSM-5 Diagnosis of CUD, Past-Year Marijuana Use, and Past-Year Heavy Marijuana Use
| Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|
| CUD | ||||
| NIAAA | 0.54 (0.47–0.60) | 0.87 (0.85–0.89) | 0.40 (0.34–0.46) | 0.92 (0.91–0.94) |
| CRAFFT | 0.88 (0.83–0.92) | 0.80 (0.78–0.82) | 0.41 (0.36–0.45) | 0.98 (0.97–0.99) |
| PESQ-PS | 0.91 (0.86–0.94) | 0.79 (0.77–0.81) | 0.41 (0.37–0.46) | 0.98 (0.97–0.99) |
| AUDIT | 0.32 (0.26–0.39) | 0.96 (0.94–0.97) | 0.54 (0.44–0.62) | 0.90 (0.88–0.92) |
| Past-year marijuana use | ||||
| NIAAA | 0.39 (0.35–0.44) | 0.93 (0.92–0.95) | 0.77 (0.72–0.82) | 0.73 (0.70–0.75) |
| CRAFFT | 0.68 (0.64–0.72) | 0.92 (0.90–0.94) | 0.84 (0.80–0.87) | 0.83 (0.81–0.86) |
| PESQ-PS | 0.72 (0.68–0.75) | 0.93 (0.91–0.94) | 0.85 (0.81–0.88) | 0.85 (0.83–0.87) |
| AUDIT | 0.20 (0.17–0.24) | 0.99 (0.98–0.99) | 0.91 (0.84–0.95) | 0.68 (0.66–0.71) |
| Past-year heavy marijuana use | ||||
| NIAAA | 0.48 (0.42–0.54) | 0.88 (0.87–0.90) | 0.50 (0.44–0.56) | 0.88 (0.86–0.89) |
| CRAFFT | 0.84 (0.79–0.88) | 0.83 (0.81–0.85) | 0.54 (0.49–0.59) | 0.96 (0.94–0.97) |
| PESQ-PS | 0.86 (0.81–0.90) | 0.82 (0.80–0.84) | 0.54 (0.49–0.59) | 0.96 (0.95–0.97) |
| AUDIT | 0.29 (0.24–0.34) | 0.97 (0.96–0.98) | 0.67 (0.59–0.75) | 0.85 (0.83–0.87) |
95% confidence intervals obtained by using exact binomial confidence limits.42
Discussion
To our knowledge, this is the first study to use the DSM-5 AUD and CUD criteria to compare several different screeners for youth. We compared 4 adolescent screeners for various levels of alcohol and marijuana use and impairment with a large, racially and ethnically diverse multisite primary care population. The CRAFFT and PESQ-PS, which address AOD, identified about one-third of youth as at risk, the NIAAA SG identified ∼19%, and the AUDIT identified ∼8%. The CRAFFT and PESQ-PS had excellent sensitivity for detecting an AUD and also did well for CUD. The NIAAA SG, briefer and focused exclusively on alcohol, was better at identifying youth with an AUD versus a CUD, and the AUDIT had lower sensitivity for both disorders. Screeners had higher sensitivity for identifying youth who reported past-year heavy alcohol use compared with any past-year drinking. This result is not surprising given that one would generally expect youth with heavy drinking to be easier to identify as high risk compared with youth who have had any past drinking in the last year (which may include very light drinkers). In addition, screeners that addressed AOD versus only alcohol did better at identifying youth who reported heavy past-year marijuana use and any past-year marijuana use.
Overall, specificity was lower than sensitivity for all screeners for AUD and CUD, although it was still good. Given that the potential harm of a false positive is low in this setting, whereas risks associated with a false negative are high, providers might be more willing to accept lower specificity for high sensitivity to ensure that at-risk youth are identified. As expected, the PPV was better for all screeners in identifying past-year use for alcohol and marijuana use versus AUD and CUD because the PPV depends on both specificity of the test and prevalence of the condition. The NPV was also high for all screeners.
An important tradeoff in primary care settings is precision versus practicality. Incorporating screening into everyday practice can be difficult if the screener is long or not intuitive. Because of its brevity and focus on frequency of drinking, the NIAAA SG offers ease of administration, and results show that it is good at identifying youth with problematic drinking levels. However, the PESQ-PS and the CRAFFT correctly identify more at-risk youth for alcohol than the NIAAA SG. One recent study found that computer self-entry for the CRAFFT was valid and time-efficient.43 Other work has also shown that brief screens can be completed electronically as part of routine care.44 Future work is needed to elucidate how to most efficiently and accurately identify more at-risk youth in the primary care setting, including determining the best cutoff points to use to increase sensitivity.
Although the NIAAA SG performed well for alcohol outcomes, it did not do as well identifying youth who report marijuana use. This result is not surprising because the NIAAA SG questions focus on alcohol. However, identifying youth at risk for marijuana use is also important, particularly because it affects more domains of functioning in adolescence than alcohol.45 Furthermore, in this sample, 72% of youth who reported past-year alcohol use also reported past-year marijuana use, and CUD in this sample was >3 times as common as AUD, emphasizing the importance of asking about marijuana use in primary care settings. If a positive screen occurs, providers need to discuss potential harms of marijuana so youth better understand how the drug can affect functioning in both adolescence and adulthood. This discussion is particularly important because many youth view marijuana use as less harmful than alcohol use,46 perhaps because of continuing changes in state laws regarding medical and recreational use of marijuana. Interventions as brief as 15 minutes can lead to subsequent reductions in youth AOD use,47,48 but time constraints, issues of confidentiality, and knowledge of what to do after a positive a screen remain common barriers to screening in the primary care setting.49–51 Training designed to increase provider confidence in screening and discussing AOD use with at-risk youth52,53 and addressing confidentiality concerns51 is 1 approach to decreasing barriers, as is incorporating technology and providing computerized brief interventions in primary care settings for at-risk youth.54
Limitations of the current study include self-report, although self-report limitations are often exaggerated,55 and rates of AOD use in our sample were similar to national rates.41 The sample may also not be generalizable to all adolescents in primary care clinics; however, clinics did cross 2 states, and the sample was racially and ethnically diverse, with a wide range of ages. We also always administered the NIAAA SG to youth first, which could be a reason for lower sensitivity.43 Finally, we cannot speak to performance of these screeners as clinical tools. For example, knowing their doctor will see their answers might change an adolescent’s likelihood of disclosure. Future studies should examine screener performance under more realistic clinical conditions.
Conclusions
In sum, many youth were identified as at risk by these 4 screeners; however, some screeners performed better than others, depending on the substance and severity (eg, frequency of use versus a disorder). Future work could focus on quantifying tradeoffs in precision and practicality of these screeners in primary care.
Acknowledgments
The authors thank the clinics that participated and supported this project. We also thank Kirsten Becker and Jennifer Parker for overseeing the survey administrations and Michael Woodward for help with manuscript formatting.
Glossary
- AOD
alcohol and other drug
- AUD
alcohol use disorder
- AUDIT
Alcohol Use Disorders Identification Test
- CRAFFT
Car–Relax–Alone–Forget–Family and Friends–Trouble
- CUD
cannabis use disorder
- DISC-IV
Diagnostic Interview Schedule for Children Version IV
- DSM-5
Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition
- NIAAA SG
National Institute on Alcohol Abuse and Alcoholism Screening Guide
- NPV
negative predictive value
- PESQ-PS
Personal Experience Screening Questionnaire Problem Severity Scale
- PPV
positive predictive value
Footnotes
Dr D’Amico conceptualized and designed the study and drafted the initial manuscript; Dr Parast and Ms Ewing carried out initial analyses and reviewed and revised the manuscript; Drs Meredith, Shadel, and Stein reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
This trial has been registered at www.clinicaltrials.gov (identifier NCT01797835).
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by a grant from the National Institute of Alcohol Abuse and Alcoholism (R01AA021786) to Elizabeth D’Amico. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
References
- 1.Berg N, Kiviruusu O, Karvonen S, et al. A 26-year follow-up study of heavy drinking trajectories from adolescence to mid-adulthood and adult disadvantage. Alcohol Alcohol. 2013;48(4):452–457 [DOI] [PubMed] [Google Scholar]
- 2.Brook JS, Lee JY, Brown EN, Finch SJ. Comorbid trajectories of tobacco and marijuana use as related to psychological outcomes. Subst Abus. 2012;33(2):156–167 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.van Gastel WA, Tempelaar W, Bun C, et al. Cannabis use as an indicator of risk for mental health problems in adolescents: a population-based study at secondary schools. Psychol Med. 2013;43(9):1849–1856 [DOI] [PubMed] [Google Scholar]
- 4.Bloom B, Cohen RA, Freeman G. Summary health statistics for US children: National Health Interview Survey, 2007. Vital Health Stat 10. 2009;10(239):1–80 [PubMed] [Google Scholar]
- 5.Elster AB, Kuznets NK. Guidelines for Adolescent Preventive Services (GAPS): Recommendations and Rationale. Baltimore, MD: Williams & Wilkins; 1993 [Google Scholar]
- 6.Kokotailo PK; Committee on Substance Abuse . Alcohol use by youth and adolescents: a pediatric concern. Pediatrics. 2010;125(5):1078–1087 [DOI] [PubMed] [Google Scholar]
- 7.Johnson SB, Millstein SG. Prevention opportunities in health care settings. Am Psychol. 2003;58(6–7):475–481 [DOI] [PubMed] [Google Scholar]
- 8.Carballo JJ, Oquendo MA, Giner L, et al. Prevalence of alcohol misuse among adolescents and young adults evaluated in a primary care setting. Int J Adolesc Med Health. 2006;18(1):197–202 [DOI] [PubMed] [Google Scholar]
- 9.Van Hook S, Harris SK, Brooks T, et al. ; New England Partnership for Substance Abuse Research . The “Six T’s”: barriers to screening teens for substance abuse in primary care. J Adolesc Health. 2007;40(5):456–461 [DOI] [PubMed] [Google Scholar]
- 10.Gordon AJ, Ettaro L, Rodriguez KL, Mocik J, Clark DB. Provider, patient, and family perspectives of adolescent alcohol use and treatment in rural settings. J Rural Health. 2011;27(1):81–90 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Kelly SM, Gryczynski J, Mitchell SG, Kirk A, O’Grady KE, Schwartz RP. Validity of brief screening instrument for adolescent tobacco, alcohol, and drug use. Pediatrics. 2014;133(5):819–826 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ozer EM, Adams SH, Lustig JL, et al. Can it be done? Implementing adolescent clinical preventive services. Health Serv Res. 2001;36(6 pt 2):150–165 [PMC free article] [PubMed] [Google Scholar]
- 13.Klein JD, Allan MJ, Elster AB, et al. Improving adolescent preventive care in community health centers. Pediatrics. 2001;107(2):318–327 [DOI] [PubMed] [Google Scholar]
- 14.Chung PJ, Lee TC, Morrison JL, Schuster MA. Preventive care for children in the United States: quality and barriers. Annu Rev Public Health. 2006;27:491–515 [DOI] [PubMed] [Google Scholar]
- 15.Stevens GD, Shi L. Racial and ethnic disparities in the primary care experiences of children: a review of the literature. Med Care Res Rev. 2003;60(1):3–30 [DOI] [PubMed] [Google Scholar]
- 16.Millstein SG, Marcell AV. Screening and counseling for adolescent alcohol use among primary care physicians in the United States. Pediatrics. 2003;111(1):114–122 [DOI] [PubMed] [Google Scholar]
- 17.Halpern-Felsher BL, Ozer EM, Millstein SG, et al. Preventive services in a health maintenance organization: how well do pediatricians screen and educate adolescent patients? Arch Pediatr Adolesc Med. 2000;154(2):173–179 [DOI] [PubMed] [Google Scholar]
- 18.Ozer EM, Adams SH, Gardner LR, Mailloux DE, Wibbelsman CJ, Irwin CE Jr. Provider self-efficacy and the screening of adolescents for risky health behaviors. J Adolesc Health. 2004;35(2):101–107 [DOI] [PubMed] [Google Scholar]
- 19.NIAAA Alcohol Screening and Brief Intervention for Youth: A Practitioner’s Guide. Rockville, MD: NIAAA, DHHS and AAP; 2011. NIH publication 11-7805 [Google Scholar]
- 20.Clark DB, Martin CS, Chung T, et al. Screening for underage drinking and Diagnostic and Statistical Manual of Mental Disorders, 5th Edition alcohol use disorder in rural primary care practice. J Pediatr. 2016;173:214–220 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Chung T, Colby SM, Barnett NP, Rohsenow DJ, Spirito A, Monti PM. Screening adolescents for problem drinking: performance of brief screens against DSM-IV alcohol diagnoses. J Stud Alcohol. 2000;61(4):579–587 [DOI] [PubMed] [Google Scholar]
- 22.Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res. 2003;27(1):67–73 [DOI] [PubMed] [Google Scholar]
- 23.Shields AL, Campfield DC, Miller CS, Howell RT, Wallace K, Weiss RD. Score reliability of adolescent alcohol screening measures: a meta-analytic inquiry. J Child Adolesc Subst Abuse. 2008;17(4):75–97 [Google Scholar]
- 24.Reinert DF, Allen JP. The Alcohol Use Disorders Identification Test: an update of research findings. Alcohol Clin Exp Res. 2007;31(2):185–199 [DOI] [PubMed] [Google Scholar]
- 25.Cook RL, Chung T, Kelly TM, Clark DB. Alcohol screening in young persons attending a sexually transmitted disease clinic. Comparison of AUDIT, CRAFFT, and CAGE instruments. J Gen Intern Med. 2005;20(1):1–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Winters KC. Development of an adolescent alcohol and other drug abuse screening scale: Personal Experience Screening Questionnaire. Addict Behav. 1992;17(5):479–490 [DOI] [PubMed] [Google Scholar]
- 27.Winters KC, Kaminer Y. Screening and assessing adolescent substance use disorders in clinical populations. J Am Acad Child Adolesc Psychiatry. 2008;47(7):740–744 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.White HR, Labouvie EW. Towards the assessment of adolescent problem drinking. J Stud Alcohol. 1989;50(1):30–37 [DOI] [PubMed] [Google Scholar]
- 29.Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156(6):607–614 [DOI] [PubMed] [Google Scholar]
- 30.Cummins LH, Chan KK, Burns KM, Blume AW, Larimer M, Marlatt GA. Validity of the CRAFFT in American-Indian and Alaska-Native adolescents: screening for drug and alcohol risk. J Stud Alcohol. 2003;64(5):727–732 [DOI] [PubMed] [Google Scholar]
- 31.Subramaniam M, Cheok C, Verma S, Wong J, Chong SA. Validity of a brief screening instrument: CRAFFT in a multiethnic Asian population. Addict Behav. 2010;35(12):1102–1104 [DOI] [PubMed] [Google Scholar]
- 32.Mitchell SG, Kelly SM, Gryczynski J, et al. The CRAFFT cut-points and DSM-5 criteria for alcohol and other drugs: a reevaluation and reexamination. Subst Abus. 2014;35(4):376–380 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Orenstein A, Davis RB, Wolfe H; CASPAR Alcohol and Drug Education Program . Comparing screening instruments. J Alcohol Drug Educ. 2001;40(3):119–131 [Google Scholar]
- 34.Clark DB, Chung T, Martin C. Alcohol use frequency as a screen for alcohol use disorders in adolescents. Int J Adolesc Med Health. 2006;18(1):181–187 [DOI] [PubMed] [Google Scholar]
- 35.Miech RA, Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE. Monitoring the Future National Survey Results on Drug Use, 1975–2015. Vol. I: Secondary School Students. Ann Arbor, MI: Institute for Social Research, The University of Michigan; 2016 [Google Scholar]
- 36.Babor TF, de la Fuente JR, Saunders J, Grant MA. AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care. Geneva, Switzerland: World Health Organization; 1992 [Google Scholar]
- 37.Shaffer D, Fisher P, Lucas C, Comer J. Diagnostic Interview Schedule for Children (DISC-IV) Scoring Manual. New York, NY: NIMH DISC Editorial Board and Mark Davies; 2007 [Google Scholar]
- 38.Wasserman GA, McReynolds LS, Lucas CP, Fisher P, Santos L. The voice DISC-IV with incarcerated male youths: prevalence of disorder. J Am Acad Child Adolesc Psychiatry. 2002;41(3):314–321 [DOI] [PubMed] [Google Scholar]
- 39.NIMH NHANES data documentation National Institute of Mental Health Diagnostic Interview Schedule for Children (DISC) (YDQ): years of coverage: 1999–2004. Available at: www.cdc.gov/nchs/data/nhanes/limited_access/ydq.pdf. Accessed October 30, 2011
- 40.Shaffer D, Lucas CP, Richters JE. Diagnostic Assessment in Child and Adolescent Psychopathology. New York, NY: Guilford Press; 1999 [Google Scholar]
- 41.Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE. Monitoring the Future National Survey Results on Drug Use, 1975–2012. Vol. I: Secondary School Students. Ann Arbor, MI: Institute for Social Research, The University of Michigan; 2013 [Google Scholar]
- 42.Collett D. Modelling Binary Data. 2nd ed. Boca Raton, FL: CRC Press; 2003 [Google Scholar]
- 43.Harris SK, Knight JRJ Jr, Van Hook S, et al. Adolescent substance use screening in primary care: validity of computer self-administered versus clinician-administered screening. Subst Abus. 2016;37(1):197–203 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Levy S, Weiss R, Sherritt L, et al. An electronic screen for triaging adolescent substance use by risk levels. JAMA Pediatr. 2014;168(9):822–828 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.D’Amico EJ, Tucker JS, Miles JNV, Ewing BA, Shih RA, Pedersen ER. Alcohol and marijuana use trajectories in a diverse longitudinal sample of adolescents: examining use patterns from age 11 to 17 years. Addiction. 2016;111(10):1825–1835 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.D’Amico EJ, Houck JM, Hunter SB, Miles JNV, Osilla KC, Ewing BA. Group motivational interviewing for adolescents: change talk and alcohol and marijuana outcomes. J Consult Clin Psychol. 2015;83(1):68–80 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.D’Amico EJ, Miles JNV, Stern SA, Meredith LS. Brief motivational interviewing for teens at risk of substance use consequences: a randomized pilot study in a primary care clinic. J Subst Abuse Treat. 2008;35(1):53–61 [DOI] [PubMed] [Google Scholar]
- 48.Jensen CD, Cushing CC, Aylward BS, Craig JT, Sorell DM, Steele RG. Effectiveness of motivational interviewing interventions for adolescent substance use behavior change: a meta-analytic review. J Consult Clin Psychol. 2011;79(4):433–440 [DOI] [PubMed] [Google Scholar]
- 49.Ozechowski TJ, Becker SJ, Hogue A. SBIRT-A: adapting SBIRT to maximize developmental fit for adolescents in primary care. J Subst Abuse Treat. 2016;62:28–37 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Helitzer DL, Sussman AL, de Hernandez BU, Kong AS. The “ins” and “outs” of provider–parent communication: perspectives from adolescent primary care providers on challenges to forging alliances to reduce adolescent risk. J Adolesc Health. 2011;48(4):404–409 [DOI] [PubMed] [Google Scholar]
- 51.Committee on Adolescence American Academy of Pediatrics . Achieving quality health services for adolescents. Pediatrics. 2008;121(6):1263–1270 [DOI] [PubMed] [Google Scholar]
- 52.Lustig JL, Ozer EM, Adams SH, et al. Improving the delivery of adolescent clinical preventive services through skills-based training. Pediatrics. 2001;107(5):1100–1107 [DOI] [PubMed] [Google Scholar]
- 53.Buckelew SM, Adams SH, Irwin CE Jr, Gee S, Ozer EM. Increasing clinician self-efficacy for screening and counseling adolescents for risky health behaviors: results of an intervention. J Adolesc Health. 2008;43(2):198–200 [DOI] [PubMed] [Google Scholar]
- 54.Harris SK, Knight JR. Putting the screen in screening: technology-based alcohol screening and brief interventions in medical settings. Alcohol Res. 2014;36(1):63–79 [PMC free article] [PubMed] [Google Scholar]
- 55.Chan D. So why ask me? Are self-report data really that bad? In: Lance CE, Vandenberg RJ, eds. Statistical and Methodological Myths and Urban Legends: Doctrine, Verity and Fable in the Organizational and Social Sciences. New York, NY: Psychology Press; 2008:309–336 [Google Scholar]