Abstract
Background:
The CRAFFT, previously validated against DSM-IV diagnostic criteria, is the most widely used screening instrument for alcohol and other substance misuse in adolescents. The present secondary analysis study sought to compare the CRAFFT with the new DSM-5 diagnostic criteria in order to assess the CRAFFT’s psychometric properties and determine the optimal cut-point for identifying adolescents in need of further assessment.
Methods:
Participants were primary care patients ages 12-17 (N=525) who were recruited while waiting for a medical appointment in an urban federally qualified health center in Baltimore, Maryland, USA. Participants were administered the CRAFFT and the Composite International Diagnostic Interview, second edition, modified to include the new DSM-5 craving item. We examined the performance of the CRAFFT in identifying any problem use (defined as 1 or more DSM-5 criteria) and any DSM-5 substance use disorder (2 or more DSM-5 criteria) for alcohol or drugs other than tobacco. We examined sensitivity, specificity, and receiver operating characteristic areas under the curve (AUC) to determine the optimal CRAFFT cut-point(s) for predicting any problem use and any DSM-5 substance use disorder (SUD).
Results:
Examining the CRAFFT as a continuous measure, AUC values were 0.93 for problem use or higher, and 0.97 for DSM-5 SUD. Consistent with previously recommended cut-points for the CRAFFT, the cut-point of 2 performed optimally for identifying adolescents both exhibiting problem use of alcohol or drugs and meeting DSM-5 SUD criteria for alcohol or other drugs.
Conclusions:
Despite changes in the DSM substance use diagnostic criteria, the CRAFFT continues to demonstrate excellent sensitivity and specificity at its established cut-point of 2. Additional studies examining the CRAFFT in light of the new DSM-5 diagnostic criteria with more diverse populations are warranted.
INTRODUCTION
The American Academy of Pediatrics (AAP)1-3 and World Health Organization4 have recommended that health care providers routinely screen adolescents for alcohol and substance use. The CRAFFT5, a screening instrument that asks about problems related to use of alcohol or drugs (other than tobacco), is a widely used screen for alcohol and other substance misuse in adolescents6 and has been endorsed by the AAP for use with adolescents.1,2
A 2002 study by Knight and colleagues conducted among 14-18 year-olds attending routine healthcare appointments in a hospital-based clinic was the first to examine the validity of the CRAFFT.7 A CRAFFT score of 2 or higher was found to be optimal in identifying any substance-related problem (including alcohol or drugs; sensitivity=0.76; specificity=0.94), any Diagnostic and Statistical Manual, Fourth Edition (DSM-IV)8 substance use disorder (either abuse or dependence; sensitivity=0.80; specificity=0.86) and DSM-IV substance dependence (sensitivity=0.92; specificity=0.80). Subsequent studies of the CRAFFT in various clinical settings have found it to be valid and reliable with sensitivities ranging from .61 to 1.00 and specificities ranging from .33 to .97.9-15
Past studies that have evaluated the psychometric properties of the CRAFFT have used the DSM-IV. The DSM-5, published in May, 2013,16 included several major changes to substance use diagnoses, including merging DSM-IV abuse and dependence into a single diagnosis of substance use disorder (SUD), dropping the legal-involvement item, and adding an item about craving.16 The current study examined the performance of the CRAFFT relative to the DSM-5 diagnostic criteria.
METHODS
Participants
Participants were 12-17 years old, English speaking patients seeking medical services at one of three community health centers in Baltimore, USA (N= 525). Sixty percent were enrolled in high school, 37% were enrolled in middle school, and 13% reported not being enrolled in school.
Procedure
Adolescent patients were identified by clinic staff and approached by the research assistant in the waiting area at one of three primary health care clinics between June 2012 and February 2013. Patients in the targeted age range were invited to participate in an anonymous survey development study. Patients providing verbal assent were taken to a private room and administered several questionnaires by a trained interviewer. Participants received a $20US gift card to a sandwich shop chain. The study was approved by the Friends Research Institute Institutional Review Board with a waiver of written informed parental consent and written assent.
Measures
CRAFFT
The CRAFFT is a substance use screening instrument for adolescents that has substantial empirical support. 5,6,12 It contains 3 pre-screening questions inquiring about past year use of alcohol, cannabis, or other drugs and asks 6 yes/no questions about risk indicators or problems they may have experienced from alcohol or drug use (C= ridden in a CAR driven by someone intoxicated; R= use alcohol/drugs to RELAX; A= use alcohol/drugs while ALONE; F= FORGET things you did while intoxicated; F= FAMILY or friends tell you to reduce drinking/drug use; T= gotten into TROUBLE while using alcohol/drugs).
Modified Composite International Diagnostic Interview, second edition, Substance Abuse Module (CIDI-2 SAM)
Substance use disorders (SUDs) were determined using selected items from the CIDI-2 SAM (CIDI-2),17 which map to the diagnostic criteria for DSM-5 SUDs.18 Given the new SUD criteria in DSM-5 criteria, our scoring omitted the DSM-IV item on legal problems and included the CIDI-2 SAM item on craving. Consistent with the initial CRAFFT validation study,7 and because the CRAFFT scoring does not differentiate between alcohol and other drugs, we combined alcohol and drugs to determine general problem use (defined as 1 or more DSM-5 criteria for alcohol or other drugs) and substance use disorder (defined as 2 or more DSM-5 criteria for alcohol or other drugs). The CIDI-2 was administered regardless of the participant’s CRAFFT score.
Statistical Analysis
Consistent with the previous study of the CRAFFT by Knight et al.,7 we examined sensitivity, specificity, and receiver operating characteristic (ROC) areas under the curve (AUC) to determine the optimal CRAFFT score for predicting any problem use of alcohol or drugs and any DSM-5 SUD. We then examined sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the optimal CRAFFT score by gender. We could not examine CRAFFT scores by age categories due to low prevalence of problem use and SUD in the younger age group.
RESULTS
Participant Characteristics and Substance Use Patterns
A total of 584 adolescent patients were approached for study inclusion, of whom 525 agreed to participate (a 9.2% refusal rate). Participants were 54.5% female and 92% African American, representing the demographics of the larger adolescent patient population at the clinics. The sample was split in age range, with 50.9% ages 12-14, and 49% ages 15-17. During the past year, 162 (30.9%) reported using alcohol and/or drugs (9.7% alcohol only, 7.2% drugs only, 13.9% both). Sixty-five participants (12.4%) reported using alcohol or drugs but did not endorse any DSM-5 criteria, whereas 32 (6.1%) endorsed 1 symptom and were classified as experiencing “problem use”, and 65 (12.4%) met DSM-5 criteria for alcohol or drug SUD. Of the 65 participants who meet DSM-5 criteria for SUD, 23% met SUD criteria for both cannabis and alcohol, 63% met SUD criteria for cannabis only, and 14% met SUD criteria for alcohol only. Other drug SUDs were rare (n=3) and were not present in isolation of cannabis or alcohol SUD. The breakdown of substance use patterns by gender and age group is shown in Table 1. Rates of substance use, problem use, and DSM-5 SUD were higher in the 15-17 year olds than in the 12-14 year olds. Among the younger group, only 6 youth each had problem use or SUD (4.6%), compared to a third of the older-age subsample.
TABLE 1.
Alcohol and Other Drug Use Among 525 Adolescent Primary Care Patients
| No Use | Non-Problem Use (0 DSM 5 symptoms) |
Problem use (1 DSM 5 symptom) |
DSM-5 SUD (≥2 DSM 5 symptoms) |
|
|---|---|---|---|---|
| Total Sample (N= 525) | 363 (69.1) | 65 (12.4) | 32 (6.1) | 65 (12.4) |
| By Gender | ||||
| Male (n= 239) | 156 (65.3) | 30 (12.6) | 16 (6.7) | 37 (15.5) |
| Female (n= 286) | 207 (72.4) | 35 (12.2) | 16 (5.6) | 28 (9.8) |
| By Age Group | ||||
| Ages 12-14 (n= 267) | 229 (85.8) | 26 (9.7) | 6 (2.3) | 6 (2.3) |
| Ages 15-17 (n= 258) | 134 (51.9) | 39 (15.1) | 26 (10.1) | 59 (22.9) |
Optimal CRAFFT Cut-Points
Figure 1 shows ROC curves for the CRAFFT in predicting problem use or greater as determined by the CIDI-2 SAM. Examining the CRAFFT scores as a continuous measure, AUC values were 0.93 for problem use and 0.97 for DSM-5 SUD. Consistent with recommended cut-points, the cut-point of 2 performed optimally for both DSM-5 problem use and SUD (Table 2). At the CRAFFT score of 2, sensitivity and specificity for problem use or any DSM-5 SUD were .79 and .97, respectively, while sensitivity and specificity for SUD were .91 and .93, respectively.
FIGURE 1.
Receiver Operating Characteristics of the CRAFFT for DSM-5 problem use or higher and DSM-5 SUD for alcohol or other drugs.
TABLE 2.
Sensitivity and Specificity of CRAFFT Scores for DSM-5 Problem Use, DSM-5 SUD, and Moderate-Severe DSM-5 SUD
| Problem use or any DSM-5 SUD (≥1 DSM 5 symptom) |
DSM-5 SUD (≥2 DSM 5 symptoms) |
Moderate-severe DSM-5 SUD (≥4 DSM 5 symptom) |
||||
|---|---|---|---|---|---|---|
|
| ||||||
|
CRAFFT
Score |
Sensitivity | Specificity | Sensitivity | Specificity | Sensitivity | Specificity |
| 1 | .94 (0.89-0.98) |
.74 (0.70-0.78) |
1.00 (1.0-1.0) |
.70 (0.66-0.75) |
1.0 (1.0-1.0) |
0.66 (0.61-0.70) |
| 2 | .79 (0.71-0.87) |
.97 (0.95-0.98) |
.91 (0.83-0.97) |
.93 (0.90-0.95) |
0.88 (0.75-0.97) |
0.87 (0.84-0.90) |
| 3 | .57 (0.47-0.66) |
.99 (0.97-1.0) |
.74 (0.63-0.84) |
.97 (0.96-0.99) |
0.79 (0.63-0.91) |
0.93 (0.91-0.95) |
| 4 | .37 (0.28-0.46) |
1.00 (1.0-1.0) |
.52 (0.41-0.65) |
.99 (0.98-1.0) |
0.61 (0.44-0.77) |
0.97 (0.95-0.98) |
| 5 | .21 (0.13-0.29) |
1.00 (1.0-1.0) |
.31 (0.20-0.42)) |
1.00 (1.0-1.0) |
0.42 (0.24-0.6) |
0.99 (0.98-1.0) |
| 6 | .05 (0.01-0.10) |
1.00 (1.0-1.0) |
.08 (0.02-0.15) |
1.00 (1.0-1.0) |
0.15 (0.03-0.29) |
1.00 (1.0-1.0) |
| AUC | .93 | .97 | 0.95 | |||
Performance of the CRAFFT by Gender
Table 3 shows values for sensitivity, specificity, PPV, and NPV of the CRAFFT at a cut-point of 2 for the total sample and stratified by gender. There was substantial consistency in the sensitivity, specificity, PPV, and NPV across gender, indicating that CRAFFT performed equally well for males and females with respect to DSM-5-defined substance use risks.
TABLE 3.
Sensitivity, Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV) of the CRAFFT at Scores of 2 or Greater
| Sensitivity | Specificity | PPV | NPV | |
|---|---|---|---|---|
| Problem use or any DSM-5 SUD (≥1 DSM 5 symptom) | ||||
| Overall | 0.79 (0.71-0.87) |
0.97 (0.95-0.98) |
0.84 | 0.95 |
| Male | 0.81 (0.70-0.91) |
0.96 (0.92-0.98) |
0.84 | 0.95 |
| Female | 0.77 (0.64-0.89) |
0.97 (0.95-0.99) |
0.83 | 0.96 |
| DSM 5 SUD (≥2 DSM 5 symptoms) | ||||
| Overall | 0.91 (0.83-0.97) |
0.93 (0.90-0.95) |
0.64 | 0.99 |
| Male | 0.93 (0.78-0.98) |
0.92 (0.88-0.95) |
0.67 | 0.98 |
| Female | 0.89 (0.77-1.0) |
0.94 (0.91-00.97) |
0.61 | 0.99 |
| Moderate-severe DSM-5 SUD (≥4 DSM 5 symptoms) | ||||
| Overall | 0.88 (0.75-0.96) |
0.87 (0.84-0.90) |
0.32 | 0.99 |
| Male | 0.90 (0.65-0.99) |
0.84 (0.79-0.89) |
0.19 | 1.0 |
| Female | 0.87 (0.67-1.0) |
0.89 (0.86-0.93) |
0.14 | 1.0 |
DISCUSSION
The CRAFFT is a widely used screening tools for adolescent substance use in the US6 due to its brevity and ability to inform providers whether a longer conversation about drug and alcohol use is warranted. While screening tools such as the CRAFFT do not produce a formal diagnosis of SUD, if valid and reliable, they give the service provider the ability to focus attention on patients at greater risk, an important factor in busy medical settings where the majority of adolescent patients may not be misusing alcohol and/or illicit drugs.
Despite changes in the DSM substance use diagnostic criteria, the CRAFFT continues to demonstrate acceptable sensitivity and specificity levels with its established cut-point of 2 items.7 This is encouraging news for providers who have already integrated the CRAFFT into their screening protocols for adolescent patients, as protocol modifications and staff re-trainings will not be necessary in order to accommodate DSM-5 diagnostic criteria.
The sensitivity and specificity for identifying problem use or SUDs found in this study of the CRAFFT are nearly identical to those obtained in the original validity study using DSM-IV criteria.7 However, numerous methodological differences between our study and the original validation study, such as our inclusion of a broader adolescent age range and the use of the CIDI-II for DSM-5 diagnoses, make direct comparisons more difficult and are definite study limitations. Our sample also contained a very small number of 12-14 year olds who met any DSM-5 criteria, inhibiting our ability to conduct sub-analyses for the different age groups. In addition, this study was conducted with a mostly African-American sample in a single US city, further limiting its generalizability. Additional studies examining the CRAFFT with more diverse populations and larger sample sizes, which would permit the examination of subsamples, are warranted in light of the new DSM-5 diagnostic criteria.
Acknowledgments
FUNDING
Research reported in this publication was supported by the National Institute on Drug Abuse (NIDA) R01DA026003-03S1; (PI Schwartz). The content is solely the responsibility of the authors and does not necessarily represent the official views of NIDA or the National Institutes of Health.
REFERENCES
- 1.American Academy of Pediatrics Substance use screening, brief intervention, and referral to treatment for pediatricians. Pediatrics. 2011;126(5):e1330–e1340. doi: 10.1542/peds.2011-1754. [DOI] [PubMed] [Google Scholar]
- 2.American Academy of Pediatrics Committee on Substance Abuse. Policy statement--alcohol use by youth and adolescents: A policy concern. Pediatrics. 2010;125(5):1078–1087. doi: 10.1542/peds.2010-0438. [DOI] [PubMed] [Google Scholar]
- 3.Kulig JW. American Academy of Pediatrics Committee on Substance A. Tobacco, alcohol, and other drugs: the role of the pediatrician in prevention, identification, and management of substance abuse. Pediatrics. 2005;115(3):816–821. doi: 10.1542/peds.2004-2841. [DOI] [PubMed] [Google Scholar]
- 4.World Health Organization WHO ASSIST Project Factsheet. 2006 http://www.who.int/substance_abuse/activities/assist/en/index.html.
- 5.Knight JR, Shrier LA, Bravender TD, Farrell M, Vander Bilt J, Shaffer HJ. A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med. 1999;153(6):591–596. doi: 10.1001/archpedi.153.6.591. [DOI] [PubMed] [Google Scholar]
- 6.Pilowsky DJ, Wu LT. Screening instruments for substance use and brief interventions targeting adolescents in primary care: a literature review. Addict Behav. 2013;38(5):2146–2153. doi: 10.1016/j.addbeh.2013.01.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156(6):607–614. doi: 10.1001/archpedi.156.6.607. [DOI] [PubMed] [Google Scholar]
- 8.American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) Author; Arlington, VA: 1994. [Google Scholar]
- 9.Cook RL, Chung T, Kelly TM, Clark DB. Alcohol screening in young persons attending a sexually transmitted disease clinic. Comparison of AUDIT, CRAFFT, and CAGE instruments. J Gen Intern Med. 2005;20(1):1–6. doi: 10.1111/j.1525-1497.2005.40052.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Cummins LH, Chan KK, Burns KM, Blume AW, Larimer M, Marlatt GA. Validity of the CRAFFT in American-Indian and Alaska-Native adolescents: screening for drug and alcohol risk. J Stud Alcohol. 2003;64(5):727–732. doi: 10.15288/jsa.2003.64.727. [DOI] [PubMed] [Google Scholar]
- 11.Dhalla S, Zumbo BD, Poole G. A review of the psychometric properties of the CRAFFT instrument: 1999-2010. Curr Drug Abuse Rev. 2011;4(1):57–64. doi: 10.2174/1874473711104010057. [DOI] [PubMed] [Google Scholar]
- 12.Knight JR, Sherritt L, Harris SK, Gates EC, Chang G. Validity of brief alcohol screening tests among adolescents: a comparison of the AUDIT, POSIT, CAGE, and CRAFFT. Alcohol Clin Exp Res. 2003;27(1):67–73. doi: 10.1097/01.ALC.0000046598.59317.3A. [DOI] [PubMed] [Google Scholar]
- 13.Levy S, Sherritt L, Harris SK, et al. Test-retest reliability of adolescents' self-report of substance use. Alcohol Clin Exp Res. 2004;28(8):1236–1241. doi: 10.1097/01.alc.0000134216.22162.a5. [DOI] [PubMed] [Google Scholar]
- 14.Skogen JC, Boe T, Knudsen AK, Hysing M. Psychometric properties and concurrent validity of the CRAFFT among Norwegian adolescents. Ung@hordaland, a population-based study. Addict Behav. 2013;38(10):2500–2505. doi: 10.1016/j.addbeh.2013.05.002. [DOI] [PubMed] [Google Scholar]
- 15.Subramaniam M, Cheok C, Verma S, Wong J, Chong SA. Validity of a brief screening instrument-CRAFFT in a multiethnic Asian population. Addict Behav. 2010;35(12):1102–1104. doi: 10.1016/j.addbeh.2010.08.004. [DOI] [PubMed] [Google Scholar]
- 16.American Psychiatric Association . Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) Author; Arlington, VA: 2013. [Google Scholar]
- 17.Forman RF, Svikis D, Montoya ID, Blaine J. Selection of a substance use disorder diagnostic instrument by the National Drug Abuse Treatment Clinical Trials Network. J Subst Abuse Treat. 2004;27(1):1–8. doi: 10.1016/j.jsat.2004.03.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Hasin DS, O'Brien CP, Auriacombe M, et al. DSM-5 Criteria for Substance Use Disorders: Recommendations and Rationale. Am J Psychiatry. 2013;170(8):834–851. doi: 10.1176/appi.ajp.2013.12060782. [DOI] [PMC free article] [PubMed] [Google Scholar]