Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Jan 1.
Published in final edited form as: Subst Abus. 2015 Mar 16;37(1):197–203. doi: 10.1080/08897077.2015.1014615

Adolescent substance use screening in primary care: validity of computer self-administered vs. clinician-administered screening

Sion Kim Harris 1,2,3,4, John R Knight 1,2,3,4, Shari Van Hook 5, Lon Sherritt 1,2,4, Traci Brooks 6, John W Kulig 7, Christina Nordt 8, Richard Saitz 9,10
PMCID: PMC4573375  NIHMSID: NIHMS675435  PMID: 25774878

Abstract

Background

Computer self-administration may help busy pediatricians’ offices increase adolescent substance use screening rates efficiently and effectively, if proven to yield valid responses. The CRAFFT screening protocol for adolescents has demonstrated validity as an interview, but a computer self-entry approach needs validity testing. The aim of this study was to evaluate the criterion validity and time efficiency of a computerized adolescent substance use screening protocol implemented by self-administration or clinician-administration.

Methods

12- to 17-year-old patients coming for routine care at three primary care clinics completed the computerized screen by both self-administration and clinician-administration during their visit. To account for order effects, we randomly assigned participants to self-administer the screen either before or after seeing their clinician. Both were conducted using a tablet computer and included identical items (any past-12-month use of tobacco, alcohol, drugs; past-3-months frequency of each; and six CRAFFT items). The criterion measure for substance use was the Timeline Follow-Back, and for alcohol/drug use disorder, the Adolescent Diagnostic Interview, both conducted by confidential research assistant-interview after the visit. Tobacco dependence risk was assessed with the self-administered Hooked on Nicotine Checklist (HONC). Analyses accounted for the multi-site cluster sampling design.

Results

Among 136 participants, mean age was 15.0±1.5 yrs, 54% were girls, 53% were Black or Hispanic, and 67% had ≥3 prior visits with their clinician. Twenty-seven percent reported any substance use (including tobacco) in the past 12 months, 7% met criteria for an alcohol or cannabis use disorder, and 4% were HONC-positive. Sensitivity/specificity of the screener were high for detecting past-12-month use or disorder and did not differ between computer and clinician. Mean completion time was 49 seconds (95%CI 44-54) for computer and 74 seconds (95%CI 68-87) for clinician (paired comparison p<0.001).

Conclusions

Substance use screening by computer self-entry is a valid and time-efficient alternative to clinician-administered screening.

Keywords: Adolescents, substance use, primary care, screening, validity (epidemiology), computers, alcohol, tobacco, cannabis, drugs

INTRODUCTION

Adolescent substance use is strongly linked to a spectrum of serious health risks and problems, making routine pediatric office visits a promising venue for screening and brief counseling.1 Yet screening rates are low among pediatricians,2,3 who must see patients quickly and have a growing list of recommended behavioral health assessments. One potential solution is to use a computer system that allows adolescent patients to complete the screen before the clinician visit and yields a report for the clinician to review with the patient during the visit. Adolescents may be more likely to disclose sensitive information if there is some distance between the screening and talking with their clinician, and it leaves more time during the visit for the clinician to conduct further assessment and offer counseling.4 A computerized system also allows multiple health-risk behaviors (e.g., tobacco, alcohol, and drug use) to be combined into a single screener, and the use of automatic skip patterns increases item clarity and improves overall time efficiency. Computerization improves standardization of screening, and integration with electronic health records may boost screening and documentation rates.4-8 Studies of computerized screening of adolescents in medical settings generally show strong acceptability and feasibility among both patients and providers.4,7,9,10

The CRAFFT is a substance use screening protocol specifically designed for adolescents, and is brief enough for busy clinicians to use.11 It is the screener most thoroughly studied among adolescents12, and the most widely recommended in the U.S. and abroad. 13-15 The CRAFFT, as originally developed, included six yes/no questions referred to by the mnemonic CRAFFT (Car, Relax, Alone, Forget, Family/friends, and Trouble). A score of 2 or more “yes” responses to these six items was found to have the best combination of sensitivity (80%) and specificity (86%) for detecting a substance use disorder in a primary care population.12,16 The CRAFFT screen has since been expanded to begin with a set of opening questions that ask about any past-12-month alcohol, cannabis or other drug use, followed by the six original items (for current version, see http://www.ceasar-boston.org/CRAFFT/pdf/CRAFFT_English.pdf). These opening questions give clinicians important information on usage, facilitate early intervention, and allow a skip-pattern for non-users that minimizes confusion and improves time-efficiency. More recently, we further expanded the CRAFFT screening protocol to include items on tobacco use, since tobacco is the leading cause of mortality in the US17, and on the past-3-month frequency of use of each substance (i.e., recent consumption pattern).

While the original six-item CRAFFT screen has demonstrated criterion validity for identifying adolescents with a substance use disorder16, the added questions on substance use and frequency need validation. Moreover, the original CRAFFT was tested as an interview, and a computerized self-entry version also needs validation. To that end, we developed a computerized screening program, based on user feedback and revision, that is optimized to run on an iPad or similar tablet device, and can be used before the clinician encounter (i.e., teens read the questions and directly enter their responses), or during the visit (i.e., clinicians read the questions aloud and enter the teen’s responses, hereafter referred to as “clinician-administration” [CA]). The objective of this study was to evaluate the criterion validity of this updated CRAFFT screening protocol in computer self-administered and clinician-administered modes. Specifically, we evaluated validity of the two screening modes to identify substance use and disorders among 12- to 17-year-old primary care patients. We also examined the time required by patients to complete the screening in each mode.

METHODS

We conducted this multi-site study during 2012-2013 at three large urban, teaching hospital-affiliated primary care offices located in Massachusetts (two adolescent clinics and one general pediatric practice), with 9 clinicians (8 attending physicians, 1 nurse practitioner) participating.

The recruitment procedure was identical at all sites. Patients aged 12- to 17-years arriving for well-visits, who were medically and emotionally stable on the day of the visit, and able to read and understand English were eligible. Minimum participant age was 13 at the adolescent clinics, and 12 at the general pediatric practice. Research assistants (RAs) worked with clinic staff to identify eligible patients with upcoming well-visit appointments whom they then recruited either through phone contact before the visit or upon arrival at the clinic. Recruitment was limited to patients of participating clinicians, and was guided by an age- and gender-stratified recruitment table to ensure equal numbers of participants across the age-gender groups in the sample. At the time of visit check-in, RAs explained the study purpose, procedures, and confidentiality protections, answered questions, and obtained written adolescent assent. Assenting patients then immediately completed a short six-item computerized demographics questionnaire which assessed gender, age, grade in school, two socioeconomic status indicators (number of parents/guardians living at patient’s primary home, highest education level completed by any parents/guardians living at home), and race/Hispanic ethnicity. All participants were screened by both computer tablet-based self-administration (SA) and clinician-administration (CA). RAs provided participants with a brief orientation on how to complete the tablet screening program, and clinicians received training on the screening protocol and tablet computer program during one of their weekly staff meetings. To account for possible order effects, we randomly assigned participants to complete the SA screen either before or after the visit and CA screening. When SA occurred first, clinicians were “blinded” to the results of the SA screen.

Participants received a $25 merchandise certificate for study completion. The institutional review boards of the lead coordinating institution and each recruitment site approved the study design and protocol and authorized a waiver of the requirement for parental consent for adolescents under 18 years of age in accordance with published guidelines for adolescent health research.18

Computerized Screening Program

The SA and CA modes used the same computer program and thus included identical items in the same order and format. The opening questions on any past-12-month use of tobacco, alcohol, cannabis, and other drugs (e.g., “During the past 12 months, did you ever drink any alcohol? Don’t count one or two sips taken during religious or family events.”) were presented in a grid on the first page. The opening questions were followed by a past-3-month frequency item for each substance used (e.g., “During the past 3 months, about how often did you …?”; response scale “Never,” “Once or Twice,” “Monthly,” “Weekly,” “Almost Daily,” “Daily” derived from the Alcohol, Smoking and Substance Involvement Screening Test [ASSIST])19,20; and then by the six original CRAFFT11,16 questions. The “Car” question (driving while impaired/riding with an impaired driver) was asked of all patients, and the RAFFT questions only of those reporting past-12-month use of substances other than tobacco.

In the CA mode, the primary clinician read the questions aloud from the iPad and entered the patient’s responses. The computer screening program allowed items to be skipped, but an automated reminder message highlighted the skipped item before going to the next screen. The program automatically recorded the user-elapsed times for each click and each page which were summed to determine the total screening completion time.

Validation Measures

Our criterion measure for past-12-month and past-90-day substance use was an adapted Timeline Follow-Back (TLFB) interview.21,22 The TLFB is a reliable and valid method used extensively in substance use studies of both adults and adolescents to measure self-reported frequency and quantity of substance use during a defined time period. By utilizing a calendar and specific memory aids to enhance recall, it has been shown to yield more precise estimates of use than simple quantity and frequency questions23-25. The TLFB has also been shown to have good validity compared to biomarkers such as salivary cotinine in the assessment of smoking.26

To assess sensitivity/specificity of the screening items to detect substance use disorders, we used the Adolescent Diagnostic Interview (ADI)27-29, a structured interview for identifying a substance abuse or dependence disorder based on DSM-IV criteria. We chose the ADI to allow direct comparison of validity statistics between this study and a prior CRAFFT validation study16 in which the ADI and screening were conducted by RA interview. In a prior validation study, ADI sensitivity/specificity for detecting alcohol abuse was .87/.87, alcohol dependence .90/.95, cannabis abuse .85/.92, and cannabis dependence .92/.92.27

Both the TLFB and ADI were administered by a trained research assistant in a confidential interview immediately after the visit and completion of both screening modes. Research assistants were blind to the results of both the self-administered and clinician-interview screening. To identify risk for tobacco dependence, we administered the Hooked on Nicotine Checklist, a 10-item nicotine dependence screen previously shown to be sensitive to loss of autonomy over tobacco use, even at the early stages, and to be valid for use with adolescents ages 12 and older.30-32

Post-Visit Ratings of Care

Because the patient-provider relationship could affect the likelihood of patient disclosure of sensitive information such as substance use, participants also completed a brief post-visit questionnaire consisting of items assessing the number of prior visits with the clinician and how connected they felt to that clinician, which was assessed using the Youth Connectedness to Provider scale,33, consisting of 7 items (e.g., “How much do you feel that this doctor cares about you?”) and a five-point response scale ranging from “1=Not at all” to “5=Very much.” Item scores were summed to form a total score which ranged from 7 to 35.

Data Analyses

To validate the opening questions, we collapsed TLFB data into a dichotomous variable for any/no past-12-month use of each substance, and examined sensitivity (% of true positives identified by the screen as positive) and specificity (% of true negatives that screen negative) of each opening question compared to these dichotomous variables. To validate the past-3-month frequency items, we collapsed TLFB data for the most recent 3-month period into five categories so as to match the frequency response scale on the screening tool (i.e., “None”=0 days, “Once or twice”=1-2 days, “Monthly” =3-12 days, “Weekly”=13-60 days, “Almost daily/Daily”=≥61 days). We then conducted a paired comparison of the screening responses and the TLFB variables using the Wilcoxon signed-ranks test.

To evaluate validity for identifying a substance use disorder, we computed dichotomous variables derived from the ADI for any/no alcohol or cannabis abuse or dependence according to DSM-IV criteria. Loss of autonomy over tobacco use (dependence risk) was defined as a score≥1 on the Hooked on Nicotine Checklist30. We calculated sensitivity, specificity, positive likelihood ratios (sensitivity/1-specificity, or ratio of the true positive rate over false positive rate), and negative likelihood ratios (1-sensitivity/specificity, or ratio of the false negative rate over true negative rate). We calculated 95% confidence intervals (95%CI) for each validity statistic using SUDAAN® v.10.0 to account for correlated error arising from our multi-site cluster-sampling design.

To determine whether screening times differed between self-administered and clinician-interview modes, we conducted generalized linear mixed modeling with site and participant as random effects and screening mode as the predictor variable.

RESULTS

Sample Characteristics

Of 141 eligible invited patients, 139 agreed (98.6% participation rate), and 136 (97.8% of agreed) had complete data for analysis. Sixty-nine patients completed the SA screening first, while 67 patients received the CA first. The overall sample was slightly more than half girls, had a mean ± standard deviation age of 15.0 ± 1.5 years, and was racially and socioeconomically diverse (Table 1). Nearly equal numbers of patients saw an adolescent medicine specialist vs. a general pediatrician. Over two-thirds of participants had ≥3 prior visits with their clinician, and scores tended to be high on the Youth Connectedness to Provider scale. There were no differences in demographic characteristics or visit ratings between the SA-first and CA-first groups.

Table 1.

Characteristics of the sample (N=136).

Total
N (%)
Demographics
Females 74 (54)
Age group
 12-13 27 (19)
 14-15 55 (40)
 16-17 54 (40)
Race/ethnicity
 White non-Hispanic 25 (18)
 Black non-Hispanic 38 (28)
 Hispanic 33 (25)
 Asian 16 (12)
 Other/Multi-race 24 (18)
College graduate parent(s) 68 (58)
Two parents at home 76 (56)
Visit Characteristics
Clinician type
 Adolescent medicine specialist 66 (48)
 General pediatrician 70 (52)
≥3 prior visits with cliniciana 90 (67)
Youth Connectedness to Provider Scale scorea
 (median, IQR; scale score range 7-35)
33 (31-35)
Substance Use Prevalenceb
Tobacco
 Any past-12-month use 7 (5)
 Any past-3-month use 4 (3)
 Monthly or more frequentd 2 (2)
 Hooked on Nicotine Checklist-positivec 6 (4)
Alcohol
 Any past-12-month use 29 (21)
 Any past-3-month use 21 (15)
 Monthly or more frequentd 8 (7)
 Disorder 4 (3)
Cannabis
 Any past-12-month use 25 (18)
 Any past-3-month use 20 (15)
 Monthly or more frequentd 15 (11)
 Disorder 8 (6)

Abbreviations: IQR: interquartile range; SD: standard deviation

* Paired t-test, two-tailed p<0.01

a

n=134 with completed post-visit questionnaires

b

Based on Timeline Follow-Back and Adolescent Diagnostic Interview

c

Hooked on Nicotine Checklist score ≥1 indicates tobacco dependence risk

d

During past 3 months

Overall, 27% reported use of any substance (including tobacco) in the past 12 months according to Timeline Follow-back data. Alcohol was the most prevalent, followed by cannabis (Table 1). Only four participants reported use of drugs other than cannabis; three had non-medical use of prescribed or over-the-counter medications and one used an illicit drug other than cannabis. Because of these small numbers, we were unable to evaluate validity for specific drugs other than cannabis. On the six CRAFFT items, 15 (11%) participants had a score ≥2 on the SA screen, and 13 (10%) on the CA. Interestingly, for the “Car” item, youth disclosed this sensitive behavior more often on the SA screen (n=26) than on the CA (n=19). On the ADI, 9 participants (7%) met DSM-IV criteria for a substance use disorder (i.e., abuse or dependence), while an additional 16 (11%) reported experiencing at least one substance-related problem but did not meet full criteria for a disorder. Six participants (4%) screened positive on the Hooked on Nicotine Checklist.

Screening Time

Screening completion time differed significantly between the two modes with SA taking, on average, 49 seconds (95%CI 44-54) compared to 74 seconds (95%CI 68-87) for CA (corrected F-statistic=140.5, p<0.001). For participants reporting no past-12-month substance use, average SA and CA screening times were 40 (95%CI 36-44) and 62 (95%CI 52-73) seconds, respectively, while for those reporting any past-12-month use, the times were about twice as long (73 [95%CI 63-83] and 116 [95%CI 99-133] seconds).

Criterion Validity for Past-12-Month Use

Disclosure of any past-12-month substance use was highest on the confidential TLFB interview (35 reported any use compared to 30 for SA and 33 for CA). Sensitivity and specificity for the opening questions did not differ significantly between SA and CA, but varied by substance, with the alcohol and cannabis items having lower sensitivities compared to tobacco (Table 2). Among the 35 participants reporting past-12-month use on the TLFB, 11 were missed by either the SA or CA screening. Five of the 11 were missed by both and all had no more than two days of use in the past year. Among the remaining six that were detected by only one of the screening modes, five disclosed on the second screen (4 disclosed alcohol use, 2 cannabis use), suggesting a possible order effect. Among the original 11 missed, only three had >6 days of use in the past 12 months, and all three were identified in the CA mode.

Table 2.

Detection of past-12-month substance use: criterion validitya of computer Self-administered vs. Clinician-administered screening.

Sensitivity
%
(95% CI)
Specificity
%
(95% CI)
Self Clinician Self Clinician
Tobacco 86
(41-98)
86
(41-98)
98
(93-99)
98
(93-99)
Alcohol 62
(44-78)
69
(50-83)
98
(93-100)
96
(90-99)
Cannabis 72
(52-86)
79
(58-91)
100 99
(94-100)
a

Criterion measure: research assistant-administered Timeline Follow-Back Calendar interview

Criterion Validity for Past-3-Month Frequency

One in five participants (n=28, 21%) reported using tobacco, alcohol, or cannabis at least once in the past 3 months on the TLFB (data not shown). Eighteen (13%) reported at least weekly use, and 10 (7%) near-daily or daily use. We found no differences between past-3-month frequency of tobacco and alcohol use on the SA and CA screens compared to the TLFB. For cannabis use, however, we found a tendency to under-report frequency on the screens compared to the TLFB. On the SA screen, 11 respondents reported a lower frequency of cannabis use than indicated in their TLFB data, while four reported higher frequency (Wilcoxon signed-rank test z-statistic= −2.15, p=0.03). Similarly, on the CA screen, eight reported lower frequency compared to their TLFB report, while three reported higher (z-statistic= −1.71, p=0.09).

Validity for Detecting a Substance Use Disorder

Test characteristics for detecting a substance use disorder were generally similar between SA and CA for all three components of the CRAFFT screening protocol (i.e., past-12-month use, past-3-month frequency, and score on the six original items) (Table 3). Because of the small numbers with ≥weekly use in this sample (n=9), we collapsed the “monthly,” “weekly” and “almost daily/daily” categories for these analyses. The discrepancy in sensitivities between SA and CA for ≥monthly use of alcohol is attributable to a difference of only two respondents, so this result should be viewed with caution. The past-12-month use items had the highest sensitivities (i.e., the fewest missed positives) across the screening variables compared (Table 3). Not surprisingly, more frequent use (≥monthly) was associated with the highest specificities (i.e., the fewest false positives) and positive likelihood ratios for identifying a disorder. A CRAFFT score≥2 had specificities that were as high as the past-3-month frequency variables, but had better sensitivities.

Table 3.

Detection of a substance use disorder: sensitivity, specificity, and likelihood ratios of computer Self-administered vs. Clinician-administered screening.

Sensitivity
%
Specificity
%
Likelihood Ratio-
Positive
Likelihood Ratio-
Negative
Self Clinician Self Clinician Self Clinician Self Clinician
Tobaccoa
 Any past-12-month use 83
(36-98)
83
(36-98)
97
(92-99)
98
(93-99)
27
(10-76)
36
(11-117)
0.2
(0.0-1.0)
0.2
(0.0-1.0)
 Any past-3-month use 83
(36-98)
83
(36-98)
99
(94-100)
99
(94-100)
54
(13-224)
54
(13-226)
0.2
(0.0-1.0)
0.2
(0.0-1.0)
 Monthly or more frequentc 67
(26-92)
50
(16-84)
99
(95-100)
100 87
(11-662)
-- 0.3
(0.1-1.0)
0.5
(0.2-1.1)
Alcoholb
 Any past-12-month use 100 100 88
(81-93)
85
(78-90)
8
(4-12)
7
(4-10)
0.0
(0.0-1.6)
0.0
(0.0-1.6)
 Any past-3-month use 100 100 92
(85-95)
89
(83-94)
12
(6-19)
9
(5-15)
0.0
(0.0-1.5)
0.0
(0.0-1.6)
 Monthly or more frequentc 100 75
(23-97)
95
(89-97)
94
(88-97)
19
(8-34)
12
(5-30)
0.0
(0.0-1.5)
0.3
(0.1-1.5)
 CRAFFT ≥2 100 100 92
(85-95)
93
(87-96)
12
(6-19)
15
(6-25)
0.0
(0.0-1.5)
0.0
(0.0-1.5)
Cannabisb
 Any past-12-month use 100 100 92
(86-96)
90
(83-94)
13
(6-21)
10
(5-15)
0.0
(0.0-0.9)
0.0
(0.0-1.0)
 Any past-3-month use 88
(46-98)
86
(41-98)
93
(86-96)
91
(85-95)
11
(6-21)
10
(5-19)
0.1
(0.0-0.9)
0.2
(0.0-1.0)
 Monthly or more frequentc 63
(28-88)
57
(23-86)
97
(92-99)
95
(90-98)
20
(7-60)
12
(4-33)
0.4
(0.2-1.0)
0.5
(0.2-1.1)
 CRAFFT ≥2 88
(46-98)
88
(46-98)
94
(88-97)
95
(90-98)
14
(7-29)
19
(8-42)
0.1
(0.0-0.8)
0.1
(0.0-0.8)
a

Criterion measure: Hooked on Nicotine Checklist.

b

Criterion measure: Adolescent Diagnostic Interview.

c

During the past 3 months.

DISCUSSION

This study shows that computer self-administration and clinician-administration of the CRAFFT screening protocol are equally valid ways to screen adolescent primary care patients for substance use and disorders, but that computer self-administration (SA) is more time-efficient. Given the lengthy list of recommended health-risk screenings, pediatric primary care clinicians typically have only 2- to 3-minutes total to spend on both substance use screening and brief counseling. Computer self-administered screening before the visit would allow them to devote all of this time to counseling, and further assessment as needed. We have previously shown that a computerized CRAFFT screen, coupled with informational feedback and 2- to 3-minutes of clinician counseling significantly reduced patients’ substance use at 3- and 12-month follow-ups.10

The current study suggests that the opening questions on past-12-month alcohol and cannabis use may need to be improved because of their low sensitivity (<80%) relative to the TLFB criterion standard, particularly in the SA mode. One possible reason for this poor performance was our presentation of these items in a grid format on a single page. This type of formatting has been found to take less time to complete than one-question-at-a-time presentation.34 However, this formatting has also been found to be associated with “straight-lining” or “non-differentiation” of responses, and therefore, increased measurement error.34 This phenomenon has been found with grids containing as few as four items.35,36 The use of yes/no questions may also have contributed to lower sensitivity. Prior studies suggest that yes/no questions have greater potential for “motivated underreporting” and social desirability bias, particularly regarding sensitive topics, than questions that ask “how many” or “how often” which implicitly convey an expectation of the behavior.34

Therefore, an alternative approach would be to begin the screen with past-12-month frequency items; i.e., “During the past 12 months, on how many days did you use [substance name]?” with either a write-in response text field, or a frequency response scale (e.g., Never, Less than monthly, Monthly, About weekly, etc.). Including the instruction “Please check ‘Never’ if you haven’t used” would be important to convey that non-use is also normative. Past-12-month frequency items would also obviate the need to ask about past-3-month frequency, thus shortening the screen. The past 12 months is the preferred timeframe for these opening questions, having higher sensitivities for detecting a disorder than past-3-month frequency. For an initial screen, the highest sensitivity is preferred, even at the expense of lower specificity (i.e., more false-positive results), to avoid missing anyone who may be at risk. False-positive results can then be clarified by further assessment with questions that have high specificity. Interestingly, a CRAFFT score ≥2 had similar specificity, but better sensitivity, for detecting a disorder compared to any past-3-month use, thus making the past-3-month frequency items redundant.

One screening item for which the SA mode appears to have an advantage over the CA mode is the substance-related riding/driving risk (“Car”) item. We found greater disclosure of this dangerous behavior in the SA mode, a finding consistent with previous studies that showed that less socially desirable behaviors were more often reported in response to self-administered questionnaires compared to face-to-face interviews.37-39

Strengths of our study include a racially and ethnically diverse sample; randomization to account for a possible order effect of screening modes; and completion of both self-entry and clinician-screening by all patients allowing for direct comparison. Our findings also provide additional evidence for the validity of the six CRAFFT items for detecting substance use disorders, regardless of screening method, which is consistent with a growing body of research in a wide variety of adolescent populations12 and languages.40-43

Some potential study limitations warrant consideration. We recruited from well-visits only (i.e., not from urgent care), the sample size was small, and the prevalence of use was low. In particular, the results for tobacco should be viewed with caution due to its low prevalence in this sample. The psychometric properties described here may not be generalizable to other modes of administration (e.g., paper self-administered), or other adolescent populations such as those with more severe substance use. Moreover, screening participants twice in the same study session may have altered screener performance, compared to being screened once. The clinicians participating in this study tended to be experienced pediatric clinicians who had long-standing relationships with many of their patients. Validity of clinician-administered screening might be different in general pediatric or family medicine practices.

Based on our study findings, we recommend computerized self-administered screening as a time-efficient, and similarly valid, alternative to clinician-administration. The enhanced, computerized CRAFFT screening protocol now assesses past-12-month use, safety risk (Car question), and problematic use (RAFFT items). Future studies should examine whether single-item questions on past-12-month use frequency improves sensitivity over the yes/no “any use” items in detecting use, as well as examine validity compared to DSM-V criteria. There is an urgent need for efficient, computerized office systems for screening, brief intervention, and referral to treatment in pediatric primary care settings. Substance addiction is a costly disease with a pediatric onset. A pediatrician screening and early intervention protocol that is effective and time-efficient could be widely applied, with substantial benefits to public health.

ACKNOWLEDGEMENTS

The authors thank the physicians and staff at our three participating clinics for assistance with study implementation, including Julia Lee, PNP MPH, Department of Pediatrics, Tufts Medical Center-Floating Hospital for Children; Deborah Glotzer, MD, Ellen Sigman, MD, Margaret Coleman, MD, Paul Geltman, MD, and Jeanette Callahan, MD, at Cambridge Pediatrics, Cambridge Health Alliance; and JoséAlberto Betances, MD, Department of Pediatrics-Adolescent Center, Boston Medical Center. The authors also acknowledge research assistants Rebecca Adams, Tatyana Buynitsky, Kateryna Kuzubova, and Roman Pavlyuk for their invaluable help with data collection. Finally, the authors thank the adolescent patients who agreed to participate.

FUNDING

This study was supported by Grant 3R01DA025068-05S1 from the National Institute on Drug Abuse. The National Institute on Drug Abuse (NIDA) was involved in the review and approval of study design and conduct; review and approval of the manuscript for publication; and the decision to submit the manuscript for publication. NIDA was not involved in the collection, management, analysis, or interpretation of the data. Other support was provided by Grant K07 AA013280 from the National Institute on Alcohol Abuse and Alcoholism (JK); Grants T20MC07462 (JK, SVH) and #T71NC0009 (SKH) from the Maternal and Child Health Bureau. No author has any conflict of interest or financial relationship relevant to this article to disclose.

REFERENCES

  • 1.Centers for Disease Control and Prevention Youth Risk Behavior Surveillance - United States, 2011. MMWR Surveill Summ. 2012;61(SS-4):1–162. [PubMed] [Google Scholar]
  • 2.Ozer EM, Adams SH, Lustig JL, et al. Increasing the screening and counseling of adolescents for risky health behaviors: a primary care intervention. Pediatrics. 2005;115(4):960–968. doi: 10.1542/peds.2004-0520. [DOI] [PubMed] [Google Scholar]
  • 3.Harris SK, Herr-Zaya K, Weinstein Z, et al. Results of a statewide survey of adolescent substance use screening rates and practices in primary care. Subst Abus. 2012;33(4):321–6. doi: 10.1080/08897077.2011.645950. doi:10.1080/08897077.2011.645950. [DOI] [PubMed] [Google Scholar]
  • 4.Chisolm D, Gardner W, Julian T, Kelleher K. Adolescent satisfaction with computer-assisted behavioral risk screening in primary care. Child Adolesc Ment Heal. 2008;13(4):163–168. doi: 10.1111/j.1475-3588.2007.00474.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Tai B, Wu L-T, Clark H. Electronic health records: essential tools in integrating substance abuse treatment with primary care. Subst Abuse Rehabil. 2012;3:1–8. doi: 10.2147/SAR.S22575. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Olson AL, Gaffney C, Hedberg V, Gladstone GR. Use of inexpensive technology to enhance adolescent health screening and counseling. Arch Pediatr Adolesc Med. 2009;163(2):172–7. doi: 10.1001/archpediatrics.2008.533. doi:10.1001/archpediatrics.2008.533. [DOI] [PubMed] [Google Scholar]
  • 7.Stevens J, Kelleher KJ, Gardner W, et al. Trial of computerized screening for adolescent behavioral concerns. Pediatrics. 2008;121(6):1099–1105. doi: 10.1542/peds.2007-1878. doi:10.1542/peds.2007-1878. [DOI] [PubMed] [Google Scholar]
  • 8.Anand V, Carroll AE, Downs SM. Automated primary care screening in pediatric waiting rooms. Pediatrics. 2012;129(5):e1275–81. doi: 10.1542/peds.2011-2875. doi:10.1542/peds.2011-2875. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Kypri K, Langley JD, Saunders JB, Cashell-Smith ML, Herbison P. Randomized controlled trial of web-based alcohol screening and brief intervention in primary care. Arch Intern Med. 2008;168(5):530–6. doi: 10.1001/archinternmed.2007.109. doi:10.1001/archinternmed.2007.109. [DOI] [PubMed] [Google Scholar]
  • 10.Harris S, Csémy L, Sherritt L, et al. Computer-facilitated substance use screening and brief advice for teens in primary care: an international trial. Pediatrics. 2012;129(6):1072–82. doi: 10.1542/peds.2011-1624. doi:10.1542/peds.2011-1624. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Knight JR, Shrier LA, Bravender TD, et al. A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med. 1999;153(6):591–596. doi: 10.1001/archpedi.153.6.591. [DOI] [PubMed] [Google Scholar]
  • 12.Dhalla S, Zumbo BD, Poole G, Poolem G. A review of the psychometric properties of the CRAFFT instrument. Curr Drug Abus Rev. 2011;4(1):57–64. doi: 10.2174/1874473711104010057. [DOI] [PubMed] [Google Scholar]
  • 13.Levy SJ, Kokotailo PK. Substance use screening, brief intervention, and referral to treatment for pediatricians. Pediatrics. 2011;128(5):e1330–40. doi: 10.1542/peds.2011-1754. doi:peds.2011-1754 [pii] 10.1542/peds.2011-1754. [DOI] [PubMed] [Google Scholar]
  • 14.National Institute on Alcohol Abuse and Alcoholism . Alcohol Screening and Brief Intervention for Youth: Practitioner’s Guide. National Institutes of Health, Department of Health and Human Services; Washington, D.C.: 2011. [Google Scholar]
  • 15.Hagan JF, Shaw JS, Duncan P. Bright Futures Guidelines for Health Supervision of Infants, Children, and Adolescents. (Third) 2008 [Google Scholar]
  • 16.Knight JR, Sherritt L, Shrier LA, Harris SK, Chang G. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156(6):607–614. doi: 10.1001/archpedi.156.6.607. [DOI] [PubMed] [Google Scholar]
  • 17.U.S. Department of Health and Human Services . The Health Consequences of Smoking—50 Years of Progress: A Report of the Surgeon General. Atlanta, GA: 2014. [Google Scholar]
  • 18.Santelli JS, Smith Rogers A, Rosenfeld WD, et al. Guidelines for adolescent health research. A position paper of the Society for Adolescent Medicine. J Adolesc Heal. 2003;33(5):396–409. doi:S1054139X03002908 [pii] [PubMed] [Google Scholar]
  • 19.Humeniuk R, Ali R, Babor TF, et al. Validation of the Alcohol, Smoking And Substance Involvement Screening Test (ASSIST) Addiction. 2008;103(6):1039–1047. doi: 10.1111/j.1360-0443.2007.02114.x. doi:ADD2114 [pii] 10.1111/j.1360-0443.2007.02114.x. [DOI] [PubMed] [Google Scholar]
  • 20.Humeniuk R, Henry-Edwards S, Ali R, Poznyak V, Monteiro M. The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): Manual for Use in Primary Care. World Health Organization; Geneva, Switzerland: 2010. [Google Scholar]
  • 21.Sobell LC, Sobell MB. Timeline followback: a technique for assessing self-reported alcohol consumption. In: Litten RZ, Allen JP, editors. Measuring alcohol consumption: psychosocial and biochemical methods. Vol. 41. Humana; Totowa, NJ: 1992. p. 72. [Google Scholar]
  • 22.Sobell LC, Sobell MB. Alcohol Timeline Followback User’s Manual. Addiction Research Foundation; Toronto: 1995. [Google Scholar]
  • 23.Robinson SM, Sobell LC, Sobell MB, Leo GI. Reliability of the Timeline Followback for Cocaine, Cannabis, and Cigarette Use. Psychol Addict Behav. 2012 doi: 10.1037/a0030992. doi:10.1037/a0030992. [DOI] [PubMed] [Google Scholar]
  • 24.Levy S, Sherritt L, Harris SK, et al. Test-retest reliability of adolescents’ self-report of substance use. Alcohol Clin Exp Res. 2004;28(8):1236–1241. doi: 10.1097/01.alc.0000134216.22162.a5. [DOI] [PubMed] [Google Scholar]
  • 25.Hjorthøj CR, Hjorthøj AR, Nordentoft M. Validity of Timeline Follow-Back for self-reported use of cannabis and other illicit substances-systematic review and meta-analysis. Addict Behav. 2012;37(3):225–33. doi: 10.1016/j.addbeh.2011.11.025. doi:10.1016/j.addbeh.2011.11.025. [DOI] [PubMed] [Google Scholar]
  • 26.Lewis-Esquerre JM, Colby SM, Tevyaw TO, Eaton CA, Kahler CW, Monti PM. Validation of the timeline follow-back in the assessment of adolescent smoking. Drug Alcohol Depend. 2005;79:33–43. doi: 10.1016/j.drugalcdep.2004.12.007. [DOI] [PubMed] [Google Scholar]
  • 27.Winters KC, Stinchfield RD, Fulkerson J, Henly GA. Measuring alcohol and cannabis use disorders in an adolescent clinical sample. Psychol Addict Behav. 1993;7(3):185–196. [Google Scholar]
  • 28.Martin CS, Winters KC. Diagnosis and assessment of alcohol use disorders among adolescents. Alcohol Heal Res World. 1998;22(2):95–105. [PMC free article] [PubMed] [Google Scholar]
  • 29.Winters K, Henly G. Adolescent Diagnostic Interview (ADI) Manual. Los Angeles: Western Psychological Services. 1993 [Google Scholar]
  • 30.Wheeler KC, Fletcher KE, Wellman RJ, Difranza JR. Screening adolescents for nicotine dependence: the Hooked On Nicotine Checklist. J Adolesc Heal. 2004;35(3):225–230. doi: 10.1016/j.jadohealth.2003.10.004. doi:10.1016/j.jadohealth.2003.10.004 S1054139X03005317 [pii] [DOI] [PubMed] [Google Scholar]
  • 31.Wellman RJ, Difranza JR, Pbert L, et al. A comparison of the psychometric properties of the Hooked on Nicotine Checklist and the modified Fagerström Tolerance Questionnaire. Addict Behav. 2006;31:486–495. doi: 10.1016/j.addbeh.2005.05.031. doi:10.1016/j.addbeh.2005.05.031. [DOI] [PubMed] [Google Scholar]
  • 32.Carpenter MJ, Baker NL, Gray KM, Upadhyaya HP. Assessment of nicotine dependence among adolescent and young adult smokers: a comparison of measures. Addict Behav. 2010;35(11):977–982. doi: 10.1016/j.addbeh.2010.06.013. doi:S0306-4603(10)00178-4 [pii] 10.1016/j.addbeh.2010.06.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Harris SK, Woods ER, Sherritt L, et al. A youth-provider connectedness measure for use in clinical intervention studies. J Adolesc Heal. 2009;44(2 Suppl):S35–36. [Google Scholar]
  • 34.Couper M, Tourangeau R, Conrad F, Zhang C. The design of grids in web surveys. Soc Sci Comput Rev. 2012;31(3):322–345. doi: 10.1177/0894439312469865. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Toepoel V, Das M, VanSoest A. Design of web questionnaires: the effect of the number of items per screen. Field methods. 2009;21:200–213. [Google Scholar]
  • 36.Tourangeau R, Couper M, Conrad F. Spacing, position, and order: interpretive heuristics for visual features of survey questions. Public Opin Q. 2004;68:368–393. [Google Scholar]
  • 37.Gfroerer J, Wright D, Kopstein A. Prevalence of youth substance use: the impact of methodological differences between two national surveys. Drug Alcohol Depend. 1997;47(1):19–30. doi: 10.1016/s0376-8716(97)00063-x. [DOI] [PubMed] [Google Scholar]
  • 38.Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280(5365):867–73. doi: 10.1126/science.280.5365.867. [DOI] [PubMed] [Google Scholar]
  • 39.Perlis TE, Des Jarlais DC, Friedman SR, Arasteh K, Turner CF. Audio-computerized self-interviewing versus face-to-face interviewing for research data collection at drug abuse treatment programs. Addiction. 2004;99(7):885–96. doi: 10.1111/j.1360-0443.2004.00740.x. doi:10.1111/j.1360-0443.2004.00740.x. [DOI] [PubMed] [Google Scholar]
  • 40.Pérez-Gómez A, Scoppetta-Díazgranados O. El Crafft/Carlos como instrumento para la identificación temprana de consumo de alcohol y otras SPA: una adaptación al Español. Rev Colomb Psicol. 2011;20(2):265–274. [Google Scholar]
  • 41.Csemy L, Knight JR, Starostova O, Sherritt L, Kabicek P, Van Hook S. Adolescents substance abuse screening program: experience with Czech adaptation of the CRAFFT screening test. Vox Pediatr. 2008;8(6):35–36. [Google Scholar]
  • 42.Bernard M, Bolognini M, Plancherel B, et al. French validity of two substance use screening tests among adolescents: A comparison of the CRAFFT and DEP-ADO. J Subst Use. 2005;10(6):385–395. doi:10.1080/14659890412331333050. [Google Scholar]
  • 43.Rumpf H-J, Wohlert T, Freyer-Adam J, Grothues J, Bischof G. Screening questionnaires for problem drinking in adolescents: performance of AUDIT, AUDIT-C, CRAFFT and POSIT. Eur Addict Res. 2013;19(3):121–7. doi: 10.1159/000342331. doi:10.1159/000342331. [DOI] [PubMed] [Google Scholar]

RESOURCES