A brief patient self-administered substance use screening tool for primary care: two-site validation study of the Substance Use Brief Screen (SUBS)

Abstract

Background

Substance use screening is widely encouraged in healthcare settings, but the lack of a screening approach that fits easily into clinical workflows has restricted its broad implementation. The Substance Use Brief Screen (SUBS) was developed as a brief, self-administered instrument to identify unhealthy use of tobacco, alcohol, illicit drugs, and prescription drugs. We evaluated the validity and test-retest reliability of the SUBS in adult primary care patients.

Methods

Adults age 18-65 were enrolled from urban safety net primary care clinics to self-administer the SUBS using touch-screen tablet computers for a test-retest reliability study (n=54) and a two-site validation study (n=586). In the test-retest reliability study, the SUBS was administered twice within a 2-week period. In the validation study, the SUBS was compared to reference standard measures, including self-reported measures and saliva drug tests. We measured test-retest reliability and diagnostic accuracy of the SUBS for detection of unhealthy use and substance use disorder for tobacco, alcohol, and drugs (illicit and prescription drug misuse).

Results

Test-retest reliability was good or excellent for each substance class. For detection of unhealthy use, the SUBS had sensitivity and specificity of 97.8% (95% CI 93.7 to 99.5) and 95.7% (95% CI 92.4 to 97.8), respectively, for tobacco; and 85.2% (95% CI 79.3 to 89.9) and 77.0% (95% CI 72.6 to 81.1) for alcohol. For unhealthy use of illicit or prescription drugs, sensitivity was 82.5% (95% CI 75.7 to 88.0) and specificity 91.1% (95% CI 87.9 to 93.6). With respect to identifying a substance use disorder, the SUBS had sensitivity and specificity of 100.0% (95% CI 92.7 to 100.0) and 72.1% (95% CI 67.1 to 76.8) for tobacco; 93.5% (95% CI 85.5 to 97.9) and 64.6% (95% CI 60.2 to 68.7) for alcohol; and 85.7% (95% CI 77.2 to 92.0) and 82.0% (95% CI 78.2 to 85.3) for drugs. Analyses of area under the receiver operating curve (AUC) indicated good discrimination (AUC 0.74-0.97) for all substance classes. Assistance in completing the SUBS was requested by 11% of participants.

Conclusions

The SUBS was feasible for self-administration and generated valid results in a diverse primary care patient population. The 4-item SUBS can be recommended for primary care settings that are seeking to implement substance use screening.

Introduction

Screening coupled with brief office-based interventions for tobacco and alcohol use in primary care is among the top ten recommended prevention practices in the U.S.^1-5 Screening and brief intervention to reduce unhealthy alcohol use in primary care patients has a robust evidence base,^6,7 and “Screening, Brief Intervention, and Referral to Treatment (SBIRT)” programs, which include drugs as well, are widely promoted by government health authorities and supported by insurance billing codes.^8,9 Yet screening rates remain low in primary care settings.^10-12 A predominant challenge has been the lack of an efficient screening approach that fits easily into existing clinical workflows.^13-20

A self-administered screening approach could reduce barriers to identifying unhealthy substance use. Existing brief screening questionnaires such as the interviewer-administered single-item screening questions for alcohol²¹ and drugs²² still require significant staff time and training to administer. Fidelity may decline as providers modify the screening language, potentially compromising the validity of the instrument.^23,24 Patients may be reluctant to report stigmatized behavior to an interviewer face-to-face.^25,26 Workflow could be streamlined by having patients self-administer questionnaires using waiting room-based kiosk or tablet computers, or internet portals into the electronic medical record, with results delivered to the provider at the point of care.

There is currently no brief and comprehensive substance use screening instrument that has been validated for patient self-administration. To address this need, we developed the Substance Use Brief Screen (SUBS). The SUBS is designed to meet the demands of primary care settings by being simple enough for self-completion, integrating screening for all clinically relevant classes of substances (tobacco, alcohol, illicit drugs, prescription drugs used non-medically); and being sufficiently precise to streamline the subsequent assessment of substance use disorders in those who screen positive. The SUBS was based on the National Institute on Drug Abuse (NIDA) Quick Screen V1.0, which has not yet been validated.²⁷ Presented here are the results of two studies of the SUBS in primary care patients; a single-site test-retest reliability study, and a two-site validation study.

Methods

Participants and Recruitment

The primary study site (Site A) was the adult primary care clinic of a large municipal hospital in New York City, and conducted the test-retest reliability and validation studies. For the validation study, data was collected at an additional site (Site B), a safety net hospital-based adult primary care clinic in Boston. Data were collected at Site A from April 2011-April 2012 for the test-retest reliability study, and from July 2012-June 2013 for the validation study. Data were collected at Site B from June-July 2012. Eligible individuals were 21-65 years of age, English-speaking, and current clinic patients. Individuals over age 65 were excluded because unhealthy drug and alcohol use is less prevalent in this age group,²⁸ and the sample size in our study would not support meaningful analyses.

In recruiting for the test-retest reliability study, a purposeful sampling approach was used to achieve approximately equal numbers of male and female participants. For the validation study, participants were recruited consecutively while they were waiting for medical appointments. At Site A, participants were approached in the clinic waiting area using a pre-specified path through the seats. At Site B, each patient who presented for a scheduled clinic visit was approached. The institutional review boards of NYU School of Medicine and Boston University Medical Center reviewed and approved all study procedures.

Study Procedures

Test-Retest Reliability Study

Test-retest reliability was examined in 61 participants, who self-administered the SUBS using a touch-screen tablet computer at the initial visit (Time 1; “T1”), and then were scheduled to return within 7-14 days for a second visit (Time 2; “T2”) to repeat it.

Validation Study

Participants completed the SUBS independently. Following the SUBS, a series of instruments were administered by a research assistant, and served as reference standard comparison measures. Using a two-step consent process, after completing all self-reported measures Site A participants were asked to consent to saliva drug testing.

Measures

Standard demographic data were collected. Health literacy was measured using the Rapid Estimate of Adult Literacy in Medicine (REALM), and standard cutoffs were applied to interpret scores as being below or at/above the high school level.²⁹

Experimental Instrument: SUBS

The SUBS (Figure 1) screens for unhealthy use of tobacco, alcohol, illicit drugs, and non-medical use of prescription medications. Its three response categories were chosen to promote reporting of even occasional use, but intended to be dichotomized into negative and positive screening results. The timeframe of “past 12 months use” was chosen as the most clinically relevant period, given that most guidelines recommend annual screening for drugs and alcohol, and the DSM-V criteria for substance use disorders are based on this period.

Figure 1.

Substance Use Brief Screen

Reference Standard Measures

Self-reported reference standard measures were previously validated instruments measuring unhealthy substance use and substance use disorders for tobacco, alcohol, illicit drugs, and non-medical use of prescription drugs (Table 1). Biologic measures were oral fluid tests for the nicotine metabolite cotinine, and for common drugs of abuse (marijuana, benzodiazepines, cocaine, amphetamines, opiates, PCP). Oral fluid has accuracy equivalent to urine drug screening tests, and a window of detection of up to 4 days for nicotine, and 1 to 3 days for most drugs.^30-32 At Site B, reference standard comparison measures were not collected for tobacco and oral fluid tests were not performed.

Table 1.

Combination of reference standard measures defining unhealthy use and substance use disorder for each of the four substance classes in the SUBS

	ASSISTa	Timeline follow-backb	MINI-Plusc	Fagerstrom Test*d	NicAlert oral fluid test*e	Intercept oral fluid test*f
Tobacco – unhealthy use	+			+	+
Tobacco - disorder				+
Alcohol – unhealthy use	+	+	+
Alcohol - disorder			+
Illicit drug – unhealthy use	+	+	+			+
Illicit drug - disorder			+
Rx drug – unhealthy use	+	+	+			+
Rx drug – disorder			+

^*Administered at Site A only.

^aAlcohol, Smoking ad Substance Involvement Screening Test (ASSIST) Version 3.0 measured unhealthy use, defined as moderate- or high-risk use, based on standard cutoffs (Humeniuk, 2008). The ASSIST 3.0 was adapted to include prescription opioids and prescription stimulants.

^bTimeline follow-back (TLFB) measured unhealthy use of alcohol, illicit drugs, and prescription drugs in the past 30 days (Sobell and Sobell, 1992). Unhealthy alcohol use was defined as alcohol in excess of guideline-recommended limits (5 drinks/day or 14 drinks/week for men; 4 drinks/day or 7 drinks/week for women). Unhealthy drug use was defined as any use of an illicit drug or non-medical use of a prescription medication.

^cMINI-Plus, Version 6.0 (Lecrubier, Sheehan, Weiller et al., 1997; Sheehan, Lecrubier, Sheehan et al., 1998) measured past 12 months unhealthy use of alcohol (MINI Question I1) and drugs (MINI Question J1). MINI-Plus additionally provided measures of alcohol and drug use disorders (defined by DSM-IV abuse or dependence).

^dFagerstrom Test for Nicotine Dependence (FTND) measured nicotine dependence, based on the standard cutoff score of 4 or higher (Heatherton et al., 1991). Unhealthy tobacco use was defined as a positive response to: “Have you used tobacco products even one time in the past 12 months?”

^eNicAlert™ test (Nymox pharmaceuticals) is an oral fluid test for the nicotine metabolite cotinine (Cooke et al., 2008). A positive oral fluid test in the absence of recent use of oral nicotine replacement therapy was considered unhealthy tobacco use.

^fIntercept™ immunoassay (OraSure Technologies) is an oral fluid test for common drugs of abuse (Heltsley et al., 2011; Cone et al., 2002). A positive oral fluid test in the absence of medical use of medications that may be detected with the Intercept™ immunoassay was considered unhealthy drug use.

References for Table 1:

Humeniuk R. Validation of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) and pilot brief intervention: A technical report of phase II findings of the WHO ASSIST Project. 2008. http://www.who.int/substance_abuse/activities/assist_technicalreport_phase2_final.pdf. Accessed February 5, 2010.

Sobell LC, Sobell MB. Timeline follow-back. Measuring alcohol consumption: Springer; 1992:41-72.

Lecrubier Y, Sheehan D, Weiller E, et al. The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. European psychiatry. 1997;12(5):224-231.

Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. The Journal of clinical psychiatry. 1998;59 Suppl 20:22-33;quiz 34-57.

Heatherton TF, Kozlowski LT, Frecker RC, Fagerstrom KO. The Fagerstrom Test for Nicotine Dependence: a revision of the Fagerstrom Tolerance Questionnaire. Br J Addict. Sep 1991;86(9):1119-1127.

Cooke F, Bullen C, Whittaker R, McRobbie H, Chen MH, Walker N. Diagnostic accuracy of NicAlert cotinine test strips in saliva for verifying smoking status. Nicotine & tobacco research : official journal of the Society for Research on Nicotine and Tobacco. Apr 2008;10(4):607-612.

Heltsley R, DePriest A, Black DL, et al. Oral fluid drug testing of chronic pain patients. I. Positive prevalence rates of licit and illicit drugs. Journal of analytical toxicology. Oct 2011;35(8):529-540.

Cone EJ, Presley L, Lehrer M, et al. Oral fluid testing for drugs of abuse: positive prevalence rates by Intercept immunoassay screening and GC-MS-MS confirmation and suggested cutoff concentrations. Journal of analytical toxicology. Nov-Dec 2002;26(8):541-546.

Statistical Analysis

We examined descriptive statistics for the sample including, in the validation study, prevalence of substance use reported on the Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST). Responses to the SUBS were dichotomized for each of its four items, with a response of “never” representing a negative screen, and any other response a positive screen. Illicit drugs and prescription drugs were analyzed separately and then in combination as an “any drug’” category.

Test-Retest Reliability

In the test-retest reliability analysis we compared responses to each SUBS item at the first versus second administration of the instrument. The level of agreement between responses at T1 and T2 was evaluated using phi coefficients and McNemar's tests.

Validation

The reference standard measures were used in combination to identify unhealthy use and substance use disorder, for each of the four substance classes in the SUBS. A composite reference standard may be used when reference tests are imperfect,³³ and has been adopted in similar validation studies.^21,22 Based on the composite reference standard measures, we calculated the sensitivity and specificity of each SUBS item. To provide an additional measure of the diagnostic value of the SUBS, positive and negative diagnostic likelihood ratios (DLRs) were calculated.³⁴ We computed receiver operator characteristic (ROC) curves and examined the area under each curve (AUC). An AUC of ≥.90 represents excellent discrimination, an AUC of 0.8 or higher indicates good discrimination, and an AUC lower than 0.7 poor discrimination.³⁵ Exact 95% confidence intervals (CIs) were calculated for all accuracy estimates.

Calculations were made individually for each of the four items in the SUBS and for the combined “any drug” category. We then examined differences in the SUBS results (in comparison to reference standard measures) between the two sites by conducting chi-square analyses. To compare sites on sensitivity, among those who were positive on the reference standard measures we examined the crosstabulation of site and the SUBS result. To compare sites on specificity, we examined the crosstabulation of site and the SUBS result among those who were negative on the reference standards. SUBS results differed significantly for the two sites only for the comparison of specificity with respect to unhealthy illicit drug use and prescription drug use disorder. At Site B, there was a higher proportion of false positive results on the SUBS for unhealthy illicit drug use, with the false positive fraction 0.02 at Site A and 0.06 at Site B (p = 0.02). A similar pattern of site differences was observed for specificity with respect to prescription drug use disorder, with a false positive fraction of 0.09 at Site A and 0.15 at Site B (p = 0.05). Because specificity was acceptable (greater than 85%) at each site, the decision was made to combine the two sites for all analyses.

In a second step, the sensitivity and specificity of the SUBS for detecting unhealthy use were estimated for pre-specified subgroups, in which prior studies have found substance use screening questionnaires to have reduced precision or feasibility.^22,36-38 These subgroups were: male, age greater than 50, Hispanic/Latino, primary language other than English, born outside US, and education or health literacy lower than high school level. To determine whether there were significant differences in SUBS accuracy for each subgroup, we performed chi-square analyses, crosstabulating each subgrouping variable with the SUBS screening result, within groups that were positive (sensitivity) or negative (specificity) on the reference standard measures. Analyses were conducted using version 13 of Stata (StataCorp, 2013) and its diagnostic testing module.³⁹

Results

Participant characteristics

Participants were racially and ethnically diverse, half had a high school level of education or less, and 26% reported drug use in the past three months (Table 2). A limited set of demographic characteristics of eligible individuals who refused to participate was collected during the validation study, at Site A. Compared to those who participated, non-participants tended to be female (57%) and white (36%), and had a lower average age (42 years).

Table 2.

Demographic characteristics and prevalence of substance use among participants in the test-retest reliability study and validation study

Characteristic	Validation study N (%) N=586	Test-retest reliability study N (%) N=54
Demographics
Age (years)
Mean, SD	46, SD 11.8	45, SD 11.2
Median	49	47
Range	21-65	19-64
Interquartile range	16	18
Gender
Female	292 (49.8)	25 (46.3)
Male	292 (49.8)	29 (53.7)
Transgender	2 (0.3)	0
Race/Ethnicity
Black/African American	293 (50.2)	32 (59.3)
White/Caucasian	109 (18.7)	11 (20.4)
Hispanic	127 (21.7)	8 (14.8)
Other	51 (8.7)	3 (5.6)
Don't Know/Ref	4 (0.7)	0
Primary language
English	467 (79.7)	49 (90.6)
Spanish	48 (8.2)	3 (5.6)
Other	71 (12.1)	2 (3.7)
Country of birth
U.S.	389 (66.4)	13 (24.1)
Other	197 (33.6)	41 (75.9)
Education (highest level completed)
Less than HS	93 (15.9)	8 (14.8)
HS grad or GED	199 (34.0)	14 (25.9)
Some college or trade school	148 (25.3)	21 (38.9)
College degree (4-year)	143 (24.4)	11 (20.4)
Other	3 (0.5)	0
Health Literacya		N/A
Below high school	213 (40.7)
High school or greater	310 (59.3)
Employment
Employed	114 (29.2)	20 (37.0)
Unemployed	275 (70.5)	30 (55.6)
Other	0	4 (7.4)
Don't know/Refused	1 (0.3)	0
Incomeb
<$5,000	89 (22.8)	21 (38.9)
$5,000-14,999	90 (23.1)	11 (20.4)
$15,000-24,999	61 (15.6)	6 (11.1)
$25,000-49,999	70 (17.9)	8 (14.8)
≥ $50,000	23 (5.9)	3 (5.6)
Don't know/Refused	57 (14.6)	5 (9.3)
Perceived health statusb,c		N/A
Very good or excellent	96 (24.6)
Good	127 (32.6)
Fair or poor	162 (41.5)
Don't know/Refused	5 (1.3)
Substance use prevalenced
Tobacco use		N/A
Lifetime	376 (64.2)
Current	216 (36.9)
Low risk	345 (58.9)
Moderate risk	201 (34.3)
High risk	40 (6.8)
Alcohol use		N/A
Lifetimee	497 (84.8)
Currente	327 (55.8)
Low risk	474 (80.9)
Moderate risk	81 (13.8)
High risk	31 (5.3)
Illicit drug use		N/A
Lifetime	348 (59.4)
Current	136 (23.2)
Low risk	414 (71.1)
Moderate risk	134 (23.0)
High risk	34 (5.8)
Prescription drug use		N/A
Lifetime	143 (24.4)
Current	50 (8.5)
Low risk	529 (90.6)
Moderate risk	46 (7.9)
High risk	9 (1.5)
Any drug use (illicit or prescription)		N/A
Lifetime	361 (61.6)
Current	153 (26.1)
Low risk	401 (69.1)
Moderate risk	141 (24.3)
High risk	38 (6.6)
Specific drug class
Marijuana		N/A
Lifetime	329 (56.1)
Current	99 (16.9)
Low risk	477 (81.4)
Moderate risk	96 (16.4)
High risk	13 (2.2)
Cocaine		N/A
Lifetime	213 (36.3)
Current	46 (7.8)
Low risk	495 (84.8)
Moderate risk	72 (12.3)
High risk	17 (2.9)
Hallucinogens		N/A
Lifetime	117 (20.0)
Current	10 (1.7)
Low risk	565 (96.4)
Moderate risk	20 (3.4)
High risk	1 (0.2)
Sedatives		N/A
Lifetime	103 (17.6)
Current	30 (5.1)
Low risk	554 (94.7)
Moderate risk	26 (4.4)
High risk	5 (0.9)
Heroin		N/A
Lifetime	101 (17.2)
Current	22 (3.8)
Low risk	542 (92.6)
Moderate risk	31 (5.3)
High risk	12 (2.0)
Rx opioids		N/A
Lifetime	75 (12.8)
Current	25 (4.3)
Low risk	555 (94.7)
Moderate risk	25 (4.3)
High risk	6 (1.0)
Rx stimulants		N/A
Lifetime	60 (10.2)
Current	16 (2.7)
Low risk	562 (95.9)
Moderate risk	22 (3.8)
High risk	1 (0.2)
Methamphetamine		N/A
Lifetime	55 (9.4)
Current	1 (0.2)
Low risk	579 (99.0)
Moderate risk	6 (1.0)
High risk	0
Inhalants		N/A
Lifetime	52 (8.9)
Current	5 (0.9)
Low risk	574 (98.0)
Moderate risk	12 (2.0)
High risk	0

^aBased on REALM-Short Form at Site A (introduced after enrollment of the first 63 participants), and the full REALM at Site B. Standard cutoffs applied to determine education level in terms of years of completed schooling.

^bData collected at Site A only

^c“Would you say your health in general is excellent, very good, good, fair, or poor?”

^dBased on responses to the ASSIST V3.0.

^eReport of any alcohol use, regardless of quantity.

Test-retest Reliability Study

In the test-retest reliability study, 54 of 61 participants (89%) completed both study visits and were included in the analysis. There were no statistically significant differences in screening results for any substance between T1 and T2 administrations of the SUBS (Table 3). Reliability was excellent for tobacco (Φ=.96) and drugs (Φ=.78), and good for alcohol (Φ=.63).

Table 3.

Test-retest reliability, based on results from the 54 participants who completed the SUBS at Time 1 and Time 2.

	Individuals with Positive Screen		Change in screening result between T1 and T2			Significance of change	Correlation of scores
Substance	Time 1 N (%)	Time 2 N (%)	No change N (%)	+ to - N	- to + N	P	phi ΦT1T2
Tobacco	21 (39)	20 (37)	53 (98)	1	0	1.00	0.96
Alcohol	36 (65)	36 (67)	45 (83)	4	5	1.00	0.63
Illicit drugs	21 (39)	20 (37)	47 (87)	4	3	1.00	0.73
Prescription drugs	14 (26)	13 (24)	41 (76)	7	6	1.00	0.36
Any Drugsa	27 (50)	25 (46)	48 (89)	4	2	.69	0.78

^aIllicit drug use and/or non-medical use of prescription drug(s) reported

Validation Study

In the validation study, 588 individuals (49% of those eligible) enrolled (Figure 2). After removing data for 2 individuals with lost or incomplete SUBS data, a total of 586 cases were analyzed. Site A contributed 390 cases, and Site B 196 cases.

Figure 2.

Flowchart of participant recruitment for the validation study

*Of the 390 participants who completed the interview at Site A, 331 (84%) had saliva test results. Site B did not offer saliva testing.

Feasibility measures were collected at Site A. Forty-two (11%) participants requested assistance to complete the SUBS. Of this group, 28 asked for help in comprehending the questions, and 14 requested technical assistance.

Analyses of the accuracy of the SUBS in comparison to reference standard measures are presented in Table 4. For tobacco (evaluated at Site A only), 144 participants (37%) had a positive SUBS screening result. The SUBS had 98% sensitivity and 96% specificity for detecting unhealthy tobacco use, based on self-reported measures only. These results did not change when results of the oral fluid cotinine test were also taken into account. For detection of tobacco dependence, the sensitivity of SUBS was 100%, and specificity was 72%.

Table 4.

Sensitivity, specificity, likelihood ratios, and area under the curve of the Substance Use Brief Screen (SUBS) for detecting unhealthy use and substance use disorders (N=586).

Substance class	Positive on SUBS N (%)	Positive on reference standards N (%)	Sensitivity % (95% CI)	Specificity % (95% CI)	Positive LR (95% CI)	Negative LR (95% CI)	AUC (95% CI)
Any unhealthy use, self-reported
Tobaccoa	144 (36.9)	136 (34.9)	97.8 (93.7, 99.5)	95.7 (92.4, 97.8)	22.6 (12.7, 40.3)	0.02 (0.01, 0.07)	0.97 (0.95, 0.99)
Alcohol	252 (43.1)	189 (32.3)	85.2 (79.3, 89.9)	77.0 (72.6, 81.1)	3.7 (3.1, 4.5)	0.19 (0.14, 0.27)	0.81 (0.78, 0.84)
Illicit drugs	133 (22.9)	148 (25.5)	81.1 (73.8, 87.0)	97.0 (94.9, 98.4)	27.0 (15.7, 46.4)	0.20 (0.14, 0.27)	0.89 (0.86, 0.92)
Prescription drugs	74 (12.8)	54 (9.3)	55.6 (41.4, 69.1)	91.6 (88.9, 93.8)	6.6 (4.6, 9.6)	0.49 (0.36, 0.65)	0.74 (0.67, 0.80)
Any drugs	169 (29.4)	160 (27.9)	82.5 (75.7, 88.0)	91.1 (87.9, 93.6)	9.2 (6.7, 12.7)	0.19 (0.14, 0.27)	0.87 (0.84, 0.90)
Substance use disorder, self-reported
Tobacco1	144 (36.9)	49 (12.6)	100.0 (92.7, 100.0)	72.1 (67.1, 76.8)	3.6 (3.0, 4.2)	0.01 (0.00, 0.22)	0.86 (0.84, 0.89)
Alcohol	252 (43.1)	77 (13.2)	93.5 (85.5, 97.9)	64.6 (60.2, 68.7)	2.6 (2.3, 3.0)	0.10 (0.04, 0.24)	0.79 (0.76, 0.83)
Illicit drugs	133 (22.7)	95 (16.2)	82.1 (72.9, 89.2)	88.8 (85.6, 91.4)	7.3 (5.6, 9.6)	0.20 (0.13, 0.31)	0.85 (0.81, 0.90)
Prescription drugs	76 (13.1)	27 (4.7)	59.3 (38.8, 77.6)	89.2 (86.3, 91.6)	5.5 (3.7, 8.1)	0.46 (0.29, 0.72)	0.74 (0.65, 0.84)
Any drugs	171 (29.5)	98 (16.9)	85.7 (77.2, 92.0)	82.0 (78.2, 85.3)	4.8 (3.9, 5.8)	0.17 (0.11, 0.28)	0.84 (0.80, 0.88)

^aReference standard measures collected from Site A only (N=390)

All other substances were evaluated based on data from both sites. With respect to alcohol, the SUBS had 85% sensitivity and 77% specificity for detecting unhealthy alcohol use, and 94% sensitivity and 65% specificity for detecting an alcohol use disorder. For unhealthy drug use, the SUBS had 81% sensitivity and 97% specificity for detection of unhealthy use of illicit drugs, and lower sensitivity (56%) and specificity (92%) for detection of unhealthy use of prescription drugs. When illicit and prescription drugs were considered together as “any drugs,” the SUBS had 83% sensitivity and 91% specificity for detection of unhealthy use, and 86% sensitivity and 82% specificity for detection of a drug use disorder. When results of oral fluid testing were taken into account at Site A, sensitivity of the SUBS for unhealthy drug use was reduced to 77%, while specificity increased to 92%. Positive screens were at least twice as likely and negative screens were at least half as likely among individuals with versus without unhealthy substance use or a substance use disorder. AUCs were greater than 0.70 for all substance classes, with respect to identifying unhealthy use and substance use disorders.

Subgroup analyses

The SUBS had statistically significant lower sensitivity and higher specificity among females for detecting unhealthy use of any drug (p<0.01) (Table 5). Sensitivity in detecting tobacco use was lower among Hispanic versus non-Hispanic participants (p<0.01). There were no other statistically significant differences between subgroups in detecting unhealthy use of tobacco, alcohol, or drugs.

Table 5.

Sensitivity, specificity, and area under the curve (AUC) for detection of any unhealthy use among select subgroups, based on comparison to reference standard measures (N=586). Statistically significant differences between subgroups are in bold text.

	Positive on reference standard N (% of subpopulation)	Sensitivity % (95% CI)	Specificity % (95% CI)	AUC (95% CI)
Tobacco
Female	41 (21.9)	95.1 (83.5, 99.4)	97.3 (93.1, 99.2)	0.96 (0.93, 1.00)
Male	95 (46.8)	98.9 (94.3, 100.0)	93.5 (87.1, 97.4)	0.96 (0.94, 0.99)
Age 21-50	71 (35.0)	97.2 (90.2, 99.7)	94.7 (89.4, 97.8)	0.96 (0.93, 0.99)
Age 51-65	65 (34.8)	98.5 (91.7, 100.0)	96.7 (91.8, 99.1)	0.98 (0.95, 1.00)
Hispanic	41 (38.3)	92.7 (80.1, 98.5)	98.5 (91.8, 100.0)	0.96 (0.91, 1.00)
Non-Hispanic	95 (33.9)	100.0 (96.2, 100.0)	94.6 (90.3, 97.4)	0.97 (0.96, 0.99)
English primary language	124 (39.7)	98.4 (94.3, 99.8)	95.2 (91.1, 97,8)	0.97 (0.95, 0.99)
Non-English primary language	12 (15.4)	91.7 (61.5, 99.8)	97.0 (89.5, 99.6)	0.94 (0.86, 1.0)
Born in US	115 (43.6)	98.3 (93.9, 99.8)	95.3 (90.6, 98.1)	0.97 (0.95, 0.99)
Born outside US	21 (16.7)	95.2 (76.2, 99.9)	96.2 (90.5, 99.0)	0.96 (0.91, 1.00)
Education or health literacy < High school level	31 (44.3)	100.0 (88.8, 100.0)	94.9 (82.7, 99.4)	0.97 (0.94, 1.00)
Education or health literacy High school level or greater	105 (32.8)	97.1 (91.9, 99.4)	95.8 (92.2, 98.1)	0.97 (0.94, 0.99)
Alcohol
Female	67 (23.0)	82.1 (70.8, 90.4)	75.0 (68.8, 80.5)	0.79 (0.73, 0.84)
Male	122 (41.5)	86.9 (79.6, 92.3)	79.7 (72.9, 85.4)	0.83 (0.79, 0.88)
Age 21-50	116 (34.9)	82.8 (74.6, 89.1)	73.6 (67.2, 79.4)	0.78 (0.74, 0.83)
Age 51-65	73 (28.9)	89.0 (79.5, 95.1)	81.1 (74.6, 86.5)	0.85 (0.81, 0.90)
Hispanic	42 (33.1)	88.1 (74.4, 96.0)	71.8 (61.0, 81.0)	0.80 (0.73, 0.87)
Non-Hispanic	147 (32.3)	84.4 (77.5, 89.8)	78.2 (73.2, 82.7)	0.81 (0.78, 0.85)
English primary language	160 (34.3)	84.4 (77.8, 89.6)	76.8 (71.7, 81.4)	0.81 (0.77, 0.84)
Non-English primary language	29 (24.4)	89.7 (72.6, 97.8)	77.8 (67.8, 85.9(	0.84 (0.77, 0.91)
Born in US	147 (37.9)	83.7 (76.7, 89.3)	76.8 (70.9, 81.9)	0.80 (0.76, 0.84)
Born outside US	42 (21.3)	90.5 (77.4, 97.3)	77.4 (70.0, 83.7)	0.84 (0.78, 0.90)
Education or health literacy < High school level	34 (36.6)	91.2 (76.3, 98.1)	79.9 (67.2, 89.0)	0.85 (0.78, 0.93)
Education or health literacy High school level or greater	155 (31.5)	83.9 (77.1, 89.3)	76.6 (71.7, 81.0)	0.80 (0.77, 0.84)
Any drug
Female	55 (19.3)	70.9 (51.7, 82.4)	94.3 (90.5, 97.0)	0.83 (0.76, 0.89)
Male	105 (36.3)	88.6 (80.9, 94.0)	87.0 (81.2, 91.5)	0.88 (0.84, 0.92)
Age 21-50	104 (31.7)	84.6 (76.2, 90.9)	90.2 (85.5, 93.7)	0.87 (0.83, 0.91)
Age 51-65	56 (22.8)	78.6 (65.6, 88.4)	92.1 (87.3, 95.5)	0.85 (0.80, 0.91)
Hispanic	35 (27.6)	77.1 (59.9, 89.6)	91.3 (83.6, 96.2)	0.84 (0.77, 0.92)
Non-Hispanic	125 (28.2)	84.0 (76.4, 89.9)	90.9 (87.2, 93.8)	0.88 (0.84, 0.91)
English primary language	142 (31.1)	82.4 (75.1, 88.3)	91.4 (87.7, 94.3)	0.87 (0.83, 0.90)
Non-English primary language	18 (15.3)	83.3 (58.6, 96.4)	90.0 (82.4, 95.1)	0.87 (0.77, 0.96)
Born in US	131 (34.6)	84.7 (77.4, 90.4)	89.5 (85.0, 93.0)	0.87 (0.84, 0.91)
Born outside US	29 (14.9)	72.4 (52.8, 87.3)	93.4 (88.5, 96.6)	0.83 (0.74, 0.91)
Education or health literacy < High school level	31 (33.7)	71.0 (52.0, 85.8)	86.9 (75.8, 94.2)	0.79 (0.70, 0.88)
Education or health literacy High school level or greater	129 (26.8)	85.3 (78.0, 90.9)	91.8 (88.4, 94.4)	0.89 (0.85, 0.92)

Discussion

The SUBS is currently the only brief, self-administered, and comprehensive screening instrument that has been validated in primary care patients. We found that the SUBS had good test-retest reliability, sensitivity, and specificity for detection of past-year unhealthy use of tobacco, alcohol, and other drugs in primary care patients. It was feasible for self-administration on a tablet computer, and generated valid results in a diverse population.

Although the efficacy of screening and brief interventions for reducing drug use per se has not been established,^40-43 there are strong clinical reasons for including illicit and prescription drug misuse in a comprehensive screening approach. Drug use has profound effects on the management of medical conditions,⁴⁴ including drug-medication interactions,^45,46 medication adherence,^47,48 risk of overdose from prescription opioid misuse,⁴⁹ and overall health-related quality of life.⁵⁰ Furthermore, given the high prevalence of drug use in many primary care populations^22,40,50,51 the efficacy of alcohol and tobacco interventions could be compromised if providers are unaware of comorbid drug use.^52,53 Having information about a patient's drug use can thus assist the clinician in carrying out activities that are integral to the quality and safety of the medical care they provide, including safer prescribing, making correct diagnoses, identifying common comorbidities, and engaging patients in management of their other medical conditions.⁵⁴

The SUBS had high sensitivity and moderate to high specificity for detection of unhealthy use of tobacco, alcohol, and other drugs, and compares favorably to other widely recommended brief substance use screening tools--all of which are interviewer-administered.^21,22,55-59 While some interviewer-administered screening tools demonstrated superior psychometric properties in validation studies, instruments that rely on an interviewer are likely to lose their fidelity when administered in a non-research setting by clinical or lay staff.²³ Self-administered tools such as the SUBS, which do not rely on a having a trained interviewer reading the questionnaire verbatim,²³ and can put patients at greater ease in reporting stigmatized behavior,^25,26 may be more consistently accurate in real-world practice.

Sensitivity of the SUBS items was lower for prescription or illicit drugs alone, and increased when they were combined into a single “any drug” category. This may reflect lack of clarity among drug users themselves about what constitutes illicit versus prescription drug misuse.⁶⁰ Despite the lower sensitivity of the prescription drug item, we believe it should be included to capture individuals who misuse prescription but not illicit drugs. However, an assessment for all classes of drug use should be conducted for any individual who screens positive for illicit or prescription drugs on the SUBS.

The SUBS employs a cutoff of 4 or more drinks/day, instead of using the NIAAA-recommended gender specific cutoffs.⁶¹ Use of the lower cutoff could reduce specificity, but we found the converse; specificity of the alcohol item was slightly higher in men than in women, and the difference was not statistically significant. There were gender-based differences for detection of unhealthy drug use, but sensitivity and specificity were adequate among both men and women.

Limitations

Both sites enrolled patients from safety net hospital-based primary care clinics. Many individuals were ineligible due to limited English fluency, or age. The prevalence of drug use in our study was relatively high for primary care populations: 29% of participants reported drug use on the SUBS, while in the general population the prevalence of past year use is just 16%.⁶² In clinical settings with a very low prevalence of drug use, it may not be feasible or cost-effective to screen, because true positives could be outnumbered by false positives in this context. While the overall prevalence was high, rates of prescription drug misuse were relatively low in our study populations, limiting the precision of our estimates for the prescription drug item. Future studies should examine the SUBS in older patients, in additional languages, and in settings with different patterns of substance use.

Our validation analyses relied primarily on self-report measures, which have consistently shown good accuracy in research studies,^63-66 but are nonetheless dependent on accurate and truthful disclosure of use. Oral fluid tests were collected for tobacco and drugs as an additional check on the self-reported data, but these tests are limited by their relatively brief window of detection.^30-32 Hair testing, which has a much longer detection period, was cost prohibitive, and may not be reliable for detecting very occasional use.³² There is currently no reliable biomarker with sufficient sensitivity and specificity to detect unhealthy alcohol use.^67,68

A limitation of the SUBS common to all currently validated very brief substance use screening tools is that it was studied under conditions in which subjects were assured anonymity, to enhance the accuracy of reporting. Social desirability bias can cause patients to under-report substance use, even on a self-administered questionnaire, if they are uncomfortable having the information reported to their medical providers.^23,69 Privacy concerns could also inhibit disclosure of substance use by patients who worry about who will have access to their screening results, or the inclusion of this information in the medical record. Exactly how the sensitivity and specificity of the SUBS would change if it were introduced into routine care is not known, and would likely vary depending on specifics of the practice site and implementation approach.

Conclusions

As a brief self-administered tool that integrates screening for all classes of commonly used substances, the SUBS has the potential to greatly ease barriers to integrating screening for unhealthy substance use in health care settings. The SUBS represents the first such screening instrument to be rigorously validated in primary care patients. We found that it was feasible to administer, with good sensitivity and specificity. Although the SUBS, like any brief screening approach, may require further assessment of patients to guide clinical interventions, we believe its use can be recommended in primary care settings seeking to implement a screening program for tobacco, alcohol, and other drugs.

• The Substance Use Brief Screen (SUBS) is a brief computer self-administered screening tool that accurately detected past year unhealthy use of tobacco, alcohol, and drugs in primary care patients.

• The SUBS was feasible and well accepted in diverse populations of safety-net primary care patients.

• Use of the SUBS could ease barriers to screening for unhealthy substance use in primary care.