VALIDATION OF AN AUDIO COMPUTER ASSISTED SELF INTERVIEW (ACASI) VERSION OF THE ALCOHOL, SMOKING AND SUBSTANCE INVOLVEMENT SCREENING TEST (ASSIST) IN PRIMARY CARE PATIENTS

Abstract

Aims

To address barriers to implementing the “Alcohol, Smoking and Substance Involvement Screening Test (ASSIST)” in medical settings, we adapted the traditional interviewer-administered (IA) ASSIST to an audio-guided computer assisted self-interview (ACASI) format. This study sought to validate the ACASI ASSIST by estimating the concordance, correlation, and agreement of scores generated using the ACASI versus the reference standard IA ASSIST. Secondary aims were to assess feasibility and compare ASSIST self-report to drug testing results.

Design

Participants completed the ACASI and IA ASSIST in a randomly assigned order, followed by drug testing.

Setting

Urban safety-net primary care clinic.

Participants

A total of 393 adult patients.

Measurements

Scores generated by the ACASI and IA ASSIST; drug testing results from saliva and hair samples.

Findings

Concordance between the ACASI and IA ASSIST in identifying moderate-high risk use was 92–99% for each substance class. Correlation was excellent for global scores (ICC=0.94, CI 0.92–0.95) and for substance-specific scores for tobacco (ICC=0.93, CI 0.91–0.94), alcohol (ICC=0.91, CI 0.89–0.93) and illicit drugs (ICC=0.85, CI 0.85–0.90), and good for prescription drugs (ICC=0.68, CI 0.61–0.73). Ninety-four percent of differences in global scores fell within anticipated limits of agreement. Among participants with a positive saliva test, 74% self-reported use on the ACASI ASSIST. The ACASI ASSIST required a median time of 3.7 minutes (range 0.7–15.4), and 21 (5.3%) participants requested assistance.

Conclusions

The computer self-administered Alcohol, Smoking and Substance Involvement Screening Test appears to be a valid alternative to the interviewer-administered approach for identifying substance use in primary care patients.

Introduction

Alcohol and drug use disorders are among the top ten causes of preventable death in the United States. Screening followed by brief intervention for unhealthy alcohol use in adult primary care patients has a strong evidence base, and is among the most cost-effective preventive health services.(1–4) While the efficacy of this approach for reducing drug use in U.S. populations has not been established,(5–8) in medical practice settings screening for drugs may be justified on clinical grounds. Drug use can have serious implications for the prevention and treatment of other medical conditions,(9) and primary care providers may have the ability to offer patients treatment services including pharmacotherapy and referral to specialty care. Yet integrating screening and interventions for substance use into busy medical settings has proven challenging.(10–14)

Identification of drug and alcohol use could be facilitated by a unified approach that combines screening and assessment for tobacco, alcohol, and drugs, and quickly and reliably gathers enough information to provide a detailed and accurate risk assessment to guide clinical interventions. One such instrument is the “Alcohol, Smoking and Substance Involvement Screening Test (ASSIST),” a structured interview developed by the World Health Organization (WHO) for use in general healthcare settings, and validated in a large multi-site international study.(15–18) However, the ASSIST has not been widely adopted,(10, 19) in part because it takes approximately 5–15 minutes of face-to-face interaction with the patient, includes complex skip patterns, and requires computation of a score by the interviewer.

A patient self-administered version of the ASSIST, which could be completed prior to the medical encounter using a tablet or kiosk computer, has the potential to facilitate its implementation in health care settings. We thus adapted the previously validated interviewer-administered ASSIST to a patient self-administered format using audio-guided computer assisted self-interview (ACASI) technology. ACASI technology supports patients with limited reading ability because relevant text is read aloud in real time, and response options are clearly indicated using symbols. Computerized self-interview questionnaires have proven sensitive for detecting stigmatized behaviors, have comparable validity to traditional interview formats, are easily adapted to multiple languages, and can be integrated into medical settings.(20–26)

A self-administered screening and assessment tool has a number of potential advantages in busy medical practices. Time constraints have been identified as a primary barrier to implementation of substance use screening and interventions in primary care.(13, 27–36) Although well-resourced clinics may be able to utilize medical staff to conduct face-to-face screening, this requires personnel time and training, can threaten fidelity when screening items are not delivered exactly as written,(37, 38) and patients may be less willing to report substance use in a face-to-face interview.(39, 40) Moreover, by occupying patients’ waiting time with an activity that has personal relevance, this approach has the potential to increase patient satisfaction.(41, 42)

A prior study found that the ACASI ASSIST had excellent test-retest reliability in a sample of 101 adult primary care patients.(43) However, the validity of adapting what was designed as an interviewer-administered screening instrument into an ACASI format must be demonstrated before it can be recommended for widespread adoption. To provide that evidence, we sought to validate the ACASI ASSIST by estimating the concordance, correlation, and agreement of results generated using the ACASI versus the reference standard IA ASSIST. Secondary aims were to compare self-reported drug use on the ASSIST to biologic drug testing results, and to examine the feasibility and acceptability of the ACASI ASSIST.

Methods

Recruitment

The study was conducted in the adult primary care clinic of a large municipal hospital in New York City from July 2012 to June 2013. A convenience sample of 399 participants was enrolled. Based on a prior study conducted at this clinical site,(44) this sample was anticipated to include least 50 participants with moderate- to high-risk substance use to inform our comparison of the two instruments. While we did not conduct prior simulations to determine the precision of comparisons between the ACASI and IA ASSIST as a function of prevalence and sample size, 50 individuals with moderate-high risk alcohol or drug use would provide sufficient power to distinguish between good (ICC=0.6) and excellent (ICC=0.8) agreement.

Participants were consecutively recruited using pre-specified paths through the seats of the clinic’s waiting area. Eligible individuals were age 21–65 years, English speaking, and current clinic patients. Individuals over age 65 were excluded because unhealthy drug and alcohol use is less prevalent in this age group (44, 45) and the sample size in our study would not support meaningful analyses. Participants were randomly assigned in counterbalanced order to complete either the computer (ACASI) or interviewer (IA) ASSIST first. All participants completed both instruments in sequence, with one ASSIST version directly following the other. Interviews were conducted anonymously and in a private room. Participants completed the ACASI ASSIST independently using a touch-screen tablet computer, with headphones. Any requests for assistance were tracked by the research assistant (RA) using a standardized form. After completing both versions of the ASSIST, all participants were asked to participate in saliva drug testing, and a randomly selected sample of 39 participants was additionally offered hair testing. The institutional review board of the NYU School of Medicine reviewed and approved all study procedures.

Study Instruments: IA ASSIST and ACASI ASSIST

The ASSIST used for both the IA and ACASI versions was based on the WHO ASSIST V3.0; (17) a brief structured interview that covers nine substances (tobacco, alcohol, cannabis, cocaine, stimulants, inhalants, sedatives, hallucinogens, opioids) and assesses lifetime and current use, consequences of use, and failure to stop or cut down. The ASSIST instruments used in our study were adapted to include two additional substance classes: prescription opioids and prescription stimulants. These were added in response to the emergence of prescription drug misuse as a public health concern in the U.S., and were included in the ACASI ASSIST instrument used in our prior test-retest reliability study.(43)

IA ASSIST

The IA ASSIST was delivered by the RA as an interview, following standard procedures for ASSIST administration.(46, 47) Participants were provided with a written response card, and the RA read aloud and verbatim the introduction and all items and responses. For participants who had questions about a prescription medication item, the RA provided a clarification based on language from the ASSIST introduction, which states that medications should be reported if they have been “used for reasons other than prescription, or taken more frequently or at higher doses than prescribed.”

ACASI ASSIST

The ACASI ASSIST items were identical to those of the IA ASSIST, but additionally included clarifying items for each of the prescription drug classes (see Table 1). These items were added because participants frequently misinterpret questions concerning misuse of prescription drug items on self-administered questionnaires,(48) including on an earlier version of the ACASI ASSIST.(49) If the individual’s responses to the clarification items were not consistent with non-medical use of a prescription drug, in the analysis we recoded the response to zero (no use in the past 3 months) and the subsequent ASSIST items for that substance were coded as zero. Recoding was required for 3/10 individuals who reported prescription stimulants, for 12/32 who reported prescription sedatives, and for 19/35 who reported prescription opioids.

Table 1.

Items added to the ASSIST to clarify non-medical use of prescription stimulants, sedatives, and opiates.

	IA ASSIST		ACASI ASSIST
	Question 1	Question 2	Question 1	Question 2	Clarifying items for prescription medications
					Item	Response options	Action	Interpretation of a positive response
Prescription opioids (morphine, codeine, fentanyl, oxycodone [OxyContin, Percocet], hydrocodone [Vicodin], methadone, buprenorphine [Suboxone], etc.)	Added	Added	Added	Added	a. Is this a medication that you can buy in the store without a prescription (over the counter)?	Yes/No	If ‘Yes,’ skip items b–d.	Not a prescription medication; not non-medical use.
				Clarifying items if response is ‘Yes’	b.Was it prescribed for you?	Yes/No	If ‘Yes,’ administer items c and d.	Is a prescription medication; unknown if non-medical use.
Prescription stimulants (Ritalin, Concerta, Dexedrine, Adderall, diet pills, etc.)	Added	Added	Added	Added	c. Do you ever use MORE of your [stimulant/sedative/opiate] medication, that is, take a higher dosage, than is prescribed for you?	Yes/No	Regardless of response, administer item d.	Non-medical use
				Clarifying items if response is ‘Yes’	d. Do you ever use your [stimulant/sedative/opiate] medication MORE OFTEN, that is, shorten the time between dosages, than is prescribed for you?	Yes/No	No further follow-up items, return back to ASSIST Question 2.	Non-medical use
Prescription sedatives: Same as in ASSIST V3.0	-	-	-	-
				Clarifying items if response is ‘Yes’

The ASSIST items were delivered in their entirety, and written text on the computer screen was identical to the words of the voice instruction. The ACASI ASSIST was created using QDS Software (Nova Research Co).

Measures

Prevalence

Prevalence of lifetime and current (past 3 months) use was based on responses to the IA ASSIST Questions 1 and 2, respectively.

Risk scores

ASSIST global scores and substance specific involvement scores (SSIS) were calculated using standard ASSIST methodology, for both the IA and the ACASI versions.(17) The global score represents the sum of all responses to ASSIST Questions 1–8. With inclusion of the prescription opioids and stimulants categories, the global score has a potential range of 0 to 498. Following the standard approach to scoring the ASSIST, the SSIS is the sum of responses to ASSIST Questions 2–7, for each substance, and has a potential range of 0 to 39. ASSIST scores were further aggregated into two summary categories: ‘prescription drugs’ (prescription opioids, sedatives, and stimulants) and ‘illicit drugs’ (all other drugs, excluding tobacco and alcohol).

Risk level

WHO-recommended cutoffs were applied to SSIS scores for each substance to determine substance-specific levels of risk (low, moderate, or high).(17) In some analyses we collapsed the moderate and high risk levels into a single ‘moderate-high risk’ category because high risk substance use was relatively infrequent in our sample, and in a general medical setting the priority is to distinguish between individuals whose substance use requires clinical intervention (i.e., moderate or high risk use) versus those who do not require intervention.(17) Level of risk for the aggregate ‘prescription drugs’ and ‘illicit drugs’ categories was based on the highest risk level for any substance in that category.

Time required to complete the ACASI ASSIST was automatically recorded by the computer, while time for the IA ASSIST was recorded by the RA using a stopwatch. Following completion of both versions of the ASSIST, the RA administered a structured questionnaire asking participants if they “prefer to be asked these questions by a person (as an interview) or by the computer,” followed by a standard demographic questionnaire.

Biologic tests

Saliva and hair tests were conducted for cannabis, benzodiazepines, cocaine, amphetamines, opioids, and phencyclidine (PCP). Participants reported any medical use of prescription sedatives, opioids, or stimulants, and test results consistent with medical use were classified as ‘negative’ with respect to drug misuse. Saliva testing, which was performed with the Intercept™ immunoassay (OraSureTechnologies), has equivalent accuracy to urine drug screening tests, and a window of detection of up to 3 days for most drugs.(50–52) Hair testing, performed by Omega Laboratories, was conducted in a randomly selected sample of 39 individuals. Two samples had an insufficient quantity for testing, so 37 tests were reported.

Statistical analysis

We examined responses for an order effect with paired sample Mann-Whitney U tests comparing ASSIST scores for those who received the ACASI version first (N=191) against those who received the IA version first (N=202). This was done for global scores, SSIS for each substance, and for the combined categories of illicit drugs and of prescription drugs. Scores were examined for both the ACASI and the IA versions of the ASSIST. We found no significant difference in scores based on assignment to the ACASI-first versus IA-first group and proceeded to conduct the remainder of the analyses without regard to order of administration.

To achieve the primary aim of evaluating the ACASI ASSIST by comparison to the IA ASSIST, we conducted analyses of concordance, correlation, and agreement. Concordance of scores indicating low versus moderate-high risk use was examined for each substance, and for the combination categories of illicit drugs and prescription drugs. Concordance indicates whether the ACASI and IA ASSIST made the same classification of individuals whose substance use requires clinical intervention (due to moderate- or high-risk use) and those who do not require intervention (due to low-risk use). We examined the proportion of individuals who had either concordant risk levels or an increased or decreased risk level on the ACASI versus the IA ASSIST. Correlation of results, for low versus moderate-high risk use, was examined using Cohen’s Kappa. Kappa coefficients were computed for all substance classes having prevalence greater than 10% in the study population, and were not calculated for lower prevalence substances because the dependence of Kappa on prevalence can compromise its interpretation in conditions of markedly low prevalence.(53, 54) Exact 95% confidence intervals (CIs) were calculated, and Kappa coefficients were interpreted using standard cutoffs for level of agreement.(55)

We also examined correlation between the ACASI and IA ASSIST scores as continuous variables using Intraclass Correlation Coefficients (ICCs). The measures of concordance and correlation are complementary; concordance demonstrates exact agreement, while a correlation coefficient gives information about the degree of change. ICCs were calculated using a single measurement, absolute agreement definition, 2-way mixed model for the following scores: global score, SSIS for each substance, and the summary ‘illicit drugs’ and ‘prescription drugs’ scores. We additionally computed the ICC for the global score limited to the items that comprise ASSIST V3.0 (i.e. without including the prescription stimulant and prescription opioid items). ICCs were interpreted using recommended guidelines for reliability of clinical instruments.(56)

The Bland and Altman method was used to measure agreement between the ACASI and IA versions of the ASSIST. As opposed to the ICC, which compares two measures that are not ordered a priori with regard to accuracy, the Bland and Altman approach allows comparison of a new approach (ACASI ASSIST) against an established reference standard (IA ASSIST). Providing a measure of agreement in addition to the ICC is desirable because correlation only measures the strength of the linear relationship between two measures, and does not take into account the scale and true value of the item being measured. As a result, two measures that have very poor agreement can still be highly correlated.(57) Employing the standard Bland and Altman approach (58) we computed the limits of agreement for global scores generated for the ACASI versus IA ASSIST. Bland and Altman analyses for the illicit and prescription drug categories were not performed because differences for these scores did not follow a normal distribution.

As a secondary analysis, we compared ASSIST responses to results of saliva and hair testing, to evaluate the accuracy of self-report of current drug use on the ACASI and IA ASSIST. Results of the biologic tests were compared to ASSIST Question 2, which asks about use in the past 3 months. All analyses were conducted using IBM SPSS Statistics 20.

Results

Figure 1 shows recruitment enrollment data. After removing 6 individuals with missing ASSIST data, there were a total of 393 cases for analysis, of which 84% had saliva tests.

Figure 1.

Flowchart of participant recruitment

Participants had diverse demographic characteristics (Table 2). A limited set of demographic characteristics was also collected from eligible individuals who refused to participate. Non-participants were more frequently female (57%) and white (36%), and had a lower average age (42 years). Drug use characteristics of participants, based on responses to the IA ASSIST, are presented in Table 3.

Table 2.

Demographics of participants (N=393).

Characteristic	N (%)
Age (years)
Mean, SD	47, 12
Median	49
Range	19–65
Interquartile range	17
Gender
Female	190 (48.3)
Male	202 (51.4)
Transgender	1 (0.3)
Race/Ethnicity
Black/African American	176 (45.0)
White/Caucasian	60 (15.3)
Hispanic	112 (28.6)
Other	40 (10.2)
Don’t Know/Refused	3 (0.8)
Primary language
English	316 (80.6)
Spanish	36 (9.2)
Other	40 (10.2)
Country of birth
U.S.	265 (67.4)
Other	128 (32.6)
Education (highest level completed)
Less than HS	69 (17.6)
HS grad or GED	96 (24.4)
Some college or trade school	119 (30.3)
College degree (4-year)	90 (22.9)
Other	19 (4.8)
Employment
Employed	137 (34.9)
Unemployed	103 (26.2)
Other	152 (38.8)
Don’t know/Refused	1 (0.3)
Income
<$5,000	92 (23.5)
$5,000–14,999	90 (22.9)
$15,000–24,999	60 (15.3)
$25,000–49,999	69 (17.6)
≥ $50,000	23 (5.8)
Don’t know/Refused	59 (15.1)
Perceived health status*
Very good or excellent	98 (24.9)
Good	128 (32.6)
Fair or poor	163 (41.4)
Don’t know/Refused	4 (1.0)

^*“Would you say your health in general is excellent, very good, good, fair, or poor?”

Table 3.

Prevalence of substance use, based on responses to the interviewer-administered ASSIST, in the 393 participants. Risk categorization is based on standard ASSIST cutoffs.(17)

Substance	Lifetime use N (%)	Current use N (%)	Low risk N (%)	Moderate risk N (%)	High risk N (%)
Tobacco	254 (64.6)	135 (34.3)	238 (60.6)	128 (32.6)	27 (6.9)
Alcohol	337 (85.8)	210 (53.4)	323 (82.2)	52 (13.2)	18 (4.6)
Any illicit druga	240 (61.1)	82 (20.9)	282 (72.1)	91 (23.2)	18 (4.6)
Any prescription drugb	105 (26.7)	32 (8.1)	357 (91.1)	31 (7.9)	4 (1.0)
Specific drug class
Cannabis	232 (59.0)	58 (14.8)	324 (82.4)	64 (16.3)	5 (1.3)
Cocainea	156 (39.7)	31 (7.9)	328 (83.7)	54 (13.8)	10 (2.6)
Hallucinogens	83 (21.1)	7 (1.8)	379 (96.4)	13 (3.3)	1 (0.3)
Sedatives	76 (20.1)	19 (4.8)	372 (94.7)	20 (5.1)	1 (0.3)
Heroina	71 (18.1)	13 (3.3)	368 (93.9)	18 (4.6)	6 (1.5)
Prescription opioids	47 (12.0)	14 (3.6)	377 (95.9)	13 (3.3)	3 (0.8)
Prescription stimulantsb	40 (10.2)	14 (3.6)	375 (95.7)	16 (4.1)	1 (0.3)
Methamphetamine	42 (10.7)	0	388 (98.7)	5 (1.3)	0
Inhalants	39 (9.9)	3 (0.8)	384 (97.7)	9 (2.3)	0

^aMissing value for 2 participants

^bMissing value for 1 participant

The median time required to complete the ASSIST was 3.7 minutes (range 0.7–15.4) for the ACASI ASSIST, and 4.4 minutes (range 1.2–19.1) for the IA ASSIST. The majority (85%) of participants said they either preferred the computer to an interviewer, or had no preference. Twenty one (5.3%) participants requested assistance using the ACASI ASSIST, while 47 (12%) requested assistance with the IA ASSIST. For the ACASI ASSIST, 33% of requests were for technical assistance, while the remainder was for difficulty with comprehension of the items or reading. For the IA ASSIST, all requests were for assistance with comprehension, and 47% were to clarify non-medical use of prescription medications.

The ACASI and IA ASSIST results were in agreement for 92–99% of participants with respect to detection of moderate-high risk substance use (Table 4). Where there was lack of concordance between the two measures, more participants reported alcohol and illicit drug use on the ACASI ASSIST, and more participants reported tobacco and prescription drug misuse on the IA ASSIST. Kappa statistics indicated substantial to near-perfect agreement between the ACASI and IA ASSIST instruments.

Table 4.

Concordance of results from the ACASI and IA versions of the ASSIST, for detection of moderate-high risk substance use (N=393).

Substance Use Variable	Individuals in risk category N (%)		Concordant risk level N (%)	Higher on ACASI N	Lower on ACASI N	Kappa (95% CI)
	ACASI ASSIST	IA ASSIST
Tobacco			367 (93.4)	11	15	0.861(0.810–0.912)
Low risk	242 (61.6)	238 (60.6)
Mod or high risk	151 (38.4))	155 (39.4)
Alcohol			361 (91.9)	18	14	0.728(0.640–0.816)
Low risk	319 (81.2)	323 (82.2)
Mod or high risk	74 (18.8)	70 (17.8)
Illicit Drugs			364 (92.6)	21	8	0.822(0.759–0.885)
Low risk	271 (69.0)	284 (72.3)
Mod or high risk	122 (31.0)	109 (27.7)
Prescription Drugs			364 (92.6)	10	19	--
Low risk	367 (93.4)	358 (91.1)
Mod or high risk	26 (6.6)	35 (8.9)
Cannabis			367 (93.3)	20	6	0.7880.710–0.866)
Low risk	310 (78.9)	324 (82.4)
Mod or high risk	83 (21.1)	69 (17.6)
Cocaine			367 (93.4)	17	9	0.769(0.685–0.853)
Low risk	321 (81.7)	321 (81.7)
Mod or high risk	72 (18.3)	72 (18.3)
Heroin			382 (97.2)	9	2	--
Low risk	362 (92.1)	369 (93.9)
Mod or high risk	31 (7.9)	24 (6.1)
Hallucinogens			380 (96.7)	9	4	--
Low risk	374 (95.2)	379 (96.4)
Mod or high risk	19 (4.8)	14 (3.6)
Inhalants			389 (99.0)	2	2	--
Low risk	384 (97.7)	384 (97.7)
Mod or high risk	9 (2.3)	9 (2.3)
Methamphetamine			388 (98.7)	3	2	--
Low risk	387 (98.5)	388 (98.7)
Mod or high risk	6 (1.5)	5 (1.3)
Sedatives			374 (95.2)	7	12	--
Low risk	377 (95.9)	372 (94.7)
Mod or high risk	16 (4.1)	21 (5.3)
Opioids			380 (96.7)	6	7	--
Low risk	378 (96.2)	377 (96.0)
Mod or high risk	15 (3.8)	16 (4.0)
Stimulants			378 (96.2)	2	13	--
Low risk	387 (98.5)	376 (95.7)
Mod or high risk	6 (1.5)	17 (4.3)

Correlation of ACASI and IA ASSIST scores was excellent for the global ASSIST score and the substance specific scores for tobacco, alcohol, and the combined class of illicit drugs, and lower for prescription drugs (Table 5). We additionally examined the correlation between global scores when the items were restricted to those included in ASSIST V3.0, and found similar results to those derived using the modified ASSIST: mean score was 31 for the ACASI and 30 for the IA ASSIST; ICC 0.929 (95% CI 0.914 to 0.942).

Table 5.

Correlation between ACASI and IA ASSIST, based on the intraclass correlation coefficient (ICC), N=393.

Substance Use Variable	ACASI	Interviewer	ICC
	Mean score ± SD	Mean score ± SD
Global ASSIST score	31.6 ± 32.6	31.3 ± 32.7	0.937 (.924–.948)
Tobacco	6.5 ± 9.3	7.0 ± 9.9	0.927 (.912–.940)
Alcohol	6.1 ± 8.3	5.8 ± 8.3	0.912 (.893–.927)
Illicit Drugs	6.9 ± 13.4	6.3 ± 12.8	0.854 (.854–.900)
Cannabis	2.6 ± 5.8	2.5 ± 5.8	0.858 (.829–.882)
Cocaine	2.4 ± 6.2	2.2 ± 6.1	0.861 (.833–.884)
Heroin	1.0 ± 4.3	1.0 ± 4.4	0.924 (.908–.937)
Hallucinogens	0.5 ± 2.4	0.4 ± 2.2	0.748 (.700–.788)
Inhalants	0.2 ± 1.4	0.2 ± 1.5	0.903 (.883–.920)
Methamphetamine	0.1 ± 0.8	0.1 ± 0.9	0.461 (.379–.535)
Rx Drugs	3 ± 9	2 ± 6	0.676 (.613–.729)
Sedatives	0.3 ± 2.3	0.6 ± 3.0	0.728 (.676–.772)
Opioids	0.4 ± 3.0	0.6 ± 3.2	0.939 (.926–.950)
Stimulants	0.1 ± 1.0	0.4 ± 2.4	0.258 (.164–.347)

Analysis of agreement using the Bland and Altman approach compared ACASI and IA ASSIST global scores (Figure 2). Twenty-four individuals had scores that fell outside the limits of agreement. Four of them reported illicit drug use on the ACASI but not on the IA ASSIST, reflecting a similar pattern to the concordance results (Table 4), in which reporting of illicit drug use was higher on the ACASI instrument.

Figure 2.

Bland and Altman analysis of differences, for ACASI ASSIST and IA ASSIST global scores.

Among the 331 individuals with completed saliva tests, 19 (5.7%) tested positive for at least one drug. Of those with a positive saliva test, 12 participants reported use on both versions of the ASSIST, and 2 participants reported use on the ACASI ASSIST but not the IA ASSIST. No participants with positive saliva tests reported current drug use on the IA ASSIST but not the ACASI ASSIST. For the 37 participants who participated in hair testing, 5 (14%) tested positive for at least one drug. Among those with a positive hair test, there were no differences in reporting of current drug use on the ACASI versus IA ASSIST. Hair test results were congruent with the ASSIST results in 33 cases (89%). Of those that were incongruent, one test was positive for cocaine and opioids in an individual who reported only current use of opioids on the ASSIST, and 3 tests were positive in individuals who reported no current drug use on the ASSIST.

Discussion

In this large sample of primary care patients, we found high levels of concordance, correlation, and agreement between responses to the ACASI ASSIST and the IA ASSIST. The ACASI ASSIST was well accepted and feasible for self-administration, with just 5% of participants requesting assistance to complete it. The ACASI ASSIST appears to be a valid alternative to the traditional IA ASSIST for identifying moderate-high risk substance use in primary care patients.

Although the overall concordance between the two modalities was high, we observed slightly more reporting of alcohol and illicit drug use on the ACASI ASSIST. This finding is consistent with multiple prior studies showing that self-administered instruments generate higher rates of reporting of stigmatized behaviors.(39, 40, 59)

Limitations

Our study does have limitations. Although our analyses did not reveal an order effect, repeat administration of a similar instrument has the potential to bias responses. Despite the overall high prevalence of substance use in our sample, few participants reported use of certain drug classes queried by the ASSIST, and relatively few had high-risk use of any substance. This limited our ability to draw some comparisons between the ACASI and IA ASSIST versions.

We examined the ACASI ASSIST only in comparison to the IA ASSIST. Supporting this approach is the fact that the IA ASSIST has been previously validated in a large multi-site study, and demonstrated sufficiently high sensitivity and specificity to be considered a reference standard measure.(18) Yet because both the IA and ACASI versions of the ASSIST rely on self-reported information, we are left with some uncertainty about which instrument best captures the truth about participants’ drug use. Self-report measures have consistently shown good accuracy in research,(60–63) but are nonetheless dependent on accurate and truthful disclosure. The biologic tests for drug use generally supported the self-reported responses to the ACASI ASSIST, but are limited by the relatively brief window of detection for saliva tests, the small proportion of participants who could be offered hair testing, and the lack of additional measures of alcohol use.

Generalizability of our findings to other clinical settings is limited by having conducted the study at a single adult primary care clinic site, and restricting eligibility to English speakers under the age of 65 years. The diversity of our sample, which had good representation of individuals with low levels of formal education, can be considered a strength, since a computer self-administered approach may be more challenging in populations with limited literacy and computer skills.(21, 24, 64) However, it is possible that substance use screening using an ACASI approach would be less acceptable in other patient populations, such as highly educated or elderly patients. The feasibility and validity of the ACASI ASSIST in other languages would need to be assessed before non-English versions can be recommended for use in clinical practice.

Conclusion

The ACASI ASSIST can be recommended for use in primary care settings as an alternative to the traditional interviewer-administered instrument. An ACASI ASSIST could be completed prior to the medical visit, either in the waiting area or at home via an internet portal, and have its results incorporated into the electronic health record at the point of care. This approach has the potential to ease barriers to implementation of substance use screening in health care settings.

While the ASSIST instrument combines screening and assessment, in some clinical settings it may be attractive to use the ACASI ASSIST only as an assessment tool, reserved for patients who have positive responses on an initial brief screen. Validated tools exist for accomplishing alcohol and drug screening in as few as two questions.(65–68). However, because as many as one-third of patients may be expected to have a positive screening result (66), an efficient approach to assessment is still essential. The ACASI ASSIST could provide this assessment efficiently and with enough detail and accuracy to guide clinical interventions to address unhealthy substance use in primary care patients.