Performance of the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) Tool for substance use screening in primary care patients

Abstract

Background

Substance use is a leading cause of morbidity and mortality that is under-identified in medical practice.

Objective

The Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) Tool was developed to address the need for a brief screening and assessment instrument that includes all commonly used substances, and fits into clinical workflows. The goal of this study was to assess the performance of the TAPS Tool in primary care patients.

Design

Multi-site study conducted within the National Drug Abuse Treatment Clinical Trials Network, comparing the TAPS Tool against a reference standard measure.

Setting

Five adult primary care clinics.

Participants

2,000 adult patients were consecutively recruited from clinic waiting areas.

Measurements

Interviewer- and self-administered versions of the TAPS Tool were compared to the reference standard modified Composite International Diagnostic Interview (CIDI), which measures problem use and substance use disorders (SUD).

Results

Interviewer- and self-administered versions of the TAPS Tool had similar diagnostic characteristics. For identifying problem use (at a cutoff of 1+), the TAPS Tool had sensitivity 0.93 (95% CI 0.90–0.95) and specificity 0.87 (95% CI 0.85–0.89) for tobacco, and sensitivity 0.74 (95% CI 0.70–0.78), specificity 0.79 (95% CI 0.76–0.81) for alcohol. For problem use of illicit and prescription drugs, sensitivity ranged from 0.82 (95% CI 0.76–0.87) for marijuana to 0.63 (95% CI 0.47–0.78) for sedatives, and specificity was 0.93 or higher. For identifying any SUD, sensitivity was lower, but a score of 2+ greatly increased the likelihood of having a SUD.

Limitations

Low prevalence of some drug classes led to poor precision in some estimates. Research assistants were not blinded to the participant’s TAPS Tool responses when they administered the CIDI.

Conclusions

In a diverse population of adult primary care patients, the TAPS Tool detected clinically relevant problem substance use. While it may also detect tobacco, alcohol, and marijuana use disorders, further refinement is needed before the TAPS Tool can be broadly recommended as a screener for SUD.

Introduction

Tobacco and alcohol use are among the leading causes of preventable death in the US, (1, 2) and illicit substance use is a significant contributor to the HIV and opioid overdose epidemics(3, 4). Health care settings offer an opportunity to identify substance use and related problems, provide timely interventions, and link patients to treatment. Tobacco screening and treatment is a core clinical quality measure for primary care(5), and screening followed by brief intervention (BI) for unhealthy alcohol use is recommended by the United States Preventive Services Task Force(6–9). While the efficacy of BI for reducing drug use has not been clearly established(10–13), screening for substance use in medical settings may be clinically justified by the impact of substance use on the prevention and treatment of other medical conditions(14–16), and the potential for drug-medication interactions(17, 18) and overdose(19).

Primary care settings require a screening and assessment approach that is efficient, accurate, and informs clinical care(20–23). Very brief screening tools to efficiently identify alcohol and drug use have been developed(24–27), but they do not provide enough information about the specific substances used, or the patient’s risk level, to guide clinical actions. A structured substance use assessment can provide this information, but current options are either too lengthy(28), or do not provide sufficient detail(29, 30) to meet the needs of medical providers.

We developed the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) Tool as a more optimal instrument for substance use screening in primary care settings. The TAPS Tool consists of a 4-item screen for tobacco, alcohol, illicit drugs, and non-medical use of prescription drugs, followed by a substance-specific assessment of risk level for individuals who screen positive. The TAPS Tool has the flexibility to be administered face-to-face or self-administered using a tablet computer, to accommodate a variety of clinical workflows. The current paper presents the results from a large multi-site study conducted within the National Institute on Drug Abuse (NIDA) National Drug Abuse Treatment Clinical Trials Network (CTN) to assess the performance of the TAPS Tool in comparison to a reference standard measure in adult primary care patients.

Methods

Participants & Recruitment

Participants were recruited from five primary care clinics located in Baltimore, MD; New York, NY; Richmond, VA; and two sites in Kannapolis, NC. Clinics were selected to provide a geographically diverse sample of urban and suburban participants, which would be expected to vary in their substance use patterns. A sample size of 2,000 was set prior to initiating recruitment, and sites enrolled competitively from August, 2014 to April, 2015, until the total sample was achieved. Sample size was determined through computer simulations to calculate the precision of the estimates of sensitivity and specificity as a function of the number of participants and substance use prevalence. We determined that a sample size of 2,000 allowed us to estimate sensitivity with precision (defined as 95th percentile of the half-width of the 95% confidence interval) of 12.5% for substances with a prevalence of 5%, up to 2.5% for substances with a prevalence of 50%.

Eligibility Criteria

Adults 18 years and older who were in clinic for a medical visit and able to provide informed consent were eligible to participate. Individuals were excluded if they were unable to comprehend spoken English, physically unable to complete the self-administered TAPS Tool, or previously enrolled in this study.

Recruitment

Research assistants (RAs) consecutively approached each individual in the waiting area. Interested patients were screened for eligibility and provided verbal consent. Participants were given the option of having the study visit before their appointment if they had more than a 1-hour wait time, otherwise the study visit was done after the medical visit.

Study Procedures

Study visits were completed in a private room. Participants were assigned a unique identifier and informed that responses were confidential, and would not be shared with anyone in the clinic. The electronic data capture system randomly assigned half of the participants to begin with the self-administered TAPS Tool, and half to begin with the interviewer-administered TAPS Tool. After completing the TAPS Tool in the first format (e.g. self-administered), they completed it in the alternate format (e.g. interviewer). The self-administered version was delivered on a tablet computer (iPad^®) that gave participants the option of hearing the questions and response options read verbatim by a recorded female voice. For the interviewer version, all questions and response options were read aloud by the RA. Following completion of both versions, the RA administered a questionnaire about the feasibility and acceptability of the TAPS Tool. This was followed by additional substance use measures that were collected for comparison purposes. RAs were not blinded to participant responses on the interviewer-administered TAPS Tool when they administered the reference standard measure (CIDI). After completing self-reported assessments, all participants were asked to provide verbal consent to participate in oral fluid testing for drugs. Individuals received $20 for the main study and an additional $10 for oral fluid testing. All study procedures were approved by local Institutional Review Boards (Duke University Health System, Friends Research Institute, New York University School of Medicine, and Virginia Commonwealth University).

Measures

Experimental instrument: TAPS Tool

The TAPS Tool was developed as a two-step screening and brief assessment tool from instruments that had not been validated. TAPS-1, the screening component, was adapted from the National Institute on Drug Abuse (NIDA) Quick Screen V1.0.(31) TAPS-2, the brief assessment component, is a modified version of the Alcohol, Smoking, and Substance Involvement Screening Test-Lite (ASSIST-Lite).(32) A prototype of the TAPS Tool was evaluated using cognitive interviewing (33–35) with 30 adult primary care patients from 3 of the study sites. Minor modifications to the wording of the TAPS Tool items were made based on these interviews, prior to finalizing the instrument that was used in this study.

The TAPS Tool is shown in Figure 1. The TAPS-1 asks about frequency of use in the past 12 months of tobacco, alcohol above guideline-recommended daily limits (>5 drinks/day for men, >4 drinks/day for women),(36) illicit drugs, and non-medical use of prescription medications (sedatives, opioids, and stimulants). Respondents choose from five response options ranging from ‘never’ to ‘daily or almost daily.’ The TAPS-2 assesses use in the past three months of tobacco, alcohol, six different classes of illicit drugs, and ‘other’ drugs using a yes/no format. When the answer is ‘yes’ to any use, the participant receives 2–3 follow-up items specific to that substance class.

Figure 1.

TAPS Tool Part 1 and Part 2, showing items, skip pattern, and scoring system

For the TAPS-1, any response other than ‘never’ constitutes a positive screen. Those with a positive screen complete the corresponding items on the TAPS-2, and responses are summed within each substance class to generate a substance-specific risk score. TAPS Tool scores have a potential range of 0–3 for tobacco and other drugs, and 0–4 for alcohol. In clinical practice the TAPS-2 items would only be administered to those with a positive screen, but for the purposes of the study, participants completed all TAPS-2 items, regardless of their TAPS-1 responses.

Reference measures

The modified Composite International Diagnostic Interview, Second Edition, Substance Abuse Module (CIDI) was the ‘gold standard’ reference measure(37–41). The CIDI has been widely used in epidemiological and clinical research to assess substance use disorders (SUDs) based on the Diagnostic and Statistical Manual of Mental Disorders (DSM), 4^th edition. As in our previous research(42), we used the existing CIDI items that mapped onto the DSM-5 SUD classifications (by omiting the item on legal problems and including the CIDI item on craving).

Problem use was defined as past-year use with endorsement of one or more items on the CIDI. This approach has been used in prior screening tool studies to identify clinically important substance use that may not be severe enough to meet criteria for a SUD(42, 43). SUDs were defined using the standard diagnostic threshold of meeting two or more DSM criteria on the CIDI.

Oral Fluid Testing

The Intercept^™ immunoassay (OraSure Technologies) provided an objective measure of point prevalence for the following drug classes: marijuana, cocaine, PCP, opiates, amphetamines, opiates, benzodiazepines, and barbiturates. It has a window of detection of up to 3 days for most drugs(44–46). To assist in interpretation of results, participants were assessed for medical use of medications that would be detected by the test. Oral fluid test results were not shared with participants, and were linked to the self-report data by the participant’s unique identifier.

Statistical Analysis

Concurrent validity of the interviewer-administered and self-administered versions of the TAPS Tool, in comparison to the reference standard CIDI, was assessed for the risk categories of problem use and SUD for each substance class. We calculated sensitivity, specificity, and positive and negative diagnostic likelihood ratios (LRs)(47), with exact 95% confidence intervals (CIs). There were 5 participants who did not complete the entire CIDI, and one participant who did not complete the entire interviewer-administered TAPS Tool. When the score for the CIDI or TAPS Tool was missing for a given substance, these cases were excluded from the analyses. For identification of problem use, we selected cutoffs that maximized sensitivity. For SUD, cutoffs were selected that were both higher than the cutoff for problem use and had highest sensitivity.

To assess for differences based on order of administration of the TAPS Tool, we examined differences in prevalence for each substance related to order of administration and instrument (self-administered vs interviewer-administered) using a generalized estimating equation (GEE) approach.(48, 49) A two-way crossed model was fit, and to account for the crossover nature of the data, an unstructured working correlation matrix was used under the GEE framework. The most important parameter of interest was the interaction between order and instrument. This variable was highly significant for alcohol, suggesting there is a difference in the participant responses to self-administered and interviewer-administered TAPS tool due to order of administration. Reporting of alcohol use was higher (38.6%) among participants who had the self-administered tool first, compared to participants who had the self-administered tool second (30.7%). For substances other than alcohol, analyses of diagnostic accuracy were conducted without regard to order of administration. For alcohol, analyses were also conducted separately, restricted to the first version received (see Supplementary Tables 3a and 3b). All analyses were conducted in SAS^® version 9.3 or 9.4.

Role of the funding source

The study was supported by a National Institute on Drug Abuse (NIDA) Center for Clinical Trials Network (CCTN) Cooperative Agreement. Co-authors include NIDA CCTN staff (GS and CC), who contributed to protocol development including study design and methods, conduct, and the preparation of the manuscript.

RESULTS

Participants

A total of 12,473 individuals were assessed for eligibility, of which 48% were eligible for participation (Figure 2). Among those eligible, 35% agreed to participate and were randomized. Of the 2,000 participants who completed the measures, 1,802 (90%) subsequently agreed to oral fluid testing.

Figure 2.

Recruitment summary

As shown in Table 1, the mean (SD) age of the participants was 46.0 (14.7) and more than half (56.2%) were women. The prevalence of past-year use, as reported on the modified CIDI, is shown in Table 2. Among the participants who provided a sample for oral fluid testing, point prevalence was 11.3% for illicit substances, and 5.0% for non-medical use of prescription medications.

Table 1.

Demographic characteristics of participants (N=2000)

Characteristic	N (%)
Gender
Male	874 (43.7%)
Female	1124 (56.2%)
Other/Refused	2 (0.1%)
Age (years) at Enrollment
Mean (SD)	46.0 (14.7)
Range	18–94
Ethnicity
Hispanic	233 (11.7%)
Non-Hispanic	1761 (88.1%)
Other/Refused	6 (0.3%)
Race
White	667 (33.4%)
Black/African American	1112 (55.6%)
Asian	35 (1.8%)
Multiracial	66 (3.3%)
Other*	113 (5.7%)
Refused	7 (0.4%)
Highest Completed Education Level
Less than high school	383 (19.2%)
High school graduate or GED	578 (28.9%)
Some college	426 (21.3%)
Associate’s degree	224 (11.2%)
Bachelor’s degree	279 (14.0%)
Graduate degree	109 (5.5%)
Don’t know	1 (0.1%)
Employment
Employed	712 (35.6%)
Unemployed	419 (21.0%)
Disability	472 (23.6%)
Retired	172 (8.6%
Student	118 (5.9%)
Other†	107 (5.4%)

^*Race-Other category refers to American Indian or Alaska Native (N=12), Unknown (N=32), and Other/not specified (N=69).

^†Employment-Other category refers to ‘keeping house’ (N=66) and Other/not specified (N=41).

Table 2.

Substance use prevalence, based on reference standard measures

	Past Year Use (from CIDI) N=2,000 N (%)	Point Prevalence (from Oral Fluid Test) N=1802 N (%)
Substance
Tobacco	882 (44.1%)	N/A
Alcohol	1239 (62.0%)	N/A
Illicit substance(s)*	511 (25.6%)	204 (11.3%)
Prescription Medication(s)†	147 (7.4%)	91 (5.0%)
Substance Class
Marijuana	416 (20.8%)	103 (5.7%)
Cocaine	145 (7.3%)	85 (4.7%)
Methamphetamine	14 (0.7%)	4 (0.2%)
Heroin	78 (3.9%)	20 (1.1%)
Prescription Opioids	96 (4.8%)	90 (5.0%)
Sedatives	82 (4.1%)	19 (1.1%)
Prescription Stimulants	23 (1.2%)	2 (0.1%)
Combined Substance Categories
Opioids (Heroin or Prescription Opioids)	145 (7.3%)	93 (5.2%)
Stimulants (Cocaine, Methamphetamine, or Prescription Stimulants)	161 (8.1%)	90 (5.0%)
Drugs Other than Marijuana‡	265 (13.3%)	181 (10.0%)

Notes to Table 2

N=2,000 for the CIDI, and N=1802 for point prevalence because some participants declined to participate in the optional oral fluid test.

^*Includes marijuana, cocaine/crack, heroin, methamphetamine, hallucinogens, and inhalants

^†Includes prescription opioids, amphetamine-containing medications, and sedatives.

^‡Includes all drug classes with the exception of marijuana, alcohol, and tobacco

Acceptability of the TAPS Tool

The vast majority (99%) of participants said they felt comfortable answering the TAPS Tool questions, and said they would be comfortable sharing the results with their doctor (95%). Participants differed in their preferences for interviewer-versus self-administered screening: 24% preferred the iPad, 31% preferred the interview, and 45% had no preference.

Comparison of Interviewer and Self-administered versions

Overall, the interviewer-administered and self-administered versions of the TAPS Tool performed similarly, and generated the same cutoffs for problem use and SUD. Because they generated similar results, only the interviewer-administered version is presented in Table 3, while data for the self-administered version is in Supplemental Table 2. Small differences in sensitivity were observed, but they did not follow a consistent pattern. Specificity was almost uniformly high across substance classes, for both versions.

Table 3.

Diagnostic accuracy of the interviewer-administered TAPS tool for identifying unhealthy use and substance use disorder (N=2000)

	True Positive (n)	False Negative (n)	True Negative (n)	False Positive (n)	Sensitivity (95% CI)	Specificity (95% CI)	Positive LR (95% CI)	Negative LR (95% CI)
Risk Level: Problem use (1 or more items positive on CIDI)
Tobacco (Cutoff=1)	599	47	1174	179	0.93 (0.90, 0.95)	0.87 (0.85, 0.89)	7.01 (6.10, 8.05)	0.08 (0.06, 0.11)
Alcohol (Cutoff=1)	352	122	1198	327	0.74 (0.70, 0.78)	0.79 (0.76, 0.81)	3.46 (3.10, 3.86)	0.33 (0.28, 0.38)
Marijuana (Cutoff=1)	189	42	1637	128	0.82 (0.76, 0.87)	0.93 (0.91, 0.94)	11.28 (9.45, 13.47)	0.20 (0.15, 0.26)
Cocaine, meth* (Cutoff=1)	82	38	1855	20	0.68 (0.59, 0.77)	0.99 (0.98, 0.99)	64.06 (40.74, 100.73)	0.32 (0.25, 0.42)
Heroin (Cutoff=1)	54	15	1919	6	0.78 (0.67, 0.87)	1.00 (0.99, 1.00)	251.09 (111.86, 563.60)	0.22 (0.14, 0.34)
Rx opiate† (Cutoff=1)	42	17	1908	28	0.71 (0.58, 0.82)	0.99 (0.98, 0.99)	49.22 (32.93, 73.57)	0.29 (0.20, 0.44)
Sedative‡ (Cutoff=1)	26	15	1926	28	0.63 (0.47, 0.78)	0.99 (0.98, 0.99)	44.25 (28.64, 68.38)	0.37 (0.25, 0.56)
Risk Level: Substance use disorder (2 or more items positive on CIDI)
Tobacco (Cutoff=2)	372	134	1332	161	0.74 (0.69, 0.77)	0.89 (0.88, 0.91)	6.82 (5.84, 7.96)	0.30 (0.26, 0.34)
Alcohol (Cutoff=2)	194	84	1466	255	0.70 (0.64, 0.75)	0.85 (0.83, 0.87)	4.71 (4.11, 5.40)	0.35 (0.30, 0.42)
Marijuana (Cutoff=2)	105	42	1764	85	0.71 (0.63, 0.79)	0.95 (0.94, 0.96)	15.54 (12.33, 19.58)	0.30 (0.23, 0.39)
Cocaine, meth* (Cutoff=2)	61	46	1873	15	0.57 (0.47, 0.67)	0.99 (0.99, 1.00)	71.76 (42.23, 121.94)	0.43 (0.35, 0.54)
Heroin (Cutoff=2)	43	22	1926	3	0.66 (0.53, 0.77)	1.00 (1.00, 1.00)	425.37 (135.50, 1335.39)	0.34 (0.24, 0.48)
Rx opiate† (Cutoff=2)	23	25	1941	6	0.48 (0.33, 0.63)	1.00 (0.99, 1.00)	155.49 (66.35, 364.39)	0.52 (0.40, 0.69)
Sedative‡ (Cutoff=2)	15	13	1947	20	0.54 (0.34, 0.72)	0.99 (0.98, 0.99)	52.69 (30.22, 91.86)	0.47 (0.32, 0.70)

Notes to Table 3. CI = Confidence Interval; LR=Likelihood Ratio; CIDI = Composite International Diagnostic Interview Composite Index; TAPS = Tobacco, Alcohol, Prescription medication, and other Substance use

There were 5 participants who did not complete the entire CIDI, and one participant who did not complete the entire interviewer-administered TAPS Tool. Therefore, when the score for CIDI or TAPS Tool is missing for a given drug, these cases were excluded from the analyses.

^*Described in TAPS as ‘Cocaine (crack) or methamphetamine’

^†Described in TAPS as ‘Prescription opiate pain reliever’

^‡Described in TAPS as ‘Medication for anxiety or sleep’

Identification of problem use (Table 3)

The optimal cutoff score for problem use was 1+. At this cutoff, the interviewer-administered TAPS Tool had good sensitivity and specificity for identifying any problem use of tobacco (0.93 and 0.87, respectively) and alcohol (0.74 and 0.79, respectively). For illicit drugs, sensitivity ranged from 0.82 (marijuana) to 0.68 (cocaine). For non-medical use of prescription drugs, sensitivity was 0.71 for opiates and 0.63 for sedatives. The 95% confidence intervals were broad for substances with lower prevalence in the study population. Specificity for identifying problem use was high (0.93 or greater) for all illicit and prescription drug classes. Positive LRs ranged from 3.5 for tobacco to over 250 for heroin, while negative LRs ranged from 0.08 for tobacco to 0.37 for sedatives.

Identification of substance use disorder (Table 3)

The optimal cutoff score on the TAPS Tool for identifying SUD was 2+. A lower cutoff of 1+ had higher sensitivity for identifying SUD, but because this was the optimal cutoff for problem use, it was not selected. Conversely, a higher cutoff of 3 demonstrated high specificity for identifying SUDs, but was not selected due to unacceptably low sensitivity (<0.50). At a score of 2+, sensitivities were lower for identification of SUD than for problem use (although they were 0.70 or greater for tobacco, alcohol and marijuana), and specificities were no lower than 0.85. Positive LRs ranged from 4.7 to over 990, and negative LRs ranged from 0.30 to 0.52. The small number of participants with prescription stimulant use, (7 with problem use and 4 with SUD were identified on the CIDI), did not allow us to make meaningful estimates for this substance class.

DISCUSSION

This multi-site study of a substance use screening instrument found that for those substances that are most commonly used by primary care patients (i.e. tobacco, alcohol, and marijuana), the TAPS Tool has good sensitivity and specificity for identifying problem use. For substances that are less frequently encountered in primary care, sensitivity and specificity estimates were lower and less precise, and sensitivity for detection of SUD was unacceptably low. Thus, while the TAPS Tool cutoffs of 1+ for problem use and 2+ for SUD can be applied for alcohol, tobacco, and marijuana, for other drugs, any patient with a score of 1+ should be further assessed for presence of a SUD.

In comparison to other substance use screening instruments, the TAPS Tool has several characteristics that make it attractive for primary care. First, the TAPS Tool screens and assesses tobacco, alcohol, and all major drug classes in a single instrument. It has the potential to be easily integrated into regular clinical workflows, which in most settings have already been designed to screen for tobacco use (required under Meaningful Use Stage 2(50)). Second, the TAPS Tool gives substance-specific risk information, which is essential for ensuring patient safety, providing feedback and education, and guiding treatment decisions. Neither the widely-used Drug Abuse Screening Test (DAST-10)(25, 51) nor the newer Screen of Drug Use (SoDU)(29) provides this level of detail. Third, the TAPS Tool provides the option of a patient self-administered format, which can facilitate more accurate reporting of stigmatized behavior(52, 53), ensure fidelity of administration(54, 55), increase patient comfort(56), and reduce the burden on staff. It has the potential to be completed through a web-based patient portal, or on a kiosk or tablet computer in the clinic, that would upload screening results into the electronic health record.

In some practice settings, there may still be a role for very brief screeners (e.g. Substance Use Brief Screen (SUBS)(27), or single-item screening questions (SISQs) for alcohol and drugs(24–26)) to quickly identify patients with any unhealthy substance use. The four-item TAPS-1 screener could potentially accomplish this, and future reports will examine its performance as a stand-alone instrument. Yet, for the significant proportion of patients who are positive on a brief screener (28–29% in some studies(26, 27)), it is essential to have an efficient follow-up assessment that can risk-stratify patients and guide care. For alcohol, this can be accomplished with the AUDIT or AUDIT-C, which have been widely adopted(36, 43, 57, 58). There has been no similarly brief structured assessment tool for other drugs. Until now, the WHO ASSIST has been the only screening tool that provides substance-specific risk stratification for drugs, but its length and complexity has hindered its implementation in primary care settings(32, 59), (though a computer self-administered version could be more feasible(60)). The TAPS Tool streamlines the ASSIST to perform this assessment relatively quickly. Future research may explore whether the TAPS-2 could be simplified even further, either by reducing either the number of substances queried or the number of items.

The TAPS Tool has some shortcomings. Sensitivity was low for detecting problem use of some substances. It is possible that this reflects differences between the timeframe of our reference standard measure and the TAPS Tool. Although the TAPS Tool screens for use in the past 12 months (TAPS-1), the final score is based on use in the past 3 months (TAPS-2). As a result, the TAPS Tool could fail to identify individuals who had problem use in the past year that had not continued into the most recent 3-month period. However, by focusing on current use, the TAPS Tool identifies those patients who are most in need of clinical intervention, which is important in primary care settings where providers have multiple demands on their time. We have limited ability to draw comparisons with other instruments that screen for illicit and prescription drug use, because only the ASSIST, (and its shortened version the ASSIST-Lite), provide substance-specific results. Validation studies of the ASSIST have not reported its diagnostic accuracy at the WHO-recommended cutoffs, and were conducted in samples that included drug treatment and psychiatric patients(28, 61, 62). The ASSIST-Lite (on which the TAPS-2 is based) was developed from secondary analysis of data from the validation study of the full ASSIST(32), and its performance has not been previously assessed.

Performance of the TAPS Tool for detecting problem use and SUD was lowest for prescription medications, particularly on the self-administered version, but still compares favorably to an existing brief screening tool that specifically queries non-medical use of prescription drugs(27). The relatively low sensitivity of screening for this substance class could be due to confusion among participants about what constitutes non-medical use(63), poor comprehension, or question fatigue due to the length and complexity of these items on the TAPS-1. The variability in how non-medical use is described in the TAPS-1 versus the TAPS-2 could have confused some respondents. In practice, performance of the TAPS and similar screening tools may be further compromised by patients’ reluctance to disclose misuse of a medication to the physician who is prescribing it.

Interviewer-administered screening approaches can be challenging to implement in practice because they require staff time and training, and interviewers may modify the screening language in ways that compromise the tool’s accuracy(54, 64). Self-administered tools could help to promote disclosure of substance use(52, 53, 65–67), but this format may not be feasible in all practice settings. Self-administration on an iPad could be problematic in patients with low literacy or poor vision, though the audio guidance can help to address these barriers. Elderly patients may have difficulty using an iPad. Tablet computers are currently uncommon in primary care settings, and their use would require considerations for workflow, security, and hygiene.

Our study design has several limitations. Although our sample was large, for most drug classes we did not have enough participants with problem use or SUD to develop and then test the cutoffs in separate samples. The low prevalence of certain drugs also led to poor precision in some estimates, particularly with respect to identifying a SUD. RAs administering the CIDI were not blinded to participant responses on the interviewer-administered TAPS Tool, which could potentially bias the CIDI responses. Our analyses relied on self-reported substance use, which has consistently shown good accuracy in research studies(68–71), but nonetheless depends on accurate and truthful disclosure of use. Data were collected with an assurance of confidentiality and anonymity, as they have been for all prior studies examining the concurrent validity of substance use screening tools, to increase the accuracy of self-report. In clinical practice, where patients are aware that medical providers will view their screening results, the diagnostic accuracy of the TAPS Tool might be expected to differ(54, 56, 69, 72).

The TAPS Tool was developed and evaluated only in English, which may limit its application in some settings. Though we recruited from clinics that had geographic and demographic diversity, our participants may not be representative of patients seen in all primary care settings. In particular, we had a high proportion of African-Americans and individuals with lower levels of education in our sample. Many eligible patients declined to participate in the study, and we are unable to assess how their inclusion would have impacted the performance of the TAPS Tool. Our study also has notable strengths, including enrollment of a large and diverse sample of adult primary care patients, high rates of completion of the reference standard measures, and a rigorous approach to testing both a self-administered and interviewer-administered version of the TAPS Tool.

Conclusion

Having information about a patient’s substance use is essential for ensuring the quality and safety of medical care. This study supports the use of the TAPS Tool (at a cutoff of 1+) in screening primary care patients for problem substance use. It may also detect alcohol, tobacco, and marijuana use disorders, although further refinement is needed before it can be broadly recommended as a screener for SUD. Because it identifies problem use of all commonly used substances with a limited number of questions, and has the flexibility to be either self-administered or completed as an interview, the TAPS Tool has the potential to ease barriers to incorporating substance screening into busy clinical environments.