Evaluating brief screeners to discriminate between drug use disorders in a sample of treatment-seeking adults

Abstract

Objective

To identify a potential core set of brief screeners for the detection of individuals with a substance use disorder (SUD) in medical settings.

Method

Data were from two multisite studies that evaluated stimulant use outcomes of an abstinence-based contingency management intervention as an addition to usual care (National Drug Abuse Treatment Clinical Trials Network [CTN] trials 006-007). The sample comprised 847 substance-using adults who were recruited from 12 outpatient substance abuse treatment settings across the United States. Alcohol and drug use disorders were assessed by the DSM-IV Checklist. Data were analyzed by factor analysis, item response theory (IRT), sensitivity, and specificity procedures.

Results

Comparatively prevalent symptoms of dependence, especially inability to cut down for all substances, showed high sensitivity for detecting a SUD (low rate of false negative). IRT-defined severe (infrequent) and low discriminative items, especially withdrawal for alcohol, cannabis, and cocaine, had low sensitivity in identifying cases of a SUD. IRT-defined less severe (frequent) and high discriminative items, including inability to cut down or taking larger amounts than intended for all substances and withdrawal for amphetamines and opioids, showed good-to-high values of area under the receiver operating characteristic curve in classifying cases and non-cases of a SUD.

Conclusion

Findings suggest the feasibility of identifying psychometrically reliable substance dependence symptoms to develop a two-item screen for alcohol and drug disorders.

1. Introduction

Substance use is considered common in medical settings as it increases risk for injuries and a wide range of health problems [1]. However, substance use and related problems are not routinely assessed and managed in medical settings [2,3]. In the United States, treatment for substance use disorders (SUDs) occurs primarily in specialty substance abuse service settings that are often separated from mainstream health care [3,4]. The absence of infrastructural support for substance abuse services in medical care settings can be associated with a lack of routine preventive services for SUDs, contributing to underdetection or undertreatment. Data from the National Ambulatory Medical Care Survey show that SUD diagnoses are recorded in only 0.9% of general and family practice visits, 0.8% of internal medicine visits, and 5.1% of psychiatric visits [5]. Similarly, surveys of the general population indicate that the vast majority of adults with a SUD in the past year have not used any substance abuse service during the year [6,7]. Recent findings from the Treatment Episode Data Set further confirm that failure to make prompt treatment contact is a pervasive unmet need for substance abuse care: An average of 15.6 years elapsed between first use of the primary substance of abuse and first-time treatment entry [8].

The majority of individuals make at least one medical visit to a doctor yearly [9]. Medical contacts provide a window of opportunity for clinicians to screen for substance problems, intervene, or coordinate substance abuse care within mainstream health care. The Patient Protection and Affordable Care Act of 2010 emphasizes prevention and integration of SUD treatments into the mainstream health care system [10]. When fully enacted, preventive services for substance problems (screening, brief intervention) will be covered, and SUD treatments will be provided within the primary care setting [11]. However, the potential benefits of integrating SUD prevention and treatment into medical care settings will not be realized unless brief and sensitive screens for substance problems are identified that can easily fit into busy work schedules of medical settings.

Screening can be improved with brief screening techniques for alcohol and drug use disorders. The U.S. Preventive Services Task Force (USPSTF) has recommended screening and intervention for alcohol problems among adults in primary care [12]. Although illicit or nonmedical drug use frequently coexists with alcohol use and increases medical problems and health care use [13,14], empirical data are insufficient to guide screening efforts for drug problems [15]. One major gap concerns the lack of brief screens for drug problems in a busy primary care setting [16]. Due to time constraint and competing priorities (e.g., assessment and treatment for primary medical conditions), the screen must be brief and sensitive to drug problems to be utilized by clinicians in a busy setting. However, the USPSTF’s reviews have not found a brief screen for drug problems that demonstrates adequate clinical utility in a busy primary care setting [16]. Concerns with prior studies include a focus on lifetime substance use problems, length of instruments, and small samples of study participants.

Presently, a stepped strategy for screening is used that combines an initial single-question screen to assess the presence of substance use with a substance problem screen to detect SUDs among individuals initially screened positive for substance use [17]. Smith and colleagues have studied a single-question screen for alcohol or drug use and found a high level of accuracy in identifying use of those substances in primary care patients [18,19]. Regarding the screen for SUDs, Brown et al. [20] examined a two-item screen for SUDs, including items similar to substance dependence questions (“taking larger amounts” or “need to cut down on use”): “In the last year, have you ever drunk or used drugs more than you meant to?” and “Have you felt you wanted or needed to cut down on your drinking or drug use in the last year?” Having at least one positive response to the two-item screen detected a current SUD with nearly 80% sensitivity and specificity in primary care settings; this two-item screen, however, has not been tested further to establish its utility.

One challenge for developing screens for SUDs concerns the uncertainty regarding whether a similar set of screening questions is applicable to alcohol and various drugs. Studies of alcohol use show that Alcohol Use Disorders Identification Test–Consumption (AUDIT–C) is useful in detecting self-reported cases of heavy drinking or alcohol disorders [21, 22]. AUDIT–C is a three-item quantity-frequency measure (“How often do you have a drink containing alcohol?”, “How many drinks containing alcohol do you have on a typical day when you are drinking?”, “How often do you have six or more drinks on one occasion?”). However, the level of accuracy in detecting a case of alcohol disorder depends on cutoff points of AUDIT–C, and an inclusion of additional items about alcohol problems improves the case identification [23-25]. Other investigators found that a two-item DSM-IV alcohol screen (i.e., recurrent drinking in hazardous situations, drinking more than intended) performs better than AUDIT–C in detecting cases of alcohol disorder in emergency departments [26,27]. The two-item screen for alcohol or drug disorders by Brown et al. [20] does not distinguish between alcohol and drugs, and its generalizability to a wide range of substances is unclear. Recently, Wu et al. [14] examined the use of substance-specific screening questions in distinguishing between cases and noncases of substance-specific disorders. Across five substances examined (alcohol, cannabis, amphetamines, cocaine, and sedatives), item response theory (IRT) analysis showed that items with low severity and high discrimination (i.e., items closely related to the underlying substance problem continuum) had good-to-high values of the area under the receiver operating characteristic curve (ROC-AUC) in distinguishing between cases and non-cases of a SUD. For all five substances, taking large amounts, inability to cut down, and tolerance exhibited high ROC-AUC, while withdrawal had low ROC-AUC [14]. ROC-AUC denotes the level of accuracy in classifying cases and noncases of a SUD by taking into account the trade-off between sensitivity and specificity. An IRT approach was included to provide psychometric properties of substance-specific screening questions for ROC-AUC [28,29]. Both IRT and ROC-AUC results for the five substances demonstrate the potential feasibility of identifying “a core set of substance dependence items” as a simplified screen to facilitate detection of alcohol or drug disorders [14].

To promote integration of substance abuse treatments with primary care settings, a short screen that is sensitive to identifying drug problems and useful for clinical decision making (distinguishing between cases and noncases of a SUD) is needed [14,20]. The recent study by Wu et al. [14] is the only research known to compare substance-specific screening questions for alcohol and various drug disorders. Given the national priority and a need for brief screens for SUDs, it is critical that substance-specific screening questions are evaluated in different samples. Here, we examine substance dependence criteria as screeners for alcohol and drug disorders among 847 substance-using adults who were recruited from 12 outpatient substance abuse settings for two national multisite studies in the National Drug Abuse Treatment Clinical Trials Network (CTN 006 and 007). The samples include geographically different groups of substance users with sufficient prevalences of substance problems that were assessed by the same diagnostic instrument to provide useful data about screening items for SUDs commonly seen in clinical settings.

Specifically, we (1) compare the prevalences of substance-specific SUDs and dependence symptoms across multiple substances to determine the extent of SUDs, (2) conduct factor and IRT analyses of dependence items to inform item-level psychometric properties of substance-specific screening questions for each SUD (item discrimination, severity), and (3) calculate sensitivity, specificity, and ROC-AUC to determine the level of accuracy in classifying cases and non-cases of a SUD for each dependence criterion. We hypothesize that items showing low discrimination and high severity estimates (low relevancy to the underlying trait of a disorder) will have lower ROC-AUC values in identifying cases of a SUD than items with high discrimination and low severity estimates will have.

2. Methods

2.1. Data source

The CTN includes 13 nodes (research centers) allied with substance abuse treatment providers in 39 states across the United States, the District of Columbia, and Puerto Rico [30]. For this study, analyses were performed on data from two multisite CTN studies that evaluated stimulant use outcomes of an abstinence-based contingency management intervention as an addition to usual care in non-opioid agonist treatment [31] and methadone maintenance treatment (MMT) settings [32]. All programs were outpatient, community-based treatment providers associated with the CTN.

Non-opioid agonist treatment participants were recruited from eight treatment programs that did not administer methadone or other opioid agonists [31]. Six of these programs were located in eastern, southeastern, or southwestern urban regions of the United States; one was located in the suburban southeast and one in the rural southwest. Eligible participants included individuals who (1) reported stimulant use within 2 weeks of study entry, (2) used stimulants within 2 weeks of entering a controlled environment (a detoxification unit, hospital, or correctional facility) and exited the controlled environment within 2 weeks of study entry, or (3) submitted a stimulant-positive urine sample at treatment entry.

MMT participants were recruited from six programs located in urban areas in the northeastern, eastern, or southwestern United States [32]. Eligible participants included opioid-dependent individuals who had (1) been enrolled in MMT for a minimum of 30 days but not longer than 3 years, (2) submitted a stimulant-positive (cocaine or amphetamines) urine sample in treatment or within 2 weeks of study enrollment based on clinic records, (3) reported that they were not in recovery from a gambling problem, and (4) demonstrated understanding of study procedures by correctly answering 80% or more of the questions on a quiz covering the requirements, risks, and benefits of participation in the parent study.

2.2. Study variables

At intake, each participant’s age, sex, race/ethnicity, education, and employment status were assessed by the CTN common demographic form. Additionally, all participants were administered the DSM-IV Checklist [33,34] by CTN-affiliated trained interviewers (research staff who completed CTN-specified training for administering the DSM-IV Checklist) to assess substance use commonly seen in an addiction treatment setting, including alcohol, cannabis, cocaine, amphetamines, and opioids. Participants who responded affirmatively to the initial substance use question (“Have you ever used [NAME OF THE SUBSTANCE] in the past 12 months?”) then were assessed for seven substance-specific DSM-IV dependence criteria in the past year, including tolerance, withdrawal, substance often taken in large amounts or for longer periods of time, persistent desire or unsuccessful attempt to cut down, a great deal of time spent in activities necessary to get the substance, important activities given up because of substance use, and continued substance use despite knowledge of having recurrent physical or psychological problems from substance use. Endorsement of at least three of the seven dependence criteria resulted in a dependence diagnosis. Following the DSM-IV logic, individuals who did not meet the criteria for a substance-specific dependence disorder were then assessed for a substance-specific abuse disorder, including role interference, hazardous use, legal problems, and relationship problems. Endorsement of at least one of the four abuse criteria resulted in an abuse diagnosis. A substance-specific disorder included abuse of or dependence on that substance.

Consistent with prior studies showing that dependence symptoms are good candidate screeners for SUDs [14,20], the analysis focused on DSM-IV criterion questions for dependence. Dependence symptoms are considered more reliable manifestations of a disruptive pattern of compulsive substance use and physical/psychological dependence than abuse problems, which are directly related to the extent of substance use; abuse symptoms (role interference, hazardous use, legal problems, relationship problems) measure the social and legal consequences of substance use behaviors, which are likely to be affected and biased by a user’s age, social role, and environmental factors [14,35,36].

2.3. Data analyses

Distributions of study variables and potential differences in sociodemographic and substance use status were examined by descriptive analyses. Prevalence rates of substance dependence in the total sample and among substance users (conditional rates of SUDs) then were calculated.

Among substance users, discrete factor analysis for categorical data was conducted using Mplus, version 6 [37] to describe factor loadings of dependence symptoms. The Tucker–Lewis index (TLI), comparative fit index (CFI), standardized root mean square residual (SRMR), and the root mean square error of approximation (RMSEA) were used to assess the model fit for a one-factor model, as substance dependence is generally considered a single factor [38,39]. A higher value of TLI and CFI (>0.94) and a lower value of SRMR (<0.8) and RMSEA (<0.07) indicate an excellent fit to the data [40,41]. The one-factor model for substance dependence also was assessed by the scree test [42] and the ratio of the first to the second eigenvalues.

Consistent with Wu et al. [14], two-parameter IRT analysis was conducted to provide psychometric properties of substance-specific questions: the unidimensionality (one factor) of the dependence criteria [38,39] and item-level discrimination and severity [43,44]. Seven substance-specific dependence criteria (dichotomous variables) among past-year users of the corresponding substance were analyzed. The two-parameter IRT model assumes that item responses are a function of severity and discrimination of the items included, which is conveyed by a monotonically increasing S-shaped item characteristic curve (ICC) [43]. An ICC is characterized by item severity and discrimination parameters to describe the relationship between an item performance and the trait underlying item performance. An item severity parameter indicates the location of the ICC in relation to the latent continuum, in which a substance user has a 50% probability for endorsing a given item (typically ranging from -3 to +3 standardized units below and above a mean score of 0) [44]. An item discrimination parameter measures the degree of precision with which an item discriminates between substance users with levels of the latent trait above or below its severity level [14].

Lastly, we calculated sensitivity (the proportion of individuals with a substance-specific SUD according to the DSM-IV Checklist who had a positive response to the item), specificity (the proportion of individuals without a substance-specific SUD according to the DSM-IV Checklist who had a negative response to the item), and the ROC-AUC for each item. A high ROC-AUC value indicates a high level of accuracy in classifying cases versus noncases of a substance-specific SUD according to the DSM-IV Checklist (1.0=perfect classification; <0.7=poor) [45].

3. Results

3.1. Selected characteristics of the sample

Of the 847 patients (Table 1), 49.9% were female, 68.9% were non-white (black, 45.9%; Hispanic 16.9%, other, 6.1%), 55.5% were aged 18–39 years, and 53.5% were in a non-MMT program.

Table 1.

Selected characteristics of adults in addiction treatment

Demographic characteristic	n	%
Sex
Male	423	49.9
Female	424	50.1
Race/ethnicity
White, non-Hispanic	263	31.1
Non-white	584	68.9
Age in years
18–39	470	55.5
40–49	294	34.7
50+	83	9.8
Years of education
0–11	291	34.4
12	336	39.7
13+	220	26.0
Marital status
Married/cohabitating	161	19.0
Separated/divorced/widowed	288	34.0
Never married	398	47.0
Past month employment
Employed	273	32.2
Student	4	0.5
Unemployed/not employed	444	52.4
Retired	126	14.9
Substance use in past year
Alcohol	476	56.2
Cannabis	316	37.3
Cocaine	683	80.6
Amphetamines	166	19.6
Opioids	449	53.0
Study number
Non-methadone maintenance treatment (CTN006)	453	53.5
Methadone maintenance treatment (CTN007)	394	46.5

3.2. Past-year substance use and dependence

Past-year use of cocaine (80.6%), alcohol (56.2%), and opioids (53.0%) was more prevalent than cannabis (37.3%), and amphetamines (19.6%). Overall, 64.6% met criteria for cocaine dependence, and 42.3% met criteria for opioid dependence; lower proportions met criteria for dependence on alcohol (23.1%), amphetamines (16.2%), and cannabis (12.9%). Among the subset that used the substance in the past year, conditional rates of substance-specific dependence were higher among amphetamine (82.5%), cocaine (80.1%), and opioid (79.7%) users than among alcohol (41.2%) and cannabis (34.5%) users.

3.3. Exploratory factor analysis of dependence symptom items

Cannabis or alcohol users had lower prevalences of dependence symptoms than users of other drugs (Fig. 1): cannabis (17%–40%), alcohol (25%–44%), amphetamines (49%–78%), cocaine (50%– 79%), and opioids (52%–77%). Inability to cut down was among the most prevalent symptoms for all substances; withdrawal was among the least prevalent symptoms among users of alcohol, cannabis, and cocaine but was prevalent among users of amphetamines and opioids. Estimates of the CFI, the TLI, the RMSEA, the ratio of the first to the second eigenvalues, and factor loadings indicated one dependence factor for each substance (Table 2), which indicated evidence of unidimensionality to allow IRT modeling.

Fig. 1.

Prevalence (12 months) of substance dependence symptoms among users of specific substance

Note: Lines extending from bars indicate 95% confidence intervals of the estimates. If two error bars overlap, the difference between two groups is not statistically significant (P > 0.05).

Table 2.

Exploratory factor analysis (EFA) of substance dependence symptoms among users of specific substance

Factor loading of dependence symptoms among users	Alcohol	Cannabis	Cocaine	Amphetamines	Opioids
Sample size, n	476	316	683	166	449
Tolerance	0.88	0.80	0.82	0.86	0.85
Withdrawal	0.89	0.78	0.75	0.92	0.90
Taking larger amounts	0.92	0.91	0.87	0.86	0.89
Inability to cut down	0.88	0.83	0.90	0.84	0.89
Time spent using	0.95	0.92	0.92	0.95	0.84
Giving up activities	0.97	0.90	0.86	0.80	0.91
Medical or psychological problems	0.92	0.84	0.82	0.58	0.82
CFI	0.99	0.99	0.99	0.99	0.98
TLI	0.99	0.98	0.98	0.99	0.98
RMSEA	0.07	0.08	0.08	0.02	0.10
SRMR	0.03	0.06	0.05	0.005	0.07
First/second eigenvalues	5.97/0.30	5.29/0.63	5.25/0.59	5.09/0.68	5.40/0.66

The EFA used a weighted least-squares approach.

CFI, comparative fit index; TLI, Tucker–Lewis index; RMSEA, root mean square error of approximation; SRMR, standardized root mean square residual.

3.4. IRT analysis and ROC-AUC of dependence symptom items

ICCs considering item-level discrimination and severity estimates simultaneously (Fig. 2) showed that withdrawal was among the most severe symptoms of the IRT-defined latent trait problems (shifted to right side) and that inability to cut down and tolerance were among the least severe indicators for alcohol, cannabis, and cocaine dependences. By contrast, giving up activities and medical/psychological problems were severe indicators for amphetamine and opioid dependences. Overall, time spent using the substance showed a high level of discrimination estimates (1.62–3.19) for all substances (steep lines).

Fig. 2.

Item response theory (IRT) analysis of substance dependence symptoms among users of specific substance: (a) alcohol, (b) cannabis, (c) cocaine, (d) amphetamines, and (e) opiates

3.5. Sensitivity and specificity of dependence symptom items

Sensitivity estimates (Fig. 3a) were similar to patterns of dependence symptoms with low prevalent symptoms showing low sensitivity: cannabis (sensitivity ranging from 0.35 for withdrawal to 0.71 for inability to cut down), alcohol (from 0.45 for withdrawal to 0.73 for inability to cut down), amphetamines (from 0.55 for medical/psychological problems to 0.87 for inability to cut down), cocaine (from 0.59 for withdrawal to 0.92 for inability to cut down), and opioids (from 0.66 for giving up activities to 0.92 for withdrawal). On the other hand, all items exhibited good-to-excellent levels of specificity (0.83–1.00), indicating that individuals who did not endorse the item were unlikely to have a SUD (Fig. 3b).

Fig. 3.

(a) Sensitivity, (b) specificity, and (c) area under the receiver operating characteristic curve (ROC-AUC) of substance dependence symptoms among users of specific substance

ROC-AUC (Fig. 3c) considers sensitivity and specificity together, and a value of 1.0 (100%) indicates that the item accurately classifies cases of a SUD defined by the DSM-IV Checklist. Consistent with IRT results, items indicative of severe ends on ICCs (Fig. 2) showed low ROC-AUC values: withdrawal for cannabis (0.67), alcohol (0.72), and cocaine (0.79). Items indicative of low severity and high discrimination on ICCs exhibited high ROC-AUC values: withdrawal for opioids (0.92) and amphetamines (0.89); taking larger amounts (0.80–0.87), inability to cut down (0.78–0.94), and tolerance (0.77–0.89) for all substances.

4. Discussion

4.1. Main findings

This study evaluated item-level psychometric (IRT) and classification (ROC-AUC) information of substance-specific dependence questions to identify a core set of screeners that can be studied further for developing a simplified screen for detecting SUDs. First, withdrawal was least prevalent among users of alcohol, cannabis, and cocaine, and showed greater substance-specific variations than other symptoms, especially inability to cut down, which was among the most prevalent symptoms across five substances. Second, comparatively low prevalent symptoms exhibited poor sensitivity (<0.60) in identifying a positive case of SUD according to the DSM-IV Checklist: withdrawal (alcohol, cannabis, and cocaine), time spent (alcohol), giving up activities (alcohol, cannabis), and medical/psychological problems (cannabis, amphetamines). Third, IRT-defined severe and low discriminative items had low ROC-AUC in classifying cases of a SUD (withdrawal for alcohol and cannabis). IRT-defined less severe and high discriminative items showed good-to-high values of ROC-AUC in identifying cases of a SUD (inability to cut down, taking larger amounts than intended for all substances; withdrawal for amphetamines and opioids).

4.2. What this study adds

IRT methods are the primary tool used by the Patient-Reported Outcomes Measurement Information System (PROMIS) to identify and develop reliable, precise, and responsive assessment tools that measure patient–reported health status [28]. However, they have been underutilized in research on screeners for SUDs to support the underlying construct of patient-reported measures. Here, both factor and IRT analyses show that various substance dependence criteria measure a one factor, supporting a dependence syndrome for all substances examined [38,39]. Of note, IRT’s item-level parameters have unique strengths in specifying psychometrically more reliable items to inform the development of a sensitive and simplified screen. These findings reveal striking similarities in item-level results from both IRT and ROC-AUC analyses. IRT-defined low reliable items, such as withdrawal, demonstrate a higher level of errors in distinguishing between cases and noncases of a SUD than IRT-defined reliable items, such as inability to cut down or taking larger amounts than intended. This pattern supports findings from different samples of adults. Wu et al. [14] examined dependence questions for five SUDs (alcohol, cannabis, cocaine, amphetamines, and sedatives) among 920 opioid-dependent adults. Likewise, IRT analysis revealed that items closely related to the underlying latent trait of a SUD showed good-to-high ROC-AUC values in identifying cases of SUDs (taking large amounts, inability to cut down), and that severe (infrequent) and less discriminative items exhibited low sensitivity (high rate of false negative) and a fair ROC-AUC in accurately classifying cases of a SUD according to the DSM-IV Checklist (withdrawal for all substances; time using for alcohol and sedatives; giving up activities for sedatives). Brown et al. [20] examined five screening questions for any alcohol or drug disorder in a primary care setting (blackouts, use alcohol or drugs more than intended, use for feelings, need to cut down, or regret). Although Brown et al. [20] did not distinguish symptoms between alcohol and drug use, the two dependence items (use more than intended, need to cut down) also demonstrated higher sensitivity in detecting a SUD than the other items.

Of note, the consistency in patient-reported dependence symptoms indicative of “compulsive substance use” and “a loss of control in limiting intake” across multiple substances reveals the presence of “core symptoms” of addictions that are of clinical implications for assessment [14,36]. As clinicians often need to assess multiple clinically relevant conditions within a constrained period of time, sensitivity and efficiency (using a minimal number of screening questions, combining items for alcohol and drugs) are crucial for incorporating a screen into routine practice [46,47]. Selective substance dependence questions can distinguish between cases and noncases of a SUD, meet the principles of standardization and brevity for a clinical screen, and are likely to be adopted by clinicians (due to familiarity of the criteria). Together, these results support further examining taking large amounts and inability to cut down as part of a simplified screener to facilitate detection of SUDs in medical settings [14].

Findings also reveal an important connection between the prevalence of a candidate item and sensitivity. The less common or IRT-defined severe items (e.g., withdrawal) had low sensitivity in identifying a SUD, suggesting that use of infrequent items as brief screeners disproportionately miss cases of SUD (false negative). For example, sensitivity for withdrawal was 0.35 for cannabis disorder and 0.45 for alcohol disorder, indicating that 65% and 55% of individuals who met criteria for a cannabis or alcohol disorder, respectively, in this sample did not endorse withdrawal. While this study and the study by Wu et al. [14] examined different substance-using samples, the overall patterns of IRT and ROC-AUC results are remarkably consistent across the two studies. Both revealed poor sensitivity (high rate of false negative) for withdrawal from alcohol, cannabis, and cocaine. Therefore, the prevalence of a clinical condition is an important factor to consider when selecting candidate screening items, and use of infrequent conditions or symptoms as screens can miss a substantial proportion of cases [14,23]. It also indicates that withdrawal alone (for alcohol, cannabis, or cocaine) may not be a good target for outcome-related assessments, as it is not a sensitive indicator of a SUD [14].

These findings have timely implications for research on screening instruments. The USPSTF has not recommended screening and intervention for alcohol use problems in youth and drug use problems because of inadequate research data [11,12,15]. Except for alcohol-specific screening tools for adults (e.g., CAGE, AUDIT, AUDIT–C), reliable and effective screening tools for alcohol-specific problems in youth and drug-specific problems that are useful for routine use in busy medical settings are lacking [11,15]. The simplified three-item AUDIT–C focuses exclusively on the quantity and frequency of (licit) alcohol use behaviors [21,25], which is not applicable to screening for illicit or nonmedical drug use disorders. The various levels of scoring algorithms for AUDIT–C also can be difficult to apply to patients in a hectic clinical setting [26], highlighting a need for developing efficient tools that can screen both alcohol and drug disorders to facilitate clinical use and intervention. Findings from this and other studies reveal the feasibility of identifying psychometrically reliable substance dependence symptoms to develop a two-item screen for alcohol and drug disorders [11,20, 27].

Lastly, as demonstrated by the PROMIS, modern psychometrical methods like IRT are needed for developing screeners to support the psychometric quality of patient-reported items [28]. This study supports the value of combining IRT and ROC-AUC procedures to select psychometrically appropriate items to develop an efficient, simplified, and reasonably sensitive tool to screen for SUDs in medical settings [14]. The Affordable Care Act and the Health Information Technology for Economic and Clinical Health Act provide the opportunity to develop standardized and clinically meaningful screening tools and to incorporate them into the electronic health records (EHRs) system to facilitate preventive services and treatment for SUDs in primary care [46,47]. A clinically useful screening tool also has the potential to generate standardized patient-reported health indicators in the longitudinal EHRs system for monitoring clinical courses and improving decision making.

4.3. Limitations and strengths

The results are based on treatment-seeking substance-using adults who are not necessarily representative of all adults with a SUD. Although cannabis is the most prevalent illicit drug used in the general population [48], opioid use accounts for more substance abuse treatment admissions in the United States than that of cannabis use [49]. Additionally, the majority of primary cannabis abusers seeking substance abuse treatment are adolescents and young adults, whereas primary abusers of opioids and cocaine in treatment are predominantly adults aged 30 years or older [49]. The high prevalent of cocaine disorder in this sample is related to an inclusion criterion of the data sources, which target stimulant use as studies have documented a prevalent rate of cocaine use among patients in substance abuse treatment programs, especially opioid abusers [50,51]. Treatment-seeking individuals differ from those who do not seek care for SUDs in primary care or other medical settings, and considerably more research in the primary care setting on the items examined here is needed before they can be adopted for widespread use. Nonetheless, these findings are consistent across multiple substances, and the overall patterns were similar to a recent study among opioid-dependent adults [14]. Although assessments of SUDs rely on self-reports and may be influenced by reporting bias, IRT analysis was included to provide independent, psychometric evaluation for self-reported measures [28]. Until reliable and sensitive biomarkers become available for SUD screening and diagnosis [52,53], self-reports are arguably the best available method for assessing SUDs [54,55].

Strengths of the data sources include geographic diversity (from 12 outpatient settings across the nation), the rigorous conduct of the original trials (standardized assessments, comprehensive research staff training, protocol monitoring, and regulatory control), and assessment of the withdrawal criterion for cannabis, which is not considered presence for dependence in DSM-IV but has important implications for the emerging DSM-5. The sample was much more heterogeneous than that found at a single site. All participants were assessed for substance-specific SUDs using the same instrument, allowing for comparisons of all substance-specific dependence criteria across various substances; this provides vital information regarding whether a similar set of questions can apply to alcohol and various drugs [14,20].

4.4. Conclusions

Findings add to the evidence for initial single-item screens for alcohol or drug use that have shown good sensitivity and specificity in medical settings [18,19], and they expand prior research on the two-item screen for alcohol and drug disorders [20]. The combined results from IRT and ROC-AUC analyses indicate the value of further testing taking large amounts and inability to cut down as part of a simplified screen to facilitate detection of problematic substance users with a high probability for having a SUD [14].