Comparison of the Substance Use Brief Screen (SUBS) to the AUDIT-C and ASSIST for detecting unhealthy alcohol and drug use in a population of hospitalized smokers

Abstract

Hospitalized patients have high rates of unhealthy substance use, which has important impacts on health both during and after hospitalization, but is infrequently identified in the absence of screening. The Substance Use Brief Screen (SUBS) was developed as a brief, self-administered instrument to identify use of tobacco, alcohol, illicit drugs, and non-medical use of prescription drugs, and was previously validated in primary care patients. This study assessed the diagnostic accuracy of the SUBS in comparison to longer screening instruments to identify unhealthy and high-risk alcohol and drug use in hospitalized current smokers. Participants were 439 patients, aged 18 and older, who were admitted to either two urban safety-net hospitals in New York City and enrolled in a smoking cessation trial. We measured the performance of the SUBS for identifying illicit drug and non-medical use of prescription drugs in comparison to a modified Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) and its performance for identifying excessive alcohol use in comparison to the Alcohol Use Disorders Identification Test–Consumption (AUDIT-C). At the standard cutoff (response other than ‘never’ indicates a positive screen), the SUBS had a sensitivity of 98% (95% CI 95–100%) and specificity of 61% (95% CI 55–67%) for unhealthy alcohol use, a sensitivity of 85% (95% CI 80–90%) and specificity of 75% (95% CI 78–87%) for illicit drug use, and a sensitivity of 73% (95% CI 61–83%) and specificity of 83% (95% CI 78–87%) for prescription drug non-medical use. For identifying high-risk use, a higher cutoff (response of ‘3 or more days’ of use indicates a positive screen), the SUBS retained high sensitivity (77–90%), and specificity was 62–88%. The SUBS can be considered as an alternative to longer screening instruments, which may fit more easily into busy inpatient settings. Further study is needed to evaluate its validity using gold standard measures in hospitalized populations.

1. Introduction

Alcohol and drug use is highly prevalent among hospitalized patients, (Holt et al., 2012; Katz, Goldberg, Smith, & Trick, 2008; Smothers & Yahr, 2005), and has important health impacts, but in the absence of systematic screening it frequently goes unidentified. Adults with substance use disorders have higher hospitalization rates, and sustain higher acute health care costs, in comparison to the general population (Gryczynski et al., 2016; Santora & Hutton, 2008; Stein, 1999). The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) recognizes alcohol screening as an inpatient quality measure (The Joint Commission, 2016) and it is important to identify drug use in the inpatient hospital setting, since both alcohol and drug use influence the diagnosis and management of medical conditions (American Psychiatric Association, 2006; Bryson et al., 2008).

Unhealthy substance use is defined as use of alcohol in excess of guideline-recommended levels (National Institute on Alcohol Abuse and Alcoholism, 2016) or any use of illicit drugs or non-medical use of prescription drugs (use for the feeling or experience, to get high, or taking more often or at higher doses than prescribed), and encompasses the full spectrum of risky use, hazardous use, and substance use disorder (SUD) (American Society of Addiction Medicine, 2013). Based on the DSM-5, the diagnosis of substance use disorder is based on a pattern of use that causes clinically significant functional impairment within a 12-month period, and ranges from mild to severe (American Psychiatric Association, 2013). Substance use has important implications for patient safety, particularly with respect to withdrawal while patients are hospitalized, which can be severe (e.g. opioids) or even life threatening (e.g. alcohol and benzodiazepines) (Kosten & O’Connor, 2003), and potential drug-medication interactions. Substance use may also impact discharge planning. Patients with substance use disorders may be less adherent to treatment and have weaker connections to primary care (Hinkin et al., 2007; Samsone & Sansone, 2008; Tucker et al., 2004). Hospitalized patients have high rates of severe SUDs in comparison to patients seen in outpatient settings (Holt et al., 2012), and the inpatient stay can be an opportunity to engage these patients in SUD treatment.

Screening for alcohol and drug use is particularly important for smokers, who have high rates of concurrent use of other substances, especially alcohol (Breslau, 1995; Katz et al., 2008). Drug and alcohol use may have profound effects on the management of chronic conditions, such as cardiovascular diseases that are common among smokers, and on their health-related quality of life (Breslau, 1995; Danaei et al., 2009; Strine et al., 2005). In addition, the efficacy of tobacco cessation interventions could be compromised if medical providers are not aware of concurrent drug use (Sullivan & Covey, 2002).

Rates of substance use screening are low among hospitalized patients (Rumpf, Bohlmann, Hill, Hapke, & John, 2001). While the JCAHO recommends alcohol screening for hospitalized patients, there is a limited evidence base of studies to support it (Makdissi & Stewart, 2013). Further, there are a lack of studies that seek to validate or compare substance use screening tools for hospitalized patients as most studies were performed in the primary care setting (Lanier & Ko, 2008; Makdissi & Stewart, 2013; Zgierska, Amaza, Brown, Mundt, & Fleming, 2014). Screening for substance use in the inpatient setting faces many of the same barriers encountered in primary care (Aira, Kauhanen, Larivaara, & Rautio, 2003; Anderson, 2009; Friedmann, McCullough, & Saitz, 2001; Johnson, Jackson, Guillaume, Meier, & Goyder, 2011; McCormick et al., 2006; Sterling, Kline-Simon, Wibbelsman, Wong, & Weisner, 2012; Yoast, Wilford, & Hayashi, 2008). The most prominent barriers include the lack of time and the challenges of incorporating screening into the regular clinical workflow. Even existing brief screening questionnaires and interviewer-administered single-item screening questions for alcohol and drugs, require substantial training and time to administer, and lose reliability as even well-trained staff modify the screening language (Saitz, Cheng, Allensworth-Davies, Winter, & Smith, 2014; Smith, Schmidt, Allensworth-Davies, & Saitz, 2009; Smith, Schmidt, Allensworth-Davies, & Saitz, 2010). Patients also may be uncomfortable reporting stigmatized behavior such as substance use in a face-to-face interview (Tourangeau & Smith, 1996).

Brief substance use screening tools have been developed for use in primary care settings, and could also reduce barriers to screening in the inpatient setting. One such tool is the Substance Use Brief Screen (SUBS). The SUBS is a brief screener for tobacco, alcohol, and drug use (illicit and prescription) that is self-administered. The advantage for SUBS is it generally takes 1–2 min to finish compared to commonly used screening instruments that can range from 2 to 15 min to complete (Babor, Higgins-Biddle, Saunders, & Monteiro, 2001; Mdege & Lang, 2011). In a two-site study, SUBS was feasible for self-administration, and had good sensitivity and specificity for detecting past-year unhealthy use of tobacco (sensitivity of 97.8% and specificity of 95.7%), alcohol (sensitivity of 85.2% and specificity of 77.0%), and drugs (sensitivity of 82.5% and specificity of 91.1%) compared with reference standard measures including an oral fluid drug test in the primary care setting for a diverse population of adult safety-net patients (McNeely et al., 2015).

The SUBS, like many other substance use screening instruments, have not been validated in hospitalized patients. Therefore, the goal of this study was to assess the diagnostic accuracy of the SUBS in comparison to previously validated screening tools for alcohol and drugs, among a sample of hospitalized smokers. This study used the opportunity of an ongoing clinical trial of inpatient smokers for evaluating the use of SUBS in the inpatient setting. We compared the diagnostic accuracy of the SUBS for identifying unhealthy alcohol use with the validated Alcohol Use Disorders Identification Test – Consumption (AUDIT-C) (Bush, Kivlahan, McDonell, Fihn, & Bradley, 1998) instrument, and unhealthy drug use with a modified version of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) Version 3.0 (Humeniuk, 2008).

2. Materials and methods

2.1. Participants and recruitment

Data were collected as part of a large clinical trial of hospitalized smokers (Effectiveness of Smoking-cessation Interventions for Urban Hospital Patients, a study in the Consortium of Hospitals Advocating for Advancing Research on Tobacco Treatment (CHART) network), funded by the National Heart, Lung, and Blood Institute (NHLBI) (trial #NCT01363245). The design of the CHART study including recruitment, participants, clinical sites, and intervention has been described previously (Grossman et al., 2012; Sherman et al., 2016). Briefly, the study was a randomized trial of adult current tobacco smokers who were admitted (to medicine, psychiatry, surgery, neurology, rehabilitation, pediatrics, and gynecology services) at two urban hospitals in New York City (Bellevue Hospital Center and the VA New York Harbor Healthcare System). Both hospitals function as “safety net” hospitals delivering care to mainly low-income and underserved patients (Werner, Goldman, & Dudley, 2008). Participants were enrolled while they were hospitalized, and had baseline data collected prior to discharge, including socio-demographics, smoking history, health status, depression, and substance abuse screening data. Participants were then randomized upon discharge to either referral to the New York State Quitline or to proactive multi-session telephone counseling for smoking cessation. This study examines the baseline data for a subset of participants in the CHART study who completed substance use screening instruments at the baseline study visit. The institutional review boards at New York University School of Medicine and the VA New York Harbor Healthcare System reviewed and approved all study procedures.

2.2. Eligibility criteria

Individuals eligible for the CHART study were hospitalized patients 18 years of age or older who reported smoking during the prior 30 days, had an active U.S. phone number and spoke English, Spanish or Mandarin. Exclusion criteria were: 1) pregnant or breastfeeding, 2) lacking cognitive or physical ability to participate or 3) discharge to an institution that would limit the research team’s ability to deliver telephone counseling or where patients lacked control over their ability to smoke (e.g., jail, nursing home). Since the SUBS was available only in English, this sub-study was only administered to study participants who could read the instrument in English.

2.3. Study instrument: SUBS

The SUBS (McNeely et al., 2015) (Fig. 1) was developed as a brief, self-administered instrument to identify unhealthy use of tobacco, alcohol, illicit drugs, and non-medical use of prescription drugs (4 total questions), and has been validated in the primary care setting (McNeely et al., 2015; National Institute on Drug Abuse, NIDA, 2016). It has three response categories (Never, One or two days, Three or more days), but is designed to be dichotomized into negative and positive screening results, with any response other than ‘Never’ considered a positive screen. The timeframe of past 12 months is consistent with most guidelines for annual screening of alcohol and drug use, and with the Diagnostic and Statistical Manual of Mental Disorders, 4th and 5th edition criteria (American Psychiatric Association, 2000; American Psychiatric Association, 2013) for substance use disorders. As discussed previously it usually takes 1–2 min to administer.

Fig. 1.

Substance use brief screen.

2.4. Reference standard measures

Commonly used and previously validated screening instruments that measure unhealthy and high-risk alcohol and drug use were used as reference standards for comparison of the SUBS (Bush et al., 1998; Humeniuk, 2008; Reinert & Allen, 2002; Rubinsky, Kivlahan, Volk, Maynard, & Bradley, 2010; VA Quality Enhancement Research Initiative, 2016). Drug use was assessed with a modified version of the World Health Organization Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) Version 3.0 (Humeniuk, 2008). The ASSIST instrument identifies lifetime use, current (past 3 months) use, and level of risk (low, moderate, and high-risk use) of tobacco, alcohol, and 7 classes of commonly used drugs. The modified ASSIST used in our study did not include tobacco or alcohol, but did include prescription opioids and prescription stimulants; a modification that we have used in prior studies (McNeely, Strauss, Rotrosen, Ramautar, & Gourevitch, 2016; McNeely et al., 2014). Based on standard recommended cutoffs for the ASSIST, a score of 4–26 indicates moderate-risk use, and a score of 27+ indicates high-risk use of any drug class. A score of ≥4 was thus used to classify ‘unhealthy use’, while a score of ≥27 classified high-risk use (Humeniuk, 2008).

Instead of using the ASSIST to measure alcohol use, the study administered the Alcohol Use Disorders Identification Test – Consumption (AUDIT-C) instrument (Bush et al., 1998), which measures alcohol use in the past year. The cutoffs for AUDIT-C for unhealthy alcohol use were ≥4 for males and ≥3 for females, and ≥8 for high-risk alcohol use, which are consistent with the standard recommended cutoffs (Rubinsky et al., 2010; VA Quality Enhancement Research Initiative, 2016). Since the entire sample was composed of current smokers, we did not evaluate the SUBS tobacco item in comparison to a reference standard measure. However, the prevalence of tobacco dependence was determined for the sample using the Heaviness of Smoking Index score of ≥4 (Heatherton, Kozlowski, Frecker, Rickert, & Robinson, 1989).

2.5. Study procedures

For the CHART study, using the electronic medical record for each hospital a daily list was created of inpatients who were documented as current smokers on hospital admission. Research assistants reviewed the list twice daily and approached the patients in their hospital room, in the emergency department, or in the intensive care unit. Interested participants consented to participate in the CHART study, which included the administration of the SUBS. For the study, research assistants administered a series of questions and instruments sequentially, which included the AUDIT-C and ASSIST. The SUBS was given to participants to complete at the end of the baseline data collection for CHART. Participants were asked to self-administer the SUBS on paper. Study participants did not receive compensation for completing the SUBS or other baseline assessments, but could receive compensation for participation in telephone follow ups as part of the CHART study.

2.6. Statistical analysis

We examined descriptive statistics of the participant sample, and the prevalence of substance use reported on the reference standard measures. We examined two approaches to scoring the SUBS. First, we followed the standard scoring approach in which responses were dichotomized for each of the 4 SUBS items, with a response of “never” representing a negative screen, and any other response representing a positive screen for unhealthy use. Second, we also examined SUBS responses dichotomized at a higher cutoff, with “never” and “one or two days” categorized as a negative screen and “three or more days” as a positive screen. Illicit drugs and prescription drugs were analyzed separately and then in combination as an “any drug” category.

We calculated the sensitivity and specificity of each SUBS item in comparison to the reference standard measure. In addition, positive and negative diagnostic likelihood ratios were calculated (Simel, Samsa, & Matchar, 1991). We computed receiver operator characteristic curves and examined the area under each curve (AUC). An AUC of ≥0.90 represents excellent discrimination, an AUC of 0.8 or higher indicates good discrimination, and an AUC lower than 0.7 indicates poor discrimination (Hanley & McNeil, 1982). Exact 95% confidence intervals were calculated for all accuracy estimates. Calculations were made individually for the items in the SUBS and then for the combined “any drug” category. We also calculated the above by stratifying by gender. Analyses were conducted using SAS 9.3 (Cary, NC).

3. Results

3.1. Characteristics of the study population

Following the introduction of the SUBS into the CHART study, 6897 hospitalized patients were identified as current smokers (Fig. 2), of which 721 were discharged before they could be evaluated for study eligibility. Of the remaining 6176, 4133 were ineligible, 1563 refused, and 3 withdrew from the study. Of the 477 that completed the baseline measures, 27 did not read English and were thus ineligible to complete the SUBS, and 11 had lost or incomplete SUBS data. Data from the remaining 439 participants were analyzed. Eighty-eight participants (20%) needed assistance in completing the SUBS, due to confusion over the wording of the SUBS items or needing assistance reading it because of visual impairment (usually because the participant was without their reading glasses). However, the specific breakdown of reasons for needing assistance was not collected.

Fig. 2.

CONSORT diagram for enrollment and survey completion.

Demographic characteristics are shown in Table 1. Participants were racially and ethnically diverse, and predominantly male (81%). Eighty-one percent completed high school and 67.5% had high health literacy based on a validated health literacy scale (Sarkar, Schillinger, López, & Sudore, 2011). Over 30% reported their health status as ‘excellent’ or ‘very good.’

Table 1.

Demographic characteristics of study participants.

Characteristics	N (%), n = 439
Demographics
Age (years)
Mean, SD	47.1 (13.4)
Median	49
Range	19, 80
Interquartile range	36, 57
Gender
Male	355 (80.9%)
Female	83 (19.0%)
Other	1 (0.2%)
Ethnicity
Hispanic	134 (30.5%)
Race
Black	179 (40.8%)
White	129 (29.4%)
Other	65 (14.8%)
Multi-race	32 (7.3%)
Asian	11 (2.5%)
Indian	8 (1.8%)
Don’t know/refused	15 (3.4%)
Country of birth
U.S.	368 (83.8%)
Education (highest level completed)
8th grade or less	14 (3.2%)
Some high school	69 (15.7%)
High school/GED	128 (29.2%)
Associate degree	25 (5.7%)
Some college	135 (30.8%)
4-year college graduate or higher	68 (15.5%)
Health insurance
Public	198 (45.1%)
VA	132 (30.1%)
None	59 (13.4%)
Private	49 (11.2%)
Don’t know/refused	1 (0.2%)
Health literacya
High HL	295 (67.2%)
Moderate HL	116 (26.4%)
Low HL	26 (5.9%)
Missing/not completed	2 (0.5%)
Perceived health statusb
Excellent	47 (10.7%)
Very good	85 (19.4%)
Good	132 (30.1%)
Fair	125 (28.5%)
Poor	50 (11.4%)

^aHow confident are you filling out medical forms by yourself? Would you say • Extremely (HIGH) • Quite a bit/somewhat (MODERATE) • A little/not at all (LOW) (Sarkar et al., 2011).

^bIn general, would you say your health is excellent, very good, good, fair, or poor?

3.2. Self-reported prevalence of substance use and risk level

The prevalence of alcohol and drug use by risk level is shown in Table 2. The prevalence of moderate- and high-risk alcohol use was gathered from the AUDIT-C. In the study population, 29% had moderate risk alcohol use and 19% had high risk alcohol use. Prevalence of moderate- or high-risk use was slightly higher among women (50.6%) than men (47.0%), though this difference was not statistically significant. Prevalence of drug use was assessed using the ASSIST. Cannabis was the substance with the highest prevalence of lifetime use (81%) as well as current use (38%). Heroin and cocaine were the substances with the highest prevalence of high-risk use. As expected, the level of tobacco dependence, based on the Heaviness of Smoking Index score of ≥4 (Heatherton et al., 1989), was high at 98.9%.

Table 2.

Substance use prevalence among study participants (n = 439).

Substance	Lifetime	Currenta	Low riskb	Moderate riskc	High riskd
Alcohol use	NA	331 (75.4%)b	229 (52.2%)	127 (28.9%)	82 (18.7%)
Any drug use (illicit or prescription)	386 (87.9%)	253 (57.6%)	179 (40.8%)	186 (42.4%)	74 (16.9%)
Illicit drug use	383 (87.2%)	228 (51.9%)	190 (43.3%)	183 (41.7%)	66 (15.0%)
Cannabis use	357 (81.3%)	165 (37.6%)	276 (62.9%)	146 (33.3%)	17 (3.9%)
Cocaine use	287 (65.4%)	104 (23.7%)	312 (71.1%)	98 (22.3%)	29 (6.6%)
Hallucinogen use	143 (32.6%)	13 (3.0%)	420 (95.7%)	17 (3.9%)	2 (0.5%)
Inhalant use	56 (12.8%)	12 (2.7%)	430 (98.0%)	9 (2.1%)	0 (0%)
Methamphetamine use	59 (13.4%)	11 (2.5%)	428 (97.5%)	9 (2.1%)	2 (0.5%)
Street opioid use	145 (33.0%)	57 (13.0%)	361 (82.2%)	49 (11.2%)	29 (6.6%)
Prescription drug use	159 (36.2%)	77 (17.5%)	360 (82.0%)	61 (13.9%)	18 (4.1%)
Prescription stimulant use	53 (12.1%)	16 (3.6%)	424 (96.6%)	13 (3.0%)	2 (0.5%)
Sedative use	111 (25.3%)	55 (12.5%)	387 (88.2%)	44 (10.0%)	8 (1.8%)
Prescription opioid use	98 (22.3%)	46 (10.5%)	388 (88.4%)	39 (8.9%)	12 (2.7%)

^aCurrent alcohol use is defined as the past 12 months and current drug use is defined as the past 3 months.

^bASSIST V3.0 score 0–3 and AUDIT-C score < 3 for females or <4 for males.

^cASSIST V3.0 score 4–26 and AUDIT-C score 3–7 for females or 4–7 for males.

^dASSIST V3.0 score ≥ 27 and AUDIT-C score ≥ 8.

3.3. Comparison of SUBS to reference standard measures

Analyses of the concurrent validity of the SUBS in comparison to the reference standard instruments (AUDIT-C for alcohol, ASSIST for drugs) are presented in Table 3. Two scoring cutoffs were used in our analysis of the SUBS. First we applied the standard SUBS cutoff of “never” as a negative screen and any other response as a positive screen (McNeely et al., 2015). Second, we examined the results designating a higher SUBS cutoff of “three or more days” as a positive screen, and other responses as a negative screen. Both cutoffs were tested for identifying the conditions of any unhealthy use and high-risk use, for each substance category.

Table 3.

Sensitivity, specificity, likelihood ratios, and area under the curve of SUBS for detecting unhealthy use and substance use disorders (n = 439).

Substance class	Positive on SUBS N (%)	Positive on reference standards N (%)a	Sensitivity % (95% CI)	Specificity % (95% CI)	Positive likelihood ratio (95% CI)	Negative likelihood ratio (95% CI)	AUCb (95% CI)
Any response > ‘never’ for that substance class = positive SUBS
Any unhealthy use
Alcohol	289 (67.7%)	209 (47.6%)	97.6% (95.5%, 99.7%)	61.0% (54.5%, 67.5%)	2.5 (2.1, 2.9)	0.0 (0.0, 0.06)	0.8 (0.8, 0.8)
Illicit drugs	240 (56.1%)	224 (51.0%)	85.1% (79.8%, 89.5%)	75.2% (68.8%, 81.0%)	3.4 (2.6, 4.3)	0.2 (0.1, 0.3)	0.8 (0.8, 0.8)
Prescription drugs	115 (27.5%)	79 (18.0%)	72.7% (61.4%, 82.3%)	82.8% (78.3%, 86.6%)	4.2 (3.1, 5.4)	0.3 (0.2, 0.5)	0.8 (0.7, 0.8)
Any drugs	268 (62.3%)	245 (55.8%)	88.5% (83.8%, 92.2%)	72.0% (65.0%, 78.4%)	3.2 (2.4, 3.9)	0.2 (0.1, 0.2)	0.8 (0.8, 0.8)
High-risk use
Alcohol	289 (67.7%)	83 (18.9%)	100% (NA)	40.1% (34.9%, 45.5%)	1.7 (1.5, 1.8)	0 (NA)	0.7 (0.7, 0.7)
Illicit drugs	240 (56.1%)	45 (10.3%)	93.2% (81.3%, 98.6%)	48.2% (43.1%, 53.3%)	1.8 (1.6, 2.0)	0.1 (0.0, 0.3)	0.7 (0.7, 0.8)
Prescription drugs	115 (27.5%)	18 (4.1%)	94.4% (72.7%, 99.9%)	75.6% (71.1%, 79.7%)	3.9 (3.1, 4.7)	0.1 (0.0, 0.2)	0.9 (0.8, 0.9)
Any drugs	268 (62.3%)	58 (13.2%)	98.3% (90.8%, 100.0%)	43.3% (38.2%, 48.5%)	1.7 (1.6, 1.9)	0.0 (0.0, 0.1)	0.7 (0.7, 0.7)
Response “three or more days” = positive SUBS
Any unhealthy use
Alcohol	167 (39.1%)	209 (47.6%)	68.4% (62.1%, 74.7%)	89.0% (84.8%, 93.2%)	6.2 (3.8, 8.6)	0.4 (0.3, 0.4)	0.8 (0.8, 0.8)
Illicit drugs	166 (38.8%)	224 (51.0%)	62.2% (55.8%, 68.5%)	86.4% (81.7%, 91.1%)	4.6 (2.9, 6.2)	0.4 (0.4, 0.5)	0.7 (0.7, 0.8)
Prescription drugs	64 (15.3%)	79 (18.0%)	42.9% (31.8%, 53.9%)	90.9% (87.9%, 94.0%)	4.7 (2.7, 6.7)	0.6 (0.5, 0.8)	0.7 (0.6, 0.7)
Any drugs	189 (44.0%)	245 (55.8%)	64.8% (58.8%, 70.8%)	83.3% (78.0%, 88.7%)	3.9 (2.6, 5.2)	0.4 (0.4, 0.5)	0.7 (0.7, 0.8)
High-risk use
Alcohol	167 (39.1%)	83 (18.9%)	90.4% (81.9%, 96.7%)	73.3% (68.6%, 77.9%)	3.4 (2.7, 4.0)	0.1 (0.0, 0.2)	0.8 (0.8, 0.9)
Illicit drugs	166 (38.8%)	45 (10.3%)	77.3% (64.9%, 89.7%)	65.6% (60.0%, 70.4%)	2.3 (1.8, 2.7)	0.4 (0.2, 0.5)	0.7 (0.7, 0.8)
Prescription drugs	64 (15.3%)	18 (4.1%)	88.9% (74.4%, 100.0%)	88.0% (54.9%, 91.2%)	7.4 (5.1, 9.8)	0.1 (0.0, 0.3)	0.9 (0.8, 1.0)
Any drugs	189 (44.0%)	58 (13.2%)	86.2% (77.3, 95.1%)	62.6% (57.7%, 67.6%)	2.3 (1.9, 2.7)	0.2 (0.1, 0.4)	0.7 (0.7, 0.8)

^aASSIST V3.0 score of ≥4 and AUDIT score of ≥3 for females and ≥4 for males.

^bAUC = area under the curve.

For identification of unhealthy alcohol use, at the standard cutoff, the SUBS had a sensitivity of 97.6% and specificity of 61.0% for the total sample. Among men, sensitivity was 98.8% and specificity was 58.8%, while among women sensitivity was 92.9% and specificity was 72.5%. For identification of high-risk alcohol use at this cutoff, the SUBS had 100% sensitivity and 40.1% specificity. At the higher cutoff for the entire sample, the SUBS had lower sensitivity (68.4%) for identifying unhealthy alcohol use, but higher sensitivity (90.4%) for high-risk use, and specificity was higher compared to the standard cutoff. Gender-based differences for identifying unhealthy alcohol use were also observed using the higher cutoff. Among men, sensitivity was 73.7% and specificity was 89.3%, while among women sensitivity was 47.6% and specificity was 90.0%.

For unhealthy illicit drug use, at the standard cutoff the SUBS had 85.1% sensitivity and 75.2% specificity for detection of unhealthy use, and 93.2% sensitivity and 48.2% specificity for high-risk use. At the higher cutoff, with respect to identifying any unhealthy use, sensitivity was lower, and specificity higher compared to the standard cutoff. For identifying high-risk use, applying the higher SUBS cutoff resulted in sensitivity of 77.3% and specificity of 65.6%.

The results for the prescription drug category followed a similar pattern. At the standard cutoff, the SUBS had sensitivity of 72.7% and specificity of 82.8% for identifying unhealthy use, and 94.4% (sensitivity) and 75.6% (specificity) for high-risk use. At the higher cutoff, sensitivity was low (42.9%) for unhealthy use, but higher (88.9%) for high-risk use. The combined category of ‘any drugs’ showed a similar pattern of results to that seen when illicit and prescription drugs were examined separately, with higher sensitivity for identifying unhealthy use at the lower cutoff, and slightly lower sensitivity but higher specificity for identifying high-risk use at the higher cutoff. Sensitivity was higher for the combined category of ‘any drugs’ than for either illicit or prescription drugs alone, with the exception of identifying high-risk use or prescription drugs using the higher SUBS cutoff.

The positive likelihood ratios for all substance classes and risk levels ranged from 1.7 to 7.4. At the lower cutoff, individuals with unhealthy use were at least 2.5 times more likely to have a positive result on the SUBS. At the higher cutoff, individuals with high-risk use were at least 2.3 times more likely to have a positive screen. For each substance class and risk category, the higher SUBS cutoff generated higher positive likelihood ratios. Applying the standard cutoff, AUCs were ≥0.70 for all substance classes for identifying unhealthy use and high-risk use. The higher cutoff also gave AUCs ≥0.70 across all substances and risk categories, except for the result for unhealthy use of prescription drugs at the higher SUBS cutoff. In subgroup analyses by gender (Supplemental Table 1), females had a lower sensitivity, but higher specificity for all substances using the lower cutoff for any unhealthy use. For the higher cutoff for any unhealthy use, females also had a lower sensitivity but higher specificity for alcohol, illicit drugs, any drugs, but not for prescription drugs. The number of females in the subgroup analyses were too small for screening for high-risk use to be able to make meaningful comparisons.

4. Discussion

The SUBS is a brief, self-administered screening instrument that has good sensitivity for the detection of past-year unhealthy and high-risk use of alcohol and drugs, in comparison to longer screening instruments, in this population of hospitalized smokers. Information about a patient’s drug and alcohol use during an inpatient admission is important for making accurate diagnoses, providing appropriate treatment during hospitalization, and formulating discharge plans that effectively address the management of chronic conditions and treatment for substance use disorders. This study further highlights the relevance of substance use screening among medical patients with known tobacco use, given the high prevalence of unhealthy substance use in this population.

Our study showed that at the standard cutoff, wherein any SUBS response other than ‘never’ indicates a positive screen, the SUBS had high sensitivity for detecting any unhealthy use and high-risk use of alcohol, illicit drugs, and prescription drugs. At the higher cutoff, wherein any SUBS response of ‘3 or more days’ was considered a positive screen, the SUBS retained high sensitivity for detecting high-risk use, and specificity was improved. These findings suggest that a higher cutoff on SUBS could be used to screen for high-risk use. This is consistent with a study that found single screening questions for alcohol and drugs could identify dependence with fairly high specificity when higher cutpoints were used (Saitz et al., 2014). We also found that for the SUBS, females had a lower sensitivity and higher specificity for detecting any unhealthy use for many substances compared to men. This is consistent with the previous validation study of the SUBS (McNeely et al., 2015). Future studies are needed to explore gender differences in detecting substance use by the SUBS.

Participants consistently reported higher rates of unhealthy use for each substance class on the SUBS versus the AUDIT-C or ASSIST. This difference may be attributed to the SUBS items themselves, and it may reflect differences in the mode of administration. ASSIST and AUDIT-C were both interviewer-administered, while the SUBS was self-administered. Patients may be more open about their alcohol or drug use on self-administered instruments, in comparison to reporting it face-to-face with an interviewer (Tourangeau & Smith, 1996). Given the demands of providers in the inpatient hospital setting, the use of a self-administered tool such as the SUBS, which does not rely on having a trained interviewer administer the questionnaire, may not only help patients report stigmatized behavior, but fit well into the clinical work flow without adding further pressures on providers. The SUBS could be given to patients on admission along with other paperwork often given to patients. The admitting team would then be notified of a positive screen on the SUBS, and that information would assist clinicians in safely caring for the patient while hospitalized and facilitate referral for further evaluation.

4.1. Limitations of the study

Our study has several limitations. First, we assessed the accuracy of the SUBS in comparison to longer, previously validated, and commonly used screening instruments. However, the AUDIT-C and ASSIST are not considered true ‘gold standard’ diagnostic instruments. Our findings thus indicate that the briefer SUBS instrument could be used instead of the AUDIT-C or ASSIST to identify unhealthy and high-risk substance use, but it does not establish the true validity of the SUBS for identifying these conditions. Future studies should examine performance of the SUBS in comparison to gold standard measures (e.g. biologic measures to detect use, diagnostic interview to detect substance use disorders) in inpatient samples.

Another limitation in this study is different timeframes of the screening instruments. While the SUBS and AUDIT-C ask about use in the past 12 months, ASSIST uses a 3-month timeframe. This potentially could affect the specificity results in the study for the SUBS if a respondent had used a substance in the 3 to 12-month window (testing negative for the ASSIST, but positive for the SUBS). Additionally, our study relied on self-report measures, which may have recall and social desirability biases. The latter may be particularly important given the sensitive nature of substance use. Social desirability bias may cause patients to under-report substance use, even on a self-administered instrument, particularly if there is a fear of having such information reported to their providers (Bradley et al., 2011; Davis, Thake, & Vilhena, 2010). This may be more of a problem in the inpatient setting where patients are not as familiar or do not have existing, trusting relationships with their providers such as in primary care. Therefore, privacy concerns could affect the sensitivity and specificity of the SUBS if used in routine inpatient care.

Since the SUBS was always given after the AUDIT-C and ASSIST, participant responses to SUBS may have been influenced by having completed the other screening instruments. For example, participants may have learned from answering the ASSIST what types of drug use are considered non-medical use of prescription drugs, so that when they encountered this question on the SUBS they were able to answer it accurately. This could explain why we found higher sensitivity for identifying unhealthy and high-risk prescription drug use in this study, in comparison to the prior SUBS validation study in primary care patients. Future studies should examine the performance of the SUBS when it is given before any other substance use measures, since this is how it would be administered in practice.

The generalizability of these findings is limited, as all participants were current smokers who were hospitalized in an urban, “safety net” hospital. Further, participants were patients who had agreed to enroll in a randomized clinical trial of smoking cessation interventions. Given the high rates of tobacco-related morbidity in this population, and high rates of concurrent unhealthy alcohol and drug use (Kalman, Morissette, & George, 2005; Ward, Kedia, Webb, & Relyea, 2012), our study did enroll a population who would be considered high priority for substance use screening. It is not clear how well the SUBS would perform in population with a lower prevalence of unhealthy substance use, therefore future studies should examine the validity of the SUBS among broader samples of hospital inpatients.

4.2. Conclusions

Our study shows that the SUBS can be considered as an alternative to longer screening instruments, for identification of unhealthy and high-risk use of alcohol, illicit drugs, and prescription drugs, among current smokers in a hospital setting. The high sensitivity of the SUBS suggests that it may be an appropriate initial screen that, as a very brief self-administered instrument, could fit easily into the busy clinical workflow of inpatient care. However, further validation work is necessary. The SUBS is not intended to be used as a diagnostic instrument for substance use disorders. However, given the high likelihood of patients with high-risk use giving a SUBS response of ‘3 or more days’ of use, a diagnostic interview to identify substance use disorders may be indicated in this group. Future studies using the SUBS should be done in comparison to gold reference standard measures in a larger and more general population of hospitalized patients.