A systematic review of brief, freely accessible, and valid self-report measures for substance use disorders and treatment

Abstract

Background

Self-report measures can improve evidence-based assessment practices in substance use disorder treatment, but many measures are burdensome and costly, limiting their utility in community practice and non-specialty healthcare settings. This systematic review identified and evaluated the psychometric properties of brief, free, and readily accessible self-report measures of substance use and related factors.

Methods

We searched two electronic databases (PsycINFO and PubMed) in May 2021 for published literature on scales, measures, or instruments related to substance use, substance use treatment, and recovery, and extracted the names of all measures. Measures were included if they were: (1) brief (25 items or fewer), (2) freely accessible in a ready-to-use format, and (3) had published psychometric data.

Results

An initial search returned 411 measures, of which 73 (18%) met criteria for inclusion. Included measures assessed a variety of substances (e.g., alcohol, nicotine, opioids, cannabinoids, cocaine) and measurement domains (e.g., use, severity, expectancies, withdrawal). Among these measures, 14 (19%) were classified as psychometrically “excellent,” 27 (37%) were rated as “good,” 32 (44%) were “adequate.”

Conclusions

Despite the shift toward evidence-based assessment in substance use disorder treatment in the last twenty years, key areas of public health concern are lacking pragmatic, psychometrically valid measures. Among the brief measures we reviewed, less than a fifth met criteria for psychometric “excellence” and most of these instruments fell into one measurement domain: screening for problematic substance use. Future research should focus both on improving the evidence base for existing brief self-report measures and creating new low-burden measures for specific substances and treatment constructs.

1. Introduction

Evidence-based assessment (EBA) is an important tool for the accurate screening of problematic substance use and the reliable evaluation of an individual’s substance use disorder (Miller, Forcehimes, & Zweben, 2019). Biological measures, such as urinalysis and plasma testing, are commonly used in substance use disorder (SUD) research, screening, and contingency management. However, these tests only reliably screen for use of certain substances (e.g. cocaine, opioids, cannabis) within the previous days (Allen et al., 2003), can lack sensitivity (Connors et al., 2016), often require costly testing equipment, and may be impractical to implement in general health settings (Pilowsky and Wu, 2013). Biological markers are also incapable of evaluating the dynamic and multidimensional facets of an individual’s SUD and recovery (Carroll, 1995; W. R. Miller et al., 2019). There is now consensus in the literature that biological assays should be used in combination with self-report measures and seldom on their own (Allen et al., 2003; Carroll, 1995; Miller et al., 2019).

Self-report assessments can provide a much more comprehensive view of SUD beyond screening for consumption and help to evaluate important constructs such as severity of dependence, expectancies, antecedents and motivations for use, and readiness for change (Miller, 2009). An understanding of these dimensions can help providers in a variety of settings (including inpatient, outpatient, residential, and primary care) optimize treatment plans, monitor progress more precisely, and strategize for relapse prevention (Marlatt et al., 2008; Miller et al., 2019). Reliable and sensitive self-report measures are a key tool for measurement-based care (MBC), a practice involving routine measurement and progress monitoring to adaptively tailor behavioral health treatment delivery and improve collaboration between patients and clinicians (Lewis et al., 2019; Tauscher et al., 2021). Self-administered screeners could also alleviate confidentiality issues raised by pediatricians surveyed about implementing Screen, Brief Intervention, and Referral to Treatment (SBIRT) among at-risk adolescents (Hammond et al., 2021).

Despite their role in EBA, standardized self-report measures remain underutilized across a range of behavioral health settings, including SUD treatment (Curry and Hanson, 2010; Pavlick et al., 2009). Although there is a paucity of literature on the barriers to implementing EBA in outpatient SUD treatment, a recent survey of alcohol and drug use counselors indicates that the use of standardized measures in initial assessment and progress monitoring may be hindered by their lack of perceived practicality and clinical utility (Revill et al., 2022). The organizational burden involved with implementing EBA may be especially problematic when integrating substance use screening in general health settings where providers commonly cite time pressures and a lack of clinical knowledge, preparedness, and resources to treat SUD (Hammond et al., 2021; McNeely et al., 2018; Palmer et al., 2019). Improved access to reliable, quick-to-administer, self-report measures would ease the implementation of EBA across all SUD treatment settings and would be especially impactful to community providers who lack resources to pay for expensive assessments and scoring (Saunders et al., 2019). A compendium of such brief measures would also serve researchers who are designing large, pragmatic trials of SUD treatments with time-constrained protocols (Loflin et al., 2020), as well as policymakers and behavioral health payers who desire to implement low-burden substance use screeners and other outcome measures in health systems (O’Donnell et al., 2014).

To date, there have been limited efforts to systematically collate and review the existing measures to support broad EBA practices for substance use. The Joint Commission (accessed April 2022) hosts an online repository of measures for behavioral health but includes only two publicly available, self-report substance use measures, the Brief Addiction Monitor and the Brief Situational Confidence Questionnaire. Deady (2009) provided a comprehensive overview of standardized tools to screen for alcohol and drug use disorders - emphasizing tools that require limited training to use and are freely available. This review identified 19 measures related to general drug and alcohol use (n=8), severity (n=7), and craving (n=4). More recently, Becker-Haimes and colleagues (2020) reviewed free and brief substance use screening tools for use with adolescents, identifying 13 measures that were intended to support screening for general substance use (n=6), nicotine addiction (n=3), cannabis (n=3) and alcohol (n=1). The U.S. Prevention Services Task Force recommends screening all adults 18 years or older for unhealthy substance use in primary care settings and reviewed 30 screening tools to support clinicians who seek to implement SBIRT (Patnode et al., 2020). Of note, the authors of this review highlighted the need to track outcomes beyond frequency of use, such as severity of unhealthy use and consequences of use. Indeed, consumption reflects only a small portion of SUD-related constructs that may warrant assessment and monitoring throughout treatment (Bjornestad et al., 2020). In particular, the ability to assess negative consequences, expectancies, urges, motives, and recovery attitudes is desirable because it may help clinicians tailor treatment to better address the specific issues that increase a patient’s vulnerability to relapse (Moos, 2007; Tiffany et al., 2012). Consequently, an expanded review of practical measures that encompasses all constructs of substance use is needed.

We conducted a systematic review of brief, free, valid, and readily accessible measures for SUD and its related constructs. Specifically, our goal was to identify a comprehensive list of self-report measures, provide a concise overview of each measure’s psychometric strengths, and create a resource for clinicians, researchers, and policymakers that can inform EBA practices. We also aimed to provide recommendations for future measure development and psychometric validation in SUD.

2. Materials and Methods

We employed a comparable methodology to that used in previously completed reviews of psychometrically valid, free, brief, and readily accessible measures in youth mental health (Becker-Haimes et al., 2020; Beidas et al., 2015). Specifically, we used a three-stage systematic review process that included: (1) measure identification, (2) measure eligibility assessment, and (3) evaluation of psychometric properties. Reporting of this systematic review was guided by the standards of the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) Statement. Figure 1 illustrates the PRISMA flow diagram.

Figure 1.

PRISMA Flow Diagram

2.1. Measure Identification

We searched PubMed and PsycINFO in May 2021 using the following search terms: (“instrument” OR “survey” OR “questionnaire” OR “measure” OR “assessment”) AND (“psychometric or “measure development” or “measure validation”) AND (“substance abuse” OR “substance use” OR “substance use disorder” OR “misuse” OR “dependen*” OR “alcohol OR “opioid” OR “tobacco” OR “nicotine” OR “marijuana” OR “cannabis” OR “addiction” OR “crack” OR “cocaine” OR “heroin” OR “fentanyl” OR “drug use” OR “drug abuse” OR “methamphetamine” OR “benzodiazepine” OR “narcotic”). Additionally, we performed a search for recovery. We used the following search terms: (“instrument” OR “survey” OR “questionnaire” OR “measure” OR “assessment”) AND (“psychometric or “measure development” or “measure validation”) AND (“substance abuse” OR “substance use” OR “substance use disorder” OR “misuse” OR “dependen*” OR “alcohol OR “opioid” OR “tobacco” OR “nicotine” OR “marijuana” OR “cannabis” OR “addiction” OR “crack” OR “cocaine” OR “heroin” OR “fentanyl” OR “drug use” OR “drug abuse” OR “methamphetamine” OR “benzodiazepine” OR “narcotic”) AND (“recovery”).

This initial search returned 3135 articles, including systematic reviews and handbooks. Members of the authorship team (RS, EBH, NC, AS, and CB) screened abstracts for relevance. We included abstracts if there was an indication of a report on a measure of substance use or misuse, severity of dependence, withdrawal, problems, or consequences associated with a SUD, and or other symptomology of a SUD (including effect expectancy, relapse propensity, etc.). In addition, we included additional constructs of direct relevance to SUD treatment, including evaluations of quality of life and degree of lifestyle change (including motivational based measures, barriers to recovery, stage of change, treatment readiness/intentions). We also hand-searched three textbooks returned by the search: the Clinician’s Handbook of Adult Behavioral Assessment, Clinician’s Handbook of Child Behavioral Assessment, and Handbook of Assessment and Treatment Planning for Psychological Disorders. Although needed in a separate review, we excluded measures related to the stigma of substance use disorder and its treatment in the current review (Earnshaw, 2020). We also excluded non-English articles, duplicates, and articles that reported on a measure that was not intended to be self-administered (i.e., diagnostic interviews). The team initially met and screened a subset of articles as a group (n=50) and achieved perfect reliability. The remaining articles were screened independently, and the team met to discuss any concerns and came to consensus when a coder was uncertain regarding the relevance of an abstract. A total of 823 abstracts were classified as relevant. The full texts of these abstracts were screened and the names of all measures mentioned were extracted; 406 measures were identified across the 823 articles. The textbook search returned five additional measures, for a total of 411 measures to be screened for eligibility.

2.2. Measure Eligibility

Measures were required to meet four inclusion criteria to be eligible for this review. Specifically, measures had to: (1) be brief (i.e., defined as 25 items or fewer, including sub-items), (2) be free (i.e., no cost for individual use), (3) be readily accessible (i.e., be readily downloadable for use from the Internet), and (4) have at least some published data for their psychometric properties. To meet the “free” criterion, measures must have been accessible and reproducible without cost or copyright infringement. To meet our “readily accessible” criteria, measures needed to be immediately ready for use. For instance, measure items published in an appendix of an open-access journal were not included if the measure would have needed to be reformatted before it could be implemented in practice. We included measures for both adolescents and adults in the current review.

2.3. Measure Psychometrics

This review is based on established guidelines for evidence-based updates (Southam-Gerow et al., 2016) which encourages high-level analysis of an area of assessment to help mental health professionals better identify and implement EBA. Substance use disorder is a vast and complex phenomenon that involves a range of measures over the treatment cascade (Williams et al., 2019). To account for this, our search was purposefully broad to capture measures across a variety of substance types and measurement constructs. In anticipation of the large number of measures that would meet inclusion criteria, we considered a full psychometric review on each measure to be infeasible for the current study. Consequently, we modified the rubric offered by De Los Reyes & Langer (2018) that guided a recent review of youth measures with parallel aims (Becker-Haimes et al., 2020) to assess measures on fewer criteria and focus on the presence or absence of evidence rather than quality of specific psychometric properties. For instance, given the number of ways a measure’s validity can be indexed (Cook & Beckman, 2006; Newton, 2013), we condensed the four validity categories from the original rubric (i.e. construct, content, discriminative, and prescriptive validities) and prioritized collecting evidence that supported a measure’s usefulness, such as construct validity and discriminative validity (Hughes, 2018). We also did not include review criteria for interrater reliability (due to its irrelevance for assessing self-report measures), repeatability, and clinical utility. Despite these omissions, we took steps to improve the rigor of our psychometric evaluation. In particular, we avoid penalizing shorter measures by using the Spearman Brown Prophecy Formula (Nunnally, 1978) to provide a threshold for internal consistency that is scaled to measure length (assuming a low inter-item correlation, r = 0.3)(Streiner et al., 2015). We also assess the internal consistency of multidimensional measures at the subscale level to avoid underestimation of reliability (Revelle and Zinbarg, 2009).

We leveraged existing systematic reviews (See Supplemental Table 1) and employed an exhaustive citation searching approach (Wright et al., 2014) to evaluate the quality of evidence for each eligible measure. Using the adapted rubric, we summarized the psychometric properties of each measure and characterized the quality of evidence as, “excellent”, “good,” or “adequate.” While each eligible measure is reviewed briefly below in text, additional information about their psychometric properties (including exact values for each psychometric property, normative data, sample population, and criterion used for validation) is included in Supplemental Table 1. Of note, we included a number of measures that are brief versions of more well-established original measures, but do not review the original measures directly. Where applicable, we report correlation coefficients between the brief and original versions of measures in Supplemental Table 1.

Measures achieved a rating of “excellent” if we located at least two independent studies that provided evidence for (1) test-retest reliability greater than .7 over a period of several days or weeks, (2) internal consistency (e.g. Cronbach’s alpha, split-half reliability, McDonald’s Omega) greater than the threshold based on the number of items on the measure (e.g. .91 for 25-item measures, .87 for 15-item measures, .81 for 10-item measures, etc.), (3) a type of validity (i.e. convergent, discriminant, predictive, concurrent, discriminative), (4) normative data from a clinical sample, and (5) treatment sensitivity (if not intended to be used as a screening measure). Measures were rated as “good” if they met the above criteria for “excellent” in one study, but we were unable to validate the information with a second source or if they met all of the following criteria in at least two independent studies: (1) test-retest reliability or treatment sensitivity data (if a non-screening instrument), (2) internal consistency, (3) one type of psychometric validity, (4) any normative data. Measures had an “adequate” quality of evidence if we located any evidence of validity or reliability (internal consistency or test-retest reliability) or if it met the above criteria for “good,” but had an internal consistency or test-retest reliability lower than the aforementioned thresholds.

2.4. Measure Organization

The identified measures were categorized either by substance (general or specific) or as SUD treatment related measures. Finally, to optimize interpretation and usability of the set of measures identified, we tagged each measure with one or more labels, defined post hoc by the authorship team, with final determination guided by author JM, an expert in SUD, to describe the specific constructs assessed by each (See Table 1). We also refer readers to Supplementary Tables 1 and 2 for more detailed information on the psychometric strengths of each included measure and direct links for accessing each measure.

Table 1.

Substance Use Disorder Constructs

Label	Definition
Use	Identifies, screens, or indicates clinically significant substance use problems; typically administered upon entering treatment and has a discrete outcome (e.g., cut-off scores separating those with and without clinically significant substance use disorder).
Severity	Measures degree of drug or alcohol use disorder, typically with a continuous scale.
Consequences	Measures problems, dysfunctions, and negative life outcomes (e.g., social, emotional, physical) related to substance use.
Withdrawal	Evaluates physical withdrawal symptoms attributed to substance use cessation.
Cravings/Urges	Evaluates psychological withdrawal symptoms, inclinations, or feelings that may be felt in the absence of physical withdrawal symptoms.
Expectancies	The beliefs regarding the positive and negative behavioral, cognitive, and emotional effects of a substance (Cooper et al., 2015).
Motives	The use of a given substance in order to achieve a desired effect or outcome (Cooper et al., 2015).
Self-efficacy	The belief that one has the ability to implement the behaviors to produce a desired effect (Kadden and Litt, 2011).
Recovery	Assess a patient’s desire for behavior change, engagement with treatment or abstinence, as well as cognitions, attitudes, and beliefs related to recovery.

3. Results

The 411 measures that met our initial screening criteria included measures related to general substance use (n=91), alcohol (n=115), nicotine (n=72), cannabinoids (n=28), opioids (n=19), stimulants (n=20), benzodiazepines (n=7), hallucinogens (n=2), multi-substance use (n=2), and SUD treatment (n=55). Measures were first screened and excluded for the brief criterion, then accessibility, and validity. Of the 411 measures initially identified, 147 measures were excluded for length (i.e. n=112 with 26–50 items, n=24 measures with 51–100 items, and n=11 with over 100) and 119 were not found after an exhaustive Internet search. Among the (n=72) measures that were categorized as brief and accessible, 42 measures were excluded for not being accompanied by instructions or scoring information and 16 were excluded due to copyright or prohibitions on free use. We removed additional measures that were outdated versions of measures already captured by the review (n=6), lacked psychometric data on validity or reliability (n=6), did not have a verifiable original source (n=1), or were not available in a ready-to-use format (n=1). Ultimately, 73 (18% of the original 411) measures were brief (25 or fewer items), free, readily accessible, and had at least minimal published psychometric data. The ratings of psychometric strength and properties for the final set of 73 measures are reported in Table 2. For further information regarding the psychometrics of each measure and how to access each, please refer to Supplemental Tables 1–2. Below we summarize each measurement area’s quality of evidence, the intended use of included measures, and the quantity of evidence for each psychometric property.

Table 2.

Psychometric Properties of Included Measures

Measure	Construct	Rating	Norms	Internal Consistency	Test-retest reliability	Validity	Treatment Sensitivity
General
Knight et al., 1999 CRAFFT Substance Abuse Screening Instrument	Use	Excellent	Excellent	Excellent	Excellent	CONCUR, CONVERG, DIVERG	Not applicable
Bohn et al., 1991 Drug Abuse Screening Test-10	Use	Excellent	Good	Excellent	Excellent	CONCUR, CONVERG	Not applicable
Skinner & Goldberg, 1986 Drug Abuse Screening Test-20	Use	Excellent	Good	Excellent	Excellent	CONCUR, CONVERG, DIVERG	Not applicable
Berman et al., 2003 Drug Use Disorders Identification Test	Use	Excellent	Good	Excellent	Excellent	CONCUR, DIVERG	Not applicable
Raistrick et al., 1994 Leeds Dependence Questionnaire	Severity	Excellent	Excellent	Excellent	Excellent	CONCUR, CONVERG, DIVERG, DISCRIM	X
Gossop et al., 1995 Severity of Dependence Scale	Severity	Excellent	Excellent	Excellent	Excellent	CONCUR, DIVERG	X
Brown & Rounds, 1995 CAGE-Adapted to Include Drugs	Use	Good	Good	Good	Good	CONVERG, DIVERG	Not applicable
Schlesinger et al., 2007 Indigenous Risk Impact Screen	Use	Good	Good	Good	Good	CONCUR, CONVERG	X
McRee et al., 2018 Alcohol, Smoking and Substance Involvement Screening Test - Frequency and Concern	Severity, Use	Adequate	Good	Adequate	Good	CONCUR, DIVERG	Not applicable
Cacciola et al., 2013 Brief Addiction Monitor - Continuous	Severity	Adequate	Good	Good	Good	PREDICT	X
Hiller et al., 2000 Drug Abstinence Self-Efficacy Scale	Self-Efficacy	Adequate		Excellent	Good	CONVERG, DISCRIM
Sklar & Turner, 1997 Drug-Taking Confidence Questionnaire 8	Self-Efficacy	Adequate	Adequate	Good	Good	CONVERG, DISCRIM
Center for Substance Abuse Treatment, 1994 Simple Screening Instrument - Substance Abuse	Use	Adequate	Adequate	Adequate	Good	CONVERG, DIVERG	Not applicable
Alcohol
Bohn et al., 1995 Alcohol Urge Questionnaire	Urges	Excellent	Excellent	Excellent	Excellent	CONCUR, CON VERG, DISCRIM, PREDICT	X
Saunders et al., 1993 Alcohol Use Disorders Identification Test	Use	Excellent	Excellent	Excellent	Good	CONCUR, DIVERG, PREDICT	Not applicable
Bush et al., 1998 Alcohol Use Disorders Identification Test - Consumption	Use	Excellent	Good	Excellent	Good	CONVERG, DIVERG, PREDICT	Not applicable
Mayfield, 1974 Cut down, Annoyed, Guilt, Eye-opened Screening Tool	Use	Excellent	Excellent	Excellent	Good	CONVERG, DIVERG	Not applicable
Selzer, 1971 Michigan Alcohol Screening Test	Use	Excellent	Good	Excellent	Good	CONVERG, DIVERG	Not applicable
DiClemente et al., 1994 Alcohol Abstinence Self-Efficacy Scale	Self-Efficacy	Good	Excellent	Good		DISCRIM, PREDICT	X
Pokorny et al., 1972 Brief Michigan Alcohol Screening Test	Use	Good	Good	Excellent	Good	CONCUR, CONVERG, DIVERG	Not applicable
Breslin et al., 2000 Brief Situational Confidence Questionnaire	Self-Efficacy	Good	Good	Excellent	Excellent	CONCUR, CONVERG, PREDICT	X
Hodgson et al., 2002 Fast Alcohol Screening Test	Use	Good	Good	Excellent	Good	CONCUR, DIVERG, PREDICT	Not applicable
Stockwell et al., 1983 Severity of Alcohol Dependence Questionnaire	Severity	Good	Excellent	Excellent	Adequate	CONCUR, DIVERG
Raistrick et al., 1983 Short Alcohol Dependence Data Questionnaire	Severity	Good	Excellent	Good	Excellent	CONCUR, DISCRIM	X
Tonigan et al., 1995 Short Inventory of Problems	Conseque nces	Good	Good	Excellent	Excellent	CONVERG, PREDICT	X
Kiluk et al., 2013 Short Inventory of Problems - Revised	Conseque nces	Good	Good	Excellent		CONCUR, CONVERG, DISCRIM, PREDICT	X
Selzer et al., 1975 Short Michigan Alcohol Screening Test	Use	Good	Good	Good	Good	CONCUR, CONVERG, DIVERG, DISCRIM	Not applicable
Singleton, 1995 Alcohol Craving Questionnaire Short Form Revised	Urges	Adequate	Adequate	Good		CONCUR	X
Ham et al., 2005 Brief Version of the Comprehensive Effects of Alcohol Questionnaire	Expectancies	Adequate		Adequate		CONCUR, CONVERG
Oei et al., n.d. Drinking Expectancy Questionnaire - Adolescent Version	Expectancies	Adequate	Adequate	Adequate		CONCUR, DIVERG
Patton et al., 2018 Drinking Expectancy Questionnaire - Shortened Adolescent Version	Expectancies	Adequate		Good		CONCUR, PREDICT
Cooper, 1994 Drinking Motives Questionnaire-Revised	Motives	Adequate	Good	Excellent	Excellent	CONCUR	X
Keyson & Janda, n.d. Drinking-Related Locus-of-Control Scale	Expectancies, Self-Efficacy	Adequate	Excellent	Good		CONCUR, DIVERG, DISCRIM
Stockwell et al., 1994 Severity of Alcohol Dependence Questionnaire - Community Samples	Severity	Adequate	Good	Good		CONCUR	X
Gossop et al., 2002 Short Alcohol Withdrawal Scale	Withdrawal	Adequate	Good	Good		CONCUR
Collins & Lapp, 1992 Temptation and Restraint Inventory	Urges	Adequate	Good	Good		CONCUR, PREDICT
Nicotine
Heatherton et al., 1991 Fagerstrom Test for Nicotine Dependence	Severity	Excellent	Excellent	Excellent	Excellent	CONVERG	X
DiFranza et al., 2002 Hooked on Nicotine Checklist	Use	Excellent	Excellent	Excellent	Good	CONCUR, CONVERG, DIVERG, DISCRIM, PREDICT	Not applicable
Prokhorov et al., 1996 Modified Fagerstrom Tolerance Questionnaire	Severity, Use	Excellent	Excellent	Excellent	Excellent	CONCUR, DIVERG, DISCRIM, PREDICT	X
DiFranza et al., 2009 Autonomy over Tobacco Scale	Severity, Urges, Withdrawal	Good	Good	Excellent	Adequate	CONCUR, DISCRIM
Glover et al., 2005 Glover-Nilsson Smoking Behavioral Questionnaire	Severity	Good	Good	Good	Good	CONCUR, CONVERG, DISCRIM	Not applicable
Heatherton et al., 1989 Heaviness of Smoking Index	Severity	Good	Good	Excellent	Adequate	CONVERG, DIVERG, PREDICT
Smith et al., 2021 Wisconsin Smoking Withdrawal Scale - Brief	Consequences, Withdrawal	Good	Good	Good	Adequate	CONVERG, DISCRIM, PREDICT
Smith et al., 2021 Wisconsin Smoking Withdrawal Scale - Long	Withdrawal	Good	Good	Good	Adequate	CONVERG, DISCRIM, PREDICT
Rash & Copeland, 2008 Brief Smoking Consequences Questionnaire	Consequences	Adequate	Good	Adequate		CONVERG, DIVERG
Morean et al., 2019 E-cigarette Dependence Scale	Severity	Adequate		Good		CONCUR, CONVERG
Ebbert et al., 2006 Fagerstrom Test for Nicotine Dependence for Smokeless Tobacco Users	Severity	Adequate	Good	Good		CONCUR, CONVERG
Foulds et al., 2015 Penn State E-cigarette Dependence Index	Use	Adequate	Good	Adequate		CONVERG
Severson et al., 2003 Severson 7-item Smokeless Tobacco Dependence Scale	Severity	Adequate	Good	Good		CONCUR, CONVERG, DIVERG
Myers et al., 2003 Short Form of the Smoking Consequences Questionnaire	Consequences	Adequate	Good	Excellent		CONCUR, CONVERG, PREDICT
Blake et al., 2016 Smoking Restraint Questionnaire	Self-Efficacy	Adequate	Adequate	Good		PREDICT
Cannabinoid
Adamson et al., 2010 Cannabis Use Disorder Identification Test - Revised	Use	Good	Good	Excellent	Good	CONVERG, DIVERG, DISCRIM, PREDICT	Not applicable
Bashford et al., 2010 Cannabis Use Problems Identification Test	Use	Good	Good	Good	Good	CONVERG, DIVERG, DISCRIM, PREDICT	Not applicable
Allsop et al., 2011 Cannabis Withdrawal Scale	Withdrawal	Good	Good	Good	Excellent	CONCUR, PREDICT	X
Simons et al., 1998 Marijuana Motives Measure	Motives	Good	Good	Excellent		CONCUR, CONVERG	X
Stephens et al., 2000 Marijuana Problems Scale	Consequences	Good	Good	Good		CONVERG, DIVERG	X
Copeland et al., 2005 Cannabis Problems Questionnaire	Consequences	Adequate	Good	Adequate	Adequate	CONCUR, CONVERG, DIVERG
Opioid
Butler et al., 2007 Current Opioid Misuse Measure	Use	Good	Excellent	Good	Excellent	CONVERG, DIVERG, DISCRIM, PREDICT	X
Butler et al., 2008 Screener and Opioid Assessment for Patients with Pain - Revised	Use	Good	Excellent	Good	Good	CONCUR, DIVERG, PREDICT	Not applicable
Wickersham, 2015 Rapid Opioid Dependence Screen	Use	Adequate		Good		CONVERG, DIVERG	Not applicable
Stimulant
Handelsman et al., 1987 Subjective Opiate Withdrawal Scale	Withdrawal	Adequate	Excellent	Adequate	Excellent	DIVERG	X
McGregor et al., 2008 Amphetamine Cessation Symptom Assessment	Withdrawal	Adequate	Excellent	Adequate		CONVERG, DISCRIM, PREDICT
Sussner et al., 2006 Cocaine Craving Questionnaire Brief	Urges	Adequate		Excellent		CONCUR, CONVERG, PREDICT	X
Benzodiazepine
Tyrer et al., 1990 Benzodiazepine Withdrawal Symptom Questionnaire	Withdrawal	Adequate	Excellent	Adequate	Excellent	DISCRIM, PREDICT	X
Treatment Related
Luoma et al., 2011 Acceptance and Action Questionnaire - Substance Abuse	Recovery, Self-Efficacy, Urges	Good	Excellent	Good		CONCUR, CONVERG, DISCRIM	X
Rollnick et al., 1992 Readiness to Change Questionnaire	Recovery	Good	Good	Excellent	Adequate	CONCUR, CONVERG, PREDICT
Spek et al., 2013 Smoking Abstinence Self-efficacy Questionnaire	Recovery, Self-Efficacy	Good	Good	Excellent		DISCRIM, PREDICT	X
Miller & Tonigan, 1997 Stages of Change Readiness and Treatment Eagerness Scale Version 8	Recovery	Good	Good	Good	Adequate	CONVERG, PREDICT
Heather & Honekopp, 2008 Treatment Version - Readiness to Change Questionnaire	Recovery	Good	Good	Excellent	Excellent	CONCUR, CONVERG, PREDICT	X
Simmons et al., 2010 Abstinence-related Motivational Engagement	Recovery	Adequate	Adequate	Excellent		CONCUR, CONVERG, DIVERG, DISCRIM, PREDICT
Vilsaint et al., 2017 Brief Assessment of Recovery Capital	Recovery	Adequate	Excellent	Good		CONCUR, DIVERG, PREDICT
Winters et al., 1987 Problem Recognition Questionnaire	Recovery	Adequate	Adequate	Good		PREDICT
DiClemente et al., 1991 Processes of Change Questionnaire	Recovery	Adequate	Good	Excellent	Adequate	CONVERG

CONCUR = Concurrent Validity, CONVERG = Convergent Validity, DIVERG = Divergent (Discriminant) Validity, DISCRIM = Discriminative Validity, PREDICT = Predictive Validity.

“X” indicates that we found evidence for treatment sensitivity for that measure.

3.1. General Substance Use

Thirteen of 91 identified measures of general substance use met inclusion criteria (14%; Table 2). Measures ranged from two to 20 items. Measures with “excellent” psychometric properties included four screening measures: the CRAFFT Substance Abuse Screening Instrument (Knight et al., 1999) the 10-item version of the Drug Abuse Screening Test (Bohn et al., 1995), the 20-item version of the Drug Abuse Screening Test (Skinner and Goldberg, 1986) and the Drug Use Disorders Identification Test (Berman et al., 2005), as well as two assessments of substance dependence severity: the Leeds Dependence Questionnaire (LDQ) (Raistrick et al., 1983), and the Severity of Dependence Scale (Gossop et al., 1995). Among the remaining seven measures, one measure was rated as “good,” and six measures were rated as “adequate.”

Overall, included measures were primarily intended to screen for or assess the severity a range of substance use disorder subtypes (e.g., alcohol use disorder, opioid use disorder, stimulant use disorder) The Drug-Taking Confidence Questionnaire (DTCQ-8) (Sklar & Turner, 1999) and Drug Abstinence Self-Efficacy Scale (DASES) (Hiller et al., 2000) were the only measures in this category to meet inclusion criteria which evaluates a construct other than use. The DTCQ-8 had “good” psychometric properties and was used to assess situational self-confidence in high-risk situations related to drug or alcohol use (Sklar & Turner, 1999). Every measure, except the Alcohol, Smoking and Substance Involvement Screening Test – Frequency and Concern (McRee et al., 2018) and DASES, had evidence for both internal consistency and test-retest reliability. We found at least one report of normative data for every measure in this category except the DASES. We found evidence for treatment sensitivity for all three measures to assess the severity of substance use related behaviors: the Brief Addiction Monitor – Continuous (Cacciola et al., 2013), SDS, and LDQ.

3.2. Alcohol

Twenty-three of 115 identified alcohol measures met inclusion criteria (20%; Table 2). Measures ranged from three to 25 items and have been administered in both clinical and community populations. Five measures had “excellent” psychometric properties: the Alcohol Use Disorder Identification Test (Saunders et al., 1993), the AUDIT-Consumption (Bush, 1998), the Alcohol Urge Questionnaire (Bohn et al., 1995), the Cut down, Annoyed, Guilt, Eye-opened Screening Tool (Mayfield et al., n.d.), and the Michigan Alcohol Screening Test (Selzer, 1971). Nine measures were rated as having “good” psychometric properties and the remaining nine were rated as “adequate.”

Most measures in this category were primarily used to screen for hazardous drinking or assess severity of dependence. However, several other included measures are intended to evaluate constructs beyond consumption, such as expectancies, urges, motives, and drinking-related consequences. All measures had evidence for internal consistency and most provided some data for test-retest reliability, validity, and norms. Among the 16 non-screening measures, nine had evidence supporting sensitivity to change related to a clinical or therapeutic intervention.

3.3. Nicotine

Fifteen of 72 measures related to nicotine met inclusion criteria (21%; Table 2). Measures ranged from two to 25 items. Three measures, the Hooked On Nicotine Checklist (DiFranza, 2002), the Fagerström Test for Nicotine Dependence (FTND) (Heatherton et al., 1991), and the Modified Fagerström Tolerance Questionnaire (mFTQ)(Prokhorov et al., 1996) were rated as having excellent overall psychometric properties. Five measures were rated as “good” and seven were rated as “adequate.”

Most included measures were intended for use to assess severity of nicotine addiction on a continuous scale. However, we also included measures related to urges, withdrawal, nicotine-related consequences, and restraint self-efficacy. Additionally, we included four scales related to methods of nicotine administration other than cigarette use. The FTND for Smokeless Tobacco Users (Ebbert et al., 2006), the Severson 7-item Smokeless Tobacco Dependence Scale (Severson et al., 2003), the E-cigarette Dependence Scale (ECDS)(Morean et al., 2019) and the Penn State E-cigarette Dependence Index (Foulds et al., 2015), were all rated as adequate. All nicotine-related measures had evidence supporting their internal consistency and validity. Most measures had evidence supporting their test-retest reliability. The ECDS was the only measure in this category without any normative data. We identified evidence supporting sensitivity to treatment among the mFTQ, FTND, and Heaviness of Smoking Index (Heatherton et al., 1989).

3.4. Cannabinoids

Six of 28 measures related to cannabinoids met inclusion criteria (21%; Table 2). Measures ranged from eight to 25 items. No cannabinoid measure met criteria to be psychometrically “excellent”. Five measures were rated as having “good” psychometric properties and one was rated as “adequate.”

This category included measures that are used to screen for problematic cannabis use, assess and monitor withdrawal symptoms, understand motives for use, and characterize problems due to excessive use. All six cannabinoid measures had evidence supporting their internal consistency and validity and had at least one report of normative data. Four out of the six measures had evidence for test-retest reliability. The Cannabis Withdrawal Scale (Allsop et al., 2011), the Marijuana Motives Measure (Simons et al., 1998), and Marijuana Problems Scale (Stephens et al., 2000) all showed evidence for treatment sensitivity.

3.5. Opioids

Four of 19 measures related to opioids met inclusion criteria (21%; Table 2). Measures ranged from eight to 24 items. The Current Opioid Misuse Measure (COMM) and its sister-measure, the Screener and Opioid Assessment for Patients with Pain – Revised (SOAPP-R) (Butler et al., 2007) were both rated as “good”. The two remaining measures were rated as “adequate”.

Included measures are intended to screen for opioid misuse or rapidly assess opioid withdrawal symptoms. The COMM has been tested in two large (n=277 and n=302) clinical samples of chronic pain patients to document opioid misuse behavior on a periodic basis. Also developed by Butler and colleagues, the SOAPP-R compliments the COMM by predicting future opioid misuse behavior based on past behaviors and cognitions (Butler et al., 2007). All measures had evidence for validity and internal consistency. Only the Rapid Opioid Dependence Screen (Wickersham et al., 2015) lacked evidence for test-retest reliability. Both non-screening measures, the COMM and the Subjective Opiate Withdrawal Scale (SOWS-Handlesman)(Handelsman et al., 1987) had evidence supporting its sensitivity to treatment. However, more recent evidence shows that the SOWS-Handelsman failed to detect treatment related changes to subjective withdrawal symptoms(Yu et al., 2008).

3.6. Stimulants

Two out of 20 measures related to cocaine, amphetamines, and other stimulants met inclusion criteria (10%; Table 2). The 10-item Cocaine Craving Questionnaire Brief (CCQ-Brief) (Sussner et al., 2006) and 16-item Amphetamine Cessation Symptom Assessment (ACSA) (McGregor et al., 2008) were both categorized as psychometrically “adequate.”

The CCQ-Brief has excellent internal consistency and is highly correlated with its well-established parent measure, the CCQ-Now. However, there is minimal normative data and limited data on the validity of the brief CCQ. The ACSA had limited data for its internal consistency, test-retest reliability, and norms, but had compelling evidence for its treatment sensitivity.

3.7. Benzodiazepines

Only the 20-item Benzodiazepine Withdrawal Symptom Questionnaire (BWSQ)(Tyrer et al., 1990) met inclusion criteria out of 7 measures related to benzodiazepines (14%; Table 2). The BSWQ’s psychometric properties were rated as “adequate.”

The BWSQ has two forms: the BWSQ1 which asks patients to self-report severe symptoms of withdrawal at any time and the BWSQ2 which asks its completers about any withdrawal symptoms within the past two-weeks. The BWSQ2 has evidence for adequate internal consistency (α=.84-.87) for 20-item measures, high test-retest reliability at one-week (α=.75-.88), and normative data from a clinical sample. In addition, individuals in a benzodiazepine discontinuation program showed reduced scores of BWSQ2, suggesting that the measure is sensitive to treatment.

3.8. Other measures

None of the (n=4) identified measures related to hallucinogenic substances or specific multi-substance use met inclusion criteria.

3.8. Treatment-Related Measures

Nine out of 55 measures related to treatment met inclusion criteria (16%; Table 2) and ranged from six to 25 items. No measure was rated as “excellent”. The Stages of Change Readiness and Treatment Eagerness Scale (SOCRATES-8)(Miller and Tonigan, 1997), the original (RCQ) (Rollnick et al., 1992) and treatment versions of the Readiness to Change Questionnaire (RCQ[TV])(Heather and Hönekopp, 2008), the Processes of Change Questionnaire (PCQ) (DiClemente et al., 1991), and the Abstinence-Related Motivational Engagement (ARME)(Simmons et al., 2010) scale all had “good” psychometric properties. The remaining five measures were rated as having “adequate” psychometric properties.

Overall, the included measures were intended to evaluate a variety of treatment-related constructs such as change readiness, motivational engagement, and recovery self-efficacy. The SOCRATES-8, RCQ, and PCQ are all reliable assessments of the status of an individual who is attempting to resolve their drinking or drug use dependency. Most included treatment-related measures had normative data available as well as evidence supporting its internal consistency and validity. Four of the nine measures had evidence for test-retest reliability and three had evidence for treatment sensitivity.

4. Discussion

This investigation is the most comprehensive and focused review to date of brief, free, readily accessible, and validated self-report substance use measures. These pragmatic measures can be used in routine and low-resource clinical settings, specialty and non-specialty care, as well as other health care treatment organizations, health systems, and payers to implement measurement-based care, assess and reward clinical outcomes, or design value-based purchasing payment initiatives. A strength of this review is that we included measures that assessed a variety of constructs that may be relevant over the duration of a SUD. For instance, we include brief screening measures for alcohol and substance use that may be useful for primary care providers seeking to implement routine screenings. We also provide a number of withdrawal and craving measures so that clinicians may better monitor patients’ experiences during detoxification. We assessed and compiled both consumption-type measures (which could complement biological assessments) as well as measures related to other substance use symptomology and sequelae such as cognitions, consequences, and expectancies. The latter set of constructs cannot be measured biologically and could be critically important for tailoring treatments and mitigating risk of relapse (Tiffany et al., 2012). We labeled our final list of measures by intended construct to facilitate measure selection. Finally, we included treatment-related measures to provide a resource on instruments that assess change processes, growth, and recovery attitudes, another important domain of substance use treatment. To our knowledge, this is the first published review of brief measures related to substance use disorder recovery.

Our review identified over 400 self-report measures of substance use disorder and treatment, yet only a fraction were pragmatic tools that could be easily accessed and implemented in low-resource healthcare settings. Our final set of measures covered general and specific substance use, and a wide range of domains involved with substance use disorder experience. Almost half of the brief, free, accessible, and validated measures focus on nicotine or alcohol. We found only four measures for opioids, two for stimulants (cocaine and amphetamine), and zero for hallucinogens or measures specific to combined substance use. The scarcity of measures related to illicit substances is notable given the decades-long opioid epidemic, now in its fourth wave, and the dramatically rising cocaine, amphetamine, and polysubstance mortality rate (Ciccarone, 2021). More so, there is a lack of brief measures on concurrent stimulant and opioid use despite the shifting patterns of stimulant use among individuals with opioid use disorder (Ellis et al., 2021). Among the seventeen included opioid and general substance use measures, only the ASSIST-FC, BAM-C, and DUDIT include questions that capture use of synthetic opioids, which are not only more lethal, but the primary opioid used in many parts of the United States (Mattson et al., 2021). In addition, there are a relatively limited number of brief measures evaluating cannabis use despite the expansion of the cannabinoid-using population in recent decades (Hasin and Walsh, 2021).

With regards to psychometric validity, despite our less conservative criteria for categorizing measure quality, only a small subset of measures across the broad domains were classified as having “excellent” psychometric properties. Among non-screening measures, fewer than half had information regarding the measure’s ability to detect treatment-related changes in outcome or process studies, a critically important property that is relatively neglected in validation studies (Hunsley and Mash, 2008). The instruments that achieved psychometric excellence were primarily composed screeners of consumption for general substance use, nicotine, and alcohol. Most of the measures of negative consequences, expectancies, motives, self-efficacy, or recovery only achieved a rating of “adequate”. Taken together, these results suggest more work is needed to advance the science and practice of pragmatic self-report measures to be used in substance use practice and research. In totality, this review highlights the need for measurement in several key areas of public health concern, as well as areas for future psychometric validation.

We offer three suggestions to increase the development and uptake of validated self-report measures for substance use in practice and research. First, we can address accessibility. It is important that measures not only be free but “easy” to access, that is, in a quickly obtainable and ready-to-use format on the Internet. Among the 411 relevant measures pulled from the article screen, we excluded almost half due to accessibility reasons. The majority were not found from a standard Internet search or were blocked by copyright. Copyright concerns are more challenging to overcome but represented only a minority of excluded measures. In fact, a substantial portion were freely available but not in a print-ready format or lacked instructions and a scoring guide. Measure developers could be encouraged to publish ready-to-use formats and to link to these measures on their professional websites. Federal bodies such as the National Institute of Drug Abuse and the Substance Abuse and Mental Health Services Administration could also host measure repositories to promote access and data sharing. In the interim, we are providing our own online supplement with an interactive list of over four-hundred measures along with access links and information on the measures assessed in the current review (https://cmsdev1.pmacs.upenn.edu/stewartlab/eba-repository.html).

Second, in addition to increasing access, there is also a clear need to develop new and additional measures to address the evolving public health epidemic, focusing on measures for synthetic opioids and stimulants (alone or concurrently) and polysubstance use. Measure development and validation would be enhanced by including multiple stakeholders such as, clinicians, counselors, peer specialists, and individuals with lived experience. Routine outcome measurement is not at all standard in community settings despite calls to integrate this practice across all behavioral health settings (American Psychological Association, 2009; Boswell et al., 2015). Our list of free, brief, and valid measures will help community providers who lack staff or the training to use and incorporate assessments into treatment planning. A community-partnered approach to ensure measures are feasible, acceptable, and clinically useful may improve uptake of newly developed measures. Consequently, further research on the implementation of MBC in community substance use treating settings should be a priority.

Third, future measure validation and development should employ more rigorous, comprehensive, and standardized methodology. The studies we reviewed for evidence of internal consistency almost exclusively reported Cronbach’s alpha (Cronbach, 1951). Despite the ubiquity of alpha, modern reliability parameters, such as McDonald’s Omega have been recommended for at least a decade due to the effect of measure length and multidimensionality on alpha (Cortina, 1993; Revelle and Zinbarg, 2009). Future psychometric works should report on a broader range of measure properties that go beyond reliability and validity, such as clinical utility and sensitivity to treatment with respect to interventions they would be likely to assess in practice (Hunsley and Mash, 2008). Finally, standardization of statistical terminology must be emphasized to improve comparisons of psychometric quality across measures. A substantial portion of articles reviewed for validity evidence conflated concurrent, convergent, divergent, and predictive validity or did not specify the type of validity evidence at all. Statistical rules-of-thumb that may vary from study to study must be substituted by scholarly guidelines that are used by all (Rönkkö and Cho, 2022).

The current review has a number of limitations that warrant further discussion. Although we used a search strategy that emphasized breadth, due to the vastness of the substance use disorder literature it is likely we missed relevant measures. In addition, we elected to use a non-systematic citation searching approach to collect evidence on the psychometric properties of each measure. Consequently, the reviewed studies should not be considered the complete set for each instrument. Due to the lack of a definition of what constitutes a “brief” (e.g., number of items or duration) measure and minimal reporting of the duration involved with a given measure’s administration, we relied on the number of items (i.e., ≤ 25) as a proxy for brevity although we recognize that some included 10-item measures, for example, could take as long as an excluded 30-item measure. We adapted the De Los Reyes & Langer (2018) rubric to promote comparison of psychometric evidence across measures that vary widely by length, dimensionality, and intended use. However, these adjustments come at the cost of foregoing finer details and of the potential overestimation of the strength of the psychometric support for certain measures. Lastly, by focusing our review on self-report measures we recognize that we excluded a number of widely-used, clinician-administered gold standard measures (e.g. ASI, CIDI, SCID) in the field that shown to have high utility in a number of treatment settings (Connors et al., 2016; McLellan et al., 1992). We also acknowledge the large and contentious literature regarding the concordance of self-report and biological measures and advocate for the cautious application of these instruments (Richter and Johnson, 2001; Sobell and Sobell, 1990). We hope this review promotes future research that improves the reliable administration of brief self-report measures across a range of substances and measurement constructs in order to enhance patient progress monitoring and maximize utility to clinicians.

5. Conclusions

If a clinician were search the web for a substance use measure to implement in their practice, what would they find? In this systematic review, we identified 73 brief, free, readily accessible, and psychometrically valid self-report measures that can be adopted in substance use treatment. Of note, we were unable to find a readily usable form of almost two-hundred measures. We report a paucity of data on treatment sensitivity and a scarcity of measures that had psychometric properties that met our highest standards. While we found a number of free, valid, and accessible measures, we call for a heightened focus on the development of measures that match the needs of the current public health crises in the United States and facilitate their implementation into community settings. While further investigation on the barriers to the implementation of evidence-based assessment is needed, we hope the current review will advance efforts to validate, disseminate, and implement brief self-report measures and subsequently improve the quality of substance use treatment and the lives of individuals with SUD.

Supplementary Material

1

Supplementary Table 1: Extracted Psychometrics and Sources.

Complete data set and associated sources used to categorize measures as “excellent”, “good”, or “adequate”. Includes: number of items, measure description, further information on norms, internal consistency values, test-retest reliability values and time window, treatment sensitivity information, validity evidence types, and correlation with parent measure.

2

Supplementary Table 2: Measure Links and Synopses.

Links to all measures in print-ready format and the measurement constructs in which they are applicable. We provide synopses of the evidence supporting a measure, when applicable.

Highlights:

• We conducted a systematic review of brief, valid, freely accessible measures for substance use disorder and its related constructs.

• Out of 411 measures, 73 (18%) met our inclusion criteria and had their psychometric strength evaluated.

• Among included measures, 14 (19%) were classified as psychometrically “excellent,” 27 (37%) were rated as “good,” 32 (44%) were “adequate.”

• Many areas of substance use disorder health care are lacking pragmatic, psychometrically valid measures.