Agreement between DSM-5 and DSM-IV measures of substance use disorders in a sample of adult substance users

Abstract

Background:

In DSM-5, definitions of substance use disorders (SUD) were changed considerably from DSM-IV, yet little is known about how well DSM-IV and DSM-5 SUD diagnoses agree among substance users. Because data from many studies are based on DSM-IV diagnostic criteria, understanding the agreement between DSM-5 and DSM-IV SUD diagnoses and reasons for discordance between these diagnoses is crucial for comparing results across studies.

Measurements:

Prevalences and chance-corrected agreement of DSM-5 SUD and DSM-IV substance dependence were evaluated in 588 substance users in a suburban inpatient addiction program and an urban medical center, using a semi-structured interview (PRISM-5). Alcohol, tobacco, cannabis, cocaine, heroin, opioid, sedative, and stimulant use disorders were examined. Cohen’s kappa was used to assess agreement between DSM-5 and DSM-IV SUD (abuse or dependence), DSM-5 SUD and DSM-IV dependence, and DSM-5 moderate/severe SUD and DSM-IV dependence.

Results:

Agreement between DSM-5 and DSM-IV SUD was excellent for all substances (κ = 0.84–0.99), except for cannabis and tobacco (κ = 0.75; 0.80, respectively). The most common reason for diagnostic discrepancies was a positive DSM-5 SUD diagnosis but no DSM-IV diagnosis, due to the lowered DSM-5 SUD threshold. Agreement between DSM-5 SUD and DSM-IV dependence was excellent for all substances (κ = 0.88–0.94), except for alcohol, tobacco, and cannabis (κ = 0.63–0.75). Agreement between moderate/severe DSM-5 SUD and DSM-IV dependence was excellent across all substances.

Conclusion:

While care should be used in interpreting results of studies using different methods, studies relying on DSM-IV or DSM-5 SUD diagnostic criteria offer similar information and thus can be compared when accumulating a body of evidence.

1. Introduction

Substance use and substance use disorders (SUD) are a leading cause of morbidity and mortality (Centers for Disease Control and Prevention, 2020; Kranzler and Soyka, 2018; Patnode et al., 2020; Schulte and Hser, 2014). Adults diagnosed with SUD are at increased risk of impaired functioning, psychiatric comorbidity, and low income (Grant et al., 2015, 2016; O’Brien et al., 2004). Considering the burden of disease associated with SUD in the US and globally, substance use remains a serious public health concern (GBD Alcohol and Drug Use Collaborators, 2018). To successfully identify etiologic factors and effective treatments for SUD, reliable and valid measures of SUD are a critical component of research.

In 1980, the third edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-III) was published. This was the first U.S. nomenclature to base diagnoses of substance and mental disorders on specific diagnostic criteria. DSM-III divided SUD into two disorders, abuse and dependence. Since then, successive versions of DSM have become the standard classification reference for clinical, research, policy, and reimbursement in the US and in many other countries (American Psychiatric Association Publishing, 2013; Kupfer et al., 2013; van Heugten-van der Kloet and van Heugten, 2015). This includes DSM-III-R (revised) in 1987 (American Psychiatric Association, 1987), which reorganized the SUD criteria into a concept of substance dependence (Edwards and Gross, 1976) developed through empirical clinical research and recommended by the World Health Organization (Edwards et al., 1981), and a residual category for abuse. DSM-IV (American Psychiatric Association, 1994), the next iteration, was published in 1994, and largely maintained the DSM-III-R SUD categories of dependence and abuse. Many studies supported the reliability and validity of the dependence diagnosis, but raised questions about the reliability and validity of the abuse category (Hasin et al., 1997, 2006; Shmulewitz et al., 2010) and problems due to “diagnostic orphans”, i.e., those with fewer than three dependence criteria but who failed to receive a DSM-IV substance use disorder diagnosis because they had no abuse criteria and did not meet the dependence threshold (Bartoli et al., 2015; Hasin and Paykin, 1998, 1999; Pollock and Martin, 1999; Ray et al., 2008).

Following the publication of DSM-IV, the body of knowledge on substance use disorders continued to grow, including substantial evidence indicating psychometric problems created by the hierarchical distinction between the DSM-III-R and DSM-IV abuse and dependence distinction (Hasin et al., 2006), and numerous studies demonstrating a unidimensional structure of the abuse and dependence criteria (Compton et al., 2009; Hasin et al., 2006, 2013; Saha et al., 2012). This led to recommendations to replace the abuse and dependence diagnoses with one combined SUD diagnosis (Hagman and Cohn, 2011; Hasin et al., 2012; Saha et al., 2012; Shmulewitz et al., 2010). Consequently, major and significant changes were made to the DSM-5 criteria for SUD (Hasin et al., 2013), the most recent version of DSM, published in 2013. Diagnostic changes included: 1) merging the criteria of substance abuse and substance dependence into one all-encompassing disorder; 2) requiring two criteria to be met for the diagnosis of SUD, as opposed to the DSM-IV requirements that three criteria be met for dependence and one criterion be met for the diagnosis of abuse; 3) adding craving – a strong desire or urge to use the substance – as a diagnostic criterion; 4) adding cannabis withdrawal as a new disorder to the list of other withdrawal syndromes already found in DSM-III-R and DSM-IV, and as a criterion for DSM-5 cannabis use disorder.

Since its publication, the DSM-5 SUD diagnostic criteria have been widely adopted by clinicians and researchers to assess risk factors and consequences of substance use, and to identify participants for inclusion in clinical trials of treatment effectiveness. Nevertheless, many researchers continue to rely on DSM-IV diagnoses because data from many studies, including large clinical trials (Anton et al., 2020; Brick et al., 2019; Compton et al., 2019; Hagman and Cohn, 2011; Santaella-Tenorio et al., 2019; Williams et al., 2019), genetic studies (Li et al., 2017; Walters et al., 2018) and U.S yearly national general population surveys (e.g., the National Survey on Drug Use and Health) are based on DSM-IV diagnoses of SUD (Center for Behavioral Health Statistics and Quality, 2016).

Recently, test-retest reliability of DSM-5 SUD diagnosis and severity levels (mild, moderate and severe disorder; 2–3, 4–5, and ≥6 criteria, respectively) were shown to have substantial to excellent reliability for most substances (Hasin et al., 2020). Nevertheless, little empirical information is available on the agreement of DSM-5 SUD and its severity levels with DSM-IV abuse and dependence diagnoses. The DSM-5 work group concluded that using a threshold of two or more criteria to diagnose SUD would maximize agreement on the prevalence of DSM-IV substance abuse and dependence disorders combined (American Psychiatric Association Publishing, 2013). However, only few studies have examined agreement between DSM-5 and DSM-IV diagnoses in clinical samples (Agrawal et al., 2011; Kelly et al., 2014; Kopak et al., 2014; Peer et al., 2013; Proctor et al., 2012). These studies, while important, were limited by assessing only lifetime and not current SUD. Further, one of the studies examined only alcohol use disorder (AUD), while another focused on cocaine use disorder and used a subpopulation of prison inmates (Kopak et al., 2014).

Given the widespread use of DSM-5 SUD diagnoses in many studies alongside continued publication of studies based on DSM-IV SUD diagnoses, more information is needed on the extent to which diagnoses based on the DSM-5 diagnostic criteria are concordant with diagnoses based on DSM-IV dependence. In particular, since the DSM-IV abuse category had weaker reliability and validity evidence than dependence, information is needed on whether the low threshold for DSM-5 SUD (two criteria) leads to inclusion of cases who would formerly have received an abuse diagnosis, potentially capturing a group of individuals with less diagnostic validity than a group formed by using a higher threshold more analogous to the DSM-IV dependence diagnosis. Therefore, using the Psychiatric Research Interview for Substance and Mental Disorders (PRISM-5), which has been shown to be a reliable instrument to measure DSM-5 SUD (Hasin et al., 2020), the current study aimed to determine the agreement between: 1) DSM-5 and DSM-IV past-year substance use disorder diagnoses; 2) DSM-5 past-year substance use disorder and DSM-IV past-year dependence diagnoses; and 3) DSM-5 past-year moderate or severe substance use disorder and DSM-IV past-year dependence diagnoses. In addition, reasons for discrepancies between DSM-IV and DSM-5 SUD diagnoses were explored.

2. Methods

2.1. Study Population and Procedures

As detailed elsewhere (Hasin et al., 2020), study participants were adults aged 18 years or older recruited from a suburban inpatient addiction program and from a large urban medical center using newspaper and social media advertising. To be eligible, participants needed to report substance use in the prior 30 days or the 30 days prior to inpatient admission and endorse at least one DSM-5 SUD criterion in pre-study screening. Of those eligible, on-site research coordinators described the study and obtained informed consent from eligible participants. Baseline interviews were conducted with 588 participants (150 inpatients; 438 community participants) that constituted the analytic sample. Participants received $50 for completion of the interview. All interviews were conducted between 05/11/2016 and 06/17/2019. Procedures were approved by Institutional Review Boards of New York State Psychiatric Institute and South Oaks Hospital.

2.2. Diagnostic interview

The PRISM-5 interview is a semi-structured, computer-assisted interview designed for clinician interviewers, which covers the DSM-IV and DSM-5 symptoms and criteria of substance and psychiatric disorders. PRISM-5 differs from other diagnostic interviews by assessing substance disorders first, and by providing more detailed symptom data. Substances covered by the PRISM-5 include alcohol, cannabis, cocaine, hallucinogens, heroin, opioid painkillers, sedatives/tranquilizers, stimulants, and tobacco (cigarettes). In this study, we included all substances except for hallucinogens, for which DSM-IV dependence and DSM-5 moderate/severe SUD showed low prevalence (≤2%).

2.3. PRISM-5 substance screening

The substance disorder module begins with screening questions regarding non-medical use (without a prescription or other than prescribed, for example, to get high) of each substance 6 or more times within any 12-month period. Among those screened positive, the SUD criteria are assessed.

2.3.1. PRISM-5 substance use disorder measures

For each substance, current DSM-5 SUD was considered positive if participants endorsed ≥2 criteria of the 11 criteria in the past 12 months. A second variable indicated the DSM-5 SUD severity measure: no disorder (0–1 criteria), mild (2–3 criteria), moderate (4–5 criteria), or severe (≥6 criteria). A third DSM-5 variable was constructed, which was positive for those with moderate or severe DSM-5 SUD and negative for those with no or mild disorder. DSM-IV current substance dependence was diagnosed when ≥3 of the DSM-IV criteria were endorsed within the past 12 months. A second binary variable was constructed to indicate DSM-IV substance use disorder, coded positive if participants endorsed dependence or abuse (based on endorsing ≥1 abuse criteria in the past 12 months).

2.3.2. Sociodemographic variables

These were collected in an introductory PRISM-5 module. They included: sex (male; female), race/ethnicity (White; Black; Hispanic; other), age (18–29; 30–39; 40–49; 50+), marital status (not married; living together/married), education (high-school diploma or less; some college or more), and employment (unemployed; employed full- or part-time).

2.4. Interviewers, training and supervision

All interviewers had at least a master’s degree in psychology or social work and an average of 4.5 years of clinical experience (range, 1–10 years). PRISM-5 training included a manual, 2-day workshop, practice interviewing, role-playing, and certification. To become certified, trainees recorded mock interviews that underwent structured review by PRISM trainers. Trainees became interviewers after 2 recordings were rated satisfactory or better. Supervision of the study interviewers was conducted by trained, highly experienced trainer/supervisors with clinical masters degrees (psychology or social work) whose mean years of clinical experience utilizing the PRISM interview in research settings was 7.6 years (range: 3–10), and whose mean years of supervisory experience with the PRISM-5 was 6 years (range: 2–8). After certification, PRISM-5 supervisors met weekly with interviewers. They also reviewed recordings of 10 % of the interviews, scoring interviewer performance, which occasionally indicated typical issues in such interviewing that required supervision (e.g., reading probes as written) but generally indicated that PRISM-5 interviews were conducted in a standardized way according to the PRISM-5 training procedures. Further information about the PRISM-5 and procedures used in this study is provided elsewhere (Hasin et al., 2020).

2.5. Analysis

Three analyses were conducted to determine the degree of agreement between substance-specific past-year DSM-5 and DSM-IV SUD measures. The first analysis compared DSM-5 with DSM-IV substance use disorder diagnoses. The second compared DSM-5 substance use disorder with DSM-IV past-year substance dependence diagnoses. The third compared DSM-5 moderate or severe substance use disorder with DSM-IV past-year substance dependence diagnoses. McNemar’s test evaluated whether prevalences differed between the DSM-5 and DSM-IV measures. Cohen’s kappa, a statistical measurement of chance-corrected agreement (McHugh, 2012), was calculated for each analysis. The following standard interpretations were used for the degree of agreement indicated by kappa values: 0–0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, excellent (Landis and Koch, 1977). Further analyses were conducted to determine the proportion of participants in the entire sample with discordant substance-specific DSM-5 and DSM-IV SUD diagnoses, and among participants with discordant diagnoses, potential reasons for these discrepancies; specifically, whether they stemmed from the lower DSM-5 SUD threshold or from the addition of new diagnostic criteria. All analyses were conducted using SAS 9.4 (SAS Institute Inc, Copyrigh ©., 2013).

3. Results

3.1. Sociodemographic characteristics of the sample

Respondents were primarily male (70 %), age 50 years or above (42 %), black (48 %), with a high school diploma or less (55 %), unmarried (80 %), and unemployed (74 %) (Supplementary Table 1).

3.2. DSM-5 and DSM-IV Substance use disorder

Table 1 shows prevalences of DSM-5 SUD and DSM-IV SUD (indicating a diagnosis of dependence or abuse). In DSM-5, prevalences of AUD, TUD and CUD were 66.0 %, 62.1 %, 44.6 %, while corresponding prevalences in DSM-IV for these disorders were 59.4 %, 52.2 % and 33.2 %. DSM-5 prevalences of cocaine, heroin and prescription opioid use disorders were 44.7 %, 24.1 % and 15.6 %, while DSM-IV prevalences were 40.5 %, 23.6 % and 14.6 %. As indicated by the McNemar tests, the prevalences of DSM-5 SUD and DSM-IV SUD differed for all substances except heroin and stimulants. Agreement between DSM-5 and DSM-IV SUD was excellent for alcohol, cocaine, heroin, opioids, sedatives, and stimulants (κ = 0.84–0.99) and substantial for cannabis and tobacco (κ = 0.75; 0.80, respectively; Table 1).

Table 1.

Agreement between PRISM-5 DSM-5 and DSM-IV past-year substance use disorder diagnoses (N = 588).

Substance	% DSM-5	% DSM- IV	McNemar’s testa p-value	Kappab (95% CI)
Alcohol	66.0	59.4	<.001	0.84 (0.79, 0.88)
Tobacco	62.1	52.2	<.001	0.80 (0.75, 0.85)
Cannabis	44.6	33.2	<.001	0.75 (0.70, 0.80)
Cannabis without withdrawal	43.7	33.2	<.001	0.76 (0.71, 0.81)
Cannabis without craving	39.5	33.2	<.001	0.78 (0.73, 0.84)
Cannabis without both withdrawal and craving	39.0	33.2	<.001	0.83 (0.78, 0.88)
Cocaine	44.7	40.5	<.001	0.91 (0.87, 0.94)
Heroin	24.1	23.6	0.083	0.99 (0.97, 1.00)
Opioids	15.6	14.6	0.014	0.96 (0.93, 0.99)
Sedatives	12.8	11.4	0.011	0.92 (0.87, 0.97)
Stimulants	2.9	2.7	0.317	0.97 (0.91, 1.00)

^aMcNemar’s test evaluated whether there were significant differences between DSM-IV and DSM-5 SUD prevalences.

^bDegree of agreement, indicated by kappa values: 0–0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, excellent.

3.3. DSM-5 substance use disorder and DSM-IV dependence

Table 2 shows prevalences of DSM-5 SUD and DSM-IV substance dependence. The prevalences of DSM-IV dependence diagnoses were 53.7 %, 46.4 %, 26.9 %, 39.1 %, 23.0 %, and 13.1 % for alcohol, tobacco, cannabis, cocaine, heroin, and prescription opioid, respectively (Table 2). Except for stimulants, dependence prevalence significantly differed from DSM-5 SUD prevalence, as indicated by the McNemar tests. Agreement between DSM-5 SUD and DSM-IV substance dependence was excellent for cocaine, heroin, opioids, sedatives, and stimulants (κ = 0.88–0.97) and substantial for alcohol, tobacco, and cannabis (κ = 0.63–0.75; Table 2).

Table 2.

Agreement between PRISM-5 DSM-5 past-year substance use disorder and DSM-IV past-year substance dependence diagnoses (N = 588).

Substance	% DSM-5	% DSM-IV	McNemar’s testa p-value	Kappab (95% CI)
Alcohol	66.0	53.7	<.001	0.75 (0.70, 0.80)
Tobacco	62.1	46.4	<.001	0.69 (0.64, 0.75)
Cannabis	44.6	26.9	<.001	0.63 (0.57, 0.69)
Cocaine	44.7	39.1	<.001	0.89 (0.85, 0.92)
Heroin	24.1	23.0	<.01	0.97 (0.94, 0.99)
Opioids	15.6	13.1	<.001	0.90 (0.85, 0.95)
Sedatives	12.8	10.4	<.001	0.88 (0.82, 0.94)
Stimulants	2.9	2.6	0.157	0.94 (0.85, 1.00)

^aMcNemar’s test evaluated whether there were significant differences between DSM-IV dependence and DSM-5 SUD prevalences.

^bDegree of agreement, indicated by kappa values: 0–0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, excellent.

3.4. DSM-5 moderate or severe substance use disorder and DSM-IV dependence

Table 3 shows prevalences of combined moderate and severe levels of DSM-5 SUD and DSM-IV substance dependence. Differences in prevalence of DSM-5 moderate-severe SUD and DSM-IV substance dependence were negligible for most substances except for cannabis use disorder, with 30.8 % for moderate-severe DSM-5 SUD and 26.9 % for DSM-IV dependence (p < .001; Table 3). Agreement between moderate and severe DSM-5 SUD and DSM-IV dependence diagnosis was excellent across all substances (κ = 0.86–0.96), and near-perfect for cocaine and heroin (κ = 0.97; 0.98, respectively; Table 3).

Table 3.

Agreement between PRISM-5 DSM-5 past-year moderate or severe substance use disorder and DSM-IV past-year substance dependence diagnoses (N = 588).

Substance	% DSM- 5	% DSM- IV	McNemar’s testa p-value	Kappab (95% CI)
Alcohol	53.7	53.7	1.000	0.92 (0.89, 0.96)
Tobacco	47.4	46.4	0.180	0.93 (0.90, 0.96)
Cannabis	30.8	26.9	<.001	0.86 (0.81, 0.90)
Cocaine	38.9	39.1	0.705	0.97 (0.96, 0.99)
Heroin	23.0	23.0	1.000	0.98 (0.96, 1.00)
Opioids	12.6	13.1	0.317	0.93 (0.89, 0.98)
Sedatives	10.4	10.4	1.000	0.96 (0.93, 1.00)
Stimulants	2.4	2.6	0.317	0.96 (0.90, 1.00)

^aMcNemar’s test evaluated whether there were significant differences between DSM-IV dependence and DSM-5 moderate or severe SUD prevalences.

^bDegree of agreement, indicated by kappa values: 0–0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, excellent.

3.5. DSM-5 and DSM-IV SUD diagnoses discrepancies

Table 4 shows prevalences of participants diagnosed with DSM-5 SUD but not DSM-IV SUD and vice versa, as well as reasons for these discrepancies. Among the entire sample, between 0.2 %–11.7 % of participants met the DSM-5 diagnostic criteria for SUD but did not meet the DSM-IV diagnostic criteria for SUD (Table 4a). Across all substances, a large proportion of these discrepancies were due to the lowered DSM-5 SUD threshold (alcohol − 76.2 %; tobacco − 72.4 %; cannabis, 58.0 %; cocaine, 80.8 %; heroin, 66.7 %; opioids, 83.3 %; sedatives, 100.0 %; stimulants, 100.0 %, respectively). Conversely, 0 %–42 % of these discrepancies were due to newly added criteria to the DSM-5 SUD diagnosis.

Table 4.

Participants with discrepant diagnoses, N = 588.

4a. DSM-5 SUD positive, DSM-IV SUD negative
	% of total sample	% among those discrepant
Alcohol (n = 42)	7.1
Only 2 DSM-IV dependence criteria (no DSM-IV abuse criteria)		76.2
1 DSM-IV dependence criterion (no abuse) and craving		23.8
Tobacco (n = 58)	9.9
Only 2 DSM-IV dependence criteria (no abuse)		72.4
0–1 DSM-IV dependence criterion (no abuse) with craving, hazardous use, social problems, or neglecting roles		27.6
Cannabis (n = 69)	11.7
Only 2 DSM-IV dependence criteria (no abuse)		58.0
0–1 DSM-IV dependence criterion (no abuse) and craving or withdrawal		42.0
Cocaine (n = 26)	4.4
Only 2 DSM-IV dependence criteria (no abuse)		80.8
1 DSM-IV dependence criterion (no abuse) and craving		19.2
Heroin (n = 3)	0.5
Only 2 DSM-IV dependence criteria (no abuse)		66.7
1 DSM-IV dependence criterion (no abuse) and craving		33.3
Opioids (n = 6)	1.0
Only 2 DSM-IV dependence criteria (no abuse)		83.3
1 DSM-IV dependence criterion (no abuse) and craving		16.7
Sedatives (n = 9)	1.5
Only 2 DSM-IV dependence criteria (no abuse)		100.0
Stimulants (n = 1)	0.2
Only 2 DSM-IV dependence criteria (no abuse)		100.0
4b. DSM-IV SUD positive, DSM-5 SUD negative
	% of total sample	% among those discrepant
Alcohol (n = 3)	0.5
Only one DSM-IV abuse criterion (no dependence), without craving		100.0
Cannabis (n = 2)	0.3
Only one DSM-IV abuse criterion (no dependence), without craving or withdrawal		100.0
Cocaine (n = 1)	0.2
Only one DSM-IV abuse criterion (no dependence) without craving		100.0
Sedatives (n = 1)	0.2
Only one DSM-IV abuse criterion (no dependence), without craving		100.0

Note: no participant met the DSM-IV diagnostic criteria but did not meet the DSM-5 diagnostic criteria for tobacco, heroin, opioid, and stimulant use disorders.

Among the entire sample (Table 4b), almost all participants who met the DSM-IV SUD diagnosis also met the DSM-5 diagnostic criteria for SUD. All discrepant cases stemmed from participants endorsing only one DSM-IV abuse criterion and not endorsing any newly added DSM-5 SUD criteria; no discrepant cases were because participant only endorsed the DSM-IV legal problems criterion (which was excluded from DSM-5).

4. Discussion

In recent years, the DSM-5 SUD diagnostic criteria have been used extensively by researchers and clinicians worldwide. However, DSM-IV continues to be used in numerous large, important studies, for example, ongoing analyses of large existing genetic or clinical trial datasets, or in the yearly national surveys of the National Drug Use and Health Survey (NSDUH). Therefore, determining differences in prevalence and level of agreement between DSM-5 and DSM-IV SUD diagnostic systems is important to inform investigators and others about whether results from studies using DSM-5 or DSM-IV SUD are comparable. Drawing on data from several hundred participants that utilized rigorous assessment procedures, we examined prevalences and chance-corrected agreement between DSM-5 and DSM-IV SUD diagnoses for eight commonly used substances.

We found that the prevalence of DSM-5 SUD was slightly higher than the prevalence of DSM-IV SUD for all substances, with differences ranging from 4.2 %–11.4 %. Further, for all substances, differences in prevalence between DSM-5 SUD and DSM-IV dependence were consistently greater than the differences in prevalence between DSM-5 and DSM-IV SUD prevalence. Together, these findings indicate that the higher rates of DSM-5 SUD stem primarily from the cases who were diagnostic “orphans” in DSM-IV (those with two dependence criteria and no abuse criteria) that DSM-5 classified as positive (most notably for alcohol, tobacco, cannabis and cocaine). Diagnostic “orphans” accounted for 58 %–100 % of these differences across the various substances. These findings are generally aligned with those from previous studies in which use of DSM-5 SUD diagnostic criteria produced slightly greater prevalences than those produced by DSM-IV (Agrawal et al., 2011; Bartoli et al., 2015; Kelly et al., 2014; Kopak et al., 2014; Peer et al., 2013). Nevertheless, one study found slightly lower prevalences using DSM-5 SUD diagnostic criteria compared to when using DSM-IV (Proctor et al., 2012), while another study found a substantially greater prevalence of AUD when using DSM-5 diagnostic criteria (Mewton et al., 2011). This last study was conducted several years prior to publication of DSM-5; therefore, it did not use the final version of the DSM-5 AUD diagnostic criteria, which may explain the substantially larger difference reported between DSM-5 and DSM-IV AUD prevalences.

Regardless of diagnostic threshold, DSM-5 and DSM-IV SUD diagnoses as well as DSM-5 SUD and DSM-IV dependence had substantial to excellent levels of agreement with each other for all substances, with kappas ranging from 0.63 to 0.99, and 0.63 to 0.97, respectively. These levels of agreement are consistent with those found in 3 previous studies that reported Cohen’s Kappa (Kelly et al., 2014; Mewton et al., 2011; Peer et al., 2013). Together, these 3 studies examined a relatively wide range of substances but did not include heroin, stimulants, or sedatives. Further, none of these studies examined levels of agreement between DSM-IV and DSM-5 SUD in a clinical sample of substance users. Despite the differences between the current study and previous studies in terms of their sample population (e.g., adults vs. adolescents, clinical vs. non-clinical) and design (e.g., past-year vs. lifetime timeframes), the similar levels of agreement between DSM-IV and DSM-5 SUD point to the robustness of the current findings. To our knowledge, this study is the first to report agreement between DSM-5 and DSM-IV heroin, sedative, and stimulant use disorders in adults (κ = 0.99; 0.92; 0.97, respectively).

Differences in prevalences of DSM-5 moderate-severe SUD (four or more criteria) and DSM-IV dependence were negligible. Accordingly, agreement between DSM-5 and DSM-IV was stronger (near-perfect for many substances) for this set of comparison than for others. This finding suggests that these two diagnoses may indicate similar clinical conditions; investigators utilizing information from studies that employ one diagnostic classification system or the other can have confidence in the comparability of these studies’ results.

Questions could be raised about the research and clinical utility of the DSM-5 SUD mild severity level given the present findings of greater agreement between DSM-5 moderate-severe SUD and DSM-IV dependence than other comparisons. Substance users that endorse criteria for mild SUD may be recognized less frequently by clinicians but may still benefit from interventions that are appropriate for their lower severity level, e.g., the SBIRT model (Screening, Brief Intervention, Referral to Treatment (Babor et al., 2007), which has been shown to be effective particularly for alcohol (Saitz, 2010). Longitudinal studies that examine whether brief preventive and therapeutic interventions in primary care settings mitigate the development of more severe SUD among individuals with mild DSM-5 SUD are warranted, as these may have important individual and public health consequences.

Across the comparisons performed in this study, level of agreement between DSM-5 and DSM-IV for CUD was slightly lower (k = 0.75) than for other substances. DSM-5 added only one new criterion across most substances (craving) but added two for cannabis use disorder: craving and withdrawal. Therefore, we explored the effects of these two criteria on agreement between DSM-5 and DSM-IV CUD (Table 1). First, we removed withdrawal from the DSM-5 CUD variable, leaving 10 criteria instead of 11 and re-ran the kappa. This made little change in level of agreement between DSM-5 CUD and DSM-IV CUD (k = 0.76). We then replaced withdrawal but removed craving from the DSM-5 CUD variable. This also made little difference in agreement (k = 0.78). We then removed both withdrawal and craving from the DSM-5 CUD diagnosis and found that this moved agreement into the excellent range (k = 0.83). Altogether, these findings suggest that for cannabis use disorder, additional care should be taken when comparing results between DSM-5 and DSM-IV. Further studies should examine if these differences lead to differential validity of DSM-IV and DSM-5 CUD.

Study limitations are noted. First, the sample included adults treated for substance use problems in inpatient settings and a community sample recruited via newspaper and social media advertising for assessment at an urban medical center. Additional studies that examine agreement for DSM-5 and DSM-IV SUD in adolescents, in patients treated in mental health and primary care settings, and in representative general population samples are warranted. Second, analyses did not include some substances, such as hallucinogens, as prevalences for these were low. Studies that report agreement between DSM-5 and DSM-IV SUD diagnoses for these substances are needed. Finally, the current study utilized self-reported measures of substance use, which could have led to reporting inaccuracies. Despite these limitations, this study had several important strengths. First, the sample used in this study was relatively large and demographically diverse; it included substance users with a wide range of SUD severity levels and from various treatment settings. Second, it included adequate representation of different substances. Finally, it employed rigorous study methods and reliable measures (Hasin et al., 2020).

5. Conclusion

The current study assessed the agreement between DSM-5 and DSM-IV SUD, using several comparison models, in which measurements were operationalized differently. In all comparisons performed and across all substances, agreements between DSM-5 and DSM-IV diagnoses were substantial to excellent. Further, excellent agreement between moderate or severe levels of DSM-5 SUD and DSM-IV dependence, a highly reliable measure, indicate that these specific DSM-5 SUD severity levels had even better agreement with DSM-IV dependence measures than DSM-5 SUD measures that included mild cases. Substance use and SUD remain a major public health concern. Use of reliable and valid diagnostic measures of these disorders is crucial if studies are to generate informative results on etiology, prevention and treatment. This study suggests that while care should always be used in interpreting the results of studies using different diagnostic modalities, studies relying on DSM-IV or DSM-5 SUD diagnostic criteria can be considered to offer similar information and thus can be compared when accumulating a body of evidence.