Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Nov 1.
Published in final edited form as: Drug Alcohol Depend. 2021 Sep 21;228:109056. doi: 10.1016/j.drugalcdep.2021.109056

Test-retest reliability and cross-cultural applicability of DSM-5 adopted diagnostic criteria for ketamine use disorders

Nicole D Fitzgerald a,*, Catherine W Striley a, Joseph J Palamar b, Jan Copeland c, Steven Kurtz d, Linda B Cottler a
PMCID: PMC8678918  NIHMSID: NIHMS1762208  PMID: 34592704

Abstract

Background:

Despite increasing prevalence of nonmedical ketamine use globally, data on ketamine use disorders, which are classified in the DSM-5 under criteria for phencyclidine, are limited. This study assessed the reliability and applicability of DSM-based diagnostic criteria for ketamine use disorder.

Methods:

Participants who used ecstasy were recruited through the Tri-City Study of Club Drug Use, Abuse, and Dependence in St. Louis, Miami, and Sydney. Those who reported using ketamine (lifetime use >5 times) were included in these analyses (n = 205). Participants were interviewed using the computerized Substance Abuse Module for Club Drugs (CD-SAM) at baseline and 7 days later for the reliability of diagnoses and individual diagnostic criteria.

Results:

Overall, 29.3% met DSM-5 adopted criteria for ketamine use disorder at Time 1. Moderate to excellent test-retest reliability was observed consistently across study sites for any ketamine use disorder (κ = 0.57, Y = 0.61) and severe ketamine use disorder (κ = 0.62, Y = 0.79). Continued use of ketamine despite knowledge of physical or psychological problems was the most frequently endorsed individual criterion (59.0%), followed by reported withdrawal (30.2%) and physically hazardous use (29.8%). All individual criteria had acceptable reliability estimates (κ ≥ 0.41).

Conclusions:

Diagnoses of ketamine use disorder can be reliably evaluated using this fully structured diagnostic instrument’s questions and algorithm. Ketamine-related withdrawal among people who use ketamine should be re-evaluated. Considering that after-effects of this dissociative anesthetic can last for many hours, it is important to explore a different timeframe for possible withdrawal effects.

Keywords: Ketamine, Reliability, Test-retest, Nosology, Epidemiology, DSM-5

1. Introduction

Ketamine, a derivative of phencyclidine (PCP), is a dissociative drug with anesthetic, analgesic, and psychedelic properties (Huang and Lin, 2020). In recent years, the nonmedical use of ketamine has become more prevalent globally, particularly in East and South-East Asia (Huang and Lin, 2020; Kalsi et al., 2011; Rao et al., 2015; Sassano-Higgins et al., 2016). While lifetime use of ketamine in the United States is estimated to be 1.5% among the general adult population (Center for Behavioral Health Statistics and Quality, 2020), prevalence of use is higher among certain populations such as electronic dance music party attendees. For example, one New York City-based study found that past 12-month ketamine use increased from 5.9% in 2016 to 15.3% in 2019 among party attendees (Palamar and Keyes, 2020). Ketamine has also received considerable media coverage over the past few years following findings of its potential effectiveness for treating depression and other psychiatric disorders (Na and Kim, 2020; Nowacka and Borczyk, 2019; Sanacora et al., 2017) that could further influence the prevalence of nonmedical ketamine use (Palamar and Le, 2021).

Ketamine use disorder is recognized and assessed in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) under the diagnostic criteria for PCP use disorder (American Psychiatric Association, 2013). While epidemiological data on ketamine use disorders are needed to plan treatment and prevention programs, they are scarce (Jansen and Darracot-Cankovic, 2001; Kalsi et al., 2011; Morgan et al., 2012). Only a handful of studies (Fernández-Calderón et al., 2016; Tang et al., 2013, 2015; Tung et al., 2014; Uosukainen et al., 2015; Winstock et al., 2012) have measured ketamine dependence, and of those, only three used DSM-based criteria. Tung et al. (2014) found that over two thirds (68%) of treatment-seeking ketamine users met criteria for DSM-IV ketamine dependence based on a structured interview with a psychiatrist. Winstock et al. (2012) found that 17% of persons out-of-treatment who used ketamine met three or more DSM-IV dependence criteria using an online interview methodology. In a later study using the same methodology, approximately the same percentage (15.6%) of self-reported ketamine users met three or more DSM-IV dependence criteria, which also included withdrawal relief (Uosukainen et al., 2015). However, investigators did not administer a complete structured diagnostic interview, limiting the results.

Although these prior studies show preliminary evidence for ketamine use disorder, criteria appear to have been applied inconsistently. Perhaps most importantly, these data would best be based on valid and reliable classifications in order to be most clinically useful. This requires testing the reliability of diagnoses and their cross-cultural applicability. To our knowledge, no study has yet examined the test-retest reliability of DSM-based diagnoses for persons who use ketamine, or assessed how reliable these diagnoses are for individuals across populations. We had the opportunity to assess DSM-5-based ketamine use disorders among community-recruited individuals who use ketamine, applying a structured diagnostic instrument, and to examine the test-retest reliability and cross-cultural applicability of DSM-5 adopted diagnostic criteria using data from three geographically diverse sites.

2. Material and methods

2.1. Sample

This analysis was based on data collected as part of the NIDA-funded Tri-City Study of Club Drug Use, Abuse, and Dependence, a multi-site epidemiological study designed to examine the test-retest reliability of DSM-IV adopted criteria for “club drugs” (ecstasy [MDMA], ketamine, gamma-hydroxybutyrate [GHB], and flunitrazepam [Rohypnol]). The study was conducted in St. Louis, Missouri, Miami, Florida, and Sydney, Australia from 2002 to 2005. Participants were eligible if they reported ecstasy use more than five times in their lifetime with at least one use occurring in the past 12 months, a selection threshold which has been used in large-scale epidemiological studies (Anthony and Helzer, 1991; Cottler et al., 1995; Halkitis et al., 2007). Target sampling plans, described elsewhere (Leung et al., 2010), were employed to systematically recruit participants from the three communities through flyers, internet postings, posters at universities and high schools, street and nightclub outreach, and announcements in local newspapers. All study protocols were approved by the Institutional Review Board (IRB) at Washington University School of Medicine and at each of the participating institutions.

2.2. Assessments and procedures

Assessments included the Substance Abuse Module for Club Drugs (CD-SAM) and the Washington University Risk Behavior Assessment for Club Drugs (WU-RBA-CD). The fully structured CD-SAM was an expanded version of what was then the Composite International Diagnostic Interview–Substance Abuse Module (CIDI-SAM) (Compton et al., 1996; Horton et al., 2000) now adopted for DSM-IV criteria for club drugs, assessed individually (Cottler et al., 2001). Similar to the original CIDI-SAM, the CD-SAM captured information on the use of a wide range of psychoactive substances, including substance-specific onset and recency of use, withdrawal symptoms, medical, physical, and psychological consequences of use, and DSM adopted diagnostic criteria. The original CIDI-SAM has been found to have excellent reliability, with an average kappa (κ) of 0.84 for substance use disorder diagnoses (Cottler et al., 1989), and previous studies have shown good test-retest agreement for the CD-SAM in measuring ecstasy abuse and dependence using DSM-IV adopted criteria (Cottler et al., 2009, 2001).

The DSM-based diagnostic algorithm originally included meeting at least three of seven dependence criteria in a 12-month period for DSM-IV dependence, or at least one DSM-IV abuse criterion. Ketamine withdrawal, which is not recognized as a criterion for PCP disorders in the DSM-IV or DSM-5, was assessed and required the endorsement of at least three of 19 symptoms compiled from all drug-specific withdrawal symptoms in the DSM, or the endorsement of withdrawal relief. To rescore these DSM-IV adopted criteria for the DSM-5, the legal problems abuse criterion was dropped, and a craving criterion was added from an existing item that assessed craving for ketamine after several hours or days of not using ketamine (used previously for ICD-10). With regard to the DSM-5, a ketamine use disorder included meeting two or more of 11 adopted criteria, with at least two being met in the same 12-month period. Severity levels were also assessed using DSM-5 nomenclature (American Psychiatric Association, 2013): a mild ketamine disorder was characterized by meeting two to three criteria, a moderate ketamine disorder was characterized by meeting four to five criteria, and a severe ketamine disorder was characterized by meeting six to 11 criteria.

The WU-RBA-CD is a computerized assessment adopted from NIDA’s Risk Behavior Assessment (Coyle, 1998; Dowling-Guyer et al., 1994). The revised version, based on information gathered in focus groups of persons who used club drugs, included questions related to patterns of drug use, simultaneous use of other drugs, and contextual factors of club drug use. For the purpose of these analyses, questions regarding co-use of ketamine, motivations for ketamine use, places of ketamine consumption, and simultaneous polysubstance use with ketamine were examined to provide additional context for CD-SAM diagnoses.

The computerized CD-SAM and WU-RBA-CD were administered in-person at both the baseline (Time 1) and retest (Time 2) interviews. To minimize bias, baseline and retest interviews (conducted 5–7 days later) were conducted with independent interviewers, with the second interviewer blinded to baseline responses. All interviewers were graduate-level students who completed intensive training and certification (provided by LBC and CWS) prior to administering the assessments. Participants were compensated $15 USD after the baseline and $40 USD after the retest for their time and effort.

2.4. Analyses

Data analyses were conducted using SAS version 9.4 (SAS Institute Inc., Cary NC). Descriptive analyses examined characteristics of participants who reported ketamine use according to diagnosis. To determine agreement between the two interviews, κ (Cohen, 1960) and Yule’s Y (Yule, 1912) were calculated. While both κ and Y correct for chance agreement, Y is typically preferred when prevalence is extremely high or low (Spitznagel and Helzer, 1985); however, both statistics tend to demonstrate high agreement for symptom/criteria prevalence between 20% and 80%. κ ranges from −1.00 (total disagreement) to 1.00 (total agreement). Generally, agreement with values of 0.61–1.00 is considered excellent, with values of 0.41–0.60 is considered moderate, with values of 0.21–0.40 is considered fair, and with values 0.20 or lower is considered poor (Landis and Koch, 1977). Homogeneity of κ values (H0: κ1 = κ2 = κ3 = κ) was examined using chi-square (Donner et al., 1996; Fleiss et al., 2013). κ was calculated for a ketamine use disorder diagnosis if no significant differences were found in κ values by each study site.

3. Results

3.1. Sample description

Among the 640 participants enrolled in the Tri-City Study, 216 participants reported using ketamine more than five times at the baseline interview and thus were asked the questions from the CD-SAM. Eleven participants who were missing a retest interview were excluded from analysis, yielding a final sample size of 205 (32%)––90 participants in St. Louis, 68 in Miami, and 47 in Sydney. Sample characteristics are presented in Table 1. The mean age was 23.5 years (SD=4.9, range 18–47) and the majority of participants were male (68.8%), identified as non-Hispanic White (68.3%), were never married (93.7%), and had completed at least some college education (61.5%). All participants reported using ecstasy more than five times in their lifetime with at least one use in the past 12 months, per the study’s eligibility criteria, and the majority also reported simultaneous polysubstance use: the most frequent substances co-used with ketamine on at least one occasion in participants’ lifetimes were ecstasy (65.7%), cannabis (58.3%), and alcohol (43.1%), followed by amphetamine or other stimulants (24.5%) and LSD, mushrooms, or other psychedelics (21.1%). The majority of participants used ketamine by either snorting or sniffing (92.7%). Participants reported using ketamine primarily at a personal residence (91.2%) or at a “rave” (66.2%), and most used ketamine with others, including roommates, co-workers, or friends (94.6%) or their spouse or partner (58.8%), while 31.4% of participants reported using ketamine alone. The most commonly cited motivations for using ketamine were out of curiosity (90.7%), for no reason (66.7%), and to bond with friends (43.6%).

Table 1.

Characteristics of participants who used ketamine (n = 205) by DSM-5 past 12-month ketamine use disorder diagnosis at baseline interview.

Total (n = 205) Ketamine use, no use disorder (n = 145) Ketamine use, use disorder (n = 60) p
Study site, % .115
St. Louis, Missouri 43.9 48.3 33.3
Miami, Florida 33.2 31.7 36.7
Sydney, Australia 22.9 20.0 30.0
Age, mean (SD)
Age at interview 23.5 (4.9) 23.8 (4.7) 22.8 (5.3) .165
Age at first ketamine use 19.9 (4.1) 19.8 (3.6) 20.2 (5.2) .646
Age at onset of heaviest ketamine use 20.8 (4.3) 20.7 (4.0) 21.0 (5.0) .670
Male, % 68.8 69.0 68.3 .929
Race/ethnicity, % .925
Non-Hispanic White 68.3 67.6 70.0
Hispanic/Latino 16.1 15.2 18.3
Non-Hispanic Asian 3.9 4.1 3.3
Non-Hispanic Black/African-American 3.4 4.1 1.7
Biracial or Multiracial 3.4 4.1 1.7
Othera 4.9 4.9 5.1
Marital status, % .894
Married 3.4 3.5 3.3
Divorced or separated 2.9 3.5 1.7
Never married 93.7 93.1 95.0
Education, % .473
<High school 15.6 14.5 18.3
High school 22.9 21.4 26.7
≥College 61.5 64.1 55.0
Routes of ketamine administration, %
Snorted or sniffed 92.7 92.4 93.3 .818
Oral 19.5 18.6 21.7 .617
Intramuscular 8.8 11.0 3.3 .076
Smoked 5.4 4.1 8.3 .305
Intravenous 4.9 4.8 5.0 .958
Ever used other substance together with ketamine, %
Ecstasy/MDMA 65.7 64.1 69.5 .465
Cannabis 58.3 53.8 69.5 .039
Alcohol 43.1 37.2 57.6 .008
Amphetamine or other stimulants 24.5 21.4 32.2 .103
LSD, mushrooms, or other hallucinogens 21.1 22.1 18.6 .587
Cocaine or crack 14.7 12.4 20.3 .147
Nitrous oxide or other inhalants 9.8 9.7 10.2 .911
Sedatives or tranquilizers 9.8 6.9 17.0 .029
GHB 6.9 5.5 10.2 .236
Places used ketamine, %
At a house, apartment, or dorm 91.2 90.3 93.2 .512
At a rave 66.2 63.5 72.9 .197
At a bar or club 53.4 48.3 66.1 .021
At a beach, park, or other public place 37.8 35.9 42.4 .384
Used ketamine with others, %
With roommate, co-worker, or friend 94.6 92.4 100.0 .036
With spouse or partner 58.8 53.1 72.9 .009
With a stranger 40.2 37.9 45.8 .301
With a dealer, not mentioned 33.8 31.0 40.7 .187
Alone 31.4 31.7 30.5 .865
Motivations for using ketamine, %
Out of curiosity 90.7 91.0 89.8 .789
For no reason 66.7 63.5 74.6 .126
To bond with friends 43.6 37.9 57.6 .010
To numb mind or forget problems 32.4 28.3 42.4 .051
To have a spiritual experience 27.9 24.8 35.6 .120
To get more in touch with self 21.6 20.0 25.4 .393
To relieve stress 20.6 15.9 32.2 .009
Pressured by others 15.2 11.7 23.7 .030
Open in a new tab
a

Other includes Alaskan Native, American Indian, Middle Eastern, and other race/ethnicity.

SD: standard deviation; MDMA: 3,4-methylenedioxymethamphetamine; LSD: lysergic acid diethylamide; GHB: gamma-hydroxybutyrate. Bolded: significant at p < .05.

Overall, 29.3% of participants who used ketamine (n = 60) met adopted DSM-5 criteria for a past 12-month ketamine use disorder at baseline. When stratified by study site, 33.3% of participants in St. Louis, 36.7% in Miami, and 30.0% in Sydney met criteria for any past-12-month ketamine use disorder. There were no significant differences in sociodemographic characteristics or reported routes of administration between those with and without our adopted diagnosis of past 12-month ketamine use disorder. However, significantly more participants who met criteria for a ketamine use disorder reported using alcohol (p = .008) and sedatives or tranquilizers (p = .029) together with ketamine compared to those who did not meet criteria for a ketamine use disorder. A greater percentage of those with ketamine use disorder also reported using ketamine at a bar or club (p = .021), with a roommate, co-worker, or friend (p = .036), and with a spouse or partner (p = .009) than those without the disorder. They were also more likely to endorse using ketamine to bond with friends (p = .010) and to relieve stress (p = .009) than those who did not meet criteria for ketamine use disorder.

3.2. Reliability of ketamine use disorder

Table 2 presents test-retest reliability statistics for DSM-5 adopted criteria. At baseline, 15.1% of participants qualified for a mild ketamine use disorder (2–3 symptoms), 5.4% qualified for a moderate ketamine use disorder (4–5 symptoms), and 8.8% qualified for a severe ketamine use disorder (≥6 symptoms). Though not shown, this included 14.4% in St. Louis, 14.7% in Miami, and 17.0% in Sydney with a mild diagnosis; 4.4% in St. Louis, 4.4% in Miami, and 8.5% in Sydney with a moderate diagnosis; and 3.3% in St. Louis, 13.2% in Miami, and 12.8% in Sydney with a severe diagnosis. The percentage of participants who met criteria for a diagnosis did not vary significantly between baseline and retest interviews at any site.

Table 2.

Percentage of participants who used ketamine (n = 205) meeting DSM-5 adopted criteria for past 12-month ketamine use disorder and test-retest diagnostic reliability.

DSM-5 diagnosis and severity Int. 1 Int. 2 % Positive in Int. 1 κ 95% CI Yule’s Y statistic
+ −
+ A B
− C D
Ketamine use disorder (≥2 symptoms) 39 21 29.3 0.57 0.45, 0.70 0.61
14 131
Mild ketamine use disorder (2–3 symptoms)a 13 18 15.1 – – –
10 164
Moderate ketamine use disorder (4–5 symptoms)b 4 7 5.4 – – –
13 181
Severe ketamine use disorder (≥6 symptoms) 10 8 8.8 0.62 0.41, 0.82 0.79
3 184
Open in a new tab
a

Differences in κ between cities for mild ketamine use disorder diagnoses were statistically significant (p = .039); κ not calculated.

b

Differences in κ between cities for moderate ketamine use diagnoses were statistically significant (p < .001); κ not calculated. Int: interview; κ: simple kappa coefficient; CI: confidence intervals.

Diagnosis of any ketamine use disorder versus no ketamine use disorder showed moderate agreement (κ = 0.57, Y = 0.61), and diagnosis of severe ketamine use disorder versus no severe ketamine use disorder demonstrated excellent agreement (κ = 0.62, Y = 0.79). No significant differences were detected in κ values between St. Louis, Miami, and Sydney for any and severe ketamine use disorder diagnoses. However, the differences in κ between cities for mild and moderate ketamine use disorder diagnoses reached statistical significance (mild: p = .039; moderate: p < .001), where in both cases users were more reliable reporters in St. Louis for mild and moderate diagnoses than Miami (mild: κ = 0.65 vs. κ = 0.21; moderate: κ = 0.73 vs. κ = −0.07). Therefore, the common κ was not calculated for mild or moderate ketamine use disorders.

3.3. Ketamine use disorder individual criteria reliability

Table 3 presents reliability of the individual ketamine use disorder adopted criteria. Continued use of ketamine despite knowledge of physical or psychological problems was the most frequently reported criterion at baseline (59.0%). Other commonly endorsed individual criteria included withdrawal (30.2%), physically hazardous use (29.8%), using more ketamine than intended (23.4%), too much time involved in getting or using ketamine (22.9%), and tolerance (18.0%). Though not shown, the most frequently reported withdrawal symptoms included feeling tired, sleepy, or weak (24.9%), having trouble concentrating (22.0%), headache (18.5%), and feeling anxious, restless, or irritable (15.1%) after not using ketamine. The least commonly reported criteria were a persistent desire to cut down or control ketamine use (3.4%), important activities given up to use ketamine (7.3%), and failure to fulfill role obligations due to ketamine use (7.3%). Across sites, percentages varied significantly only for the criterion of use despite knowledge of ketamine causing social problems during the retest interview (p = .049), which was more commonly endorsed in Sydney (17.0%) than in St. Louis (4.4%).

Table 3.

Percentage of participants who used ketamine (n = 205) meeting individual DSM-5 adopted criteria for past 12-month ketamine use disorder and test-retest diagnostic reliability of individual criteria.

Diagnostic criteria Int. 2 % Positive in Int. 1 κ 95% CI Yule’s Y statistic
+ −
Int. 1 + A B
− C D
Continued use despite knowledge of physical or psychological problemsa 93 28 59.0 0.51 0.39, 0.63 0.52
21 63
Withdrawalb 37 25 30.2 0.50 0.36, 0.63 0.54
17 126
Physically hazardous use 43 18 29.8 0.60 0.48, 0.72 0.63
16 128
Using more ketamine than intended 28 20 23.4 0.55 0.41, 0.69 0.62
11 146
Too much time involved in getting or using ketamine 27 20 22.9 0.59 0.45, 0.73 0.69
7 151
Tolerance 22 15 18.0 0.55 0.40, 0.70 0.64
11 157
Craving for ketamine 19 8 13.2 0.71 0.56, 0.86 0.80
5 173
Use despite knowledge of causing social problemsc, d 16 6 10.7 – – –
4 179
Failure to fulfill role obligations 7 8 7.3 0.44 0.21, 0.68 0.65
7 183
Important activities given up to use ketamine 8 7 7.3 0.54 0.31, 0.77 0.73
5 185
Persistent desire to cut down or control ketamine use 3 4 3.4 0.41 0.08, 0.74 0.72
4 194
Open in a new tab
a

Criterion of continued use of ketamine despite knowledge of physical or psychological problems was met if either continued use despite knowledge of physical problems or continued use of ketamine despite knowledge of psychological problems was reported.

b

Withdrawal criterion was met if three or more than three withdrawal symptoms were reported (withdrawal symptoms ≥3) or if withdrawal relief was reported.

c

Criterion for use despite knowledge of it causing social problems was met if a participant reported any of the associated symptoms (problems with family/problems with friends/problems at work or school/physical fights) together with continued use despite knowledge of the problems.

d

Differences in κ between cities for criterion reached statistically significant level (p = .015); κ was not calculated.

Int: interview; κ: simple kappa coefficient; CI: confidence intervals.

All individual criteria showed moderate to excellent agreement (κ range = 0.41–0.71, Y range = 0.52–0.80). The most reliable individual criteria were craving ketamine (κ = 0.71, Y = 0.80), physically hazardous use (κ = 0.60, Y = 0.63), and spending too much time in getting, using, or recovering from ketamine (κ = 0.59, Y = 0.69). Similarly, while the most reliable criteria tended to be more frequently endorsed, the least reliable individual criteria (persistent desire to cut down or control ketamine use and failure to fulfill role obligations) were those with the lowest base prevalence. Only those criteria with more than 20% endorsement in either interview had Y values similar to their κ estimates; the rest varied due to low positivity. κ statistics for all criteria were consistent between sites except for use despite knowledge of ketamine causing social problems (p = .015), where the κ value for St. Louis was significantly lower than Miami and Sydney (κ = 0.33 vs. κ = 0.86 and κ = 0.92, respectively); thus, κ was not calculated for this criterion.

4. Discussion

This study sought to examine DSM-5-based ketamine use disorders in a community-recruited population from three geographically diverse cities and to determine test-retest reliability of these DSM-5 diagnoses and adopted diagnostic criteria. To our knowledge, this is the first study to assess the reliability and applicability across diverse populations of DSM-based ketamine use disorder diagnoses. Among the 640 participants who used ecstasy at least five time in their lives, 205 (32%) reported using ketamine; almost a third (29.3%) of them met adopted criteria for a DSM-5 ketamine use disorder. When considering levels of severity, 15.1% met criteria for a mild use disorder, 5.4% met criteria for a moderate use disorder, and almost a tenth (8.8%) met criteria for a severe use disorder. The percentage meeting criteria for each level of severity did not differ significantly across sites at baseline or retest. Considering the increasing attention given to ketamine as a treatment for major depressive and other mood disorders (Daly et al., 2018; Daly et al., 2019), clinicians should be aware of the potential for misuse.

The validity of any diagnosis depends on its reliability: diagnoses tend to have little clinical utility if they cannot be replicated consistently (Cottler et al., 2009; Helzer et al., 1977a,b; Spitzer and Fleiss, 1974). Diagnoses of ketamine use disorders using the structured CD-SAM instrument were found to have good to excellent agreement for overall ketamine use disorder and severe ketamine use disorder. Due to differences in κ values between cities for mild and moderate ketamine use disorders, κ was not calculated for these diagnoses. These differences may be attributed to low base rates for mild and moderate diagnoses. However, severe ketamine use disorder had a low base rate and the highest level of reliability, which is consistent with prior studies that have shown that lower substance use disorder severity in baseline interviews is associated with more discordant interviews at retest, since a difference of one or two criteria can result in the presence or absence of a diagnosis for mild disorders (Denis et al., 2015; Hasin et al., 2006, 2020).

Our findings also demonstrate reliability both within and between study sites for the diagnosis of an overall ketamine disorder and severe ketamine disorder, with no significant differences found between sites for these diagnoses. This reliability across cities suggests that these diagnoses are applicable to different communities, which is particularly important given the diffusion of nonmedical ketamine use cross-nationally (Huang and Lin, 2020). While there were no site differences for overall and severe diagnoses, St. Louis had significantly greater reliability when compared to Miami for both mild and moderate ketamine use disorders; we restricted further calculation of reliability to severe diagnosis. No individual criterion either at baseline or retest varied significantly between St. Louis and Miami. However, compared to those in Miami, participants in St. Louis were younger on average, which could account for differences in diagnostic reliability between the two cities.

The most commonly endorsed DSM-5 adopted diagnostic criteria for ketamine use disorder were continued use of ketamine despite knowledge of physical or psychological problems, withdrawal, physically hazardous use, and using more ketamine than intended. Compared to the DSM-IV dependence symptoms reported by individuals who used ketamine in the web-based survey conducted by Uosukainen et al. (2015), a much higher percentage of our sample reported continuing to take ketamine despite physical or psychological problems (59.0% versus 9.8%). However, it is unclear how Uosukainen et al. assessed this criterion or if participants were assessed with every question. The CD-SAM measured the occurrence of 23 physical and 13 psychological symptoms due to ketamine before asking if the participant had continued taking the drug after realizing it had caused any of the aforementioned problems. A greater number of participants in our sample reported using despite recognizing physical symptoms than psychological symptoms, and the most frequently endorsed symptoms of headaches or dizziness, blurred vision, blackouts, memory lapses, and nausea are consistent with earlier reports (Copeland and Dillon, 2005; Gińe et al., 2016; Jansen and Darracot-Cankovic, 2001; Morgan et al., 2010, 2004; Sassano-Higgins et al., 2016; Zhang et al., 2020). The CD-SAM did not assess the prevalence of urinary symptoms, which have also been shown to be common among persons dependent on ketamine (Winstock et al., 2012). Dillon et al. (2003) found that many individuals who used ketamine who reported experiencing “adverse” symptoms perceived them as positive effects of the drug, which might explain why such a high percentage of our sample continued taking ketamine despite recognizing these symptoms.

Despite the lack of consensus regarding the existence of a ketamine-specific withdrawal syndrome, almost a third of our sample met the withdrawal criterion for ketamine at baseline. This criterion was met if the participant reported three or more withdrawal symptoms after not using ketamine or reported withdrawal relief. Withdrawal relief was less common than reporting three or more symptoms (5.9% versus 28.8%). Uosukainen et al. (2015), who assessed only withdrawal relief among persons who used ketamine, found a similar result (4.6%). Consistent with the extant literature, frequently endorsed withdrawal symptoms after cessation of ketamine use included anxiety, tiredness, a change in appetite, depression, trouble sleeping, and craving ketamine (Chen et al., 2020; Critchlow, 2006; Goyal et al., 2014; Lin et al., 2016). Participants who experienced these withdrawal symptoms from cessation of ketamine use were able to report all of them fairly reliably in this study. However, since the wording of the withdrawal question asked about symptoms which occurred “during the first few hours or days after not using ketamine,” it is possible or likely that participants may have misreported lingering side-effects of ketamine use as withdrawal symptoms. Future research is needed to further evaluate the potential for withdrawal symptoms among people who use. Considering that after-effects of this dissociative anesthetic can last for many hours, it is important to explore a different timeframe for possible withdrawal effects.

Beyond diagnostic reliability, item-by-item reliability for individual symptoms is another important component of an instrument’s clinical utility (Cottler et al., 1989). All individual DSM-5 diagnostic criteria for ketamine use disorder demonstrated acceptable test-retest reliability. With the exception of persistent desire to cut down or control ketamine use and failure to fulfill role obligations, all items showed moderate to excellent agreement; those two criteria with agreement below 0.50 had much higher Yule’s Y statistics, which suggests that their low kappa agreement may be attributed to low base rates (Spitznagel and Helzer, 1985). The same is true for associated symptoms with low base rates, relatively low κ values, but higher Y values. Individual criteria were once again reliable within and between sites, with significant differences between κ values only found for use despite knowledge of ketamine causing social problems. Since this criterion depended on the endorsement of one or more associated symptoms (problems with family, problems with friends, problems at work or school, or physical fights due to ketamine), and since all symptoms except problems with friends had relatively low endorsement by respondents in either interview, this inconsistency in κ values may also be attributed to low base rates.

4.1. Limitations

First, while this study was designed to test the reliability of adopted diagnostic criteria for club drugs, ketamine was not the focus of the parent study. Participants were eligible if they reported using ecstasy more than five times in their lifetime and at least once in the past 12 months, which means that the sample for this analysis did not include any individuals who used ketamine but not ecstasy. Those who used ketamine but were not eligible for inclusion in the study may differ from those who were included, and thus our findings should not be generalized to all persons who use ketamine. Similarly, results cannot be generalized to those who used ketamine less frequently. Second, the data were collected between 2002 and 2005. While the age of the data may have little impact on the reliability of the CD-SAM’s diagnostic algorithm or the nosology of ketamine use disorders, possible changes in use, populations who use, and the purity and composition of ketamine itself could impact symptom base rates. Third, some of the severity-level diagnoses and individual criteria included small numbers of participants, which resulted in relatively wide confidence intervals for κ values. Fourth, there were no measures specific to frequency of ketamine use beyond lifetime use greater than five times. Regular or more frequent use of ketamine may differentiate ketamine use disorder from no use disorder, and should be considered in future studies. Finally, polysubstance use may have impacted symptom reporting. Consistent with previous reports (Barrett et al., 2005; Giné et al., 2016; Morgan et al., 2009; Pavarin et al., 2019; Zhang et al., 2020), the vast majority of our participants who used ketamine reported lifetime use of other substances, as well as using ketamine together with other substances––for instance, almost two-thirds (65.7%) of our sample reported using ketamine together with ecstasy. While the CD-SAM queried individual criteria separately for each drug used more than five times, it is possible that users may have had difficulty differentiating the consequences and effects of ketamine from that of other substances. Future research should evaluate persons with both ketamine histories and other drug histories to compare differences in symptom reports.

4.2. Conclusions

Our results demonstrate that DSM-5-based diagnoses of ketamine use disorder can be reliably evaluated with a structured diagnostic interview in diverse communities of people who use. Consistent test-retest reliability of diagnostic results and individual criteria suggests cross-cultural applicability of the CD-SAM’s diagnostic algorithm for disordered use of ketamine. The endorsement of ketamine-related withdrawal among some participants suggests a need for future research regarding the inclusion of withdrawal symptoms when assessing ketamine use disorders, though ketamine-specific after-effects should be differentiated from withdrawal symptoms.

Role of Funding Source

Research reported in this publication was supported by the National Institute on Drug Abuse of the National Institutes of Health under Award Numbers R01DA14584 (PI: Cottler), T32DA035167 (PI: Cottler), and R01DA044207 (PI: Palmar). The funding source had no role in the study design, collection, analysis, interpretation of data, or writing of the manuscript, nor the decision to submit the present research.

Footnotes

Declaration of Competing Interest

Dr. Palamar has consulted for Alkermes. The authors have no other potential conflicts to declare.

References

  1. American Psychiatric Association, 2013. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Publishing,. [Google Scholar]
  2. Anthony JC, Helzer JE, 1991. Syndromes of drug abuse and dependence. In: Robins LN, Regier DA (Eds.), Psychiatric Disord in America: The ECA Study. The Free Press, New York, pp. 116–154. [Google Scholar]
  3. Barrett SP, Gross SR, Garand I, Pihl RO, 2005. Patterns of simultaneous polysubstance use in Canadian rave attendees. Subst. Use Misuse 40, 1525–1537. [DOI] [PubMed] [Google Scholar]
  4. Center for Behavioral Health Statistics and Quality, 2020. 2019 National Survey on Drug Use and Health: Detailed Tables. Subst Abuse Mental Health Serv Adm. [Google Scholar]
  5. Chen LY, Chen CK, Chen CH, Chang HM, Huang MC, Xu K, 2020. Association of craving and depressive symptoms in ketamine-dependent patients undergoing withdrawal treatment. Am. J. Addict 29 (1), 43–50. [DOI] [PubMed] [Google Scholar]
  6. Cohen J, 1960. A coefficient of agreement for nominal scales. Educ. Psychol. Meas 20 (1), 37–46. [Google Scholar]
  7. Compton WM, Cottler LB, Dorsey KB, Spitznagel EL, Magera DE, 1996. Comparing assessments of DSM-IV substance dependence disorders using CIDI-SAM and SCAN. Drug Alcohol Depend. 41 (3), 179–187. [DOI] [PubMed] [Google Scholar]
  8. Copeland J, Dillon P, 2005. The health and psycho-social consequences of ketamine use. Int. J. Drug Policy 16 (2), 122–131. [Google Scholar]
  9. Cottler LB, Leung KS, Ben-Abdallah A, 2009. Test–re-test reliability of DSM-IV adopted criteria for 3, 4–methylenedioxymethamphetamine (MDMA) abuse and dependence: a cross-national study. Addiction 104 (10), 1679–1690. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cottler LB, Robins LN, Helzer JE, 1989. The reliability of the CIDI-SAM: A comprehensive substance abuse interview. Br. J. Addict 84 (7), 801–814. [DOI] [PubMed] [Google Scholar]
  11. Cottler LB, Schuckit MA, Helzer JE, Crowley T, Woody G, Nathan P, Hughes J, 1995. The DSM-IV field trial for substance use disorders: major results. Drug Alcohol Depend. 38 (1), 59–69. [DOI] [PubMed] [Google Scholar]
  12. Cottler LB, Womack SB, Compton WM, Ben-Abdallah A, 2001. Ecstasy abuse and dependence among adolescents and young adults: applicability and reliability of DSM-IV criteria. Hum. Psychopharmacol 16 (8), 599–606. [DOI] [PubMed] [Google Scholar]
  13. Coyle SL, 1998. Women’s drug use and HIV risk: Findings from NIDA’s cooperative agreement for community-based outreach/intervention research program. Women Health 27 (1–2), 1–18. [DOI] [PubMed] [Google Scholar]
  14. Critchlow DG, 2006. A case of ketamine dependence with discontinuation symptoms. Addiction 101 (8), 1212–1213. [DOI] [PubMed] [Google Scholar]
  15. Daly EJ, Singh JB, Fedgchin M, Cooper K, Lim P, Shelton RC, Drevets WC, 2018. Efficacy and safety of intranasal esketamine adjunctive to oral antidepressant therapy in treatment-resistant depression: a randomized clinical trial. JAMA Psychiatry 75 (2), 139–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Daly EJ, Trivedi MH, Janik A, Li H, Zhang Y, Li X, Singh JB, 2019. Efficacy of esketamine nasal spray plus oral antidepressant treatment for relapse prevention in patients with treatment-resistant depression: a randomized clinical trial. JAMA Psychiatry 76 (9), 893–903. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Denis C, Gelernter J, Hart A, Kranzler H, 2015. Inter-observer reliability of DSM-5 substance use disorders. Drug Alcohol Depend. 153, 229–235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Dillon P, Copeland J, Jansen K, 2003. Patterns of use and harms associated with nonmedical ketamine use. Drug Alcohol Depend. 69 (1), 23–28. [DOI] [PubMed] [Google Scholar]
  19. Donner A, Eliasziw M, Klar N, 1996. Testing the homogeneity of kappa statistics. Biometrics 52, 176–183. [PubMed] [Google Scholar]
  20. Dowling-Guyer S, Johnson ME, Fisher DG, Needle R, Watters J, Andersen M, Williams M, Kotranski L, Booth R, Rhodes F, Weatherby N, 1994. Reliability of drug users’ self-reported HIV risk behaviors and validity of self-reported recent drug use. Assessment 1 (4), 383–392. [Google Scholar]
  21. Fernández-Calderón F, Vidal-Giné C, López-Guerrero J, Lozano-Rojas ÓM, 2016. Reliability, convergent and structural validity and cut-off score of the Severity of Dependence Scale for recreational ketamine users. Addict. Behav 60, 1–7. [DOI] [PubMed] [Google Scholar]
  22. Fleiss JL, Levin B, Paik MC, 2013. Statistical Methods for Rates and Proportions. John Wiley & Sons. [Google Scholar]
  23. Giné CV, Calderón FF, Guerrero JL, 2016. Patterns of use, harm reduction strategies, and their relation to risk behavior and harm in recreational ketamine users. Am. J. Drug Alcohol Abuse 42 (3), 358–369. [DOI] [PubMed] [Google Scholar]
  24. Goyal S, Ambekar A, Ray R, 2014. Ketamine dependence in an anesthesiologist: an occupational hazard? Indian. J. Psychol. Med 36, 335–337. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Halkitis PN, Palamar JJ, Mukherjee PP, 2007. Poly-club-drug use among gay and bisexual men: a longitudinal analysis. Drug Alcohol Depend. 89 (2–3), 153–160. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Hasin D, Samet S, Nunes E, Meydan J, Matseoane K, Waxman R, 2006. Diagnosis of comorbid psychiatric disorders in substance users assessed with the Psychiatric Research Interview for Substance and Mental Disorders for DSM-IV. Am. J. Psychiatry 163 (4), 689–696. [DOI] [PubMed] [Google Scholar]
  27. Hasin D, Shmulewitz D, Stohl M, Greenstein E, Roncone S, Aharonovich E, Wall M, 2020. Test-retest reliability of DSM-5 substance disorder measures as assessed with the PRISM-5, a clinician-administered diagnostic interview. Drug Alcohol Depend. 216, 108294. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Helzer JE, Clayton PJ, Pambakian R, Reich T, Woodruff RA, Reveley MA, 1977a. Reliability of psychiatric diagnosis. II. The test/retest reliability of diagnostic classification. Arch. Gen. Psychiatry 34, 136–141. [DOI] [PubMed] [Google Scholar]
  29. Helzer JE, Robins LN, Taibleson M, Woodruff RA, Reich T, Wish ED, 1977b. Reliability of psychiatric diagnosis. I. A methodological review. Arch. Gen. Psychiatry 34, 129–133. [DOI] [PubMed] [Google Scholar]
  30. Horton J, Compton W, Cottler LB, 2000. Reliability of substance use disorder diagnoses among African–Americans and Caucasians. Drug Alcohol Depend. 57 (3), 203–209. [DOI] [PubMed] [Google Scholar]
  31. Huang M-C, Lin S-K, 2020. Ketamine abuse: Past and present. In: Hashimoto K, Ide S, Ikeda K (Eds.), Ketamine. Springer, Singapore, pp. 1–14. [Google Scholar]
  32. Jansen KLR, Darracot-Cankovic R, 2001. The nonmedical use of ketamlne, part two: A review of problem use and dependence. J. Psychoact. Drugs 33 (2), 151–158. [DOI] [PubMed] [Google Scholar]
  33. Kalsi SS, Wood DM, Dargan PI, 2011. The epidemiology and patterns of acute and chronic toxicity associated with recreational ketamine use. Emerg. Health Threats J 4, 7107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Landis JR, Koch GG, 1977. The measurement of observer agreement for categorical data. Biometrics 33 (1), 159–174. [PubMed] [Google Scholar]
  35. Leung KS, Ben-Abdallah A, Copeland J, Cottler LB, 2010. Modifiable risk factors of ecstasy use: Risk perception, current dependence, perceived control, and depression. Addict. Behav 35 (3), 201–208. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Lin PC, Lane HY, Lin CH, 2016. Spontaneous remission of ketamine withdrawal-related depression. Clin. Neuropharmacol 39, 51–52. [DOI] [PubMed] [Google Scholar]
  37. Morgan CJA, Muetzelfeldt L, Curran HV, 2010. Consequences of chronic ketamine self-administration upon neurocognitive function and psychological wellbeing: a 1–year longitudinal study. Addiction 105 (1), 121–133. [DOI] [PubMed] [Google Scholar]
  38. Morgan CJA, Curran HV, I.S.C.o.D, 2012. Ketamine use: a review. Addiction 107 (1), 27–38. [DOI] [PubMed] [Google Scholar]
  39. Morgan CJA, Muetzelfeldt L, Curran HV, 2009. Ketamine use, cognition and psychological wellbeing: a comparison of frequent, infrequent and ex-users with polydrug and non-using controls. Addiction 104 (1), 77–87. [DOI] [PubMed] [Google Scholar]
  40. Morgan CJA, Riccelli M, Maitland CH, Curran HV, 2004. Long-term effects of ketamine: evidence for a persisting impairment of source memory in recreational users. Drug Alcohol Depend. 75 (3), 301–308. [DOI] [PubMed] [Google Scholar]
  41. Na K-S, Kim Y-K, 2021. Increased use of ketamine for the treatment of depression: Benefits and concerns. Prog. Neuropsychopharmacol. Biol. Psychiatry 104, 110060. [DOI] [PubMed] [Google Scholar]
  42. Nowacka A, Borczyk M, 2019. Ketamine applications beyond anesthesia – a literature review. Eur. J. Pharmacol 860, 172547. [DOI] [PubMed] [Google Scholar]
  43. Palamar JJ, Keyes KM, 2020. Trends in drug use among Electronic Dance Music party attendees in New York City, 2016–2019. Drug Alcohol Depend. 209, 107889. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Palamar JJ, Le A, 2021. Media coverage about medical benefits of MDMA and ketamine affects perceived likelihood of engaging in recreational use. Addict. Res. Theory 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Pavarin RM, Marani S, Turino E, 2019. Ketamine abusers referring to emergency departments in Northern Italy: a cross-sectional study. Ann. Ist. Super. Sanità 55 (4), 338–344. [DOI] [PubMed] [Google Scholar]
  46. Rao SS, Wood DM, Dargan PI, 2015. Ketamine––Epidemiology of misuse and patterns of acute and chronic toxicity. In: Yew DT (Ed.), Ketamine: Use and Abuse. CRC Press, pp. 104–136. [Google Scholar]
  47. Sanacora G, Frye MA, McDonald W, Mathew SJ, Turner MS, Schatzberg AF, Summergrad P, Nemeroff CB, American Psychiatric Association, 2017. A consensus statement on the use of ketamine in the treatment of mood disorders. JAMA Psychiatry 74 (4), 399–405. [DOI] [PubMed] [Google Scholar]
  48. Sassano-Higgins S, Baron D, Juarez G, Esmaili N, Gold M, 2016. A review of ketamine abuse and diversion. Depress. Anxiety 33 (8), 718–727. [DOI] [PubMed] [Google Scholar]
  49. Spitzer RL, Fleiss JL, 1974. A re-analysis of the reliability of psychiatric diagnosis. Br. J. Psychiatry 125, 341–347. [DOI] [PubMed] [Google Scholar]
  50. Spitznagel EL, Helzer JE, 1985. A proposed solution to the base rate problem in the kappa statistic. Arch. Gen. Psychiatry 42 (7), 725–728. [DOI] [PubMed] [Google Scholar]
  51. Tang WK, Liang HJ, Lau CG, Tang A, Ungvari GS, 2013. Relationship between cognitive impairment and depressive symptoms in current ketamine users. J. Stud. Alcohol Drugs 74 (3), 460–468. [DOI] [PubMed] [Google Scholar]
  52. Tang WK, Morgan CJA, Lau GC, Liang HJ, Tang A, Ungvari GS, 2015. Psychiatric morbidity in ketamine users attending counselling and youth outreach services. Subst. Abuse 36 (1), 67–74. [DOI] [PubMed] [Google Scholar]
  53. Tung C-K, Yeung S-W, Chiang T-P, Xu K, Lam M, 2014. Reliability and validity of the severity of dependence scale in a Chinese sample of treatment-seeking ketamine users. East Asian Arch. Psychiatry 24 (4), 156–164. [PubMed] [Google Scholar]
  54. Uosukainen H, Tacke UR, Winstock A, 2015. Self-reported prevalence of dependence of MDMA compared to cocaine, mephedrone and ketamine among a sample of recreational poly-drug users. Int. J. Drug Policy 26 (1), 78–83. [DOI] [PubMed] [Google Scholar]
  55. Winstock AR, Mitcheson L, Gillatt DA, Cottrell AM, 2012. The prevalence and natural history of urinary symptoms among recreational ketamine users. BJUI Int. 110 (11), 1762–1766. [DOI] [PubMed] [Google Scholar]
  56. Yule GU, 1912. On the methods of measuring association between two attributes. J. R. Stat. Soc 75 (6), 579–652. [Google Scholar]
  57. Zhang C, Xu Y, Zhang B, Hao W, Tang WK, 2020. Cognitive impairment in chronic ketamine abusers. Psychiatry Res. 291, 113206. [DOI] [PubMed] [Google Scholar]

RESOURCES