Test retest reliability of DSM-IV adopted criteria for MDMA abuse and dependence: A cross-national study

Abstract

Aims

This study evaluated the prevalence and reliability of DSM-IV adopted criteria for 3,4-methylenedioxymethamphetamine (MDMA) abuse and dependence with a purpose to determine whether it is best conceptualized within the category of hallucinogens, amphetamines, or its own category.

Design

Test-retest study.

Participants

MDMA users (lifetime use >5 times) were recruited in St. Louis, Miami, and Sydney (n=593). The median lifetime MDMA consumption was 50 pills at the baseline.

Measurements

The computerized Substance Abuse Module for Club Drug (CD-SAM) was used to assess MDMA abuse and dependence. The Discrepancy Interview Protocol (DIP) was used to determine the reasons for the discrepant responses between the two interviews. Reliability of diagnoses, individual diagnostic criteria, and withdrawal symptoms was examined using the kappa coefficient (κ).

Findings

For baseline data, 15% and 59% meeting MDMA abuse and dependence respectively. Substantial test-retest reliability of the diagnoses was consistently observed across cities (κ=0.69). `Continued use despite knowledge of physical/psychological problems' (87%) and `Withdrawal' (68%) were the two most prevalent dependence criteria. `Physically hazardous use' was the most prevalent abuse criterion. Six dependence criteria and all abuse criteria were reliably reported across cities (κ: 0.53 to 0.77). Seventeen out of 19 withdrawal symptoms showed consistency in the reliability across cities. The most commonly reported reason for discrepant responses was `Interpretation of question changed'. Only a small proportion of the total discrepancies were attributed to lying or social desirability.

Conclusion

The adopted DSM-IV diagnostic classification for MDMA abuse and dependence was moderately reliable across cities. Findings on MDMA withdrawal support the argument that MDMA should be separated from other hallucinogens in DSM.

INTRODUCTION

MDMA (3,4-methylenedioxymethamphetamine) or ecstasy is categorized as a controlled substance worldwide (Schedule I in the US) with a high abuse/dependence liability [1]. After a marked decline in the rates of use from 2002 to 2003, the trend leveled off for two years and then increased in 2006 [2-4]. In 2006, over 12 million individuals 12 or older reported using MDMA at least once in their lifetime, with 2.1 million reporting use in the past 12 months [2], indicating MDMA's continued popularity among youth.

MDMA is currently classified in the Diagnostic and Statistical Manual of Mental Disorders-fourth edition (DSM-IV) as an hallucinogen, despite its strong stimulant properties. The typical preparation of MDMA, known as ecstasy, contains the racemic mixture of bioactive (S) and (R) MDMA enantiomers which are known to cause both stimulant and hallucinogenic effects in humans. Although MDMA is structurally similar to both mescaline-type hallucinogens and amphetamine-type stimulants, it is pharmacologically different from any other substance classes [5,6]. Given the high prevalence rate of MDMA use, the harmful consequences of its use, and the distinctive pharmacology of MDMA, we have been arguing for a separate substance class with its own set of substance specific diagnostic criteria for this substance [7-9].

Early studies on MDMA use provided some evidence for disordered use. Three studies emphasized MDMA's amphetamine-like chemical structure and applied the DSM criteria for amphetamine use disorders to determine MDMA abuse and dependence. Hando et al. [10] found that 64% of MDMA users (N=185) met criteria for dependence and 21% met criteria for abuse, with withdrawal (98%), tolerance (83%), and unsuccessful attempts to stop (43%) common. Yen and Hsu [11] found that 22% (N= 200) of adolescent MDMA users from a juvenile abstinence center met adopted DSM criteria for MDMA dependence, with 24% reporting tolerance and 6.5% withdrawal. Schuster et al. [12] found the prevalence of MDMA dependence to be 20% among 3021 community adolescents. Other studies treated MDMA as an hallucinogen (as in DSM-IV) and found that MDMA users had a higher risk of dependence (OR= 6.3) compared to LSD-only users within 24 months after the onset [13,14].

Previous studies provided preliminary evidence for MDMA abuse and dependence, yet the diagnostic criteria applied either excluded MDMA withdrawal (as for hallucinogens) or limited the number of withdrawal symptoms to six (as for amphetamines). Previous research found a much wider range of withdrawal symptoms in addition to those six symptoms [7], which imply that the criteria utilized by those studies are too restrictive for MDMA. More importantly, two properties of the clinical utility of such a classification need testing-- reliability of MDMA abuse and dependence, and cross-cultural applicability.

Cottler et al.'s study [7], one of the first to report not only the evidence for MDMA abuse and dependence, but also the reliability of the diagnosis, was limited by a small sample size (n=52) of young club drug users in St. Louis. Diagnostic criteria for MDMA dependence were based on the seven generic criteria for substance dependence listed in DSM-IV. To determine MDMA withdrawal, a list of 19 withdrawal symptoms was compiled from all other drug specific withdrawal symptoms in DSM. In this study, 43% and 34% of the MDMA users met the DSM criteria for dependence and abuse respectively. Specifically, over half (59%) met criteria for withdrawal by either reporting at least 3 of the 19 withdrawal symptoms or endorsing withdrawal relief. Furthermore, the diagnosis for abuse/dependence consequent to MDMA use was reliable (kappa=0.58).

The present study extends our previous investigation of disordered use of MDMA evaluating MDMA use, abuse and dependence and reliability of DSM-IV adopted criteria in three geographically diverse cities, to further the discussion that MDMA should be conceptualized within its own category. Demonstrating reliability among users from three different sites would be a first step in arguing a separate substance use disorder category for MDMA.

METHODS

Participant recruitment

Data were collected as part of the NIDA funded TriCity Study of Club Drug use, Abuse and Dependence among club drug users in St. Louis, Missouri; Miami, Florida, and Sydney (Australia) from 2002 to 2005. Eligible participants were those who used ecstasy more than five times lifetime, with at least one use occurring in the past 12 months. Participants were actively and systematically recruited from the three communities through advertising flyers, internet postings, posters in universities and high-schools, street and club outreach, and public announcements in local newspapers [15]. Study protocols were approved by the Washington University School of Medicine Institutional Review Board (IRB), and the IRB at each of the participating institutes. Informed consent was obtained from all participants.

Instruments and Procedures

The Substance Abuse Module for Club Drugs (CD-SAM)

The computerized CD-SAM assessed abuse and dependence associated with MDMA use. The CD-SAM is an expanded version of the Composite International Diagnostic Interview - Substance Abuse Module (CIDI-SAM) [16,17]. As with the original SAM, the CD-SAM is structured to elicit detailed information about substance use, onset and recency of use, withdrawal symptoms for each substance, and DSM substance specific diagnostic criteria. It also includes substance specific items on medical, physical, and psychological consequences.

An important feature of the CD-SAM is the separate assessment of each club drug, including MDMA, ketamine, gamma-hydroxybutyric acid (GHB), and flunitrazepam (rohypnol). To determine disordered use of MDMA, the DSM based diagnostic algorithm used by Cottler et al. [7] included meeting at least three of the seven dependence criteria in a 12-month period, while abuse of MDMA was defined as meeting at least one DSM-IV abuse criterion. To determine withdrawal for MDMA, a list of 19 withdrawal symptoms was compiled from all drug specific withdrawal symptoms in DSM. A positive diagnosis of MDMA withdrawal required at least 3 of these or withdrawal relief. The CD-SAM also collects information about the use, abuse and dependence on 13 other substances, including alcohol, marijuana, hallucinogens, stimulants, cocaine, sedatives, inhalants, opioids, ketamine, GHB, rohypnol, PCP, and anabolic steroids. The threshold for assessing this history is use of the substance more than 5 times lifetime.

The computerized CD-SAM was used in both the baseline (the first interview) and the retest (the second interview). Most participants returned for the retest exactly a week after the baseline (52%, median duration between the two interviews= 7days, range 3-14 days). No statistical difference was found between cities in the duration between the two interviews [Kruskal-Wallis test (2, N=593) = 0.60, p=0.74]. To minimize bias, a second independent interviewer, blind to the baseline responses, conducted the retest. All interviewers completed a five-day intensive training (provided by LBC), then practiced another week, before being certified to administer the instrument. The interviews were tape-recorded and reviewed by the quality control manager. Participants were remunerated $15 after the baseline and $40 after the retest for time and effort.

The Discrepancy Interview Protocol (DIP)

The DIP, developed by Cottler et al. [18-20], determined reasons for the differences in the answers between the baseline and the retest. Due to the large number of questions in the CD-SAM, a set of 14 predetermined questions with presumably low reliability were selected with 10 possible reasons for discrepancies between test and re-test. They included answers like not understanding the question, can't remember the answer, not paying attention, and saying no to shorten the interview.

The DIP is administered at the end of the retest interview; the interviewer checks the responses to the predetermined questions for the two interviews. When a discrepant response is found, the interviewer asks for the explanation.

Analyses

The simple and weighted kappa coefficients [21,22] were used to examine the agreement between the two interviews. The homogeneity of kappas across cities (H₀: κ₁= κ₂= κ₃= κ) was examined using the χ² statistic [23,24]. A common kappa was calculated to represent the overall test-retest reliability when no statistical significance difference between cities was found [23]. Kappas of 0.61 or greater are generally considered substantial to excellent agreement; values between 0.41 and 0.60 represent moderate agreement and values smaller than 0.4 indicate poor agreement [25]. All analyses used SAS, version 9.1.

RESULTS

Characteristics of participants

Among the 640 participants enrolled, two who used ecstasy fewer than six times lifetime and 33 without a retest were excluded. Among the remaining 605, eleven participants with a retest outside the two-week window and one outside the eligible age range (15-50 years) were excluded. The final sample consisted of 593 participants (93% completed), with 278 participants from St. Louis (56% male), 178 from Miami (61% male), and 137 from Sydney (61% male). The median age was 22 years [mean=23.31 (5.00), range 16 - 47 years]. No gender or age differences were noted between sites. Educational status varied across sites [χ² (2, N=593) = 8.26, p<0.05], with Miami having the highest percentage with a high school diploma (49% vs. 36%). While the majority of the participants in St. Louis and Sydney were Caucasian (≅73%), 57% in Miami were Hispanic [χ² (6, N=593) = 283.93, p<0.001]. In Miami, all participants endorsed being comfortable conducting an interview in English.

The median number of lifetime MDMA pills consumed was 50 [mean =212(SD=502)]; Sydney and Miami had significantly more pills than St. Louis [Kruskal-Wallis test (2, N=592) = 28.39, p<0.001]. The mean onset age of MDMA was 19 years (SD=4.10); 10% of users began MDMA use before age 16.

Nearly all reported lifetime alcohol or marijuana use (99.7% and 97.5% respectively). A majority reported using hallucinogens (64%), stimulants (62%), or cocaine (61%); nearly half reported sedatives (51%), inhalants (50%), or opioids (47%). On average, six other substances were reported (SD = 2.64). While most started using marijuana before MDMA (86%), over two-thirds reported sedatives, rohypnol, cocaine, ketamine, anabolic steroids, or GHB after or simultaneous with the onset of MDMA use. Over half (57%) of hallucinogen users and 41% of stimulant users reported using those substances before MDMA.

MDMA abuse and dependence

Neither abuse nor dependence, abuse only, and dependence categories were determined based on DSM-IV nomenclature [7]. Table 1 presents the percentage of participants meeting each diagnostic category and the reliability of the diagnoses between the baseline and the retest (weighted kappas). Overall, the proportion meeting MDMA dependence/abuse only criteria at baseline and retest was 59%/15% and 57%/18% respectively. The proportion of the categories varied by city [χ² (4, N=593) = 11.50, p=0.02 at baseline; χ² (4, N=593) = 12.37, p=0.02 at retest] (results not shown). St. Louis had a lower percentage of MDMA dependent users compared to Miami and Sydney. Test-retest reliability of MDMA use disorder diagnosis was substantial: the weighted kappas for St. Louis, Miami, and Sydney were 0.71, 0.65, and 0.62 respectively; test-retest reliability of diagnoses was consistent across cities [χ² (2, N=593) = 2.14, p=0.34], with a common kappa of 0.69 for the overall reliability between the two interviews.

Table 1.

Percentage of MDMA users meeting DSM-IV criteria for abuse and dependence and the test-retest reliability of diagnostic results.

	St. Louis (n=278)				Miami (n=178)				Sydney (n=137)				Total (N=593)
Diagnosis	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ b	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ b	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ b	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	Common κc	95% CI
Dependence (with or without abuse)	52.2 / 50.0	44.6			64.0 / 61.8	55.1			66.4 / 64.2	56.2			59.0 / 56.8	50.4
Abuse only	16.6 / 19.0	10.1			14.6 / 18.5	11.2			10.2 / 13.1	4.4			14.5 / 17.5	9.1
Neither abuse nor dependence	31.3 / 30.9	24.5			21.4 / 19.7	14.6			23.4 / 22.6	16.8			26.5 / 25.6	19.7
Total		79.1				80.9				77.4				79.3
			0.71	0.65-0.78			0.65	0.54-0.76			0.62	0.49-0.75			0.69	0.63-0.74

^aRaw agreement between the two interviews.

^bWeighted kappa

^cThe common kappa was calculated where differences in kappas between cities were not statistically significant (p>0.05).

Dependence criteria

Table 2 presents percentages, ordered by DSM-IV, of individual dependence criteria and reliability of these criteria. For the dependence criteria with more than one associated symptom in CD-SAM (i.e., `Withdrawal' and `Continued use despite knowledge of physical or psychological problems'), the associated symptoms were examined separately. `Continued use of MDMA despite knowledge of physical or psychological problems from it' and `Withdrawal' were the two most frequently reported dependence criteria at the baseline (87% and 68% respectively) and the retest (84% and 62% respectively); the least reported was `Persistent desire to cut down or control MDMA use' (17% at both interviews). `Tolerance' was the most reliably reported criterion (common κ= 0.77), followed by `Withdrawal' (common κ= 0.58), and `Important activities given up to MDMA use' (common κ= 0.57). The least reliable was `Persistent desire to cut down' (common κ= 0.53). Over 60% of users reported at least three withdrawal symptoms from MDMA (common κ = 0.57); 11% reported withdrawal relief (common κ = 0.59).

Table 2.

Percentage of MDMA users meeting individual DSM-IV criteria for substance dependence and the test-retest reliability of individual dependence criteria.

	St. Louis (n=278)				Miami (n=178)				Sydney (n=137)				Total (N=593)
Dependence Criteria (associated symptoms)	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	Common κb	95% CI
Toleranceg	43.7 / 43.3	90.3	0.80	0.73-0.87	55.6 / 51.7	82.6	0.65	0.54-0.76	56.2 / 54.0	90.5	0.81	0.71-0.91	50.2 / 48.3	88.0	0.77	0.72-0.82
Withdrawale,g,h	64.4 / 56.1	78.1	0.55	0.45-0.64	67.4 / 65.7	80.3	0.56	0.43-0.69	76.6 / 70.1	87.6	0.68	0.55-0.82	68.1 / 62.2	80.9	0.58	0.52-0.65
(Withdrawal symptoms ≥ 3)	63.7 / 55.0	77.7	0.54	0.44-0.64	64.6 / 65.2	79.2	0.54	0.42-0.67	75.9 / 70.1	86.9	0.67	0.53-0.81	66.8 / 61.6	80.3	0.57	0.50-0.64
(Withdrawal relief)	11.2 / 9.4	92.5	0.59	0.43-0.75	14.6 / 11.2	91.0	0.60	0.43-0.78	13.9 / 13.9	89.8	0.57	0.37-0.77	12.8 / 11.0	91.4	0.59	0.49-0.69
Using more MDMA than intendedg	37.8 / 35.3	77.3	0.51	0.41-0.62	47.8 / 41.0	78.7	0.57	0.45-0.69	48.2 / 46.7	81.0	0.62	0.49-0.75	43.2 / 39.6	78.6	0.56	0.49-0.63
Persistent desire to cut down or control MDMA use	14.8 / 15.1	84.5	0.39	0.24-0.54	18.0 / 15.7	87.6	0.56	0.40-0.73	21.9 / 20.4	88.3	0.65	0.49-0.81	17.4 / 16.5	86.3	0.53	0.44-0.62
Too much time involved in getting or using MDMAc,g,h	48.6 / 42.8	86.3	0.73	0.64-0.81	61.8 / 52.8	76.4	0.52	0.40-0.66	62.8 / 55.5	82.5	0.64	0.51-0.77	55.8 / 48.7	82.5	__	__
Important activities given up to MDMA useg,h	20.5 / 14.8	87.1	0.56	0.43-0.68	24.2 / 16.9	88.2	0.64	0.50-0.78	34.3 / 24.8	78.8	0.50	0.34-0.65	24.8 / 17.7	85.5	0.57	0.49-0.65
Continued use of MDMA despite knowledge of physical or psychological problems from itf	83.8 / 82.4	86.3	0.51	0.38-0.65	89.3 / 88.2	93.3	0.66	0.49-0.84	89.1 / 81.0	86.1	0.46	0.26-0.66	86.7 / 83.8	88.4	0.54	0.45-0.64
(Used despite knowledge of physical problems)	75.9 / 75.2	82.0	0.51	0.40-0.63	80.9 / 79.8	83.2	0.47	0.30-0.63	78.9 / 75.9	88.3	0.67	0.52-0.82	78.1 / 76.7	83.8	0.55	0.47-0.63
(Used despite knowledge of psychological problems) d	74.8 / 68.7	80.9	0.53	0.42-0.64	83.2 / 78.1	91.6	0.73	0.60-0.86	77.4 / 66.4	78.8	0.48	0.33-0.64	77.9 / 71.0	83.6	−	−

^aRaw agreement between baseline and retest.

^bCommon kappas were calculated where differences in kappas between cities were not statistically significant (p>0.05).

^cDifferences in kappa between cities reached statistically significant level (Likelihood χ², 2df=7.40, p=0.025), the common kappa was not calculated.

^dDifferences in kappa between cities reached statistically significant level (Likelihood χ², 2df=7.60, p=0.022), the common kappa was not calculated.

^eWithdrawal criterion was met if 3 or more than 3 withdrawal symptoms were reported (withdrawal symptoms ≥ 3) or when withdrawal relief was reported.

^fCriterion of continued use of MDMA despite knowledge of physical of psychological problems was met if either continued use despite knowledge of physical problems or continued use despite of psychological problems were reported.

^gDenoted a statistically significant variation in the prevalence across cities at the baseline (χ²test: p<0.05).

^hDenoted a statistically significant variation in the prevalence across cities at the retest (χ²test: p<0.05).

Abuse criteria

As shown in Table, reports of `Failure to fulfill role obligations' and `Use despite knowledge of social problems' varied across cities. Overall, the most frequently reported abuse criterion was `Physically hazardous use', followed by `Use despite knowledge of social problems', `Failure to fulfill role obligations', and `Legal problems'. Criteria were moderately reliable (common κ ranged from 0.59 to 0.73). Five associated symptoms of `Use despite knowledge of social problems' were examined separately. The overall prevalence ranged from 2% (`Physical fights') to 28% (`Continued use despite knowledge of problems'), with a test-retest reliability (common κ) ranging from 0.47 (`Problems with friends') to 0.72 (`Problems with family'), indicating consistency in the test-retest reliability across cities.

Withdrawal symptoms

Table 4 shows the rates of individual withdrawal symptoms and the corresponding reliability. Symptoms were ordered by the baseline prevalence (in descending order). The five most commonly reported symptoms (at both interviews) were `Feel depressed', `Feel tired or weak', `Change in appetite', `Have trouble concentrating', and `Feel anxious, restless, or irritable'. The kappas for those ranged from 0.44 to 0.66, in the moderate to substantial range. The least reported were running eyes or nose and seizures. Among the 19 withdrawal symptoms, eight showed statistically significant variations in prevalence across cities. Two symptoms (`Tremble or twitch' and `Have vivid, unpleasant dreams') also varied across cities in one of the interviews with Sydney having higher prevalence rates of `Seeing, hearing or feeling thing that weren't there', trouble concentrating, and trouble sleeping; Sydney also had a lower rate of MDMA craving compared to St. Louis and Miami (z-tests, p<0.05, results not shown). Seventeen out of 19 symptoms showed consistency in the reliability across cities.

Table 4.

Percentage of MDMA users reporting MDMA specific withdrawal symptoms and the test-retest reliability of individual withdrawal symptoms.

	St. Louis (n=278)				Miami (n=178)				Sydney (n=137)				Total (N=593)
Withdrawal symptoms	% of users reported the symptom at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users reported the symptom at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users reported the symptom at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users reported the symptom at baseline/retest	Rawa agreement (%)	Common κb	95% CI
1) Feel depressed	52.2 / 44.6	85.3	0.71	0.62-0.79	57.3 / 51.1	81.5	0.63	0.51-0.74	62.8 / 53.3	80.3	0.60	0.47-0.73	56.2 / 48.6	83.0	0.66	0.60-0.72
2) Feel tired, sleepy or weak	52.2 / 45.0	72.7	0.46	0.35-0.56	51.1 / 48.3	71.4	0.43	0.29-0.56	62.0 / 49.6	71.5	0.43	0.29-0.58	54.1 / 47.1	72.0	0.44	0.37-0.51
3) Have a change in appetitee,f	42.8 / 33.8	75.9	0.49	0.39-0.60	51.7 / 52.3	74.7	0.49	0.37-0.62	70.1 / 68.6	82.5	0.59	0.44-0.74	51.8 / 47.4	77.1	0.52	0.45-0.59
4) Have trouble concentratinge,f	37.4 / 29.9	76.6	0.48	0.37-0.59	37.6 / 32.0	76.4	0.48	0.35-0.62	63.5 / 57.7	79.6	0.57	0.43-0.71	43.5 / 36.9	77.2	0.51	0.43-0.58
5) Feel anxious, restless, or irritablee,f	38.5 / 35.6	73.4	0.43	0.32-0.54	39.9 / 36.5	79.8	0.57	0.45-0.70	54.7 / 53.3	79.6	0.59	0.45-0.72	42.7 / 40.0	76.7	0.52	0.45-0.59
6) Have trouble sleepinge,f	30.6 / 27.0	78.4	0.47	0.36-0.59	32.6 / 36.0	73.0	0.40	0.26-0.54	49.6 / 43.1	74.5	0.49	0.34-0.63	35.6 / 33.4	75.9	0.46	0.38-0.53
7) Have a headache	32.0 / 28.8	79.5	0.52	0.41-0.63	33.2 / 29.2	79.2	0.52	0.38-0.65	32.9 / 27.7	80.3	0.53	0.38-0.69	32.6 / 28.7	79.6	0.52	0.45-0.60
8) Craving for Ecstasye,f	27.8 / 22.0	88.5	0.69	0.59-0.79	29.8 / 21.9	80.9	0.51	0.36-0.65	13.1 / 11.0	89.1	0.48	0.26-0.71	25.0 / 19.4	86.3	0.62	0.54-0.69
9) Have muscle pains	23.7 / 21.2	87.4	0.64	0.53-0.75	21.4 / 20.2	84.3	0.52	0.37-0.68	24.1 / 24.8	86.1	0.62	0.47-0.78	23.1 / 21.8	86.2	0.61	0.53-0.68
10) Yawn a lot	22.3 / 16.2	83.8	0.48	0.35-0.61	20.8 / 19.1	83.7	0.49	0.33-0.65	21.9 / 21.2	81.8	0.46	0.28-0.64	21.8 / 18.2	83.3	0.48	0.39-0.57
11) Have a fast heartbeate,f	18.7 / 12.2	87.8	0.54	0.40-0.67	16.3 / 12.9	84.3	0.37	0.18-0.56	28.5 / 24.8	81.8	0.53	0.37-0.69	20.2 / 15.4	85.3	0.50	0.41-0.59
12) Feel nauseated to vomit	15.8 / 12.6	88.9	0.54	0.40-0.69	21.9 / 17.4	83.2	0.47	0.31-0.63	15.3 / 14.6	89.1	0.57	0.38-0.76	17.5 / 14.5	87.2	0.52	0.43-0.62
13) See, hear or feel things that weren't theree,f	12.6 / 8.3	89.2	0.43	0.26-0.59	14.0 / 11.2	90.5	0.57	0.39-0.75	27.7 / 20.4	85.4	0.60	0.45-0.76	16.5 / 12.0	88.7	0.54	0.44-0.63
14) Sweat or have fevere,f	11.9 / 11.2	91.4	0.58	0.42-0.73	16.9 / 20.2	85.4	0.52	0.36-0.68	24.1 / 21.9	83.2	0.53	0.36-0.70	16.2 / 16.4	87.7	0.54	0.45-0.63
15) Have vivid, unpleasant dreamsf	14.0 / 10.4	87.8	0.43	0.27-0.59	15.2 / 16.9	83.7	0.39	0.21-0.57	21.9 / 19.7	86.1	0.58	0.41-0.75	16.2 / 14.5	86.2	0.47	0.37-0.57
16) Tremble or twitche	11.5 / 10.1	92.1	0.59	0.43-0.74	11.8 / 12.9	86.5	0.38	0.18-0.58	21.9 / 16.8	83.2	0.46	0.28-0.65	14.0 / 12.5	88.4	0.50	0.39-0.60
17) Have diarrhea or a stomachache	10.8 / 9.4	92.1	0.56	0.40-0.73	16.9 / 14.0	83.7	0.38	0.19-0.56	13.1 / 14.6	91.2	0.63	0.44-0.82	13.2 / 12.0	89.4	0.53	0.42-0.63
18) Have runny eyes or nosec	6.1 / 3.2	95.7	0.52	0.28-0.75	6.2 / 3.9	91.0	0.07	-0.14-0.27	4.4 / 7.3	94.2	0.47	0.16-0.78	5.7 / 4.4	93.9	−	−
19) Have seizuresd	0.4 / 0.4	99.3	−	−	1.1 / 0.6	99.4	−	−	0 / 0	100.0	−	−	0.5 / 0.3	99.5	−	−

^aRaw agreement between baseline and retest.

^bCommon kappas were calculated where differences in kappas between cities were not statistically significant (p>0.05).

^cDifferences in kappa between cities reached statistically significant level (Likelihood χ², 2df=9.45, p=0.009), the common kappa was not calculated.

^dKappas were not calculated due to low base rates for positive cases (less than 1%).

^eDenoted a statistically significant variation in the prevalence across cities at the baseline (χ²test: p<0.05).

^fDenoted a statistically significant variation in the prevalence across cities at the retest (χ²test: p<0.05).

Sources of discrepancy

Table 5 shows discrepant answers from baseline to retest, the percentages in favor of the responses given at the baseline and the retest, and reasons for discrepant answers. The three items with the highest number of discrepancies were `Feeling anxious, restless, or irritable', trouble concentrating and `Using more MDMA than intended'. Despite this, these three items were still reliably reported both within and across cities. The item with the lowest number of discrepancies was `Withdrawal relief', one of the associated symptoms of `Withdrawal'. Participants were asked to determine which response was correct (baseline or retest). Of the 10 predetermined reasons for discrepancy, the most commonly reported reasons were `Interpretation of question changed' (39%), `Didn't understand the question' (13%), and `Don't know' (12%). Fortunately, the least reported reasons were `the interviewer was wrong and miscoded or misunderstand the response' (0.3%) and `Participant was too embarrassed or thought that the interview would disapprove' (0.6%). Only eight of the 446 withdrawal related discrepancies (1.8%) were due to a participant's misinterpreting a withdrawal symptom of MDMA that was also attributed to other substances used with MDMA. Additionally, the DIP was used to determine whether participants frequently gave their second answer as the correct answer. Findings of the `resolved response' showed that, of the 13 DIP items, only three items were resolved in favor of the baseline and only two were resolved in favor of the retest, suggesting that there was no systematic bias in favor of either interviews.

Table 5.

Frequencies of discrepancies, the proportions of reasons attributed to discrepant responses, and the results of resolved response.

			Reasons for discrepancy											Resolved response
Question no.	Symptoms	Total No. of Discrepant answers with reasons given in DIPa	A %	B %	C %	D %	E %	F %	G %	H %	I %	J %	K %	% in favor of baseline response	% in favor of retest response
D604AA	Have there been times when you used MDMA in a situation when you could have gotten yourself or others hurt?	82	7	15	1	0	6	0	0	2	52	6	10	45	55
D904AA	Have you often used more MDMA or used MDMA over a longer period of time than you intended?	122	16	12	2	0	7	2	1	6	44	8	2	53	47
D1004AA	Have you wanted to quit or cut down on MDMA or tried to quit or cut down but were unable to for at least a month?	75	25	12	4	1	12	1	0	1	28	9	5	44	56
D1104AA	Have you ever spent a lot of time using, planning to get, or recovering from the effects of MDMA?	97	16	7	5	0	11	1	0	3	37	13	5	63b	37b
D1204AB	Did you ever find you had to use much more MDMA to get the effect you wanted or the same amount of MDMA had much less effect than before?	68	12	12	1	1	10	3	0	7	35	10	7	54	46
D1304AA	Did you ever give up or reduce any important activities to get or use MDMA?	82	11	18	2	0	7	5	0	0	39	9	9	66b	34b
D1404AA01	During the first few hours or days of not using MDMA, did you ever feel depressed?	99	10	8	6	0	6	2	0	11	41	13	2	45	55
D1404AA02	During the first few hours or days of not using MDMA, did you ever feel anxious, restless, or irritable?	136	11	13	5	1	13	1	0	5	36	14	1	36b	64b
D1404AA03	During the first few hours or days of not using MDMA, did you ever have trouble concentrating?	131	9	9	6	0	18	2	1	3	38	12	1	45	55
D1404AA19	During the first few hours or days of not using MDMA, did you ever crave MDMA?	80	9	6	8	0	13	1	1	8	31	21	3	43	57
D1504AA	Have you ever used MDMA to avoid or get rid of those so-called withdrawal symptoms caused by MDMA?	51	22	10	2	0	8	0	2	0	39	10	8	48	52
D1604AB	Did you continue to use MDMA after you realized it was causing any of these physical health problems?	84	21	10	4	0	4	2	2	0	44	12	1	39b	61b
D1704AB	Did you continue to use MDMA after you realized it was causing any of these emotional or psychological problems?	91	10	8	2	0	10	2	1	1	44	13	9	61b	39b
Totals		1198
Proportion of total of 1198(%)			13.3	10.8	4.0	0.3	10.0	1.8	0.6	3.9	39.4	11.8	4.3

Reasons for discrepancy were: A - Respondent (R) didn't understand the question; B - R couldn't remember the answer at that time; C - R's sitution has changed since the first interview; D - the interviewer was wrong and miscoded or misunderstood the response; E - R was not paying attention on one occasion; F - R said `No' to shorten the interview/lied; G - Rwas too embarrassed/thought interviewer would disapprove; H - R didn't really know what the right answer was; I - R's interpretation of question changed; J - Don't know; K - other.

^aNumber of participants completing the DIP=590. Not all participants answered all questions.

^bDifferences in proportion reached the statistical significant level (z-statistics, p<0.05).

DISCUSSION

The aim of this study was to examine the prevalence and reliability of adopted DSM -IV diagnostic criteria for MDMA in different communities, and to determine whether MDMA should be considered a separate substance category. Lifetime MDMA abuse or dependence was found among over 70% of users, with more than half meeting criteria for MDMA dependence--comparable with our previous studies in the US [7,26] and an Australian study [10].

One of the hallmarks of a classification system is its reliability and cross-cultural applicability. Validity of the clinical diagnosis is constrained by its reliability [27-31] and the diagnostic classifications have little practical value if the diagnostic results cannot be reproduced consistently. In the present study, although the proportion of MDMA abuse and dependence varied across cities, the diagnostic results were reliably obtained both within and across cities. Our findings not only demonstrate substantial reliability for the diagnostic results in each study site, but also consistency of reliability across sites, suggesting the applicability of the diagnostic classifications in different communities. The present findings are important given the absence of cross-national research on the reliability of diagnostic classifications for MDMA use disorder in the field.

Perhaps the finding most relevant to our argument for separating MDMA from other hallucinogens is that more than 68% (baseline) of users reported enough symptoms to meet a `Withdrawal' syndrome, a syndrome not yet recognized in DSM-IV. In the present study, `Withdrawal' was the second most frequently reported criterion for MDMA dependence, with consistent test-retest reliability across different cities. Subsequent analyses on the two associated symptoms for `Withdrawal' showed that, at the baseline, 67% of the MDMA users reported at least three of the19 withdrawal symptoms and 13% had ever used MDMA to avoid or get rid of any of those withdrawal symptoms (`Withdrawal relief'). These findings agree with the DSM-IV diagnostic features of Substance Withdrawal which highlight a substance-specific maladaptive behavioral change (with physiological and cognitive concomitants) due to the cessation of, or reduction in substance use. Users did experience MDMA withdrawal and were able to reliably report these specific experiences. Indeed, the structural and pharmacological distinctiveness of MDMA, the evidence for disordered use of the substance, and the reliable self-report of MDMA specific withdrawal together provide strong support for separating MDMA from other hallucinogens in a diagnostic classification system such as DSM.

It is less clear whether MDMA should also be separated from the amphetamine class when individual withdrawal symptoms are considered. Although MDMA users reported a wide range of withdrawal symptoms associated with the cessation of, or reduction in MDMA use, four of the six most commonly reported symptoms overlap with those for amphetamines, including `Feel depressed', `Have a change in appetite', `Feel anxious, restless, or irritable', and `Have trouble sleeping'. While this may suggest that both substances share some commonalities, it is not possible to say whether MDMA should be grouped as amphetamine at this stage because many non-amphetamine-specific withdrawal symptoms were also reliably reported by users. An important question regarding the self-reported withdrawal symptoms is whether these symptoms can be corroborated by findings in controlled settings of humans. To date most research on the physiological dependence on MDMA focuses on tolerance, acute and sub-acute effects, and long term neurological damages [32-34]; withdrawal symptoms for MDMA have rarely been examined in controlled settings. A recent animal study showed that acute administration of 5-HT_1/2 - serontonergic antagonist (metergoline) and β_1/2-adrenergic antagonist (timolol) induced a withdrawal syndrome in mice chronically treated with MDMA, while both D₁ and D₂ - dopaminergic antagonists (SCH23390 and raclopride) failed to produce relevant behavioral manifestations of withdrawal, such as paw tremor and face rubbing, suggesting the possibility of a MDMA-specific withdrawal syndrome not attributable to its amphetamine-like properties [35]. However, due to the lack of controlled studies on MDMA withdrawal, the biological mechanisms underlying these non-amphetamine-specific withdrawal symptoms remain unclear.

Related to this issue is the polysubstance issue. Consistent with other MDMA studies, polysubstance use was prevalent here [7,10-12]. One way to test the hypothesis that MDMA users can differentiate consequences from their use of MDMA and other substances would be to evaluate users with both MDMA histories and other drug histories. They would then be compared on symptom reports to both substances to understand how they can differentiate the behaviors. If no differences were found between consequences for MDMA and other substances among users of both, it could be said that users are not able to separate the effects. This type of analysis was carried out on sample of cocaine users; they were found to be able to differentiate different classes of substances [36].

Other dependence criteria most frequently reported by the participants include `Continued use of MDMA despite knowledge of physical or psychological problems', `Too much time involved in getting or using MDMA', and `Tolerance'. The prevalence of `Continued use of MDMA despite knowledge of physical or psychological problems' was high (87%) and should be re-evaluated for all of the substances. While most users were aware of the deleterious consequences associated with MDMA use, the possession of this knowledge did not deter use of the substance. This discrepancy between `doing' and `knowing' poses a serious challenge to the educational programs aimed to increase the awareness of the harmful consequences associated with MDMA use. It is hypothesized that actual drug use behaviors are influenced more by the perception of risks associated with the substance rather than the awareness of risks per se [37].

The test-retest reliability of individual dependence criteria was generally lower than that of the final diagnostic results. This is not unexpected since a high item-by-item reliability is more difficult to achieve compared to that of a composite score [38]. Since the composite diagnosis for dependence was determined only by the total number of criteria reported, the final diagnoses obtained in the two interviews could be the same even when two different groups of individual criteria were reported in the two interviews. Despite the inherent difficulties in obtaining a high item-by-item reliability, moderate to substantial test-retest reliability was observed in all individual dependence criteria and reliability was consistent across cities.

Consistent with previous studies [7,10,26], `Physically hazardous use' was the most frequently reported abuse criterion (51% ). All four abuse criteria were reliably reported in each city and the reliability was consistent across study sites. Interestingly, as with the case of `Continued use despite knowledge of physical or psychological problems', the discrepancy between knowledge of harmful consequences and actual drug use behaviors was also observed in the context of social problems. Among those who reported knowledge of problems with family, friends, at work/school, or physical fights, most continued to use MDMA despite the fact that they were aware of the problems. Although abuse is considered to be a less severe substance use problem than dependence [2,39,40], the findings that many users engaged in dangerous activities under the influence of MDMA and continued to the substance despite the knowledge of problems is a concern.

An important feature of this study is the Discrepancy Interview Protocol (DIP), mixed with a test-retest study. The DIP identifies the sources of discrepancies between the two interviews. Withdrawal symptoms and `Using more MDMA than intended' had the highest number of discrepant responses. Over a third of users with discrepant responses on these items attributed the discrepancies to a change in the way they interpreted the questions. Interpretation change constituted 39% of the reason for discrepant responses. More importantly, contrary to the common belief that a self-report of substance use behaviors is subject to desire to deceive and social desirability [41-43], in the present study, only 1.8% of the discrepancies were attributed to lying or `saying no to shorten the interview' and 0.6% were attributed to social desirability (`Too embarrassed or thought interviewer would disapprove').

To conclude, these results not only highlight the existence of MDMA abuse and dependence, but also demonstrate the reliability of the diagnostic results based on DSM-IV nomenclature [7]. Consistent test-retest reliability of the diagnostic results has been found across different communities despite the variation of MDMA abuse and dependence across study sites, suggesting a cross-national applicability of our diagnostic algorithm. The high prevalence of MDMA withdrawal among users underscores the need to evaluate this. First, in DSM-IV, MDMA is classified as an hallucinogen where `Withdrawal' is ignored. However, the exclusion of this phenomenon for MDMA may increase the false negative rate of diagnosis. Some MDMA users who have developed dependence on MDMA may be misdiagnosed as `No disorder' or abuse and thus may not receive proper interventions. Second, the current DSM classification for MDMA may elicit false impressions about the addictiveness of the substance by ignoring the existence of MDMA withdrawal. Thus, it is advised that the DSM-V Workgroup considers separating MDMA from other hallucinogens in the future versions of the Diagnostic Manual.

Table 3.

Percentage of MDMA users meeting individual DSM-IV criteria for substance abuse and the test-retest reliability of individual abuse criteria.

	St. Louis (n=278)				Miami (n=178)				Sydney (n=137)				Total (N=593)
Abuse Criteria (associated symptoms)	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	κ	95% CI	% of users meeting the criterion at baseline/retest	Rawa agreement (%)	Common κb	95% CI
Failure to fulfill role obligationsd,e	19.8 / 18.7	88.9	0.64	0.53-0.76	23.0 / 14.0	84.3	0.49	0.33-0.65	37.3 / 31.4	81.0	0.58	0.44-0.72	24.8 / 20.2	85.7	0.59	0.51-0.66
Physically hazardous use	50.0 / 54.0	86.0	0.72	0.64-0.80	51.1 / 52.3	87.6	0.75	0.66-0.85	51.8 / 51.1	84.7	0.69	0.57-0.81	50.8 / 52.8	96.2	0.72	0.67-0.78
Legal problems	3.6 / 3.6	98.6	0.79	0.59-0.99	6.7 / 3.9	96.1	0.61	0.35-0.87	4.4 / 4.4	100.0	1.00	−	4.7 / 3.9	98.2	0.73	0.57-0.88
Use despite knowledge of it causing social problemsc,e	23.7 / 20.1	87.8	0.64	0.53-0.75	28.1 / 20.2	82.0	0.51	0.37-0.66	32.1 / 30.7	82.5	0.59	0.45-0.74	27.0 / 22.6	84.8	0.60	0.52-0.67
(Problems with family)	15.8 / 14.4	92.1	0.69	0.57-0.81	23.6 / 15.2	89.3	0.66	0.53-0.80	30.7 / 23.4	91.2	0.78	0.66-0.90	21.6 / 16.7	91.1	0.72	0.64-0.79
(Problems with friends)	15.1 / 11.9	88.9	0.52	0.38-0.67	18.5 / 8.4	85.4	0.39	0.21-0.57	17.5 / 11.7	86.9	0.50	0.27-0.68	16.7 / 10.8	87.4	0.47	0.37-0.57
(Problems at work/school)	8.6 / 6.1	93.9	0.55	0.37-0.74	11.2 / 6.2	91.6	0.47	0.25-0.70	16.8 / 16.8	89.8	0.63	0.46-0.81	11.3 / 8.6	92.2	0.57	0.46-0.68
(Physical fights)	2.9 / 2.9	97.8	0.61	0.33-0.90	1.7 / 3.9	97.8	0.59	0.23-0.95	1.5 / 0.7	99.3	0.66	0.04-1.00	2.2 / 2.7	98.2	0.61	0.40-0.82
(Continued use despite knowledge of problems)	25.2 / 21.9	88.1	0.67	0.57-0.77	29.8 / 21.4	82.6	0.55	0.41-0.69	32.9 / 31.4	82.5	0.60	0.46-0.74	28.3 / 24.0	85.2	0.62	0.55-0.69

^aRaw agreement between baseline and retest.

^bCommon kappas were calculated where differences in kappas between cities were not statistically significant (p>0.05).

^cCriterion for 'Use despite knowledge of it causing social problems' was met if a participant reported any of the associated symptoms (problems with family/problems with friends/problems at work or at school/physical) together with 'continued use despite knowledge of the problems'.

^dDenoted a statistically significant variation in the prevalence across cities at the baseline (χ²test: p<0.05).

^eDenoted a statistically significant variation in the prevalence across cities at the retest (χ²test: p<0.05).