Abstract
Background
New psychoactive substances (NPS) continue to emerge; however, few surveys of substance use ask about NPS use. Research is needed to determine how to most effectively query use of NPS and other uncommon drugs.
Objective
To determine whether prevalence of self-reported lifetime and past-year use differs depending on whether or not queries about NPS use are preceded by “gate questions.” Gate questions utilize skip-logic, such that only a “yes” response to the use of specific drug class is followed by more extensive queries of drug use in that drug class.
Methods
We surveyed 1,048 nightclub and dance festival attendees (42.6% female) entering randomly selected venues in New York City in 2016. Participants were randomized to gate vs. no gate question before each drug category. Analyses focus on eight categories classifying 145 compounds: NBOMe, 2C, DOx, “bath salts” (synthetic cathinones), other stimulants, tryptamines, dissociatives, and non-phenethylamine psychedelics. Participants, however, were asked about specific “bath salts” regardless of their response to the gate question to test reliability. We examined whether prevalence of use of each category differed by gate condition and whether gate effects were moderated by participant demographics.
Results
Prevalence of use of DOx, other stimulants, and non-phenethylamine psychedelics was higher without a gate question. Gate effects for other stimulants and non-phenethylamine psychedelics were larger among white participants and those attending parties less frequently. Almost one in ten (9.3%) participants reporting no “bath salt” use via the gate question later reported use of a “bath salt” such as mephedrone, methedrone, or methylone.
Conclusion
Omitting gate questions may improve accuracy of data collected via self-report.
Keywords: New psychoactive substances, survey methodology, nightclub party attendees, synthetic cathinones
Introduction
Hundreds of new psychoactive substances (NPS) have emerged in recent years in the United States and throughout much of the world (1–3). Determining the prevalence of use of these new compounds is essential to guide prevention efforts. However, most monitoring of NPS use focuses on seizure data (1,2) or data on reported poisonings (4–7). While use of NPS is often unintentional or unknown as they are often adulterants in other more traditional drugs such as ecstasy (8–12), research is needed to inform survey researchers how to appropriately query known use of such drugs. In this paper, we test gate questions as a method of reducing respondent burden when presented with lists of potentially unfamiliar NPS and some other uncommon drugs.
Electronic surveys are now the most common method for recording study participants’ survey responses. Electronic surveys have many advantages and a main advantage is use of skip-logic. Skip-logic is when an individual is asked a question and his or her response determines whether follow-up questions will be asked regarding that topic. This method reduces participant burden as they will not be asked irrelevant follow-up questions and it allows researchers to easily query a wide variety of phenomena (13). Skip-logic also makes data management and analysis easier for researchers—in part, because skip-logic prevents contradictory answers (e.g., reporting no lifetime ecstasy use, but then later reporting past-year ecstasy use). However, studies have found that measurement error can occur when skip-patterns are used. For example, a study focusing on the National Household Survey on Drug Abuse (NHSDA) found that the prevalence of past-year self-reported cocaine use increased when participants were provided multiple chances to report use (14), and other studies have discovered measurement error when skip-patterns were used to assess psychiatric symptoms in a diagnostic manner (13,15).
Gate questions have been utilized for the National Survey of Drug Use and Health (NSDUH; formerly the NHSDA) since 1999 (16). Gate questions on such drug surveys ask a single yes/no question to establish whether the participant has ever used a particular drug or drug from a particular drug class. If “yes” is answered then subsequent questions are provided on the following page. Answering “no” allows the participant to skip the follow-up questions and proceed to the following section (16). While gate questions on surveys such as the NSDUH have been used successfully for some time, there is little research examining the impact of gate questions on prevalence and accuracy of reporting. In addition, it may be assumed that most participants in major drug surveys have at least heard of the majority of drugs queried and it is unknown how participants respond to lists of largely unfamiliar drugs.
Few surveys query use of any NPS. Monitoring the Future (MTF), a nationally representative survey of high school students, queries use of “synthetic marijuana” (synthetic cannabinoids) and “bath salts” (synthetic cathinones) (17–19), but as single items representing full NPS categories. However, many individuals may be unaware what specific substances are classified as “bath salts” and examples (e.g., methylone) are not provided. To our knowledge, the annual Global Drug Survey (GDS) is the only large-scale drug survey to query use of dozens of individual NPS (20) and GDS utilizes lists without gate questions.
In order to inform researchers how to most efficiently query use via lists of individual NPS, we examined whether there were differences in self-reported prevalence of use in a high-risk population—electronic dance music (EDM) party attendees (20–22). High prevalence of NPS use in this scene allows us to adequately compare self-reported use according to gate condition.
Methods
Participants and procedure
We surveyed 1,048 individuals entering EDM parties in New York City throughout the summer of 2016 using time–space sampling (23). Individuals were eligible if they (1) were about to attend the randomly selected party and (2) identified as aged 18–40. Each week, a list of EDM venues and parties planned for that week (typically Thursday through Saturday) were created. Specific venues that hold EDM parties on consistent nights each week were included in the sample space as were parties that were (1) recommended by key informants and/or (2) listed on a popular EDM ticket website as having at least 15 tickets purchased in advance. Dance festivals and “secret location” warehouse parties were included.
Trained recruiters approached passersby (alone or in groups) and asked if they were going to the randomly selected party. Those confirmed eligible were asked if they would take a drug survey. Participants provided informed consent and completed the self-administered survey on tablets. Participants who completed the survey were compensated $10 cash, and the response rate for those believed to be eligible was 77%. This study was approved by the first authors’ institutional review board.
Measures
Participants were first asked about demographic characteristics (e.g., age, gender, race/ethnicity, educational attainment) and level of nightclub/festival attendance over the past year. The survey then informed participants that the following section was going to ask about “use of relatively uncommon synthetic drugs, some of which are called ‘legal highs’ or research chemicals”. Participants were asked about known use of drugs within eight separate categories. Below each category name was a list of individual compounds within that class, along with street names if available. For example, the NBOMe (pronounced “N-bomb”) category was accompanied with a list of six compounds (Figure 1). The Qualtrics survey program randomized participants to the gate question or to no gate question and survey recruiters were blind to assigned condition. Those who received and checked off “yes” to a gate question were taken to a similar page, but with checkboxes next to each drug name for participants to check off which compound(s) they had used. Checking off “no” to a gate question directed the participant to the next gate question for another drug class. The only exception was that all participants randomized to gate questions (N = 520) were asked about use of specific “bath salts”, regardless of their answer to the gate question, to examine reliability of responses.
Figure 1.
Comparison of NBOMe gate vs. non-gate survey page.
Note. (A) represents a gate page in which answering “no” takes the participant to the following drug class and answering “yes” takes the participant to a nearly identical page (B) that includes check-boxes to indicate which NBOMe compound(s) was used. Those not randomized to the gate page (A) are queried directly with the non-gate page (B).
Those randomized to no gate question received the identical page with the checklist as those who checked “yes” to the gate question. Only “yes” boxes were included in the checklists; thus participants were not required to check off “no” for every compound not used. If the participant did not use any listed compounds, he or she was to check off “I have not used any of the above” in order to proceed.
Participants were thus queried about 145 compounds categorized into the following eight classes: NBOMe (e.g., 25i-NBOMe; n = 6), 2C (e.g., 2C–B, n = 23), “bath salts” (synthetic cathinones; e.g., alpha-PVP, n = 26), DOx (e.g., DOB, n = 5), other stimulants (e.g., 4-FA, n = 37), trypta-mines (e.g., 4-MeO-DMT, n = 27), dissociatives (e.g., methoxetamine [MXE], n = 12), and other (non-phe-nethylamine) psychedelics (e.g., AL-LAD, n = 9). Those who checked off that they used a specific compound were taken to an additional page listing the compound(s) checked off, asking if they had used within the past 12 months. Those reporting use of one or more compounds in a category were coded as using a drug in that category and this was done for both lifetime and past-year use. After participants were queried about NPS, they were asked about traditional drugs such as ecstasy/3,4-methy-lenedioxymethamphetamine (MDMA)/Molly, lysergic acid diethylamide (LSD), and ketamine.
Analyses
We first examined descriptive statistics, and using chi-square, we then compared self-reported lifetime and past-year prevalence of reported use of any compound within each category according to whether the participant received a gate question. Effect size (Φ) was also computed for each comparison. To determine whether any differences existed regarding specific compounds, we repeated these analyses for compounds with self-reported lifetime prevalence of at least 2%. Regarding “bath salt” use within the subsample receiving the gate question, we compared self-reported use of any “bath salts” (checking off “yes” to the gate question) to checking off use of any “bath salt” on the checklist which was provided regardless of their answer to the gate question. We also determined whether any specific compounds in this class were more likely to be reported in a discordant manner (i.e., reporting no “bath salt” use, but then reporting use of a specific “bath salt” on the following page). Finally, to examine whether there were differences in the magnitude of gate effects according to age, sex, race/ethnicity, educational attainment, level of party attendance, and lifetime use of ecstasy/MDMA/Molly, LSD, and ketamine, we computed separate logistic regression models specifying an interaction between gate (yes/no) and the covariate, with reported lifetime use of a compound within the NPS class as the outcome. We then repeated analyses focusing on past-year use as the outcome. When an interaction was significant, we presented gate effects stratified by categories of the covariate with a significant interaction. All bivariable comparisons were conducted using chi-square and Fisher’s exact test and all statistics were computed using Stata SE 13 (StataCorp, 2009).
Results
Sample characteristics are presented in Table 1 and self-reported prevalence of use of compounds in each drug class according to gate condition is presented in Table 2. Lifetime prevalence of DOx (p = 0.002), other stimulants (p = 0.037), and other (non-phenethylamine) psychedelics (p = 0.023) was significantly higher among those not queried with an initial gate question. Similarly, past-year DOx use (p = 0.038) was higher among those not assessed initially with a gate question. Differences in lifetime prevalence according to gate question randomization varied by as much as 4.7%. In Table 3, we present comparisons of prevalence according to gate condition for the 15 compounds with at least 2% lifetime prevalence. 2C-B was the only compound with significantly higher reported prevalence among those who did not receive the gate question (p = 0.035).
Table 1.
Sample characteristics (N = 1,048).
| Full sample N | Full sample % | Gate % | No gate % | |
|---|---|---|---|---|
| Age* | ||||
| 18–24 | 602 | 57.2 | 53.5 | 60.7 |
| 25–40 | 446 | 42.8 | 46.5 | 39.0 |
| Sex | ||||
| Male | 602 | 57.4 | 54.4 | 60.4 |
| Female | 446 | 42.6 | 45.6 | 39.6 |
| Race/ethnicity | ||||
| White | 625 | 59.6 | 60.0 | 59.3 |
| Black | 76 | 7.3 | 7.7 | 6.8 |
| Hispanic | 146 | 13.9 | 12.5 | 15.3 |
| Asian | 109 | 10.4 | 11.2 | 9.7 |
| Other | 92 | 8.8 | 8.7 | 8.9 |
| Educational attainment | ||||
| Less Than bachelor’s degree | 504 | 48.1 | 47.1 | 48.1 |
| Bachelor’s degree or higher Party attendance | 544 | 51.9 | 52.9 | 51.9 |
| Less than biweekly | 630 | 60.1 | 60.2 | 60.0 |
| Biweekly or more often | 418 | 39.9 | 39.8 | 40.0 |
| Lifetime party drug use | ||||
| Ecstasy/MDMA/Molly | 640 | 61.2 | 63.0 | 59.5 |
| LSD | 298 | 28.5 | 30.3 | 26.8 |
| Ketamine | 223 | 21.3 | 22.5 | 20.1 |
p < .05.
Table 2.
Self-reported prevalence of use of any compounds in each NPS class according to gate question condition.
| NPS category | Self-reported use | Prevalence of full sample % (95% CI) | Gate (a) % (95% CI) | No gate (b) % (95% CI) | Difference (a – b) | Φ | p |
|---|---|---|---|---|---|---|---|
| NBOMe | Lifetime | 5.0 (3.6, 6.3) | 4.2 (2.5, 6.0) | 5.7 (3.7, 7.7) | −1.5 | .03 | 0.279 |
| Past year | 1.4 (0.7, 2.2) | 1.4 (0.4, 2.3) | 1.5 (0.5, 2.6) | −0.1 | .01 | 0.818 | |
| 2C series | Lifetime | 11.3 (9.3, 13.2) | 11.5 (8.8, 14.3) | 11.0 (8.3, 13.7) | 0.5 | .01 | 0.777 |
| Past year | 5.1 (3.7, 6.4) | 3.9 (2.2, 5.5) | 6.3 (4.2, 8.3) | −2.4 | .05 | 0.076 | |
| “Bath salts”± | Lifetime | 11.4 (9.4, 13.3) | 12.1 (9.3, 14.9) | 10.6 (8.0, 13.2) | 1.5 | .02 | 0.441 |
| Past year | 3.7 (2.6, 4.9) | 3.3 (1.7, 4.8) | 4.2 (2.5, 5.9) | −0.9 | .02 | 0.443 | |
| DOx series | Lifetime | 2.5 (1.5, 3.4) | 1.0 (0.1, 1.8) | 4.0 (2.3, 5.6) | −3.0 | .10 | 0.002 |
| Past year | 1.1 (0.5, 1.8) | 0.4 (0.0, 0.9) | 1.9 (0.7, 3.1) | −1.5 | .07 | 0.038 | |
| Other stimulants | Lifetime | 13.4 (11.3, 15.4) | 11.2 (8.4, 13.9) | 15.5 (12.4, 18.6) | −4.3 | .06 | 0.037 |
| Past year | 8.8 (7.1, 10.5) | 7.7 (5.4, 10.0) | 9.9 (7.3, 12.4) | −2.2 | .04 | 0.217 | |
| Tryptamines | Lifetime | 8.0 (6.4, 9.7) | 8.5 (6.1, 10.9) | 7.6 (5.3, 9.8) | 0.9 | .02 | 0.597 |
| Past year | 3.6 (2.5, 4.8) | 4.0 (2.3, 5.7) | 3.2 (1.7, 4.7) | 0.8 | .02 | 0.478 | |
| Dissociatives | Lifetime | 10.6 (8.7, 12.5) | 9.8 (7.2, 12.4) | 11.4 (8.6, 14.1) | −1.6 | .03 | 0.413 |
| Past year | 6.6 (5.1, 8.1) | 6.5 (4.4, 8.7) | 6.6 (4.5, 8.8) | −0.1 | .00 | 0.953 | |
| Other psychedelics | Lifetime | 13.2 (11.1, 15.2) | 10.8 (8.1, 13.4) | 15.5 (12.4, 18.6) | −4.7 | .07 | 0.023 |
| Past year | 8.5 (6.8, 10.2) | 7.7 (5.4, 10.0) | 9.3 (6.8, 11.8) | −1.6 | .03 | 0.357 |
“Other” psychedelics are non-phenethylamine psychedelics. ± Specific “bath salts” were assessed on the following page regardless of their answer to the “bath salt” gate question. CI = confidence interval, Φ (phi) = effect size.
Table 3.
Specific NPS and other uncommon drugs with lifetime prevalence of 2% or higher.
| NPS category | Self-reported use | Prevalence of full sample % (95% CI) | Gate (a) % (95% CI) | No gate (b) % (95% CI) | Difference (a – b) | Φ | p |
|---|---|---|---|---|---|---|---|
| 25i-NBOMe | Lifetime | 3.1 (2.0, 4.1) | 2.3 (1.0, 3.6) | 3.8 (2.2, 5.4) | −1.5 | .04 | 0.164 |
| Past year | 0.7 (0.2, 1.2) | 0.2 (0.0, 0.6) | 1.1 (0.2, 2.0) | −0.9 | .06 | 0.124 | |
| 2C-B | Lifetime | 6.1 (4.7, 7.6) | 6.7 (4.6, 8.9) | 5.5 (3.5, 7.4) | 1.2 | .03 | 0.403 |
| Past year | 3.1 (2.0, 4.1) | 1.9 (0.7, 3.1) | 4.2 (2.5, 5.9) | −2.3 | .07 | 0.035 | |
| 2C-E | Lifetime | 2.4 (1.5, 3.3) | 2.7 (1.3, 4.1) | 2.1 (0.9, 3.3) | 0.6 | .02 | 0.518 |
| Past year | 0.7 (0.2, 1.2) | 0.4 (0.0, 0.9) | 1.0 (0.1, 1.8) | −0.6 | .03 | 0.452 | |
| 2C-I | Lifetime | 4.1 (2.9, 5.3) | 4.0 (2.3, 5.7) | 4.2 (2.5, 5.9) | −0.2 | .00 | 0.917 |
| Past year | 1.0 (0.4, 1.5) | 0.8 (0.0, 1.5) | 1.1 (0.2, 2.0) | −0.3 | .02 | 0.753 | |
| “Bath salt” unknown | Lifetime | 2.8 (1.8, 3.8) | 3.1 (1.6, 4.6) | 2.5 (1.1, 3.8) | 0.6 | .02 | 0.544 |
| Past year | 0.5 (0.1,0.9) | 0.6 (0.0, 1.2) | 0.4 (0.0, 0.9) | 0.2 | .01 | 0.685 | |
| Mephedone | Lifetime | 2.3 (1.4, 3.2) | 1.7 (0.6, 2.9) | 2.8 (1.4, 4.3) | −1.1 | .04 | 0.230 |
| Past year | 0.5 (0.1, 0.9) | 0.4 (0.0, 0.9) | 0.6 (0.0, 1.2) | −0.2 | .01 | 1.000 | |
| Methedrone | Lifetime | 2.4 (1.5, 3.3) | 3.1 (1.6, 4.6) | 1.7 (0.6, 2.8) | 1.4 | .05 | 0.145 |
| Past year | 0.6 (0.1, 1.0) | 0.4 (0.0, 0.9) | 0.8 (0.0, 1.5) | −0.4 | .02 | 0.687 | |
| Methylone | Lifetime | 4.0 (2.8, 5.2) | 3.3 (1.7, 4.8) | 4.7 (2.9, 6.6) | −1.4 | .04 | 0.226 |
| Past year | 1.7 (0.9, 2.5) | 1.4 (0.4, 2.3) | 2.1 (0.9, 3.3) | −0.7 | .03 | 0.358 | |
| MDA | Lifetime | 9.0 (7.2, 10.7) | 7.7 (5.4, 10.0) | 10.2 (7.6, 12.8) | −2.5 | .04 | 0.151 |
| Past year | 6.3 (4.8, 7.8) | 5.8 (3.8, 7.8) | 6.8 (4.7, 9.0) | −1.0 | .02 | 0.485 | |
| DMT | Lifetime | 5.6 (4.2, 7.0) | 6.2 (4.1, 8.2) | 5.1 (3.2, 7.0) | 1.1 | .02 | 0.465 |
| Past year | 2.6 (1.6, 3.5) | 2.7 (1.3, 4.1) | 2.5 (1.1, 3.8) | 0.2 | .01 | 0.814 | |
| 2-MeO-ketamine | Lifetime | 7.4 (5.8, 8.9) | 6.9 (4.7, 9.1) | 7.8 (5.5, 10.1) | −0.9 | .02 | 0.601 |
| Past year | 4.9 (3.6, 6.2) | 4.8 (3.0, 6.7) | 4.9 (3.1, 6.8) | −0.1 | .00 | 0.930 | |
| NKET | Lifetime | 2.2 (1.3, 3.1) | 2.1 (0.9, 3.4) | 2.3 (1.0, 3.5) | −0.2 | .01 | 0.862 |
| Past year | 1.7 (0.9, 2.5) | 1.5 (0.5, 2.6) | 1.9 (0.7, 3.1) | −0.4 | .01 | 0.658 | |
| 1P-LSD | Lifetime | 6.7 (5.2, 8.2) | 5.6 (3.6, 7.6) | 7.8 (5.5, 10.1) | −2.2 | .04 | 0.156 |
| Past year | 4.4 (3.1, 5.6) | 3.5 (1.9, 5.0) | 5.3 (3.4, 7.2) | −1.8 | .04 | 0.146 | |
| LSZ | Lifetime | 3.7 (2.6, 4.9) | 4.0 (2.3, 5.7) | 3.4 (1.9, 5.0) | 0.6 | .02 | 0.590 |
| Past year | 2.1 (1.2, 3.0) | 2.5 (1.2, 3.8) | 1.7 (0.6, 2.8) | 0.8 | .03 | 0.369 | |
| Psychedelic unknown | Lifetime | 5.8 (4.4, 7.2) | 4.8 (3.0, 6.7) | 6.8 (4.7, 9.0) | −2.0 | .04 | 0.165 |
| Past year | 3.6 (2.5, 4.8) | 3.1 (1.6, 4.6) | 4.2 (2.5, 5.9) | −1.1 | .03 | 0.345 |
25i-NBOMe is in the NBOMe class; 2C-B, 2C-E, and 2C-I are in the 2C class; “bath salt” unknown, mephedrone, methedrone, and methylone “bath salt are in the “ class; MDA is in the other stimulant class; DMT is in the tryptamine class; 2-MeO-ketamine and N-Ethyl-Ketamine (NKET) are in the dissociative class; and 1P-LSD, Lysergic acid 2,4-dimethylazetidide (LSZ), and psychedelic unknown are in the other (non-phenethylamine) psychedelic class. CI = confidence interval, Φ (phi) = effect size.
With regard to the half of the sample who received the gate question for “bath salts” and were then asked about use of specific compounds in this class afterward, of those who reported use via the gate question (12.1% of the subsample), the majority (63.0%) of these also checked off use of a specific “bath salt” from the list on the following page (Table 4). Of those who reported no use via the gate question, 9.3% then checked off use of at least one “bath salt” on the following page. The sensitivity of the gate question correctly classifying a concordant report of an affirmative response on the list was 27.0%, but the specificity of the gate question correctly classifying a concordant response denying use was 97.8%. The most prevalent “bath salts” used in the sample were more likely to be reported after answering “no” to the gate question. Specifically, use of mephedrone, methedrone, methylone, and “bath salt” unknown was more likely to be reported after reporting no “bath salt” use via the gate question (ps < .001).
Table 4.
Self-reported prevalence of lifetime use of specific “bath salts” comparing initial gate question responses with the “bath salt” checklist.
| Prevalence among those receiving gate question % (95% CI) | Reported use after reporting no “bath salt” use % (95% CI) | |
|---|---|---|
| “Bath salts” any | 12.1 (9.3, 14.9) | 9.3 (6.8, 11.9) |
| Mephedrone | 1.7 (0.6, 2.9) | 1.2 (0.2, 2.2) |
| Methedrone | 3.1 (1.6, 4.6) | 2.4 (1.1, 3.8) |
| Methylone | 3.3 (1.7, 4.8) | 2.4 (1.1, 3.8) |
| “Bath salt” unknown | 3.1 (1.6, 4.6) | 1.0 (0.1, 1.9) |
This analysis only focuses on the half of the sample (N = 520) who received the gate question for “bath salts”. All participants in this subsample received the separate list of “bath salts” after the gate question regardless of their answer to the preceding gate question. CI = confidence interval.
All ps < .001.
Finally, we determined whether there were any differences in gate effects by covariates. Results of interaction models suggested significant differences by race/ ethnicity (white vs. non-white) and level of party attendance (less than biweekly vs. biweekly or more) with regard to both lifetime and past-year use of other stimulants and other (non-phenethylamine) psychedelics. As shown in Table 5, for both drug classes, participants identifying as white or as less-frequent party attendees were more likely to have higher reported prevalence in absence of the gate question compared to non-whites and more frequent attendees, respectively. In addition, white participants were more likely to have higher reported past-year prevalence of 2C series drugs when not assessed via gate question.
Table 5.
Self-reported prevalence stratified by race/ethnicity and level of party attendance based on significant interaction models.
| Full sample % (95% CI) | Gate (a) % (95% CI) | No gate(b) % (95% CI) | Difference (a – b) | Φ | p | |
|---|---|---|---|---|---|---|
| Other stimulants | ||||||
| Lifetime use | ||||||
| Non-white | 11.8 (8.7, 14.9) | 13.0 (8.4, 17.6) | 10.7 (6.5, 14.9) | 2.3 | .04 | 0.467 |
| White | 14.4 (11.6, 17.2) | 9.9 (6.6, 13.3) | 18.8 (14.5, 23.2) | −9.0 | .13 | 0.002 |
| Attend parties less than biweekly | 9.2 (6.9, 11.5) | 5.4 (2.9, 8.0) | 12.9 (9.2, 16.6) | −7.5 | .13 | 0.001 |
| Attend parties biweekly or more | 19.6 (15.8, 23.4) | 19.8 (14.3, 25.3) | 19.4 (14.0, 24.8) | 0.4 | .00 | 0.923 |
| Past-year use | ||||||
| Non-white | 6.4 (4.0, 8.7) | 7.7 (4.0, 11.3) | 5.1 (2.1, 8.1) | 2.6 | .05 | 0.279 |
| White | 10.4 (8.0, 12.8) | 7.7 (4.7, 10.7) | 13.1 (9.3, 16.9) | −5.4 | .09 | 0.027 |
| Attend parties less than biweekly | 5.7 (3.9, 7.5) | 3.5 (1.5, 5.6) | 7.9 (4.9, 10.9) | −4.4 | .09 | 0.025 |
| Attend parties biweekly or more | 13.4 (10.1, 16.7) | 14.0 (9.2, 18.8) | 12.8 (8.3, 17.3) | 1.2 | .02 | 0.716 |
| Other psychedelics | ||||||
| Lifetime use | ||||||
| Non-white | 13.0 (9.8, 16.2) | 13.9 (9.2, 18.7) | 12.1 (7.7, 16.5) | 1.8 | .03 | 0.572 |
| White | 13.3 (10.6, 15.9) | 8.7 (5.5, 11.8) | 17.9 (13.6, 22.2) | −9.2 | .14 | 0.001 |
| Attend parties less than biweekly | 9.7 (7.4, 12.0) | 5.8 (3.2, 8.3) | 13.6 (9.8, 17.4) | −7.8 | .13 | 0.001 |
| Attend parties biweekly or more | 18.4 (14.7, 22.2) | 18.4 (13.0, 23.7) | 18.5 (13.2, 23.8) | −0.1 | .00 | 0.974 |
| Past-year use | ||||||
| Non-white | 8.0 (5.4, 10.6) | 9.6 (5.6, 13.7) | 6.5 (3.2, 9.8) | 3.1 | .06 | 0.240 |
| White | 8.8 (6.6, 11.0) | 6.4 (3.7, 9.1) | 11.2 (7.7, 14.7) | −4.8 | .08 | 0.035 |
| Attend parties less than biweekly | 5.4 (3.6, 7.2) | 2.9 (1.0, 4.7) | 7.9 (4.9, 10.9) | −7.8 | .11 | 0.005 |
| Attend parties biweekly or more | 13.2 (9.9, 16.4) | 15.0 (10.1, 19.9) | 11.4 (7.1, 15.7) | −0.1 | .05 | 0.276 |
| 2C series | ||||||
| Past-year use | ||||||
| Non-white | 3.8 (2.0, 5.6) | 4.8 (1.9, 7.7) | 2.8 (0.6, 5.0) | 2.0 | .05 | 0.277 |
| White | 5.9 (4.1, 7.8) | 3.2 (1.2, 5.2) | 8.6 (5.5, 11.8) | −5.4 | .11 | 0.004 |
Stratification of self-reported prevalence was based on differences derived from significant interaction models using logistic regression. “Other” psychedelics are non-phenethylamine psychedelics. Φ (phi) = effect size, OR = odds ratio, CI = confidence interval. Stratified tests were conducted based on significant interaction effects in logistic regression models. Specifically, gate effects were larger for whites (vs. non-whites) for both lifetime (OR = 0.38, 95% CI = 0.18, 0.81) and past-year use (OR = 0.36, 95% CI = 0.14, 0.93) of other stimulants when not receiving the gate question, and a similar interaction was found for whites (vs. non-whites) for both lifetime (OR = 0.37, 95% CI = 0.17, 0.78) and past-year use (OR = 0.36, 95% CI = 0.14, 0.89) of other psychedelics, and for past-year use (OR = 0.20, 95% CI = 0.06, 0.71) of 2C series. Gate effects on reported use of other stimulants were also higher for infrequent attendees (vs. frequent attendees) for both lifetime (OR = 2.65, 95% CI = 1.24, 5.67) and past-year use (OR = 2.61, 95% CI = 1.04, 6.55), and a similar interaction was found for infrequent attendees (vs. frequent attendees) for both lifetime (OR = 2.55, 95% CI = 1.20, 5.44) and past-year use (OR = 3.97, 95% CI = 1.51, 10.43) of other psychedelics.
Discussion
As NPS continue to emerge, it is important to determine the most effective methods to query use via surveys. In this paper, we examined whether there were differences in self-reported prevalence of use in a high-risk population according to whether lists of compounds were preceded by a gate question.
Of the eight drug classes queried, participants reported higher prevalence of DOx, other stimulants, and other (non-phenethylamine) psychedelics when the checklist of compounds in those classes was not preceded by a gate question. Some differences in reported prevalence were as high as 4.7%. The difference for other stimulants was less unexpected as it is a longer and more heterogeneous list (name-wise) compared to some other categories (e.g., all 2C drugs begin with “2C”), and lists for the other two categories were shorter. Additionally, we found that reported prevalence of 2C-B was higher when the gate for 2C drugs was not received. While detected gate effects tended to be modest, we believe these differences in prevalence are in fact meaningful as differences of 1–5% suggest imprecise detection of prevalence of these rare substances. We are unable to determine under- or over-reporting in this study (other than for “bath salts”), but it appears that providing participants with the actual list of drugs initially may be associated with more attention paid to that list and it also may reduce survey burden. It is unknown, however, whether requiring participants to check “no” for drugs they did not use would be associated with them paying closer attention and recognizing additional drugs they had used.
Importantly, we uncovered that almost one out of ten participants who reported no “bath salt” use as per the gate question then reported use of one or more compounds in this class. An older study of the NHSDA found that prevalence of past-year self-reported cocaine use increased when participants were provided multiple chances to report use (14). Our findings do provide evidence of underreporting of “bath salts” and we believe they have important implications for other studies that query use. Studies such as MTF that ask about “bath salt” use do not list what drugs constitute “bath salts”. Technically, these compounds are called synthetic cathinones, but even many people in scenes experienced with specific compounds such as methy-lone are either unaware that these compounds are in this category or they did not read the full list of compounds. A recent analysis of MTF had similar findings in which among high school seniors denying nonmedical opioid use, 37% later reported nonmedical Vicodin use and 28% reported nonmedical OxyContin use later in the survey despite these two drugs being presented in the definition of opioids in the initial question (24). It is unknown whether the discrepancies in these two studies are due to lack of attention to the initial question/definition or due to confusion regarding which drugs are in the corresponding class (despite the drugs being listed). It is possible that since “bath salts” are now highly stigmatized drugs—reported by the media as turning users violent or into cannibals (25)—that many participants merely disregarded the list of compounds upon seeing the class name. We believe that estimates of “bath salt” use are likely underestimates when specific compounds were not queried (17,19,26).
We believe lower reported prevalence for drugs initially assessed with gate questions might also be a result of satisficing. Satisficing is when participants do not utilize full effort due to disinterest, fatigue, lack of motivation, impatience, distraction, perceived unimportance, and/or survey burden or difficulty (27,28). When satisficing occurs, participants expend less energy, they tend to be less thorough in comprehension, judgment, memory retrieval, and response selection, and may deliberately or non-deliberately find shortcuts in order to reduce workload and finish quickly (27,28). For example, some participants may check off “no” to gate questions upon discovering it allows them to skip follow-up questions. An older study of NSDUH found such occurrences (29). Gate questions also add an extra page for each drug group, which may further increase chances for exhaustion. Satisficing also can include primacy effects in which participants may essentially ignore the lower end of lists (28,30), so some participants may essentially only glance at compounds at the top of lists. Longer and unfamiliar words can also lead to satisficing (27,28) and studies have found that many participants do not even read definitions of survey concepts (30), so some might have been confused when presented with names of dozens of unfamiliar compounds. Despite participants receiving compensation, it is likely that satisficing was still a factor in this survey.
Past research has found that less educated participants are more likely to satisfice on surveys (28,31), but level of education was not a significant factor in this analysis. However, we found that white and less-frequent party attending participants were more likely to report higher prevalence via non-gate questions for some drug classes. More research is needed to determine why these differences arose, but it seems plausible that less-frequent attendees would be less familiar with many of the compounds listed and thus potentially more confused. Participants are more likely to check “no” when they are presented with an unfamiliar phenomenon (27), so it should likely be assumed that participants who deem a drug unfamiliar would report no use. However, interestingly, some participants sought assistance from recruiters saying they did not know how to proceed when they were presented with lists of unfamiliar drugs.
In light of these findings, we recommend researchers who wish to query NPS to group compounds together by class but to keep lists as short as possible. NSDUH asks about 18 different opioids via one gate question (as the participant looks at a card containing labeled photos of each opioid), and gate questions are also used to query use of 13 tranquilizers, 12 stimulants, and 8 sedatives (32). GDS, on the other hand, recently listed as many as 80 NPS on a single survey page and without any gate questions, but satisficing on GDS is likely not an issue because it is a self-selected online sample without compensation. Regardless, shorter eye-friendly lists without gate pages may hold participants’ attention regarding a subject they may find uninteresting and at a time they are about to enter a party. Researchers can also maximize participant motivation by explaining the purpose (and importance of) the study, include the most important questions early (before potential survey fatigue), ensure familiarity with as many terms as possible, keep questions simple, minimize distraction, and try to offer adequate compensation for their time (33). We referred to synthetic cathinones as “bath salts” throughout our survey as this is the common street name for these compounds. We feel it is important to list as many familiar drug names as possible (e.g., street names). For example, a participant may not have previously realized that “M1” is a street name for the synthetic cathinone called methylone, similar to how it has also been found that many adolescents appear to be unaware that “Molly” is a street name for ecstasy/MDMA (34). Thus, a key finding was that classes must be defined and drugs in these classes need to be listed.
Limitations
Results may not be fully generalizable to the general population as this sample consisted of nightclub and dance festival attendees and this population tends to have high rates of NPS use (20–22). It is also possible that some participants “correctly” reported what drug they believe they used, but they could have actually used a drug that was replaced by or adulterated with one or more different drugs. For example, a recent study found that many ecstasy/Molly users who deny use, of NPS such as “bath salts” actually tested positive for these substances (12). We could only examine reliability with regard to “bath salt” use, and under- and overreporting cannot adequately be inferred. Not all NPS are actually “new” and definitions of NPS vary. Hundreds (e.g., 2Cx, and DOx) were discovered decades ago (35,36), so many drugs commonly classified as “NPS” are simply less common drugs. We do not refer to all compounds in this study as NPS, as classes contained at least two compounds which are uncommon, but not always deemed NPS—DMT (N,N-dimethyltryptamine) and 3,4-methylenedioxyamphetamine (MDA). Even though DMT was discovered in 1931, numerous studies have categorized it as an NPS (20,22,37–41) and the annual GDS found that the popularity of DMT has recently boomed (42). In addition, while MDA was first discovered over a century ago, it is not queried via major drug surveys, and it has recently regained popularity (under the names Sassafras, Sass, and Sally) (43), so we included it with NPS while programming the survey.
Conclusion
While under- and overreporting could not be determined, results of this study suggest excluding gate questions when querying groups of NPS or other uncommon drugs may lead to more reliable responses. The inclusion of gate pages adds more survey pages for participants with affirmative responses and this may contribute to survey fatigue and thus possibly to satisficing. It appears that some participants are less likely to pay full attention to lists of drugs when a single dichotomous yes/no question is presented. Omitting gate questions appears to increase the likelihood of the participant reading the list of each drug class, as when they do not recognize the drug class—whether “bath salts” or opioids (24)—they may be more likely to be attentive to more items on the checklist. As NPS continue to emerge, research focusing on use of these compounds is essential in order to inform prevention.
Acknowledgments
Funding: This project was funded by the NIH (K01 DA-038800, PI: Palamar; P30 DA011041, PI: Deren).
Footnotes
Declaration of interest
None of the authors have any conflicts of interest to declare.
References
- 1.US Drug Enforcement Administration, Office of Diversion Control. National Forensic Laboratory Information System Special Report: Synthetic Cannabinoids and Synthetic Cathinones Reported in NFLIS, 2013–2015. Springfield, VA: US Drug Enforcement Administration; 2016. [Google Scholar]
- 2.US Drug Enforcement Administration, Office of Diversion Control. National Forensic Laboratory Information System Special Report: 2C-Phenethylamines, Piperazines, and Tryptamines Reported in NFLIS, 2011–2015. Springfield, VA: US Drug Enforcement Administration; 2017. [Google Scholar]
- 3.European Monitoring Centre for Drugs and Drug Addiction. An update from the EU early warning system (March 2015) Luxembourg: Publications Office of the European Union; 2015. New psychoactive substances in Europe. [Google Scholar]
- 4.American Association of Poison Control Centers. Bath Salts Data. 2015. [Google Scholar]
- 5.American Association of Poison Control Centers. Synthetic Cannabinoids Data. 2015. [Google Scholar]
- 6.Forrester MB. NBOMe designer drug exposures reported to Texas poison centers. J Addict Dis. 2014;33:196–201. doi: 10.1080/10550887.2014.950027. [DOI] [PubMed] [Google Scholar]
- 7.Palamar JJ, Su MK, Hoffman RS. Characteristics of novel psychoactive substance exposures reported to New York City Poison Center, 2011–2014. Am J Drug Alcohol Abuse. 2016;42:39–47. doi: 10.3109/00952990.2015.1106551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Brunt TM, Nagy C, Bucheli A, Martins D, Ugarte M, Beduwe C, Ventura VM. Drug testing in Europe: monitoring results of the Trans European Drug Information (TEDI) project. Drug Test Anal. 2016;9:188–198. doi: 10.1002/dta.1954. [DOI] [PubMed] [Google Scholar]
- 9.Caudevilla-Gálligo F, Ventura M, Indave Ruiz BI, Fornís I. Presence and composition of cathinone derivatives in drug samples taken from a drug test service in Spain (2010–2012) Hum Psychopharmacol. 2013;28:341–344. doi: 10.1002/hup.2296. [DOI] [PubMed] [Google Scholar]
- 10.Gine CV, Espinosa IF, Vilamala MV. New psychoactive substances as adulterants of controlled drugs. A worrying phenomenon? Drug Test Anal. 2014;6:819–824. doi: 10.1002/dta.1610. [DOI] [PubMed] [Google Scholar]
- 11.Gine CV, Ventura Vilamala M, Fornis Espinosa I, Gil Lladanosa C, Calzada Alvarez N, Fito Fruitos A, Rodriguez Rodriguez J, et al. Crystals and tablets in the Spanish ecstasy market 2000–2014: are they the same or different in terms of purity and adulteration? Forensic Sci Int. 2016;263:164–168. doi: 10.1016/j.forsciint.2016.04.016. [DOI] [PubMed] [Google Scholar]
- 12.Palamar JJ, Salomone A, Vincenti M, Cleland CM. Detection of “bath salts” and other novel psychoactive substances in hair samples of ecstasy/MDMA/”Molly” users. Drug Alcohol Depend. 2016;161:200–205. doi: 10.1016/j.drugalcdep.2016.02.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Swanson SA, Brown TA, Crosby RD, Keel PK. What are we missing? The costs versus benefits of skip rule designs. Int J Methods Psychiatr Res. 2014;23:474–485. doi: 10.1002/mpr.1396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lessler JT, Caspar RA, Penne MA, Barker PR. Developing computer assisted interviewing (CAI) for the National Household Survey on Drug Abuse. J Drug Issues. 2000;30:9–33. [Google Scholar]
- 15.Barber ME, Marzuk PM, Leon AC, Portera L. Gate questions in psychiatric interviewing: the case of suicide assessment. J Psychiatr Res. 2001;35:67–69. doi: 10.1016/s0022-3956(01)00004-8. [DOI] [PubMed] [Google Scholar]
- 16.Gfroerer J, Eyerman J, Chromy J. Redesigning an ongoing National Household Survey: methodological Issues. Rockville, MD: Substance Abuse and Mental Health Services Administration, US Dept of Health and Human Services; 2002. [Google Scholar]
- 17.Miech RA, Johnston LD, O’Malley PM, Bachman JG, Schulenberg JE. Monitoring the future national survey results on drug use, 1975–2015: vol. I, Secondary school students. Ann Arbor, MI: Institute for Social Research, The University of Michigan; 2016. [Google Scholar]
- 18.Palamar JJ, Acosta P. Synthetic cannabinoid use in a nationally representative sample of US high school seniors. Drug Alcohol Depend. 2015;149:194–202. doi: 10.1016/j.drugalcdep.2015.01.044. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Palamar JJ. “Bath salt” use among a nationally representative sample of high school seniors in the United States. Am J Addict. 2015;24:488–491. doi: 10.1111/ajad.12254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Palamar JJ, Barratt MJ, Ferris JA, Winstock AR. Correlates of new psychoactive substance use among a self-selected sample of nightclub attendees in the United States. Am J Addict. 2016;25:400–407. doi: 10.1111/ajad.12403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Palamar JJ, Acosta P, Sherman S, Ompad DC, Cleland CM. Self-reported use of novel psychoactive substances among attendees of electronic dance music venues. Am J Drug Alcohol Abuse. 2016;42:624–632. doi: 10.1080/00952990.2016.1181179. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Palamar JJ, Martins SS, Su MK, Ompad DC. Self-reported use of novel psychoactive substances in a US nationally representative survey: prevalence, correlates, and a call for new survey methods to prevent under-reporting. Drug Alcohol Depend. 2015;156:112–119. doi: 10.1016/j.drugalcdep.2015.08.028. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.MacKellar DA, Gallagher KM, Finlayson T, Sanchez T, Lansky A, Sullivan PS. Surveillance of HIV risk and prevention behaviors of men who have sex with men: a national application of venue-based, time-space sampling. Public Health Rep. 2007;122:39–47. doi: 10.1177/00333549071220S107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Palamar JJ, Shearston JA, Cleland CM. Discordant reporting of nonmedical opioid use in a nationally representative sample of US high school seniors. Am J Drug Alcohol Abuse. 2016;42:530–538. doi: 10.1080/00952990.2016.1178269. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Dahl J. CBS News. 2012. May 30, Bath salts, drug alleged “face-chewer” Rudy Eugene may have been on, plague police and doctors. [Google Scholar]
- 26.Stogner JM, Miller BL. Investigating the ‘bath salt’ panic: the rarity of synthetic cathinone use among students in the United States. Drug Alcohol Rev. 2013;32:545–549. doi: 10.1111/dar.12055. [DOI] [PubMed] [Google Scholar]
- 27.Tourangeau R, Rips LJ, Rasinski K. The pyschology of survey response. 6. New York, New York: Cambridge University Press; 2006. [Google Scholar]
- 28.Krosnick JA. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cognit Psychol. 1991;5:213–236. [Google Scholar]
- 29.Gfroerer J, Bose J, Painter D, Jones M, Kennet J. Redesigning the National Survey on Drug use and health. 2009 URL https://s3.amazonaws.com/sitesusa/wp-content/uploads/sites/242/2014/05/2009FCSM_Gfroerer_X-B.pdf.
- 30.Galesic M, Tourangeau R, Couper MP, Conrad FG. Eye-tracking data: new insights on response order effects and other cognitive shortcuts in survey responding. Public Opin Q. 2008;72:892–913. doi: 10.1093/poq/nfn059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Schuman H, Presser S. Questions and answers in attitude surveys: experiments on question form, wording, and context. 2. Thousand Oaks, CA, London, New Delhi: Sage Publications; 1996. [Google Scholar]
- 32.United States Department of Health and Human Services. Substance Abuse and Mental Health Services Administration. Center for Behavioral Health Statistics and Quality. National Survey on Drug Use and Health. Ann Arbor, MI: Inter-University Consortium for Political and Social Research; 2014. [Google Scholar]
- 33.Krosnick J. The threat of satisficing in surveys: the shortcuts respondents take in answering questions. Surv Methods Newslett. 2000;20:1–8. [Google Scholar]
- 34.Palamar JJ, Keyes K, Cleland CM. Underreporting of ecstasy use among high school seniors in the US. Drug Alcohol Depend. 2016;165:279–282. doi: 10.1016/j.drugalcdep.2016.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Shulgin A, Shulgin A. Pihkal: a chemical love story. Berkeley, CA: Transform Press; 1991. [Google Scholar]
- 36.Shulgin A, Shulgin A. Tikhal: the continuation. Berkeley, CA: Transform Press; 1997. [Google Scholar]
- 37.Hondebrink L, Nugteren-Van Lonkhuyzen JJ, Van der Gouwe D, Brunt TM. Monitoring new psychoactive substances (NPS) in The Netherlands: data from the drug market and the Poisons Information Centre. Drug Alcohol Depend. 2015;147:109–115. doi: 10.1016/j.drugalcdep.2014.11.033. [DOI] [PubMed] [Google Scholar]
- 38.Sutherland R, Peacock A, Whittaker E, Roxburgh A, Lenton S, Matthews A, et al. New psychoactive substance use among regular psychostimulant users in Australia, 2010–2015. Drug Alcohol Depend. 2016;161:110–118. doi: 10.1016/j.drugalcdep.2016.01.024. [DOI] [PubMed] [Google Scholar]
- 39.Burns L, Roxburgh A, Matthews A, Bruno R, Lenton S, Van Buskirk J. The rise of new psychoactive substance use in Australia. Drug Test Anal. 2014;6:846–849. doi: 10.1002/dta.1626. [DOI] [PubMed] [Google Scholar]
- 40.Dinger J, Woods C, Brandt SD, Meyer MR, Maurer HH. Cytochrome P450 inhibition potential of new psychoactive substances of the tryptamine class. Toxicol Lett. 2016;241:82–94. doi: 10.1016/j.toxlet.2015.11.013. [DOI] [PubMed] [Google Scholar]
- 41.Bruno R, Matthews AJ, Dunn M, Alati R, McIlwraith F, Hickey S, Burns L, Sindicich N. Emerging psychoactive substance use among regular ecstasy users in Australia. Drug Alcohol Depend. 2012;124:19–25. doi: 10.1016/j.drugalcdep.2011.11.020. [DOI] [PubMed] [Google Scholar]
- 42.Winstock AR, Kaar S, Borschmann R. Dimethyltryptamine (DMT): prevalence, user characteristics and abuse liability in a large global sample. J Psychopharmacol. 2014;28:49–54. doi: 10.1177/0269881113513852. [DOI] [PubMed] [Google Scholar]
- 43.Stuart T. What Is MDA, or sally, one of the drugs Scott Weiland overdosed on? Rolling Stone. 2015 Dec 18; [Google Scholar]