Abstract
Objective Individuals with chronic pain who misuse prescription opioids are at high risk for developing opioid use disorder and/or succumbing to opioid overdose. The current study conducted a survey to evaluate sex-based differences in pain catastrophizing, opioid withdrawal, and current pain in persons with co-occurring chronic pain and opioid misuse. We hypothesized that women with chronic pain who misused prescription opioids would self-report higher pain ratings compared with men and that the relationship between pain catastrophizing and self-reported current pain would be moderated by symptoms of opioid withdrawal in women only.
Design Survey assessment of the relationship between pain and opioid misuse.
Setting Online via Amazon Mechanical Turk.
Participants Persons with ongoing chronic pain who also misused prescription opioids on one or more days in the last 30 days were eligible (N = 181).
Methods Participants completed demographic and standardized assessments including the Brief Pain Inventory (BPI), Pain Catastrophizing Scale (PCS), and Subjective Opiate Withdrawal Scale (SOWS).
Results Women reported higher levels of current (P < 0.001), average (P < 0.001), and worst (P = .002) pain in the last 24 hours compared with men. Women also endorsed higher scores on the PCS (P = 0.006) and marginally higher past-30-day SOWS ratings (P = 0.068) compared with men. SOWS ratings moderated the relationship between PCS and BPI Worst Pain in women (ΔR2 < 0.127, ΔF(1, 78) = 12.39, P = 0.001), but not in men (ΔR2 < 0.000, ΔF(1, 98) = 0.003, P = 0.954).
Conclusions These data suggest a strong relationship between opioid withdrawal, pain catastrophizing, and the experience of pain in women with chronic pain who misuse opioids.
Keywords: Opioids, Opioid Misuse, Chronic Pain, Sex Differences, Opioid Withdrawal
Introduction
Misuse of prescription opioids has fueled the opioid epidemic in America [1]. Public health initiatives have sought to curb the rising rates of opioid use disorder (OUD) by reducing prescription opioid diversion and the incidence of opioid overprescribing [2]. As a result, recent national trends include a decreased number of individuals receiving opioids for pain management between 2006 and 2015, though the average duration for each opioid prescription has increased from 13.3 to 17.6 days [3]. Thus, people are still being exposed to opioids for an extended period of time despite evidence that such therapy is not effective for many individuals with chronic pain [1].
Growing evidence suggests that men and women experience chronic pain differently. Relative to men, women are more likely to report having chronic pain [4], to endorse more locations of pain [5,6], and are more susceptible to cognitive dysfunction associated with chronic pain [7]. In addition, women score higher than men on the Pain Catastrophizing Scale (PCS) [8], which has been shown to mediate sex-based differences in self-reported recent daily pain [6]. Women who meet criteria for OUD are also more likely than men to use opioids to cope with pain and negative affect [9]. However, compared with women, men who have chronic pain may initiate prescription opioid misuse more quickly [10] and source their prescription opioids illicitly [11].
Differences in how pain is experienced may also inform differences in opioid response. A large observational study of chronic pain patients transitioning off opioid therapies reported that opioid use and withdrawal exacerbated pain severity and that improvement in pain severity and functioning was evident after withdrawal [12]. Sex-based differences in opioid withdrawal may also confer unique risk to women using opioids for pain management. Preclinical research on sex-based differences in opioid withdrawal has been equivocal, although studies suggest that males may gain tolerance to opioids more rapidly whereas females may experience elevated levels of opioid withdrawal after acute exposure to opioids [13]. In addition, women in a large clinical trial on supervised withdrawal in persons with OUD experienced higher levels of opioid craving and marginally higher scores on the Clinical Opioid Withdrawal Scale (COWS) compared with men [14]. In general, women who misuse opioids tend to report more medical problems, including anxiety and depression, relative to men [15,16]. Although it is clear that co-occurring chronic pain and opioid misuse could increase individual risk for OUD and/or opioid overdose [17], research to identify sex-based differences in pain as it relates to opioid use within this high-risk population is lacking.
The current study compared self-reported sex-based differences in persons reporting chronic pain regarding a) point prevalence ratings of subjective pain, opioid withdrawal, and pain catastrophizing and b) the source of prescription opioids. Based on previous studies [5,6,9], we hypothesized that women would report higher levels of pain and pain catastrophizing than men. We also postulated that opioid withdrawal would moderate the relationship between pain catastrophizing and self-reported worst pain assessed by the Brief Pain Inventory (BPI) [18], given that pain catastrophizing is defined by exaggerated emotional responses to pain and worst pain is defined as the most severe self-reported pain in the last 24 hours. As an exploratory analysis, we also examined whether opioid withdrawal would moderate the relationship between pain catastrophizing and self-reported pain interference in daily activities as measured by the BPI. In each regression analysis, we stratified the sample by sex, as the higher levels of opioid withdrawal experienced by women could influence their self-report of worst pain and/or pain interference. Finally, we hypothesized that women would be more likely than men to source opioids from a physician, based on evidence that they self-report higher levels of pain.
Methods
Participants
Women (N = 80) and men (N = 101) who self-reported both chronic pain and past-30-day opioid misuse were included in the study. These participants were a subset of a larger study that examined the relationship between treatment access and treatment preference in persons misusing prescription opioids who were recruited between November 2016 and January 2017 [19]. Participants were registered “Workers” on Amazon Mechanical Turk (AMT), a regularly used platform that enables biomedical research studies to sample from nationally representative groups, including persons with chronic pain [20,21]. AMT Workers consist of persons given access to survey assignments through the AMT platform. As a quality metric, “Requesters” (persons distributing surveys) can rate Workers based on task performance, for example, whether they successfully completed surveys and correctly answered distractor questions. AMT workers in this study met quality assurance standards (>90% worker approval rating), and consent was indicated via completion of the survey.
Eligibility criteria for these analyses were 1) age ≥18 years, 2) US residency, 3) misuse of prescription opioids (getting “high” or using other than prescribed) more than once in the last 30 days, and 4) presence of chronic pain, as evidenced by endorsement of allof the following criteria: a) experience of pain other than everyday kinds of pain, b) pain lasting at least three months, c) long-term pain not explained by withdrawal from opioids, d) reporting at least mild daily pain on average (≥2 on a 0–10 visual analog scale). The survey was hosted on Qualtrics (Provo, UT, USA), a survey design program that allows advanced control over survey flow and data collection.
Measures
Survey questions included participant demographics, the number of days misusing opioids in the last 30 days, and the location from which prescription opioids were sourced the first time the participant misused opioids, as well as within the last 30 days (Table 1). OUD status was determined via a self-report version of the DSM-5 checklist for OUD that resulted in nondiagnostic classification of none, mild, moderate, or severe OUD. The Subjective Opiate Withdrawal Scale (SOWS) [22] was administered to gauge severity of symptoms associated solely with opioid withdrawal (after abstaining from opioid use) in the last 30 days. Values were summed to a total severity score for analyses.
Table 1.
First Misuse |
Misuse Last 30 Days |
|||||
---|---|---|---|---|---|---|
Men (N = 101), % | Women (N = 80), % | χ2(1), P Value | Men (N = 101), % | Women (N = 80), % | χ2(1), P Value | |
Physician | 65 | 80 | 4.74, 0.030 | 54 | 74 | 7.13, 0.008 |
Friend/family for pain | 13 | 11 | 0.11, 0.740 | 33 | 24 | 1.74, 0.188 |
Friend/family to “get high” | 16 | 3 | 8.87, 0.003 | 41 | 24 | 5.72, 0.017 |
Taken from medicine cabinet | 6 | 4 | 0.45, 0.501 | 9 | 3 | 3.21, 0.073 |
Bought illicitly | 0 | 0 | na | 14 | 14 | 0, 0.983 |
Online | 0 | 3 | 2.55, 0.110 | 1 | 3 | 0.62, 0.429 |
Men and women were compared regarding their source of opioids the first time they ever misused, as well as their source of the opioids that were misused in the last 30 days. Between-group comparisons were conducted with chi-square analyses, and significant group differences are in bold.
Participants completed the following questionnaires regarding their pain and propensity for opioid misuse: the Brief Pain Inventory (BPI) [23]was used to characterize self-reported chronic, current, and 24-hour pain as well as pain interference in daily activities [18]; the Pain Catastrophizing Scale (PCS) was administered to assess participant emotional response to the pain they experience [24]; and the Screener and Opioid Assessment for Patients with Pain–Revised (SOAPP-R) was used to distinguish the degree to which participants were susceptible to opioid misuse [25]. Summary and subscale scores were calculated for each questionnaire.
Statistical Analyses
Sex-based differences on pain and opioid-related measures were compared between groups using chi-square analyses for binary dependent variables and independent sample t tests for continuous dependent variables. Linear regression analyses were used to evaluate relationships between continuous variables. “Worst Pain” from the BPI was used as a primary dependent variable because it represents the most severe pain experienced in the past 24 hours, which is both clinically relevant and provides the best opportunity to examine whether any sex-based differences exist regarding pain catastrophizing and self-reported pain. To evaluate whether SOWS Total Score moderated the relationship between the PCS Total Score and BPI Worst Pain, SOWS Total score was added as an interaction term in step 2 of the linear regression model. Analyses were also run on data stratified by sex to examine whether the relationship between PCS Total Score, SOWS, and BPI Worst Pain was sex-specific. Alpha levels for significant findings were set at P < 0.05, and analyses were conducted using SPSS, version 24.0.
Results
Study participants were 55.8% male, 90.1% Caucasian, and 8.8% Hispanic. The top three reported locations of pain were lower back (74%), upper back (42%), and knees (36.5%). Women (M = 36.04, SD = 9.2) and men (M = 33.9, SD = 8.6) did not significantly differ in age (t(179) = 1.58, P = 0.115) or percentage with any self-reported OUD (82.5% vs 80.2%, χ2(1) = 0.115, P = 0.694). Women reported misusing opioids on more days in the past 30 (M = 18.1, SD = 11.1) than men (M = 14.3, SD = 10.0, t(179) = 2.39, P = 0.018) and had marginally higher past-30-day withdrawal ratings on the SOWS (M = 23.9, SD = 19.0) than men (M = 19.4, SD = 14.5, t(179) = 1.84, P = 0.068). Differences in where participants sourced misused opioids can be found in Table 1.
Women scored significantly higher on the PCS Total Score (M = 26.16, SD = 12.0) than men (M = 21.1, SD = 12.3, t(179) = 2.80, P = 0.006). Differences were driven by women scoring significantly higher on the Rumination (M = 8.8, SD = 4.2 vs M = 7.2, SD = 4.3, t(179) = 2.55, P = 0.011) and Helplessness (M = 11.8, SD = 5.9 vs M = 9.0, SD = 5.8, t(179) = 3.19, P = 0.002) subscales relative to men, respectively. No differences were detected in the Magnification subscale or the SOAPP-R. Figure 1 displays sex-based differences on BPI scales.
Linear regression revealed a significant relationship between PCS Total Score and BPI Worst Pain (R2 = 0.143, F(1, 179) = 29.94, P < 0.001). The addition of SOWS Total Score increased the variance explained by the regression model (ΔR2 = 0.041, ΔF(1, 178) = 9.00, P = 0.003). Further, stratifying the group by sex revealed that the significant association between PCS Total Score and BPI Worst Pain remained true in men (R2 = 0.162, F(1, 99) = 19.09, P < 0.001), but the addition of the SOWS Total Score did not increase the variance explained by the regression model(ΔR2 < 0.000, ΔF(1, 98) = 0.003, P = 0.954). However, in women, the association between PCS Total Score and BPI Worst Pain was significant (R2 = 0.081, F(1, 78) = 6.89, P = 0.010), and adding the SOWS Total score further increased the variance explained by the regression model (ΔR2 = 0.127, ΔF(1, 78) = 12.39, P = 0.001) (Figure 2 ).
Linear regression also revealed a significant relationship between PCS Total Score and BPI Pain Interference (R2 = 0.333, F(1, 179) = 89.17, P < 0.001). The addition of SOWS Total Score increased the variance explained by the regression model (ΔR2 = 0.034, ΔF(1, 178) = 9.68, P = 0.002). Further, stratifying the group by sex revealed that the significant association between PCS Total Score and BPI Pain Interference remained true in men (R2 = 0.371, F(1, 99) = 58.29, P < 0.001), and the addition of the SOWS Total Score increased the variance explained by the regression model (ΔR2 = 0.036, ΔF(1, 98) = 5.81, P = 0.017). In women, the association between PCS Total Score and BPI Pain Interference was significant (R2 = 0.237, F(1, 78) = 24.22, P < 0.001), and adding the SOWS Total Score marginally increased the variance explained by the regression model (ΔR2 = 0.033, ΔF(1, 78) = 3.50, P = 0.065).
Discussion
Understanding sex-based differences in the relationship between chronic pain and opioid withdrawal is critical in identifying persons who may be at elevated risk for misusing prescription opioids and developing OUD. The current study found that, among persons who have chronic pain and currently misuse prescription opioids, women reported greater levels of pain interference, average current pain, and worst pain in the last 24 hours compared with men (Figure 1). Women also scored higher on the PCS Total as well as the Rumination and Helplessness subscales, but not the Magnification subscale compared with men; this replicates and extends outcomes reported in healthy young adults [8]. Compared with men, women in this study reported misusing opioids more frequently, despite the absence of sex-based differences in prescription opioid misuse propensity as measured by the SOAPP-R or differences in the presence of self-reported OUD as measured by the OUD DSM-5 checklist. Diagnosing OUD via DSM-5 criteria can be challenging in a chronic pain population, as some diagnostic criteria, including tolerance and duration of use, might be present due to a chronic pain condition and not OUD. Regardless, women also self-reported marginally higher levels of past-30-day opioid withdrawal severity compared with men. Thus, although women were engaging in more opioid misuse and experiencing more opioid-related consequences than men, they may not have been identified as at-risk based on standard screening measures.
Opioid withdrawal severity, as measured by the SOWS, moderated the relationship between pain catastrophizing and past-24-hour worst pain. We hypothesized that this relationship would be significant in women but not men based upon evidence that women experience higher levels of opioid withdrawal [13,14]. Stratifying the sample by sex revealed that this effect was indeed specific to women but not men (Figure 2). These data are correlational and cannot confirm whether opioid withdrawal contributes to higher levels of self-reported pain in women. Yet it remains possible that, relative to men, women with chronic pain may be at higher risk of catastrophizing their experience of pain because they experience more severe opioid withdrawal, and this could result in higher levels of self-reported pain and perhaps corresponding differences in the frequency of prescription opioid misuse. Indeed, for many individuals with chronic pain, the need to increase opioid consumption to treat worsening pain could be exacerbated by pain related to opioid withdrawal, and women could be more vulnerable to ongoing opioid misuse if opioid withdrawal does in fact contribute to their overall perception of pain. The same sex-based differences were not found in the relationship between pain catastrophizing, opioid withdrawal severity, and pain interference. Pain catastrophizing was strongly associated with pain interference in daily activities for the entire sample, and opioid withdrawal severity moderated this relationship in both men and women (although the latter finding was marginally significant in women). Based on these findings, opioid withdrawal might uniquely contribute to self-reported worst pain in women; however, opioid withdrawal might also exacerbate pain interference in daily activities regardless of sex. It is not possible to completely separate self-reported pain from opioid withdrawal in this study, but this topic could and should be addressed in future human laboratory research.
Sex-based differences in the source of misused opioids were also explored to understand differences in where men and women obtain opioids for misuse. Previous studies have found that women report higher health care utilization than men [26] and initially source opioids through legitimate sources such as physicians [9], whereas men are more likely to source opioids illicitly [11]. Indeed, the current study found that women were more likely than men to initially obtain opioids by prescription and that men were more likely to initially source opioids from friends or family with the expressed purpose of “getting high” (Table 1). This trend was also significant for opioids sourced in the last 30 days (Table 1). Clinicians should be aware that women misusing prescription opioids might be more likely than men to seek opioids from a physician, especially given that women generally report higher levels of pain and pain catastrophizing [6,8]. Physicians should consider directing resources to preventing opioid misuse in patients who are receiving prescriptions for extended periods, particularly women, to help balance the risk of undertreating pain with contributing to opioid misuse. Evidence also suggests that women with chronic pain might be willing to utilize long-acting opioids that have reduced abuse potential, such as buprenorphine [27].
This study is limited by the sample size and the emphasis on point prevalence vs longitudinal data. The use of AMT for participant recruitment also limits the interpretation of these results given that there was no clinical confirmation of self-reported pain diagnosis or opioid misuse. Previous studies have confirmed the validity and reliability of data collected through AMT compared with traditional in-person surveys [28,29], although the findings from this study could be strengthened through in-person human laboratory research. Our regression models utilized Worst Pain as the dependent variable because it has theoretical support to be associated with pain catastrophizing and is a frequently reported outcome in clinical trials [23]. In addition, the associations described in this research are self-reported domains that cannot be interpreted as causative findings. However, the results are consistent with extant literature and representative of individual perceptions of the experience of chronic pain and opioid withdrawal. In that regard, they may be useful to clinicians who are treating chronic pain patients and researchers who are interested in sex-based differences in opioid analgesia, misuse, and withdrawal.
In general, the relationship between self-reported pain and opioid misuse is a major issue in pain medicine, and the current study suggests that men and women who have chronic pain may experience pain and opioid effects differently and in a manner that could differentially contribute to continued misuse of prescription opioids. This study observed a relationship between pain catastrophizing, opioid withdrawal, and self-reported pain in women but not men. Human laboratory studies that can elucidate relationships between opioid withdrawal and self-reported pain in persons with chronic pain are warranted. Additional research aimed at delineating sex-based differences in motivation to misuse opioids may also have a meaningful impact on chronic pain treatment as precision medical approaches are developed to maximize analgesia and minimize risk of opioid misuse and subsequent issues such as the onset of OUD and/or risk for opioid overdose.
Funding sources: The work described in the manuscript was funded by the National Institute on Drug Abuse R01 DA035246 (Dunn) and T32 DA007209 (Bigelow).
Conflicts of interest: DAT has received medication supplies from Indivior (formerly Reckitt Benckiser Pharmaceuticals) for an investigator-initiated study, was site PI for a clinical trial sponsored by Alkermes, and is on a scientific advisory board for Alkermes. There are no conflicts to report for ASH, CMC, or KED.
References
- 1. Volkow ND, McLellan AT. Opioid abuse in chronic pain—misconceptions and mitigation strategies. N Engl J Med 2016;37413:1253–63. [DOI] [PubMed] [Google Scholar]
- 2. Kolodny A, Courtwright DT, Hwang CS et al. . The prescription opioid and heroin crisis: A public health approach to an epidemic of addiction. Annu Rev Public Health 2015;36:559–74. [DOI] [PubMed] [Google Scholar]
- 3. Schuchat A, Houry D, Guy GP. New data on opioid use and prescribing in the United States. JAMA 2017;3185:425–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Tsang A, Von Korff M, Lee S et al. . Common chronic pain conditions in developed and developing countries: Gender and age differences and comorbidity with depression-anxiety disorders. J Pain 2008;910:883–91. [DOI] [PubMed] [Google Scholar]
- 5. Andersson HI, Ejlertsson G, Leden I, Rosenberg C. Chronic pain in a geographically defined general population: Studies of differences in age, gender, social class, and pain localization. Clin J Pain 1993;93:174–82. [DOI] [PubMed] [Google Scholar]
- 6. Edwards RR, Haythornthwaite JA, Sullivan MJ, Fillingim RB. Catastrophizing as a mediator of sex differences in pain: Differential effects for daily pain versus laboratory-induced pain. Pain 2004;1113:335–41. [DOI] [PubMed] [Google Scholar]
- 7. Roth RS, Geisser ME, Theisen-Goodvich M, Dixon PJ. Cognitive complaints are associated with depression, fatigue, female sex, and pain catastrophizing in patients with chronic pain. Arch Phys Med Rehabil 2005;866:1147–54. [DOI] [PubMed] [Google Scholar]
- 8. Sullivan MJ, Tripp DA, Santor D. Gender differences in pain and pain behavior: The role of catastrophizing. Cogn Ther Res 2000;241:121–34. [Google Scholar]
- 9. McHugh RK, DeVito EE, Dodd D et al. . Gender differences in a clinical trial for prescription opioid dependence. J Subst Abuse Treat 2013;451:38–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Ives TJ, Chelminski PR, Hammett-Stabler CA et al. . Predictors of opioid misuse in patients with chronic pain: A prospective cohort study. BMC Health Serv Res 2006;61:46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Back SE, Payne RL, Simpson AN, Brady KT. Gender and prescription opioids: Findings from the National Survey on Drug Use and Health. Addict Behav 2010;3511:1001–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Townsend CO, Kerkvliet JL, Bruce BK et al. . A longitudinal study of the efficacy of a comprehensive pain rehabilitation program with opioid withdrawal: Comparison of treatment outcomes based on opioid use status at admission. Pain 2008;1401:177–89. [DOI] [PubMed] [Google Scholar]
- 13. Bodnar RJ, Kest B. Sex differences in opioid analgesia, hyperalgesia, tolerance and withdrawal: Central mechanisms of action and roles of gonadal hormones. Horm Behav 2010;581:72–81. [DOI] [PubMed] [Google Scholar]
- 14. Back SE, Payne RL, Wahlquist AH et al. . Comparative profiles of men and women with opioid dependence: Results from a national multisite effectiveness trial. Am J Drug Alcohol Abuse 2011;375:313–23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Simoni-Wastila L, Ritter G, Strickler G. Gender and other factors associated with the nonmedical use of abusable prescription drugs. Subst Use Misuse 2004;391:1–23. [DOI] [PubMed] [Google Scholar]
- 16. Kosten TR, Rounsaville BJ, Kleber HD. Ethnic and gender differences among opiate addicts. Int J Addict 1985;208:1143–62. [DOI] [PubMed] [Google Scholar]
- 17. Dunn KM, Saunders KW, Rutter CM et al. . Opioid prescriptions for chronic pain and overdose: A cohort study. Ann Intern Med 2010;1522:85–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Mendoza T, Mayne T, Rublee D, Cleeland C. Reliability and validity of a modified Brief Pain Inventory short form in patients with osteoarthritis. Eur J Pain 2006;104:353–61. [DOI] [PubMed] [Google Scholar]
- 19. Huhn AS, Tompkins DA, Dunn KE. The relationship between treatment accessibility and preference amongst out-of-treatment individuals who engage in non-medical prescription opioid use. Drug Alcohol Depend 2017;180:279–285. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Tompkins DA, Johnson PS, Smith MT et al. . Temporal preference in individuals reporting chronic pain: Discounting of delayed pain-related and monetary outcomes. Pain 2016;1578:1724–1732. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Tompkins DA, Huhn AS, Johnson PS et al. . To take or not to take: The association between perceived addiction risk, expected analgesic response and likelihood of trying novel pain relievers in self-identified chronic pain patients. Addiction 2018;1131:67–79. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Handelsman L, Cochrane KJ, Aronson MJ et al. . Two new rating scales for opiate withdrawal. Am J Drug Alcohol Abuse 1987;133:293–308. [DOI] [PubMed] [Google Scholar]
- 23. Cleeland C. The Brief Pain Inventory: User guide. 2009. Available at:https://www.mdanderson.org/documents/Departments-and-Divisions/Symptom-Research/BPI_UserGuide.pdf. (accessed December 2, 2018).
- 24. Sullivan MJ, Bishop SR, Pivik J. The Pain Catastrophizing Scale: Development and validation. Psychol Assess 1995;74:524. [Google Scholar]
- 25. Butler SF, Fernandez K, Benoit C, Budman SH, Jamison RN. Validation of the Revised Screener and Opioid Assessment for Patients with Pain (SOAPP-R). J Pain 2008;94:360–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Unruh AM. Gender variations in clinical pain experience. Pain 1996;65(2–3):123–167. [DOI] [PubMed] [Google Scholar]
- 27. Malinoff HL, Barkin RL, Wilson G. Sublingual buprenorphine is effective in the treatment of chronic pain syndrome. Am J Ther 2005;125:379–384. [DOI] [PubMed] [Google Scholar]
- 28. Kim HS, Hodgins DC. Reliability and validity of data obtained from alcohol, cannabis, and gambling populations on Amazon’s Mechanical Turk. Psychol Addict Behav 2017;311:85–94. [DOI] [PubMed] [Google Scholar]
- 29. Mortensen K, Hughes TL. Comparing Amazon’s Mechanical Turk platform to conventional data collection methods in the health and medical research literature. J Gen Intern Med 2018;334:533–8. [DOI] [PMC free article] [PubMed] [Google Scholar]