Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Mar 1.
Published in final edited form as: Pharmacol Biochem Behav. 2021 Jan 12;202:173112. doi: 10.1016/j.pbb.2021.173112

Reinforcing effects of synthetic cathinones in rhesus monkeys: Dose-response and behavioral economic analyses.

Fernando B de Moura a,b, Alexander Sherwood c, Thomas E Prisinzano d, Carol A Paronis a,b, Jack Bergman a,b, Stephen J Kohut a,b
PMCID: PMC7938610  NIHMSID: NIHMS1667615  PMID: 33444603

Abstract

The abuse of synthetic cathinones (“bath salts”) with psychomotor stimulant and/or entactogenic properties emerged as a public health concern when they were introduced as “legal” alternatives to drugs of abuse such as cocaine or MDMA. In this study, experiments were conducted in nonhuman primates to examine how differences in transporter selectivity might impact the reinforcing effects of synthetic cathinones. Rhesus monkeys (N=5) were trained to respond for intravenous injections under a fixed-ratio (FR) 30, timeout 60-sec schedule of reinforcement. The reinforcing effects of selected cathinones (e.g., MDPV, αPVP, MCAT, and methylone) with a range of pharmacological effects at dopamine and serotonin transporters were compared to cocaine and MDMA using dose-response analysis under a simple FR schedule and behavioral economic procedures that generated demand curves for two doses of each drug. Results show that one or more doses of all drugs were readily self-administered in each subject and, excepting MDMA (21 injections/session), peak levels of self-administration were similar across drugs (between 30–40 injections/session). Demand elasticity for the peak and the peak + ½-log dose of each drug did not significantly differ, and when data for the two doses were averaged for each drug, the following rank-order of reinforcing strength emerged: cocaine > MCAT = MDPV = methylone > αPVP = MDMA. These results indicate that the reinforcing strength of synthetic cathinones are not related to their selectivity in binding dopamine or serotonin transporter sites .

Keywords: cathinone, rhesus monkeys, self-administration, behavioral economics

1. Introduction

The use of synthetic cathinones, commonly referred to as “bath salts”, emerged as a public health concern over the past decade (German et al. 2014). Bath salt preparations, which primarily contain mixtures of several cathinones with pharmacological activity via several monoaminergic systems (Caudevilla-Gálligo et al. 2013; Davies et al. 2010; Miotto et al, 2013; Schneir et al. 2014; Spiller et al. 2011; Zuba and Byrska 2013), are often used as adulterants in or as substitutes for methamphetamine (MA) or 3,4-methylenedioxy-methamphetamine (MDMA; Seely et al. 2013). Recent epidemiological data indicate that annual rates of bath salt use by high school students increased between 2015–2018 (Johnston et al. 2019), illustrating their continuing popularity.

Synthetic cathinones are structural congeners of β-keto-amphetamine and, like other amphetamines, their abuse-related effects are thought to be mediated primarily by their ability to either block the reuptake or increase the release of neuronal dopamine (DA) and/or serotonin (5-HT). As with other amphetamine-type drugs, the reinforcing strength of cathinones appears to be related to their monoaminergic selectivity (Wee et al. 2005; Wee and Woolverton, 2006; Woolverton, 1992; Negus et al. 2007; Gannon et al, 2018a). For example, under both behavioral economic and progressive ratio (PR) procedures, the dopamine-preferring uptake inhibitor methylenedioxyprovalerone (MDPV) has been shown to be more reinforcing than methylone, a relatively nonselective DA/5-HT releaser (Gannon et al. 2018b; Gannon et al. 2019; Baumann et al. 2012; Eshleman et al. 2017; Gannon et al. 2017a). In general, cathinones that have dopaminergic actions but, like methylone, retain significant 5-HT activity (i.e., <10-fold selectivity for DA/5-HT) have been shown to maintain stable self-administration behavior in rats (Aarde et al. 2013; Creehan et al. 2015; Harvey et al. 2017; Dolan et al. 2018; Gannon et al. 2018b).

Previous studies also have suggested that the DA-preferring cathinones MDPV and α-pyrrolidinopentiophenone (αPVP) may be more reinforcing than psychomotor stimulants like cocaine and methamphetamine. For example, MDPV and αPVP have been shown to maintain higher breakpoints than cocaine under a progressive ratio schedule of drug-maintained reinforcement in rats (Gannon et al. 2017a,b, 2018a; see also Aarde et al. 2015). Using the behavioral economic demand analysis, αPVP was shown to be more reinforcing than methamphetamine in rats (Huskinson et al. 2017). Recently, Collins et al (2019) extended such comparisons to studies in rhesus monkeys, finding that breakpoints for MDPV and αPVP in nonhuman primates were higher than those for cocaine and methamphetamine.

Although previous studies have clearly demonstrated that synthetic cathinones with a range of DA-mediated and 5-HT-mediated actions can serve as reinforcers in laboratory animals, their reinforcing strength typically has been compared to that of drugs of abuse with predominantly dopaminergic mechanisms of action. There is little information about how the reinforcing effects of cathinones with predominantly serotonergic (e.g., methylone; Nagai et al. 2007; Baumann et al. 2012) or dopaminergic properties (e.g., αPVP and MDPV; Rothman and Baumann 2003; Marusich et al. 2014; Eshleman et al. 2017) compare with those of a self-administered drug with predominantly serotonergic activity, e.g., MDMA (Nagai et al. 2007; Baumann et al. 2012; though see Dolan et al, 2018). This is surprising in view of the popularity of substituting some synthetic cathinones for MDMA, either alone or as a cathinone/MDMA mixture (Davies et al. 2010; Seeley et al. 2013). A comparison of the reinforcing effects of cathinones with MDMA, which displays greater serotonergic activity than methamphetamine or cocaine, may lead to a better understanding of the differential contribution of serotonergic and/or dopaminergic activity to the abuse potential of novel synthetic cathinones.

The present study was conducted to investigate the relationship between serotonergic activity and the abuse potential of synthetic cathinones by comparing the reinforcing strength of drugs varying in monoaminergic selectivity to that of cocaine and MDMA in nonhuman primates. The synthetic cathinones evaluated in these experiments vary in transporter selectivity, and based on in vitro studies, can be considered DAT-selective (MDPV, αPVP) inhibitors, NET/DAT-preferring (MCAT) releasers, SERT-preferring (MDMA) releasers, or nonselective inhibitors (cocaine) or mixed inhibitor/releasers (methylone; Rothman and Baumann 2003; Nagai et al. 2007; Baumann et al. 2012; Simmler et al. 2013; Marusich et al. 2014; Elshelman et al. 2017; Kohut et al. 2017).

2. Methods

2.1. Animals.

Adult rhesus monkeys (Macaca mulatta; n=4 males and 1 female) were used in the present studies. All monkeys previously served in IV self-administration studies of cocaine, nicotine, or methamphetamine alone and after pretreatment with a range of candidate medications (e.g., adrenergics, serotonergics, nicotinics). Each subject resided in a stainless-steel chamber with side and front visual access to the colony room. A 12-hr light-dark cycle was in effect in the colony room (excepting the 2-hr session during which lights outside the home chambers were extinguished). Each subject had continuous access to water and received High Protein Monkey Diet (Purina Mills International, Brentwod, MO) supplemented with fresh fruit and vegetables in addition to the 1-g banana-flavored pellets that were earned during experimental sessions. Enrichment was provided through access to mirrors and toys in the home-cage, music during non-experimental sessions, and interactions with technical staff throughout the day. Animal care and research were conducted according to the guidelines provided by the Institute of Laboratory Animal Resources and the National Institutes of Health Office of Laboratory Animal Welfare. The facility is licensed by the U.S. Department of Agriculture, and all experimental protocols were approved by the Institutional Animal Care and Use Committee at McLean Hospital.

2.2. Apparatus.

Experimental sessions were conducted in the subject’s home chamber. A food cup attached to the outside of the cage provided access to response-contingent food pellets that were delivered during experimental sessions from a pellet dispenser positioned above the chamber (Gerbrands Model G5210, Arlington, MA). Two syringe pumps (Model 981219, Harvard Apparatus, Inc., South Natick, MA) also were positioned above the chamber, each connected to one lumen of tubing that connected through a tether system to a double-lumen catheter implanted in the subject’s jugular or femoral vein. A custom-designed operant response panel containing three square translucent response keys arranged horizontally was attached to the front of the chamber and faced inward. Each key could be illuminated with stimulus lights of differing color (SuperBright LEDs; Fairchild Seminconductor, San Jose, CA). All experimental events were controlled and data were recorded by Med Associates software on a desktop PC (Georgia, VT).

2.3. Self-Administration Procedure.

Each subject was surgically prepared with an indwelling jugular or femoral double-lumen catheter using previously detailed procedures (Kohut et al. 2013). Initially, i.v. cocaine self-administration and food-maintained responding were established in each subject under a fixed ratio (FR) 30; timeout (TO) 60-s schedule of reinforcement. Daily 2-hr sessions comprised of three components of schedule-controlled responding separated by inter-component 5-min TO periods during which all lights were off and responding had no scheduled consequences. Under these conditions, red lights illuminating the center response key during the first and third components signaled the availability of 1-g banana-flavored pellets under the FR30;TO 10-s schedule (5-min in length); green lights illuminating the center response key during the second (self-administration) component signaled the availability of intravenous injections of test drugs (100-min in length). The self-administration component began with a 10-s illumination of a yellow stimulus light accompanied by a non-contingently delivered priming injection of what was available for i.v. self-administration thereafter, i.e., saline or a unit dose of cocaine. Illumination of the yellow stimulus light for 10-s also accompanied all deliveries of reinforcement (food or i.v. cocaine) during each session component. When responding under the multiple-component conditions was stable, dose-response functions were obtained for cocaine, MDPV, αPVP, MCAT, MDMA, and methylone by introducing a test unit dose of drug for at least 3 days and until stability criteria was met , i.e., the number of injections did not vary by ±15% over the final 2 of 3 sessions at any given dose. Once stability criteria were met, various doses of each test drug were studied in a mixed order. The order of drug/dose testing was randomized among individual subjects; however, all doses of one drug were studied before moving to the next drug. At least 4 unit doses of each drug were studied to obtain data for both the ascending and descending limbs of the bitonic dose-response functions.

2.4. Demand Curves.

Following dose-response determinations, further experiments with each self-administered drug were conducted by establishing demand curves for the unit dose at the peak of the dose-response function and the first unit dose (1/2 log unit higher) on the descending limb of the function. The peak doses and the peak + 1/2 log doses were selected for further study because they maintained robust responding in all subjects. Daily sessions were 120-min and comprised 3 components—an 80-min component of IV self-administration that was preceded and followed by 20-min components of food-maintained responding under the FR30;TO 60-s schedule of reinforcement. For each unit dose, demand curves were generated by increasing the response requirement for i.v. drug injection during the self-administration components of consecutive sessions (FR 10, 30, 56, 100, 300, 560, etc.) until an FR requirement that resulted in 0 injections during the session was reached. Demand curves were acquired in 4-monkeys for each drug; both doses of each drug were obtained in the same four subjects.

2.5. Drugs.

Cocaine and MCAT hydrochloride was supplied by the National Institute on Drug Abuse (NIDA) Drug Supply Program (Bethesda, MD). MDPV, MDMA, αPVP, and methylone were synthesized by the Prisinzano laboratory at the University of Kentucky. All drugs were dissolved in sterile saline (0.9% NaCl), and all doses are expressed as the base weight of the drug.

2.6. Data Analysis.

All statistical analyses were conducted using GraphPad Prism version 5.0 for Windows (San Diego, CA). Self-administration data are expressed as the mean (± S.E.M) number of injections per session. A one-way ANOVA with Dunnett’s multiple comparison test was used to determine whether self-administration data for doses of each drug differed significantly from values obtained with saline. A two-tailed Student’s paired t-test was used to compare intake of the peak and the peak + ½ log doses during dose-effect determinations (injections/session) and under the FR30 schedule during demand testing. Significance was set at p<0.05 in all cases.

Demand curves for each unit dose of a drug were based on the group means for consumption at each FR, and were fit to the following exponential model of demand described by Hursh and Silberberg, 2008:

logQ=logQ0+k(eαP1)

In this model, Q represents consumption; Q0 denotes consumption as the price (P) approaches 0; and k is a fixed parameter that denotes the range of the exponential model (experimentally derived as 3.5 for the current data sets). The free parameter α quantifies the elasticity of demand as a measure of the rate of change in elasticity across the function. The α values for each monkey’s demand functions for the peak and the peak + ½-log dose of each drug were compared using a paired two-tailed Student’s t-test and, in the absence of significant differences, the curves were averaged for each drug. α values for the averaged curves were used to generate rank-order of reinforcing strength across drugs, using a repeated-measures one-way ANOVA with a Tukey’s multiple comparisons test to evaluate statistical significance. Straight lines were fitted to the linear portion of the ascending limb of the self-administration dose-response functions to compare self-administration potency between test drugs. This was accomplished with linear regression calculated by interpolating the unit dose that engendered 50% (ED50) drug injections earned per session. The calculated ED50 values for each test drug were then related to their selectivity for DAT/SERT (Eshleman et al. 2017) using a Pearson correlation.

3. Results

3.1. Self-Administration.

Averaged results for each drug describes an inverted U-shaped function that is commonly observed for self-administered drugs under FR schedules of reinforcement. Thus, the maximal number of injections/session of cocaine (40±5.1), MDPV (32±5.4), αPVP (39±8.1), MCAT (41±7.0), MDMA (21±5.2), and methylone (34±7.4) were significantly above levels recorded during sessions of saline availability (mean: 6.4±1.3 injections/session; Figure 1). Post-hoc analyses reveal that the number of unit doses that were self-administered significantly above saline levels varied among drugs. Thus, a tenfold range of doses of cocaine (0.01, 0.032, 0.1 mg/kg), MDPV (0.001, 0.0032, 0.01 mg/kg), αPVP (0.001, 0.0032, 0.01 mg/kg), and MCAT (0.0032, 0.01, and 0.032 mg/kg) maintained i.v. self-administration behavior whereas only a threefold range of doses of MDMA (0.032 and 0.1 mg/kg) and one dose of methylone (0.1 mg/kg) were self-administered at levels significantly above values obtained for saline (Figure 1). There was no concordance (r4=−0.28, p=0.59) between the rank of potency of drugs in the present self-administration studies with the rank order of DAT binding selectivity (Eshleman et al., 2017). Results of the drug effects described in this section are summarized in Table 1.

Figure 1.

Figure 1.

Dose-response functions for self-administration of cocaine, MDPV, αPVP, MCAT, MDMA, and methylone. Ordinate: injections per session. Abscissa: dose (mg/kg). Filled circles represent points in which self-administration of drug were significantly greater than saline as determined by a Dunnett’s Multiple Comparisons test (cocaine: F4,28=13, p<0.0001; MDPV: F4,44=8.5, p<0.0001; αPVP: F5,40=6.1, p=0.003; MCAT: F6,51=7.2, p<0.0001; MDMA: F4,44=5.5, p=0.0012; methylone: F5,38=5.2, p=0.001) . Error bars represent S.E.M.

Table 1:

Summary of Results of Fixed Ratio Self-Administration of Drugs Studied

Drug Peak Dose (mg/kg) Avg.Max # of Injections/Session ED50 Valueb (mg/kg [95% CI]) DAT/SERT Selectivity
Cocaine 0.01 40.3 0.0044 (0.0022–0.0089) 1.25
αPVP 0.0032 38.8 0.0016 (0.0002–0.013) 2893
MCAT 0.0032 41.2 0.0023 (0.00045–0.012) 136
MDPV 0.0032 31.9 0.00062 (0.00031–0.0012) 110
Methylone 0.1 34.4 0.013 (0.001–0.19) 5.6
MDMA 0.032 21.1 0.055 (0.016–0.18) 0.88
a

DAT selectivity values from Eshleman et al. 2017 calculated as 1/hDAT IC50:1/hSERT IC50

b

ED50 values with 95% Confidence Intervals of the ascending limb of the self-administration curves

3.2. Behavioral Economics.

Demand curves in Fig 2 show that consumption of all drugs decreased as response requirement increased. Excepting the peak + ½-log dose of MDPV, the number of injections of each drug earned under the FR30 schedule in demand studies did not significantly differ for either dose from the number of injections earned under the same schedule during earlier dose-effect determinations (p>0.05). Self-administration of the peak + ½-log dose of MDPV was significantly decreased under the FR30 schedule in demand studies than during previous dose-effect determinations (28 ± 5.4 vs. 16 ± 0.82 inj/session, respectively; t6=2.98, p=0.025). The exponential model of demand provided a good fit of the data generated for all doses of test drugs (R2 values: 0.82–0.99; Figure 2). For each drug, demand elasticity of the two doses studied did not significantly vary (p>0.05). The observed rank order of reinforcing effectiveness (based on non-overlapping 95% confidence intervals of the averaged demand curve) was: cocaine>methcathinone=MDPV=methylone>MDMA=αPVP (Figure 3; see also Table 2). Q0 values varied from 23 (MDMA and methylone) to 80 (MCAT), whereas α values varied from 0.000013 (cocaine) to 0.000043 (MDMA). Results described in this section are summarized in Table 2.

Figure 2.

Figure 2.

Demand curves for the peak and peak + ½-log dose of cocaine, MDPV, αPVP, MCAT, MDMA, and methylone. Ordinate: consumption (injections per session). Abscissa: price (FR). Error bars represent S.E.M.

Figure 3.

Figure 3.

Demand curves for the average of the peak and peak + ½-log dose of cocaine, MDPV, αPVP, MCAT, MDMA, and methylone. Ordinate: consumption (injections per session). Abscissa: price (FR value). Error bars represent S.E.M.

Table 2:

Comparison of behavioral economic indices of demand intensity (Q0) and elasticity (α) for cocaine, MDPV, αPVP, MCAT, MDMA, and methylone. T-test comparing the peak and peak + 1/2 log demand curves.

Drug α Q0 r2 T Test
Cocaine:
 Peak 1.2 × 10−5 68 0.98
 + ½-log 1.7 × 10−5 25 0.97 T3=1.0, p=0.38
 Grouped 1.3 × 10−5 40 0.97
MDPV:
 Peak 3.0 × 10−5 52 0.98
 + ½-log 1.7 × 10−5 18 0.93 T3=2.4, p=0.093
 Grouped 1.8 × 10−5 25 0.94
αPVP:
 Peak 3.6 × 10−5 57 0.99
 + ½-log 6.1 × 10−5 47 0.96 T3=1.5, p=0.11
 Grouped 4.2 × 10−5 52 0.98
MCAT
 Peak 1.9 × 10−5 75 0.91
 + ½-log 2.5 × 10−5 21 0.82 T3=1.4, p=0.12
 Grouped 1.7 × 10−5 80 0.89
MDMA
 Peak 4.7 × 10−5 20 0.86
 + ½-log 4.9 × 10−5 16 0.93 T3=0.10, p=0.46
 Grouped 4.3 × 10−5 23 1.0
Methylone
 Peak 3.5 × 10−5 57 0.99
 + ½-log 1.9 × 10−5 18 0.83 T3=0.76, p=0.50
 Grouped 1.8 × 10−5 23 0.88

4. Discussion

The present studies were conducted to evaluate the reinforcing effects of four commonly used synthetic cathinones by comparing them to those of well-characterized monoaminergic drugs acting predominantly via dopaminergic (cocaine) and serotonergic (MDMA) mechanisms of action. Results from dose-effect determinations under the FR30 schedule of IV self-administration indicate that peak unit doses of cocaine and the four cathinones maintained levels of IV self-administration that were comparable (approximately 30–40 injections per session) and that were considerably greater than those maintained by MDMA (approximately 20 injections per session). These findings are consistent with previous reports showing relatively low rates of MDMA self-administration (Dolan et al., 2018), and higher rates of self-administration of the other five compounds in both rats (Gannon et al. 2017b; Huskinson et al. 2017; Dolan et al. 2018) and nonhuman primates (Collins et al. 2019).

Differences in peak levels of self-administration between MDMA and other drugs in the present studies likely were not due to differing time courses of drug action, as MDMA, MDPV, and αPVP are thought to have similar durations of action (de la Torre et al., 2000; Anizan et al., 2016; Aarde et al. 2015). Rather, it is likely that the lesser intake of MDMA than of the other drugs reflects differences in neurochemical mechanisms of action. Namely, the ability of monoaminergic drugs to inhibit DAT has been closely related to their capacity to maintain drug-taking behavior (Ritz et al. 1987, 1988; Ritz and Kuhar 1989), whereas their ability to increase synaptic levels of serotonin has been proposed to dampen such DA-mediated reinforcing effects (Woods and Tessel, 1974; Glowa et al. 1997; Wee et al. 2005; Rothman and Bauman, 2006; Negus et al. 2007). For example, the addition of the serotonin releaser fenfluramine has been shown to significantly attenuate self-administration of amphetamine in rhesus monkeys (Wee and Woolverton, 2006). Additionally, slower acquisition as well as lower and more variable levels of IV self-administration of MDMA (compared to cocaine) have been attributed to its ability to selectively stimulate the release of serotonin from presynaptic neurons and/or block its reuptake (Fantegrossi et al. 2002; Schenk et al. 2003, 2007; Baumann et al, 2012; Bradbury et al. 2014). Thus, the present findings add to a considerable literature suggesting that the serotonergic actions of MDMA constrain its reinforcing effects, leading to lower levels of self-administration than those observed for monoaminergic drugs with less prominent serotonin-releasing activity.

The cathinones studied here display a relatively wide range of selectivity for increasing neuronal DA vs. 5-HT-ranging from 5.6 (methylone) to >100 (MDPV, aPVP, MCAT; Eshleman et al. 2017 ; Table 1). Despite this range of DAT:SERT selectivity, each of the cathinones engendered approximately the same peak number of injections as cocaine under the FR30 schedule conditions employed here. These findings may appear to challenge the presumed relationship between DAT:SERT selectivity and the expression of reinforcing effects (i.e., cocaine vs. MDMA); however, several other considerations may govern this relationship. First, a threshold of serotonin-selectivity, e.g., DA:5-HT selectivity ≥1 as reported for MDMA (Rothman and Bauman, 2003), may be required for serotonin to measurably dampen DA-mediated reinforcing effects. Some support for this idea comes from previous studies investigating the reinforcing efficacy of a series of amphetamine analogs in rhesus monkeys. In those studies, Wee et al. (2005) found that the analog PAL-314, which is a monoamine releaser with DA:5-HT release selectivity of 0.15, was self-administered at lower levels than the dopaminergic monoamine releaser d-amphetamine and that PAL-313, with DA:5-HT selectivity of 0.83, was not self-administered at all. A second key factor in the present results may be differences in the nature of MDMA’s and methylone’s interaction with the SERT. In this regard, in vitro studies have shown that MDMA has greater potency as a serotonin releaser than as an uptake inhibitor (Rothman and Baumann, 2003) whereas methylone is equipotent in both capacities (Baumann et al. 2012; López-Arnau et al. 2012). Thus, the balance between uptake inhibition and release may dictate whether a serotonergic compound can effectively modulate dopaminergic reinforcing effects.

A third consideration in the relationship between IV self-administration of monoaminergic drugs and pharmacological mechanisms may be that, as previously suggested, the reinforcing effects of DA transport blockers like cocaine are related more closely to DAT affinity than to DA:5-HT selectivity (Lile et al. 2003; see also Ritz et al, 1987; Bergman et al, 1989). For example, although cocaine, MDMA, and methylone are considered to be relatively nonselective for DAT and SERT, cocaine can bind the DAT with >50-fold higher affinity than MDMA (0.49 vs. 32.4 μM) but only with approximately 10-fold higher affinity than methylone (0.49 vs 5.02 μM; Eshleman et al. 2013; 2017). Taken together, methylone’s activity as an uptake inhibitor at the SERT coupled with its higher affinity for the DAT may explain why methylone was self-administered more avidly than MDMA, albeit across a narrow dose range.

Demand analysis provided a second means of evaluating the reinforcing effects—and, in particular, reinforcing strength—of the monoaminergic drugs studied here. In agreement with previous findings—and, more generally, behavioral economic theory—between-dose differences in reinforcing strength were not evident for any drug in the present studies (Hursh et al, 1988; Bickel et al, 1990; Wade-Galuska et al 2007; Gannon et al, 2018a; however, note Moreton et al, 1977; Marquis et al, 1989; Kohut and Bergman, 2016). Yet, differences in reinforcing effectiveness among drugs were clearly evident, which differs from the results of other types of comparative analyses. For example, in contrast to conclusions drawn from PR studies in rhesus monkeys (Collins et al, 2019), the rank-order of demand elasticity here (cocaine > methcathinone = MDPV = methylone > MDMA = αPVP) is not predicted by either binding affinity or DA/5-HT selectivity. While the pharmacodynamic basis for such differences in demand elasticity and, presumably, reinforcing strength is unclear, it may be helpful to note that contemporary views of reinforcement suggest that such processes are most likely circuit-driven rather than the consequence of activating singular receptor mechanisms (Volkow et al., 2004). From this perspective, differences in the reinforcing strength of the drugs studied here may more accurately reflect differential activation of various brain networks rather than the activation of discrete dopaminergic or serotonergic mechanisms. Consistent with this point of view, recent neuroimaging studies have demonstrated divergent patterns of disrupted brain activity (both qualitative and quantitative) following two weeks of treatment with either methamphetamine (MA) (DA:5-HT=0.02; Eshleman et al, 2017) or the relatively nonselective monamine releaser and synthetic cathinone, mephedrone (DA:5-HT=0.64; Gregg et al. 2015) in rodents (see den Hollander et al, 2015). Specifically, MA treatment resulted in widespread decrease in activity in both cortical and subcortical regions whereas mephedrone increased activity, primarily in subcortical regions relative to vehicle-treated subjects (den Hollander et al, 2015). This line of thinking provides an attractive means for understanding the differences in reinforcing strength evident in the present studies. However, it must remain speculative in the absence of data identifying differential activation patterns of neuronal circuitry by drugs of abuse that can be more closely related to their differing profiles of pharmacodynamic activity.

Demand analyses showed that the reinforcing strength of αPVP, as reflected by elasticity, was comparable to that of MDMA and below that of cocaine. These findings were unexpected because the peak session intake of αPVP was similar to that of cocaine and higher than that of MDMA. Also, αPVP has been reported to engender higher breakpoints than cocaine in studies employing progressive ratio procedures to examine its reinforcing strength in rodents and rhesus monkeys (Gannon et al. 2018b; Collins et al. 2019). Based upon the reported concordance between progressive-ratio and demand analyses of reinforcing strength (Foster et al. 1997; Johnson and Bickel, 2006), the present findings, e.g., greater demand elasticity in the reinforcing effects of αPVP than cocaine, was surprising and led to a closer inspection of individual data. This analysis revealed that the number of αPVP injections/session in one monkey was substantially below the group average (14 v 39 injections/session) but that the variability of αPVP consumption in the demand studies was relatively low, i.e., the relative elasticity of αPVP was generally consistent among all test subjects. Further, removal of this subject’s data from the demand analyses did not appreciably alter the α value (i.e, 4.6e-5 vs 3.6e-5; see Table 2). The elasticity of the peak unit dose of αPVP here (reflected in its α value) agrees with that reported previously (Huskinson et al. 2017), providing additional assurance of its reproducibility. Reasons for discordance between data from progressive-ratio and demand analyses of reinforcing strength—particularly in studies in the same species (rhesus monkeys here and in Collins et al, 2019)--are unclear but may include differences in the drug history of subjects in the different studies. In this regard, the reinforcing strength of self-administered drugs, reflected in demand elasticity, has been shown to vary with drug history (Wade-Galuska et al. 2011; Kohut and Bergman, 2016). Perhaps, differences in the particular drug history of subjects in the present experiments and the previous work by Collins et al. (2019) contributed to the difference in the expression of the reinforcing effects of αPVP in the two studies.

In summary, the results of this study indicate that the illicit use of synthetic cathinones is most likely attributable to their dopaminergically-mediated actions. Moreover, the present data suggest that 5-HT activity has negligible influence on the reinforcing strength of synthetic cathinones with equal or greater affinity for DA than 5-HT. Thus, the nonselective drugs methylone and cocaine were readily self-administered in nonhuman primates, and in fact, demand analysis found that a relatively nonselective drug, methylone, was more reinforcing than a highly DA-selective one, αPVP. The overall findings also emphasize the value of utilizing drugs with a wide range of DA/5-HT selectivity in evaluating the contribution of DA and 5-HT mechanisms to the reinforcing effects of monoaminergic drugs.

Highlights.

  • Synthetic cathinones maintain self-administration behavior in nonhuman primates

  • Reinforcing strength of synthetic cathinones does not appear to be related to selectivity for dopamine vs. serotonin

  • Illicit use of synthetic cathinones is most likely attributable to their dopaminergically-mediated actions.

Acknowledgements

The authors would like to thank Timothy Gillis, Olga Smirnova, and Daniel Borgatti for their technical assistance and Dr. Roger D. Spealman for comments on an earlier version of this manuscript. The authors would also like to acknowledge the International Study Group Investigating Drugs as Reinforcers for providing Dr. Moura with the William L. Woolverton Award to present a portion of this data at their 2018 annual meeting in San Diego, CA.

Funding: DA002519, DA048150, DA393306, and DA048150

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts of Interests: None

References

  1. Aarde SM, Creehan KM, Vandewater SA, Dickerson TJ, Taffe MA 2015. In vivo potency and efficacy of the novel cathinone α-pyrrolidinopentiophenone and 3,4-methylenedioxypyrovalerone: self-administration and locomotor stimulation in male rats. Psychopharmacology (Berl) 232:3045–3055. Doi: 10.1007/s00213-015-3944-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Anizan S, Concheiro M, Lehner KR, Bukhari MO, Suzuki M, Rice KC, Baumann MH, Huestis MA 2016. Linear pharmacokinetics of 3,4-methylenedioxypyrovalerone (MDPV) and its metabolites in the rat: relationship to pharmacodynamic effects. Addict Biol 21:339–347. Doi: 10.111/adb.12201. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Baumann MH, Ayestas MA Jr., Partilla JS, Sink JR, Shulgin AT, Daley PF et al. 2012. The designer methcathinone analogs, mephedrone and methylone, are substrates for monoamine transporters in brain tissue. Neuropsychopharmacology 37:1192–1203. Doi: 10.1124/dmd.107.016261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bergman J, Madras BK, Johnson SE, Spealman RD 1989. Effects of cocaine and related drugs in nonhuman primates. III. Self-administration by squirrel monkeys. J Pharmacol Exp Ther 251:150–155. [PubMed] [Google Scholar]
  5. Bickel WK, DeGrandpre RJ, Higgins ST, Hughes JR 1990. Behavioral economics of drug self-admnistration. I. Functional equivalence of response requirement and drug dose. Life Sci 47:1501–1510. Doi: 10.1016/0024-3205(90)90178-t. [DOI] [PubMed] [Google Scholar]
  6. Bradbury S, Bird J, Colussi-Mas J, Mueller M, Ricaurte G, Schenk S 2014. Acquisition of MDMA self-administration: pharmacokinetic factors and MDMA-indueced serotonin release. Addict Biol 19:874–884. Doi: 10.1111/adb.12069. [DOI] [PubMed] [Google Scholar]
  7. Caudevilla-Gálligo F, Ventura M, Indave Ruiz BI, Fornís I 2013. Presence and composition of cathinone derivatives in drug samples taken from a drug test service in Spain (2010–2012). Hum Psychopharmacol 28:341–344. Doi: 10.1002/hup.2296. [DOI] [PubMed] [Google Scholar]
  8. Collins GT, Sulima A, Rice KC, France CP 2019. Self-administration of synthetic cathinones 3,4-methlenedioxypyrovalerone (MDPV) and α-pyrrolidiniopentiophenone (α-PVP) in rhesus monkeys. Psychopharmacology (Berl). 236:3677–3685. Doi: 10.1007/s00213-019-05339-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Creehan KM, Vandewater SA, Taffe MA 2015. Intravenous self-administration of mephedrone, methylone and MDMA in females rats. Neuropharmacology 92:90–97. Doi: 10.1016/neuropharm.2015 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Davies S, Wood DM, Smith G, Button J, Ramsey J, Archer R, Holt DW, Dargan PI 2010. Purchasing ‘legal highs’ on the internet—is there consistency in what you get? QJM 103:489–493. Doi: 10.1093/qjmed/hcq056. [DOI] [PubMed] [Google Scholar]
  11. de la Torre R, Farré M, Ortuño J, Mas M, Brenneisen R, Roset PN, Segura J, Camí J 2000. Non-linear pharmacokinetics of MDMA (‘ecstasy’) in humans. Br J Clin Pharmacol 49:104–109. Doi: 10.1046/j.1365-2125.2000.00121.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. den Hollander B, Dudek M, Ojanperä I, Kankuri E Hyytiä P, Korpi ER 2014. Manganese-enhanced magnetic resonance imaging reveals differential long-term neuroadaptation after methamphetamine and the substituted cathinone 4-methylmethcathinone (mephedrone). Int J Neuropsychopharmacol 18:pyu106. Doi: 10.1093/ijnp/pyu106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Dolan SB, Chen Z, Huang R, Gatch MB 2018. “Ecstasy” to addiction: mechanisms and reinforcing effects of three synthetic cathinone analogs of MDMA. Neuropharmacology 133:171–180. Doi: 10.1016/j.neuropharm.2018.01.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Eshleman AJ, Wolfrum KM, Hatfield MG, Johnson RA, Murphy KV, Janowsky A Substituted methcathinones differ in transporter and receptor interactions. Biochem Pharmacol 85:1803–1815. Doi: 10.1016/j.bcp.2013.04.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Eshleman AJ, Wolfrum KM, Reed JF, Kim SO, Swanson T, Jonhson RA, Janowsky A 2017. Structure-activity relationships of substituted cathinones, with transporter binding, uptake, and release. J Pharmacol Exp Ther 360:33–47. Doi: 10.1124/jpet.116.236349. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Fantegrossi WE, Gannon BM, Zimmerman SM, Rice KC 2013. In vivo effects of abused “bath salt” constituent 3,4-methylenedioxypyrovalerone (MDPV) in mice: drug discrimination, thermoregulation, and locomotor activity. Neuropsychopharmacology 38:563–573. Doi: 10.1038/npp.2012.233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Foster TM, Temple W, Cameron B, Poling A 1997. Demand curves for food in hens: similarity under fixed-ratio and progressive-ratio schedules. Behav Processes 39:177–185. Doi: 10.1016/s0376-6357(96)00051-4. [DOI] [PubMed] [Google Scholar]
  18. Gannon BM, Galindo KI, Rice KC, Collins GT 2017a. Individual differences in the relative reinforcing effects of 3,4-methylenedioxypyrovalerone under fixed and progressive ratio schedules of reinforcement in rats. J Pharmacol Exp Ther 361:181–189. Doi: 10.1124/jpet.116.239376. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Gannon BM, Rice KC, Collins GT 2017b. Reinforcing effects of abused “bath salts” constituents 3,4-methylenedioxypyrovalerone and α-pyrrolidinopentiophenone and their enantiomers. Behav Pharmacol 28:578–581. Doi: 10.1097/FBP.0000000000315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Gannon BM, Baumann MH, Walther D, Jimenez-Morigosa C, Sulima A, Rice KC, Collins GT 2018a. The abuse-related effects of pyrrolidine-containing cathinones are related to their potency and selectivity to inhibit the dopamine transporter. Neuropsychopharmacology 43:2399–2407. Doi: 10.1038/s41386-018-0209-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gannon BM, Galindo KI, Mesmin MP, Rice KC, Collins GT 2018b. Reinforcing effects of binary mixtures of common bath salt constituents: studies with 3,4-methylenedioxypyrovalerone (MDPV), 3,4-methylenedioxymethcathinone (methylone), and caffeine in rats. Neuropsychopharmacology 43:761–769. Doi: 10.1038/npp.2017.141. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Gannon BM, Mesmin MP, Sulima A, Rice KC, Collins GT 2019. Behavioral economic analysis of the reinforcing effects of “bath salts” mixtures: studies with MDPV, methylone, and caffeine in male Sprague-Dawley rats. Psychopharmacology (Berl) 236:1031–1041. Doi: 10.1007/s00213-018-5046-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. German CL, Fleckenstein AE, Hanson GR 2014. Bath salts and synthetic cathinones: an emerging designer drug phenomenon. Life Sci 97:2–8. Doi: 10.1016/j.ifs.2013.07.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Glowa JR, Rice KC, Matecka D, Rothmann RB 1997. Phentermine/fenfluramine decreases cocaine self-administration in rhesus monkeys. NeuroReport 8:1347–1351. Doi: 10.1097/00001756-199704140-00006. [DOI] [PubMed] [Google Scholar]
  25. Gregg RA, Baumann MH, Partilla JS, Bonano JS, Vouga A, Tallarida CS, Velvadapu V, Smith GR, Peet MM, Reitz AB, Negus SS, Rawls SM 2015. Stereochemistry of mephedrone neuropharmacology: enatiomer-specific behavioural and neurochemical effects in rats. Br J Pharmacol 172:883–894. Doi: 10.1111/bph.12951. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Harvey EL, Burroughs RL, Baker LE 2017. Effects of D1 and D2 receptor antagonists on the discriminative stimulus effects of methylendioxypyrovalerone and mephedrone in male Sprague-Dawley rats trained to discriminate D-amphetamine. Behav Pharmacol 28:586–589. Doi: 10.1097/fbp.0000000000328. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Hursh SR, Raslear TG, Shurtleff D, Baumann R, Simmons L 1988. A cost-benefit analysis of demand for food. J Exp Anal Behav 50:419–440. Doi: 10.1901/jeab.1988.50-419. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hursh SR, Silberberg A 2008. Economic demand and essential value. Psychol Rev 155:186–198. Doi: 10.1037/0033-295X.115.1.186. [DOI] [PubMed] [Google Scholar]
  29. Huskinson SL, Naylor JE, Townsend EA, Rowlett JK, Blough BE, Freeman KB 2017. Self-administration and behavioral economics of second-generation synthetic cathinones in male rats. 234:589–598. Doi: 10.1007/s00213-016-4492-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Johnson MW, Bickel WK 2006. Replacing relative reinforcer efficacy with behavioral economic demand curves. J Exp Anal Behav 85:73–93. Doi: 10.1901/jeab.2006.102-04. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Johnston LD, Miech RA, O’Malley PM, Bachman JG, Schulenberg JE, M.E. Patrick 2019. Monitoring the future national survey results on drug use 1975–2018: overview, key findings on adolescent drug use. Institute for Social Research, University of Michigan, Ann Arbor. [Google Scholar]
  32. Kohut SJ, Fivel PA, Mello NK 2013b. Differential effects of acute and chronic treatment with the α2-adrenergic agonist, lofexidine, on cocaine self-administration in rhesus monkeys. Drug Alcohol Depend 133:593–599. Doi: 10.1016/j.drugalcdep.2013.07.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kohut SJ, Bergman J 2016. Reinforcing effectiveness of nicotine nonhuman primates: effects of nicotine dose and history of nicotine self-administration. Psychopharmacology (Berl) 233:2451–2458. Doi: 10.1007/s00213-016-4293-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kohut SJ, Jacobs DS, Rothman RB, Partilla JS, Bergman J, Blough BE 2017. Cocaine-like discriminative stimulus effects of “norepinephrine-preferring” monoamine releasers: time course and interaction studies in rhesus monkeys. Psychopharmacology (Berl) 234:3455–3465. Doi: 10.1007/s00213-017-4731-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Lile JA, Wang Z, Woolverton WL, France JE, Gregg TC, Davies HM, Nader MA 2003. The reinforcing efficacy of psychostimulants in rhesus monkeys: the role of pharmacokinetics and pharmacodynamics. J Pharmacol Exp Ther 307:356–366. Doi: 10.1124/jpet.103.0498525. [DOI] [PubMed] [Google Scholar]
  36. López-Arnau R, Martínez-Clemente J, Pubill D, Escubedo E, Camarasa J 2012. Comparative neuropharmacology of three psychostimulant cathinone derivates: butylone, mephedrone and methylone. Br J Pharmacol 167:407–420. Doi: 10.1111/j.1476-5381.2012.01998.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Marquis KL, Webb MG, Moreton JE 1989. Effects of fixed ratio size and dose on phencyclidine self-administration by rats. Psychopharmacology (Berl) 97:179–182. Doi: 10.1007/bf00442246. [DOI] [PubMed] [Google Scholar]
  38. Marusich JA, Antonazzo KR, Wiley JL, Blough BE, Partilla JS, Baumann MH 2014. Pharmacology of novel synthetic stimulants structurally related to the “bath salts” constituent 3,4-methylenedioxypyrovalernone (MDPV). Neuropharmacology 87:206–213. Doi: 10.1016/j.neuropharm.2014.02.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Miotto K, Striebel J, Cho AK, Wang C 2013. Clinical and pharmacological aspects of bath salt use: a review of the literature and case reports. Drug Alcohol Depend 132:1–12. Doi: 10.1016/j.drugalcdep.2013.06.016. [DOI] [PubMed] [Google Scholar]
  40. Moreton JE, Meish RA, Stark L, Thompson T 1977. Ketamine self-administration by the rhesus monkey. J Pharmacol Exp Ther 203:303–309. [PubMed] [Google Scholar]
  41. Nagai F, Nonaka R, Satoh HKK 2007. The effects of non-medically used psychoactive drugs on monoamine neurotransmission in rat brain. Eur J Pharmacol 22:132–137. Doi: 10.1016/j.ejphar.2006.11.075. [DOI] [PubMed] [Google Scholar]
  42. Negus SS, Mello NK, Blough BE, Baumann MH, Rothman RB 2007. Monoamine releasers with varying selectivity for dopamine/norepinephrine versus serotonin release as candidate “agonist” medications for cocaine dependence: studies in assays of cocaine discrimination and cocaine self-administration in rhesus monkeys. J Pharmacol Exp Ther 320:627–636. Doi: 10.1124/jpet.106.107383. [DOI] [PubMed] [Google Scholar]
  43. Ritz MC, Lamb RJ, Goldberg SR, Kuhar MJ 1987. Cocaine receptors on dopamine transporters are related to self-administration of cocaine. Science 237:1219–1223. Doi: 10.1126/science.2820058. [DOI] [PubMed] [Google Scholar]
  44. Ritz MC, Kuhar MJ 1989. Relationship between self-administration of amphetamine and monoamine receptors in brain: comparisons with cocaine. J Pharmacol Exp Ther 248:1010–1017. [PubMed] [Google Scholar]
  45. Rothman RB, Baumann MH 2003. Monoamine transporters and psychostimulant drugs. Eur J Pharmacol 479: 23–40. Doi: 10.1016/j.ephar.2003.08.054. [DOI] [PubMed] [Google Scholar]
  46. Rothman RB, Baumann MH 2006. Balance between dopamine and serotonin release modulates behavioral effects of amphetamine-type drugs. Ann N Y Acad Sci 1074:245–260. Doi: 10.1196/annals.1369.064. [DOI] [PubMed] [Google Scholar]
  47. Schenk S, Gittings D, Johnstone M, Daniela E 2003. Development, maintenance and temporal pattern of self-administration maintained by ecstasy (MDMA) in rats. Psychopharmacology (Berl) 169:21–27. Doi: 10.1007/s00213-003-1407-0. [DOI] [PubMed] [Google Scholar]
  48. Schenk S, Hely L, Lake B, Daniela E, Gittings D, Mash DC 2007. MDMA self-administration in rats: acquisition, progressive ratio responding and serotonin transporter binding. Eur J Neurosci 26:3229–3336. Doi: 10.1111/j.1460-9568.2007.05932.x. [DOI] [PubMed] [Google Scholar]
  49. Schneir A, Ly BT, Casagrande K, Darracq M, Offerman SR, Thornton S, Smollin C, Vohra R, Rangun C, Tomaszewski C, Gerona RR 2014. Comprehensive analysis of “bath salts” purchased from California stores and the internet. Clin Toxicol (Phila) 52:651–658. Doi: 10.3109/15563650.2014.933231. [DOI] [PubMed] [Google Scholar]
  50. Seely KA, Patton AL, Moran CL, Womack ML, Prather PL, Fantegrossi WE, et al. 2013. Forensic investigation of K2, spice, and “bath salt” commercial preparations: a three-year study of new designer drug products containing synthetic cannabinoid, stimulant, and hallucinogenic compounds. For Sci Int 233:416–422. Doi: 10.1016/j.forsciint.2013.10.002. [DOI] [PubMed] [Google Scholar]
  51. Simmler LD, Buser TA, Donzelli M, Schramm Y, Dieu LH, Huwyler J, Chaboz S, Hoener MC, Liechti ME 2013. Pharmacological characterization of designer cathinones in vitro. Br J Pharmacol 168: 458–470. Doi: 10.1111/j.1476-5381.2012.02145.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Spiller HA, Ryan ML, Weston RG, Jansen J 2014. Clinical experience with and analytical confirmation of “bath salts” and “legal highs” (synthetic cathinones in the United States. Clin Toxicol (Phila) 49:499–505. Doi: 10.3109/15563650.2011.590812. [DOI] [PubMed] [Google Scholar]
  53. Volkow ND, Fowler JS, Wang GJ 2004. The addicted brain viewed in the light of imaging studies: brain circuits and treatment strategies. Neuropharmcology 47:3–13. Doi: 10.1016/j.neuropharm.2004.07.019. [DOI] [PubMed] [Google Scholar]
  54. Wade-Galuska T, Winger G, Woods JH 2007. A behavioral economic analysis of cocaine and remifentanil self-administration in rhesus monkeys. Psychopharmacology (Berl) 194:563–572. Doi: 10.1007/s00213-007-0858-0. [DOI] [PubMed] [Google Scholar]
  55. Wee S, Anderson KG, Baumann MH, Rothmann RB, Blough BE, Woolverton WL 2005. Relationship between the serotonergic activity and reinforcing effects of a series of amphetamine analogs. J Pharmacol Exp Ther 313:848–854. Doi: 10.1124/jpet.104.080101. [DOI] [PubMed] [Google Scholar]
  56. Wee S, Woolverton WL 2006. Self-administration of mixtures of fenfluramine and amphetamine by rhesus monkeys. Pharmacol Biochem Behav 84:337–343. Doi: 10.1016/j.pbb.2006.05.022. [DOI] [PubMed] [Google Scholar]
  57. Woods JH, Tessel RE 1974. Fenfluramine: amphetamine congener that fails to maintain drug-taking behavior in the rhesus monkey. Science 185:1067–1069. Doi: 10.1126/dcience.185.4156.1067. [DOI] [PubMed] [Google Scholar]
  58. Woolverton WL 1992. Determinants of cocaine self-administration by laboratory animals. Ciba Found Symp 166:149–166. Doi: 10.1002/9789470514245.ch9. [DOI] [PubMed] [Google Scholar]
  59. Zuba D, Byrska B 2013. Prevalence and co-existence of active components of ‘legal highs’. Drug Test Anal 5:420–429. Doi: 10.1002/dta/1365. [DOI] [PubMed] [Google Scholar]

RESOURCES