Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 Aug 1.
Published in final edited form as: Drug Alcohol Depend. 2016 May 28;165:103–110. doi: 10.1016/j.drugalcdep.2016.05.021

Development of a Translational Model to Screen Medications for Cocaine Use Disorder I: Choice Between Cocaine and Food in Rhesus Monkeys*

Amy R Johnson a, Matthew L Banks a, Bruce E Blough b, Joshua A Lile c, Katherine L Nicholson a, S Stevens Negus a
PMCID: PMC4939093  NIHMSID: NIHMS792280  PMID: 27264165

Abstract

Background

Homologous cocaine self-administration procedures in laboratory animals and humans may facilitate translational research for medications development to treat cocaine dependence. This study, therefore, sought to establish choice between cocaine and an alternative reinforcer in rhesus monkeys responding under a procedure back-translated from previous human studies and homologous to a human laboratory procedure described in a companion paper.

Methods

Four rhesus monkeys with chronic indwelling intravenous catheters had access to cocaine injections (0, 0.043, 0.14, or 0.43 mg/kg/injection) and food (0, 1, 3, or 10 1g banana-flavored food pellets). During daily 5h sessions, a single cocaine dose and a single food-reinforcer magnitude were available in 10 30-min trials. During the initial “sample” trial, the available cocaine and food reinforcer were delivered non-contingently. During each of the subsequent nine “choice” trials, responding could produce either the cocaine or food reinforcer under an independent concurrent progressive-ratio schedule.

Results

Preference was governed by the cocaine dose and food-reinforcer magnitude, and increasing cocaine doses produced dose-dependent increases in cocaine choice at all food-reinforcer magnitudes. Effects of the candidate medication lisdexamfetamine (0.32–3.2 mg/kg/day) were then examined on choice between 0.14 mg/kg/injection cocaine and 10 pellets. Under baseline conditions, this reinforcer pair maintained an average of approximately 6 cocaine and 3 food choices. Lisdexamfetamine dose-dependently decreased cocaine choice in all monkeys, but food choice was not significantly altered.

Conclusions

These results support utility of this procedure in rhesus monkeys as one component of a platform for translational research on medications development to treat cocaine use disorder.

Keywords: cocaine, choice, rhesus monkey, human, translation, lisdexamfetamine

1. INTRODUCTION

Cocaine use disorder remains a significant clinical challenge for which there are no medications currently approved by the Food and Drug Administration. Research to evaluate the efficacy and safety of new medications for drug abuse or other disorders benefits from a translational path from preclinical to clinical studies, and a key step along this path occurs at the transition from research in animals to human laboratory studies (Comer et al., 2008; Haney and Spealman, 2008; Mello and Negus, 1996; Rush and Stoops, 2012). A change in species is unavoidable at this transition; however, the fidelity of translation may benefit from (1) the use of nonhuman primates as animal subjects due to their high biological similarity with humans, especially with regard to monoaminergic systems affected by cocaine and many candidate medications (Weerts et al., 2007; Bradberry, 2008), and (2) the use of homologous experimental procedures that minimize discrepancies in variables other than species and maximize potential for direct comparison of results across species (Czoty et al., 2016b; Foltin et al., 2015; Yu, 2011). In particular, recent reviews on medications development for drug abuse research have suggested that translation is optimized by use of procedures that test effects of medication maintenance on choice between the abused drug and an alternative non-drug reinforcer (Haney and Spealman, 2008; Jones and Comer, 2013; Banks et al., 2015a; Czoty et al., 2016b). Although choice procedures have been developed for use in both animals and humans, there remain discrepancies in many procedural details for the most commonly used approaches (Banks et al., 2012; Jones and Comer, 2013). As a result, the opportunity exists to achieve closer alignment between preclinical and human laboratory choice procedures as one strategy to facilitate preclinical to clinical translation of results obtained with those procedures

As one step to address this opportunity, the goal of this project and the companion study conducted in humans (Lile et al., 2016; see companion paper in this issue) was to develop homologous drug self-administration procedures in nonhuman primates and humans as a platform for translational research on candidate medications to treat drug abuse. In particular, these studies sought to harmonize three sets of procedural variables: (1) the route and doses of self-administered cocaine, (2) the schedule of reinforcement that governed availability of cocaine and an alternative non-drug reinforcer, and (3) the treatment regimen for delivery of a candidate medication. With regard to the schedule of self-administration, previous human-laboratory studies have identified concurrent independent progressive-ratio schedules of choice between drug and money as a sensitive tool for medication evaluation (Jones and Comer, 2013; Moeller and Stoops, 2015; Stoops et al., 2012; Sullivan et al., 2006). Accordingly, cocaine self-administration was established in rhesus monkeys and human subjects under nearly identical concurrent independent progressive-ratio schedules of choice between cocaine and a species-specific non-drug alternative reinforcer (food in monkeys; money in humans). The cocaine dose and magnitude of the non-drug alternative were then systematically manipulated in each species, with the same unit doses of cocaine being used in both species. Results are reported here for the study in nonhuman primates and in a companion paper for the study in human subjects (Lile et al., 2016; this issue). We hypothesized that comparable patterns of cocaine choice could be demonstrated in rhesus monkeys and humans, and that specific parameters of cocaine dose and alternative reinforcer magnitude could be identified for subsequent evaluation of candidate medications in both species.

The present study also evaluated effects of lisdexamfetamine as a representative candidate medication. Lisdexamfetamine is an amphetamine prodrug approved for treatment of ADHD and compulsive eating disorder (Blick and Keating, 2007; Hutson et al., 2014), and it was selected for initial testing because preclinical and clinical research suggests that it might also be useful for treating cocaine use disorder (Banks et al., 2015; Mooney et al., 2015). Furthermore, maintenance on its metabolite, d-amphetamine, has been shown to decrease cocaine self-administration across a broad range of experimental conditions in rats, rhesus monkeys, human-laboratory studies, and placebo-controlled, double-blind clinical trials (Herin et al., 2010; Negus and Henningfield, 2015; Nuijten et al., 2016). Each lisdexamfetamine dose was tested using a subchronic, 7-day treatment regimen, because medications to treat drug use disorders are administered chronically in humans, and it has been argued that preclinical animal- and human-laboratory studies should also evaluate effects of repeated treatment delivery to more accurately predict clinical effectiveness (Czoty et al., 2016b; Banks et al., 2015a; Haney and Spealman, 2008; Mello and Negus, 1996). We hypothesized that 7-day treatment with lisdexamfetamine would produce a dose-dependent decrease in cocaine choice and a reciprocal increase in choice of the food alternative in this concurrent independent progressive-ratio choice procedure.

2. METHODS

2.1. Subjects

Studies were conducted in four adult male rhesus monkeys (Macaca mulatta) with various drug histories. One monkey (M1524) was experimentally naïve at the start of the study. A second monkey (M1501) was exposed to two one-month regimens of oxycodone administration over a period of one year, and these studies terminated 8 months before transfer to choice studies. The third monkey (M1498) was exposed to three one-month regimens of oxycodone administration over a period of approximately 16 months. This monkey was then involved in drug self-administration studies with phencyclidine (two months) and cocaine (six months), and these studies terminated one month before transfer to choice studies. The fourth monkey (M1416) had a history of morphine dependence as a juvenile, followed by a history of heroin self-administration (three months), cocaine self-administration (five months), and cocaine discrimination (four years) as an adult. These studies terminated 1 month before transfer to choice studies. Monkeys M1498 and M1416 also received acute exposure to various classes of test drugs, and all monkeys had intermittent exposure to ketamine as an anesthetic for husbandry procedures.

Each monkey had a surgically implanted venous catheter with a single lumen (Braintree Inc., Braintree, MA) or double lumen (STI Components, Roanoke, VA). Monkeys could earn 1g banana-flavored pellets (5TUR Grain-based Precision Primate Pellets; Test Diets, St. Louis, MO) during daily experimental sessions. In addition, monkeys received daily food rations (Lab Diet High Fiber Monkey Biscuits; PMI Feeds, St. Louis, MO), and the biscuit ration size was individually determined for each monkey to maintain a healthy body weight. Biscuit rations were delivered in the afternoons after behavioral sessions to minimize the effects of biscuit availability and consumption on food-maintained operant responding. Animals also received fresh fruit 7 afternoons per week. Water was continuously available in each monkey’s home chamber, which also served as the experimental chamber. A 12h light/dark cycle was in effect (lights on from 0600 to 1800 h). Environmental enrichment (foraging devices, novel treats, movies and music) was also provided after behavioral sessions. Facilities were accredited by the Association for Assessment and Accreditation of Laboratory Animal Care. The Institutional Animal Care and Use Committee approved all experimental protocols.

2.2. Apparatus

Each home cage was equipped with an operant response panel, which had two response levers with three stimulus lights above each lever. The lights over the left and right levers were white and red, respectively. Additionally, the cages were equipped with a pellet dispenser that delivered food pellets to a receptacle within the cage. The externalized section of the intravenous catheter for drug self-administration was routed through a jacket and tether system (Lomir Biomedical, Quebec, Canada) to the rear of the cage and connected to a peristaltic fluid pump (Cole-Parmer, Chicago, IL). Catheter patency was periodically evaluated with intravenous (IV) ketamine (4 mg/kg) administration, and the catheter was considered patent if IV ketamine administration produced overt loss of muscle tone within 20 sec.

2.3. Single Alternative Training

Initial training for food-maintained responding proceeded in a series of incremental steps, during which only one lever and associated stimulus lights were active (the “food-associated lever,” counterbalanced between monkeys). Under the terminal progressive-ratio (PR) schedule, daily 5-h behavioral sessions consisted of 10 discrete 30-min trials. The first trial was a “sample” trial, in which subjects received non-contingent delivery of 10 pellets. The remaining 9 trials were “response” trials, in which food pellets were available under the PR schedule. Stimulus lights were illuminated over the lever at the start of each trial, and completion of the ratio requirement produced food pellet delivery, initiated a time out (TO) for the remainder of the trial, and incremented the ratio for the next trial. If a monkey failed to complete the ratio requirement within 30 min, the trial terminated without reinforcement, the response counter reset to “0,” a 1-min TO period ensued, and the ratio requirement did not increment for the next trial. The starting ratio was 200 in 2 monkeys (M1498, M1524) and 400 in the other 2 monkeys (M1416, M1501), and the increment after each completed ratio was 100 for all monkeys (i.e., PR values were 200, 300, 400…1000 for two monkeys; 400, 500, 600…1200 for the other 2 monkeys). The lower starting ratio was used in two monkeys because they failed to complete ≥8 trials with higher starting ratios. Once monkeys reliably completed ≥8 trials for the 10-pellet reinforcer magnitude under the terminal schedule, a pellet magnitude-effect curve was determined at magnitudes of 0, 1, 3 and 10 pellets. During these studies, the designated pellet magnitude was delivered non-contingently during the sample trial of each daily session, and responding under the PR schedule produced this pellet magnitude during subsequent response trials. Each pellet magnitude was presented for a minimum of 7 consecutive days and until responding stabilized (number of trials completed for the last 3 days within 1 of the running mean, with no increasing or decreasing trends). Responding maintained by 10 pellets was determined first in all monkeys, and the remaining pellet magnitudes were studied in a mixed order across monkeys.

Once the pellet magnitude-effect curve was completed, an intravenous catheter was surgically implanted using aseptic procedures, and cocaine training began. The training regimen for cocaine self-administration was identical to that for food-maintained responding with the exception that the other lever and associated stimulus lights were active (the “cocaine-associated lever”), and responding produced intravenous cocaine injections. Training proceeded until responding maintained by 0.43 mg/kg/injection cocaine was stable under the same terminal schedule used for food in that monkey (i.e., starting ratio of 200 in 2 monkeys and 400 in the other 2 monkeys, with an increment of 100 in all monkeys). Subsequently, a cocaine dose-effect curve was determined at doses of 0, 0.043, 0.14 and 0.43 mg/kg/injection cocaine using test durations and stability criteria identical to those used for the pellet magnitude-effect curve. The cocaine doses were selected to match approximate unit cocaine doses used in the parallel human-laboratory study (i.e., 0.043, 0.14 and 0.43 mg/kg/injection unit doses in monkeys are equivalent to doses of 3, 10 and 30 mg for a 70 kg human subject; Lile et al., 2016; see companion paper in this issue). Responding maintained by 0.43 mg/kg/injection was determined first in all monkeys, and the remaining doses were studied in a mixed order across monkeys.

2.4. Cocaine vs. Food Choice Procedure

After determination of magnitude-effect functions for food and cocaine alone, concurrent-choice studies were initiated to assess cocaine choice dose-effect curves during concurrent availability of 1, 3 or 10 pellets. Choice session were identical to sessions under the terminal schedule for food or cocaine alone with the following exceptions: (1) a single pellet magnitude and a single cocaine dose were concurrently available, (2) both the food and drug reinforcers available during that session were delivered non-contingently at the start of the sample trial, with food delivered first, and cocaine delivered 5 min later, (3) both food- and cocaine-associated levers were active at the start of each choice trial, and lights above both levers were illuminated, (4) the first response during each trial locked in choice for that reinforcer during that trial, deactivated the alternative lever, and extinguished lights above the alternative lever, and (5) completion of a ratio produced the chosen reinforcer and incremented the ratio requirement only for that reinforcer in the next trial. If a monkey failed to complete a ratio requirement within 30 min, then the trial terminated without reinforcement, the response counter reset to “0” for both levers, a 1-min TO period ensued, the ratio did not increment for either reinforcer for the next trial, and the trial was counted as an “omission.” Each combination of pellet magnitude and cocaine dose was in effect for 7 consecutive days, and all cocaine doses were tested in combination with a single pellet magnitude before proceeding to a different pellet magnitude. Both the order of cocaine doses within a pellet magnitude and the order of pellet magnitudes were randomized across monkeys.

2.5. Effects of Lisdexamfetamine

Prior to testing lisdexamfetamine, choice performance was first re-established between 0.14 mg/kg/injection cocaine and 10 pellets (see Results for rationale). Each lisdexamfetamine dose (0.32, 1.0, 1.8, and 3.2 mg/kg/day) was tested for 7 consecutive days, and baseline choice performance was re-established over a period of at least 4 days between each 7-day lisdexamfetamine dose test. On test days, lisdexamfetamine was administered by slow IV infusion over a period of 30 min beginning 1 h before the start of the daily choice session. The order in which lisdexamphetamine doses were tested was randomized across monkeys. Throughout the study, including during lisdexamfetamine studies, measurement of choice variables was supplemented by daily observation of overt behavior for signs of behavioral toxicity (e.g., abnormal postures or behaviors, convulsions, reduced consumption of food rations), and provisions were in place to terminate treatment in the event that behavioral toxicity was observed. No treatment in this study was terminated; however, in one monkey, the dose of 1.8 mg/kg/day lisdexamfetamine was tested before scheduled testing of a higher dose of 3.2 mg/kg/day, and in this monkey 1.8 mg/kg/day lisdexamfetamine fully suppressed responding. To avoid the potential for toxicity, 3.2 mg/kg/day lisdexamfetamine was not tested in this subject.

2.6. Data Analysis

The primary dependent variables were the mean numbers of cocaine choices, food choices, and omissions per session. Data from the last three days of each test condition were first averaged within a monkey and then averaged across monkeys to generate group means. Data were analyzed by one- or two-way repeated-measures ANOVA, as appropriate, and a significant ANOVA was followed by either a Dunnet’s or Holm-Sidak post hoc test. The criterion for significance was p<0.05. Also, as shown in below in Results, lisdexamfetamine decreased cocaine choices in all monkeys, but effects on food choice varied across monkeys. Accordingly, selected individual data with lisdexamfetamine are also shown.

2.7. Drugs

(−)-Cocaine HCl (NIDA, Rockville, MD) and lisdexamfetamine mesylate (B. E. Blough, Research Triangle Institute) were dissolved in sterile saline for IV injection.

3. RESULTS

3.1. Responding maintained by food or cocaine alone

Training took an average of 2.5 months to reach the terminal schedule of food presentation (range = 43 – 142 days). Food pellets maintained a magnitude-dependent increase in responding (Figure 1A). When 0 pellets were available, subjects completed an average of approximately 1 ratio requirement. As the number of pellets available increased, subjects increased the number of trials completed (F3,9 = 17.96, p < 0.001), such that an average of approximately 8 trials were completed when 10 pellets were available.

Figure 1. Effects of reinforcer magnitude in units of pellet number (A) or cocaine dose (B) on the number of trials completed during single-alternative training.

Figure 1

Open in a new tab

Each condition was presented for a minimum of 7 days and until stable responding was observed. All points show mean±SEM for the final 3 days in 4 monkeys. Asterisks (*) indicate statistical significance (p < 0.05) compared to 0 pellets (A) or saline (B).

Cocaine self-administration training took an average of 34 days to reach the terminal schedule (range = 24 – 48 days), and cocaine also maintained a dose-dependent increase in responding (Figure 1B). When saline was available, subjects completed an average of approximately 1 ratio requirement. As the dose of cocaine increased, the number of trials completed increased (F3,9 = 53.42, p < 0.0001), such that an average of at least 8 trials were completed during availability of 0.14 and 0.43 mg/kg/injection cocaine.

3.2. Choice between food and cocaine

Figure 2 shows the mean numbers of completed cocaine trials, completed food trials, and omissions during the final three days for each cocaine dose at each pellet magnitude. Data within each panel were analyzed by two-way ANOVA [cocaine dose (0, 0.043, 0.14 and 0.43 mg/kg/injection) x trial outcome (cocaine choice, food choice, or omission)], and this analysis revealed a significant interaction at each pellet magnitude (Panel A: F6,18 = 9.03, p < 0.001; Panel B: F6,18 = 10.82, p < 0.0001; Panel C: F6,18 = 17.02, p < 0.0001). Across all 3 pellet magnitudes, cocaine maintained a dose-dependent increase in the number of cocaine trials completed, and doses of 0.14 and 0.43 mg/kg/injection were always chosen in significantly more trials than saline, as denoted by asterisks over open bars in Figure 2A–C. Similarly, across all three pellet magnitudes, the mean number of food trials completed tended to decrease as cocaine dose increased; however, this trend was significant only during availability of 3 and 10 pellets. Under those conditions, the number of food choices was higher during concurrent availability of saline than during concurrent availability of 0.14 and 0.43 mg/kg/injection cocaine, as denoted by asterisks over closed bars in Figure 2B,C. Omissions tended to be highest when low magnitudes of the food and cocaine reinforcers were concurrently available (e.g., during concurrent availability of 1 pellet and saline injections in Figure 2A), and the mean number of omissions tended to decrease as cocaine dose increased. This tendency attained significance during the availability of 1 pellet, when the number of omissions was higher during availability of saline than during availability of 0.14 and 0.43 mg/kg/injection cocaine, as denoted by asterisks over gray bars in Figure 2A).

Figure 2. Trials completed for either cocaine or food when 1 pellet (A), 3 pellets (B) or 10 pellets (C) was available as the alternative to cocaine (0–0.43 mg/kg/injection).

Figure 2

Open in a new tab

Each combination of cocaine dose and pellet reinforcer magnitude was available for 7 days. All bars show mean ± SEM for the final 3 days in 4 monkeys. Asterisks (*) indicate statistical significance (p < 0.05) within a trial outcome (cocaine choice, food choice, or omission) compared to the 0 cocaine data. Dollar signs ($) indicate statistical significance (p<0.05) within a cocaine dose between the numbers of cocaine vs. food trials completed.

The analysis of choice results as shown in Figure 2 also permitted evaluation of preference between food and cocaine at each combination of pellet magnitude and cocaine dose (see dollar signs in Figure 2A–C). Both 0.14 and 0.43 mg/kg/injection cocaine were preferred to 1 pellet (Figure 2A). During availability of 3 pellets, food was preferred to saline injections, whereas 0.14 and 0.43 mg/kg/injection cocaine were preferred to food (Figure 2B). During availability of 10 pellets, food was preferred to saline and the lowest dose of 0.043 mg/kg/injection cocaine, whereas the highest dose of 0.43 mg/kg/injection cocaine was preferred to food (Figure 2C). During the availability 10 pellets, preference for the 0.14 mg/kg/injection cocaine dose was not significant.

3.3. Effects of lisdexamfetamine treatment

Choice between 0.14 mg/kg/injection cocaine and 10 pellets was selected as the baseline for the experiment with lisdexamfetamine because (1) it yielded a trend, albeit non-significant, toward cocaine preference (approximately 6 cocaine and 3 food trials completed) with few omissions, and (2) a reduction in cocaine dose produced reallocation of choice that resulted in significant preference for this food magnitude, again with few omissions. Thus, behavior maintained by this pair of reinforcer magnitudes was likely to be sensitive to reductions in the relative reinforcing efficacy of cocaine during pharmacological treatment. As a prelude to presentation of lisdexamfetamine effects, Figure 3A shows four hypothetical changes in choice between 0.14 mg/kg/injection cocaine and 10 pellets that could be observed during candidate medication treatment. Outcome #1 is interpreted as therapeutically desirable and consists of a decrease in cocaine trials completed with a reciprocal increase in food trials completed. This outcome indicates a reallocation of behavior from cocaine choice to food choice and a decrease in the relative reinforcing efficacy of cocaine in comparison to food. Outcomes #2–4 show three other possible outcomes interpreted as therapeutically undesirable. Specifically, outcome #2 shows a concurrent decrease in both cocaine and food trials with an increase in omissions, suggestive of non-selective behavioral suppression; outcome #3 shows an increase in completed cocaine trials with a reciprocal decrease in food trials, suggestive of increased relative reinforcing efficacy of cocaine; and outcome #4 shows no treatment effect. Of course, graded outcomes between these extremes are also possible.

Figure 3. Hypothetical treatment effects (A) or lisdexamfetamine effects (B) on choice between 0.14 mg/kg/injection cocaine and 10 pellets.

Figure 3

Open in a new tab

The four types of hypothetical outcomes in Panel A are described in text. Panel B shows mean ± SEM for the final 3 days in 4 monkeys (0–1.8 mg/kg/day lisdexamfetamine) or 3 monkeys (3.2 mg/kg/day lisdexamfetamine). Asterisks (*) indicate statistical significance (p < 0.05) within a trial outcome (cocaine choice, food choice, omission) compared to the 0 lisdexamfetamine treatment dose in Panel B. Dollar signs ($) indicate statistical significance (p<0.05) within a lisdexamfetamine dose between the numbers of cocaine vs. food trials completed.

Figure 3B shows choice between 0.14 mg/kg/injection cocaine and 10 pellets during 7-day treatments with different lisdexamfetamine doses (0, 0.32, 1.0, 1.8, and 3.2 mg/kg/day) Data were analyzed by two-way ANOVA (trial outcome x lisdexamfetamine dose), which revealed a significant interaction (F6,18 = 4.82, p < 0.01). Lisdexamfetamine doses of 0.32 and 1.0 mg/kg/day did not significantly alter cocaine or food trials completed or the number of omissions. A dose of 1.8 mg/kg/day lisdexamfetamine decreased cocaine trials completed, had no effect on completed food trials, and increased omissions. The high dose of 3.2 mg/kg/day lisdexamfetamine was tested in only 3 monkeys and produced a profile of effects similar to 1.8 mg/kg/day lisdexamfetamine.

Lisdexamfetamine time course effects are shown in Supplemental Figure 11, and saline substitution effects are included for comparison. Under baseline conditions, preference between 0.14 mg/kg/day cocaine and 10 pellets was relatively stable across all 7 days. Saline substitution decreased the number of trials completed on the cocaine-associated key and produced a reciprocal increase in food trials completed. This reallocation of behavior was evident on day 1 and sustained throughout the 7-day experiment. Lisdexamfetamine produced a dose-and time-dependent decrease in cocaine trials completed while having smaller and more transient effects on completed food trials. Thus, the decline in cocaine choice was associated with sustained food choice and an increase in trial omissions.

Individual subject data during 1.8 mg/kg/day lisdexamfetamine treatment are shown in Supplemental Figure 22. This dose of lisdexamfetamine decreased the number of completed cocaine trials in all four monkeys, but the degree to which this decrease in cocaine choice was accompanied by a reciprocal increase in food choice varied across monkeys. Monkey 1501 showed full reallocation of behavior from cocaine to food choice without an increase in omissions. Monkeys 1498 and 1416 showed only partial reallocation of behavior from cocaine to food choice together with small increases in omissions (Figure 5B–C). Finally, in Monkey 1524, lisdexamfetamine decreased both cocaine and food trials completed together with an increase in omissions. The lower dose of 1.0 mg/kg/day lisdexamfetamine produced little change in cocaine vs. food choice in this monkey. There was no clear relationship between individual differences in sensitivity to lisdexamfetamine and individual differences in other variables such as drug history or starting ratio.

4. DISCUSSION

The goal of this study was to develop a cocaine-vs.-food choice procedure in rhesus monkeys homologous to a cocaine-vs.-money procedure in humans as an experimental tool to facilitate translational research for the development of medications to treat cocaine use disorder (Lile et al., 2016; this issue). There were three main findings. First, rhesus monkeys could be trained to choose between cocaine and food under a concurrent progressive-ratio schedule back-translated from a procedure used in humans to study choice between cocaine and money. Second, the allocation of behavior between cocaine and food varied systematically as a function of cocaine dose and food reinforcer magnitude. Moreover, these results for choice between cocaine and food in monkeys correlated with choice between identical cocaine doses and money in humans (Lile et al., 2016; this issue). Lastly, repeated 7-day treatment with the candidate medication lisdexamfetamine produced a dose-and time-dependent decrease in cocaine choice in all monkeys, and did not significantly impact food choice. These results illustrate the use of the procedure to study a candidate medication and provide qualified support for further consideration of lisdexamfetamine maintenance to treat cocaine use disorder.

4.1. Choice between cocaine and food

This study extends the range of conditions under which cocaine-vs.-food choice has been established in rhesus monkeys (Foltin et al., 2015; Nader and Woolverton, 1991; Negus, 2003; Paronis et al., 2002; Woolverton and Balster, 1981). Specifically, this study used a concurrent independent progressive-ratio procedure to mimic drug-vs.-money choice procedures used previously in human laboratory studies in general (Jones and Comer, 2013; Moeller and Stoops, 2015; Stoops et al., 2012; Sullivan et al., 2006) and to match the cocaine-vs.-money choice procedure used in the companion human laboratory study in particular (Lile et al., 2016; this issue). As such, this study represents an example of back-translation, in which a procedure originally developed for use in humans was modified for use in laboratory animals. Back-translation is one approach that has been used in other disciplines to strengthen the procedural concordance between animal and human studies and improve the predictive power of forward animal-to-human translational research (Insel et al., 2013; Keeler and Robbins, 2011). This approach of back-translation has also been recommended as a strategy to strengthen translational research on medications development for cocaine abuse (Czoty et al., 2016b). To our knowledge, this is the first demonstration of cocaine vs. food choice by rhesus monkeys under this type of schedule, although both food and cocaine-maintained responding have been established separately under progressive-ratio schedules in rhesus monkeys (Bedford et al., 1978; Negus and Mello, 2003b; Rowlett et al., 1996; Stafford et al., 1999).

Previous studies have demonstrated that choice between cocaine and food is sensitive to manipulation of both the cocaine dose and food-reinforcer magnitude in both rhesus monkeys (Nader and Woolverton, 1991; Negus, 2003) and rats (Thomsen et al., 2013). In the present study, similar effects were obtained. In general, increasing the magnitude of the available cocaine dose resulted in increased cocaine choice and decreased food choice, whereas increasing the magnitude of the food reinforcer increased food choice and decreased cocaine choice. The reciprocal effects of reinforcer magnitude on preference were especially apparent when cocaine dose was manipulated during concurrent availability of 10 pellets. Under these conditions, increasing cocaine doses produced a systematic shift from robust food preference to robust cocaine preference, and omissions were rare. These results in the monkey cocaine-vs.-food choice procedure closely approximate the shift from money preference to cocaine preference produced by increasing cocaine doses in the human cocaine-vs.-money choice procedure described in the companion manuscript (Lile et al., 2016; this issue). This concordance in results from monkey and human cocaine choice procedures provides one source of evidence to support predictive validity of these homologous procedures for translational research on determinants of cocaine choice.

It is also notable that the dose-dependent increases in cocaine-vs.-food choice in rhesus monkeys observed here and in a previous study (Foltin et al., 2015) were obtained under discrete-trial procedures that limited the frequency of cocaine injections. These findings contrast with a recent report suggesting that cocaine vs. saccharin preference could be established in rats when intervals between choice opportunities were short (0 or 1 min) but not when inter-trial intervals were longer (10 min) (Vandaele et al., 2015). The reasons for this discrepancy are not clear and may be related to various procedural differences including species and identity of the non-drug alternative reinforcer; however, in the present study using discrete 30-min trials, the highest cocaine dose (0.43 mg/kg/inj) was preferred to food at all food magnitude alternatives.

4.2. Effects of 7-day lisdexamfetamine treatment on cocaine vs. food choice

Results of the present study confirm and extend previous reports that cocaine vs. food choice can be reduced by maintenance either on lisdexamfetamine in rhesus monkeys (Banks et al., 2015b) or on its primary metabolite amphetamine in rhesus monkeys or rats (Banks et al., 2013; Negus, 2003; Thomsen et al., 2013). Amphetamine maintenance also decreased cocaine self-administration maintained under other, non-choice schedules of reinforcement in rhesus monkeys and rats (Chiodo et al., 2008; Czoty et al., 2010; Negus and Mello, 2003a, 2003b), as well as cocaine choice in human laboratory studies and metrics of cocaine use in clinical trials (Grabowski et al., 2001; Levin et al., 2015; Nuijten et al., 2016; Rush et al., 2010; Stoops and Rush, 2013). The present proof-of-concept study used intravenous administration to permit precise control of the lisdexamfetamine dose, although an oral formulation is approved for use in humans and would likely be used in clinical studies. Future translational studies with candidate medications might benefit from use of the same route of administration for treatment drugs in monkeys and humans to parallel use of the same route of administration for cocaine (e.g., Czoty et al., 2016a). Additionally, lisdexamfetamine was administered here for a duration of 7 days to match anticipated dosing regimens for human laboratory studies; however, longer treatment regimens would also be possible in preclinical studies to provide insight on the longer treatment durations that would be used for clinical trials.

A recent double-blind, placebo-controlled pilot clinical trial found that lisdexamfetamine maintenance was not significantly better than placebo at reducing cocaine use in a group of 43 cocaine-dependent individuals (Mooney et al., 2015). However, three caveats warrant mention in comparing that clinical trial in humans to the present study in monkeys. First, the highest dose evaluated in that clinical trial was 70 mg/day, which is approximately equivalent to the dose of 1 mg/kg/day dose tested in monkeys. Both 70 mg/day lisdexamfetamine in humans and 1 mg/kg/day lisdexamfetamine in monkeys produced similar, small and non-significant decreases in metrics of cocaine use. Thus, there was evidence for concordance in effects produced by similar lisdexamfetamine doses in humans and monkeys. Second, the authors of the clinical trial appreciated the impact of regulatory constraints on the doses selected for initial testing, and they noted that “evaluation of higher doses of lisdexamfetamine may provide clearer evidence of its efficacy in treating cocaine dependence.” Results of the present study illustrate how preclinical studies might be useful to address this type of suggestion and inform decisions on whether to pursue testing of higher doses in humans. Specifically, the present study found that cocaine choice was significantly reduced by a higher dose of 1.8 mg/kg/day lisdexamfetamine in monkeys (equivalent to 126 mg/day in a 70 kg human), and this supports the speculation by the clinical trial authors that higher lisdexamfetamine doses might also be more effective to decrease cocaine use in humans. Lastly, the clinical trial revealed individual differences in some adverse events, in medication adherence, and in study retention. The present study also identified individual differences in undesirable lisdexamfetamine effects in monkeys. Specifically, although lisdexamfetamine significantly reduced choice of 0.14 mg/kg/inj cocaine doses in all subjects, the degree to which this decrease in cocaine choice was accompanied by a reciprocal increase in food choice varied across subjects. This variability in lisdexamfetamine effectiveness to promote behavioral reallocation to food choice observed in the present study may be related to the individual differences in the adverse effects of lisdexamfetamine in humans, which would further support the concordance between non-human primate data using these procedures and clinical trial results.

The results of the present study with lisdexamfetamine provide initial evidence for sensitivity of this procedure in rhesus monkeys to effects of a representative candidate medication. As such, these results provide a preclinical treatment profile that can be compared to results with other candidate medications as they are tested in the future. In particular, it would be of interest to identify treatments that not only reduce cocaine choice, but that also produce a more robust and reliable reallocation of responding to food choice than was produced here by lisdexamfetamine. Additionally, these results provide an outcome in monkeys that could be directly compared to results obtained in the complementary cocaine-vs.-money choice procedure in humans. A comparison of treatment effects with lisdexamfetamine and other candidate medications on cocaine choice in rhesus monkeys and humans will be important for continued validation and refinement of this platform for translational research.

Supplementary Material

supplement

Highlights.

  • Homologous procedures were established for cocaine choice in monkeys and humans

  • Choice was governed by cocaine dose and magnitude of the alternative non-drug reinforcer

  • Cocaine choice dose-effect curves in monkeys and humans were highly correlated

  • Lisdexamfetamine treatment selectively decreased cocaine choice in monkeys

  • This platform will be useful for translational testing of candidate medications

Acknowledgments

Role of Funding Source This research and the preparation of this manuscript were supported by grants awarded to Dr. Joshua Lile (National Institute on Drug Abuse grants K02 DA031766 and R01 DA033364) as well as a training grant awarded to Virginia Commonwealth University (NIDA grant T32 DA007027). Additionally, ARJ received a Travel Award to present this work to the 2015 meeting of the International Study Group Investigating Drugs as Reinforcers (ISGIDAR). These funding sources had no further role in study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.

The authors would like to thank Ellen Soehngen and James Gillespie for technical contributions to the project.

Footnotes

*

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

1

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

2

Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi:…

Contributors

Ms. Johnson and Drs. Banks, Blough, Lile, Nicholson and Negus designed the study and managed literature searches and summaries of previous related work. Ms. Johnson and Dr. Negus wrote the protocol and the first draft of the manuscript. Ms. Johnson and Drs. Lile and Negus undertook the statistical analysis and graphical representation of the data. Ms. Johnson and Drs. Banks, Nicholson and Negus oversaw conduct of the study. Dr. Blough provided lisdexamfetamine and also provided insights on its pharmacology relevant to experimental design. All authors contributed to, and have approved, the final manuscript.

Conflict of Interest

There are no relevant conflicts of interest to declare. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Dr. Negus has consulted on topics unrelated to this manuscript with Alkermes, Depomed, and Grunenthal.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  1. Banks ML, Blough BE, Negus SS. Effects of 14-day treatment with the schedule III anorectic phendimetrazine on choice between cocaine and food in rhesus monkeys. Drug Alcohol Depend. 2013;131:204–213. doi: 10.1016/j.drugalcdep.2013.05.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Banks ML, Hutsell BA, Schwienteck KL, Negus SS. Use of preclinical drug vs. food choice procedures to evaluate candidate medications for cocaine addiciton. Curr Treat Options Psychiatry. 2015a;2:136–150. doi: 10.1007/s40501-015-0042-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Banks ML, Negus SS. Preclinical determinants of drug choice under concurrent schedules of drug self-administration. Adv Pharmacol Sci. 2012;2012 doi: 10.1155/2012/281768. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Banks ML, Hutsell BA, Blough BE, Poklis JL, Negus SS. Preclinical assessment of lisdexamfetamine as an agonist medication candidate for cocaine addiction: effects in rhesus monkeys trained to discriminate cocaine or to self-administer cocaine in a cocaine versus food choice procedure. Int J Neuropsychopharmacol. 2015b;18:1–10. doi: 10.1093/ijnp/pyv009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bedford JA, Bailey LP, Wilson MC. Cocaine reinforced progressive ratio performance in the rhesus monkey. Pharmacol Biochem Behav. 1978;9:631–638. doi: 10.1016/0091-3057(78)90214-9. [DOI] [PubMed] [Google Scholar]
  6. Blick SKA, Keating GM. Lisdexamfetamine. Paediatr Drugs. 2007;9:129–135. doi: 10.2165/00148581-200709020-00007. [DOI] [PubMed] [Google Scholar]
  7. Bradberry CW. Comparison of acute and chronic neurochemical effects of cocaine and cocaine cues in rhesus monkeys and rodents: focus on striatal and cortical dopamine systems. Rev Neurosci. 2008;19:113–128. doi: 10.1515/revneuro.2008.19.2-3.113. [DOI] [PubMed] [Google Scholar]
  8. Chiodo KA, Läck CM, Roberts DCS. Cocaine self-administration reinforced on a progressive ratio schedule decreases with continuous D-amphetamine treatment in rats. Psychopharmacology (Berl ) 2008;200:465–473. doi: 10.1007/s00213-008-1222-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Comer SD, Ashworth JB, Foltin RW, Johanson CE, Zacny JP, Walsh SL. The role of human drug self-administration procedures in the development of medications. Drug Alcohol Depend. 2008;96:1–15. doi: 10.1016/j.drugalcdep.2008.03.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Czoty PW, Blough BE, Fennell TR, Snyder RW, Nader MA. Attenuation of cocaine self-administration by chronic oral phendimetrazine in rhesus monkeys. Neuroscience. 2016a;34:367–376. doi: 10.1016/j.neuroscience.2016.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Czoty PW, Martelle JL, Nader MA. Effects of chronic d-amphetamine administration on the reinforcing strength of cocaine in rhesus monkeys. Psychopharmacology (Berl) 2010;209:375–382. doi: 10.1007/s00213-010-1807-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Czoty PW, Stoops WW, Rush CR. Evaluation of the “pipeline” for development of medications for cocaine use disorder: a review of translational laboratory, human laboratory, and clinical trial research. Pharm Rev. 2016b doi: 10.1124/pr.115.011668. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Foltin RW, Haney M, Rubin E, Reed SC, Vadhan N, Balter R, Evans SM. Development of translational preclinical models in substance abuse: effects of cocaine administration on cocaine choice in humans and non-human primates. Pharmacol Biochem Behav. 2015;134:12–21. doi: 10.1016/j.pbb.2015.03.023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Grabowski J, Rhoades H, Schmitz J, Stotts a, Daruzska La, Creson D, Moeller FG. Dextroamphetamine for cocaine-dependence treatment: a double-blind randomized clinical trial. J Clin Psychopharmacol. 2001;21:522–6. doi: 10.1097/00004714-200110000-00010. [DOI] [PubMed] [Google Scholar]
  15. Haney M, Spealman R. Controversies in translational research: drug self-administration. Psychopharmacology (Berl) 2008;199:403–419. doi: 10.1007/s00213-008-1079-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Herin DV, Rush CR, Grabowski J. Agonist-like pharmacotherapy for stimulant dependence: preclinical, human laboratory, and clinical studies. Ann N Y Acad Sci. 2010;1187:76–100. doi: 10.1111/j.1749-6632.2009.05145.x. [DOI] [PubMed] [Google Scholar]
  17. Hutson PH, Pennick M, Secker R. Preclinical pharmacokinetics, pharmacology and toxicology of lisdexamfetamine: a novel d-amphetamine pro-drug. Neuropharmacology. 2014;87:41–50. doi: 10.1016/j.neuropharm.2014.02.014. [DOI] [PubMed] [Google Scholar]
  18. Insel TR, Voon V, Nye JS, Brown VJ, Altevogt BM, Bullmore ET, Goodwin GM, Howard RJ, Kupfer DJ, Malloch G, Marston HM, Nutt DJ, Robbins TW, Stahl SM, Tricklebank MD, Williams JH, Sahakian BJ. Innovative solutions to novel drug development in mental health. Neurosci Biobehav Rev. 2013;37:2438–2444. doi: 10.1016/j.neubiorev.2013.03.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Jones JD, Comer SD. A review of human drug self-administration procedures. Behav Pharmacol. 2013;24:384–95. doi: 10.1097/FBP.0b013e3283641c3d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Keeler JF, Robbins TW. Translating cognition from animals to humans. Biochem Pharmacol. 2011;81:1356–1366. doi: 10.1016/j.bcp.2010.12.028. [DOI] [PubMed] [Google Scholar]
  21. Levin FR, Mariani JJ, Specker S, Mooney M, Mahony A, Brooks DJ, Babb D, Bai Y, Eberly LE, Nunes EV, Grabowski J. Extended-release mixed amphetamine salts vs placebo for comorbid adult attention-deficit/hyperactivity disorder and cocaine use disorder: a randomized clinical trial. JAMA Psychiatry. 2015;72:593–602. doi: 10.1001/jamapsychiatry.2015.41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Lile JA, Stoops WW, Rush CR, Negus SS, Glaser PEA, Hatton KW, Hays LR. Development of a translational model to screen medications for cocaine use disorder II: choice between intravenous cocaine and money in humans. Drug Alcohol Depend. doi: 10.1016/j.drugalcdep.2016.05.022. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Mello NK, Negus SS. Preclinical evaluation of pharmacotherapies for treatment of cocaine and opioid abuse using drug self-administration procedures. Neuropsychopharmacology. 1996;6:375–424. doi: 10.1016/0893-133X(95)00274-H. [DOI] [PubMed] [Google Scholar]
  24. Moeller SJ, Stoops WW. Cocaine choice procedures in animals, humans, and treatment-seekers: can we bridge the divide? Pharmacol. Biochem Behav. 2015;138:133–141. doi: 10.1016/j.pbb.2015.09.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Mooney ME, Herin DV, Specker S, Babb D, Levin FR, Grabowski J. Pilot study of the effects of lisdexamfetamine on cocaine use: a randomized, double-blind, placebo-controlled trial. Drug Alcohol Depend. 2015;153:94–103. doi: 10.1016/j.drugalcdep.2015.05.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Nader MA, Woolverton WL. Effects of increasing the magnitude of an alternative reinforcer on drug choice in a discrete-trials choice procedure. Psychopharmacology (Berl) 1991;105:169–171. doi: 10.1007/BF02244304. [DOI] [PubMed] [Google Scholar]
  27. Negus SS. Rapid assessment of choice between cocaine and food in rhesus monkeys: effects of environmental manipulations and treatment with d-amphetamine and flupenthixol. Neuropsychopharmacology. 2003;28:919–931. doi: 10.1038/sj.npp.1300096. [DOI] [PubMed] [Google Scholar]
  28. Negus SS, Henningfield J. Agonist medications for the treatment of cocaine use disorder. Neuropsychopharmacology. 2015;40:1815–1825. doi: 10.1038/npp.2014.322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Negus SS, Mello NK. Effects of chronic d-amphetamine treatment on cocaine- and food-maintained responding under a progressive-ratio schedule in rhesus monkeys. Psychopharmacology (Berl) 2003a;167:324–332. doi: 10.1016/S0376-8716(02)00339-3. [DOI] [PubMed] [Google Scholar]
  30. Negus SS, Mello NK. Effects of chronic d-amphetamine treatment on cocaine- and food-maintained responding under a second-order schedule in rhesus monkeys. Drug Alcohol Depend. 2003b;70:39–52. doi: 10.1016/S0376-8716(02)00339-3. [DOI] [PubMed] [Google Scholar]
  31. Nuijten M, Blanken P, vad de Wetering B, Nuijen B, van den Brink W, Hendriks VM. Sustained-release desamfetamine in the treatment of chronic cocaine-dependent patients on heroin-assisted treatment: a randomised, double-blind, placebo-controlled trial. Lancet. 2016 doi: 10.1016/S0140-6736(16)00205-1. in press. [DOI] [PubMed] [Google Scholar]
  32. Paronis CA, Gasior M, Bergman J. Effects of cocaine under concurrent fixed ratio schedules of food and IV drug availability: a novel choice procedure in monkeys. Psychopharmacology. 2002;163:283–291. doi: 10.1007/s00213-002-1180-5. [DOI] [PubMed] [Google Scholar]
  33. Rowlett JK, Massey BW, Kleven MS, Woolverton WL. Parametric analysis of cocaine self-administration under a progressive- ratio schedule in rhesus monkeys. Psychopharmacology (Berl) 1996;125:361–370. doi: 10.1007/BF02246019. [DOI] [PubMed] [Google Scholar]
  34. Rush CR, Stoops WW. Agonist replacement therapy for cocaine dependence: a translational review. Future Med Chem. 2012;4:245–265. doi: 10.4155/fmc.11.184. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Rush CR, Stoops WW, Sevak RJ, Hays LR. Cocaine choice in humans during D-amphetamine maintenance. J Clin Psychopharmacol. 2010;30:152–159. doi: 10.1097/JCP.0b013e3181d21967. [DOI] [PubMed] [Google Scholar]
  36. Stafford D, LeSage M, Glowa J. Effects of phentermine on responding maintained by progressive-ratio schedules of cocaine and food delivery in rhesus monkeys. Behav Pharmacol. 1999;10:775–784. doi: 10.1097/00008877-199912000-00009. [DOI] [PubMed] [Google Scholar]
  37. Stoops WW, Lile JA, Glaser PEA, Hays LR, Rush CR. Influence of acute bupropion pre-treatment on the effects of intranasal cocaine. Addiction. 2012;107:1140–1147. doi: 10.1111/j.1360-0443.2011.03766.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Stoops WW, Rush CR. Agonist replacement for stimulant dependence: a review of clinical research. Curr Pharm Des. 2013;19:7026–7035. doi: 10.2174/138161281940131209142843. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sullivan MA, Vosburg SK, Comer SD. Depot naltrexone: antagonism of the reinforcing, subjective, and physiological effects of heroin. Psychopharmacology (Berl) 2006;189:37–46. doi: 10.1007/s00213-006-0509-x. [DOI] [PubMed] [Google Scholar]
  40. Thomsen M, Barrett AC, Negus SS, Caine SB. Cocaine versus food choice procedure in rats: environmental manipulations and effects of amphetamine. J Exp Anal Behav. 2013;99:211–233. doi: 10.1002/jeab.15. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Vandaele Y, Cantin L, Serre F, Vouillac-Mendoza C, Ahmed SH. Choosing under the influence: a drug-specific mechanism by which the setting controls drug choices in rats. Neuropsychopharmacology. 2015;41:646–657. doi: 10.1038/npp.2015.195. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Weerts EM, Fantegrossi WE, Goodwin AK. The value of nonhuman primates in drug abuse research. Exp Clin Psychopharmacol. 2007;15:309–327. doi: 10.1037/1064-1297.15.4.309. [DOI] [PubMed] [Google Scholar]
  43. Woolverton WL, Balster RL. Effects of antipsychotic compounds in rhesus monkeys given a choice between cocaine and food. Drug Alcohol Depend. 1981;8:69–78. doi: 10.1017/CBO9781107415324.004. [DOI] [PubMed] [Google Scholar]
  44. Yu D. Translational research: Current status, challenges and future strategies. Am J Transl Res. 2011;3:422–433. [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

supplement
NIHMS792280-supplement.pdf (330.8KB, pdf)

RESOURCES