Cocaine-associated decision-making: toward isolating preference

Abstract

Ever-increasing evidence suggests that substance use disorder is mediated by decision-making processes, and as such, providing nondrug alternatives can shift maladaptive preferences away from drug reinforcers, such as cocaine. Of note, a recent hypothesis suggests that preference for cocaine is simply a byproduct of cocaine intake, such that the ‘direct’ effects of cocaine weaken the impact of non-drug alternatives while measuring choice. Conversely, existing quantitative theories of decision-making suggest preference is determined by various dimensions of concurrent reinforcers that in turn determine the relative value of available alternatives. Toward teasing apart the conflicting theories above, we developed a novel drug-choice procedure to control for reinforcer frequency and magnitude (two reinforcer dimensions well known to influence preference) that consequently controls for overall cocaine intake. As predicted by quantitative choice theory, results suggest that cocaine intake and preference are dissociable while measuring choice, with reinforcer frequency and magnitude having independent influence on the relative value of choice alternatives. Furthermore, we demonstrate that the choice procedure is sensitive to various manipulations known to alter cocaine reinforcement, all while keeping cocaine intake constant. Finally, the results point to the process of economic substitution as an important avenue of future neurobehavioral investigation toward the improvement of behavioral and pharmacological therapies for substance use disorders. Overall, the proposed choice procedure will allow for improved isolation of the neurobehavioral processes that mediate drug-associated decision-making in future studies.

1. Introduction

Understanding the decision processes underlying drug preference is important to improved substance-use disorder treatments (Kalivas & Volkow 2005; Heyman 2013). Choice-based models in human laboratories have demonstrated many conditions under which non-drug reinforcers reduce drug preference (Hart et al. 2000; Foltin et al. 2015; Lile et al. 2016). Hence, preclinical drug vs. food choice procedures are increasingly common, and an emerging literature has examined putative neurobehavioral mechanisms mediating drug preference (Ahmed et al. 2013; Banks & Negus 2012). Importantly, preclinical models have established concordance with human laboratory findings (Thomsen et al. 2013; Nader & Banks 2014) and exhibited predictive validity regarding candidate medication effectiveness (Banks et al. 2015).

According to quantitative choice theory, cocaine preference depends on the relative position of concurrent reinforcers on each reinforcer dimension (e.g., magnitude, frequency, etc.; Anderson et al. 2002; Iglauer & Woods 1974) in the same way as nondrug preference (e.g., food and water). For example, generalized concatenated matching (Hutsell et al. 2015) predicts the choice proportion allocated to cocaine vs. nondrug reinforcers (e.g., food) varying in both relative reinforcer frequency and magnitude as:

$$\frac{B_C}{B_C+B_F}=\frac{1}{1+{\left(\frac{R_F}{R_C}\right)}^{s_r}\ast {\left(\frac{M_F}{M_C}\right)}^{s_m}}$$ (1)

where B_i, R_i, and M_i denote choices, reinforcer frequency, and reinforcer magnitude for cocaine (C) and food (F), respectively. The free parameters S_r and S_m indicate sensitivity to relative frequency and magnitude of cocaine vs. food reinforcement, respectively. Thus, matching equation predicts that cocaine preference is determined by the relative values of cocaine and food in a context defined by the combination of relative reinforcer frequency and magnitude.

Under one drug-choice procedure, rats can display a preference for palatable, non-caloric reinforcers (i.e., saccharin) over cocaine, independent of dose (Lenoir et al. 2007; Cantin et al. 2010). Ahmed and colleagues proposed that “direct” anorectic effects of cocaine on saccharin promote cocaine choice, implying that cocaine choices are determined by cocaine plasma concentrations (Vandaele et al. 2016; Freese et al. 2018) independent of the cocaine reinforcement dimensions (e.g., infusion frequency and magnitude) that determine a given concentration. Moreover, according to this hypothesis, dose-dependent changes in cocaine preference previously reported in human and preclinical studies (Nader & Woolverton 1990) are a byproduct of cocaine intake. Conversely, the generalized matching framework predicts distinct and dissociable effects of cocaine infusion frequency and magnitude on cocaine relative value and consequent preference.

Importantly, distinguishing among predictions derived from these two hypotheses using existent cocaine choice models is not possible, as current choice metrics confound preference with intake and resulting cocaine plasma concentrations. That is, cocaine preference is correlated with obtained relative reinforcer frequency and/or magnitude. Consequently, the independent variables specified by choice theory are dependent on (directly co-vary with) preference in existing cocaine choice procedures. A large literature body has demonstrated, as predicted by the matching framework, that relative reinforcer frequency is a chief determinant of preference in a variety of human and non-human choice procedures (McCarthy & Davison 1984; Johnson & Alsop 2000). Additionally, confounding cocaine preference with reinforcer frequency and consequent intake in preclinical models precludes assessment of competing predictions associated with hypothesized neurobehavioral mechanisms. To address the confound noted above, controlled reinforcer frequency procedures have been used in several previous clinical and preclinical studies to guarantee equality between experimenter-scheduled and subject-obtained reinforcer frequency (Stubbs & Pliskoff 1969; Llewellyn et al. 1976; Pizzagalli et al. 2005; Pope et al. 2015). The present study aim was to demonstrate independence between cocaine preference and intake by using a controlled reinforcer frequency procedure to control the obtained frequency of reinforcement. We demonstrate that cocaine preference is independent from intake, while measuring choice over a wide range of cocaine doses (0.032-1.0 mg/kg/infusion). Unlike the “direct” anorectic effects hypothesis, generalized matching predicts dissociable effects of relative cocaine reinforcer frequency and magnitude on cocaine preference, providing a unifying framework to evaluate predictions regarding relative cocaine value. Thus, the controlled reinforcer frequency procedure offers an empirical and theoretical approach to isolate specific neurobehavioral mechanisms of drug preference.

2. Method

2.1. General Choice Procedure

All procedures were approved by the Institutional for Animal Care and Use Committee at the University of Kentucky. Following a series of initial training procedures (see Table 1), Sprague Dawley rats were assigned (counterbalanced) to either the controlled or uncontrolled reinforcer frequency schedule for cocaine vs. food choice. Within both procedures, each session was divided into 5 choice blocks, separated by a 2-min blackout inter-block-interval. Blocks were distinctly signaled by an accompanying tone pattern (see supplemental Table S1; Pope et al. 2015). Within each block, responses on the food lever (counterbalanced for side across rats) resulted in the delivery of a single 45-mg food pellet (Bio-Serv Precision Pellets, product #F0021, Flemington, NJ), while responses on the cocaine lever (counterbalanced for side across rats) resulted in an infusion of cocaine that varied in working dose across blocks (0, 0.032, 0.10, 0.32, and 1.0 mg/kg/infusion, determined by infusion time; Yates et al. 2017). In each block, upon food-pellet delivery, the lever retracted along with a 5.9-s cue-light onset above the corresponding food lever. Upon cocaine infusion, levers were retracted and the cue-light above the corresponding cocaine lever turned on for a duration that matched the infusion length (0, 0.189, 0.59, 1.89, or 5.9 s) that achieved the working dose for a given block. Each trial began with the illumination of the house-light, and an orienting response into the magazine turned off the house-light and extension of the response lever(s). All programmed reinforcers (i.e., reinforcers set up by the computer to be delivered upon completion of the associated schedule) were scheduled according to a fixed ratio (FR) and required consecutive responses on the same lever to complete the ratio; a changeover response would reset the response counter. Upon completion of the FR, levers retracted and reward delivery with accompanying signals occurred. Rats were initially trained on a FR1 and systematically incremented to reach a terminal FR5. All trials were separated by a 10-s blackout inter-trial-interval prior to house-light onset. Sessions ended following completion of all 5 blocks.

Table 1.

Example training timetable for a given animal during cocaine-food choice. Note: Order of conditions (food restriction, removal of cocaine cues, and removal of head-entry orienting responses) within the experiment proper was scheduled according to a partial Latin-square design (baseline first), and some animals had up to 5 additional days within a given condition to establish stability. During controlled reinforcer frequency sessions the cocaine:food reinforcer frequency ratio used was constant at 3:3. See the supplemental methods for details regarding the initial training phases.

Initial Training	Days
Magazine shaping	2
Lever training	8
Orienting response	3
Catheter surgery and recovery	8
Drug self-administration training	8
Drug-food lever training	4
Experiment Proper	Days
Baseline Controlled Reinforcer Frequency (3:3)	14
Food restriction - controlled reinforcer frequency (3:3)	10
Baseline - controlled reinforcer frequency (3:3)	7
No cocaine cues - controlled reinforcer frequency ratio (3:3)	10
Baseline - controlled reinforcer frequency ratio (3:3)	7
No head-entry orienting response - controlled reinforcer frequency ratio (3:3)	10
Baseline Uncontrolled Reinforcer Frequency	14
Food restriction - uncontrolled reinforcer frequency	10
Baseline - uncontrolled reinforcer frequency	7
No cocaine cues - uncontrolled reinforcer frequency	10
Baseline - uncontrolled reinforcer frequency	7
No head-entry orienting response - uncontrolled reinforcer frequency	10

2.2. Uncontrolled Reinforcer Frequency

The uncontrolled reinforcer frequency choice procedure (see Figure 1), based on Thomsen et al. (2013), consisted of blocks composed of sample and choice trials. Sample trials (2) consisted of a single lever (1 food and 1 cocaine) extended in a random sequence. Completion of all sample trials was required to advance to choice trials. After completion of the sample trials, choice trials (6) with both levers extended began. With both levers extended, rats were provided the opportunity to distribute 6 total choices across the two options within 30 minutes. After 6 total reinforcers within a block were earned or 30 minutes had elapsed, the block would end and enter into the inter-block interval. Cocaine preference was calculated as the total number of cocaine choice responses divided by the total number of cocaine and food choice responses (see Figure 1).

Figure 1.

Example session from a single subject under the uncontrolled reinforcer frequency schedule. (A) A trial-by-trial (rows) and block-by-block (columns) breakdown during an uncontrolled reinforcer frequency session, where the left lever is associated with food and the right lever is associated with drug. Within each choice trial, both reinforcers are available and a check mark over the food/drug label represents choice made (i.e., FR5 completion) by the animal. (B) Graphical representation of the number of reinforcers earned across blocks as a function of dose. (C) Graphical representation of the percent choice for cocaine as a function of dose.

2.3. Controlled Reinforcer Frequency

The controlled reinforcer frequency choice procedure (see Figure 2) consisted of a total of six trials per block: 3 drug and 3 food. Each trial always had both levers (cocaine and food) extended, yet only one of the two reinforcers (cocaine or food) was made available randomly in an unpredictable manner on each trial. Because reinforcement was scheduled randomly and unpredictably for the two choice alternatives, the animal had no way of determining which alternative (cocaine or food) would be reinforced on each trial. However, regardless of which lever the rat responded on, the reinforcer scheduled on that trial had to be collected to advance to the next trial. Thus, all responses for reinforcers scheduled on a particular trial were defined as forced responses for that reinforcer type (3 cocaine and 3 food); these forced responses were then removed from the total number to determine preference (total responses - forced responses = choice responses). Cocaine preference was calculated as the total number of cocaine choice responses divided by the number of cocaine choice plus food choice responses (see Figure 2). Overall, by using the controlled reinforcer frequency method, the relative number of cocaine to food reinforcers earned (3 each) is kept constant across all sessions between all animals (i.e., the cocaine:food reinforcer frequency ratio was held constant at 3:3)¹. After completion of all 6 trials, the block would end and enter into the inter-block interval.

Figure 2.

Example session from a single subject under the controlled reinforcer frequency schedule. (A) Trial-by-trial (rows) and block-by-block (columns) breakdown during a controlled reinforcer frequency session, where the left lever is associated with food and the right lever is associated with drug. Within each trial, only one reinforcer is randomly scheduled in an unpredictable manner, represented by bolded text with (+) sign. The number above each illustrated lever (below food/drug labels) represents the number of responses made on that lever. Numbers that are under bolded labels with (+) signs represent forced responses; numbers that are under un-bolded labels with (−) signs represent choice responses. (B) Graphical representation of the number of choice responses for food and cocaine across blocks as a function of dose. (C) Graphical representation of the percent cocaine choice as a function of dose.

2.4. Manipulations

Following stability of preference, defined as no linear trends in choice performance parameters for four consecutive days under baseline conditions within either choice procedure (controlled or uncontrolled reinforcer frequency), all rats were then assigned, via a partial Latin square design (baseline first), to the environmental manipulations. Each environmental manipulation was in effect for a minimum of ten days and until stable. Following stability, rats were returned to baseline conditions for seven days before being assigned to the next condition. After completing each condition under the initially assigned baseline choice procedure (counterbalanced across animals), rats were switched to the other choice procedure (controlled or uncontrolled reinforcer frequency) and trained to stability under baseline conditions according to the same stability criteria above; rats then underwent the same series of environmental manipulations according to a partial Latin square (baseline first) design different from that used for the first procedure. Therefore, each rat was exposed to each environmental manipulation under each choice procedure (controlled and uncontrolled reinforcer frequency) in a counterbalanced order.

Food Restriction

To determine the effects of food motivation on cocaine choice, rats were food restricted and maintained at approximately 85% of their free-feeding body weights during the testing period.

Infusion Cue Removal

To determine the effects of cocaine-associated conditioned reinforcement on choice, the cue-light associated with cocaine infusion was removed; thus, only cocaine infusion deliveries went unsignaled across all blocks.

Orienting-response Removal

To determine the effects of subject-determined trial initiation on choice (inter-trial interval effects), the orienting response was removed. All trials were no longer initiated by a head-entry into magazine; thus, the house-light (discriminative stimulus for orienting response) was not used, and all trials began immediately with the extension of the response lever(s).

Reinforcer Frequency Ratio

To determine the effect of systematic reinforcer frequency ratio manipulation on cocaine preference, 6 animals were trained under the controlled reinforcer frequency baseline conditions as described above, with a cocaine:food frequency ratio of 3:3 (see Table 2). Half were then trained on a controlled reinforcer frequency schedule identical to that under baseline conditions, except each block included 5 drug trials and 1 food trial (randomly determined) for a cocaine:food frequency ratio of 5:1; the other half were trained on an identical controlled reinforcer frequency schedule, except each block included 1 drug trial and 5 food trials (randomly determined) for a cocaine:food frequency ratio of 1:5. Following stability, the reinforcer frequency ratios were returned to baseline conditions for a minimum of seven days. Finally, rats were trained on the reinforcer frequency ratio opposite to the initial condition.

Table 2.

Example training timetable for a given animal during cocaine-food choice under the controlled reinforcer frequency procedure with cocaine:food reinforcer frequency ratio manipulations (3:3, 5:1, and 1:5). Note: During the experiment proper, order of cocaine:food reinforcer frequency ratio condition was conducted using a partial Latin-square design (baseline first, 3:3), and some animals had up to 5 additional days within a given condition to establish stability. See the supplemental methods for details regarding the initial training phases.

Initial Training	Days
Magazine shaping	2
Lever training	8
Orienting response	3
Catheter surgery and recovery	8
Drug self-administration training	8
Drug-food lever training	4
Experiment Proper	Days
Baseline controlled reinforcer frequency (cocaine:food = 3:3)	14
5:1 cocaine:food reinforcer frequency	10
Baseline controlled reinforcer frequency (cocaine:food = 3:3)	7
1:5 cocaine:food reinforcer frequency	10

Upon completion of all experimental testing, the resulting n-sizes were: n=20 for controlled and uncontrolled reinforcer frequency under baseline conditions; n=14 for controlled and n=11 for uncontrolled reinforcer frequency under food restriction; n=15 for controlled and n=10 for uncontrolled reinforcer frequency under no cocaine infusion cue conditions; n=12 for controlled and n=9 for uncontrolled reinforcer frequency under no head-entry orienting response conditions; n=5 for cocaine:food reinforcer frequency ratio manipulations (see Table 1 and Table 2 for training timeline details).

2.5. Analysis

When the relative reinforcer frequency ratio was held constant (3:3), the generalized matching model applied was of the form:

$$\frac{B_C}{B_C+B_F}=\frac{1}{1+{\left(\frac{a{M}_F}{M_C}\right)}^{s_m}}$$ (2)

where B represents responses allocated to cocaine (C) or food (F) alternatives, and M_C represents cocaine dose while M_F represents food magnitude. The free parameter a in place of M_F is a scaling constant, acting as an exchange rate for food and cocaine that scales food pellets in units of cocaine. For example, a cocaine-food exchange value of 0.3 defines a single food pellet is equivalent in value as a 0.3 mg/kg infusion of cocaine. Thus, like monetary exchange rates, larger cocaine-food exchange values indicate greater relative magnitude of the food pellet in cocaine units, effectively serving as an index of reinforcer substitutability. The free parameter S represents the sensitivity to changes in the relative magnitude between drug and food reinforcers. Best-fit model parameters (a and S) were determined via nonlinear-mixed-effects model (Pinheiro et al. 2007) fits of Equations 2 to cocaine choice data, with schedule (nominal), condition (nominal), and dose (continuous) as within-subject factors, and subject as a random factor.

When the reinforcer frequency ratio under the reinforcer frequency ratio was systematically manipulated, Equation 1 was fit with a set as a constant of 0.2, derived from baseline fits where relative reinforcer frequency was equal (cocaine:food = 3:3). Best-fit model parameters (S_r and S_m) were determined via nonlinear-mixed-effects model fits of Equations 1 to cocaine choice data, with both reinforcer frequency ratio and dose as continuous within-subject factors and subject as a random factor.

The use of intravenous administration of cocaine allows for accurate estimates of whole-body cocaine concentrations based upon recorded infusion times (Tsibulsky & Norman 1999; Zimmer et al. 2011). Thus, estimated whole-body cocaine concentrations at the time of reinforcer delivery were determined according to the following equation (Weiss et al. 2003):

$$(B_{n-1}+D)e^{-kt}$$ (3)

where B_n represents current cocaine concentrations (mg/kg), B_n−1 represents cocaine concentrations from the previous infusion, D represents the dose of cocaine for the given block, k represents the decay constant (0.0383), and t represents minutes since last infusion. Average whole-body cocaine concentration at reinforcer delivery was analyzed via linear mixed-effects modeling (Gelman & Hill 2006) with schedule (nominal), condition (nominal) and dose (continuous) as within-subject factors, and subject as a random factor. Correlations between cocaine-food exchange (a) and average estimated whole-body cocaine concentration prior to reinforcer delivery during the last block (i.e., 1.0 mg/kg/infusion cocaine) were calculated using Pearson’s r (α = 0.05). All correlations were calculated using the Pearson correlation coefficient. For all tests, α was set to 0.05.

3. Results

3.1. Cocaine versus Food Choice

Figure 3A illustrates percent cocaine choice under controlled and uncontrolled reinforcer frequency procedures at baseline (individual data in Figure S1). Nonlinear-mixed-effects analysis of baseline preference revealed that the controlled reinforcer frequency produced greater sensitivity to relative reinforcer magnitude (S) [F(1,172)=10.47, p<0.05], while there were no significant differences in cocaine-food exchange (a). Thus, while both procedures produced similar dose-dependent increases in cocaine preference, sensitivity to the relative reinforcer magnitude ratio was greater under the controlled reinforcer frequency schedule as indicated by a steeper cocaine choice dose-effect curve.

Figure 3.

Effects of environmental manipulations on cocaine preference. (A) Mean (±SEM) percent choice for cocaine under the controlled and uncontrolled reinforcer frequency choice procedures at baseline; n = 20/procedure. (B) Mean (±SEM) percent choice for cocaine under controlled reinforcer frequency at baseline (BL), food restricted (n = 14), no cocaine cues (NCC; n = 15), and no head-entry orienting response conditions (NHE; n = 12). (C) Mean (±SEM) percent choice for cocaine under uncontrolled reinforcer frequency for baseline (BL), food restricted (FR; n = 11), no cocaine cues (NCC; n = 10), and no head entry conditions (NHE; n = 9). Best-fit parameter estimates from nonlinear-mixed-effects generalized matching fits (Equation 2), namely (D) cocaine-food exchange and (E) sensitivity to relative cocaine-food magnitude under the different schedules and conditions. * indicates p <0.05 between schedule and ^# indicates p <0.05 versus baseline. Lines are best fits of Equation 2 determined by nonlinear-mixed-effects analysis.

Figure 3B and 3C illustrates percent cocaine choice across the different environmental manipulations under the controlled and uncontrolled reinforcer frequency procedures, along with cocaine-food exchange estimates (3D) and magnitude sensitivity estimates (3E) from generalized matching fits. Nonlinear-mixed-effects analysis revealed a significant main effect of environmental manipulation [F(3,515)=57.13, p<0.05] and a significant interaction with reinforcement schedule [F(3,515)=6.63, p<0.05] on the cocaine-food exchange rate parameter (a), indicating that cocaine-food substitution was affected by the different environmental manipulations, and these differences were procedure-dependent. Post hoc pairwise comparisons (Bonferroni corrected) indicated that cocaine-food exchange was increased under the controlled reinforcer frequency during food restriction, while there was no effect of food restriction when reinforcer frequency was uncontrolled. Removal of the cocaine cue increased cocaine-food exchange under both the controlled and uncontrolled reinforcer frequency procedure, and removal of the orienting response decreased cocaine-food exchange under both procedures. Finally, non-linear-mixed effects analysis revealed a significant main effect of schedule [F(1,515)=3.35, p<0.05] on sensitivity to relative reinforcer magnitude (S), indicating that sensitivity to relative magnitude was greater overall under the controlled reinforcer frequency ratio. Altogether, the results demonstrate that the environmental manipulation effects on cocaine-food relative value were differentially affected under food restriction when the reinforcer frequency ratio was controlled vs. uncontrolled, and overall sensitivity to changes in relative reinforcer magnitude was increased under the controlled reinforcer frequency procedure.

See the supplemental file for details regarding response latency and rate data (Figures S2 and S3).

3.2. Whole-body Cocaine Concentrations

Figure 4A and 4C illustrate cumulative average calculated whole-body cocaine concentrations at reinforcer delivery under the controlled and uncontrolled reinforcer frequency procedures, respectively (individual data in Figure S4). Linear-mixed-effects analysis revealed a main effect of dose [F(1,24.93)=533.32, p<0.05], schedule [F(1,22.98)=22.38, p<0.05], and condition [F(3,45.45)=4.96, p<0.05] on estimated cocaine concentrations, suggesting that cocaine concentrations increased with dose, cocaine concentrations were generally higher when reinforcer frequency was uncontrolled, and the environmental conditions significantly affected estimated cocaine concentrations. Additionally, linear-mixed-effects analysis revealed a dose x schedule x condition interaction [F(3,42.03)=3.46, p<0.05], indicating that whole-body cocaine concentrations increased with dose, but the rate of increase was differentially affected by the environmental manipulations under the controlled vs. uncontrolled reinforcer frequency procedures.

Figure 4.

Effects of environmental manipulations on cocaine intake. Mean (±SEM) whole-body cocaine concentrations at reinforcer delivery, averaged during choice for each block, under the (A) controlled and (C) uncontrolled reinforcer frequency procedures. Correlations between individual indifference points (constrained at 2) and whole-body cocaine concentrations reached during choice trials in the last block under the (B) controlled and (D) uncontrolled reinforcer frequency procedures for the different conditions. * indicates p <0.05.

Figure 4B and 4D illustrate the correlations between cocaine-food exchange rate and cumulative whole-body cocaine concentrations under the controlled and uncontrolled reinforcer frequency procedures, respectively. As expected, because of the preference-intake confound allowed by the uncontrolled reinforcer frequency procedures, there was a strong negative correlation between these measures (r = −0.78, p<0.05). Importantly, controlling reinforcer frequency prevented the preference-reinforcer frequency covariation, eliminating any correlation between preference and intake (r = 0.08, NS). Collectively, these correlation analyses confirm that cocaine preference can be dissociated from intake under specific experimental procedures designed to isolate these variables while measuring choice.

3.3. Reinforcer Frequency Effects

Manipulating the cocaine:food reinforcer frequency ratio in favor of cocaine (5:1) or food (1:5) reinforcement significantly shifted preference toward cocaine or food (Figure 5A). Fits of Equation 1 to the data via nonlinear mixed effects modeling demonstrated that the independent effects of relative reinforcer frequency (S_r = 1.14) and magnitude (S_m = 1.82) collectively accounted for 90% of the variance in cocaine choices. Thus, as predicted by generalized matching, relative reinforcer frequency and magnitude primarily determined cocaine choice, illustrating that relative reinforcer frequency is itself a determinant of cocaine choice. This result further highlights that the confounding of preference and reinforcer frequency when using uncontrolled reinforcer frequency procedures makes it impossible to determine the influence of this choice determinant on preference.

Figure 5.

Relative reinforcer frequency effects on cocaine preference and intake. (A) Mean (±SEM) percent choice for cocaine under the controlled reinforcer frequency procedure for an equal cocaine:food reinforcer frequency ratio (3:3), a ratio in favor of cocaine (5:1), and a ratio in favor of food (1:5); n=5/reinforcer frequency ratio condition. (B) Mean (±SEM) whole-body cocaine concentrations at reinforcer delivery under the controlled reinforcer frequency procedure for each cocaine:food frequency ratio. (C) Cocaine choices at the minimum whole-body cocaine concentration that produced cocaine-food indifference (0.18 mg/kg) under the controlled reinforcer frequency procedure for each cocaine:food reinforcer ratio. Lines are best fits of matching (Equation 1) determined by nonlinear-mixed-effects analysis.

Additionally, estimated whole-body cocaine concentrations changed as a function of cocaine dose across the session (Figure 5B), as indicated by a significant main effect of dose [F(1,5.29)=3134.80, p<0.05]; there was a significant main effect of reinforcer frequency ratio on whole-body cocaine concentrations across the session [F(2,10.02)=265.47, p<0.05]; and, the increase in whole-body cocaine concentration was dependent upon reinforcer ratio [F(2,10.05)=164.70, p<0.05], with the 5:1 cocaine:food reinforcer frequency ratio producing the highest cocaine concentrations. Finally, to determine the role of whole-body cocaine concentration in determining cocaine choice, we analyzed the relationship between cocaine-food exchange rate (a) and estimated whole-body cocaine concentration at each reinforcer frequency ratio. The lowest estimated whole-body concentration associated with equal cocaine-food preference (50% cocaine-food preference) was 0.18 mg/kg under the 5:1 cocaine:food reinforcer ratio. Thus, if cocaine concentration is itself the sole determinant of cocaine value, cocaine-food preference should be equivalent once the 0.18 mg/kg whole-body concentration is reached under all reinforcer ratio conditions; in other words, as soon as 0.18 mg/kg whole-body concentration is achieved, preference between food and cocaine should be equal, and any value above 0.18 mg/kg should produce cocaine preference. Accordingly, Figure 5C compares cocaine choices from each reinforcer ratio at the 0.18 mg/kg whole-body cocaine concentrations; as illustrated, the same whole-body concentration produced very different cocaine preferences under the different reinforcer frequency ratios, suggesting that the ‘direct’ anorectic effects of cocaine are not the exclusive determinant of cocaine-food relative value.

4. Discussion

This study revealed several important features of cocaine choice. First, as previously demonstrated (Thomsen et al. 2013; Iglauer & Woods 1974), relative reinforcer magnitude had a large effect on cocaine vs. food preference, modulating choice from exclusive food preference at low cocaine doses to exclusive cocaine preference at high cocaine doses. Second, relative reinforcer frequency had a prominent effect on cocaine-food preference (Figure 5), shifting the cocaine choice dose-effect curve left or right on the dose axis. Third, relative reinforcer magnitude and frequency effects were independent, and their combined effects accounted for most of the variance in cocaine preference. Fourth, unlike traditional choice procedures that do not control relative reinforcer frequency, cocaine preference can be dissociated from cocaine intake when relative reinforcer frequency is held constant while measuring choice. Collectively, these findings support the generalized matching prediction that cocaine preference is determined by the value of cocaine relative to a concurrent nondrug alternative, and these values are determined by differences on orthogonal reinforcer dimensions.

4.1. The role of ‘direct’ anorectic effects in cocaine value

While the ‘direct effects’ hypothesis may help explain many of the behavioral effects of cocaine, it struggles in accounting for much of the cocaine preference data in the literature. For example, human studies have demonstrated dose-dependent increases in cocaine preference over concurrently available money (regardless of whether cocaine was onboard; Foltin et al. 2015), a reinforcer that is presumably not affected by the anorectic effects of cocaine (Lile et al. 2015). Relatedly, one of the most effective pharmacological treatments known to reduce cocaine preference over food (Banks et al 2013; Hutsell et al. 2015) and money (Grabowski et al. 2004) is chronic d-amphetamine treatment, a drug known to have long-lasting anorectic effects similar to cocaine (Negus & Mello 2003). Finally, the present controlled reinforcer frequency results illustrate manipulations that clearly decreased (food restriction, cocaine cue removal) or increased (removal of orienting response) cocaine relative value in the absence of changes in cocaine intake.

To provide a more direct comparison of the present results to the existing literature, as proof of principle we calculated whole-body cocaine concentrations using the parameters from the choice procedures described in Lenoir et al. (2007) and Kearns et al. (2017). Because the time of each choice was not reported within Lenoir et al. (2007) or Kearns et al. (2017), we relied on the sample phase (2 forced choice trials for cocaine and food) that took place prior to the first-choice opportunity to determine whole-body cocaine concentrations at time of first choice. Thus, the fixed, precise sample-trial timing used in each publication along with intravenous administration of cocaine allows for precise estimation of existing whole-body cocaine concentrations at the time of first choice. For Lenoir et al (2007), the sample phase (Figure 6A and 6B) consisted of a fixed 5-min limited hold, where if no response was made the trial was omitted, and the dose-dependent inter-trial interval began (10-min for 0.25 mg/kg, 16.4-min for 0.75 mg/kg, and 23.1 min for 1.5 mg/kg). For Kearns et al. (2017), a 10-min inter-trial interval was used, but the order of cocaine and food sample trials was randomized (Figure 6C and 6D); thus, the maximum (food-food-cocaine-cocaine sample order) and minimum (cocaine-cocaine-food-food sample order) whole-body cocaine concentrations via sample-trial order was calculated. Using the above parameters, we calculated whole-body cocaine concentrations present upon the first choice-trial, covering the entire possible range allowed by each procedure.

Figure 6.

Simulated whole-body cocaine concentrations during the sample-phase in Lenoir et al (2007) for (A) a perfect responder (i.e., immediate responding upon sample-trial start) and (B) a last-second responder (i.e., responding right before the end of 5-min limited-hold during forced-trials). Simulated whole-body cocaine concentrations during the sample-phase in Kearns et al (2017) for whole-body cocaine concentration determined via sample-trial order prior to choice with a (C) 10-min and (D) 60-min inter-trial interval. Xs represent the time of first choice. (E) Representative range (rectangle size) of whole-body cocaine concentrations from methods described in Lenoir et al. (2007; gray rectangles) and Kearns et al (2017; white rectangles) at time of first choice. Gray rectangles were calculated based on the given inter-trial interval and 5-min limited hold described in Lenoir et al. 2007; bottom of the gray rectangles represents whole-body cocaine concentrations for a perfect responder (immediate responding upon trial start) and the top of the gray rectangle represents a last-second responder (responding right before the end of the trial limited-hold). White rectangles were calculated based on the given inter-trial interval in Kearns et al. 2017; since sample-trials were randomized, the bottom of the white rectangles represent a sample trial order of cocaine-cocaine-food-food, whereas the top of the white triangles represent a sample trial order of food-food-cocaine-cocaine, capturing the entire whole-body cocaine concentration range possible at first choice. The hatched, horizontal lines represent estimated whole-body cocaine concentrations when preference switched from food to cocaine (> 50% preference) for each reinforcer ratio condition (5:1, 3:3, and 1:5).

Figure 6E summarizes the calculated whole-body cocaine concentrations from trials using Lenoir et al. (2007) and Kearns et al. (2017) methods, along with the present cocaine-food reinforcer frequency manipulation. There are three notable results. First, there is a large range of whole-body cocaine concentrations across the experimental manipulations. Second, the Lenoir et al. (2007) procedure produced a whole-body cocaine range of 0.09-0.27 (Figure 6E, gray squares), and 10-20% of individuals were reported as cocaine-preferring across all conditions. However, the Lenoir et al. (2007) procedure was also used by Kearns et al. (2017) and resulted in a much larger whole-body cocaine concentration range of 0.07-1.14 (Figure 6E, open squares). Importantly, Kearns et al. (2017) reported ~55% of rats as cocaine-preferring at both the low (Figure 6E, small open square) and high concentration ranges (Figure 6E, large open square), illustrating a dissociation between preference and intake. Third, the relative cocaine frequency manipulation in this study produced a whole-body cocaine range of 0.14-0.58 across all conditions (5:1, 3:3, 1:5). The hatched, horizontal lines in Figure 6E represent the whole-body cocaine concentration at the point where preferences switched from food to cocaine (i.e., > 50% preference) for each condition, illustrating another dissociation between preference and intake. Collectively, comparing calculations from prior studies and the present results provide further evidence against the hypothesis that the ‘direct’ anorectic effects of cocaine are the single determinant of preference in cocaine choice procedures.

4.2. Matching and the economics of substance-use disorders

Like matching, economic demand theory posits that the value of a reinforcer is not an inherent, absolute attribute of the reinforcer. Rather, value is defined by the decision-making context in which the reinforcer is made available (Rachlin et al. 1976; Hursh & Roma 2016). Economic approaches have identified two main determinants of drug consumption: cost-benefit factors and reinforcer type interactions (Hursh & Roma 2016). The cost-benefit ratio of a reinforcer consists of price (e.g., specified by an FR schedule) and magnitude factors (e.g., specified by the unit dose in mg/kg/infusion), and has usually been combined into a single variable called unit price (UP = price/magnitude). Reinforcer interaction is the degree of substitution, complementarity, or independence between different reinforcer types (Hursh & Roma 2016). Previous drug-choice studies have applied the unit-price approach by assuming preference is determined by relative unit price, and some support for unit-price predictions has been reported (Madden et al. 2000).

Contrary to the above, current unit price-based models (Hursh & Roma 2016) are dependent on consumption as the dependent variable and are not readily applicable to the present data where reinforcer consumption is held constant. Furthermore, while the unit price approach suggests that cost-benefit comparisons lie on a single reinforcer dimension, more recent evidence suggests that reinforcer cost-benefit dimensions have dissociable effects (Smith et al. 2016) and are mediated by different neurobehavioral mechanisms (Brenton et al. 2013). Consistent with cost-benefit independence, the generalized matching framework used here incorporates sensitivity parameters that independently scale the effectiveness of each cost-benefit dimension on cocaine preference. Importantly, the effects on sensitivity to reinforcement dimensions in this study are consistent with previous findings. First, sensitivity to reinforcer magnitude was not significantly affected by environmental manipulations in the present study, consistent with prior findings with a cocaine vs. food choice procedure in rhesus monkeys (Hutsell et al. 2014). Second, sensitivity to reinforcer magnitude and frequency were constant when relative reinforcer frequency was varied. This finding suggests that relative reinforcer frequency affects preference with a constant sensitivity and that the effect of reinforcer frequency is independent of cocaine dose (reinforcer magnitude), consistent with previous studies (Hollard & Davison 1971). Collectively, the existing unit-price approach appears limited in scope, and its success may be more exemplary of application to a specific decision-making context that is engendered by the majority of existing experimental designs, rather than a general rule.

A further difference between traditional unit-price and generalized matching approaches to drug choice is the addition of an exchange rate parameter that provides a common scale for reinforcer magnitude. The cocaine-food exchange rate parameter functions effectively as a substitutability coefficient in the present formulation (Hursh & Roma 2016), and this substitutability was not because the food pellet used herein was an inferior non-drug alternative (see Figure S5). Accordingly, environmental manipulations affected drug preference by altering cocaine vs. food substitution, rather than sensitivity to magnitude or price. For example, food restriction shifted the cocaine choice curve to the right, with relative reinforcer price and magnitude held constant, by increasing the exchange rate from 0.17 to 0.41 mg/kg/infusion of cocaine per food pellet; this effect is similar to changes in demand elasticity under open vs. closed economies (Hursh & Roma 2016). Likewise, removal of the cocaine infusion cue similarly increased the exchange rate from baseline to 0.43 mg/kg/infusion of cocaine per pellet, possibly by decreasing choice-cocaine associability (Killeen & Sitomer 2003). Conversely, decreasing the time between choice opportunities by removing the head-entry response decreased exchange rate (without affecting intake) from baseline to 0.11 mg/kg/infusion of cocaine per pellet, consistent with some observed effects of inter-trial interval on cocaine choice (Elsmore et al. 1980). Overall, the general pattern of environmental manipulation effects on choice were mostly comparable between the controlled and uncontrolled reinforcer frequency procedures; however, importantly, manipulations under the controlled reinforcer frequency procedure produced these effects while maintaining whole-body cocaine concentrations, relative reinforcer price, relative reinforcer frequency, and relative reinforcer magnitude equal and constant across individuals and conditions.

4.3. Translation, validity, and conclusions

While drug choice models have many methodological advantages over single-schedule measures (Banks et al. 2015), they also better represent behavior-environment relations outside the laboratory. While the controlled reinforcer frequency procedure has not yet been utilized in human clinical studies on substance-use disorder, it has been used successfully to isolate reinforcement processes in human clinical models of anhedonia (Pizzagalli et al. 2005) and attention deficit hyperactivity disorder (Alsop et al. 2016), illustrating its translational utility. Furthermore (in addition to drug reinforcer magnitude), utilizing the controlled reinforcer frequency procedures allows for the precise manipulation of relative reinforcer frequency, modeling environments that vary in alternative reinforcement, a factor known to modulate human drug use and often used successfully therapeutically (Brewer et al. 2017). Relatedly, studying individual differences in drug preference within environments devoid of or replete with alternative reinforcers via a controlled reinforcer frequency procedure may inform future research regarding the neurobehavioral mechanisms that underlie resilience to substance-use disorder and sensitivity to therapeutic alternatives, respectively, all while controlling for individual differences in intake. Additionally, because alternative reinforcers that decrease relative drug value function as economic substitutes, the present results highlight the importance of future research toward understanding the underlying neurobiological processes that govern substitution effects and provides a neurobehavioral target for future pharmacotherapies that function similarly to reduce drug preference while increasing preference for therapeutic alternatives (Hutsell et al. 2015). Finally, it is well known that differential drug intake itself can have considerable effects on neurobiological outcome measures (Hyman et al. 2006), leaving drug choice procedures that confound preference with drug intake susceptible to considerable interpretive issues regarding the underlying neurobiology. Thus, dissociating drug preference from intake with the controlled reinforcer frequency procedure while measuring drug choice offers a unique avenue to a better understanding of the neurobehavioral mechanisms that govern drug-associated decision-making.

Highlights.

• Relative reinforcer frequency and magnitude affected cocaine preference independently

• Cocaine preference is dissociable from cocaine intake

• Reinforcer substitutability is an important determinant of cocaine preference