Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Jan 1.
Published in final edited form as: Drug Alcohol Depend. 2020 Oct 18;218:108369. doi: 10.1016/j.drugalcdep.2020.108369

Remifentanil-food choice follows predictions of relative subjective value

Jonathan J Chow 1, Joshua S Beckmann 1
PMCID: PMC7750270  NIHMSID: NIHMS1638939  PMID: 33109461

Abstract

Background

Preclinical studies into drug vs. nondrug choice have emerged to better model and investigate the neurobehavioral mechanisms underlying drug preference. Current literature has suggested that drugs of abuse have inherently low value, thus promoting food preference. Herein, we examined remifentanil vs. food choice to test both the relative value hypothesis and the ‘direct effects’ (pharmacological effects of drugs on alternative reinforcers) hypothesis of opioid preference.

Methods

Adult male rats were trained under two choice procedures (controlled vs. uncontrolled reinforcer frequency) for remifentanil vs. food choice. Furthermore, a series of procedural manipulations known to affect drug reinforcement were tested under both choice procedures. Using remifentanil self-administration data, pharmacokinetic profiles were calculated and analyzed to determine if opioid intake was related to opioid preference.

Results

Both choice procedures produced dose-dependent preference. Moreover, procedural manipulations produced comparable changes in remifentanil preference under both choice procedures. In addition, calculated pharmacokinetic data revealed that preference was dissociable from intake under the controlled reinforcer frequency choice procedure.

Conclusions

When compared to the ‘direct effects’ hypothesis, remifentanil preference was better predicted by the relative value hypothesis, formalized in generalized matching. Use of a controlled reinforcer frequency schedule successfully removed the drug preference-intake confound found in most drug-choice procedures. Importantly, drug preference under the controlled reinforcer frequency schedule remained sensitive to procedural manipulations known to affect drug reinforcement. Thus, given that differential drug intake itself affects neurobiological measurements, future use of controlled reinforcer frequency schedules may help to better isolate the neurobehavioral mechanisms that mediate opioid preference.

Keywords: remifentanil, choice, matching, magnitude, rat

1. Introduction

Substance-use disorder is, at least partially, characterized as an increased amount of time allocated towards drug-related behavior, relative to nondrug alternatives (Heyman, 2013). Hence, an increase in preclinical drug vs. nondrug choice studies have emerged to better model and investigate this human problem (Ahmed et al., 2013; Banks and Negus, 2012). Although preclinical research into drug choice has provided results that are in concordance with human research (Banks and Negus, 2017; Beckmann et al., 2019; Bickel et al., 2010; Cantin et al., 2010; Higgins et al., 2004; Lenoir et al., 2007; Petry et al., 2005; Thomsen et al., 2013), the determinants of drug choice, including opioids, are largely unknown.

According to a relative-value hypothesis, like that formalized within the generalized matching framework (Davison and McCarthy, 1988), preference is determined by the relative differences in reinforcer dimensions (e.g., magnitude, probability, delay, etc.) across all concurrently-available alternatives. For example, consistent with matching predictions, preclinical research into drug vs. nondrug choice has demonstrated that relative reinforcer magnitude, price, delay, and frequency are reinforcer dimensions that determine preference (Anderson et al., 2002; Beckmann et al., 2019; Negus, 2003; Panlilio et al., 2017; Secci et al., 2016; Thomsen et al., 2013), and these results also parallel reported choice studies using only nondrug reinforcers (e.g., Alsop and Elliffe, 1988; Heyman and Monaghan, 1994; Landon et al., 2003) or only drug reinforcers (e.g., Anderson and Woolverton, 2000; Koffarnus and Woods, 2008; Woolverton, 1996).

Contrary to matching predictions, the ‘direct effects’ hypothesis states that drug preference is determined by the pharmacological presence of a drug of abuse, such that intake of said drug directly modulates the nondrug alternative (Vandaele et al., 2016). Specifically, the ‘direct effects’ hypothesis suggests that psychostimulants (e.g., cocaine) that have noted anorectic effects (Woolverton et al., 1978) ‘directly’ modulate food reinforcement, consequently increasing psychostimulant preference. Likewise, the ‘direct effects’ hypothesis suggests that opioids (e.g., heroin) that have noted orexigenic effects (Rideout and Parker, 1996) directly modulate food reinforcement, consequently increasing preference for food.

To tease apart the two hypotheses above, the current study investigated remifentanil, a fast-acting synthetic opioid (Haidar et al., 1997; Panlilio and Schindler, 2000), vs. food choice. Because remifentanil has a very rapid half-life (~0.7 min; Haidar et al., 1997), ‘direct effects’ are minimized. Moreover, the ‘direct effects’ hypothesis suggests that, absent of any ‘direct’ effects, the value of drugs of abuse is inherently low (Ahmed, 2018; Lenoir et al., 2013; Vandaele et al., 2016); thus, remifentanil preference should be minimal, even nonexistent. Further, as the dose of remifentanil increases, the likelihood of ‘direct’ effects increases; consequently, the ‘direct effects’ hypothesis predicts that animals should be more likely to choose food when under the influence of remifentanil (per orexigenic ‘direct’ effects associated with opioids). Therefore, dose-dependent opioid preference should not be observed. On the contrary, if opioid preference is determined by relative subjective remifentanil-food value, as formalized in generalized matching, then changes to remifentanil dose (i.e., magnitude) should influence remifentanil preference independent of overall remifentanil intake.

To address the above, we examined remifentanil vs. food choice under two choice procedures: uncontrolled and controlled reinforcer frequency. Most drug-choice studies rely on procedures where drug preference is determined by the number of drug reinforcers earned (i.e., intake), which consequently results in the confounding of drug preference and drug intake (i.e., uncontrolled reinforcer frequency). To dissociate drug preference from intake, we employed the use of a controlled reinforcer frequency procedure to control for differences in the frequency of reinforcement across available reinforcers, which subsequently controls for relative differences in reinforcer intake. Furthermore, under an uncontrolled reinforcer frequency schedule, subjects determine the relative ratio of drug:food reinforcers earned. Importantly, the difference in reinforcer frequency/rate across given alternatives is an important determinant of choice (Anderson et al., 2002; Beckmann et al., 2019; McCarthy and Davison, 1984), and under the controlled reinforcer frequency schedule ,differences in drug:food reinforcer ratio are experimentally controlled. Finally, several procedural manipulations were conducted to determine if remifentanil vs. food choice was sensitive to factors known to influence drug reinforcement. If remifentanil vs. food choice is solely driven by ‘direct’ effects, then these manipulations should not influence remifentanil preference, as remifentanil intake is constant within the controlled reinforcer frequency schedule. However, if remifentanil preference follows the predictions of relative subjective value, then choice should be affected systematically by the procedural manipulations.

2. Methods

2.1. Subjects

Twenty-four adult male (data was collected prior to current NIH standards) Sprague Dawley rats (PND 60; ~250-275 g) from Harlan Inc. (Indianapolis, IN) were individually housed (12:12 hr light:dark cycle) upon arrival with ad libitum access to water. Food was unrestricted for the majority of the experiment; however, during periods of food restriction, rats were maintained at ~85% of their free-feeding body weights. Rats were given a week to acclimate upon arrival and handled daily prior to experimentation. All experiments were conducted according to the 2010 NIH Guide for the Care and Use of Laboratory Animals (8th edition) and were approved by the Institutional Animal Care and Use Committee at the University of Kentucky.

2.2. Apparatus

Experiments were conducted in operant chambers (ENV-008, MED Associates, St. Albans, VT) enclosed within sound-attenuating boxes (ENV-018M). Each chamber was connected to a personal computer (SG-502), and all chambers were operated using MED-PC IV. Within each chamber, a recessed food receptacle (ENV-200R2MA) outfitted with a head-entry detector (ENV-254-CB) was located on the front intelligence panel of the chamber, two retractable response levers were mounted on either side of the food receptacle (ENV-122CM), and a white cue-light (ENV-221M) was mounted above each response lever. The back-intelligence panel was outfitted with two nosepoke response ports (ENV-114BM) directly opposite to front response levers, a house-light (ENV-227M) was located at the top of the back panel between the two nosepoke response receptacles with Sonalert© tones (ENV-223 AM and ENV223-HAM) located on either side of the house-light. Food pellets (45-mg Bio-Serv Precision Pellets; Flemington, NJ) were delivered via a dispenser (ENV-203M-45). Drug infusions were delivered via a syringe pump (PHM-100), located outside of the sound-attenuating boxes, through tubing strung through a leash (PHM-120) that attached to a swivel (PHM-115) above the chamber.

2.3. Drugs

Remifentanil hydrochloride, purchased from Sigma-Aldrich (St. Louis, MO), was mixed in sterile saline (0.9% NaCl).

2.4. Establishing Procedures

All rats were trained on a series of establishing procedures to prepare for remifentanil vs. food choice. See supplementary materials for detailed description regarding food-training, catheterization, drug self-administration, and food and remifentanil lever training.

2.5. Experiment Proper

Following establishing procedures, rats were randomly assigned to either the controlled reinforcer frequency or uncontrolled reinforcer frequency schedule for remifentanil versus food choice; choice procedures were identical to those used in Beckmann et al (2019). Briefly, each session was divided into 5 distinct blocks separated by a dark 2-min inter-block-interval. In each block, a distinct cycling-tone pattern (49 kHz and 20 kHz at 1.8/0, 1.5/0.3, 0.9/0.9, 0.3/1.5, and 0/1.8 seconds, respectively) was used as a discriminative stimulus. In all 5 blocks, the nondrug alternative was kept constant, such that upon delivery of a single 45-mg food pellet, lever(s) retracted and the cue-light above the corresponding food-lever would turn on for 5.9s. The dose of remifentanil increased as a function of block (0, 0.32, 1.0, 3.2, and 10 μg/kg/infusion), such that upon remifentanil infusion, lever(s) would retract and the cue-light above the corresponding remifentanil-lever would turn on for a varying duration (0, 0.189, 0.59, 1.89, and 5.9s) which matched the pump duration needed to achieve the dose for the given block (see Supplementary Table 1). Within each block, each trial began with the illumination of the house-light where an orienting response into the magazine would turn off the house-light and extend the response lever(s). All response requirements were scheduled on a fixed ratio (FR) and required consecutive responding; a changeover (i.e., switch) in responding would reset the FR count. Upon completion of the FR requirement, lever(s) would retract and reward delivery, signaled by a corresponding cue-light, would occur. Rats were initially trained on a FR1 and were incrementally progressed up to an FR5. All trials were separated by a 10-s intertrial-interval (ITI) blackout (i.e., not stimuli present). Sessions ended upon completion of all 5 blocks.

2.5.1. Controlled Reinforcer Frequency

The controlled reinforcer frequency choice procedure consisted of a total of 3-drug and 3-food trials per block, totaling 6 trials. Both levers were extended during each trial. During each trial only one of the two reinforcers (remifentanil or food) was randomly scheduled for reinforcement. Regardless of which lever the rat responded on, the reinforcer that was scheduled had to be earned to advance onto the next trial. Responses on the lever that was scheduled for reinforcement were recorded as forced responses since responding on the scheduled lever was required, while responses on the lever that was not scheduled for reinforcement were also recorded. Forced responses were then subtracted from total responses made for a given trial to determine preference (total responses – forced responses = choice responses). Importantly, the randomization of reinforcer (i.e., remifentanil vs. food) on each trial ensured that the availability for a given reinforcer was unpredictable, thus rats initiate responding on the preferred option (Beckmann et al., 2019). After all 6 trials were completed, the block ended and entered the inter-block-interval blackout. An example session structure is available in the supplementary files (Figure S1).

2.5.2. Uncontrolled Reinforcer Frequency

The uncontrolled reinforcer frequency choice procedure consisted of a sample-phase and choice-phase in each block. Sample-phases consisted of two trials, where a single lever corresponding with food or remifentanil was randomly presented; the subsequent trial was the opposite lever. Rats were required to complete each sample-trial to advance. After completion of the sample-trials, the choice-phase started where both levers were extended on trial start. With both levers extended, rats had the opportunity to distribute 6 total choices across the two options within 30 minutes. After 6 total reinforcers within a block were earned or 30 minutes had elapsed, the block would end and enter the inter-block-interval blackout. Example session structure available in supplementary files (Figure S2).

2.5.3. Manipulations

Following stability (no linear trends in choice performance parameters for four consecutive days under baseline conditions for either choice procedure, controlled or uncontrolled reinforcer frequency), rats were assigned, via partial Latin square design, to different procedural manipulations (see below). Each condition was in effect for a minimum of ten days. Following stability on a given manipulation, rats were returned to baseline for seven days prior to advancement of the next condition. After completing procedural manipulations under the initially-assigned choice procedure, rats were switched to the other choice procedure, where they completed baseline training and then underwent the procedural manipulations. The selected procedural manipulations have previously been shown to shift drug preference systematically (e.g., Beckmann et al., 2019; Thomsen et al., 2013), and were utilized to shift preferences under choice procedures where corresponding intake was controlled and uncontrolled to determine if opioid preference and intake are dissociable.

2.5.3.1. Food restriction

To determine the effects of food motivation on remifentanil choice, rats were food restricted and maintained at ~85% of their free-feeding body weights during the testing period.

2.5.3.2. Infusion cue removal

To determine the effects of remifentanil-associated conditioned reinforcement on choice, the cue-light signaling remifentanil infusion was removed; thus, remifentanil delivery went unsignaled across all blocks.

2.5.3.3. Orienting-response removal

To determine the effects of subject-determined trial initiation on choice, the orienting response was removed. All trials were no longer initiated by a head-entry into the magazine; thus, the house-light was not used, and all trials began immediately with the extension of the response lever(s).

The resulting n-sizes for each conditions is as follows: n=17 for both controlled and uncontrolled reinforcer frequency under baseline; n=9 for controlled and n=8 for uncontrolled under food restriction; n=13 for controlled and n=12 for uncontrolled under removal of remifentanil cue; n=10 for controlled and n=9 for uncontrolled under removal of orienting response (i.e., head-entry). See Supplementary Table 2 for an example timeline. All attrition was due solely to catheter failure.

2.6. Analysis

Percent remifentanil choice was calculated as the number of remifentanil choice responses made divided by total choice responses made for the controlled reinforcer frequency, while percent remifentanil choice was calculated as the number of remifentanil reinforcers earned divided by the total number of reinforcers earned for the uncontrolled reinforcer frequency for each block. Given that the scale used to calculate preference under an uncontrolled reinforcer frequency choice procedure is binary (i.e., drug vs. nondrug) and the scale used to calculate preference under the controlled reinforcer frequency choice procedure is continuous, we examined if preference (calculated as the very-first response emitted on each trial, i.e., the number of first responses for remifentanil on each trial over the total number of trials to create a binary scale akin to the uncontrolled reinforcer frequency schedule) was different depending on the scale used. We found that both preference measures (choice responses vs. first response) were nearly identical (r = 0.99), consistent with previous research using the controlled reinforcer frequency schedule in cocaine choice (Beckmann et al., 2019).

Since remifentanil and food are qualitatively different reinforcers, the following form of the generalized matching equation (Beckmann et al., 2019; Hutsell et al., 2015) was applied:

BrBr+Bf=1001+(aMr)sM (1)

where Br represents behavior for remifentanil, Bf represents behavior for food, and Mr represents the magnitude of remifentanil. The free parameter sM represents the sensitivity to relative magnitude differences for remifentanil:food reinforcement, while a represents the remifentanil-food exchange rate, which can be conceptualized as a scaling constant such that a single 45-mg food pellet is scaled in unit dose of remifentanil. For example, low exchange values would suggest less effective substitutability of remifentanil by food, while large exchange values would suggest more effective substitutability. Best-fit model parameters (a and sM) were determined via nonlinear mixed-effects modeling (NLME; Pinheiro et al., 2006), with schedule (nominal), condition (nominal), and dose (continuous) as within-subject factors, and subject as a random factor. See Figure 1 for theoretical graphs based on parameter changes in remifentanil-food exchange rate (a) and sensitivity to relative magnitude differences (sM).

Figure 1.

Figure 1.

Open in a new tab

Simulated remifentanil vs. food choice functions from Equation 1. (A) The effects of changing remifentanil-food exchange rate (a) with sensitivity to relative magnitude differences held constant (sM = 4). Changes in exchange rate shifts the choice curve while maintaining the shape. (B) The effects of changing sensitivity to relative magnitude differences (sM) with remifentanil-food exchange rate (a = 2) held constant. Changes in sensitivity to relative magnitude differences shifts the shape of the choice curves while maintaining the exchange rate (i.e., 50% mark).

Using data from the same sessions used to calculate preference, whole-body remifentanil levels at choice were estimated based on an elimination half-life of 0.7 min (Haidar et al., 1997; Panlilio et al., 2003) according to the following equation:

Bn=(Bn−1+D)e−KT (2)

where Bn represents the amount of remifentanil in the body at the current time, Bn-1 represents the amount of drug in the body at the time of previous calculation, D represents the dose of remifentanil for the given block, K represents the elimination constant (1.0043; Panlilio et al., 2003), and T represents minutes since last infusion. Importantly, pharmacokinetic data was estimated from recorded self-administration data used to calculate choice. Averaged whole-body remifentanil concentration at choice was analyzed using linear mixed-effects modeling (LME; Gelman and Hill, 2006) with schedule (nominal), condition (nominal) and dose (continuous) as within-subject factors, and subject as a random factor. Correlations between remifentanil-food exchange (a) and averaged estimated whole-body remifentanil concentrations at choice during the last block (i.e., 10 μg/kg/infusion) was calculated using Pearson’s r.

Since remifentanil is short-acting, an additional estimation of whole-body levels at choice on a trial-by-trial basis was examined. Trial-by-trial whole-body remifentanil concentrations was analyzed via LME with schedule (nominal), condition (nominal) and trial (continuous) as within-subject factors, and subject as a random factor. Correlations between remifentanil-food exchange (a) and an aggregated-estimate (i.e., averaged levels per block summed) of whole-body remifentanil concentrations at choice was calculated using Pearson’s r. For all tests, α was set to 0.05.

3. Results

Figure 2A illustrates percent remifentanil choice under controlled and uncontrolled reinforcer frequency procedures at baseline. NLME analysis of baseline preference revealed differences in remifentanil-food exchange (a) [F(1,166) = 6.28, p < 0.05], while there were no significant differences in sensitivity to reinforcer magnitude. Thus, while both choice procedures produced similar dose-dependent preference, food functioned as a more effective substitute under the uncontrolled reinforcer frequency schedule (i.e., greater a value; rightward shift).

Figure 2.

Figure 2.

Open in a new tab

Effects of procedural manipulations on remifentanil preference. (A) Mean (± SEM) percent choice for remifentanil under the controlled and uncontrolled reinforcer frequency choice procedure at baseline; n = 17/procedure. (B) Mean (± SEM) percent choice for remifentanil under controlled reinforcer frequency at baseline (BL), food restriction (FoodRes; n = 9), no remifentanil cue (NRC; n = 13), and no head-entry (NHE; n = 10) conditions. (C) Mean (± SEM) percent choice for remifentanil under uncontrolled reinforcer frequency at baseline (BL), food restriction (FoodRes; n = 8), no remifentanil cue (NRC; n = 12), and no head-entry (NHE; n = 19) conditions. Best-fit parameter estimates from NLME generalized matching fits (Eqn 1) for (D) remifentanil-food exchange (a) and (E) sensitivity to relative remifentanil-food magnitude under the different schedules and conditions. Lines are best fits of Equation 1.

Figure 2B and 2C illustrates percent remifentanil choice across different procedural manipulations under the controlled and uncontrolled choice procedures, along with parameter estimates for (2D) remifentanil-food exchange (a) and (2E) sensitivity to magnitude via generalized matching fits. NLME analysis revealed a significant main effect of schedule [F(1,441) = 16.17, p < 0.05], a significant main effect of condition [F(3,441) = 28.17, p < 0.05], and a significant schedule by condition interaction [F(3,441) = 6.18, p < 0.05] on remifentanil-food exchange (a), indicating that remifentanil-food substitution was affected by the different procedural manipulations, and these differences were schedule dependent. Post hoc comparisons indicated that food restriction and removal of remifentanil cue increased remifentanil-food exchange (a) under both choice procedures indicating that food functioned as a more effective substitute under each manipulation. Additionally, post hoc comparisons indicated that removal of the head entry response decreased remifentanil-food exchange (a) under both procedures indicating food functioned as a less effective substitute. NLME analysis revealed no significant effects on sensitivity to magnitude (sM) under either choice procedure.

Figure 3 represents averaged whole-body remifentanil levels at choice under both (3A & 3C) procedures during different manipulations. LME analysis revealed a significant main effect of schedule [F(1,18.68) = 15.50, p < 0.05], condition [F(3,33.76) = 11.53, p < 0.05], and dose [F(1,17.55) = 160.85, p < 0.05]. LME analysis also revealed a significant condition x dose interaction [F(3,35.28) = 5.58, p < 0.05]. Altogether, these results indicate that averaged whole-body levels increased with dose, but the rate of increase was differentially affected by the procedural manipulations. Furthermore, (3B & 3D) correlations between remifentanil-food exchange (a) and whole-body remifentanil levels during the last block were calculated, and the results revealed that whole-body remifentanil levels were independent of preference for both controlled (Pearson’s r = 0.096, NS) and uncontrolled (Pearson’s r = 0.21, NS) schedules.

Figure 3.

Figure 3.

Open in a new tab

Estimated remifentanil levels as a function of block. Mean (± SEM) whole-body remifentanil levels at choice, averaged during choice for each block, under the (A) controlled and (C) uncontrolled reinforcer frequency schedules. Correlations between individual remifentanil-food exchange (a) and whole-body levels reached during the last block under the (B) controlled and (D) uncontrolled reinforcer frequency procedures for the different conditions.

Figure 4 represents trial-by-trial whole-body remifentanil levels under both (4A & 4C) choice procedures during different manipulations. LME analysis revealed a significant main effect of schedule [F(1,18.33) = 109.47, p < 0.05], condition [F(3,35.08) = 13.18, p < 0.05], and trial [F(1,17.99) = 220.00, p < 0.05]. LME analysis also revealed a significant condition x trial interaction [F(3,38.74) = 12.38, p < 0.05], and a significant schedule x trial interaction [F(1,38.74) = 12.38, p < 0.05]. Altogether, these results indicate that whole-body levels increased as a function of trial, where the increase was affected by the procedural manipulation and schedule used. Furthermore, (4B & 4D) correlations between remifentanil-food exchange (a) and the aggregated whole-body levels (since remifentanil levels were eliminated during the inter-block-intervals) revealed that that whole-body remifentanil levels were independent of preference under the controlled reinforcer frequency schedule (Pearson’s r = 0.082, NS); however, under the uncontrolled reinforcer frequency schedule (Pearson’s r = 0.40, p < 0.05) a significant correlation was observed.

Figure 4.

Figure 4.

Open in a new tab

Estimated remifentanil levels on a trial-by-trial basis. Mean (± SEM) whole-body remifentanil levels at choice for each trial under the (A) controlled and (C) uncontrolled reinforcer frequency schedules. Correlations between individual remifentanil-food exchange (a) and aggregated whole-body levels under the (B) controlled and (D) uncontrolled reinforcer frequency procedures for the different conditions. * indicates p < 0.05.

4. Discussion

The experiments herein utilized controlled and uncontrolled reinforcer frequency choice procedures to examine remifentanil vs. food choice. First, dose-dependent remifentanil preference was observed under both choice procedures. Second, when whole-body remifentanil level at choice was estimated, it was revealed that preference was independent of drug intake. Finally, when remifentanil vs. food choice was examined under different procedural manipulations, both choice procedures produced similar shifts in remifentanil preference. Altogether, these results indicate that remifentanil preference was determined by differences in relative subjective reinforcer (remifentanil and food) value, as formalized by the generalized matching framework.

4.1. The role of ‘direct’ effects on remifentanil vs. food choice

According to the ‘direct effects’ hypothesis (Vandaele et al., 2016), drug intake should directly modulate choice of a nondrug alternative. Within the context of the present experiment, when remifentanil is pharmacologically-present, food preference should increase due to the ‘direct’ orexigenic effects of opioids (Rideout and Parker, 1996). However, there are several results that fail to support the ‘direct effects hypothesis’. First, dose-dependent increases in remifentanil choice was observed in both choice procedures, including all procedural conditions tested within each procedure. Importantly, rats under the controlled reinforcer frequency procedure experienced the same relative remifentanil-food reinforcement across all procedural manipulations that resulted in comparable patterns of whole-body remifentanil levels, yet choice for remifentanil varied systematically, indicating a dissociation between whole-body levels of remifentanil and remifentanil preference. Second, correlations between remifentanil-food exchange (a) and whole-body levels during the last block (reflective of block-by-block intake), or an aggregated whole-body level for the session (more reflective of trial-by-trial intake), revealed that remifentanil preference was independent from remifentanil intake under a controlled reinforcer frequency schedule. Finally, the ‘direct effects’ hypothesis postulates that drugs of abuse inherently have low value, in general, and should not be preferred over nondrug reinforcers absent of direct effects (Vandaele et al., 2016). Importantly, when whole-body trial-by-trial remifentanil levels were examined, estimated levels at the start of each choice block (i.e., after the 2-min inter-block-interval) were nearly 3 half-lives out from the previous infusion. Thus, despite the absence of ‘direct’ effects, rats still chose remifentanil over food at the highest dose.

While the ‘direct effects’ hypothesis may help explain the results for opioid vs. nondrug choice within specific contexts (e.g., Lenoir et al., 2013; Vandaele et al., 2016), it has trouble accounting for the results of many other opioid studies in the literature, including the present results. If ‘direct’ orexigenic effects determined preference between opioids and food, then at higher doses where the likelihood of an opioid being pharmacologically active at the time of choice should consequently increase food preference, eliminating dose-dependent preference. However, previous studies have examined heroin or fentanyl vs. food choice and have demonstrated dose-dependent opioid preference repeatedly, with no preference reversals toward food at high opioid doses (Negus, 2006, 2005; Townsend et al., 2019b, 2019a). The results reported herein also demonstrate opioid preference at the highest remifentanil dose, without any indication of preference reversal towards food at higher doses. Importantly, dose-dependent effects are also observed consistently in human clinical opioid vs. nondrug choice studies (Comer et al., 2013, 1997; Jones and Comer, 2013). Thus, most existing results do not support the ‘direct effects’ hypothesis. Rather, most existing results collectively are consistent with predictions from the generalized matching framework that suggest opioid preference is determined by relative subjective value, such that differences in reinforcer dimensions, like magnitude and reinforcer substitutability, determine choice between alternatives.

4.2. Controlled and uncontrolled reinforcer frequency choice procedures

While most drug choice studies to date employ uncontrolled reinforcer frequency schedules, there are some inherent issues with the design. Under uncontrolled reinforcer frequency choice procedures, preference is calculated as the number of drug reinforcers earned divided by the total reinforcers earned. While this measure is widely used (e.g., Lenoir et al., 2007; Negus, 2003; Thomsen et al., 2013), the measure for preference covaries directly with reinforcer intake. That is, the number of drug reinforcers chosen, and the number of drug reinforcers consumed are the same measure, making preference and intake synonymous. Additionally, rate or frequency of reinforcement is subject determined within uncontrolled reinforcer frequency procedures. Importantly, frequency or rate of reinforcement is a significant, well-known determinant of drug preference (Anderson et al., 2002; Beckmann et al., 2019); thus, in procedures where reinforcer frequency is uncontrolled, the impact of this reinforcer dimension on preference cannot be determined appropriately. Furthermore, differences in reinforcer intake are known to directly influence neurobiological adaptions (Hyman et al., 2006; Kalivas and O’Brien, 2008; Nestler, 2001), including within drug-choice procedures (Chow et al. under review). Thus, in studies investigating the neurobehavioral mechanisms of choice where a given subject experienced greater overall drug intake, the neurobiological changes observed may be interpreted to be adaptations associated with preference via increased drug choices; however, these changes may instead be reflective of increased drug intake. Importantly, by using a controlled reinforcer frequency schedule, these reinforcer frequency and intake issues are controlled. As seen herein and elsewhere (Beckmann et al., 2019), under a controlled reinforcer frequency schedule, the relative drug:food ratio of reinforcers earned is experimentally controlled, such that the effects of frequency of reinforcement on preference and associated neurobiology can be estimated, without covarying effects due to intake differences.

Prior research using controlled reinforcer frequency schedules to investigate drug vs. food choice was completed using cocaine vs. food choice (Beckmann et al., 2019). First, like cocaine, remifentanil preference under a controlled reinforcer frequency schedule was dose-dependent and dissociable from overall drug intake. Second, the results reported herein with remifentanil mirror those seen in cocaine vs. food choice, where procedural manipulations known to affect drug reinforcement (i.e., food restriction, removal of drug cue, and removal of orienting response) produced comparable changes in drug-food preference. Specifically, food restriction and removal of the drug cue increased the substitutability of food for remifentanil, while removal of the head-entry response decreased food substitutability for remifentanil. Also mirroring previous cocaine vs. food studies (Beckmann et al., 2019; Hutsell et al., 2015), sensitivity to relative remifentanil-food magnitude (sM) differences was not significantly affected by procedural manipulations. Additionally, response patterns (i.e., dose-dependent increases in latency to first choice and overall drug/food response rates) observed herein (see supplemental files; Figure S3 and S4) were similar to those reported in previous drug-choice studies (e.g., Beckmann et al., 2019; Iglauer and Woods, 1974; Llewellyn et al., 1976; Negus, 2003; Thomsen et al., 2013). Finally, the results obtained under the controlled reinforce frequency choice procedure are in line with results seen in uncontrolled reinforcer frequency choice procedures from our lab and others (e.g., Negus, 2006, 2003; Thomsen et al., 2013; Townsend et al., 2019b, 2019a), providing further evidence that the controlled reinforcer frequency schedule is measuring the same preference process as uncontrolled procedures, with the added benefit of removing the reinforcer frequency/intake confound.

It should be noted that this study examined remifentanil choice in male rats only. Previous investigation into sex differences in opioid vs. food choice using uncontrolled reinforcer frequency schedules have revealed mixed results (Townsend et al., 2019b; Venniro et al., 2019, 2017). Thus, future studies using controlled reinforcer frequency schedules need to investigate the possibility of sex differences. Moreover, use of a controlled reinforcer frequency choice procedure may also help elucidate if sex differences are a byproduct of differential drug intake or a sex-dependent predisposition to differential subjective drug value.

5. Conclusions

While controlled reinforcer frequency procedures have yet to be utilized in clinical substance-abuse research, these procedures have been utilized successfully in other human clinical research and have revealed insight into the reinforcement and decision-making processes associated with other behavioral disorders (Alsop et al., 2016; Pizzagalli et al., 2005), providing an impetus for adaptation of this type of schedule into substance-use research. Additionally, the majority of drug-choice results, both clinical and preclinical, from our lab and others follow the predictions formalized by generalized matching, such that drug preference is determined by differences in relative subjective value defined by the current choice context. The consistency between matching predictions and a large collection of data from multiple drug classes, clinical and preclinical studies, and various procedures, suggests relative subjective value as a possible unifying process mediating drug choice. Coupling quantitative models that formalize relative subjective value with neurobiological measurements in future drug-choice research may help to better isolate the neurobehavioral mechanisms that underlie substance-use disorder.

Supplementary Material

1

Highlights.

  • Dose-dependent remifentanil preference was observed under all procedures tested

  • Remifentanil preference was predicted by a relative value model

  • Quantitative models of relative value may help to unify drug preference results

Acknowledgements

We would like to thank Josh N Lavy and Beckmann lab members for their technical support.

Funding and Disclosure

This work was supported by the National Institute on Drug Abuse grants DA033373, DA045023, and DA047368.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest

The authors, JJC & JSB, report no conflict of interests.

Data Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Ahmed SH, 2018. Trying to make sense of rodents’ drug choice behavior. Prog. Neuro-Psychopharmacology Biol. Psychiatry 87, 3–10. [DOI] [PubMed] [Google Scholar]
  2. Ahmed SH, Lenoir M, Guillem K, 2013. Neurobiology of addiction versus drug use driven by lack of choice. Curr. Opin. Neurobiol 23, 581–587. [DOI] [PubMed] [Google Scholar]
  3. Alsop B, Elliffe D, 1988. Concurrent-schedule performance: Effects of relative and overall reinforcer rate. J. Exp. Anal. Behav 49, 21–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Alsop B, Furukawa E, Sowerby P, Jensen S, Moffat C, Tripp G, 2016. Behavioral sensitivity to changing reinforcement contingencies in attention-deficit hyperactivity disorder. J. Child Psychol. Psychiatry 57, 947–956. [DOI] [PubMed] [Google Scholar]
  5. Anderson KG, Velkey AJ, Woolverton WL, 2002. The generalized matching law as a predictor of choice between cocaine and food in rhesus monkeys. Psychopharmacology (Berl). 163, 319–326. [DOI] [PubMed] [Google Scholar]
  6. Anderson KG, Woolverton WL, 2000. Concurrent variable-interval drug self-administration and the generalized matching law: A drug-class comparison. Behav. Pharmacol 11, 413–420. [DOI] [PubMed] [Google Scholar]
  7. Banks ML, Negus SS, 2017. Insights from preclinical choice models on treating drug addiction. Trends Pharmacol. Sci 38, 181–194. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Banks ML, Negus SS, 2012. Preclinical determinants of drug choice under concurrent schedules of drug self-administration. Adv. Pharmacol. Sci 2012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Beckmann JS, Chow JJJ, Hutsell BA, 2019. Cocaine-associated decision-making: Toward isolating preference. Neuropharmacology 153, 142–152. 10.1016/j.neuropharm.2019.03.025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Bickel WK, Jones BA, Landes RD, Christensen DR, Jackson L, Mancino M, 2010. Hypothetical intertemporal choice and real economic behavior: Delay discounting predicts voucher redemptions during contingency-management procedures. Exp. Clin. Psychopharmacol 18, 546. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cantin L, Lenoir M, Augier E, Vanhille N, Dubreucq S, Serre F, Vouillac C, Ahmed SH, 2010. Cocaine is low on the value ladder of rats: possible evidence for resilience to addiction. PLoS One 5, e11592. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Comer SD, Collins ED, Fischman MW, 1997. Choice between money and intranasal heroin in morphine-maintained humans. Behav. Pharmacol [DOI] [PubMed] [Google Scholar]
  13. Comer SD, Metz VE, Cooper ZD, Kowalczyk WJ, Jones JD, Sullivan MA, Manubay JM, Vosburg SK, Smith ME, Peyser D, 2013. Comparison of a drug versus money and drug versus drug self-administration choice procedure with oxycodone and morphine in opioid addicts. Behav. Pharmacol 24, 504. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Davison M, McCarthy D, 1988. The matching law: A research view. [Google Scholar]
  15. Gelman A, Hill J, 2006. Data analysis using regression and multilevel/hierarchical models. Cambridge university press. [Google Scholar]
  16. Haidar SH, Moreton JE, Liang Z, Hoke JR, Muir KT, Eddington ND, 1997. The pharmacokinetics and electroencephalogram response of remifentanil alone and in combination with esmolol in the rat. Pharm. Res 14, 1817–1823. [DOI] [PubMed] [Google Scholar]
  17. Heyman GM, 2013. Addiction and choice: theory and new data. Front, psychiatry 4, 31. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Heyman GM, Monaghan MM, 1994. REIN FORCER MAGNITUDE (SUCROSE CONCENTRATION) AND THE MATCHING LAW THEORY OF RESPONSE STRENGTH. J. Exp. Anal. Behav 61, 505–516. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Higgins ST, Heil SH, Lussier JP, 2004. Clinical implications of reinforcement as a determinant of substance use disorders. Annu. Rev. Psychol 55, 431–461. [DOI] [PubMed] [Google Scholar]
  20. Hutsell BA, Negus SS, Banks ML, 2015. A generalized matching law analysis of cocaine vs. food choice in rhesus monkeys: Effects of candidate ‘agonist-based’medications on sensitivity to reinforcement. Drug Alcohol Depend. 146, 52–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hyman SE, Malenka RC, Nestler EJ, 2006. NEURAL MECHANISMS OF ADDICTION: The Role of Reward-Related Learning and Memory. Annu. Rev. Neurosci 29, 565–598. 10.1146/annurev.neuro.29.051605.113009 [DOI] [PubMed] [Google Scholar]
  22. Iglauer C, Woods JH, 1974. Concurrent Performances: Reinforcement By Different Doses Of Intravenous Cocaine In Rhesus Monkeys 1. J. Exp. Anal. Behav 22, 179–196. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Jones JD, Comer SD, 2013. A review of human drug self-administration procedures. Behav. Pharmacol 24, 384. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Kalivas PW, O’Brien C, 2008. Drug addiction as a pathology of staged neuroplasticity. Neuropsychopharmacology 33, 166–180. [DOI] [PubMed] [Google Scholar]
  25. Koffarnus MN, Woods JH, 2008. Quantification of drug choice with the generalized matching law in rhesus monkeys. J. Exp. Anal. Behav 89, 209–224. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Landon J, Davison M, Elliffe D, 2003. Concurrent schedules: Reinforcer magnitude effects. J. Exp. Anal. Behav 79, 351–365. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Lenoir M, Cantin L, Vanhille N, Serre F, Ahmed SH, 2013. Extended heroin access increases heroin choices over a potent nondrug alternative. Neuropsychopharmacology 38, 1209–1220. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Lenoir M, Serre F, Cantin L, Ahmed SH, 2007. Intense sweetness surpasses cocaine reward. PLoS One 2, e698. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Llewellyn ME, Iglauer C, Woods JH, 1976. Relative Reinforcer Magnitude Under A Nonindependent Concurrent Schedule Of Cocaine Reinforcement In Rhesus Monkeys 1. J. Exp. Anal. Behav 25, 81–91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. McCarthy D, Davison M, 1984. Isobias and alloiobias functions in animal psychophysics. J. Exp. Psychol. Anim. Behav. Process 10, 390. [PubMed] [Google Scholar]
  31. Negus SS, 2006. Choice between heroin and food in nondependent and heroin-dependent rhesus monkeys: effects of naloxone, buprenorphine, and methadone. J. Pharmacol. Exp. Ther 317, 711–723. [DOI] [PubMed] [Google Scholar]
  32. Negus SS, 2005. Interactions between the reinforcing effects of cocaine and heroin in a drug-vs-food choice procedure in rhesus monkeys: a dose-addition analysis. Psychopharmacology (Berl). 180, 115–124. [DOI] [PubMed] [Google Scholar]
  33. Negus SS, 2003. Rapid assessment of choice between cocaine and food in rhesus monkeys: effects of environmental manipulations and treatment with d-amphetamine and flupenthixol. Neuropsychopharmacology 28, 919–931. [DOI] [PubMed] [Google Scholar]
  34. Nestler EJ, 2001. Molecular basis of long-term plasticity underlying addiction. Nat. Rev. Neurosci 2, 119–128. [DOI] [PubMed] [Google Scholar]
  35. Panlilio LV, Katz JL, Pickens RW, Schindler CW, 2003. Variability of drug self-administration in rats. Psychopharmacology (Berl). 167, 9–19. [DOI] [PubMed] [Google Scholar]
  36. Panlilio LV, Schindler CW, 2000. Self-administration of remifentanil, an ultra-short acting opioid, under continuous and progressive-ratio schedules of reinforcement in rats. Psychopharmacology (Berl). 150, 61–66. [DOI] [PubMed] [Google Scholar]
  37. Panlilio LV, Secci ME, Schindler CW, Bradberry CW, 2017. Choice between delayed food and immediate opioids in rats: treatment effects and individual differences. Psychopharmacology (Berl). 234, 3361–3373. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Petry NM, Alessi SM, Marx J, Austin M, Tardif M, 2005. Vouchers versus prizes: contingency management treatment of substance abusers in community settings. J. Consult. Clin. Psychol 73, 1005. [DOI] [PubMed] [Google Scholar]
  39. Pinheiro J, Bates D, DebRoy S, Sarkar D, 2006. nlme: linear and nonlinear mixed effects models, R package version 3, 1–76. R Foundation for Statistical Computing, Vienna, Austria. [Google Scholar]
  40. Pizzagalli DA, Jahn AL, O’Shea JP, 2005. Toward an objective characterization of an anhedonic phenotype: a signal-detection approach. Biol. Psychiatry 57, 319–327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Rideout HJ, Parker LA, 1996. Morphine enhancement of sucrose palatability: analysis by the taste reactivity test. Pharmacol. Biochem. Behav 53, 731–734. [DOI] [PubMed] [Google Scholar]
  42. Secci ME, Factor JA, Schindler CW, Panlilio LV, 2016. Choice between delayed food and immediate oxycodone in rats. Psychopharmacology (Berl). 233, 3977–3989. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Thomsen M, Barrett AC, Negus SS, Caine SB, 2013. Cocaine versus food choice procedure in rats: environmental manipulations and effects of amphetamine. J. Exp. Anal. Behav 99, 211–233. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Townsend EA, Blake S, Faunce KE, Hwang CS, Natori Y, Zhou B, Bremer PT, Janda KD, Banks ML, 2019a. Conjugate vaccine produces long-lasting attenuation of fentanyl vs. food choice and blocks expression of opioid withdrawal-induced increases in fentanyl choice in rats. Neuropsychopharmacology 44, 1681–1689. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Townsend EA, Negus SS, Caine SB, Thomsen M, Banks ML, 2019b. Sex differences in opioid reinforcement under a fentanyl vs. food choice procedure in rats. Neuropsychopharmacology 1–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Vandaele Y, Cantin L, Serre F, Vouillac-Mendoza C, Ahmed SH, 2016. Choosing under the influence: a drug-specific mechanism by which the setting controls drug choices in rats. Neuropsychopharmacology 41, 646. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Venniro M, Russell TI, Zhang M, Shaham Y, 2019. Operant social reward decreases incubation of heroin craving in male and female rats. Biol. Psychiatry 86, 848–856. [DOI] [PMC free article] [PubMed] [Google Scholar]
  48. Venniro M, Zhang M, Shaham Y, Caprioli D, 2017. Incubation of methamphetamine but not heroin craving after voluntary abstinence in male and female rats. Neuropsychopharmacology 42, 1126–1135. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Woolverton WL, 1996. Intravenous self-administration of cocaine under concurrent VI schedules of reinforcement. Psychopharmacology (Berl). 127, 195–203. [PubMed] [Google Scholar]
  50. Woolverton WL, Kandel D, Schuster CR, 1978. Tolerance and cross-tolerance to cocaine and d-amphetamine. J. Pharmacol. Exp. Ther 205, 525–535. [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

1
NIHMS1638939-supplement-1.docx (226.1KB, docx)

RESOURCES