CHOICE BETWEEN FOOD AND COCAINE REINFORCERS UNDER FIXED AND VARIABLE SCHEDULES IN FEMALE AND MALE RHESUS MONKEYS

Abstract

Illicit drugs like cocaine may be uncertain in terms of the time and effort required to obtain them. Behavior maintained by variable schedules resembles excessive drug-taking compared with fixed schedules. However, no prior research has examined fixed vs. variable schedules in drug vs. nondrug choice. The current study evaluated cocaine vs. food choice under fixed- (FR) and variable-ratio (VR) schedules. The simpler food vs. food and cocaine vs. cocaine arrangements also were included. Adult female (n=6) and male (n=7) rhesus monkeys chose between cocaine (0.01–0.18 mg/kg/injection) and food (4 pellets/delivery), food and food (4 pellets/delivery), or cocaine and cocaine (0.018–0.03 mg/kg/injection) under FR and VR 100 and 200 schedules. In cocaine vs. food choice, cocaine’s potency to maintain choice was greatest when available under a VR 100 or 200 schedule and food under an FR schedule and was lowest when cocaine was available under an FR 200 schedule and food was available under a VR 200 schedule. In food vs. food choice, males chose food associated with a VR schedule more than food associated with an FR schedule. In cocaine vs. cocaine choice, females and males chose cocaine associated with a VR schedule more than cocaine associated with an FR schedule, particularly under VR 200. These findings suggest that uncertainty in terms of time and effort required to obtain cocaine, or perhaps the occasional low-cost access that results from VR schedules, results in greater allocation of behavior toward drug reinforcers at the expense of more certain, nondrug alternatives.

Introduction

A consistent finding in substance-use research is that drug and nondrug choice can be altered by manipulating environmental variables like response requirement, reinforcer magnitude or delay, and others (e.g., Anderson et al., 2003; Campbell and Carroll, 2000; Huskinson et al., 2015b, 2016; Maguire et al., 2013a,b; Nader and Woolverton, 1991; 1992; Negus, 2003; Pickens and Thompson, 1968; Woolverton and Anderson, 2006). In general, environmental manipulations like delay, cost, and magnitude of drug and nondrug reinforcers produce choice patterns that are concordant with human-laboratory (e.g., reduced drug choice and intake) and clinical outcomes (e.g., increased percent of drug-negative urine samples) in contingency management (e.g., Greenwald and Steinmiller, 2009; Higgins et al., 1994; Lile et al., 2016; Packer et al., 2012; Silverman et al., 1999; Stoops et al., 2012; Toegel et al., 2020). Effects of environmental and pharmacological manipulations on drug self-administration have been evaluated primarily with single-operant schedules of drug availability and often with fixed schedules of reinforcement (e.g., fixed-ratio [FR] schedules; see Huskinson, 2020). Furthermore, the schedule of reinforcement in self-administration studies often is a low-value schedule (e.g., FR5) or a relatively rich schedule of reinforcement. An overreliance on single-operant procedures using low-value schedules may impede generalization to the natural environment, where drug and nondrug reinforcers are available concurrently, and many individuals with substance-use disorders (SUDs) are not likely to receive drug and nondrug reinforcers under relatively rich and certain situations (see Lagorio and Winger, 2014; Huskinson, 2020). A nondrug reinforcer such as a paycheck generally occurs at predictable times, in exchange for specific work requirements. Other nondrug reinforcers like consumable goods or hobbies are available at relatively predictable locations and prices, while illicit drugs like cocaine may be less certain in terms of availability, quality, and in the time and effort required to obtain them.

Similarly, while some individuals obtain drugs in a relatively resource rich environment, others have access to drugs under relatively large effort requirements for relatively small amounts of drug (i.e., relatively lean conditions). Indeed, individuals with relatively higher incomes who use cocaine or heroin report higher amounts and frequencies of use compared with individuals with relatively lower incomes who report lower amounts and frequencies of use (Greenwald and Steinmiller, 2014; Roddy and Greenwald, 2009; Roddy et al., 2011). Such lean access to drugs may be especially prevalent in individuals with SUDs who also are experiencing homelessness, unemployment, and/or low incomes. Therefore, overall cost and certainty may be important aspects of reinforcement that differ for illicit drugs relative to nondrug reinforcers in this population. For example, drug seeking sometimes results in relatively immediate payoff and other times requires a large amount of time and effort in the drug-procurement process, resulting in delayed acquisition of drug effects, and yet very little research has examined the extent to which uncertain and lean drug access is an important determinant of drug choice.

There are multiple aspects of uncertainty that could affect the value of drug and nondrug reinforcers. One example is the effort required to obtain reinforcers, which can be modeled experimentally with variable response requirements using variable-ratio (VR) or random-ratio (RR) schedules of reinforcement. A VR schedule is a response-based schedule that requires a varying number of responses per reinforcer delivery, and a RR schedule is similar to a VR, but the scheduling is such that each response has a constant probability of being reinforced (see Madden et al., 2007 for an example of an RR schedule). Behavior maintained under a VR or RR schedule has been compared with an FR schedule of reinforcement, a large portion of which has been done with nondrug reinforcers like food or liquids (e.g., Madden et al. 2005). For example, VR and RR schedules result in high rates of behavior with little pausing after reinforcer delivery or between response bouts (e.g., Ferster and Skinner, 1957) and result in less elastic demand compared with behavior maintained under FR schedules (Madden et al., 2005). Moreover, reinforcers associated with VR or RR schedules are chosen over those associated with FR schedules (e.g., Fantino, 1967; Field et al., 1996; Madden and Hartman, 2006), even when the variable requirement is larger, on average, than the fixed requirement (Ahearn et al. 1992; Goldshmidt and Fantino, 2004; Johnson et al., 2011, 2012).

Recently, differences between behavior maintained by fixed and variable schedules of nondrug reinforcement have been extended to drug self-administration in an interesting way. Rodents with a history of responding under a VR schedule of saccharin delivery subsequently self-administered more amphetamine under a progressive-ratio (PR) schedule compared to subjects responding under an FR schedule of saccharin availability (Mascia et al. 2019). This work suggests that acquisition of nondrug commodities under a VR schedule can strengthen subsequent drug-taking behavior when the VR schedule of reinforcement is unrelated to drug procurement. A similar effect has been reported in human participants where uncertain monetary gains increased consumption of other consumable goods (i.e., beer and palatable foods/drinks) compared with certain monetary gains (Rauwolf et al., 2021). Others have extended research with fixed vs. variable response requirements by making the schedule of drug reinforcement fixed or variable, rather than focusing on nondrug reinforcers. Lagorio and Winger (2014) found that demand for cocaine, remifentanil, and to a lesser extent, ketamine, was less elastic under an RR schedule compared with an FR schedule in a behavioral-economic procedure. These effects were most robust under relatively large cost requirements and with relatively smaller doses (i.e., relatively lean drug access). Similarly, we demonstrated that male rhesus monkeys chose cocaine associated with a VR schedule and/or a variable dose per delivery over fixed response requirements and doses (Huskinson et al., 2017). Taken together, outcomes with drug reinforcers suggest that variable response requirements, particularly when the average response cost is relatively large, results in high-rate behavior and greater behavioral allocation toward the variable schedule compared with fixed schedules.

If one accepts that uncertainty in terms of the time and effort required to procure illicit drugs in the natural environment can be modeled with variable schedules, an important next step is to understand how a variable schedule of reinforcement affects drug vs. nondrug choice. Drug vs. nondrug choice has received considerable attention because it has good predictive validity (see Banks et al., 2015 for a review). The major goal of the current experiments was to evaluate cocaine vs. food choice under FR and VR schedules of reinforcement. In addition to drug vs. nondrug choice, choice scenarios between the same type of reinforcers also were included to evaluate how food or cocaine choice is affected by FR vs. VR schedules in the simpler food vs. food and cocaine vs. cocaine choice arrangements.

Method

Animal-use procedures were approved by the University of Mississippi Medical Center’s Institutional Animal Care and Use Committee and were conducted in accordance with the National Research Council’s Guide for Care and Use of Laboratory Animals (8^th edition, 2011). We report (below) how we determined our sample sizes, data exclusions, manipulations, and measures used in the current study.

Subjects and Apparatus

A total of six female and seven male adult rhesus monkeys (Macaca mulatta) served as subjects. Similar choice studies with rhesus monkeys in our laboratory have shown that 3–6 subjects are typically sufficient to draw meaningful conclusions on a within-subject basis (e.g., Huskinson et al., 2015a,b, 2016, 2017, 2019). However, previous work has largely been conducted with males, and the goal of the current experiments was to include 3–4 subjects per sex per condition. Subject ID’s and characteristics are shown in Table 1, including the conditions each subject experienced; sex, and for females, whether the menstrual cycle was regular or irregular; most recent history at the beginning of the study; and the reference for the most recent experiment where one exists. Menstrual cycles were monitored in female subjects via vaginal swabs to detect the presence/absence of menstruation, and while some potential sex differences were obtained in these experiments, none of the data pointed toward a systematic effect of a subject’s day in the menstrual cycle on any outcomes. Similarly, a subject’s experimental history did not appear to systematically impact the outcomes in any condition. For two subjects (42286, 160–2008), some conditions were terminated prior to completion of all doses or both FR or VR values as a result of depletion of useable veins for catheterization. Only data collected prior to confirmed catheter patency were included in the current data set.

Table 1.

Subject Characteristics

Subject ID	Conditions Each Subject Experienced			Female or Male (Menstrual Cycle)	Most Recent History at Start of Study	Reference
	Food vs. Food	Cocaine vs. Cocaine	Food vs. Cocaine
71–2006	Yes	Yes	Yes	Female (irregular)	Food and Druga SA	Unpublished Data
425–2003	Yes	No	Yes	Female (irregular)	Food and Cocaine SA	Unpublished Data
40852	Yes	Yes	Yes	Female (regular)	Naïve	N/A
42286	No	Yes	Yes	Female (regular)	Naïve	N/A
48–2005	Yes	No	No	Female (irregular)	Food and Druga SA	Unpublished Data
39608	No	Yes	No	Female (regular)	Naïve	N/A
69–2012	No	Yes	Yes	Male (N/A)	Naïve	N/A
64–2012	Yes	Yes	Yes	Male (N/A)	BZ and BZ-type SA	Unpublished Data
160–2008	Yes	No	Yes	Male (N/A)	BZ and BZ-type SA	Unpublished Data
1356	No	No	Yes	Male (N/A)	Food and Druga SA	Zamarripa et al., 2020; Zamarripa et al., in prep
0342	Yes	No	No	Male (N/A)	Behavioral Observation	Huskinson et al., 2020
321–2009	Yes	No	No	Male (N/A)	Cocaine and BZ SA	Huskinson et al., 2019
1010	No	Yes	No	Male (N/A)	Food and Druga SA	Zamarripa et al., 2020; Zamarripa et al., in prep

Abbreviations: SA = self-administration; BZ = benzodiazepine

^aDrug experience included cocaine, benzodiazepines, and/or mu- and kappa-opioids

All subjects were housed individually and were given unlimited access to water. Food was mildly restricted in consultation with veterinary staff to maintain subjects at stable body weights by food provided during the session (1-g BioServ flavored pellets) as well as supplemental feeding (Teklad 25% Monkey Diet, Harlan/Teklad, Madison, WI). Supplemental feeding always occurred at least 30 minutes after the choice session ended. Female subject weights ranged from 5.9–8.6 kg and male subject weights ranged from 10.5–13.9 kg at the beginning of the experiment. Fresh fruit or vegetables and forage material (e.g., dried fruit and nuts) were provided daily, and a multivitamin was given three times per week, and all subjects were part of an environmental enrichment program that included access to radio, TV, toys, puzzles, and foraging devices. One female (71–2006) and one male (69–2012) completed the experiment in older caging (1.0 m³, Plaslabs, Lansing, MI) that required animals to be fit with a mesh jacket and tether (Lomir Biomedical, Malone, NY), even in conditions that did not require a catheter (food vs. food). This caging, equipment, and computer software have been described in detail previously (e.g., Huskinson et al., 2015a, 2017). All other subjects were housed in stainless steel enrichment-style cages (each unit: 0.76 m × 0.76 m × 0.86 m; Carter2 Systems, Inc., Beaverton, OR) that were changed every two weeks and did not require a jacket or tether for food vs. food conditions (though some subjects completed food conditions with a jacket, tether, and catheter). In drug conditions, subjects always were fit with a mesh jacket and tether, and the catheter attached to a single- or double-lumen swivel (Lomir Biomedical, Inc., Malone, NY) that attached to a custom-designed operant panel (Carter2 Systems, Inc.) mounted on each home cage. Each operant panel contained two response levers (Med Associates, Inc.) with stimulus lights above each lever and a food receptacle between each lever. A 15-rpm syringe pump (flow rate=0.18 ml/s), feeder that delivered 1-g Bio Serv flavored pellets, and a Med Associates connection panel were housed on the outside of each operant panel. PC computers with Med Associates interfaces and software were used to control experimental events and record data. Lights were maintained on a 14/10-h light/dark cycle, with lights on at 0600 h.

Surgery

For drug conditions, subjects had a catheter surgically implanted using aseptic technique similar to that described previously (Huskinson et al., 2017, 2019). All monkeys were given atropine (0.04 mg/kg, i.m.) and ketamine (5–20 mg/kg, i.m.), followed by inhaled isoflurane and preoperative antibiotics (usually cefazoline; 20–25 mg/kg, i.m.) and analgesics (carprofen, 2–4 mg/kg, s.c. and/or buprenorphine SR, 0.05 mg/kg, s.c.). When anesthesia was established, a single- or double-lumen silicon or polyvinyl chloride catheter was implanted into a major vein with the tip terminating near the right atrium. The distal end of the catheter was passed subcutaneously to the mid-scapular region, where it exited the subject’s back, threaded through the tether, and connected to the swivel. Sensorcaine (0.25%) was locally applied to incision sites prior to suturing. Postoperative antibiotics (usually Keflex, 22.2 mg/kg, p.o. twice daily or i.m. once daily) were given when recommended by veterinary staff. Postoperative analgesics (carprofen, 4 mg/kg, p.o. or s.c.) were given for 3 days following surgery. Additional analgesics were given when recommended by veterinary staff. If a catheter became nonfunctional, it was removed, and the subject was removed from the condition for at least 1 week and until health was verified by veterinary staff, after which a new catheter was implanted. The catheter was filled with 40–100 U/ml heparinized-saline between sessions to prevent clotting.

Procedure

We used a discrete-trials choice procedure similar to that described previously (Huskinson et al., 2015b, 2017). Except on cage-change days and some holidays, sessions were conducted daily, beginning at approximately 9:30 am. Sessions consisted of three sample and 15 or 24 choice trials (described below), there were no limited-holds in effect, and sessions ended when all trials were completed. For cocaine conditions with larger doses (0.056–0.18 mg/kg/injection), only 15 choice trials were conducted per session to limit total cocaine intake, and for cocaine conditions with smaller doses (0.01–0.03 mg/kg/injection) and all food conditions, 24 choice trials were conducted. During sample trials, one lever was active, signaled by illumination of the corresponding white light, and the consequences associated with the active lever were available. The active lever was randomly determined at the start of each session and alternated thereafter. Sample trials were used to ensure exposure to the contingencies associated with each lever and were followed by choice trials, during which the white lights above both levers were illuminated, and the consequences associated with both levers were available. A response on either lever made the other lever inactive and engaged the requirement on the active lever. This feature has been used by others examining choice between VR and FR schedules of food (e.g., Johnson et al., 2011; 2012) and cocaine delivery (Huskinson et al., 2017) and prevents switching between levers once an initial choice has been made. The outcome associated with the chosen lever was delivered after completion of the response requirement. During reinforcer delivery, the white stimulus light darkened, the red stimulus light above the associated lever illuminated for 10 s, and cocaine or food associated with the lever that was pressed was delivered. Trials were separated by five-min timeouts, during which all stimulus lights were darkened and lever presses were recorded but had no programmed consequences.

All conditions were in effect at least three sessions and until choice was stable. Stability required: 1) completion of all trials, 2) choice of one lever to be within 20% of the mean for three consecutive sessions, and 3) no upward/downward trends for 3 consecutive sessions. There were two exceptions to these stability criteria: 1) a trend was acceptable if a ceiling or floor effect was obtained (e.g., 2, 1, and 0 choices), and 2) if stability criteria were met at indifference (40–60% choice of one option), at least one more session was conducted to ensure subjects were not in the process of switching from one lever to the other. After this additional session, if stability criteria were met, the condition was ended. If the additional session revealed a trend, the condition was continued until stability criteria were again satisfied, and the indifference criterion was not applied more than once per injection-lever or food-lever pairing. Once stable, the schedule and reinforcer associated with each lever were reversed, and choice was re-determined.

Fixed-Control Conditions.

Food (4 pellets/delivery) vs. food (4 pellets/delivery), cocaine (0.018 or 0.03 mg/kg/injection) vs. cocaine (0.018 or 0.03 mg/kg/injection), or cocaine (0.01–0.18 mg/kg/injection) vs. food (4 pellets/delivery) choice was examined using the general procedure described above when food or cocaine were available under concurrent and identical FR 100 or 200 schedules. During these conditions, a fixed number of responses (100 or 200, depending on condition) were required to receive the reinforcer associated with either the left or right lever. This condition served as a baseline against which variable conditions could be compared. For one female (71–2006) and one male (69–2012), the cocaine dose was 0.018 mg/kg/injection in the cocaine vs. cocaine conditions. These also were the subjects that were housed in different caging, and 0.018 mg/kg/injection was in the middle of their food vs. cocaine dose-response functions (see top two panels of Figures 3 and 4). When these experiments were conducted in newer caging, the food vs. cocaine dose-response functions were slightly shifted to the right (i.e., cocaine was less potent), and a larger dose (0.03 mg/kg/injection) was used for the cocaine vs. cocaine conditions for subjects in newer caging.

Variable-Schedule Conditions.

These conditions were identical to the Fixed-Control conditions except that the option on the test lever was changed to a VR 100 or 200 schedule that was equal, on average, to the concurrently available FR schedule. Three values constituted the VR requirement. In the VR 100 condition, the values were 1, 100, and 199, and in the VR 200 condition, the values were 1, 200, and 399. These values were selected to allow for small, equal, or large requirements on the VR relative to the constant FR value and this method is similar to procedures used in previous choice experiments (e.g., Johnson et al., 2011; Huskinson et al., 2017). During sample trials, each value was presented on the VR lever once, in a random order for each session. During choice trials, if the alternative associated with the VR lever (test lever) was chosen, values were presented pseudo-randomly, using an order that ensured the same value could not be presented on more than 4 trials in a row, and if the lever associated with the VR schedule was chosen on all trials, all values were presented the same number of times.

The FR/VR 100 or 200 conditions in the food vs. food, cocaine vs. cocaine, and cocaine vs. food conditions occurred in an irregular order across subjects. For food vs. food and cocaine vs. cocaine conditions, the Fixed Control was always conducted first, followed by the Variable-Schedule condition. For the cocaine vs. food conditions, the goal was to individually determine at least three doses for each subject that resulted in complete dose-response functions under Fixed-Control conditions. This also resulted in different doses for different subjects, and to allow for group comparisons, doses were categorized as low, middle, and high for each subject. In separate cocaine vs. food conditions, cocaine or food were made available under a VR schedule in an ABA design (i.e., Fixed Control, Variable Cocaine, Fixed Control or Fixed Control, Variable Food, Fixed Control). If, however, there was no effect of changing the schedule type, the final Fixed Control was periodically omitted.

Data Analysis

For all group analyses, the primary dependent measure was mean percent choice of the test lever for food vs. food and cocaine vs. cocaine or mean percent cocaine choice for food vs. cocaine. Secondary dependent measures included obtained VR values and sessions to stability. Total session time is not reported because there were no statistically significant differences between FR/VR conditions or females vs. males. The means for each subject for each dependent measure (except sessions to stability) was determined from the final three sessions from the initial-lever pairing and the final three sessions from the reversal. For the number of sessions to stability, the average of the initial-injection pairing and the reversal was used for each subject.

For each food vs. food and cocaine vs. cocaine condition at each response requirement (100 or 200), a separate two-way repeated-measures analysis of variance (ANOVA) was conducted with Fixed Control vs. Variable Food or Variable Cocaine as a within-subject factor and females vs. males as a between-subject factor. Bonferroni’s multiple comparisons tests were conducted to determine differences between the fixed and variable conditions within each sex and for males and females combined. For the food vs. cocaine conditions, 95% confidence intervals were used to make conclusions about percent cocaine choice for individual-subject data. For the group analyses, a separate repeated-measures mixed-effects model was conducted (GraphPad Prism 9©) for each response requirement (100 and 200) and for females alone, males alone, and females and males combined with cocaine dose (low, middle, high) and condition (Fixed Control, Variable Food, Variable Cocaine) as within-subjects variables. A mixed-effects model was used rather than a repeated-measures ANOVA, because there were missing values in each of the tests. Bonferroni’s multiple comparisons tests were conducted to determine differences between each condition at within each dose. Results were considered statistically significant at p<0.05. The hypotheses, design, and data reported in the current manuscript were not preregistered and data are available from the corresponding author upon request.

Drugs

Cocaine hydrochloride was provided by the National Institute on Drug Abuse (Rockville, MD). Final solutions were prepared using 0.9% saline. Doses are expressed as the salt form.

Results

Figure 1 represents data from the FR/VR 100 (panel A) or 200 (panel B) food vs. food conditions. Percent test-lever choice is shown for Fixed-Control (open symbols and bars) and Variable-Food (blue or shaded symbols and bars) conditions for the group mean as well as individual-subject data and for females, males, and all subjects combined. In Fixed-Control conditions for FR/VR 100 and 200, when both options were identical, average choice was approximately indifferent. However, comparable distribution of behavior between the levers was not the actual result. Rather, subjects tended to develop a lever bias in which the test lever was chosen nearly exclusively during one lever pairing, and the fixed lever was chosen nearly exclusively during the other lever pairing. Thus, “indifference” represented here is the result of averaging from the extremes. We have found that a biased pattern of behavior is typical under identical FR schedules when choice is between identical reinforcers or between different reinforcers (e.g., food and drug) that are subjectively equivalent (Freeman et al., 2014; Huskinson et al., 2015b, 2016, 2017, 2019; Woolverton, 2003) and is consistent with predictions of the matching law (Baum, 1974; Herrnstein, 1961), because exclusive choice of one option maximizes reinforcement, as does allocating choice among both options. Importantly, the test lever was held constant between Fixed and Variable conditions so that a lever bias had an equal impact on choice across conditions. A biased pattern also was the case for Fixed-Control conditions for cocaine vs. cocaine choice described below and was sometimes the case in food vs. drug conditions when approximately 50% choice was obtained for individual subjects.

Figure 1.

Mean percent test-lever choice for females, males, and both groups combined during food (4 pellets/delivery) vs food (4 pellets/delivery) choice conditions. Data points represent the means for individual subjects from the final 3 stable sessions for each lever pairing and its reversal, and bars represent the group means. Open bars and circles represent the Fixed-Control conditions, and blue or shaded bars and circles represent the Variable-Food conditions when responding was maintained under an FR/VR 100 (Panel A) or an FR/VR 200 (Panel B). Asterisks indicate significant differences at p<0.05.

For females in the FR/VR 100 conditions (Figure 1, panel A), mean percent test-lever choice in the Variable-Food condition was not significantly different from test-lever choice in the Fixed-Control condition (p>0.05). Conversely, mean percent test-lever choice in the Variable-Food condition was significantly increased in males under the FR/VR 100 requirement compared with the respective Fixed-Control condition (p<0.05). For females and males combined, mean percent test-lever choice was significantly increased in the Variable-Food condition compared with the Fixed-Control condition [Combined; F(1,6)=10.2, p<0.05]. Because the VR values were presented in a pseudo-random order, it was possible for obtained VR values to differ somewhat from the programed VR value (i.e., 100). In the FR/VR 100 Variable-Food condition, the average obtained VR value across subjects was 99.6 (subject range=94.9–102.0). In addition, there were no significant differences in the average number of sessions to reach stability between the Fixed and Variable conditions or between females or males [Fixed Control M=5.8, subject range=3–9; Variable Food M=6.8, subject range=4–10].

When the average response requirement was 200 (Figure 1, panel B), females’ mean percent test-lever choice also did not differ between Fixed Control and Variable Food (p>0.05), and for males, was significantly increased in the Variable-Food condition compared with Fixed Control (p<0.05). For females and males combined, mean percent test-lever choice was significantly greater in the Variable-Food condition compared with Fixed Control [Combined; F(1,6)=8.9, p<0.05]. In addition, there was a main effect of sex [F(1,6)=34.3, p<0.05], and there was a significant interaction [F(1,6)=6.3, p<0.05]. In the FR/VR 200 Variable-Food condition, the average obtained VR value across subjects was 200.1 (subject range=195.2–208.1). The average number of sessions to reach stability was significantly greater in the Variable-Food condition compared with Fixed Control [F(1,6)=9.19, p<0.05; Fixed Control M=4.3, subject range=3–7; Variable Food M=6.8, subject range=4–9], and there were no sex differences.

Figure 2 represents data from the FR/VR 100 (panel A) or 200 (panel B) cocaine vs. cocaine conditions. Percent choice of the test lever is shown for Fixed Control (open symbols and bars) and Variable Cocaine (red or shaded symbols and bars) for the group mean as well as individual-subject data and for females, males, and all subjects combined. The black symbols represent the female (71–2006) and male (69–2012) subjects that experienced a smaller dose of cocaine (0.018 mg/kg/injection). All other subjects experienced these conditions with 0.03 mg/kg/injection. When the response requirement was 100 (Figure 2, panel A), mean percent test-lever choice for females was not significantly different in the Variable-Cocaine condition compared with Fixed Control (p=0.09). For males, mean percent test-lever choice was significantly increased in the Variable-Cocaine condition compared with Fixed Control (p<0.05). For females and males combined, mean percent test-lever choice was significantly greater under Variable Cocaine compared with Fixed Control [Combined; F(1,5)=30.3, p<0.05]. The average obtained VR value across subjects was 101.0 (subject range=98.5–104.1). There were no significant differences in the average number of sessions to reach stability between the Fixed and Variable conditions or between females or males [Fixed Control M=6.6, subject range=4–10; Variable Cocaine M=6.1, subject range=4–7].

Figure 2.

Mean percent test-lever choice for females, males, and both groups combined during cocaine (0.018 or 0.03 mg/kg/injection) vs cocaine (0.018 or 0.03 mg/kg/injection) choice conditions. Data points represent the means for individual subjects from the final 3 stable sessions for each lever pairing and its reversal, and bars represent the group means. The solid data points represent the one female (71–2006) and one male (69–2012) that experienced these conditions with a smaller dose of cocaine (0.018 mg/kg/injection). Open bars and circles represent the Fixed-Control conditions, and red or shaded bars and circles represent the Variable-Cocaine conditions when responding was maintained under an FR/VR 100 (Panel A) or an FR/VR 200 (Panel B). Asterisks indicate significant differences at p<0.05.

When the response requirement was 200 (Figure 2, panel B), mean percent test-lever choice for females and males was significantly greater in the Variable-Cocaine condition compared with Fixed Control (p’s<0.05). Similarly, for females and males combined, mean percent test-lever choice also was significantly increased in the Variable-Cocaine condition compared with Fixed Control [Combined; F(1,4)=116.4, p<0.05]. The average obtained VR value across subjects was 199.2 (subject range=193.7–201.4). There were no significant differences in the average number of sessions to reach stability between the Fixed and Variable conditions or between females or males [Fixed Control M=4.9, subject range=3–12; Variable Cocaine M=6.1, subject range=4–8].

Figure 3 represents data from the FR/VR 100 food vs. cocaine conditions for each female (left column) and male (right column) subject. Percent cocaine choice is shown for Fixed Control (open circles), Variable Cocaine (red or shaded triangles), and Variable Food (blue or shaded squares). One male (subject 160–2008) experienced four doses, and because the largest dose tested (0.1 mg/kg/injection) was similar to 0.056 mg/kg/injection, the 0.056 mg/kg/injection dose was considered high, 0.03 mg/kg/injection was middle, and 0.018 mg/kg/injection was low for the group analyses (see Figure 5). Across subjects and conditions, cocaine choice at each subject’s lowest and highest doses were not different across conditions, indicated by overlapping 95% confidence intervals in all but one case (subject 64–2012, Variable Food, lowest dose). However, changes in cocaine choice were observed for some subjects at the middle cocaine dose (note: for female subject 425–2003, a middle dose was not obtained because changing the cocaine dose by one quarter log resulted in a shift from exclusive food choice to exclusive cocaine choice). For three of four females (subjects 71–2006, 40852, 42286) and one of four males (1356), cocaine choice at the middle dose was increased in the Variable-Cocaine condition compared with Fixed Control, indicated by nonoverlapping 95% confidence intervals. Conversely, under the Variable-Food condition, cocaine choice was not reduced for any subject compared with Fixed Control. In fact, some subjects showed a trend toward the opposite effect, and cocaine choice was increased compared with Fixed Control for one female (subject 42286), indicated by nonoverlapping 95% confidence intervals.

Figure 3.

Mean percent cocaine choice for individual subjects (females, left column; males, right column) during FR/VR 100, food (4 pellets/delivery) vs cocaine (0.01–0.18 mg/kg/injection) choice conditions. Data points represent the means for individual subjects from the final 3 stable sessions for each lever pairing and its reversal. Open circles represent average choice from all Fixed-Control conditions, red or shaded triangles represent average choice from the Variable-Cocaine conditions, and blue or shaded squares represent average choice from the Variable-Food conditions. Error bars represent 95% confidence intervals, and the numbers on individual graphs are subject IDs.

Figure 4 represents data from the FR/VR 200 food vs. cocaine conditions and is arranged in an identical manner as Figure 3. As in Figure 3, cocaine choice did not differ across conditions for the highest doses tested. However, there were some instances at the lowest and middle doses where cocaine choice was different across conditions. For three of four females and all four males, cocaine choice generally was increased under Variable Cocaine compared with Fixed Control, but nonoverlapping 95% confidence intervals were obtained in only one female (subject 71–2006) and one male (subject 64–2012) at the middle dose. For two males (subjects 1356, 160–2008), the increase in cocaine choice under the Variable-Cocaine condition also occurred at the lowest dose tested. For two of four females and all four males, cocaine choice generally was decreased under Variable Food compared with Fixed Control, and nonoverlapping 95% confidence intervals were obtained in two females (subjects 425–2003, 42286) and one male (subject 69–20012) at the middle dose, and in one male (subject 1356) at the lowest dose.

Figure 4.

Mean percent cocaine choice for individual subjects (females, left column; males, right column) during FR/VR 200, food (4 pellets/delivery) vs cocaine (0.01–0.18 mg/kg/injection) choice conditions. Data points represent the means for individual subjects from the final 3 stable sessions for each lever pairing and its reversal. Open circles represent average choice from all Fixed-Control conditions, red or shaded triangles represent average choice from the Variable-Cocaine conditions, and blue or shaded squares represent average choice from the Variable-Food conditions. Error bars represent 95% confidence intervals, and the numbers on individual graphs are subject IDs.

As described in the Methods in food vs. cocaine conditions, cocaine doses were determined on an individual-subject basis in an attempt to obtain a full range of cocaine choice. To evaluate effects of making cocaine or food available under a VR schedule at the group level, group means were determined by designating individual-subject doses as low, middle, or high. Figure 5 shows the group means for females (left column), males (middle column), and females and males combined (right column) under FR/VR 100 (top row) and 200 (bottom row). Symbols are identical to Figures 3 and 4. For females (Figure 5, left column) at both response requirements, cocaine dose-effect functions were orderly in that cocaine choice was low at low cocaine doses and increased as the dose was raised, indicated by significant main effects of cocaine dose [FR/VR 100, F(2,6)=51.3, p<0.05; FR/VR 200, F(2,6)=32.9, p<0.05]. In the Variable-Cocaine condition, cocaine choice for females was significantly increased at the middle dose under FR/VR 100 compared with Fixed Control and Variable Food (p’s<0.05) and was significantly increased at the middle dose under FR/VR 200 compared with Variable Food but not compared with Fixed Control. In the Variable-Food condition for females, cocaine choice was significantly reduced at the middle dose under FR/VR 200 (p<0.05) but not FR/VR 100 (p=0.96) compared with the middle dose under the respective Fixed Control [overall analysis: main effect of condition FR/VR 100, F(2,6)=9.3, p<0.05; FR/VR 200, F(2,6)=4.0, p=0.08; dose X condition interaction FR/VR 100, F(4,8)=5.5, p<0.05; FR/VR 200, F(4,6)=2.7, p=0.13].

Figure 5.

Group mean percent cocaine choice for females (left column), males (center column), and females and males combined (right column) during FR/VR 100 (top row) or 200 (bottom row) when choice was between food (4 pellets/delivery) and cocaine (0.01–0.18 mg/kg/injection). Cocaine dose is shown as low, middle, and high so that group averages could be plotted despite individual differences in cocaine dose ranges. Open circles represent the Fixed-Control conditions, red or shaded triangles represent the Variable-Cocaine conditions, and blue or shaded squares represent the Variable-Food conditions. Error bars represent +/− one standard error of the mean, * indicate significant differences compared to Fixed Control at p<0.05, and # indicate significant differences between the Variable-Cocaine and Variable-Food conditions.

For males (Figure 5, middle column) at both response requirements, cocaine dose-effect functions also were orderly at the group level, indicated by significant main effects of cocaine dose [FR/VR 100, F(2,6)=45.5, p<0.05; FR/VR 200, F(2,6)=36.8, p<0.05]. In the Variable-Cocaine condition, cocaine choice was significantly increased at the middle dose under FR/VR 100 compared with Fixed Control (p<0.05). No other significant differences were obtained for males under FR/VR 100 [overall analysis: main effect of condition, F(2,6)=4.4, p=0.07; dose X condition interaction, F(4,10)=1.5, p=0.30]. In the FR/VR 200 conditions, cocaine choice was increased at both the low and middle doses under Variable Cocaine compared with Variable Food (p’s<0.05) but not compared with Fixed Control [main effect of condition, FR/VR 200, F(2,6)=8.7, p<0.05; dose X condition interaction F(4,11)=0.8, p=0.5].

For females and males combined (Figure 5, right column) at both FR/VR 100 and 200 requirements, cocaine dose-effect functions were orderly, indicated by significant main effects of cocaine dose [FR/VR 100, F(2,14)=9.3, p<0.05; FR/VR 200, F(2,6)=76.8, p<0.05]. In addition, in the Variable-Cocaine condition, under both FR/VR 100 and 200, cocaine choice was significantly increased at the middle dose compared with Fixed Control and compared with Variable Food (p’s<0.05), and in the Variable-Food condition, cocaine choice was not significantly reduced in the FR/VR 100 conditions (p>0.05). However, cocaine choice was significantly reduced at the middle dose compared with Fixed Control and Variable Cocaine under FR/VR 200 (p’s<0.05) [overall analyses: main effect of condition FR/VR 100, F(2,14)=107.2, p<0.05; FR/VR 200, F(2,14)=12.2, p<0.05; dose X condition interaction FR/VR 100, F(4,22)=4.1 p<0.05; FR/VR 200, F(4,21)=3.5, p<0.05].

The average obtained VR value in Variable-Cocaine conditions for FR/VR 100 was 101.8 (subject range=89.0–116.9) and for FR/VR 200 was 193.8 (subject range=177.8–206.5). The average obtained VR value in Variable-Food conditions for FR/VR 100 was 105.6 (subject range=95.2–119.3) and for FR/VR 200 was 195.9 (subject range=179.89–210.5). Obtained VR values in food vs. cocaine conditions were not as close to the programmed VR values as was the case in the food vs. food and cocaine vs. cocaine conditions. This was driven largely by conditions in which the Variable Cocaine or Variable Food option were chosen less frequently (i.e., low- and high-dose conditions) compared with conditions in which the variable option was chosen more frequently (i.e., middle-dose conditions). Fewer VR trials resulted in skewed means, and we have seen this before (Huskinson et al., 2017). However, these values are still close to the programmed VR values. There were no significant differences in the average number of sessions to reach stability between the Fixed and Variable conditions or between females or males for FR/VR 100 or 200 conditions. There was, however, a significant main effect of dose in the FR/VR 100 condition [F(2,14)=6.9, p<0.05], and the number of sessions to reach stability was significantly longer in the FR/VR 100 conditions that occurred with the middle dose compared with the high dose (p<0.05). Collapsed across FR/VR 100 conditions, the subject average for the low dose was 4.7 sessions (subject range=3–10), for the middle dose was 6.0 sessions (subject range=5–9), and for the large dose was 4.1 (subject range=3–6). There were no significant differences between doses in the FR/VR 200 conditions, though there was a trend toward significantly more sessions to stability for the middle compared with the high dose [main effect of dose: F(2,14)=3.2, p=0.07; post hoc: p=0.07]. Collapsed across FR/VR 200 conditions, the subject average for the low dose was 4.6 sessions (subject range=3–10), for the middle dose was 6.8 sessions (subject range=4–10), and for the large dose was 5.6 (subject range=3–14).

Discussion

In the present set of experiments, we evaluated the ability of a VR schedule of reinforcement to shift choice away from reinforcers associated with an equal (on average) FR schedule of reinforcement and toward those associated with the VR schedule. To our knowledge, this is the only evaluation of drug vs. nondrug choice with FR and VR schedules. The most critical finding was that cocaine’s potency to maintain choice increased in the Variable-Cocaine condition when a nondrug reinforcer (food) was the alternative available under an FR schedule. The translational implication of these results is that variability in terms of the time and effort required to obtain drugs may result in greater or excessive allocation of behavior toward drug seeking and using at the expense of engaging in the acquisition of more predictable, nondrug alternatives. Conversely, making food available under a VR schedule (Variable Food) while cocaine remained under an FR schedule decreased the potency of cocaine to maintain choice at the group level under the larger response requirement but not the smaller requirement. From a clinical perspective, this outcome suggests that variability in terms of cost of nondrug reinforcers would be an effective means of reducing cocaine use, particularly if the schedule of nondrug reinforcement is a relatively lean one. Indeed, prize-based contingency management varies the probability and magnitude of receiving nondrug vouchers or prizes contingent on submitting drug-negative urine samples, and this method of contingency management is effective in reducing drug use in individuals with SUDs (e.g., Benishek et al., 2014; Olmstead and Petry, 2009; Petry et al., 2005).

Previous research with single-operant arrangements has shown that relatively lean conditions result in less elastic demand or greater behavioral output under a RR schedule compared with an FR schedule (Lagorio and Winger, 2014). In the current experiment, choice of reinforcers associated with the VR schedule was, in some cases, more robust with the larger response requirement compared with the smaller one. This effect occurred for females in the cocaine vs. cocaine conditions, though Variable-Cocaine choice for males was similar across response requirements. In a previous study, male rhesus monkeys chose between FR and VR schedules of cocaine reinforcement when the average response requirement was 30 responses (Huskinson et al. 2017). In that experiment, only two of four males reliably chose cocaine associated with a VR 30 schedule over cocaine associated with an FR 30 schedule, suggesting that with males, the larger requirement of 100 and 200 resulted in more reliable results across subjects compared with an FR or VR 30 schedule. In food vs. cocaine choice in the current experiment, increasing the requirement from 100 to 200 resulted in more robust effects with Variable Food for females at the middle dose and for males at the low and middle doses, and with Variable Cocaine for males at the low and middle doses. Taking the results from the current experiments together with previous research (e.g., Fantino, 1967; Field et al., 1996; Huskinson et al., 2017; Lagorio and Winger, 2014; Madden and Hartman, 2006), one might conclude that more reliable and robust effects of variable response requirements occur with larger average costs and lower drug doses. If translated to individuals with SUDs, this could indicate that uncertain and lean access to drugs is not likely to reduce drug-seeking behavior. In fact, if uncertainty exists, lean access is likely to increase the time and effort allocated to drug-seeking behavior at the expense of engaging in more predictable nondrug alternatives.

In food vs. food conditions, responding associated with FR or VR schedules did not differ across response requirements for females or males. This is somewhat inconsistent with the literature in rodents and pigeons with nondrug reinforcers where choice of reinforcers associated with a variable schedule becomes more extreme with larger response requirements (Fantino, 1967; Field et al., 1996; Madden and Hartman, 2006). Previous experiments included a wider range of requirements than was used in the current experiments, and it is possible that a relation between choice and the size of the response requirement would emerge if smaller and larger average values had been included in the current experiments. For example, in the food vs. food conditions for females, it is possible that a larger response requirement (e.g., 400) would have increased Variable-Food choice. Similarly, the food amount used in the current experiment was relatively large (4 pellets/delivery), and subjects often earned more than half of their ration in the food vs. food conditions. Perhaps a smaller food amount in female subjects, who are much smaller in terms of weight, would have resulted in more reliable effects of changing the test lever from an FR to a VR schedule. Future research exploring a wider range of response requirements and reinforcer magnitudes is necessary to determine whether leaner conditions of reinforcement would consistently result in more extreme allocation of behavior toward the reinforcer associated with a variable schedule in rhesus monkeys with drug and nondrug reinforcers.

Several behavioral and neurobiological mechanisms have been purported to underly effects seen with variable schedules compared with more predictable schedules with drug and nondrug reinforcers. These mechanisms are described in detail in a recent review and will be discussed only briefly here (see Huskinson, 2020). In terms of behavioral mechanisms, reinforcer choice associated with VR schedules is most robust when the smallest possible ratio value is 1, and the effect dissipates as the smallest value approaches that of the FR schedule (e.g., Fantino, 1967; Field et al., 1996). In addition, Madden and colleagues (2007, 2011) proposed that the greater allocation of behavior toward reinforcers associated with variable schedules can be explained within a delay-discounting framework. With ratio schedules, a certain amount of time is required to complete a response requirement, and one can conceptualize the time taken to complete a response requirement as a delay to reinforcer deliver (i.e., a delay from the first response to the final response and ultimate reinforcer delivery). Madden and colleagues argue that greater weight is applied to the sometimes-immediate delivery that can occur under VR schedules, and proportionately less weight is applied to the sometimes-delayed delivery that also occurs under VR schedules. Conversely, the FR option always results in some delay to reinforcement, depending on the time it takes the subject to complete the response requirement. This framework can account for the finding that choice of reinforcers associated with VR schedules is most robust when the smallest possible requirement is 1 as well as the finding that choice becomes more extreme with larger average response requirements. Thus, the excessive allocation of behavior toward reinforcers associated with VR schedules may not result from uncertainty per se, and rather, may result from the sometimes-immediate possibility of obtaining a reinforcer (i.e., when the smallest possible response requirement is 1).

From a neurobiological perspective, a relatively large literature referred to as “Dopamine Prediction Error,” suggests that uncertain reward delivery produces a larger, more sustained dopaminergic response compared to predictable reward deliveries (Fiorillo et al., 2003). The dopaminergic response also can predict subjects’ choices of certain or variable reinforcer amounts (Sugam et al., 2012; see Nasser et al., 2017; Schultz, 2016 for recent reviews on Prediction Error). Similarly, when rats’ behavior was maintained by saccharin delivery, dopamine overflow in the NAc was an increasing function of the average VR requirement and was greater when compared with the FR group. Enhanced dopamine overflow under larger VR schedules is consistent with more robust behavioral results obtained with larger VR schedules. Similarly, D’Souza and Duvauchelle (2008) trained rats with certain or uncertain cocaine and saline cues. During subsequent test sessions, the dopaminergic response to cocaine was larger in the uncertainty-trained animals compared with the certainty-trained animals. A larger, more sustained dopaminergic response could be a mechanism underlying enhanced choice of reinforcers associated with VR schedules in the current experiments, but dopaminergic response was not measured. In addition, some have suggested that uncertain access to nondrug reinforcers or rewards could be sensitizing dopamine neurons in a manner similar to drug exposure (Robinson and Anselme, 2019; Zack et al., 2014).

While choice outcomes in the food vs. cocaine conditions were more robust in female subjects compared with males, the opposite occurred in food vs. food, and to a lesser extent, in cocaine vs. cocaine choice. Because different subjects served in different conditions, it is possible that individual differences are responsible for somewhat different findings across males and females and under food vs. food, cocaine vs. cocaine, and food vs. cocaine conditions. However, discrepancies also have been obtained in choice studies evaluating delay discounting in cocaine vs. food choice compared with cocaine vs. cocaine and food vs. food choice (Huskinson et al., 2015b, 2016; Woolverton et al. 2007). Thus, effects of manipulating variables like delay to reinforcement or variable response costs seem to be context or reinforcer dependent. In other words, outcomes obtained in choice between the same type of reinforcer may not translate to outcomes obtained in choice between different types of reinforcers.

Several other possibilities exist that could help explain the different outcomes for females and males across the different choice arrangements. For example, the dose of cocaine used in cocaine vs. cocaine conditions tended to fall in the middle of the dose-response function for males in cocaine vs. food choice. This was not true for most of the females, whose dose-response functions in food vs. cocaine choice tended to be rightward shifted compared with males. Therefore, the cocaine dose used in cocaine vs. cocaine choice for three of four females was considered a low dose in cocaine vs. food choice. In addition, the food amount in all food conditions was unchanged. It is unknown where this food amount would fall on a food-amount curve or whether food-amount curves would differ for females and males if the alternative cocaine dose was held constant while food amount was adjusted. Similarly, it is unknown the extent to which changing one schedule to a VR would result in more or less robust effects with low-magnitude reinforcers compared with high-magnitude reinforcers in choice scenarios when both reinforcers are identical. Future research should evaluate whether outcomes with fixed and variable schedules vary as a function reinforcer magnitude in food vs. food and cocaine vs. cocaine choice, and multiple food magnitudes should be evaluated in food vs. cocaine choice.

It is possible that the obtained differences between females and males and between choice scenarios reflect individual variation in choice. However, it is of course feasible that our choice results represent sex differences in reinforcement processes. The current studies were not sufficiently powered to detect reliable sex differences, and systematic replications of the differences obtained between females and males in the current study are warranted. Nevertheless, it is worth noting that in choice conditions between the same type of reinforcer (i.e., food vs. food and cocaine vs. cocaine), males tended to show a more reliable, and in the case of food vs. food, a greater allocation of behavior toward the reinforcer associated with a variable schedule compared with females. Conversely, in food vs. cocaine choice, females showed a more reliable and more robust effect with Variable Cocaine compared with males, suggesting a more vulnerable phenotype in the context of drug vs. nondrug choice in females. A second potential sex difference emerged in the food vs. cocaine conditions in that females’ dose-response functions tended to be rightward shifted compared with males. Most research evaluating sex differences in drug self-administration has been done with rodents and with single-operant procedures (see Anker and Carroll, 2010; Becker and Koob, 2016; Carroll and Lynch, 2016; Kerstetter and Kippin, 2011 for recent reviews). Much less is known about sex or cycle effects on drug vs nondrug choice. In one study, oral cocaine choice and intake was greater during the follicular but not the luteal phase in female rhesus monkeys compared with males when the nondrug alternative was water (Carroll et al., 2016). When choice was between oral cocaine and a saccharin solution, there were no sex or cycle effects on cocaine choice or intake (Carroll et al., 2016). Similar to our results, in fentanyl vs. diluted ensure® choice, the fentanyl dose-response function was leftward shifted in male rats compared with females when the ensure® concentration was relatively small (18%; Townsend et al., 2019). However, sex differences were not obtained in a latter study with the same concentration (Townsend, 2021) or when the ensure® concentration was relatively large (56%; Townsend et al. 2019). Additional research is clearly needed to determine the reliability of sex or cycle effects in drug vs. nondrug choice.

Finally, it is important to highlight some limitations of the current experiments that were not addressed above. The procedure used was rigorous in multiple respects. Conducting sessions until choice was stable, completing lever reversals, and returning to a Fixed-Control baseline between each Variable-Cocaine or Variable-Food condition in food vs. cocaine choice are all aspects of the procedure that boost confidence in the reliability of the obtained data. However, including all of these aspects also resulted in a very lengthy study. In previous choice studies, we have seen that choice can fluctuate over time, though not necessarily in a systematic way (e.g., Huskinson et al., 2015b, 2016, 2017). We attempted to reduce the influence of a subject’s most recent experience by always preceding a variable condition with a fixed one and by conducting conditions in a counterbalanced order across subjects. There is some evidence that exposure to a VR schedule can enhance the reinforcing properties of drugs (Mascia et al., 2019), and one might suspect a leftward shift in the cocaine dose-response function over time as a result of experience with VR schedules. However, changes that occurred over time were not systematic or consistent across subjects. Another limitation is that we evaluated only one dimension of uncertainty (i.e., response cost), modeling the time and effort required to obtain food and cocaine. Other dimensions like uncertain drug quality could be modeled. Finally, we are working on ways to reduce the amount of time needed to conduct experiments like the current one.

Public Significance Statement:

Illicit drugs like cocaine may be uncertain in terms of their availability, quality, and time and effort required to obtain them compared to other important drivers of behavior, such as a paycheck. Uncertain response requirements (modeling time and effort) may be an important contributing factor in the excessive allocation of behavior toward drug seeking and drug taking at the expense of engaging in nondrug related activities