Disruptions in Reward-Guided Decision-Making Functions Are Predictive of Greater Oral Oxycodone Self-Administration in Male and Female Rats

Abstract

Background

Problematic opioid use that emerges in a subset of individuals may be due to preexisting disruptions in the biobehavioral mechanisms that regulate drug use. The identity of these mechanisms is not known, but emerging evidence suggests that suboptimal decision making that is observable prior to drug use may contribute to the pathology of addiction.

Methods

The current study investigated the relationship between decision-making phenotypes and opioid-taking behaviors in male and female Long Evans rats. Adaptive decision-making processes were assessed using a probabilistic reversal learning task and oxycodone- (or vehicle, as a control) taking behaviors assessed daily in 32 sessions using a saccharin fading procedure that promoted dynamic intake of oxycodone. Tests of motivation, extinction, and reinstatement were also performed.

Results

Computational analyses of decision-making data identified data-driven metrics that predicted self-administration of oxycodone and addiction-relevant behaviors. Moreover, preexisting impairments in reward-guided decision making observed in female rats were associated with greater addiction-relevant behaviors when compared with males.

Conclusions

These results provide new insights into the biobehavioral mechanisms that regulate opiate-taking behaviors and offer a novel phenotypic approach for interrogating sex differences in addiction susceptibility and opioid use disorders.

Plain Language Summary

Only a small subset of individuals who are exposed to opiates such as oxycodone will develop an opioid use disorder. However, identification of these at-risk individuals prior to drug exposure has not been possible.

We hypothesized that decision making could be a predictor of problematic oxycodone use. To test this hypothesis, decision-making and oral oxycodone self-administration behaviors were assessed in adult rats. Deficits in reward-guided decision making were found to predict oxycodone use and explain the problematic oxycodone use that we observed in female rats. These data indicate that decision-making phenotypes can serve as a biomarker of opiate risk in drug-naïve individuals.

Plain Language Summary

Only a small subset of individuals who are exposed to opiates such as oxycodone will develop an opioid use disorder. However, identification of these at-risk individuals prior to drug exposure has not been possible.

We hypothesized that decision making could be a predictor of problematic oxycodone use. To test this hypothesis, decision-making and oral oxycodone self-administration behaviors were assessed in adult rats. Deficits in reward-guided decision making were found to predict oxycodone use and explain the problematic oxycodone use that we observed in female rats. These data indicate that decision-making phenotypes can serve as a biomarker of opiate risk in drug-naïve individuals.

It has been proposed that the emergence and persistence of problematic opioid-taking behaviors is driven by preexisting differences in the neurobiological mechanisms that regulate drug use (1,2). The identity of these mechanisms is not known, in part because of the challenge of dissociating initial addiction susceptibility mechanisms from the robust neural adaptations that occur in response to drug exposure. Evidence suggests that intrinsic differences in decision making could be a useful behavioral biomarker for delineating the biological mechanisms that mediate addiction susceptibility from those that are consequent to drug use (3,4). Specifically, we found that deficits in reward-guided decision making prior to any drug exposure were predictive of greater cocaine-taking behaviors in rats (5,6) and were associated with neurobiological abnormalities that have been observed in cocaine-dependent individuals (5,7, 8, 9). Decision-making phenotypes could be used to assess addiction risk in individuals prior to any drug exposure, and therefore represent a potentially powerful approach to addressing the ongoing opioid epidemic.

The rapid rise in opioid misuse and abuse has been linked to prescription opioids such as oxycodone (10, 11, 12, 13). Approximately 13% of oxycodone users will abuse the drug, and many heroin users report that their initiation of opiates began following use of oxycodone (14,15). Despite the profound risks associated with oxycodone, it remains one of the most widely prescribed painkillers, and there is significant interest in understanding the neurobiology that leads to abuse and misuse of oxycodone (16, 17, 18, 19). Studies that have investigated the neurobiology of opioid use disorder in rodents have typically been conducted using intravenous delivery of oxycodone (20, 21, 22, 23). However, most individuals begin self-administering oxycodone orally. Given that the time course of action and metabolism of oxycodone varies across routes of administration and likely leads to different clinical features across individuals, ages, and sexes (24,25), modeling oxycodone use via an oral self-administration procedure in rodents could provide more clinically relevant insights into the pathology of opioid use.

Several oral oxycodone self-administration procedures have been developed for use with rodents (24,26,27). Many of these oral procedures lead to patterns of oxycodone use that are different from the ones that have been observed in rodents when intravenous administration has been used (20, 21, 22) and in human populations, including uniformly high rates of escalation and intake of oxycodone (27) in both male and female participants (26) [however, also see (28)]. Given that most oxycodone users do not go on to abuse the drug (29), and there is some evidence that risk for transition from oxycodone use/abuse to dependence (e.g., telescoping) is elevated in women compared with men (30, 31, 32), developing an oral self-administration procedure that is able to model the biologically mediated variation in oxycodone use is critical for understanding the neurobiology of opioid use disorder.

In the current study, we sought to understand the relationships between decision-making functions and oxycodone-taking behaviors in both male and female rats. We hypothesized that deficits in reward-guided decision making prior to any drug exposure would be predictive of greater oxycodone intake and that these biobehavioral relationships would be different in male and female rats.

Methods and Materials

Animals

Male (n = 78) and female (n = 78) Long Evans rats were purchased from Charles River Laboratories at approximately 6 weeks of age. Rats were pair housed in a climate-controlled vivarium on a 10-hour light/dark cycle (lights on at 6 am, lights off at 8 pm). Additional details are provided in the Supplement.

Probabilistic Reversal Learning Task

Decision making was assessed with a 3-choice, spatial probabilistic reversal learning (PRL) task using stochastic reward schedules in operant conditioning chambers (Med Associates) (Figure 1). On each trial, 3 noseport apertures were illuminated, with one of the apertures being associated with a higher probability of delivering a reward (e.g., 45 mg sucrose pellet; BioServ) than the other 2 apertures (Figure 1B). The reinforcement probabilities assigned to each noseport were pseudorandomly assigned at the start of each session for individual rats. Rats could make a single choice on each trial by making a noseport entry into the illuminated port. Rats were trained on a variable schedule of reinforcement for 35 sessions until their performance had stabilized (Figure 1B–E; see the Supplement).

Figure 1.

The PRL task. (A) Schematic of a single-trial event. (B) The probability of reinforcement for the 3 NP apertures under the variable schedule of reinforcement. (C) The number of reversals that rats were able to complete on the variable schedule of reinforcement increased across the 35 training sessions (session: Wald χ²₁ = 825, p < .001). (D) The number of reversals that rats completed during the last 5 sessions under the variable schedule of reinforcement was stable (Cronbach’s α = 0.86). (E) The number of trials that rats required to acquire a discrimination was less than that required to reverse the discrimination (Wald χ²₁ = 1069, p < .001). (F) The probability of reinforcement for the 3 NP apertures under the stable high schedule of reinforcement. (G) The choice behavior of rats on the stable high schedule of reinforcement. (H) The probability of reinforcement for the 3 NP apertures under the stable low schedule of reinforcement. (G) The choice behavior of rats on the stable low schedule of reinforcement. ∗∗∗p < .001. NP, noseport; PRL, probabilistic reversal learning.

Decision-making functions were assessed across 2 additional schedules of reinforcement in which a single reversal occurred when rats had completed 120 trials (Figure 1F, H). Performance of individual rats on the PRL task in the 3 different schedules of reinforcement were highly correlated (Figure S1), so dependent measures were collapsed across the different schedules of reinforcement.

Oral Self-Administration Procedure

Following the decision-making assessments, rats were divided into 2 groups and balanced based on their performance in the PRL task: in one group rats were trained to self-administer a vehicle solution (0.05% saccharin in water; n = 41) whereas rats in the other group self-administered an oxycodone solution (0.05 mg/kg/delivery of oxycodone in 0.05% saccharin solution; n = 115) in 3-hour daily sessions for 32 sessions in a novel operant environment (Figure 2A). On the ninth self-administration session, the dose of oxycodone was doubled (0.05 mg/kg/delivery to 0.1 mg/kg/delivery), and the saccharin was gradually removed from the vehicle solution (Figure 2B) such that water was the vehicle solution for both groups by the 13th self-administration session. These doses were used based on previous studies that had found that operant responding for oral delivery of these doses of oxycodone was greatest in both male and female rats (24,28).

Figure 2.

Oral self-administration of oxycodone in rats. (A) Schematic of the operant environment. (B) Dose of oxycodone and percentage of saccharin in the vehicle solution across the 32 self-administration sessions. (C) The total number of rewards earned by rats in the vehicle (n = 41) and oxycodone (n = 115) conditions decreased across the first 12 sessions (Wald χ²₁ = 51.97, p < .001) but then stabilized from session 13 to the last session (Wald χ²₁ = 1.64, p = .20). Notably, the number of rewards earned was greater in the oxycodone group compared to the vehicle group (Wald χ²₁ = 56.86, p < .001). (D) The amount of oxycodone consumed (mg/kg) across the self-administration sessions. (E) The number of active and inactive responses made under the progressive ratio schedule of reinforcement in the vehicle group (gray bars) and the oxycodone group (red bars) (group: Wald χ²₁ = 50.75, p < .001). (F) The number of active (circles) and inactive (squares) noseport responses that rats in the vehicle (gray bars) and oxycodone (red bars) groups made across the six 1-hour extinction sessions (group: Wald χ²₁ = 9.48, p = .002). (G) The number of active (circles) and inactive (square) noseport responses that rats in the vehicle (gray bars) and oxycodone (red bars) groups made in the 1-hour cue-induced reinstatement test (group: Wald χ²₁ = 71.94, p < .001). (H) Matrix showing the correlation (Pearson’s correlation) between active responses made under PR schedule of reinforcement across the 6 extinction sessions and on the cue-induced reinstatement test. ∗∗p < .01, ∗∗∗p < .001. Ext, extinction; PR, progressive ratio; Reinst, reinstatement.

Addiction-Relevant Behaviors

Additional tests of drug-seeking and drug-taking behaviors were conducted following the self-administration sessions using procedures that have been described previously (33). Motivation to obtain a delivery of oxycodone or vehicle was assessed under a progressive schedule; the ability to inhibit nonreinforced drug-taking behaviors was assessed under extinction; and drug-seeking behavior was assessed in a cue-induced reinstatement test (see the Supplement).

Data Analysis

Reinforcement Learning Model

Choice behavior data collected from rats in the PRL task were analyzed with a differential forgetting reinforcement learning (DFRL) model (34, 35, 36), which captures gradually decaying effects of previous choices and outcomes on choices. This model was found to fit the choice behavior of rats significantly better than other reinforcement learning models (Table S1), as we have shown previously (5,6,37). This reinforcement learning model contains 4 free parameters: a decay rate for the action values of chosen options (γ_C), a decay rate for the action values of unchosen options (γ_U), a parameter for the appetitive strength of rewarded outcomes (Δ₊), and a parameter for the aversive strength of unrewarded outcomes (Δ₀).

Characterizing Latent Drug-Taking Phenotypes

The number of oxycodone rewards that individual rats self-administered during the last 20 self-administration sessions, when the vehicle no longer contained saccharin, were fit with a power function:

$$f(x)=A{x}^B$$ (1)

where x is the session number, and A and B are free parameters estimating the scaling factor and the rate of growth, respectively. The A parameter determines the initial strength of drug-taking behaviors; rats with a higher A parameter self-administered more oxycodone than rats with a lower A parameter across the self-administration procedure (Figure S4). The B parameter determines the growth of the function; the number of oxycodone rewards that rats self-administered escalated in those with a higher, positive B parameter than in rats with a lower, negative B parameter (Figure S4).

Statistical Analyses

Statistical analyses are described in the Supplement.

Results

Decision Making Is Reliable Across Different Schedules of Reinforcement

Long Evans rats (female n = 78, male n = 78) were trained to make flexible choices in a 3-armed bandit PRL task (Figure 1A) using stochastic reward schedules (Figure 1B). The number of reversals that rats completed in a single session improved across the 35 training sessions under the variable schedule of reinforcement (Figure 1C) and stabilized after completion of 36.23 ± 0.49 sessions (Figure 1D). The number of trials that rats required to reach the first performance criterion was significantly less than that required to achieve the second performance criterion (Figure 1E), indicating that despite the extensive training that rats received on the PRL task, they nonetheless took more trials to reverse a discrimination. Rats chose the most frequently reinforced option both before and after the reversal under the stable schedules of reinforcement at a rate significantly greater than chance (Figure S2), demonstrating that rats were able to track dynamic and novel schedules of reinforcement.

Choice behavior of rats was fitted with the 4 different computational algorithms, and the Bayesian information criterion was calculated for each model (Table 1). The DFRL model had the lowest Bayesian information criterion value, and therefore it fit the choice data collected in rats best. The average parameter estimates obtained from the DFRL model are presented in Table 2.

Table 1.

The Total BIC for Each of the 4 Computational Algorithms

	Model 1: Q Learning	Model 2: Forgetting Q Learning	Model 3: Forgetting Reinforcement Learning	Model 4: Differential Forgetting Reinforcement Learning
BIC	3,490,441	3,159,931	3,034,649	2,995,183

The differential forgetting reinforcement learning model (model 4) had the lowest BIC value of all the models, indicating that this model fit the rat choice data best.

BIC, Bayesian information criterion.

Table 2.

Quartile Values of the Parameter Estimates From the Differential Forgetting Reinforcement Learning Model

Quartile	γC	γU	Δ+	Δ0
25th	0.59	0.67	1.08	−0.02
Median	0.65	0.79	1.59	0.21
75th	0.74	0.89	2.12	0.43

γ_C quantifies the decay rate for the action value of the chosen option; γ_U quantifies the decay rate for the action values of unchosen options; Δ₊ quantifies the appetitive strength of rewarded outcomes; and Δ₀ quantifies the aversive strength of no rewarded outcomes.

Oral Self-Administration of Oxycodone Leads to Addiction-Relevant Behaviors

Following the decision-making assessments, rats were trained to orally self-administer either oxycodone (0.05 mg/kg/delivery in 0.05% saccharin solution; n = 115) or vehicle (0.05% saccharin solution; n = 41) in 3-hour daily sessions for 32 sessions (Figure 2A, B). The number of rewards and amount of oxycodone that rats consumed during each of the self-administration sessions is presented in Figure 2C, D. The abrupt increase in the amount of oxycodone that rats consumed during the ninth self-administration session is due to the increase in oxycodone dose from 0.05 mg/kg/delivery to 0.10 mg/kg/delivery.

The number of rewards self-administered decreased during the first 12 sessions in both experimental groups (Figure 2C), which is likely due to the change in reinforcement schedule (e.g., from FR1 schedule [fixed ratio 1] to FR3 schedule) and the removal of saccharin from the vehicle solution. However, the number of rewards that rats earned from the 13th self-administration session to the last session did not change in either group (Figure 2C). The number of rewards earned was greater in the oxycodone group than in the vehicle group, indicating that rats found the oral oxycodone solution to be reinforcing.

As expected, rats in the oxycodone group made more active responses under the progressive ratio schedule of reinforcement (Figure 2E) and during the extinction sessions (Figure 2F) than rats in the vehicle group. The number of active lever responses during the cue-induced reinstatement test increased from that made during the last extinction session in both the oxycodone and vehicle group, but this effect was larger in the oxycodone group (Figure 2G). The small but significant increase in active responses in the vehicle group is likely because this cue had previously been paired with a palatable saccharin solution and therefore had acquired incentive value. These 3 addiction-relevant behaviors were positively correlated with one another (Figure 2H), suggesting that oral oxycodone self-administration led to the emergence of multiple problematic drug-seeking behaviors.

Quantifying Individual Differences in Oxycodone-Taking Behaviors

Although the amount of oxycodone that rats self-administered did not increase during the saccharin-free sessions (Figure 2D), there were individual differences in patterns of oxycodone intake, which were characterized using a power function (Figure 3A–C and Figure S4) (5,38). The A and B parameters explained non-overlapping portions of variance in the total amount of oxycodone that was consumed (R² = 0.92; A parameter: β = 0.93, p < .001; B parameter: β = 0.23, p < .001) and were not related to one another (Figure 3D), indicating that they were explaining distinct patterns of oxycodone use.

Figure 3.

Characterizing latent oxycodone-taking behaviors with the power function. (A) The power function (red lines) was able to capture latent drug-taking behaviors. (B) The frequency distribution of the A parameter and the (C) B parameter estimate. (D) No relationship was detected between the A parameter and the B parameter (R² = 0.0002, p = .88). However, both the A and B parameter estimates explained a significant amount of variance in responding under the PR schedule of reinforcement (A parameter Wald χ²₁ = 109.21, p < .001; B parameter Wald χ²₁ = 47.51, p < .001; A parameter × B parameter interaction: Wald χ²₁ = 1.57, p = .21); during extinction (A parameter Wald χ²₁ = 28.26, p < .001; B parameter Wald χ²₁ = 23.52, p < .001; A parameter × B parameter interaction: Wald χ²₁ = 0.39, p = .53); and on the reinstatement test (A parameter Wald χ²₁ = 8.41, p = .004; B parameter Wald χ²₁ = 37.37, p < .001; A parameter × B parameter interaction: Wald χ²₁ = 0.92, p = .34). (E) Active noseport responses in rats with a low A parameter (n = 58) and a high A parameter (n = 57) in the PR test, extinction, and reinstatement. (F) Active noseport responses in rats with a positive B parameter (n = 42) and a negative B parameter (n = 43) in the PR test, extinction, and reinstatement. ∗p < .05, ∗∗∗p < .001. Ext, extinction; PR, progressive ratio; Reinstate, reinstatement.

Variation in these latent drug-taking behaviors may predict the development of addiction-relevant behaviors. Both the A and B parameter estimates explained a significant amount of variance in responding under the progressive ratio schedule of reinforcement, under extinction, and in the reinstatement test (Figure 3E, F). These data indicate that both the A and B parameters contributed to the emergence of these addiction-relevant behaviors.

Poor Reward-Guided Decision Making Predicts Greater Oxycodone Use

Next, the predictive relationship between decision making and oxycodone self-administration was examined. Performance on the PRL task and self-administration session were entered into a statistical model predicting the number of oxycodone rewards that rats earned during each session. Both session and PRL task performance, but not the interaction between these factors, explained a significant amount of variance in oxycodone-taking behaviors (session: Wald χ²₁ = 175, p < .001; PRL performance: Wald χ²₁ = 7.01, p = .008; session × PRL interaction performance: Wald χ²₁ = 0.61, p = .44) (Figure 4A). Rats with poor PRL performance took significantly more oxycodone than rats with good PRL performance (Figure 4B, C). No relationship was observed in the vehicle group (Figure 4D). Therefore, poor decision making that is predictive of self-administration behaviors was specific to oxycodone.

Figure 4.

Poor reward-guided decision making is predictive of higher oxycodone self-administration. (A) Scatterplot of the relationship between PRL performance and total oxycodone consumed (mg/kg). (B) Frequency distribution of performance on the PRL task. Rats were divided into 2 groups based on a median split of PRL performance: poor PRL (n = 57) and good PRL (n = 58). (C) Rats in the poor PRL group earned significantly more oxycodone rewards than rats in the good PRL group (Wald χ²₁ = 11.05, p < .001); ∗p < .05 for group-level comparisons. (D) The number of vehicle rewards that rats earned did not differ between rats with poor and good PRL performance (group: Wald χ²₁ = 3.06, p = .22; group × session interaction: Wald χ²₁ = 3.20, p = .20). (E) Individual differences in the γ_C were not predictive of oxycodone self-administration. (F) Individual differences in the γ_U parameter were not predictive of oxycodone self-administration. (G) Individual differences in the Δ₊ parameter predicted oxycodone self-administration (R²₁₁₂ = 0.14, p < .001). (H) Individual differences in the Δ₀ parameter did not predict oxycodone self-administration. (I) Rats with a low Δ₊ parameter self-administered greater amounts of oxycodone than rats with a high Δ₊ parameter (Wald χ²₁ = 561, p < .001). (J) Variation in the Δ₊ parameter was correlated with the initial strength of oxycodone reinforcement or the A parameter (R²₁₁₂ = 0.11, p < .001). (K) Variation in the Δ₊ parameter was not correlated with the rate of growth in oxycodone use or the B parameter (R²₁₁₂ = 0.02, p = .13). (L) Rats with a low Δ₊ parameter made more active responses on the PR test (Wald χ²₁ = 1306, p < .001), during extinction sessions (Wald χ²₁ = 366; p < .001), and on the cue-induced reinstatement test (Wald χ²₁ = 32.41, p < .001) than rats with a high Δ₊ parameter. ∗p < .05, ∗∗∗p < .001. Ext, extinction; PR, progressive ratio; PRL, probabilistic reversal learning; Reinstate, reinstatement.

Differences in decision making that are predictive of oxycodone use may be associated with a specific reinforcement learning mechanism. The parameter estimates obtained from the DFRL algorithm were entered into a stepwise regression model predicting the total amount of oxycodone self-administered. The Δ₊ parameter, but not the γ_C, γ_U, or Δ₀ parameters, explained a significant amount of variance in the total amount of oxycodone consumed (R²₁₁₂ = 0.14, p < .001) (Figure 4E–H): Rats with a low Δ₊ parameter took significantly more oxycodone than rats with a high Δ₊ parameter (Figure 4I). The Δ₊ parameter explained a significant amount of variance in the A parameter (Figure 4J) but not in the B parameter (Figure 4K).

Disruptions in reward-guided decision making prior to drug exposure may also facilitate the development of addiction-relevant behaviors. The Δ₊ parameter explained a significant amount of variance in the number of active responses made under the progressive ratio schedule, during the extinction sessions, and on the cue-induced reinstatement test: Rats with a lower Δ₊ parameter made more active responses than rats with a higher Δ₊ parameter (Figure 4L). Collectively, these data indicate that poor reward-guided decision making is associated with heightened oxycodone reinforcement that elevates intake of oxycodone and other addiction-relevant behaviors.

Oxycodone Self-Administration Is Greater in Female Rats Than in Male Rats

Next, self-administration behaviors in male and female rats in the oxycodone and vehicle groups were compared. Post hoc analyses of the sex × session interaction revealed that female rats earned a greater number of oxycodone rewards (Figure 5A) and self-administered more oxycodone (Figure 5B) than male rats, particularly after saccharin had been removed from the vehicle solution. These sex differences were not due to an overall increase in operant responding because the number of inactive responses was similar in males and females (Figure 5C). Furthermore, no sex differences were detected in the vehicle group (Figure 5D). Both the A parameter and B parameter were larger in female rats than in male rats (Figure 5E), indicating that the initial strength of oxycodone reinforcement and the rate of growth in oxycodone use was greater in females than in males. Weight gain during the self-administration sessions was found to be attenuated in the oxycodone group compared with the vehicle group and more so in the female oxycodone rats than in the male oxycodone rats (Figure S3).

Figure 5.

Sex differences in oxycodone-taking behaviors. (A) Female rats earned more oxycodone rewards than male rats (sex × session interaction: Wald χ²₁ = 41.34, p < .001). Post hoc analyses revealed a significant effect of sex from self-administration session 13 until the last session that rats completed. (B) Female rats consumed a greater amount of oxycodone (mg/kg) than male rats and (C) made more active noseport responses than males. No sex differences in inactive responses were observed (right axis; Wald χ²₁ = 2.16, p = .14). (D) The number of vehicle rewards did not differ between males and females (main effect of sex: Wald χ²₁ = 2.74, p = .10; sex × session interaction: Wald χ²₁ = 2.25, p = .13). (E) The A parameter and B parameter were larger in female rats than in male rats (Wald χ²₁ > 4.30, p < .05). (F) Females made more responses under the progressive ratio schedule of reinforcement than males (active responses: Wald χ²₁ = 28.91, p < .001; inactive responses: Wald χ²₁ = 9.51, p = .002), but the difference between response type was greater in females (sex × response type interaction: Wald χ²₁ = 17.45, p < .001). Moreover, females earned more oxycodone rewards than males on the progressive ratio test (Wald χ²₁ = 43.57, p < .001). (G) Active responses were greater in female rats than in males across the six 1-hour extinction sessions (Wald χ²₁ = 16.18, p < .001). (H) Active responses on the cue-induced reinstatement test were greater in females than males (Wald χ²₁ = 31.73, p < .001). ∗p < .05, ∗∗∗p < .001.

The number of active and inactive responses made under the progressive ratio schedule of reinforcement were greater in female rats than male rats, but the difference between the response types was greater in females (Figure 5F). The number of rewards that females earned under the progressive ratio schedule of reinforcement was greater than that earned by males (Figure 5F), indicating that motivation to obtain an oxycodone reinforcer was greater in female rats. The number of active responses across the 6 extinction sessions (Figure 5G) and during the cue-induced reinstatement test (Figure 5H) was greater in females than males. No differences were observed in the vehicle group (Figure S5).

Reward-Guided Decision Making Is Lower in Female Rats

The greater oxycodone-taking behaviors that we observed in female rats may be the result of preexisting disruptions in reward-guided decision making. Female rats completed fewer reversals than male rats under the variable schedule of reinforcement training (Figure 6A) and when performance had stabilized (Figure 6B and Figure S6), suggesting that these differences were not driven by the fact that females required additional training to reach the same level of stability as males. Therefore, female rats had greater difficulty than males allocating their choices to the highest reinforced option.

Figure 6.

Sex differences in decision-making processes. (A) The number of reversals that female (dark orange) and male (light orange) rats completed across the 35 PRL sessions under the variable schedule of reinforcement (sex: Wald χ²₁ = 8.87, p = .003; session: Wald χ²₁ = 596, p < .001; sex × session interaction: Wald χ²₁ = 2.27, p = .13). (B) The number of reversals that female (dark orange) and male (light orange) rats completed once performance had stabilized or rats had completed 50 sessions on the variable PRL task (Wald χ²₁ = 22.78, p < .001). (C) The probability of repeating the same choice (e.g., p[staying] based on whether the previous trial was rewarded [e.g., win] or not [e.g., lose] in female [dark orange] and male [light orange] rats). Outcome type × sex interaction: Wald χ²₁ = 23.30, p < .001; post hoc analyses: p(staying) after a win, sex: Wald χ²₁ = 23.02, p < .001; p(staying) after a loss, sex: Wald χ²₁ = 0.05, p = .82. (D) The average γ_C, γ_U, Δ₊, and Δ₀ parameter estimates in female (dark orange) and male (light orange) rats. Parameter × sex interaction: Wald χ²₁ = 24.50, p < .001; post hoc analyses: γ_C, sex: Wald χ²₁ = 16.42, p < .001; γ_U, sex: Wald χ²₁ = 7.27, p = .007; Δ₊, sex: Wald χ²₁ = 21.51, p < .001; Δ₀, sex: Wald χ²₁ = 0.69, p = .41. (E) Individual differences in the Δ₊ parameter predicted oxycodone intake across female and male rats. Sex is no longer a predictor of oxycodone when the Δ₊ parameter is included in the model (sex: t_2,111 = 0.31, p = .76; Δ₊ parameter: t_2,111 = 3.74, p < .001). (F) Individual differences in the Δ₊ parameter predicted the A parameter in female and male rats. Sex is no longer a predictor of the A parameter when the Δ₊ parameter is included in the model (sex: t_2,111 = 0.37, p = .72; Δ₊ parameter: t_2,111 = 3.25, p = .002). (G) Mediation analysis of sex, the Δ₊ parameter, and initial strength of oxycodone reinforcement (e.g., A parameter). ∗∗p < .01, ∗∗∗p < .001. PRL, probabilistic reversal learning.

These differences in PRL performance between male and female rats might have been driven by impairments in select reinforcement learning mechanisms. Post hoc analyses of the outcome type × sex interaction effect (Figure 6C) indicated that female rats were less likely to persist with a rewarded choice than males. No sex differences were detected in the probability of repeating an unrewarded choice. Poor reversal performance in the female rats was, therefore, likely due to deficits in reward-based choice behavior.

The parameter estimates obtained from the DFRL algorithm in males and females were then compared. Post hoc analyses of the significant sex × parameter interaction indicated that the γ_C and γ_U were larger but the Δ₊ parameter was lower in female rats than in male rats (Figure 6D). The Δ₀ parameter did not differ between the sexes. A multiple regression analysis indicated that among the different parameter estimates, the Δ₊ parameter explained the greatest amount of variance in PRL performance (Δ₊ parameter: t = 5.57, p < .001, R² = 0.37) across sexes (sex × Δ₊ parameter interaction: t = 1.79, p = .08).

Preexisting disruptions in reward-guided decision making may be the mechanism that leads to sex differences in oxycodone self-administration. To directly test this hypothesis, sex and the Δ₊ parameter were entered into a multiple regression analysis predicting total oxycodone intake. Sex was not a significant predictor of total oxycodone intake when the Δ₊ parameter was included in the regression model, indicating that the sex differences in oxycodone intake were almost completely accounted for by the Δ₊ parameter (Figure 6E). A similar effect was observed when sex and the Δ₊ parameter were entered into a multiple regression analysis that predicted the A parameter (Figure 6F) but not the B parameter (sex: t = 3.15, p = .002; Δ₊ parameter: t = 0.14, p = .89).

Next, we conducted a mediation analysis to determine whether the effect of sex on oxycodone-taking behaviors was mediated by variation in the Δ₊ parameter. The significant relationship between sex and the A parameter (β = 0.19, p = .04) was attenuated when the Δ₊ parameter variable was included in the model (β = 0.03, p = .73). The indirect effect was significant (95% CI, 0.33–1.31), indicating that there was a significant reduction in the effect of sex on the A parameter when the Δ₊ parameter was included in the regression model (Figure 6G). Collectively, these data demonstrate that greater oxycodone use in female rats was associated with preexisting disruptions in reward-guided decision making.

Discussion

The current study provides new evidence for key biobehavioral mechanisms that regulate oxycodone-taking behaviors. Using translationally relevant behavioral and computational approaches, we found that disruptions in reward-guided decision-making functions predicted greater oxycodone self-administration and addiction-like behaviors in rats. Rats with poor performance on a PRL task self-administered greater amounts of oxycodone than rats with better PRL performance. Moreover, we have provided evidence that elevated oxycodone-taking behaviors in female rats was driven by preexisting deficits in reward-guided decision-making functions. Collectively, these data demonstrate the utility of decision-making and computational phenotypes for interrogating biologically based differences in opiate-taking behaviors and provide a novel approach for assessing opiate susceptibility in drug-naïve individuals.

Deficits in Reward-Guided Decision Making Predict Greater Oxycodone Use

Not all individuals who use oxycodone develop an opiate use disorder, and there is emerging evidence that some individuals are more likely than others to transition from opiate use to abuse (19,38,39). However, identifying the at-risk individuals prior to initiation of oxycodone use is a challenge. The results of the current study indicate that computationally determined decision-making functions could serve as a behavioral predictor of problematic oxycodone use. We report that rats with poor performance on the PRL task self-administered greater amounts of oxycodone, were more motivated to obtain an oxycodone reinforcer, were slower to extinguish the drug-taking response, and had greater cue-induced reinstatement than rats with better PRL performance. Moreover, decision-making deficits that predicted oxycodone use were specific to reward-mediated updating of action values: Rats with a lower Δ₊ parameter were less likely to repeat a rewarded action and self-administered more oxycodone than rats with a higher Δ₊ parameter. These findings with oxycodone are similar to what we have previously reported for methamphetamine (40) and cocaine (5), which suggests that deficits in the Δ₊ parameter are likely to represent an addiction susceptibility phenotype.

Our characterization of latent drug-taking phenotypes indicates that rats with a lower Δ₊ parameter self-administer greater amounts of oxycodone regardless of drug dose (0.05 mg/kg/delivery or 0.1 mg/kg/delivery), vehicle (saccharin or absence of saccharin), or effort requirement (FR1 schedule vs. FR3 schedule) (see Figure 4I). Therefore, the reinforcing effects of oxycodone appear to be attenuated in rats with a low Δ₊ parameter, which could encourage rats to consume greater amounts of oxycodone to achieve the same euphoric state of rats with a high Δ₊ parameter. Alternatively, rats with a lower Δ₊ parameter may be less sensitive to negative effects of drugs of abuse that regulate or constrain drug-taking behaviors (e.g., sedation, tachycardia). There is evidence that lower sensitivity to alcohol sedation, but greater self-reported rewarding effects of alcohol, predict alcohol use disorder symptoms in humans (41). Additional studies investigating the behavioral and computational processes that lead to differences in the Δ₊ parameter will undoubtedly provide insights into addiction susceptibility mechanisms.

The oral oxycodone self-administration procedure used here did not lead to a systematic, group-level escalation in oxycodone intake. This may be because access to oxycodone was restricted in the current study, in contrast to other studies, which provided continuous access and observed robust increases in oxycodone consumption (24,26). Other operant-based oral self-administration studies that restricted access to oxycodone have also found no evidence of escalation in oxycodone use at the group level (28,42). However, we did see evidence of escalation in a subset of rats that emerged when saccharin was removed from the oxycodone solution (see Figure 3A) and, notably, we saw that the rate of escalation was greater in females than males (see Figure 5E). Although it is not known why removal of saccharin led to different patterns of oxycodone use, it may reveal underlying motivations for oxycodone use and be an important factor in the transition from oxycodone use to abuse.

Sex Differences in Decision Making Are Associated With Oxycodone Use

Self-administration of oxycodone was found to be greater in female rats than male rats, and the power function indicated that both the initial strength of oxycodone reinforcement and the rate of growth in oxycodone use was greater in females. We propose that these differences in oxycodone intake indicate an increase in the vulnerability of female rats to self-administer oxycodone, but it is also possible that they reflect differences in the potency of oxycodone as a reinforcer. However, previous studies have reported vertical shifts in the dose-response curve for oral oxycodone self-administration in females compared with males (28,42), indicating that oxycodone at the doses used here serves a reinforcer in both male and female rats. Previous research groups have observed similar sex differences in oral oxycodone self-administration in both operant and non-operant environments (e.g., home cage) and in settings where access to oxycodone is restricted (3–6 hours/session) or continuous (24,28). Therefore, we propose that the differences observed here reflect sex differences in the incentive value of oxycodone.

However, sex differences are more readily observed in studies that use oral self-administration procedures than in studies that use the intravenous route. This suggests that there is something about oral administration of opioids that leads to differences in opioid reinforcement between the sexes. One possibility is that female rats find the bitter taste of oxycodone less aversive than males (43). In the current study, when oxycodone was delivered in a 0.05% saccharin solution, which likely masked the bitter flavor of the oxycodone, the effect of sex was less pronounced compared with when oxycodone was administered in water. Another possibility is that there are sex differences in first-pass metabolism of oxycodone. Systemic exposure to oxycodone has been found to be higher in females than males following oral administration (∼6.8-fold difference) and larger than that of males following intravenous administration (∼1.6-fold difference) (25). Similar sex differences in the pharmacokinetic profile of orally delivered oxycodone have been observed in humans (44) and may be the reason why females are at greater risk for transitioning from oxycodone use to dependence.

We hypothesized that reward-guided decision making assessed prior to oxycodone exposure would be attenuated in female rats and would account for sex differences in oxycodone use. We have provided direct evidence that supports this hypothesis and propose that disruptions in reward updating are the mechanism by which sex differences in drug use emerge. However, the decision-making phenotypes did not account for the sex differences in the B parameter. This suggests that the greater escalation in oxycodone use that we observed in female rats may be linked to other addiction susceptibility mechanisms that we did not measure in the current study.

The poor decision making that we observed in female rats was surprising given that in our previous studies, we have not detected effects of sex on decision-making functions across adolescent development (6,45). Therefore, sex differences in decision making may emerge during early adulthood, when prefrontal circuits have stabilized (46,47). Prefrontal neurodevelopment plateaus earlier in females than males (48,49), which may lead to a precocial stabilization of decision-making functions in females. Prefrontal circuit stabilization does not depend on the presence or absence of gonadal hormones in females (50) but may involve a more complex interaction between stress hormones, pubertal hormone exposure, and age (51,52). Additional longitudinal studies that quantify these factors of interest across the lifespan of individuals are needed to understand the long-term impact of hormones on development and cognition. The sex differences in PRL performance may also reflect underlying differences in strategy and/or approach that others have observed in mice (53). Future studies investigating sex differences in decision-making functions across different tasks that assess different decision-making strategies (40,54) could help us better understand the sex differences in reward decision-making observed here.

Conclusions

By integrating sophisticated behavioral assessments with computational tools and self-administration assays, the current study provides new evidence indicating that poor reward-guided decision making is a predictor of problematic oxycodone use. Our data from female rats reveal a novel biobehavioral understanding of how differences in drug-taking behaviors may emerge between males and females. Future studies that integrate our computational approaches with drug-taking behaviors, neuroimaging, and genomic assays will provide critical insights into the neurobiological regulators of opiate use disorder.