Mechanisms of impulsive choice: III. The role of reward processes

Abstract

Two experiments examined the relationship between reward processing and impulsive choice. In Experiment 1, rats chose between a smaller-sooner (SS) reward (1 pellet, 10 s) and a larger-later (LL) reward (1, 2, and 4 pellets, 30 s). The rats then experienced concurrent variable-interval 30-s schedules with variations in reward magnitude to evaluate reward magnitude discrimination. LL choice behavior positively correlated with reward magnitude discrimination. In Experiment 2, rats chose between an SS reward (1 pellet, 10 s) and an LL reward (2 and 4 pellets, 30 s). The rats then received either a reward intervention which consisted of concurrent fixed-ratio schedules associated with different magnitudes to improve their reward magnitude discrimination, or a control task. All rats then experienced a post-intervention impulsive choice task followed by a reward magnitude discrimination task to assess intervention efficacy. The rats that received the intervention exhibited increases in post-intervention LL choice behavior, and made more responses for larger-reward magnitudes in the reward magnitude discrimination task, suggesting that the intervention heightened sensitivities to reward magnitude. The results suggest that reward magnitude discrimination plays a key role in individual differences in impulsive choice, and could be a potential target for further intervention developments.

1. Introduction

Impulsive choice has been traditionally evaluated by presenting choices between smaller-sooner (SS) and larger-later (LL) rewards (e.g., $10 in 1 month vs. $100 in 10 months). When the LL reward is optimal in terms of reward rate, individuals who prefer the SS outcome are regarded as impulsive, while those who prefer the LL outcome are regarded as self-controlled (e.g., Galtress, Garcia, et al. 2012). This distinction is critical, as individual differences in childhood impulsivity have been shown to predict societal status and financial success in adulthood (Mischel et al. 1989) and such individual differences among substance abusers have been shown to predict treatment success from rehabilitation programs (MacKillop and Kahler 2009; Stanger et al. 2012; Washio et al. 2011). Moreover, impulsivity-based behavioral deficits have been found among substance abusers (e.g., Bickel and Marsch 2001), pathological gamblers (e.g., Dixon et al. 2003), and obese individuals (e.g., Bruce et al. 2011; Weller et al. 2008), as well as clinical populations with disorders such as attention deficit hyperactivity disorder (ADHD; e.g., Barkley et al. 2001) schizophrenia (Heerey et al. 2007), bipolar disorder (Ahn et al. 2011), depression (Imhoff et al. 2013), borderline personality disorder (Lawrence et al. 2010), and Parkinson’s disease with a comorbidity of impulsive-compulsive behavior (Housden et al. 2010). Moreover, individual differences in impulsive choice behavior are stable in both human (Baker et al. 2003; Jimura et al. 2011; Johnson et al. 2007; Kirby 2009; Matusiewicz et al. 2013; Ohmura et al. 2006; Peters and Büchel 2009) and non-human animals (Marshall et al. 2014; Peterson et al. 2015), suggesting that impulsivity reflects a trait variable. Accordingly, there has been heightened interest in understanding the underlying mechanisms that govern individual differences in impulsive choice (e.g., Galtress, Garcia, et al. 2012; Marshall et al. 2014; Peters and Büchel 2011).

Impulsive choice procedures are designed to pit reward magnitude against reward delay. Accordingly, two primary mechanisms driving individual differences in impulsive choice include sensitivity to changes in both reward magnitude and the passage of time (see Galtress, Garcia, et al. 2012). Indeed, computations of subjective value integrate the delay and magnitude of the SS and LL rewards during value-based decision making (e.g., Peters and Büchel 2011). Several previous studies have confirmed the relationship between temporal processing and impulsive choice behavior (for reviews, see Kirkpatrick et al. 2015; Wittmann and Paulus 2008). For example, the tendency to make impulsive choices has been shown to be positively correlated with poor temporal discrimination ability in rats (Marshall et al. 2014; McClure et al. 2014), and impulsive humans tend to exhibit relative deficits in timing tasks (Baumann and Odum 2012; Darcheville et al. 1992; van den Broek et al. 1992). In addition, methamphetamine increases impulsive choice and those increases have been linked to decreased sensitivity to delay (Pitts and Febbo 2004). Indeed, Kim and Zauberman (2009) suggested that previous factors that have been associated with impulsive choice, such as age (e.g., Green et al. 1994), income (e.g., Green et al. 1996), intelligence (e.g., Shamosh and Gray 2008), and drug abuse (e.g., Bickel and Marsch 2001), may be comprehensively accounted for by subjective differences in temporal processing. Thus, in both human and non-human animals, temporal processing appears to be a critical underlying mechanism of impulsive choice behavior (Kirkpatrick et al. 2015; Marshall et al. 2014; Smith et al. 2015).

In terms of the relationship between reward processing and impulsive choice, Ballard and Knutson (2009) showed that an individuals’ activations in brain regions associated with reward processing was a negative function of the tendency to make more impulsive choices (also see Benningfield et al. 2014), while Eppinger et al. (2012) reported that impulsive choice behavior was positively correlated with activity in the brain’s reward system in response to immediate reward (also see Hariri et al. 2006; McClure et al. 2007; Wilbertz et al. 2012). Additionally, lesions of the nucleus accumbens, a node within the brain’s core valuation circuit (see Galtress, Marshall, et al. 2012; Peters and Büchel 2010), increase impulsive choice (e.g., Cardinal et al. 2001; Galtress and Kirkpatrick 2010), although other reports have shown that partial inactivation of the nucleus accumbens decreased impulsive choice (Moschak and Mitchell 2014). Overall, reward processing appears to be fundamentally associated with impulsive choice, but these connections are not as well documented in the behavioral domain (particularly in rodent models) as the temporal processing deficits.

In conjunction with the neurobiological evidence connecting impulsive choice with reward processing, Locey and Dallery (2009) suggested that the traditional hyperbolic discounting equation [i.e., A/(1+kD)] should include a free parameter (z) accounting for individual differences in sensitivity to reward magnitude [i.e., A^z/(1+kD)]. This suggests that we should expect to find a significant correlation between impulsive choice and sensitivity to reward magnitude. However, recent research in our laboratory failed to find a significant relationship between the rats’ LL choice behavior and their reward magnitude discrimination in a discrete-trial multiple variable-interval schedules of reinforcement task (Marshall et al. 2014). This failure to observe the predicted relationship may have been due to task demands and task structure. The task that was used to assess reward discrimination involved presenting rats with specific magnitudes and measuring their responses to those magnitudes. Specifically, rats received pseudo-alternating variable interval schedules on the left and right levers that were associated with 1 pellet on each side. Then, the reward magnitude was increased for one of the levers and response rates were measured. In this case, the effect of magnitude on behavior was measured purely through response rates rather than through choice behavior. The effects of reward magnitude on behavior have been suggested to be augmented when individuals’ behaviors rather than experimental manipulations determine the reward experienced (see Bonem and Crossman 1988). In other words, the behavioral effects of manipulations of reward magnitude may have been stronger if the rats were able to choose one reward magnitude over another (e.g., impulsive choice task) rather than when the rats were forced to experience one reward magnitude over another (e.g., multiple variable-interval schedules). Therefore, given the input of reward magnitude into computations of subjective value, it is critical to determine the relationship between reward processing and impulsive choice using alternative measures of reward magnitude discrimination that may better capture this relationship. This was the primary goal of Experiment 1 in the present report.

A second goal was to determine whether we could change impulsive choice behavior by giving targeted training to increase reward magnitude discrimination, providing complimentary evidence for a direct link between reward discrimination and impulsive choice. There have not been any previous studies that have directly examined this issue, particularly in rats. One possible intervention targeting reward processing mechanisms was conducted by Stein et al. (2013), who implemented a reward bundling procedure between two phases of impulsive choice tasks. In this reward bundling procedure (see Ainslie and Monterosso 2003), SS and LL rewards were delivered throughout a trial. For example, if the size of the bundle was three, then an SS choice resulted in three SS rewards (e.g., 1 pellet × 3 deliveries), while an LL choice resulted in three LL rewards (e.g., 3 pellets × 3 deliveries); in the bundle conditions, each of the SS and LL rewards were separated by the length of the LL delay. Accordingly, the greater the bundle, the more that the rats would be exposed to LL delays and differential reward magnitudes. Stein et al. (2013) showed that the greater the reward bundling, the more often rats chose LL rewards in the post-test impulsive-choice phase. Interestingly, these results were explained not in terms of exposure to differential reward bundling, but in terms of the greater exposure to LL delays throughout the reward bundling procedure for the rats that received greater bundles of reward (Stein et al. 2013). This explanation corroborates the impact of a time-based intervention on impulsive choice (e.g., Smith et al. 2015), in that greater exposure to reward delays promoted more self-controlled choice. However, an alternative explanation for the effects of reward bundling may relate to exposure to differential reward magnitudes (see Białaszek and Ostaszewski 2012). In sum, an analysis of whether a reward-based intervention would reduce impulsive choice via improvements in reward processing has yet to be conducted. Experiment 2 of the present report sought to address this issue by determining the effects of a novel reward-based intervention task on impulsive choice.

2. Experiment 1

2.1. Methods

2.1.1. Animals

Twenty-four experimentally-naïve male Sprague-Dawley rats (Charles River) were used in the experiment. They arrived to the facility (Kansas State University, Manhattan, KS) at approximately 42–45 days of age. The rats were pair-housed in a dimly-lit (red light) colony room that was set to a 12-hr light:dark schedule (lights off at approximately 8 am). The rats were tested during their dark phase. There was ad libitum access to water in their home cages and in the experimental chambers. The rats were maintained at approximately 85% of their projected ad libitum weight during the experiment based on growth curves obtained from the supplier. In addition to earning food pellets during the experiment, they were fed in their home cages following the experimental session.

2.1.2. Apparatus

The experiment was conducted in 24 operant chambers (Med-Associates; St. Albans, VT) each housed within sound-attenuating, ventilated boxes (74 × 38 × 60 cm). Each chamber (25 × 30 × 30 cm) was equipped with a stainless steel grid floor, two stainless steel walls (front and back), and a transparent polycarbonate side wall, ceiling, and door. A pellet dispenser (ENV-203), mounted on the outside of the operant chamber, delivered 45-mg food pellets (Bio-Serv; Flemington, NJ) to a food cup (ENV-200R7) centered on the lower section of the front wall. Head entries into the food magazine were transduced by an infrared photobeam (ENV-254). Two retractable levers (ENV-112CM) were located on opposite sides of the food cup. The chamber was also equipped with two nosepoke lights (ENV-119M-1) that were located above the levers. Water was continuously available from a sipper tube that protruded through the back wall. Experimental events were controlled and recorded with 2-ms resolution using MED-PC IV (Tatham and Zurn 1989).

2.1.3. Procedure

2.1.3.1. Magazine and lever press training

The rats were given a random-time 60-s schedule of food deliveries for magazine training, earning approximately 120 pellets in a single 2-hr session. The rats were then given two sessions of lever-press training with a fixed ratio (FR) 1 schedule of reinforcement that was followed by a random ratio (RR) 3 schedule and then an RR 5; each of these schedules lasted until the rats earned 20 pellets on each lever. The rats that did not learn to lever press on their own were hand-shaped via successive approximations. Two rats required a third session of lever-press training.

2.1.3.2. Impulsive choice

The impulsive choice task was a modification of Green and Estle (2003) that has been used in our laboratory previously in conjunction with a range of different manipulations (e.g., Galtress, Garcia, et al. 2012; Garcia and Kirkpatrick 2013; Kirkpatrick et al. 2013; Kirkpatrick et al. 2014; Marshall et al. 2014; Peterson et al. 2015; Smith et al. 2015). This procedure yields choice behavior that correlates significantly with other commonly implemented adjusting procedures (Evenden and Ryan 1996; Mazur 1987), but, in comparison with these procedures, has stronger test-retest reliability and results in more rapid acquisition of stable choice behavior (Peterson et al. 2015).

Each session consisted of a randomly intermixed series of free choice and forced choice trials. At the beginning of each session, there was a 10-s interval preceding the first trial. On free choice trials, both the left and right levers were inserted into the chamber. The levers corresponded to a smaller-sooner (SS) outcome and a larger-later (LL) outcome; the assignment of the SS and LL outcomes to the left and right levers was counterbalanced across rats. In free-choice trials, a lever press on one lever resulted in illumination of the cue light above the chosen lever, retraction of the other lever, and initiation of the corresponding reward delay. The first lever press following this delay terminated the trial, causing the lever to retract, the cue light above the lever to turn off, food to be delivered, and a 60-s intertrial interval (ITI) to begin. A fixed interval procedure was used in place of the more typical fixed time procedure (which does not require a response to collect food) to promote sustained attention during the delays to reward. A fixed ITI was included to better mimic real-life situations by allowing for reward maximization (Odum 2011). Forced choice trials were identical to free choice trials, except that only one lever was inserted into the chamber. Each session consisted of 54 free choice trials, 14 SS forced choice trials, and 14 LL forced choice trials, and lasted until all 82 trials had been completed or approximately 2 hr had elapsed.

The primary manipulation of the impulsive choice task was the reward magnitude of the LL outcome; the SS magnitude was maintained at 1 pellet. The LL magnitude was delivered in an ascending series (i.e., 1, 2, and 4 pellets in Phases 1–3, respectively). The SS and LL reward delays were maintained at 10 and 30 s, respectively, across phases; each phase lasted for 10 sessions. One day intervened between the end of the impulsive choice task and the onset of the subsequent reward magnitude discrimination task.

2.1.3.3. Reward magnitude discrimination

The reward magnitude discrimination test involved the simultaneous presentation of both levers, for which lever pressing was reinforced on variable-interval (VI) schedules of reinforcement. Specifically, this concurrent VI-VI task involved two VI 30-s schedules of reinforcement, in which food was available to be delivered t s following lever insertion. The values of t were drawn from independent exponential distributions with means of 30 s. Food was delivered following the first lever press after t s had elapsed on that lever. The delivery of food for lever pressing on one lever did not impact the probability of reinforcement on the other. Rats could freely switch between the two levers without penalty, or changeover delay (COD), as a COD produces more uniform matching (see Herrnstein 1961), and the present experiment was specifically interested in differential biases and sensitivities to manipulations of reward magnitude.

The concurrent VI-VI task manipulated reward magnitude on one of the VI 30 s schedules. On the “small-reward” lever, one pellet of food was delivered following completion of the VI 30 s. Alternatively, on the “large-reward” lever, reward magnitude incremented across phases: 1, 2, and 4 pellets in Phases 1–3, respectively. The small and large-reward lever assignments were the same as the SS and LL levers in the impulsive choice task. Phase 1 lasted for 10 sessions, until the rats exhibited stable behavior across consecutive sessions and were responding at relatively equivalent rates on both 1-pellet levers. Phases 2–3 lasted for 5 sessions each to optimize chances of observing individual differences by avoiding overtraining. Each session lasted for approximately 2 hr.

2.1.4. Data analysis

All summary measures were obtained from the raw data using MATLAB (The MathWorks, Natick, MA). For all analyses, generalized linear mixed-effects models (Wright and London 2009) employed binomial logistic regression with a logit link function. Generalized linear mixed-effects models are comparable to repeated-measures regression analyses, but allow for parameter estimation as a function of manipulation condition (e.g., LL magnitude) and the individual subject (Young et al. 2013). Accordingly, such models permit inclusion of both fixed and random effects, respectively. Model fitting occurred in two stages: Analyses first determined the model with the best fitting random-effects structure, and then determined the model with the best fitting fixed-effects structure that incorporated the aforementioned best fitting random-effects structure. Given the current design, all potential random effects are also potential fixed effects; thus, the factor(s) within the best-fitting random effects structure were automatically included as fixed-effects (Young et al. 2013), so that factors that did not vary as a function of subject (i.e., between-subjects factors) could be entered into the model in this second stage. Here, model selection involved determining the model that minimized the Akaike information criterion (AIC), in which the doubled negative log likelihood of the model is penalized by twice the number of estimated parameters; accordingly, AIC involves determining the best approximating model of the data, such that the model with the minimum AIC reflects the best fit but with a penalty for the expected improvement in fit due to added parameters (see Burnham and Anderson 1998). Continuous predictors were mean-centered to reduce multicollinearity, and categorical predictors were effect-coded (i.e., codes summed to 0).

The dependent measures in these analyses were individual choices (impulsive choice: SS vs. LL; reward magnitude discrimination: small-reward lever vs. large-reward lever). Unless described otherwise below, all responses from all sessions across all subjects were included in analyses. This equated to a total of 37,066 observations from the impulsive choice task and 1,100,436 observations from the reward magnitude discrimination task. The fixed- and random-effects within the models for each of the task analyses are described in the corresponding results sections below.

2.2. Results

2.2.1. Impulsive choice

2.2.1.1. Overall effects

Figure 1 shows the mean proportion of choices for the LL outcome as a function of session for each LL reward magnitude. Session 5 of Phase 1 (LL=1) is missing due to the loss of data files from a computer error. The analysis included the overall intercept, as well as the slopes as a function of LL magnitude, session, and LL Magnitude × Session. Intercept, LL magnitude, session, and LL Magnitude × Session also served as random effects. These factors were treated as continuous predictors. There was no overall bias (i.e., intercept) to make SS versus LL choices, β = −0.64 (SE = 035), t (37062) = −1.84, p = .066, 95% CI [−1.33, 0.04]. However, the rats were significantly more likely to make LL choices as LL magnitude increased, β = 2.38 (SE = 0.17), t (37062) = 14.30, p < .001, 95% CI [2.06, 2.71]. There was also an overall positive relationship between LL choice behavior and session within each phase, β = 0.18 (SE = 0.04), t (37062) = 4.11, p < .001, 95% CI [.09, .26]. These main effects were qualified by a significant LL Magnitude × Session interaction, β = 0.35 (SE = 0.04), t(37062) = 8.41, p < .001, 95% CI [.27, .43]. Specifically, there was a decrease in LL choice as a function of session when an LL choice resulted in 1 pellet in Phase 1 (slope = −0.30), and increases in LL choice as a function of session when an LL choice resulted in 2 or 4 pellets in Phases 2–3 (slopes = 0.05 and 0.75, respectively).

Figure 1.

Mean proportion of choices for the larger-later (LL) outcome as a function of session and LL reward magnitude (1, 2, or 4 pellets). The first session of the LL=1 phase corresponds to the first session of the overall choice task, while the first sessions of the LL=2 and LL=4 phases correspond to the sessions immediately following the tenth sessions of the LL=1 and LL=2 phases, respectively.

2.2.1.2. Individual differences

Figure 2 shows the proportion of choices for the LL outcome for individual rats as a function of session within each phase (i.e., LL=1, LL=2, LL=4). In accordance with the patterns observed in Figure 1, all rats tended to choose the LL outcome more as LL magnitude increased. However, there were substantial individual differences in LL choice behavior across rats in terms of overall propensities to make LL choices and the relationship between LL choice behavior and LL magnitude across sessions within each phase. For example, while the overall behavioral patterns in Figure 1 were representative of some rats’ behavior, such as Rats 11 and 23, Rats 2, 15, and 18 did not exhibit any relative preference for the LL until the second half of Phase 3 (i.e., when the LL magnitude was 4 pellets). Such individual differences support the inclusion of intercept, LL magnitude, session, and LL Magnitude × Session as random effects in the model.

Figure 2.

Proportion of choices for the larger-later (LL) outcome as a function of session and LL reward magnitude for individual rats. The solid (LL=1), dotted (LL=2), and dashed lines (LL=4) represent the individual-subject fits from the best-fitting generalized linear mixed effects model.

A critical part of individual differences analyses involves the determination of whether individual differences are reliable over time. The absence of reliability would suggest a strong state-dependency of the results, such that the observed behavior is more a function of the environment rather than of the individual, thereby limiting interpretation of potential relationships between innate mechanisms (e.g., impulsivity and reward magnitude discrimination). Internal reliability of individual differences was assessed by Cronbach’s alpha (α), which assesses the consistency of choice behavior across different LL magnitudes for the individual rats (i.e., the rats that were most impulsive in the LL = 2 phase should also be the most impulsive in the LL = 4 phase). Here, the mean proportion of LL choices across sessions of each phase for each rat was logit-transformed to account for the assumptions of data analyses that are more robust to normal error score distributions (see Sheng and Sheng 2012; Warton and Hui 2011). In this calculation, a value of .001 was added to both the numerator and denominator to account for exclusive choice behavior (see Haldane 1956). The internal reliability analysis revealed that the individual differences in mean choice behavior across LL magnitudes were relatively consistent, α = .69.

2.2.2. Reward magnitude discrimination

2.2.2.1. Overall effects

Figure 3 shows the mean proportion of responses on the large-reward lever as a function of session for the different large-reward lever outcome magnitudes. As for the impulsive choice task, the first session of the Large = 1 phase corresponds to the first session of the overall reward magnitude discrimination task. Analysis included the overall intercept and the slopes as a function of large-reward lever magnitude, session, and Large-Reward Lever Magnitude × Session as fixed effects. Intercept, large-reward lever magnitude, session, and Large-Reward Lever Magnitude × Session also served as random effects. These factors were treated as continuous predictors. While the rats tended to respond at similar rates on the small- and large-reward levers when the reward magnitudes were equal (Figure 3), there was an overall bias (i.e., intercept) to respond on the large-reward lever across all magnitudes, β = 0.39 (SE = 0.08), t (1100432) = 4.94, p < .001, 95% CI [0.24, 0.55]. Additionally, the rats were significantly more likely to respond on the large-reward lever as large-reward lever magnitude increased, β = 0.24 (SE = 0.04), t (1100432) = 5.62, p < .001, 95% CI [0.15, 0.32], but there was no main effect of session on responding on the large-reward lever, β = 0.004 (SE = 0.01), t (1100432) = 0.45, p = .656, 95% CI [−0.01, 0.02]. However, there was a significant Large-Reward Lever Magnitude × Session interaction, β = 0.04 (SE = 0.01), t (1100432) = 3.30, p < .001, 95% CI [0.01, 0.06], which was driven by the relatively flat slope as a function of session when the large-reward lever magnitude was 1 pellet (slope = −0.01), and positive slopes as a function of session when the large-reward lever magnitude was 2 and 4 pellets (slopes = 0.02 and 0.09, respectively).

Figure 3.

Mean proportion of responses on the large-reward lever as a function of session and large-reward lever magnitude (1, 2, or 4 pellets).

2.2.2.2. Individual differences

Figure 4 shows the proportion of responses on the large-reward lever for individual rats as a function of session within each phase. Note the change in scaling of the ordinate relative to Figure 3. An internal reliability analysis revealed that the individual differences in mean proportion of responses for the large-reward lever (i.e., logit-transformed) were consistent across large-reward lever magnitudes, α = .83 (Marshall et al. 2014). With the exception of Rats 4 and 18, the individual rats exhibited a greater propensity to make more responses on the large-reward lever as its reward magnitude increased. Similarly, as seen in Figure 3, many of the rats did not exhibit considerable changes in responding as a function of session when the large-reward lever magnitude was 1 pellet (but see, e.g., Rats 1, 5, and 13), but did tend to increase their responding on the large-reward lever as a function of session when the large-reward lever magnitude increased to 2 and 4 pellets (but see, e.g., Rat 4). Accordingly, Figure 4 demonstrates the considerable individual differences in behavior as a function of large-reward lever magnitude and session, supporting the inclusion of intercept, large-reward lever magnitude, session, and Large-Reward Lever Magnitude × Session as random effects in the model.

Figure 4.

Proportion of responses on the large-reward (Lg.) lever as a function of session and large-reward lever magnitude (1, 2, or 4 pellets) for individual rats. The solid (Lg.=1), dotted (Lg.=2), and dashed lines (Lg.=4) represent the individual-subject fits from the best-fitting generalized linear mixed effects model.

2.2.4. Inter-task relationships

To determine the relationship between individual differences in performance measures across tasks, simple linear regression analyses were performed. In contrast to the generalized linear mixed effects models that included all sessions and trials, these analyses employed only the data from designated stages of each task, so that these analyses were not complicated in terms of the duration to both learn and exhibit stable behavior across tasks. For the impulsive choice task, the final five sessions were used in the analyses; for the reward magnitude discrimination task, the final three sessions.

For each rat, the mean proportion of LL (impulsive choice) and large-reward lever choices (reward magnitude discrimination) were computed across sessions for each phase. These mean proportions were logit-transformed, as was done for the reliability analyses above, to meet the assumptions of the general linear model (see Warton and Hui 2011). These logit-transformed means were averaged across phases to provide overall summary measures for each task.

From the impulsive choice task, the measures entered into these analyses were the mean LL choice behavior across all phases (i.e., when LL magnitude was 1, 2, and 4 pellets), and the mean LL choice behavior when the LL reward was 2 and 4 pellets. From the reward magnitude discrimination task, the measures entered into the analyses were the mean large-reward lever choice behavior across all phases (i.e., when the large-reward magnitude was 1, 2, and 4 pellets), and the mean large-reward lever choice behavior when the large-reward lever delivered 2 and 4 pellets. For the purpose of simplicity, only the significant regression results are reported.

As shown in Figure 5, the rats that responded more frequently on the large-reward lever in the reward discrimination task when the large-reward lever delivered more reward than the small-reward lever (i.e., 2 or 4 pellets) were the same rats that made more LL choices in the impulsive choice task across all phases, β = 1.53 (SE = 0.67), t(22) = 2.28, p = .032, 95% CI [0.14, 2.92]. Here, reward magnitude discrimination accounted for nearly 20% of the variance of the individual differences in LL choice behavior. While there were no other significant relationships between reward magnitude discrimination and impulsive choice, the other three correlations produced p-values of less than .12. Specifically, impulsive choice across all phases was non-significantly positively related to reward magnitude discrimination across all phases, β = 1.20, p = .079, and impulsive choice when the LL magnitude was 2 or 4 pellets was non-significantly positively related to reward magnitude discrimination when the large-reward lever delivered 2 or 4 pellets, β = 1.95, p = .055, and across all phases, β = 1.54, p = .110.

Figure 5.

The individual rat’s mean proportion of responses for the large-reward lever in the reward magnitude discrimination task when the large-reward lever delivered 2 and 4 pellets (abscissa) plotted against the mean proportion of choices for the LL outcome in the impulsive choice task (ordinate). The variance accounted for by the best fitting regression line is shown (R²).

2.3. Discussion

The present experiment determined the relationship between individual differences in impulsive choice behavior and reward processing. In the impulsive choice task (Figure 1), the rats increased their LL choice behavior as a function of LL magnitude, as observed previously in humans and non-human animals (e.g., Garcia and Kirkpatrick 2013; Green et al. 1999; Marshall et al. 2014). While the mean data were generally representative of the individual rat data (Figure 2), there was considerable inter-subject heterogeneity in both overall bias for making LL choices and sensitivity to the increases in LL magnitude. Subsequently, the reward magnitude discrimination task determined the rats’ overall discriminability of different reward magnitudes in a context without overall differences in reward delay (i.e., as there were in the impulsive choice task). In conjunction with previous results (e.g., Catania 1963), the rats increased their responding for the large-reward lever as the reward on this lever increased in magnitude (Figure 3). As in the impulsive choice task, the mean data were generally representative of the individual rat data (Figure 4). Visual comparison of Figures 2 and 4 suggests that there was less variability across individuals in the reward magnitude discrimination task than in the impulsive choice task. However, this would be expected as the rats would be projected to make exclusive SS choices in Phase 1 of the impulsive choice task (i.e., LL=1), but respond equivalently on both levers in Phase 1 of the reward magnitude discrimination task (i.e., Large=1). Thus, the impulsive choice data would be expected to span a greater range than the reward magnitude discrimination data. Overall, the present series of tasks captured a wide range of individual differences in impulsive choice and reward magnitude discrimination, supporting the rat as a strong preclinical model of impulsive choice behavior and its corresponding mechanisms (e.g., Kirkpatrick et al. 2015).

Regression analyses revealed that the rats that were more likely to respond on the large-reward lever (i.e., when large-reward magnitude was 2 or 4 pellets) were also those that made more LL choices (Figure 5), suggesting that the less impulsive rats were better at discriminating reward magnitude. While only one of four regression analyses were statistically significant, the other correlation patterns paralleled the significant relationship in Figure 5, in that that greater LL choice behavior was related to more responses on the large-reward lever. Accordingly, the present experiment provides evidence for a relationship between impulsive choice and reward discrimination behavior. This result contrasts with recent findings from our lab, using a pseudorandomly alternating VI-VI paradigm, in which no relationship was found (Marshall et al. 2014). The current findings confirm the proposal that assessments of reward processing capabilities may differ when the individual, rather than the experimenter, controls the reward to be experienced (e.g., concurrent VI-VI schedules of reinforcement, impulsive choice tasks; see Bonem and Crossman 1988). Here, the rats were able to choose the reinforcer magnitudes in the reward magnitude discrimination task, which may explain the expression of the current inter-task correlation. Thus, similar to the previously observed finding that greater temporal discrimination is related to greater LL choice in individual rats (Marshall et al. 2014; McClure et al. 2014), the current results suggest that greater reward magnitude discrimination is also related to greater LL choice behavior.

Interestingly, previous research has suggested shared neurobiological correlates for the processing of time and number (e.g., Walsh 2003). Accordingly, an individual’s sensitivity to the passage of time (i.e., temporal units) may be inherently related to an individual’s sensitivity to changes in magnitude (i.e., number units), thus accounting for the relationships between temporal processing, reward processing, and impulsive choice. As recent research has shown that exposure to a time-based intervention both improved temporal discrimination and reduced impulsive choice in rats (Smith et al. 2015), improvements in reward magnitude discrimination may also lead to reductions in impulsive choice behavior, given the potential overlap of the corresponding neurobiological correlates. Experiment 2 trained rats on a reward magnitude discrimination task (or control task) and assessed the subsequent effects on impulsive choice. We also assessed the effects of the reward intervention on reward magnitude discrimination ability. For this, a new reward magnitude discrimination task was implemented, which relied on fixed ratio schedules to expand on the analysis of the relationship between impulsive choice and reward magnitude discrimination. This procedure improved on the previous study by removing delay to reward entirely from the task. The task was used following the intervention to confirm that the intervention produced improvements in reward discrimination ability that may have accompanied any changes in choice behavior.

3. Experiment 2

3.1. Methods

3.1.1. Animals

Twenty-four experimentally-naïve male Sprague-Dawley rats, approximately 42–45 days of age on arrival, were used in the experiment. The housing and husbandry conditions were identical to Experiment 1.

3.1.2. Apparatus

The experimental apparatus was identical to Experiment 1.

3.1.3. Procedure

3.1.3.1. Magazine and lever press training

The magazine and lever-press training procedures were identical to Experiment 1, except that all rats completed lever-press training in two sessions.

3.1.3.2. Pre-intervention impulsive choice

The impulsive choice task was identical to Experiment 1 with the following exceptions. Each session consisted of 56 free choice trials, 12 SS forced choice trials, and 12 LL forced choice trials (i.e., 80 total trials). As for Experiment 1, the assignment of the SS and LL outcomes to the left and right levers was counterbalanced across rats. In addition, the current task included two phases, in which the LL magnitude was 2 and 4 pellets in Phases 1–2, respectively. Phase 1 lasted for 11 sessions and Phase 2 lasted for 10 sessions. Two days intervened between the end of the pre-intervention impulsive choice task and the onset of the subsequent reward-based intervention and control tasks.

3.1.3.3. Reward-based intervention and control tasks

Following the pre-intervention impulsive choice task, the rats were randomly divided into two groups matched for mean LL choice behavior. Half of the rats experienced a reward-based intervention task (Group Intervention; n = 12), while the other half experienced a control task (Group Control; n = 12).

The reward-based intervention and control tasks involved a randomly intermixed series of free choice and forced choice trials. There was a 10-s interval preceding the first trial. On free choice trials, both the left and right levers were inserted into the chamber. The first response on one lever caused the other lever to retract. The next lever press resulted in food delivery, lever retraction, and onset of a 60-s ITI. Therefore, each choice was reinforced on an FR 2 schedule of reinforcement; FR schedules were used to remove any further learning of reward delays as VI schedules increase self-control through improvements in timing processes (Smith et al. 2015). Thus, any intervention-induced changes in performance would more likely be attributed to potential improvements in reward discrimination. Forced choice trials were identical to free choice trials, except only one lever was inserted. There were four blocks of 20 trials per block and each block included 14 free choice trials, 3 small-lever forced choice trials, and 3 large-lever forced choice trials. Each session lasted until 80 trials were completed or approximately 2 hr had elapsed.

Initially, each lever was associated with a single food amount within each block. For Group Control, both levers always resulted in 2 food pellets. For Group Intervention, the “small-reward “ lever resulted in 1 food pellet; the “large-reward “ lever resulted in 2 or 4 food pellets. Within each session, the small/large-reward lever assignments for Group Intervention alternated between the left/right levers for each block. The initial left/right-small/large lever assignments pseudorandomly alternated between sessions. There were no explicit cues to indicate the end of the block (i.e., when the left/right lever assignments changed). For half of the rats in Group Intervention, the first phase of the reward discrimination task involved large-reward lever outcomes of 2 food pellets; for the other half of the rats, the first phase involved large-reward lever outcomes of 4 pellets. Due to experimenter error, the first two sessions of the reward-based intervention task did not include alternating left/right-small/large lever adjustments. However, the subsequent 15 sessions involved alternating left/right-small/large lever adjustments between blocks. Following these 15 sessions, the procedure was modified such that the initial left/right-small/large lever assignment was constant for 4 consecutive sessions.

Due to the rats’ inconsistent preference for the large-reward lever throughout the within-session manipulations of large-reward lever side due to relatively rapid changes in lever-magnitude assignments, the rats were transitioned to a procedure in which the lever assignments for the large- and small-reward lever were maintained throughout the session for 5 consecutive sessions (Phase 1). After which, the lever assignments were switched (Phase 2). Following these first two phases, those rats receiving a large-reward lever magnitude of 2 pellets then experienced a large-reward lever magnitude of 4 pellets, and vice versa, with a subsequent swap of large-/small-reward lever assignments (Phases 3–4). Prior to Phase 1 of this version of the task, the rats were divided into two groups, matched for mean choice behavior, corresponding to the order of 2 versus 4 pellet exposure. For half of each of these groups, the larger magnitude began on the lever opposite to the LL lever from the pre-intervention impulsive choice task (Table 1). The control task was identical to that described above. Overall, this modification of the reward-based intervention lasted for 20 total sessions. One day intervened between the end of the reward-based intervention and control tasks and the onset of the subsequent post-intervention impulsive choice task.

Table 1.

Control and reward intervention testing order conditions across phases of the intervention delivery in Experiment 2. Each phase lasted for three sessions. There were four different testing orders delivered to different sub-sets of rats within the intervention group. p = pellets

Phase	Control		Intervention (1)		Intervention (2)		Intervention (3)		Intervention (4)
	Left	Right	Left	Right	Left	Right	Left	Right	Left	Right
1	2p	2p	1p	4p	1p	2p	2p	1p	4p	1p
2	2p	2p	4p	1p	2p	1p	1p	2p	1p	4p
3	2p	2p	1p	2p	1p	4p	4p	1p	2p	1p
4	2p	2p	2p	1p	4p	1p	1p	4p	1p	2p

3.1.3.4. Post-intervention impulsive choice

The post-intervention impulsive choice task was identical to the pre-intervention impulsive choice task, except that Phases 1 and 2 both lasted for 10 sessions.

3.1.3.5. Reward magnitude discrimination

The reward discrimination test began two days following the post-intervention impulsive choice task and was identical to the reward-based intervention task described above with the following exceptions. In Phase 1, the small and large-reward levers resulted in 1 and 2 pellets, respectively. In this phase, the small-reward lever was on the same side as the LL lever from the post-intervention impulsive choice task and the large-reward lever was on the same side as the SS lever (i.e., the lever assignments were switched). For immediate reference, the lever initially associated with the small reward is referred to as Lever 1, while the lever initially associated with the large reward is referred to as Lever 2. In Phase 2, Levers 1 and 2 resulted in 3 and 2 pellets, respectively. In Phase 3, Levers 1 and 2 resulted in 3 and 4 pellets, respectively. Lastly, in Phase 4, Levers 1 and 2 resulted in 5 and 4 pellets, respectively. Each phase lasted for 3 sessions. The current task used FR schedules of reinforcement so as to match task structure from the intervention and control tasks; accordingly, controlling for such previous task exposure, differences in performance could be more likely attributed to subjective differences in reward magnitude discrimination rather than individual differences in learning a new task.

3.1.4. Data analysis

Data analysis was as described in Experiment 1, with the exception that intervention group (Intervention, Control) and Pre-/Post-Intervention were included as factors in the models. As for Experiment 1, all choices from all sessions across all subjects were included in the analyses unless described otherwise. This equated to a total of 52,941 observations from the impulsive choice task, and 16,124 observations from the reward magnitude discrimination task. Model selection was as described in Experiment 1 with the following exception. Initial analysis of the impulsive choice task revealed that the best random-effects structure included the full factorial of Pre/Post × LL Magnitude × Session. However, when Pre/Post × LL Magnitude × Session and its component interactions and main effects were entered as fixed effects, the model failed to converge, suggesting that the random effects model was overly complex (see Bolker et al. 2008). The former analysis revealed that the random effect of session had the smallest variance among the main effect terms in the random-effects structure, and that the random interactions of Pre/Post × Session and LL Magnitude × Session had the smallest variances among the interaction terms in the random-effects structure. Thus, these terms were contributing minimally to the random effects model and were good candidates for simplifying the random effects model. Accordingly, the random-effects structure of Pre/Post × LL Magnitude + Session was selected, and analysis continued to determine the model with the best fixed-effects structure (i.e., minimum AIC) and the aforementioned random-effects terms.

3.2. Results

3.2.1. Impulsive choice

3.2.1.1. Overall effects

Analysis included the overall intercept, categorical predictors of pre/post intervention and group (Control, Intervention), and continuous predictors of LL magnitude and session. The categorical predictors were effect coded with Pre/Post as −1/+1 and Intervention/Control as −1/+1. The fixed-effects structure included group, pre/post, LL magnitude, session, Group × Pre/Post, Pre/Post × LL Magnitude, Group × Session, Pre/Post × Session, LL Magnitude × Session, Group × Pre/Post × Session, and Pre/Post × LL Magnitude × Session. Intercept, pre/post, LL magnitude, session, and Pre/Post × LL Magnitude served as random effects. Analysis revealed an overall bias (i.e., intercept) to choose the LL outcome, β = 1.45 (SE = 0.23), t(52929) = 6.36, p < .001, 95% CI [1.00, 1.89]. There was no main effect of group, β = −0.07 (SE = 0.22), t (52929) = −0.31, p = .757, 95% CI [−0.50, 0.37], but the rats showed an increase in LL choice behavior from the pre-intervention to the post-intervention phases, β = 0.34 (SE = 0.09), t (52929) = 3.88, p < .001, 95% CI [0.17, 0.50], and were significantly more likely to make LL choices as a function of LL magnitude, β = 1.54 (SE = 0.08), t(52929) = 19.43, p < .001, 95% CI [1.38, 1.69], and as a function of session, β = 0.19 (SE = 0.03), t (52929) = 7.42, p < .001, 95% CI [0.14, 0.24].

In addition, there were significant two-way interactions between Pre/Post × Session, β = −0.02 (SE = 0.01), t (52929) = −4.55, p < .001, 95% CI [−0.03, −0.01], and LL Magnitude × Session, β = 0.09 (SE = 0.01), t (52929) = 16.68, p < .001, 95% CI [0.08, 0.10], as well as significant three-way interactions between Group × Pre/Post × Session, β = 0.02 (SE = 0.004), t(52929) = 3.66, p < .001, 95% CI [0.01, 0.02], and Pre/Post × LL Magnitude × Session, β = 0.03 (SE = 0.01), t (52929) = 6.27, p < .001, 95% CI [0.02, 0.04]. The three-way interactions are displayed in Figure 6, with the top panel displaying the Group × Pre/Post × Session interaction and the bottom panel displaying the Pre/Post × LL Magnitude × Session interaction. Group Intervention showed a greater increase in post-intervention LL choice behavior than Group Control, especially in the early sessions post-intervention (Figure 6, top panel), and there were larger increases in post-intervention LL choice behavior when the LL reward was 2 pellets than when it was 4 pellets that were particularly evident in the early sessions of the 2-pellet post-intervention phase (Figure 6, bottom panel).

Figure 6.

Top panel: Mean proportion of choices for the larger-later (LL) outcome before (Pre) and after (Post) the reward-based intervention for the rats in Groups Control and Intervention as a function of session, collapsed across LL magnitudes. Bottom panel: Mean proportion of choices for the LL outcome before (Pre) and after (Post) the reward-based intervention as a function of session and LL magnitude (2 or 4 pellets), collapsed across group. The three-way interactions shown here represent the highest-order significant interactions from the corresponding model.

In addition to the significant effects described above, the best-fitting model also included non-significant interactions between Group × Pre/Post, β = −0.09 (SE = 0.06), t (52929) = −1.40, p = .160, 95% CI [−0.21, 0.03], Pre/Post × LL Magnitude, β = −0.003 (SE = 0.05), t (52929) = −0.06, p = .956, 95% CI [−0.11, 0.10], and Group × Session, β = −0.01 (SE = 0.02), t(52929) = −0.45, p = .649, 95% CI [−0.05, 0.03]. While these lower-order effects were non-significant, the significance of the three-way interactions warranted their inclusion in the model (Cohen et al. 2003).

3.2.1.2. Individual differences

Figure 7 shows the proportion of choices for the LL outcome for individual rats before and after the intervention as a function of session within each phase. An internal reliability analysis conducted on the mean proportion of choices for LL outcome (i.e., logit-transformed) across the four phases (pre/post × LL magnitude) revealed that individual differences were consistent across phases, α = .78. Similar to that seen in Figure 6, individual rats chose the LL outcome more as LL magnitude increased. There were also large individual differences in LL choice behavior as a function of session. Furthermore, many of the rats exhibited similar or greater preference for the LL choice following the intervention than before the intervention (but see Rats C.3 and I.3). Moreover, there were considerable individual differences in changes in LL choice behavior before versus after the intervention as a function of LL magnitude. For instance, Rat C.6 showed a relative increase in LL choice behavior following the intervention when the LL reward was 2 pellets but not 4 pellets (also see Rats C.9, C10, and I.2), while Rat I.10 showed a similar increase in LL choice behavior when the LL reward was 4 pellets but not 2 pellets (also see Rat I.4). Such individual differences support the inclusion of intercept, LL magnitude, pre/post, session, and LL Magnitude × Pre/Post as random effects in the model.

Figure 7.

Proportion of choices for the larger-later (LL) outcome before (Pre) and after (Post) the reward-based intervention as a function of session and LL reward magnitude for individual rats. The solid (Pre: LL=2), dotted (Pre: LL=4), dashed (Post: LL=2), and dot-dash lines (Post: LL=4) represent the individual-subject fits from the best-fitting generalized linear mixed effects model. The rat numbers preceded by a “C” are individuals in Group Control, while the rat numbers preceded by an “I” are individuals in Group Intervention.

3.2.2. Reward magnitude discrimination

3.2.2.1. Overall effects

Figure 8 shows the mean proportion of choices for the larger-magnitude choice for Groups Control (top) and Intervention (bottom) as a function of session within each phase characterized by different combinations of small- and large-magnitudes (e.g., 1 pellet vs. 2 pellets; “1v2”). Analysis included the overall intercept, a categorical predictor of group (Control, Intervention), the slopes as a function of magnitude ratio, session, and Magnitude Ratio × Session, and a Group × Magnitude Ratio interaction. Group was effect coded with Intervention/Control as −1/+1, and magnitude ratio and session were treated as continuous predictors. Intercept, magnitude ratio, session, and Magnitude Ratio × Session also served as random effects. Analysis revealed an overall bias (i.e., intercept) to choose the larger-reward lever, β = 0.34 (SE = 0.07), t (16118) = 5.04, p < .001, 95% CI [0.21, 0.47]. Group Control was less likely to choose the larger-reward lever (linear-transformed bias = .54) than Group Intervention (bias = .63), as revealed by a main effect of group, β = −0.18 (SE = 0.07), t (16118) = −2.84, p = .005, 95% CI [−0.31, −0.06]. Additionally, the rats were significantly less likely to choose the large-reward lever as magnitude ratio decreased and, hence, discrimination difficulty increased, β = −0.16 (SE = 0.07), t (16118) = −2.50, p = .013, 95% CI [−0.29, −0.04], and were also significantly more likely to choose the larger-reward lever with session within each phase, β = 0.48 (SE = 0.06), t(16118) = 8.38, p < .001, 95% CI [0.36, 0.59]. The main effects of magnitude ratio and session were qualified by a significant Magnitude Ratio × Session interaction, β = −0.15 (SE = 0.04), t (16118) = −3.50, p < .001, 95% CI [−0.23, −0.06]. Specifically, as magnitude ratio decreased, there were smaller increases in choice of the larger-reward lever as a function of session (slopes = .70, .55, .40, and .25). Lastly, the main effects of magnitude ratio and group were qualified by a significant Group × Magnitude Ratio interaction, β = 0.13 (SE = 0.06), t (16118) = 2.02, p = .043, 95% CI [0.004, 0.26]. Specifically, as magnitude ratio decreased, the behavioral differences between Group Intervention and Group Control declined (slopes = −.38, −.25, −.12, .01).

Figure 8.

Mean proportion of choices for the larger-magnitude outcome as a function of session and magnitude ratio (e.g., small-magnitude = 1 pellet, large-magnitude = 2 pellets; “1v2”) for the rats in Group Control (top) and Group Intervention (bottom).

3.2.2.2. Individual differences

Figure 9 shows the proportion of responses on the larger-reward lever for individual rats as a function of session within each phase. There were considerable individual differences as a function of magnitude ratio, session, and the interaction between these two factors. For example, while some rats exhibited general increases in choice behavior for the larger-reward lever with session (e.g., Rats C.4 and C.8), some rats did not show such an increase across sessions within each phase (e.g., Rats C.6 and I.3) and some rats only showed increases across sessions within certain phases and not others (e.g., Rats C.5 and I.9). Accordingly, such individual differences support the inclusion of intercept, magnitude ratio, session, and Magnitude Ratio × Session as random effects in the model.

Figure 9.

Proportion of choices for the larger-magnitude outcome as a function of session and magnitude ratio (e.g., small-magnitude = 1 pellet, large-magnitude = 2 pellets; “1v2”) for individual rats. The solid (1v2), dotted (2v3), dashed (3v4), and dot-dash lines (4v5) represent the individual-subject fits from the best-fitting generalized linear mixed effects model. The rat numbers preceded by a “C” are individuals in Group Control, while the rat numbers preceded by an “I” are individuals in Group Intervention.

As seen in Figure 9, some rats exhibited the greatest proportion of choices for the larger-reward lever when the magnitude ratio was largest (e.g., Rats I.1 and I.8), similar to that seen in terms of the mean behavior in Figure 8. Thus, most rats displayed an overall effect of magnitude ratio on their larger-reward lever responses. However, some rats were biased to one side (e.g., Rats C.9 and C10) and thus did not show sensitivity to magnitude ratio. This latter situation would fail to capture whether individual differences were consistent over time, as such rats would primarily respond on one lever regardless of reward magnitude. Indeed, the reliability analysis of the mean proportion of choices for the large-magnitude lever (i.e., logit-transformed) across phases revealed a general inconsistency of individual differences seen in Figure 9, α < 0. A reliability analysis of such individual differences only including the final session in each phase also revealed similar inconsistency, α < 0. The inconsistency of individual differences as revealed by the reliability analysis was primarily driven by the correlations in mean proportion of choices for the larger-reward lever across phases. Specifically, across rats, there were positive correlations between the mean proportion of choices for the larger-reward lever between phases in which the larger-magnitude rewards were on the same lever (i.e., left lever - left lever), while there were negative correlations between the mean proportion of choices for the larger-reward lever between phases in which the larger-magnitude rewards were on opposite levers (i.e., left lever - right lever), |r |s > .40, ps < .055. These results reflect lever biases, such that some rats were likely less sensitive to the dynamicity of the task. However, as seen in Figure 9, many of the rats’ behaviors suggested that they were sensitive to the task parameters in the first phase of the reward magnitude discrimination (i.e., 1 pellet vs. 2 pellets); that is, a considerable proportion of rats showed an increase in choice for the larger-reward lever across sessions in this first phase. Accordingly, these mean data across sessions from the 1v2 phase of the reward discrimination task will be subsequently used to evaluate individual differences in reward magnitude discrimination.

3.2.3. Inter-task relationships

Figures 6–9 and the corresponding analyses suggest that the reward-based intervention reduced impulsive behavior in the rats in Group Intervention while also improving reward discrimination. More specifically, the effect of the reward-based intervention on impulsive choice was manifested early in the post-intervention phases. To determine whether reductions in impulsive choice across the post-intervention impulsive choice task were related to improvements in reward discrimination, a multiple linear regression was conducted. The criterion in the model was the difference score between mean post-intervention proportion of LL choices and the mean pre-intervention proportion of LL choices, such that larger values reflect a greater increase in post-intervention LL choice behavior; the mean pre-and post-intervention LL choice proportions included all sessions from both the pre- and post-intervention impulsive choice tasks so as to assess overall impact on choice behavior. These proportions were logit-transformed. There were three predictors in the model. The first predictor of Group was effect-coded with Intervention/Control as −1/+1. The second predictor was the logit-transformed mean proportion of choices for the larger-reward lever in Phase 1 (i.e., 1 pellet vs. 2 pellets) of the reward magnitude discrimination task where reward discrimination was most apparent (Figure 8). This predictor was mean-centered to reduce multicollinearity. The final predictor was the Group × Reward Magnitude Discriminability interaction. All predictors were entered simultaneously into the model.

Figure 10 shows the relationship between reward magnitude discrimination and impulsive choice, separated by group. Group Intervention exhibited a positive relationship between performance in the impulsive choice and reward magnitude discrimination tasks, while Group Control exhibited a negative relationship, as revealed by a significant interaction between group and reward magnitude discrimination, β = −0.39 (SE = 0.17), t(20) = −2.29, p = .033, 95% CI [−0.74, −0.04]. Accordingly, for Group Intervention, greater sensitivity to reward magnitude predicted greater increases in LL choice post-intervention. However, for the rats in Group Control, the rats that showed greater reductions in impulsive choice were those that were less sensitive to reward magnitude.

Figure 10.

The individual rats’ logit-transformed mean proportion of choices for the large-reward lever in Phase 1 of the reward magnitude discrimination task (i.e., mean-centered) plotted against the difference score (post-pre) of the logit-transformed proportion of choices for the LL outcome, separated by group. The solid (Group Intervention) and dotted (Group Control) lines represent the linear regression model fits.

3.3. Discussion

Theoretically, the subjective value of a delayed reward depends on magnitude and delay (e.g., Peters and Büchel 2011), such that individual differences in sensitivity to magnitude and delay should impact the subjective value of reward. Previous research has shown that improving temporal discrimination in rats leads to reductions in impulsive choice (Smith et al. 2015). The present experiment determined whether a reward-based intervention would also decrease impulsive choice behavior via improvements in reward magnitude discrimination in rats. Indeed, the rats that received the intervention exhibited (1) more LL choices early in the post-intervention phases (Figures 6–7), and (2) greater reward magnitude discrimination compared to the rats that received the control task (Figures 8–9).

In the impulsive choice task (Figures 6–7), the rats showed greater increases in LL choice behavior from the pre-intervention LL = 2 phase to the post-intervention LL = 2 phase relative to the LL = 4 phases. In regards to behavioral differences between Groups Control and Intervention, the effect of the reward-based intervention appeared to serve a more critical determinant of behavior following transitions of reward contingencies (Figure 6, top panel). Specifically, while both groups exhibited similar choice behavior before the intervention, Group Intervention showed a larger increase in post-intervention LL choice behavior during the initial sessions of each post-intervention phase. But, this difference faded with further training in the post-intervention phases. Thus, these results suggest that the reward-based intervention acted to improve sensitivity to changes in reward parameters, facilitating transitions to relatively stable levels of choice behavior.

Further support for the hypothesis that the reward-based intervention improved sensitivity to changes in response-reinforcer contingencies was found within the reward magnitude discrimination task. Here, Group Intervention exhibited a greater proportion of choices for the larger-reward lever when its spatial position changed every three sessions (Figures 8–9). While the more challenging phases of the reward magnitude discrimination task elicited greater reliance on spatial biases to respond on a particular lever in some individuals, most rats did track the relative magnitudes. It is worth noting that the initial phase of the reward magnitude discrimination task employed the same reward magnitudes as the initial phases of the pre- and post-intervention impulsive choice tasks without the longer delays to reward. Accordingly, it would be expected that the rats in both groups, having exhibited stable choice behavior when choosing between these delayed magnitudes in the impulsive choice task, would exhibit more exclusive preference for the 2-pellet reward in the reward magnitude discrimination task. However, in this first phase, only Group Intervention exhibited the predicted pattern of behavior, displaying a relative preference early in the 1v2 phase of the reward magnitude discrimination task than Group Control. Thus, the reward-based intervention seemingly served to improve sensitivity to changes in reward contingencies.

Of intrigue within the present results was the contrasting relationship between performances within the impulsive choice and reward magnitude discrimination tasks in Groups Control and Intervention (Figure 10). Groups Intervention and Control showed a positive and negative relationship, respectively, between reward magnitude discrimination and improvements in pre- versus post-intervention self-control, indicating differential behavioral effects of the control and intervention tasks. This difference may be explained in terms of task dynamics. The rats in Group Intervention were exposed to dynamic response-reinforcer contingencies prior to the reward magnitude discrimination task within the reward-based intervention; the rats in Group Control were not (i.e., reward magnitudes were constant throughout the intervention phase). Thus, for Group Control, the negative relationship between reward magnitude discrimination and increases in self-control may simply reflect slower learning of the new response-reinforcer contingencies or less adaptability to changing response-reinforcer contingencies; this relationship may have changed if they had been given longer exposure to the reward magnitude discrimination task. Recall that the larger-reward lever in Phase 1 of the reward magnitude discrimination task was not the same as the LL lever in the impulsive choice task (the lever assignments were switched). Thus, rats that were more biased to make the LL choice may then be expected to make fewer larger-reward lever choices early in the reward magnitude discrimination task, especially if those rats had not been exposed to the more dynamic reward contingencies associated with the intervention task. Indeed, Group Intervention showed more larger-reward lever choices during the 1v2 phase the reward magnitude discrimination task than Group Control (Figure 8). Accordingly, the rats that had previous exposure to changing reward contingencies (i.e., Group Intervention) may have learned that their environment involved changes in response-reinforcer contingencies, thereby facilitating transitions when such changes occurred (also see Carlson and Wielkiewicz 1976), whereas the control group did not experience this benefit. An interesting possibility for future research would be to test the effects of the reward magnitude intervention on impulsive choice within an adjusting amount task, where reward amounts change dynamically based on recent choice patterns (e.g., Richards et al. 1997).

4. General Discussion

The combined results of the two experiments indicate an important role for reward magnitude discrimination abilities in impulsive choice. Experiment 1 demonstrated that rats with better reward magnitude discrimination also chose the LL more often, demonstrating a positive correlation between reward magnitude discrimination and self-control. Experiment 2 verified this relationship and provided evidence of a more causal link between reward discrimination and self-control. Specifically, the rats in Group Intervention showed increased post-intervention LL choices relative to Group Control in the early post-intervention sessions of each magnitude, and the rats that showed greater overall increases in self-control also exhibited better discrimination between differential reward magnitudes (Figure 10).

Furthermore, the present studies indicated that rats demonstrate a broad spectrum of individual differences in both impulsive choice and reward magnitude discrimination, with good internal reliability, confirming previous reports (Galtress, Garcia, et al. 2012; Garcia and Kirkpatrick 2013; Kirkpatrick et al. 2015; Marshall et al. 2014). The mixed-effects modeling approach allowed for an extension on previous findings by isolating potential sources of individual differences. For example, in both experiments, the analysis of impulsive choice indicated that the best-fitting model included a random effect of intercept, suggesting that there were individual differences in overall biases to behave impulsively (i.e., SS vs. LL choices). Moreover, the random effect of LL magnitude slope demonstrated individual differences in sensitivities to changes in reward magnitude within the impulsive choice task (Figures 2 and 7). These differences were apparent both in terms of the speed of learning across sessions for individual magnitudes (Experiment 1), and also in terms of the overall degree of sensitivity to the individual magnitudes (Experiments 1 and 2). These patterns are highly consistent with the important role of reward processing in impulsive choice, indicating that individual differences in reward magnitude discrimination are apparent within the impulsive choice task and that these can be disclosed through random effects analyses. Experiment 2 further demonstrated random effects in the response to the intervention, which interacted with LL magnitude. These results confirm that the rats were differentially sensitive to the reward-based intervention versus the control task, and that this affected their sensitivity to LL magnitudes, consistent with the observed effects of the reward-based intervention on reward magnitude discrimination. Ultimately, these results demonstrate that individual differences in self-control are critically governed by differences in reward magnitude discrimination ability.

The reward magnitude discrimination task in Experiment 2 also showed evidence of a numerical distance effect (e.g., Gallistel and Gelman 2000), suggesting that the rats in Group Intervention were treating the reward magnitudes as numerical information (Figure 8). The intervention group demonstrated the best discrimination of the 1v2 pellets (ratio = 2) followed by 2v3 (ratio = 1.5) and 3v4 (ratio = 1.33) followed by 4v5 (ratio = 1.25). Numerical distance effects were also seen in individual rats, although these effects were clouded by side biases that emerged in some individuals (Figure 9). Numerical distance effects are well established for other types of numerical tasks, as they emerge from Weber’s law, which states that quantity discriminations follow a ratio rule. The observation of a general numerical distance effect could provide important clues as to the underlying mechanisms behind the effects of reward magnitude discrimination on impulsive choice, suggesting that numerical processing may play a role in discrimination abilities. Importantly, the numerical distance effects were only apparent in Group Intervention, suggesting that the intervention may have promoted or even induced this effect. It is possible that this may be a key element of the intervention effect on self-control. This issue should be examined in future research.

The present results complement recent research demonstrating a key role for temporal discrimination capacity in impulsive choice, and suggest similar parallel processes may occur for temporal and reward components of impulsive choice. As described above, previous studies have shown a positive correlation between impulsive choice and temporal discrimination ability, assessed by temporal discrimination (Marshall et al. 2014) and peak procedure tasks (McClure et al. 2014). Moreover, Smith et al. (2015) demonstrated that extensive exposure to the timing of different delays within differential-reinforcement-of-low-rates (DRL), fixed-interval, and variable-interval schedules of reinforcement all resulted in improved temporal discrimination, while simultaneously promoting self-control. Thus, similar to the present studies supporting the relationship between reward magnitude discrimination and self-control (see Figures 5 and 10), there is a clear connection between temporal discrimination and self-control. Accordingly, reward magnitude discrimination and temporal discrimination may reflect two of the more important factors that govern individual differences in impulsive decision making.

The examination of possible mechanisms of individual differences in self-control/impulsive choice is a critical venture, as impulsive choice has been proposed to serve as a trans-disease process (Bickel and Mueller 2009) that may be a primary liability factor for the development of a range of maladaptive diseases and disorders, as described in the Introduction. The present results, coupled with previous findings, provide important methodologies and insights into the problem. For example, robust behavioral screening measures present opportunities for identifying individuals with different profiles of deficits that all result in impulsive decision making. As described in the present report, two reward discrimination tasks predicted self-control/impulsive choice through reward discrimination abilities (Figures 5 and 10). Indeed, the impulsive choice task itself provided a robust measurement of reliable individual differences that were disclosed using a mixed-effects modeling approach. This same general approach can be used to identify the relationship between deficits in temporal discrimination deficits and choice, using either the impulsive choice procedure with manipulations of delay, or alternative methods such as temporal discrimination or peak procedure tasks (Marshall et al. 2014; McClure et al. 2014). Similar to how time-based interventions could be used to reduce impulsive choice in individuals with temporal discrimination deficits (Smith et al. 2015), the reward intervention presents an opportunity to modify impulsive choice by improving reward discrimination ability. Accordingly, the intervention could be used in conjunction with behavioral screening techniques to moderate impulsive choice in at-risk sub-populations with reward processing deficits, such as individuals with schizophrenia and ADHD (Gold et al. 2008; Wilbertz et al. 2012). Thus, the present set of experiments provides new key insights into the core mechanisms of impulsive choice, ultimately supplying new methods for advancing our understanding of this critical trans-disease process within the rodent pre-clinical model.