Decision making: effects of methylphenidate on temporal discounting in nonhuman primates

Abstract

Decisions are often made based on which option will result in the largest reward. When given a choice between a smaller but immediate reward and a larger delayed reward, however, humans and animals often choose the smaller, an effect known as temporal discounting. Dopamine (DA) neurotransmission is central to reward processing and encodes delayed reward value. Impulsivity, the tendency to act without forethought, is associated with excessive discounting of rewards, which has been documented in patients with attention deficit hyperactivity disorder (ADHD). Both impulsivity and temporal discounting are linked to the dopaminergic system. Methylphenidate (MPH), which blocks the DA transporter and increases extracellular levels of DA in the basal ganglia and prefrontal cortex, is a primary treatment for ADHD and, at low doses, ameliorates impulsivity in both humans and animals. This study tested the hypothesis that low doses of MPH would decrease the discounting rate of rhesus monkeys performing an intertemporal choice task, suggesting a reduction in impulsivity. The results support this hypothesis and provide further evidence for the role of DA in temporal discounting and impulsive behavior.

decision making is often based on selecting what is thought to be the most rewarding option. However, when faced with a choice between a small but immediate reward and a larger but more delayed reward, humans and animals (Ainslie 1974; Hwang et al. 2009; Kobayashi and Schultz 2008; Richards et al. 1997; Rodriguez and Logue 1988; Wang et al. 2014) tend to prefer the smaller reward if the difference in magnitude is sufficiently small or if the delay to receive the larger reward is too long. This seemingly irrational choice is thought to be the result of the subjective value of the larger reward decaying as a function of delay, a process known as temporal discounting (Ainslie 1975; Samuelson 1937).

Children diagnosed with attention deficit hyperactivity disorder (ADHD) have difficulty delaying gratification (Rapport et al. 1986) and discount reward value more than controls, preferring smaller, sooner rewards (Barkley et al. 2001; Green and Myerson 2004; Luman et al. 2005; Marco et al. 2009; Paloyelis et al. 2010; Peters and Buchel 2011; Reynolds 2006; Rubia et al. 2009; Sonuga-Barke et al. 1992), particularly those with the combined impulsive/hyperactive subtype (Paloyelis et al. 2010; Scheres et al. 2010). Thus steeper discounting of rewards is thought to reflect greater impulsivity and poorer self-control (Critchfield and Kollins 2001; Evenden 1999; Kobayashi and Schultz 2008; Logue 1988; Sagvolden et al. 2005; Shiels et al. 2009).

Both impulsivity and temporal discounting are linked to the dopaminergic system. Reward information is encoded by dopamine (DA) neurons (Schultz et al. 1997), and temporal discounting has been associated with the DA system and DA-modulated circuits (Ballard and Knutson 2009; Boettiger et al. 2007; Cardinal et al. 2001; Dalley et al. 2007; Figner et al. 2010; Kable and Glimcher 2007, 2009, 2010; Kim et al. 2008; McClure et al. 2004, 2007; Paloyelis et al. 2010; Wang et al. 2014). The response of DA neurons to delayed rewards, for instance, decreases at a rate similar to behavioral discounting, suggesting that DA responses encode subjective value (Kobayashi and Schultz 2008).

Impulsivity and ADHD, of which impulsivity is a cardinal symptom, have been linked to alterations within the DA system (Cook et al. 1995; Dalley et al. 2007; Gill et al. 1997; Krause et al. 2000; Pattij and Vandershuren 2008; Paloyelis et al. 2010; Rajala et al. 2014; Volkow et al. 2009) and are treated predominantly with psychostimulants that alter DA neurotransmission. Methylphenidate (MPH; Ritalin), one of the most common of such drugs (Greenhill 2001; Solanto 1998), blocks the dopamine transporter (DAT), which prevents the reuptake of DA and increases its extracellular levels (Kojima et al. 2011; Volkow et al. 1998, 2001).

The present study, therefore, tested the hypothesis that changes in extracellular levels of DA resulting from the administration of low doses of MPH would change the discounting rate of rhesus monkeys performing an intertemporal choice task, which would suggest a reduction in impulsivity. Rhesus monkeys, the closest animal model to humans used for this type of work, were selected for this study because they can be tested under highly controlled conditions, are able to perform intertemporal choice tasks (Hwang et al. 2009; Kim et al. 2008; Kobayashi and Schultz 2008), are similar to humans in the structure, function, and dopaminergic innervation of the frontal brain (Berger et al. 1991; Wise 2008), respond to MPH similarly to humans (Rajala et al. 2012), and, importantly, can be tested multiple times with various doses of the drug without the ethical constraints inherent to human experiments, particularly those involving children. The results support this hypothesis and provide further evidence for the role of the DA system in temporal discounting and impulsive behavior.

MATERIALS AND METHODS

Subjects and Surgery

Four adult (9–14 yr) male rhesus monkeys (Macaca mulatta) ranging from 9.0 to 11.5 kg and purchased from the Wisconsin Regional Primate Research Center were used in this study. Three of the four subjects were used in earlier studies of the effects of MPH on cognitive function (Rajala et al. 2012). The animals were prepared for eye-movement recording by implanting scleral search coils (Judge et al. 1980), which were constructed from Teflon-coated stainless steel wire (SA632; Cooner Wire, Chatsworth, CA), and a lightweight titanium head post, which was used to restrain the head for cleaning around the implant area as well as for experimental sessions. All surgical procedures were approved by the University of Wisconsin Animal Care and Use Committee and were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals.

Experimental Setup

The experiments were carried out in 3 × 3 × 2-m double-walled soundproof chambers (Acoustic Systems, Austin, TX) under dim illumination. Eye movements were measured with the magnetic search coil technique (Robinson 1963) using a phase-angle system (CNC Engineering, Seattle, WA). The signals representing horizontal and vertical eye position were low-pass filtered at 250 Hz (Krohn-Hite, Brockton, MA) and sampled at 500 Hz with an analog-to-digital converter (Tucker Davis Technologies, Alachua, FL). Visual stimuli were presented on a computer screen placed 56 cm in front of the animals. The coils were calibrated with a behavioral procedure that relied on the animals' tendency to look at spots of light presented in a darkened environment (Populin and Yin 1998). Subjects were trained to make eye movements to visual stimuli using operant conditioning; a water reward was delivered after the subject looked at the stimuli within the specified time window (Populin 2006). Linear equations were fit to the horizontal and vertical components of the eye-movement data, and the resulting coefficients were used to convert the voltage output from the coil system to degrees of visual angle. Data acquisition was performed using custom software. The digitized eye-position signals were stored in a relational database for offline analysis.

Intertemporal Choice Task and Experimental Sessions

The experimental task was modeled after that used by Kobayashi and Schultz (2008) and was designed to compel subjects to make a value-based decision between a small reward delivered immediately (smaller sooner; SS) and a larger reward delivered after a delay period (larger later; LL).

Images.

Eight fractal images selected at random, with no previous association or meaning to the subjects, were used to represent each of the various SS and LL conditions (please see endnote). Image size and resolution were standardized (340 × 454 pixels, 72 pixels/inch). To control for differences in luminance, the brightness of each image was measured twice at five points (4 corners and the center) using a Minolta LS-100 luminance meter (Minolta Camera, Tokyo, Japan) and then averaged. The brightness of each image was adjusted until the average luminance fell between 25 and 30 cd/m² (average luminance of the 8 images = 26.92 ± 0.78 cd/m²).

Reward conditions.

Water was used as a reward in all experimental sessions. The SS delay was fixed at 0 s and had four possible reward magnitudes (subjects MI and SH: 0.2, 0.3, 0.4, and 0.5 ml; subject GO and CO: 0.17, 0.29, 0.40, and 0.55 ml), whereas the LL reward magnitude was fixed at 0.60 ml for subjects MI and SH and 0.69 ml for subjects GO and CO and had four possible delay times (2, 4, 8, and 16 s). The different sets of reward values resulted from the use of two different experimental setups, which responded slightly differently, albeit consistently, to the same software commands. The fractal images used were the same across all four subjects, who learned the reward-image associations by trial and error in several initial training sessions.

Experimental task.

Each trial began with a red fixation point presented at the center of a computer screen, which the monkey was required to look at to proceed with the trial (Fig. 1). After the computer detected that the monkey was looking at the red dot for 350 ms, two target images appeared at fixed positions to the left and right of the fixation point, one representing the SS and the other the LL. The offset of the central point, 300 ms after the presentation of the images, commanded the subject to choose a reward option and respond by making an eye movement to one of the images within 700 ms. When the saccade reached a target, a red dot appeared overlaid on it for 500 ms, indicating the choice. The subject then either received the reward (SS) or had to wait the associated delay period before receiving the reward (LL). In the case of the SS, the images disappeared simultaneously with the red indicator dot. In the case of the LL, the images remained on the screen for the duration of the delay, but no behavioral criteria were imposed during this time. The position of the two images was randomized in every trial. If the subject failed to acquire the initial fixation point or failed to make an eye movement to one of the images within 700 ms, the trial was considered a failure, immediately terminated, and repeated until it was successfully completed.

Fig. 1.

Schematic representation of the intertemporal reward choice task. 1) The subject was required to fixate on a red dot presented in the middle of the screen, straight ahead, for 350 ms, at which point 2) 2 fractal images representing the smaller sooner reward (SS) and larger later reward (LL) were displayed to the left and right of the fixation point while the subject maintained fixation. 3) Upon offset of the fixation point, the signal to respond, the subject was expected to make a saccadic eye movement to 1 of the fractal images, indicating his choice. 4) When the computer detected the subject's eyes within the acceptance window of 1 of the images, a large red dot was overlaid on the fractal, indicating to the subject his choice. 5) In this example, if the subject chose the SS (left image), the reward would be delivered immediately, and a 16-s wait would precede the start of the next trial. Conversely, if the subject chose the LL (right image), a delay of 16 s would precede the delivery of the reward and the onset of the next trial, which would start shortly after; both images remained on the screen for the duration of the delay period. Images shown here were generated with MATLAB code from Moler (2011) and are representations of the fractals used in the experiment.

In each session, a set of 16 different blocks comprising all 16 possible pairings of SS and LL were presented. Each block consisted of 14 trials; a small subset of sessions (2 at 3 mg/kg MPH and paired controls) for subject GO were carried out with blocks consisting of 20 trials. The same pairing of SS and LL was tested within each block of trials. Subjects were required to complete all trials within a block before advancing to the next. The blocks were presented in either ascending or descending order of both SS magnitude and LL delay, and order was counter-balanced both between testing days and between control and MPH treatment sessions to control for a potential effect of presentation order. Unlike Kobayashi and Schultz (2008), who made trial length constant (22 s) across the entire session by adjusting the intertrial interval on each trial, we made trial length constant within each block. The intertrial period was adjusted for trials in which the subject chose the SS so that they were of equal length to those in which he chose the LL. Thus all trials within a given block had a fixed length, regardless of choice. For example, in a block where the LL had a 2-s delay, a 2-s wait time was added after a choice of the SS, and, in a block where the LL had a 16-s delay, a 16-s wait time was added after SS choices. This approach was implemented after difficulties keeping subjects engaged in the task with a uniformly long trial time (e.g., 22 s) during pilot testing.

Drug Delivery and Dosing

MPH (Sigma-Aldrich, St. Louis, MO) was administered orally (1.5, 3.0, and 6.0 mg/kg), dissolved in 0.5 ml of a mixture of grape and unsweetened cranberry juices, which were mixed in an effort to mask the taste of the drug, 45 min before the start of each experimental session. To ensure full delivery of the drug, it was administered while the subjects were in the primate chair. Juice alone was administered in a similar manner on control days. To ensure stable testing conditions from day to day, drug-treatment sessions were performed in the middle of the week, after two successful consecutive controls. Four sessions (2 ascending and 2 descending) were carried out in each animal for each MPH dose, with the exception of the 1.5-mg/kg dose for SH and the 3-mg/kg dose for MI, for which three sessions were carried out. Paired control sessions, collected the day before drug treatment, were matched for order of delay presentation. The order in which drug doses were given was pseudorandomized across testing days. Drug dosing was calculated in milligrams per kilogram, taking into account the subjects' weight the day of the experimental session. These doses were selected based on the results of a previous study of the effects of MPH on working memory performance (Rajala et al. 2012) and the pharmacokinetic work of Doerge et al. (2000), who determined that, in rhesus monkeys, an acute oral dose of 3 mg/kg MPH prepared as in the current study resulted in plasma levels similar to those measured in humans treated for ADHD.

Data Analysis and Statistical Model

The rate of discounting of delayed rewards was assumed to follow a hyperbolic function (Frederick et al. 2002)

$$\text{ν}\left(D\right)=\frac{1}{1+\text{κ}D}$$ (1)

where κ is the parameter value representing the discount rate fit to each individual subject, and D is the delay of the LL in seconds. Larger values of κ indicate a faster rate of discounting. The subjects' choice between SS and LL on each trial was used to estimate κ by maximizing the likelihood of the observed choices. The probability that the subject chooses the SS alternative is given by the logistic function

$$p(SS|R,D,\text{κ},\text{λ})=\frac{1}{1+e^{-\text{λ}\left(R-\text{ν}\left(D\right)\right)}}$$ (2)

where R is the ratio of the SS reward to the LL reward, with the function ν(D) defined by Eq. 1, and λ is the sensitivity of choice probability to the difference between the reward ratio and discounted value of the delayed reward. This is the inverse of randomness of choice, where λ = 0 would denote random choices with p(SS | R_i,D_i,κ,λ) = 0.5.

The MATLAB (The MathWorks, Natick, MA) constrained optimization routine fmincon was used to maximize the log-likelihood function

$${\displaystyle \sum _{i=1}^N}{y}_i\log \left[p\left(SS|R_i,D_i,\text{κ},\text{λ}\right)\right]+\left(1-y_i\right)\text{log}\left[1-p\left(SS|R_i,D_i,\text{κ},\text{λ}\right)\right]$$ (3)

where N is the number of trials and y_i codes [0,1] for choice. Sequential quadratic programming was used to perform the constrained optimization. The nonparametric bootstrap estimates of the standard error for κ and λ were obtained by taking 1,000 independent samples of size N with replacement of the subject's choice, y_i, and its paired covariates [R_i, D_i] (Efron and Tibshirani 1993). In addition, to investigate the possibility that the selected fractal images might yield idiosyncratic preferences unrelated to the conditioned reward value, we reassigned (permuted) reward-image mappings and refit the model to the observed responses 1,000 times to obtain an empirical null distribution for the control condition. This procedure destroys the ordinal relationship between reward levels and delay as though the subjects' choices were based on characteristics other than the conditioned reward value; λ provides a measure of how well the permuted reward-image predicts the observed choices.

A general linear model (GLM) was used to evaluate the associations between the discounting rate κ and choice sensitivity λ and explanatory variables or covariates MPH dose (0.0, 1.5, 3.0, 6.0 mg/kg) and session phase (1st half vs. 2nd half trials). To test for the possibility that other factors, such as satiety or fatigue throughout the session, could have changed κ or λ, session phase was added into the model to test for differences in κ and λ between the first and second half of the sessions. A simple linear GLM was used to analyze each of the four subjects. The two GLMs of interest were

$${\text{κ}}^{\text{subject}}={\text{β}}_0^{\text{subject}}+{\text{β}}_1^{\text{subject}}{\text{x}}_{\text{dose}}+{\text{β}}_2^{\text{subject}}{\text{x}}_{\text{phase}}+{\text{ε}}_{\text{subject}}$$ (4)

$${\text{λ}}^{\text{subject}}={\text{β}}_0^{\text{subject}}+{\text{β}}_1^{\text{subject}}{\text{x}}_{\text{dose}}+{\text{β}}_2^{\text{subject}}{\text{x}}_{\text{phase}}+{\text{ε}}_{\text{subject}}$$ (5)

where ε_subject is the random error term. The covariates χ_dose and χ_phase were coded as [0.0, 1.5, 3.0, 6.0] and [1, −1], respectively. Equations 4 and 5 were repeatedly fit to the 1,000 precalculated κ and λ paired with covariates dose [(3 controls + 3 MPH)] × phase (2) to estimate the sampling distributions of the intercept, slopes, and 99% confidence intervals. The MATLAB routine fitlm was used to specify the linear model and estimate the coefficients. It is important to note that the bootstrap replications each had nine degrees of freedom for each fit to the resampled data set. This provides an empirical sampling distribution for each coefficient, rather than depending on parametric statistics with possible assumption violations.

RESULTS

Four monkeys performed the intertemporal choice task, in which they chose between two images that represented an SS and an LL reward on each trial, under control conditions and after different oral doses of MPH. A total of 20,986 trials were collected and analyzed. Experimental-choice data were fit using a hyperbolic discounting function (Eq. 1) and a probability choice model (Eq. 2) using maximum likelihood (Eq. 3) to estimate κ and λ. Figure 2 illustrates averaged choice data (●) from the four subjects, plotted as the probability of choosing the SS as a function of both reward ratio (R; SS reward magnitude/LL reward magnitude) and LL delay (D) for the control condition (Fig. 2A) and the three doses of MPH studied (Fig. 2, B–D). The overlaid surface plots represent the joint model fitting of κ and λ to the data with warmer colors indicating a larger probability of choosing the SS (Eq. 3). Under control conditions (Fig. 2A), the probability of choosing the SS becomes larger when the magnitude of the SS reward increases, i.e., the value of R goes toward 1, and the delay of the LL becomes longer, i.e., the value of D increases. The surface plot color contours identify equal probability as R and D were varied and are related to the temporal discounting function at the indifference point of where the probability of choosing the SS and LL is equal [p(SS | R_i,D_i,κ,λ) = 0.5]. Importantly, these contours reflect both the contribution of κ and λ. The yellow color band is most salient in showing value discounting as a function of delay.

Fig. 2.

Averaged choice data and probability of choice surface plots resulting from joint estimation of free parameters κ and λ. Choice data and maximum likelihood surface fit from the 4 subjects in control (A), 1.5 mg/kg methylphenidate (MPH) (B), 3 mg/kg MPH (C), and 6 mg/kg MPH (D) treatment conditions. Probability of choice of the SS as a function of delay and SS reward ratio. Reward ratio (R) = SS reward magnitude/LL reward magnitude; ● represent averaged choice proportion across 14 trials of each block for all sessions for the control or MPH treatment conditions. Cooler colors indicate a lower probability of choosing the SS, and warmer colors indicate higher probability of choosing the SS. The surface plot color contours identify equal probability and are related to the temporal discounting function at the indifference point of p(SS | R,D,κ,λ) = 0.5, where D is the delay of LL in seconds. However, these contours reflect the contribution of both κ and λ.

Example hyperbolic discounting functions are shown in Fig. 3A, where a larger value of κ indicates a preference for smaller immediate rewards compared with larger delayed rewards, resulting in a steeper temporal discounting function. Median values of κ were computed from bootstrapped distributions and plotted as a function of MPH dose for each subject; the boxes represent the 25th and 75th percentiles with the whiskers extending to 99% (Fig. 3B). A GLM was used to further examine the variability of GLM coefficient estimates, taking into account their individual linear trends as a function of dose and phase. Although higher-order terms may have provided a better fit in some cases, the effects become increasingly difficult to interpret; consequently the model was restricted to linear terms. The coefficient estimates for the intercept, dose, and phase under control and MPH conditions are plotted for each subject (Fig. 3, C–E). Table 1 lists the median parameter estimates of the model fitting for κ and false discovery rate (FDR)-corrected P values.

Fig. 3.

Analysis of discounting constant (κ). A: examples of hyperbolic discounting functions for simulated values of κ. Discounted value of the LL [ν(D)] plotted as a function of LL delay (in s). As κ increases, the discounting function becomes steeper, and the LL loses more value with time, indicating greater impulsivity. B: discounting constant (κ) plotted as a function of MPH dose (mg/kg) for all 4 subjects. Data are presented as box plots for each subject at each dose. Points represent the median values of bootstrapped data; boxes represent 25th and 75th percentiles; whiskers extend to the 99th percentile. C–E: general linear model (GLM) parameter estimates for κ. Data are presented as box plots. Points represent the median estimates of bootstrapped data; boxes represent 25th and 75th percentiles; whiskers extend to the 99th percentile. C: estimate of intercept (β₀^subject) for the discounting constant, κ. Intercept values indicate subject's average κ value over conditions. D: estimate of slope of MPH dose effect (β₁^subject) on κ. Slope values indicate the linear trend resulting from varying doses of MPH treatment. E: estimate of the phase effect (β₂^subject), testing for a difference in κ between the 1st and 2nd half of the session. Values >0 would indicate a larger κ in the 1st compared with the 2nd half of the session.

Table 1.

Median parameter estimates and FDR-corrected P values for κ from the model: κ^subject = β₀^subject + β₁^subjectχ_dose + β₂^subjectχ_phase + ϵ^subject

κ	Median Estimate	P Value, FDR
Intercept
β0CO	0.090	0.000
β0SH	0.126	0.000
β0MI	0.117	0.000
β0GO	0.029	0.000
Dose Slope
β1CO	−0.016	0.000
β1SH	−0.009	0.000
β1MI	−0.025	0.05
β1GO	−0.005	0.000
Phase Slope
β2CO	−0.001	0.389
β2SH	0.002	0.754
β2MI	−0.005	0.081
β2GO	0.000	0.532

FDR, false discovery rate.

Under control conditions (a dose of 0 mg/kg MPH), the rate of discounting (κ) varied considerably both between and within the four subjects (Fig. 3B). The intercept shows that all four subjects differed from 0 overall (collapsed across conditions) in this measure of impulsivity, with subject GO having the smallest intercept and thus being the least impulsive, and subjects MI and SH the largest and most impulsive (Fig. 3C; Table 1). Subjects MI and SH were tested using slightly different reward magnitudes (see materials and methods for actual values); however, these differences were likely too small to account for the differences in the estimation of κ across subjects. The administration of different doses of MPH significantly reduced κ linearly across all four subjects (Fig. 3D; Table 1). The degree to which MPH reduced κ was largest for one of the most impulsive subjects, MI, and smallest for the least impulsive, GO (Fig. 3D).

To determine whether differences in task performance could have resulted from diminished motivation to perform the experimental task as the trials within a session progressed, an additional parameter, termed session phase, was added to the model to test for differences in κ between the first and second half of the session. As illustrated in Fig. 3E, no significant differences in κ between the first and second half of the experimental sessions were found for any of the subjects (Table 1).

The probability choice model (Eq. 2) also provided an estimate of the subjects' choice sensitivity (λ) to the single dimension [R − ν(D)] and characterizes the randomness of choice. For example, the smaller the value of λ, the more random the subject's choice between the SS and LL options (Fig. 4A). The same analysis was performed as described above for κ and median parameter estimates of the model fitting for λ, and FDR-corrected P values are listed in Table 2. Like κ, choice sensitivity varied under control conditions both between and within subjects (Fig. 4B). In the absence of MPH, choice sensitivity was similar among the subjects, with subject CO slightly less sensitive and subject SH slightly more sensitive (Fig. 4C). The administration of MPH reduced λ linearly in subjects CO and MI (Fig. 4D and Table 2). However, there was no significant linear trend of MPH on λ for subjects SH and GO (Fig. 4D and Table 2). Visual inspection of the dose-response functions (Fig. 4B), however, does reveal an effect of MPH on λ for GO, but it is nonlinear and thus not detected by the model. Thus, although all subjects preferred the LL more on average with increasing doses of MPH and discounted its value less with time, some subjects (CO and MI) became slightly more random in their choices. Subject GO, however, became less random at higher doses.

Fig. 4.

Analysis of choice sensitivity constant (λ). A: examples of choice functions for simulated values of λ. The probability of choice of the SS given a particular size of the SS (R) and delay of the LL (D) plotted as a function of the difference between R and the discounted value of the LL [ν(D)]. This value represents the difference between the axes R and D in Fig. 2, allowing for the collapse of 2 dimensions into a single dimension. When [R − ν(D)] = 0, the so-called indifference point, choices are maximally random [p(SS | R_i,D_i,κ,λ) = 0.5]. As choice sensitivity decreases and λ becomes smaller, choices become more random across all values of [R − ν(D)]. B: choice sensitivity constant (λ) plotted as a function of MPH dose for all 4 subjects. Details as in Fig. 3B. The empirical null distribution medians and 99% confidence intervals of λ resulting from repeated reassignments of the reward-image mappings are shown for each of the subjects under control conditions (data points jittered around 0 mg/kg for clarity). C–E: GLM parameter estimates for λ. Details as in Fig. 3, C–E. C: estimate of intercept (β₀^subject) for the discounting constant, κ. Intercept values indicate subject's average λ value over conditions. D: estimate of the slope of MPH dose effect (β₁^subject) on κ. Slope values indicate the linear trend resulting from varying doses of MPH treatment. E: estimate of the phase effect (β₂^subject), testing for a difference in κ between the 1st and 2nd half of the session.

Table 2.

Median parameter estimates and FDR-corrected P values for λ from the model: λ^subject = β₀^subject + β₁^subjectχ_dose + β₂^subjectχ_phase + ϵ^subject

λ	Median Estimate	P Value, FDR
Intercept
β0CO	5.74	0.000
β0SH	9.23	0.000
β0MI	7.29	0.000
β0GO	7.56	0.000
Dose Slope
β1CO	−0.72	0.000
β1SH	−0.20	0.066
β1MI	−1.08	0.000
β1GO	0.08	0.435
Phase Slope
β2CO	0.08	0.720
β2SH	−0.83	0.003
β2MI	1.01	0.000
β2GO	3.40	0.000

In addition, λ was generally stable for three of the four subjects between the first and second half of the experimental sessions (Fig. 4E and Table 2). Subjects SH and MI were slightly less and more sensitive in the first half of the sessions, respectively; these effects, while significant (Table 2), were very small. Subject GO, the most patient under control conditions (Fig. 3), exhibited significantly greater choice sensitivity in the first half of the experimental sessions compared with the second half (Fig. 4E and Table 2). It must be noted, however, that the bootstrapped estimates were highly variable. The empirical null distribution statistics of λ from the repeated reassignments of the reward-image mappings are shown for the control condition in Fig. 4B. All subjects' λ values approached 0 and were significantly below the true reward-image mapping. This result fails to support idiosyncratic preferences for particular fractal images unrelated to reward conditioning.

DISCUSSION

The results from this study demonstrate that low doses of MPH, one of the most common treatments for ADHD (Greenhill 2001; Solanto 1998), change the rate of temporal discounting of rhesus monkeys performing an intertemporal choice task, suggesting a reduction in impulsivity, in a dose-dependent and subject-specific manner. Therefore, in light of the fact that MPH increases levels of extracellular DA (Kojima et al. 2011; Volkow et al. 1998, 2001), these results provide further evidence for the role of the DA system in temporal discounting and impulsive behavior.

To best discuss the effects of MPH on temporal discounting, performance under control conditions must be considered first. The rate of discounting, κ, computed for each subject was different, revealing a spectrum of impulsivity consistent with previous characterizations of this trait in the same subjects using a different experimental task (Rajala et al. 2014). Specifically, three of the monkeys in this study were phenotyped as impulsive or calm based on the proportion of premature responses made in a memory-guided saccade task configured to create a high degree of uncertainty (Rajala et al. 2014). Subject GO, the least impulsive in the present study, made fewer premature responses and was classified as calm compared with subjects MI and SH, the most impulsive in this study, who made significantly more premature responses and were classified as impulsive. The λ computed for each subject also revealed individual differences in choice sensitivity under control conditions.

There was also considerable systematic variability in the bootstrapped estimates of κ and λ within each subject (Figs. 3B and 4B), which can be attributed to the sensitivity of the real-time nature of the intertemporal choice task to changes in the monkeys' internal state at the time of testing, given that they experience the delays and rewards on a trial-by-trial basis (Paloyelis et al. 2010; Reynolds 2006; Shiels et al. 2009). The use of conditioned monkey subjects requires the use of tasks with immediate reinforcement compared with humans, who can also work under hypothetical delayed-reward conditions. This sensitivity to transient changes and the current state of the subjects, however, is advantageous for this study because it allowed for the detection of the effects of the acute administration of MPH.

Because the subjects were conditioned to participate by using water as the reward, choice preferences could reflect how thirsty they were instead of how impulsive and thus could change during the course of an experimental session. Similarly, the subjects could become fatigued as the session progressed, affecting their estimates of κ and λ. The analysis of phase, comparing κ and λ between the first and second half of the experimental sessions, revealed no significant difference in the estimate of κ between the first and second half of the session for any of the subjects (Fig. 3E, Table 1), indicating that these factors did not influence the discounting of reward value. Choice sensitivity was significantly affected in some subjects, particularly GO and MI, whose choices were more sensitive (less random) in the first half of the experimental sessions (Fig. 4E and Table 2). This could indicate an effect of fatigue or satiety toward the end of the experimental sessions causing them to be less sensitive to the differences between the reward options. The effect was greatest in subject GO, the most patient of the monkeys (Fig. 3), who has also been shown to have differential methylation within the 3′-untranslated region of the DAT gene and differential DAT availability within the internal globus pallidus (Rajala et al. 2014), which could potentially contribute to these differences. Lastly, the order of presentation of the delays, ascending vs. descending, was counter-balanced and thus should not have affected choice.

MPH affected discounting rate (κ) and choice sensitivity (λ) in a dose-dependent manner. In general, it decreased κ linearly with increasing MPH dose, suggesting a reduction of impulsivity. The extent of the effects, however, varied from subject to subject, with the largest effect in the most impulsive (Fig. 3D). These results were expected because impulsive monkeys (Rajala et al. 2014), as well as adult humans diagnosed with ADHD (Dougherty et al. 1999), have greater availability of DAT, the molecule to which MPH binds, in the basal ganglia. Interestingly, studies in which extracellular levels of DA were increased by the administration of L-dihydroxyphenylalanine in humans performing a temporal discounting task (Pine et al. 2010) and the administration of GBR-12909, which binds to and blocks DAT, in monkeys performing a novelty-seeking task (Costa et al. 2014) reported increases in measures of impulsivity. However, direct comparison of these apparently contradictory findings is difficult because those drugs are not used for the treatment of ADHD (or impulsivity); thus it is unknown the extent to which DA levels are increased following the single doses that were used and how those levels compare to changes in extracellular levels of DA brought about by clinically relevant doses of psychostimulants. Consistent with this view, studies that have used Adderall, another highly prescribed psychostimulant for the treatment of ADHD (Greenhill 2001; Solanto 1998) that increases extracellular levels of DA, have shown reduced temporal discounting (de Wit et al. 2002; Wade at al. 2000).

Unlike the overall effect of MPH on κ, which was consistent across all subjects, choice sensitivity, λ, decreased linearly in two of the subjects (CO and MI) and was affected nonlinearly in subject GO (Fig. 4, B and D, and Table 2). The least impulsive, GO, exhibited an increase in choice sensitivity; subject SH showed no effect compared with control; and subjects CO and MI showed a small overall decrease at the largest dose of MPH. The lack of a specific pattern precludes speculation about the underlying mechanism for this result. An increase in the randomness of choice in subjects CO and MI brought about by higher doses of MPH could potentially have a detrimental effect on learning despite a reduction in impulsive behavior (Rajala et al. 2011).

The differences between subjects could be a consequence of differing baseline levels of DA (Peters and Buchel 2011; Volkow et al. 2008) potentially resulting from differential expression of DAT, which clears DA from the synaptic cleft. Genetic variabilities in the DAT gene (Vandenbergh et al. 1992) have been associated with impulsivity and ADHD (Cook et al. 1995; Daly et al. 1999; Gill et al. 1997; Waldman et al. 1998; Winsberg and Comings 1999), as well as differential performance on temporal discounting tasks (Boettiger et al. 2007; Paloyelis et al. 2010). In addition, genetic and epigenetic differences in the DAT gene correlate with impulsive behavior (Rajala et al. 2014), as well as DAT availability in areas of the basal ganglia (Dougherty et al. 1999; Rajala et al. 2014).

The fact that MPH blocks DAT suggests that it reduces impulsivity by altering DA processing. It must be noted, however, that MPH also blocks the reuptake of norepinephrine (NE) but to a much lesser extent compared with DA (Gatley et al. 1996). Thus, although the results of this study strongly support a role of DA in temporal discounting and impulsive behavior, they do not preclude the fact that other neurotransmitter systems, such as NE, may also be involved.

The responses of DA neurons to delayed reward stimuli decrease with delay at a rate similar to behavioral discounting; that is, they show temporal discounting in their discharges to delayed rewards (Kobayashi and Schultz 2008). These responses were documented outside of a decision-making context but still correlate with choice behavior, suggesting that the subjective value that they encode is relayed to other brain regions to be integrated into the decision-making process (Kobayashi and Schultz 2008). Dopamine neurons project to both the prefrontal cortex and the striatum, both of which contain DAT (Ciliax et al. 1999; Sesack et al. 1998) and have been implicated in temporal discounting processes (Ballard and Knutson 2009; Boettiger et al. 2007; Cardinal et al. 2001; Dalley et al. 2007; Figner et al. 2010; Hariri et al. 2006; Kable and Glimcher 2007, 2010; Kim et al. 2008; Kobayashi et al. 2007; McClure et al. 2004, 2007; Scheres et al. 2007; Wang et al. 2014).

Two alternative, but not necessarily mutually exclusive, hypotheses could account for the effects of MPH on temporal discounting and, by extension, impulsive behavior. First, MPH may reduce temporal discounting and impulsivity by increasing DA levels within the striatum, thereby strengthening the valuation of delayed rewards (Scheres et al. 2007; Shiels et al. 2009). Evidence from studies in rodents (Cardinal et al. 2001) and humans (Ballard and Knutson 2009; Hariri et al. 2006) supports this hypothesis. Alternatively, MPH may reduce impulsive choice by increasing cortical DA levels and thereby strengthening behavioral control processes, which are thought to be required to delay immediate gratification (Aron et al. 2003; Figner et al. 2010; McClure et al. 2004; Peters and Buchel 2011). Future studies are needed to determine the extent to which frontal and striatal systems are affected by MPH and the role they play in its effect on discounting and impulsivity.

Lastly, it is important to point out some limitations of this study. First, the experimental design does not allow us to unambiguously determine whether subjects transformed volume of the water reward to stimulus value. Padoa-Schioppa and Assad (2006) examined trade-offs between commodity (juice type) and quantity (volume) that allowed an inference of relative value. Although the commodity dimension was not varied, delay to the large reward was varied, and the subjects did indeed temporally discount the large reward value, which suggests that the subjects were making trade-offs based on a subjective value. Second, the possibility of a nonlinear utility function cannot be ruled out. Padoa-Schioppa and Assad (2006) assumed a linear utility function when computing commodity-quantity relative value in monkeys, and Glimcher et al. (2005) also assumed a linear utility function for computing Nash equilibria for the inspection game in monkeys. In general, the assumption of a linear utility function yielded good temporal discounting function fits; however, MPH might have differentially affected the degree of nonlinearity in an individual's utility function. We could have extended the probability choice model to include a flexible utility function within the temporal discounting function. However, adding additional nonlinear complexity to the model may have diminishing returns in terms of interpreting the effects of MPH. Despite these limitations, this study clearly shows that MPH significantly reduces temporal discounting in rhesus monkey subjects. This suggests that it may reduce impulsivity, and the results provide evidence for a role of DA-mediated temporal discounting in impulsive behavior.

GRANTS

This work was supported by NIH Grant DC003693, NSF Grant IOB-0517458, and the Wisconsin Institute for Discovery Seed Grant Program. A. Z. Rajala was supported by NIH Grant T32GM007507; R. L. Jenison was supported by NIH Grant AA018736.

DISCLOSURES

No conflicts of interest, financial or otherwise, are declared by the authors.

AUTHOR CONTRIBUTIONS

Author contributions: A.Z.R., R.L.J., and L.C.P. conception and design of research; A.Z.R., R.L.J., and L.C.P. performed experiments; A.Z.R., R.L.J., and L.C.P. analyzed data; A.Z.R., R.L.J., and L.C.P. interpreted results of experiments; A.Z.R., R.L.J., and L.C.P. prepared figures; A.Z.R., R.L.J., and L.C.P. drafted manuscript; A.Z.R., R.L.J., and L.C.P. edited and revised manuscript; A.Z.R., R.L.J., and L.C.P. approved final version of manuscript.

ENDNOTE

At the request of the author(s), readers are herein alerted to the fact that additional materials related to this manuscript may be found at the institutional website of one of the authors, which at the time of publication they indicate is: https://uwmadison.box.com/s/v3rocgd0tw5lyeopb9n7er5k4fcd1c24. These materials are not a part of this manuscript, and have not undergone peer review by the American Physiological Society (APS). APS and the journal editors take no responsibility for these materials, for the website address, or for any links to or from it.