Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2026 Sep 24.
Published in final edited form as: Eur J Neurosci. 2025 Apr;61(7):e70104. doi: 10.1111/ejn.70104

Dynamically Adjusting Intertemporal Choice Task in Rodents

Morteza Salimi 1,2, Milad Nazari 3,4,5, Miranda Francoeur Koloski 1,2,6, Samuel A Barnes 1,2, Jonathan Mishler 1,2, Sahar Jomehpour 2, Dhakshin S Ramanathan 1,2,6,7
PMCID: PMC13600693  NIHMSID: NIHMS2185318  PMID: 40219999

Abstract

Temporal discounting refers to the tendency for immediate rewards over delayed ones, assessed through intertemporal choice tasks where subjects choose between immediate low-value or delayed high-value rewards. Traditional rodent tasks often require extensive pre-task training, introducing species-specific biases and thus lower translational utility. We present a novel dynamically adjusting intertemporal choice task, where the delay for a large reward adjusts trial-by-trial based on prior choices. Choosing the large reward increases its delay by 500 ms, while selecting the small reward decreases the large reward delay by 500 ms. In eight Long–Evans rats tested across 50 days, key behavioral measures stabilized early, including the average delay and preference for the large reward. However, training enhanced behavioral flexibility, allowing rats to optimize rewards over time. This task enables rapid assessment of delay preferences while also revealing cognitive flexibility, offering significant advantages for investigating decision-making that may be relevant to real-world behaviors.

Keywords: decision-making, delay discounting, dynamic, intertemporal

1 |. Introduction

Delay discounting is a phenomenon whereby rewards lose value as the delay to their delivery increases. This process has been extensively investigated in both humans and animals using intertemporal choice tasks (Ainslie 1974; Shamosh and Gray 2008). This process is typically studied using an intertemporal choice task in which subjects make repeated decisions between rewards of different magnitudes and delays (Ainslie 1974; Evenden and Ryan 1996). In human studies, intertemporal choice tasks are used to explore impaired and risky decision-making associated with substance abuse (Story et al. 2014) and various other psychiatric disorders (Evenden and Ryan 1996). In such studies, an inability to wait for larger rewards reflects a more impulsive phenotype. Neuroimaging studies in humans indicate that the ventral striatum is particularly sensitive to immediate reward magnitude and strengthens preferences for immediate, rather than delayed rewards, while the medial prefrontal cortex calculates the subjective value by balancing reward size against delay (Kable and Glimcher 2007; Ballard and Knutson 2009).

In a typical rodent version of an intertemporal choice task, animals must choose between a small reward delivered immediately or a large reward delivered after a delay (Story et al. 2014; Winstanley et al. 2004; Kobayashi and Schultz 2008; Roesch and Bryden 2011; Lefner et al. 2021). Subjects discount the large reward in a hyperbolic fashion as the delay increases (Story et al. 2014; Kable and Glimcher 2007; Lefner et al. 2021; St Onge and Floresco 2009; Roesch and Bryden 2011). Most tasks use a fixed large reward delay or predetermined delays that change progressively throughout the task (Winstanley et al. 2004; St Onge and Floresco 2009; Madden and Johnson 2010; Ryan et al. 2016). Subjects typically show a preference shift from larger-later to smaller-sooner rewards as delays increase, with the rate of this shift (discounting) varying across individuals and being influenced by factors such as reward magnitude, type, and context. We have previously characterized the effects of stimulant medications on performance on this type of task (Koloski et al. 2024a). We found that stimulant medications affect preference by shifting rats to be less impulsive and flattening the discounting curve. (Koloski et al. 2024a). We also identified beta-oscillations in key reward-regions like orbitofrontal cortex and ventral striatum as markers of reward-processing that were sensitive to delays on this task (Koloski et al. 2024b). These findings support prior research highlighting the role of orbitofrontal cortex, prelimbic cortex, basolateral amygdala, and nucleus accumbens in encoding reward magnitude and delay (Winstanley et al. 2004; Lefner et al. 2021; Roesch and Bryden 2011).

The fixed delays typically used in these tasks (including our own) have several limitations. (1) Fixed delays fail to capture the dynamic nature of decision-making, limiting their translational potential to real-world scenarios where choices adapt based on changing circumstances (Luhmann et al. 2008; Worthy et al. 2013). (2) Prior work with fixed delay tasks have demonstrated that the order of delay presentation within a session can introduce biases in choice behavior (Craig et al. 2014; Slezak and Anderson 2009). This effect confounds the interpretation and translatability of discounting behavior observed, as human versions rarely use such fixed interval designs. (3) Extended training requirements: The necessity for extensive training with typical fixed- choice intertemporal choice tasks prior to behavioral stabilization can lead to over-training/habitual responses that further reduce the ecological validity of the findings (humans are typically tested just once). These limitations highlight the need for more flexible, rapidly learned and individualized approaches for studying delay discounting in rodents that better reflect the adaptive nature of real-world decision-making processes.

To address these limitations, several recent studies have developed tasks with dynamic, adjusting delays. In these tasks, the delay associated with the larger reward adapts based on recent choice history, typically increasing after choices of larger rewards and decreasing after choices of smaller rewards (Krebs and Anderson 2012; Wahab et al. 2018; McLaughlin and Redish 2023). In Krebs and Anderson (2012) and Wahab et al. (2018) studies, the adjusting procedure used larger temporal jumps after a bock of repeated high-value choices unlike our implementation including smaller adjustments after every trial. McLaughlin and Redish (2023) developed a spatial delay discounting task with dynamically adjusting delays applied on a trial by trial design, but their task was unique in that it was developed within a spatial maze rather than a more classic operant box. In summary, while other dynamically adjusting intertemporal choice tasks have been developed, they used large delay increments, inflexible delay adjustments, or were implemented within the context of a spatial maze. Our study introduces a dynamic intertemporal choice task that implements trial-by-trial delay adjustments with small increments/decrements in a standard operant chamber. We assayed behavior across extended training to better understand changes associated with early and late/over-training on this paradigm. We made several observations of this version of the temporal discounting task. (1) The delay at which animals stabilized was reached early in training and did not change significantly even with extended training. (2) Animals continue to evolve their behavioral strategy with extended learning, resulting in an overall increase in their ability to acquire rewards. While we do not directly compare this training strategy with others, our results suggest the possibility that the training strategy used here can reduce overall training time to achieve a stable delay choice and may capture naturalistic decision-making processes with translational relevance without confounding influence of spatial demands.

2 |. Materials and Methods

2.1 |. Ethics Statement

This research was conducted in strict accordance with the Guide for the Care and Use of Laboratory Animals of the National Institutes of Health. The protocol was approved by the San Diego VA Medical Center Institutional Animal Care and Use Committee (IACUC, Protocol Number A21–012).

2.2 |. Subjects

8 (5 female 3 male) experimentally naïve Long Evans rats obtained from Charles River Laboratories were utilized for this study. Upon arrival, the rats were approximately one month old and weighed 150 g. They underwent a 2-weeks habituation period before the commencement of training. All rats were pair-housed in standard rat cages (10 × 10.75 × 19.5 in.; Allentown, NJ, USA) and maintained on a 12-h light/dark cycle (lights on at 6 a.m.), with testing conducted during the light cycle. Food was provided ad libitum, while water access was restricted to 24 h before the test day and unrestricted on nontraining days (weekends) to sustain motivation for water rewards used in the operant task.

2.3 |. Behavioral Apparatus

Behavioral experiments were conducted in a custom designed operant chamber (16 × 12 × 16 in.). This chamber included five nose ports (NPs), each featuring an LED, an infrared (IR) sensor, and a metal cannula used to administer water rewards (Figure 1A). Additionally, the chamber included two speakers for auditory cues, a house light, and five peristaltic stepper motors/water pumps for water delivery into the nose ports. Control of the chamber was managed using Simulink (MathWorks) installed directly onto a Raspberry Pi system. Further details regarding the design, operation, and software control of this chamber can be found in previous publications (Buscher et al. 2020). We used the same custom boxes to run fixed-delay discounting procedures previously (Koloski et al. 2024b).

FIGURE 1 |.

FIGURE 1 |

Open in a new tab

Task description. (A) The operant chamber comprises five nose ports (NPs) equipped with LED, IR sensor, and water reward cannula, alongside auditory speakers, house lights, and water delivery pumps. (B) Schematic dynamic delay task paradigm: Each day involved trials with a fixed 500 ms delay for small rewards and a adjusting delay for larger rewards based on previous choices. Rats initiate the task by poking into NP3 when its LED is on, then choose between NP2 and NP4 for small (10 μL) and large rewards (30 μL), respectively. The task was self-paced within a 60 min/day. Failure to initiate or make a choice resulted in prompts to commence the next trial. (C) Representative sample of one early (day 3) versus one late (day 45) day from one rat. Rats' choices in each trial plotted as a function of delay. In the early days, the rat exhibited less behavioral flexibility and more persevering behaviors (either staying with the high or the low reward choice for multiple trials).

2.4 |. Behavioral Procedure

Rats were approximately 6 months old at the start of the behavioral testing.

2.4.1 |. Pretraining

Prior to training on the behavioral task, rats underwent a pretraining period (~7–10 days). During phase 1 of pre-training, rats learned that a response to an LED-illuminated nose port triggered a 20 μL water reward. In phase 2, they were required to first enter the middle nose port (NP3) to “start” the trial, after which NP2 and NP4 both illuminated. A response in NP2 triggered an immediate (500 ms) small reward (10 μL) in NP2 or a delayed (2 s) large reward (30 μL) in NP4. Rats progressed to the dynamically adjusting intertemporal choice task once they consistently performed at least 100 trials within 60 min. Previous studies from our lab using this same box/set-up have shown that rats are sensitive to the reward amounts chosen; when delays are matched, rats strongly (> 80%) prefer the large reward choice (Koloski et al. 2024a).

2.4.2 |. Dynamically Adjusting Intertemporal Choice Task

Animals were trained 5 days/week for 50 days/rat (Figure 1A,B). Each training day began with the houselights dimmed and the middle nose port (NP3) LED on. Rats initiated each trial by poking into NP3 after which the LED on NP3 extinguished, and LEDs in the adjacent NP2 and NP4 turned on, indicating the two choices: an immediate/small reward (NP2, a 10 μL reward with a delay of 500 ms) or a delayed large reward (NP4, a 30 μL reward starting with a 2 s delay at the beginning of the task). The large reward delay was then changed on each subsequent trial based on the rats' immediate choice history: a large reward choice increased the delay for that reward on a subsequent trial by +500 ms and a small reward choice decreased the delay for the high reward stimulus by −500 ms, with a minimum floor of 2 s delay for the large reward trial. Water rewards were always delivered from NP3 at a rate of 10 μL/s. The stepper motor made an audible sound indicating the start and length of the water delivery, providing an instantaneous audio cue that was time-locked to reward delivery. A 5 s intertrial interval occurred after water delivery was finished prior to NP3 turning on to indicate the start of the next trial. Houselights were turned on during the ITI period. The task was self-paced, but failure to initiate the next trial or make a choice within 60s resulted in the flashing of house lights to orient animals to the task. Animals were given 60 min of training each day and were allowed as many trials as they could perform within that fixed time window. The testing continued for up to 50 days.

Unlike traditional delay discounting paradigms that require multi-phase training with strict performance criteria, our task allows animals to begin dynamic delay training immediately after pretraining. Rather than progressing through fixed delay increments, rats experience trial-by-trial delay adjustments based on their own choices, enabling a more naturalistic and individualized learning process while minimizing exclusion rates (Foscue et al. 2012).

2.5 |. Data Analysis

2.5.1 |. Behavioral Parameters

To analyze the rats' behavior during the learning process, we extracted various parameters from the behavioral data in each training day. These parameters include the following: Trial Num: The total number of trials in each 60 min/day.

Alternating Choice Rate: The rate at which the rat alternate between the large and small reward choice during each training. It was calculated as the number of switches divided by the number of trials.

AlternatingChoiceRate=numberofswitchnumberoftrials

Total Reward: The total amount of reward received by the rat. It is calculated as the sum of rewards from large and small reward choice trials. Rats receive 3× more water on large reward trials (30 μL vs. 10 μL).

TotalReward=(Numberoflargerewardtrials)*3+(numberofsmallrewardtrials)

Large reward preference: The ratio of trials in which the rat chose the large reward choice to the total number of trials.

Largerewardpreference=numberoflargetrialsnumberoftrials

Average Delay: The average value (s) of the delay in all trials for that day.

We utilized custom Matlab code designed to extract detailed trial-by-trial data from the behavioral days. This code enabled the extraction of various key parameters essential for understanding the rats' decision-making processes. Specifically, the extracted parameters included the choice patterns across trials, response times, reward amounts obtained per trial, and the progression of delay adjustments based on the rats' choices.

2.5.2 |. Binning

After calculating these parameters for each day, we smoothed across days by averaging the parameters in each subject over three consecutive days of performance to create a “day” parameter that was used for further analysis. To balance temporal resolution with data stability, we grouped every 3 days into a “smoothed day” rather than analyzing individual days. Given the dynamic nature of our task, where rats continuously adjust their delay preferences, analyzing single days could introduce variability that may obscure broader trends in decision-making. By averaging data across 3 days of behavior, we enhance stability while maintaining sufficient resolution to capture meaningful changes in choice behavior over time. Days with fewer than 10 trials were excluded before analyses (< 3% of the total). Finally, we divided the days into four phases (quartiles) to observe how the dynamics of behavior change over both shorter and longer time spans: Q1 (Days 1–4), Q2 (days 5–8), Q3 (days 9–12), and Q4 (days 13–16) reflecting a total of 48 days of training/rat.

2.5.3 |. Statistical Tests

All graphical data representations were created using GraphPad Prism v.10. Data was analyzed in IBM SPSS v.28 (New York, USA) as a repeated measures ANOVA with Sidak's posttest A p-value less than 0.05 was considered significant. Next, we utilized a linear regression model to examine the correlation between pairs of parameters. For this analysis, we employed the Matlab function fitlm, which provides regression coefficients and p-value.

3 |. Results

3.1 |. Behavioral Performance and reward Acquisition Improved Over Time

The chamber and task structure shown in Figure 1A,B. Rats were given choices between a small water reward (10 μL) with a fixed 500 ms delay and a large water reward (30 μL) with a variable delay that adjusted based on their immediate choice history. Selection of the high-value delayed reward would increase the delay by an additional 500 ms for the next trial; selection of the low-value immediate reward would reduce the delay of the high-reward option by 500 ms on the next trial. There was a lower limit of 2 s for the high-reward condition but no upper limit. Delays on the low-reward trials were not adjusted. Figure 1C shows an example training day from one rat early and late in training. With repeated training, we observed that animal's behavior changed. For example, even with similar numbers of trials, animals were more likely to start alternating choice rather than perseveratively responding to either the high or low reward option prior to alternating (Figure 1C, insert). We quantify this as the normalized “alternating choice” behavior, which is defined as the proportion of trials in which subjects alternate between the two options. This behavior seemed to allow for an increased number of rewards as well.

To better understand behavioral performance on this task, we calculated the total number of trials performed, the alternating choice rate, total reward consumed, average delay, and preference for the large reward on each behavioral training day. For each subject, we averaged data across 3 “days” increments to form a single day parameter (updated every 3 days) and split days into four quartiles (4 days in each) to examine trends across training. Two statistical models were specified. In the first model, we analyzed changes across the 16 days (e.g., focusing on the main effect of days). In the second model, we analyzed time across two factors: a “quartile” factor focused on slow changes in behavior, and an intraquartile day factor that included the 4 days within each quartile, allowing us to better understand the time-course of changes.

We found that rats increased the total number of trials performed across days (F(4, 31) = 3.05, p = 0.027). There was a significant main effect quartile (F(2, 15) = 6.027, p = 0.01) and a main effect of intraquartile day (F(2, 15) = 4.46, p = 0.02), but no quartile × intraquartile day interaction (F(2, 15) = 0.98, p = 0.45; Figure 2). No sex effect was found (F(1, 6) = 3.2, p = 0.12). We next analyzed the normalized alternating choice rate (Figure 2). The alternating choice rate significantly increased across days (F(4, 31) = 5.29, p = 0.004). The follow-up model showed a significant main effect of quartiles (F(2, 15) = 7.91 p = 0.006) and main effect of intraquartile day (F(2, 15) = 5.47, p = 0.008), but no interaction between quartiles and intraquartile day (F(2, 15) = 0.51, p = 0.75). Post hoc analyses showed that the alternating rate significantly increased from quartiles 1 and 4 (p = 0.04) and day 1 versus day 4 within quartile 1 (p < 0.018). This suggests a rapid and then slower increase in the rate of alternating choice behaviors with training. We did not find an effect of sex for alternating rate (F(1, 6) = 0.0011, p = 0.98). The total amount of rewards collected increased over days (Figure 2C; F(4, 31) = 3.61, p = 0.012). Quartile (F(2, 15) = 9.03, p = 0.0004) and within-quartile day (F(2, 15) = 5.29, p = 0.016) were both significant, but there was no quartile × day interaction (F(2, 15) = 0.93, p = 0.45). Post hoc analyses demonstrated a significant increase in total reward from quartile 1 to quartile 4 (p = 0.03) and within quartiles 1 (day 2 > day 1, p = 0.017) and 2 (day 4 > day 2, p = 0.019). No sex effect for was found (F(1, 6) = 4.8, p = 0.07).

FIGURE 2 |.

FIGURE 2 |

Open in a new tab

Behavioral assessment for task performance over time. Trend of performance across all days. Each set of 3 days were interpolated into one which were then segregated into distinct quartiles represented by different colored rectangles for better visual representation. Repeated measurement analyses indicated a significant increase in trend of number of trials F(4, 31) = 3.05, p = 0.027; no sex differences were found. Alternating choice rate (defined as the number of alternation trials divided by the total number of trials) indicated significant effect F(4, 31) = 5.29, p = 0.004. Significant quartile effect and day effect were found: F(2, 15) = 7.91 p = 0.006 and F(2, 15) = 5.47, p = 0.008, respectively. Within-quartile analysis indicated that a significant increased from quartiles 1 and 4 (p = 0.04) and day 1 versus day 4 within quartile 1 (p = 0.018). We did not find any sex differences. Total reward showed a significant effect of training days, F(4, 31) = 3.61, p = 0.012. We found a significant increase in total reward from quartile 1 to quartile 4 (p = 0.03), within quartile 1 (p = 0.017), and quartile 2 (p = 0.019). No notable changes were found in average delay or large choice preference over days. No sex differences were found. Data were analyzed using ANOVA followed by Sidak's correction for multiple comparisons, mean ± SEM; *p < 0.05.

We did not find any significant changes in average delay (Figure 2, F(2, 15) = 0.44, p = 0.8; no sex difference, F(1, 6) = 0.23, p = 0.64) or large choice preference (Figure 2, F(2, 15) = 0.39, p = 0.7; no sex difference, F(1, 6) = 0.76, p = 0.41) across days, indicating that rats find their preferred delay length early in training day even while other task features (like alternating choice rate and reward) continue to evolve over time. In summary, our behavioral analyses demonstrate that rats improved task performance over time measured by reward consumption by increasing number of trials and switch rate across days but showed quick stabilization in average delay and large choice preference.

3.2 |. Consistent Alternating Choice Behavior and Large Choice Preference at Last Quartile

Our initial results suggested that the classic aspects of temporal discounting (particularly the “average” delay that subjects typically gravitated towards on the task and the preference for the large reward option) stabilizes early and remains stable across many days of training, which suggests that our first goal (to develop a task that may be useful for assessing intertemporal choice with minimal training) was achieved. Interestingly, however, we found that certain key aspects of behavior, particularly choice alternation and reward acquisition, improve with sustained training. To better understand the relationship between alternating choice, delay and reward, we performed a few follow-u p analyses. First, we analyzed the relationship between alternating choice rate as a function of delay during the initial training (first quartile) versus the advanced training phase (fourth quartile). We hypothesized that when animals respond perseveratively, alternating choice rate would decrease with increasing delays (animals begin to avoid the large delay choice), whereas when animals respond more flexibly there will be no relationship between the two (animals learn that alternating choice can result in a stabilization of the delay). We found, as hypothesized, a significant effect of early versus late training (Figure 3A, F(1, 14) = 6.03, p = 0.028) and a delay × training interaction (F(1, 9) = 2.19, p = 0.11). Post hoc analyses demonstrated a significant decrease in the alternating choice rate with increasing delay during the first quartile (F(9, 63) = 8.06, p = 0.002). By contrast, the alternating choice rate in the last quartile remained stable across delays (F(9, 63) = 0.55, p = 0.56). A significant difference in alternating choice rate between early and late training begins at delays exceeding 8 s (Figure 3A), which is close to the average delay animals prefer in both early and late training days.

FIGURE 3 |.

FIGURE 3 |

Open in a new tab

Alternating choice rate and large choice preference as a function of delay. Panel (A) illustrates the alternating choice rate across delays in quartile 1 (blue) and quartile 4 (red). We found a significant decrease in the alternating choice rate as a function of delay in quartile 1 (F(9, 63) = 8.06, p = 0.002), while in quartile 4, the alternating rate remained unchanged (F(9, 63) = 0.55, p = 0.56). (B) Linear regression between switch rate and average delay in each quartile 1 (blue) and quartile 4 (red) day. There was a significant positive correlation between alternating rate and average delay in quartile 1 (R = −0.62, F = 55.2, p < 0.0001) but not in quartile 4 (R = −0.07, F = 0.38, p = 0.53). Linear regression between delay and large choice preference in each (C) quartile 1 and (D) quartile 4. There was a significant negative correlation between delay and large choice preference in quartile 1 (R = −0.61, F = 48.5, p < 0.0001) while quartile 4 did not show any change (R = −0.05, F = 0.2, p = 0.65).

We next explored the relationship between alternating choice behaviors and delays in a linear regression model. Consistent with the above analysis, in early training days, there was a significant negative correlation between the alternating choice rate and average delay (Figure 3B, R = −0.62, F = 55.2, p < 0.0001). However, there was no correlation between delay and alternating choice rate in late training days (R = −0.07, F = 0.38, p = 0.53). To probe this further, we assessed the relationship between alternating choice rate and large-choice preference. There was a significant negative correlation between alternating choice rate and large reward preference in the first quartile (R = −0.61, F = 48.5, p < 0.0001). However, in quartile 4, there was no significant relationship between the two (R = −0.05, F = 0.2, p = 0.65 (see Figure 3C,D). All these data points suggest that as rats learn the task, they respond more flexibly and less perseveratively at higher delay lengths that they would tend to avoid earlier in training.

3.3 |. Increasing the Alternating Choice Rate Resulted in a Larger Amount of Rewards

If alternating choice behavior represented more flexible decision-making, we hypothesized that this flexible responding must serve some function, such as increasing reward acquisition. To test this hypothesis, we conducted a linear regression analysis to compare alternating choice rate and total reward across days. Our results revealed a significant positive correlation between the alternating choice rate within a day and the total amount of reward received (R = 0.22, F = 20.06, p < 0.0001; Figure 4). These findings suggest that the rats' adaptive choice alternation directly influenced their overall reward acquisition, supporting the role of alternating choice as a cognitive flexible strategy for maximizing rewards.

FIGURE 4 |.

FIGURE 4 |

Open in a new tab

Correlation between alternating choice rate and amount of reward. This figure depicts the linear regression analysis between switch rate and total reward. The Pearson correlation coefficient (r) indicated a significant positive correlation (R = 0.22, F = 20.06, p < 0.0001), suggesting that as the switch rate increases, there was a corresponding increase in total reward.

4 |. Discussion

In this study, we developed a novel, dynamically adjusting intertemporal choice task. The mechanics of the task design allowed us to rapidly identify an average delay for the high-reward choice that rats would settle on given the different reward amounts (30 μL vs. 10 μL of water). Importantly, the overall preference for the high reward choice and the average delay achieved across the task did not change significantly even with extended training, suggesting this task may be an efficient way of identifying an important aspect of intertemporal choice behavior. The average delay achieved on this task, which was around 8 s on average, likely reflects something close to the “50/50” point that could be identified with more classic version of an intertemporal choice task. Indeed, in our prior study that used a more standard fixed delay version of the task, we found that a 50/50 point (the point at which rats would select the low and high rewards about 50% of the time) was between 5–10 s for the majority of rats studied (Koloski et al., 2024c). We also found that even while the overall intertemporal delay on this task stayed about the same, their strategy shifted significantly as they learned the task. Animals increased their cognitive flexibility and decreased perseverative responding indicated by more alternations between high and low rewards, especially at higher delays with training, and this correlated with overall greater reward acquisition.

There are several advantages of this task compared to standard intertemporal choice tasks that have been employed in rodents. One key advantage of our dynamic delay discounting task is its reduced training time compared to traditional paradigms, which often require rigid performance thresholds and block-based delay progression before testing can begin (Foscue et al. 2012). The task described here allows for rapid identification less than 2 weeks and stabilization of delay preferences, allowing for early testing of interventions that might affect behavior. By eliminating the need for prolonged training, our approach improves accessibility while preserving the ability to measure intertemporal choice behavior with high precision and limits the potential confounds associated with over-training. The second advantage of this task is the variability of trials across days. Standard versions of these tasks implemented in rodents typically use a stereotyped pattern of delays (fixed number of trials at one delay followed by a similar number of trials at the next delay, and so on). This standardized approach, while facilitating behavioral acquisition, also influences behavioral responding in a way that calls into question what is truly being measured (Wahab et al. 2018). In our task, there is no systematic/standardized approach to delays. Rather, the trial-by-trial choice of the rats determines the delay structures on a particular training day, thus minimizing this confound. This version of the intertemporal choice task does have one important disadvantage compared to fixed-delay ascending/descending tasks. Standard versions of intertemporal choice tasks result in a negative delay-discounting curve (a plot demonstrating that with increased delay there is lower selection of the high-reward choice). The shape and slope of this curve is an important parameter that is often measured. This adaptive version of the task does not readily offer a discounting curve/slope that can be readily measured, and instead, we are more reliant on the average delay achieved as a metric of impulsivity. While this measure does seem similar to the same one acquired on a standard version of the task, we cannot say with certainty that it is identical.

As noted above, one interesting and unexpected finding was that although the delay preference of each rat was established early in training and remained consistent, flexible alternating choice behaviors increase with extended training. Although previous adjusting-choice paradigms have demonstrated that animals adapt their choices based on prior outcomes and establish stable delay preferences, our paradigm uniquely highlights an active and flexible choice strategy. The progressive increase in alternating choice behavior suggests that rats actively refined their decision-making approach rather than passively stabilizing at a fixed preference point. This dynamic behavioral flexibility enabled rats to optimize reward acquisition by alternating their choices between the high and low reward options, which effectively stabilized the delay to align with their preferred individual delay for the larger reward. The correlation between increased alternating choice rates and total reward (Figure 4) not only highlights a progressive enhancement in task efficiency but also illustrates the rats' capacity to adapt and refine their decision-making strategies (employing more flexible decision-making) in our paradigm. This adaptability directly addresses the limitations of fixed-delay tasks, and demonstrates the benefits of a behavioral paradigm that engages more complex cognitive processes and offers a more nuanced measure of learning and decision-making (Shamosh and Gray 2008; Odum et al. 2020; Mazur 1998). This version of the task may mirror some real-world decision-making environments, where delays and choices are fluid and subject to individual's adaptive strategies rather than fixed by external constraints. This responsiveness to individual differences not only provides deeper insights into the decision-making processes but also overcomes the rigidity of traditional models that often fail to reflect the nuanced nature of cognitive behavior in dynamic settings.

Despite unique findings noted above, this study has some limitations. (1) While we did not find significant sex differences in task performance, the small and unbalanced sample size may have limited our ability to detect subtle sex-related effects. Future studies with larger and more balanced groups are needed to determine whether sex influences decision-making strategies in this dynamic delay discounting task. (2) We did not perform any pharmacological manipulations in this study. Previous research has demonstrated how stimulant drugs influence delay discounting behavior. Krebs and Anderson (2012) examined the effects of D-amphetamine (0.03–1.80 mg/kg, i.p.) on impulsive choice in rats and found that its impact depended on baseline impulsivity levels increasing impulsive choice in low-impulsivity rats while reducing it in high-impulsivity rats. This suggests that stimulant effects on decision-making may be influenced by pre-existing behavioral traits (Krebs and Anderson 2012). Similarly, Wahab et al. (2018) investigated the role of dopaminergic compounds in a self-adjusting delay discounting task. Their results showed that amphetamine (0.3–1 mg/kg, i.p.) increased preference for delayed rewards, indicating reduced impulsivity, whereas D1 and D2 dopamine receptor antagonists (SCH23390, Raclopride) decreased preference for delayed rewards, highlighting the role of dopamine transmission in intertemporal choice regulation. These studies emphasize the potential for pharmacological interventions to shape impulsive choice behavior (Wahab et al. 2018). Future research using our dynamic delay discounting paradigm could explore whether these drug effects extend to more flexible and self-regulated decision-making strategies and would shed greater light on how behavior on our task compares with others. (3) One potential concern in our task design is that the non-delay trials are shorter than delay trials, thereby allowing rats to complete more total trials when favoring the small reward. While this is theoretically possible, our data does not support that animals were biased by this strategy: we found no significant differences in total trial numbers across days or quartiles, and rats exhibited increasing alternating behavior over time, indicating engagement in flexible decision-making rather than maximizing trial counts. Importantly, modifying the task to equalize trial durations (e.g., by extending the ITI for small reward choices) would eliminate the cost-of-waiting decision variable, fundamentally altering the nature of the task and preventing the assessment of adaptive delay-based decision-making strategies. While this consideration may be relevant for future work, our design prioritizes capturing the trade-off between immediate and delayed rewards, which is central to understanding intertemporal choice.

In conclusion, our novel dynamic delay discounting task represents a significant advancement in studying intertemporal choice and decision-making processes in rodent models. By adjusting delays based on immediate choice history, this paradigm captures individualized delay preferences and adaptive decision-making strategies more effectively than traditional fixed-delay tasks. Future research should leverage this task to investigate individual differences in delay discounting and to examine how various manipulations, such as pharmaceutical or chemogenetic interventions, impact choice behavior. This dynamic approach will not only deepen our understanding of the cognitive mechanisms and neural circuits involved in strategic decision-making but will also improve the ecological validity of delay discounting studies and offer valuable insights for both animal and human research.

Funding:

This work was supported by Veterans Affairs San Diego Healthcare System (IK2BX006125), Foundation for the National Institutes of Health (T32-MH018300), National Institute of Mental Health (R01MH123650), and Burroughs Wellcome Fund (1015644).

Abbreviations:

ANOVA

analysis of variance

IBM

International Business Machines

IR

infrared

ITI

intertrial interval

IACUC

Institutional Animal Care and Use Committee

LED

light-emitting diode

NP

nose port

p

p-value (statistical significance)

R

Pearson's correlation coefficient

SPSS

Statistical Package for the Social Sciences

VA

Veterans Affairs

Footnotes

Conflicts of Interest

The authors declare no conflicts of interest.

Peer Review

The peer review history for this article is available at https://www.webofscience.com/api/gateway/wos/peer-review/10.1111/ejn.70104.

Data Availability Statement

Data and code are available at https://figshare.com/s/97c7b6298d9b5a480873.

References

  1. Ainslie GW 1974. “Impulse Control in Pigeons.” Journal of the Experimental Analysis of Behavior 21, no. 3: 485–489. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Ballard K, and Knutson B. 2009. “Dissociable Neural Representations of Future Reward Magnitude and Delay During Temporal Discounting.” NeuroImage 45, no. 1: 143–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Buscher N, Ojeda A, Francoeur M, et al. 2020. “Open-Source Raspberry Pi-Based Operant box for Translational Behavioral Testing in Rodents.” Journal of Neuroscience Methods 342: 108761. [DOI] [PubMed] [Google Scholar]
  4. Craig AR, Maxfield AD, Stein JS, Renda CR, and Madden GJ. 2014. “Do the Adjusting-Delay and Increasing-Delay Tasks Measure the Same Construct: Delay Discounting?” Behavioural Pharmacology 25, no. 4: 306–315. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Evenden JL, and Ryan CN. 1996. “The Pharmacology of Impulsive Behaviour in Rats: The Effects of Drugs on Response Choice With Varying Delays of Reinforcement.” Psychopharmacology 128, no. 2: 161–170. [DOI] [PubMed] [Google Scholar]
  6. Foscue EP, Wood KN, and Schramm-Sapyta NL. 2012. “Characterization of a Semi-Rapid Method for Assessing Delay Discounting in Rodents.” Pharmacology, Biochemistry, and Behavior 101, no. 2: 187–192. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Kable JW, and Glimcher PW. 2007. “The Neural Correlates of Subjective Value During Intertemporal Choice.” Nature Neuroscience 10, no. 12: 1625–1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Kobayashi S, and Schultz W. 2008. “Influence of Reward Delays on Responses of Dopamine Neurons.” Journal of Neuroscience 28, no. 31: 7837–7846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Koloski MF, Hulyalkar S, Barnes SA, Mishra J, and Ramanathan DS. 2024. “Cortico-Striatal Beta Oscillations as a Reward-Related Signal.” Cognitive, Affective, & Behavioral Neuroscience 24: 839–859. 10.3758/s13415-024-01208-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Koloski MF, Terry A, Lee N, and Ramanathan DS. 2024. “Methylphenidate, but Not Citalopram, Decreases Impulsive Choice in Rats Performing a Temporal Discounting Task.” Frontiers in Psychiatry 15: 1385502. 10.3389/fpsyt.2024.1385502. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Krebs CA, and Anderson KG. 2012. “Preference Reversals and Effects of D-Amphetamine on Delay Discounting in Rats.” Behavioural Pharmacology 23, no. 3: 228–240. [DOI] [PubMed] [Google Scholar]
  12. Lefner MJ, Magnon AP, Gutierrez JM, Lopez MR, and Wanat MJ. 2021. “Delays to Reward Delivery Enhance the Preference for an Initially Less Desirable Option: Role for the Basolateral Amygdala and Retrosplenial Cortex.” Journal of Neuroscience 41, no. 35: 7461–7478. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Luhmann CC, Chun MM, Yi DJ, Lee D, and Wang XJ. 2008. “Neural Dissociation of Delay and Uncertainty in Intertemporal Choice.” Journal of Neuroscience 28, no. 53: 14459–14466. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Madden GJ, and Johnson PS. 2010. “A Delay-Discounting Primer.” In Impulsivity: The Behavioral and Neurological Science of Discounting, 11–37. Washington, DC, US: American Psychological Association. [Google Scholar]
  15. Mazur JE 1998. [cited 2024 Sep 1. “Choice and Self-Control.” In Handbook of Research Methods in Human Operant Behavior [Internet], edited by Lattal KA and Perone M, 131–161. Boston, MA: Springer US. 10.1007/978-1-4899-1947-2_5. [DOI] [Google Scholar]
  16. McLaughlin AE, and Redish AD. 2023. “Optogenetic Disruption of the Prelimbic Cortex Alters Long-Term Decision Strategy but Not Valuation on a Spatial Delay Discounting Task.” Neurobiology of Learning and Memory 200: 107734. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Odum AL, Becker RJ, Haynes JM, et al. 2020. “Delay Discounting of Different Outcomes: Review and Theory.” Journal of the Experimental Analysis of Behavior 113, no. 3: 657–679. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Roesch MR, and Bryden DW. 2011. “Impact of Size and Delay on Neural Activity in the Rat Limbic Corticostriatal System.” Frontiers in Neuroscience 5: 130. 10.3389/fnins.2011.00130. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Ryan NP, Catroppa C, Godfrey C, et al. 2016. “Social Dysfunction After Pediatric Traumatic Brain Injury: A Translational Perspective.” Neuroscience and Biobehavioral Reviews 64: 196–214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Shamosh NA, and Gray JR. 2008. Jul. “Delay Discounting and Intelligence: A Meta-Analysis.” Intelligence 36, no. 4: 289–305. [Google Scholar]
  21. Slezak JM, and Anderson KG. 2009. “Effects of Variable Training, Signaled and Unsignaled Delays, and d-Amphetamine on Delay-Discounting Functions.” Behavioural Pharmacology 20, no. 5–6: 424–436. [DOI] [PubMed] [Google Scholar]
  22. St Onge JR, and Floresco SB. 2009. Feb. “Dopaminergic Modulation of Risk-Based Decision Making.” Neuropsychopharmacology 34, no. 3: 681–697. [DOI] [PubMed] [Google Scholar]
  23. Story G, Vlaev I, Seymour B, Darzi A, and Dolan R. 2014. “Does Temporal Discounting Explain Unhealthy Behavior? A Systematic Review and Reinforcement Learning Perspective.” Frontiers in Behavioral Neuroscience 8: 76. 10.3389/fnbeh.2014.00076. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Wahab M, Panlilio LV, and Solinas M. 2018. “An Improved Within-Session Self-Adjusting Delay Discounting Procedure for the Study of Choice Impulsivity in Rats.” Psychopharmacology 235, no. 7: 2123–2135. [DOI] [PubMed] [Google Scholar]
  25. Winstanley CA, Theobald DEH, Cardinal RN, and Robbins TW. 2004. “Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice.” Journal of Neuroscience 24, no. 20: 4718–4722. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Worthy DA, Hawthorne MJ, and Otto AR. 2013. “Heterogeneity of Strategy Use in the Iowa Gambling Task: A Comparison of Win-Stay/Lose-Shift and Reinforcement Learning Models.” Psychonomic Bulletin & Review 20, no. 2: 364–371. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

Data and code are available at https://figshare.com/s/97c7b6298d9b5a480873.

RESOURCES