Poke And Delayed Drink Intertemporal Choice Task (POKE‐ADDICT): An open‐source behavioral apparatus for intertemporal choice testing in rodents

Abstract

Advancements in neuroscience research present opportunities and challenges, requiring substantial resources and funding. To address this, we describe here “Poke And Delayed Drink Intertemporal Choice Task (POKE‐ADDICT)”, an open‐source, versatile, and cost‐effective apparatus for intertemporal choice testing in rodents. This allows quantification of delay discounting (DD), a cross‐species phenomenon observed in decision making which provides valuable insights into higher‐order cognitive functioning. In DD, the subjective value of a delayed reward is reduced as a function of the delay for its receipt. Using our apparatus, we implemented an effective intertemporal choice paradigm for the quantification of DD based on an adjusting delayed amount (ADA) algorithm using mango juice as a reward. Our paradigm requires limited training, a few 3D‐printed parts and inexpensive electrical components, including a Raspberry Pi control unit. Furthermore, it is compatible with several in vivo procedures and the use of nose pokes instead of levers allows for faster task learning. Besides the main application described here, the apparatus can be further extended to implement other behavioral tests and protocols, including standard operant conditioning. In conclusion, we describe a versatile and cost‐effective design based on Raspberry Pi that can support research in animal behavior, decision making and, more specifically, delay discounting.

Poke And Delayed Drink Intertemporal Choice Task (POKE‐ADDICT) is an open‐source and cost‐effective apparatus for studying delay discounting (DD) in rodents, a phenomenon relevant to decision‐making and cognitive function. This versatile tool employs an adjusting delayed amount (ADA) algorithm with mango juice rewards, requiring minimal training. It's adaptable for human translation with monetary rewards, constructed using 3D‐printed components and a Raspberry Pi control unit. This setup facilitates rapid task learning using nose pokes, compatible with various in vivo procedures, and extensible for other behavioral tests. This Raspberry Pi‐based design offers a budget‐friendly solution for research in animal behavior, decision‐making, and DD.

1. INTRODUCTION

Technological advancements in neuroscience research have brought forth both new opportunities and challenges. Investigators require costly equipment and substantial funding to maintain competitiveness. Without adequate resources, the likelihood of conducting innovative studies becomes exceedingly low. ¹ Luckily, exploiting basic and established experimental approaches, such as Skinner boxes for studying animal behavior, can still produce valuable results. Nevertheless, even a very simple behavioral setup for rodents can represent a significant expense. In addition, commercial products often do not provide the required versatility for developing custom experimental protocols. To address these issues, we present an open‐source, versatile, and cost‐effective apparatus for conducting intertemporal choice testing in rodents. Among the various higher‐order psychological functions, only a few are quantifiable and can be modeled in a mathematical manner. Delay discounting (DD) is a cross‐species phenomenon observed in decision making where the subjective value of a delayed reward is reduced as a function of the delay for its receipt. DD is classically modeled using hyperbolic ² or exponential ³ functions which describe the tendency for more remote rewards to have less and less subjective value for their recipient. ⁴ Indifference points (the amount of reward that, if given immediately, is considered equal to the delayed one) can be obtained through the intertemporal choice paradigm. Gauging the steepness of curves that mirror reward devaluation also provides valuable information in relation to clinical conditions, as DD is considered a transdiagnostic process. ⁵ Indeed, subjects affected by major depression, ⁶ schizophrenia, ⁷ borderline personality disorder, ⁸ bipolar disorder, ⁹ bulimia nervosa, ¹⁰ and binge‐eating disorder display steeper discounting curves ¹¹ ; whereas anorexic patients exhibit shallower reward devaluations. ¹² , ¹³

DD also interacts with several other cognitive domains, including intelligence, ¹⁴ working memory ¹⁵ and episodic memory. ¹⁶ However, the causal relationships between psychopathology, cognitive dimensions, and DD remain unclear, highlighting the need for further research. Furthermore, the specific brain regions involved in this process are still not well understood. ¹⁷ Considering these knowledge gaps and acknowledging the presence of DD in animals, ¹⁸ we developed a custom‐built apparatus for implementing intertemporal choice paradigms for rodents. While existing literature presents various protocols for studying rodents (e.g. Refs. [19, 20, 21, 22, 23, 24]), to date, there is no standardized procedure for obtaining the indifference points for discounting curve fitting.

Starting with the tool used for the test: some researchers employ mechanical levers ²⁵ which, however, are not only more challenging for rodents to learn to use but may also be unsuitable in situations where rodents have lower muscle strength or other motor difficulties. The alternative we also adopt is the use of nose pokes, which are simpler to implement and learn. Moreover, none of the existing protocols are affordable for laboratories with limited resources: there are several companies that produce similar tests, but they typically require several thousand dollars, whereas our test setup costs just under $100 (Table 1).

TABLE 1.

List of electronic components.

Item	Quantity	Price for unit (€)
Raspberry Pi 3 Model B+	1	38.53
LED (HL3P‐BR60‐J00DD)	2	0.52
Relay (G5LE‐1 DC3)	1	1.50
Pinch Solenoid Valve (SIRAI S104 01 EG W4 12V)	1	N.A.
Transformer (VEL05US120‐US‐JA)	1	6.11
Distance sensor (Mh‐Sensor‐Series, Flying‐Fish)	2	N.A.

Note: The table displays the components usable for constructing the apparatus. The codes and products have been extracted from the website https://www.digikey.it/. The pinch solenoid valve, which, when closed, blocks the tube and the flow of mango juice, is not available on the portal. However, it can be found on other websites for approximately € 30. While the distance sensor is available on many e‐commerce websites, a package of 10 units can be found for approximately € 12.

To address this gap, our apparatus can be easily integrated into any operating chamber, is based on Raspberry Pi technology (which is widely available), three 3D‐printed parts (two nose‐pokes and one dish for liquid delivery) and a few inexpensive electrical components (see Materials section). Mango juice was selected as reward, given its pleasant flavor and great appeal for the rats. ²⁶ Additionally, we implemented an adjusting delayed amount two‐choice task where the magnitude of the delayed reward is progressively modified, which allows for precise and reliable estimation of indifference points within a one‐month timeframe. Indeed, in many protocols, the reward is incrementally augmented in a constant manner, ²² , ²³ a method that not only requires more trials to reach the indifference point but may also not identify it with sufficient precision. In this protocol, we opted to adjust the magnitude of the delayed reward, as it allows us to consider a broader quantitative range of responses without the constraint of a fixed larger reward. Importantly, procedures using either adjusting delay or adjusting reward algorithms produced no significant differences in terms of DD quantification. ²⁷

2. METHODS

2.1. Apparatus

The behavioral experiments were conducted in a modular operant test chamber. Nevertheless, our components can be easily adapted to any kind of commercial or custom testing chamber with minimal modifications. All 3D components were designed using Autodesk Fusion 360 (https://www.autodesk.com/products/fusion‐360) and printed using a commercial low‐cost 3D printer (MINGDA D2, www.3dmingdaofficial.com) and standard polylactide (PLA); design files are available at https://github.com/jacolamanna/POKE‐ADDICT (GitHub). We never observed the rat gnawing on or licking the PLA. Therefore, this material does not interfere with the task. Electronic components are listed in Table 1. The total costs of materials (excluding the testing chamber) are below 100 €.

Two 3D‐printed nose‐poke panels (Figure 1A) were positioned in opposite bottom corners of a wall, with the reward panel (Figure 1B) placed in the center. The design of the nose‐poke panels included an upper LED (blue or red) and a distance sensor (Mh‐Sensor‐Series, Flying‐Fish) integrated into each hole (Table 1). The reward panel featured a dish for collecting mango juice and two tunnels/holes: one for liquid delivery and the other for disposal of excess juice and dish cleaning. These tunnels/holes were selectively connected to a semi‐open bottle, allowing the mango juice to flow from its container into the dish via gravity, while the excess juice was directed towards a vacuum cleaning system. The vacuum was only used at the end of the session to remove mango juice from the dish. Therefore, the noise generated by the vacuum was not an issue, as it was employed only when the rat was not present. Alternatively, a syringe can also be used to extract the liquid.

FIGURE 1.

Schematic representation of the POKE‐ADDICT apparatus. (A) Diagram of the two nose poke ports (left), showing the cavity to house the LED and the housing for the distance sensor. (B) Diagram of the reward dish. (C) The testing setup involves a Raspberry Pi 3b+ as the central component, which interfaces with the surrounding testing environment. The inputs to the system are received through two distance sensors (Mh‐Sensor‐Series, Flying‐Fish) that are seamlessly integrated within the nose poke holes. On the output side, the system employs a relay component to channel a 12 V power supply to the electrovalve. Additionally, two 3.3 LED indicators are positioned at the top of the nose‐poke holes.

The flow of juice was regulated by an electrovalve (Table 1), which enabled the administration of a specific number of drops during the trials. In a preliminary phase, we assessed the efficiency of the solenoid valve by observing the weight of 25 drops of mango juice while altering the opening frequency (total period: 12.5, 50 or 100 s). The mean weight of the fastest rate (2 drops/second, 12.5 s) was slightly lower (M = 1.34 g, SD = 0.49) than that of the second rate (M = 1.35 g, SD = 0.05; 0.5 drops/second, 50 s) and the slowest rate (M = 1.36 g, SD = 0.06; 0.25 drops/second, 100 s).

These results highlight the high reproducibility of the reward release. The opening of the solenoid valve produces a noise (“click”) for each drop. However, during the tests, the animal did not appear to be startled, but used these sounds to gauge the magnitude of the reward. If desired, a user could consider acoustically isolating it. All electronic components, such as LEDs and distance sensors, were directly connected to the Raspberry Pi 3B+ and are therefore supplied using the 3.3 V voltage generated directly by the controller, except for the electrovalve. The electrovalve, requiring a 12 V power supply, was connected to an external transformer and controlled through a Raspberry Pi pin via a Switch/Relay (Table 1). Figure 1C provides a summary of the apparatus.

2.2. Software

The experimental protocols were implemented using custom Python code that ran directly on the Raspberry Pi (all codes available at https://github.com/jacolamanna/POKE‐ADDICT). Graphs and analyses were performed using R (https://www.r‐project.org/).

2.3. Subjects

The experiments were conducted on three Sprague–Dawley rats (175–200 g upon arrival; Charles River, Italy; 6 weeks of age). When not undergoing testing protocols, the rats were housed under a 12‐h light/12‐h dark cycle, with ad libitum access to food and water, and maintained at a constant temperature of 23°C. All experiments were conducted during the light phase. All procedures were carried out in compliance with relevant guidelines and regulations. The protocols were approved by the Animal Care and Use Committee of San Raffaele Scientific Institute, in accordance with the guidelines set forth by the Italian Ministry of Health (IACUC 905). The study adheres to the ARRIVE guidelines for reporting.

3. PROTOCOL

Prior to conducting the “Adjusting Amount Reward Test,” the rats underwent training phases consisting of a “Habituation Phase” and a “Learning Phase.” The task was divided into four working days, corresponding to a week. The experimental manipulations were performed between 10 a.m. and 6 p.m. as required. Before each session, food access was restricted to approximately 80% of the rats' daily requirements.

3.1. Habituation phase

During the first week, the animals were habituated to the experimenter and the testing cage. For the first two days, each rat was handled by the experimenter and placed inside the experimental cage for several minutes. Throughout the remainder of the week, the experimenter exposed the rodent to mango juice using a plastic syringe, delivering a few milliliters into the dish. An important objective of this phase was the habituation to the reward. For all the rats in our sample, this habituation occurred within the first week.

3.2. Learning phase

In this phase, the rat learned to interact with the apparatus. We recommend turning off the lights during the sessions so that the LED light is more prominent.

During the second week, the rodents were trained in the following sequence:

1. To associate poking their nose into the port with the release of a reward.

2. To recognize activation of the LED as a cue to start the task.

3. To recognize the flashing of the LED as an indication of successful nose poke activation and anticipation of a reward.

These three learning tasks were introduced sequentially in the following order. For the first task, the experimenter used an algorithm that activated both LEDs on two nose pokes, and when the rat poked its nose, 10 drops of mango juice were suddenly released onto the plate (Figure 2A). To help the rat associate nose poke activation with reward release, a few drops of mango juice were manually placed inside the nose poke using a plastic syringe.

FIGURE 2.

Training and testing algorithms. (A–C) The flowcharts presented herein serve as schematic representations outlining the specific procedures employed during the learning phase of the experiment. (D) Within the flowchart, a preconfigured scheme known as the adjusting amount reward is delineated. This scheme governs the adjustment process for the reward magnitude. (E) An illustrative example is provided within the algorithm, specifically highlighting a step where the delay is set to 15 s. The three distinct colors utilized in the representation correspond to the three individual rats involved in the study. On the y‐axis, the computed magnitude of the reward is plotted according to the adjusting amount reward algorithm, while the x‐axis indicates the step associated with adjusting the quantity of drops. The vertical dashed line signifies the initiation of the countdown for convergence.

The second task's objective was achieved by modifying the algorithm, in which the LED of one of the two nose pokes alternately lit up. The rat learned to insert its head into the illuminated nose poke to receive the reward of 10 drops of mango juice (Figure 2B).

Regarding the last task, the training algorithm was further adjusted. One nose poke produced an immediate reward of 5 drops of mango juice, while the other nose poke led to a delayed reward of 20 drops after a 5‐s delay. When the rat selected the nose poke with the delayed reward, the LED of the chosen nose poke started flashing throughout the delay, while the LED of the other nose poke turned off. We recommend alternating between the nose pokes, for example, after every 10 trials, to eliminate any potential preference bias (Figure 2C). Before proceeding to the next step, it is important to ensure that the rat completes each task within a short timeframe, specifically within 60 s.

3.3. Adjusting delayed amount reward test

From the third week onwards, we implemented the testing protocols using the adjusting delayed amount two‐choice task. Each rat went through several testing blocks, where one nose poke was linked to an immediate reward (“I”) of 4 drops and the other nose poke was associated with a delayed yet larger reward (“D”) of 9 drops. If the rat chose the nose poke for an immediate reward, the port LED flashed 4 times at 10 Hz and then turned off, releasing 4 drops of mango juice. When the rat selected the nose poke for a delayed reward, the LED on the inactive nose poke turned off, and the LED on the active port flashed at 10 Hz during the entire delay period. After the time delay elapsed, the LED was turned off, and the reward was released. Each block consisted of three forced‐alternate trials (D, I, and D) followed by four free‐choice trials. The interval between each trial was 10 s. We also considered keeping the interval between two successive nose pokes fixed. However, we still achieved an increase in the indifference point as the delay increased, and the indifference point was reached in less time. At the end of each block, we considered the outcomes of the last three trials. If the rat consistently chose the immediate reward for all three trials, we added 5 drops to the delayed choice. If not, we initiated the “countdown” by subtracting 2 drops from the delayed choice. The nose poke positions were then inverted to prevent a preference bias.

For the subsequent blocks after the countdown had begun, we again considered the outcomes of the last three trials. If the rat chose the immediate reward at least twice, we added 1 drop to the delayed choice in the next block. Otherwise, we subtracted 1 drop. In the following blocks, the same procedure was applied, adding or subtracting 1 drop based on the outcomes of the last three trials. At this point, the indifference point was determined (Figure 2D). Once the animal reached the indifference point, we randomly changed the delay and repeated the process. The randomization also serves to avoid the satiety effect, although we rarely observed the rat leaving juice in the dish. We tested three different delays: 5, 15, and 25 s. An example of this process is illustrated in Figure 2E.

After testing a specific delay, the rat was typically replaced in the testing cage. Each delay was normally tested once per day.

Prior to the third week of Adjusting Delayed Amount Reward Test, the animals were not yet expert in the task, and the results were not considered reliable. Data collection started when the rat consistently approached the nose poke within a minute, which usually occurred during the second or third week (Figure 3A). However, rather than linking the start of data collection to the number of weeks spent on the task, we recommend looking at the reaction times. Once the LEDs were activated, the rat had 300 s to perform the nose poke. In the initial weeks, this occasionally happened.

FIGURE 3.

Reaction times and indifference points. (A) Reaction times. The response times exhibited by the rodents over the initial three‐week period demonstrate a notable interindividual variability. However, it is observed that across all three rats, there is a consistent reduction in reaction times during the third week. (B) Indifference points. The plot displays the indifference points of a rat estimated with POKE‐ADDICT. A total of 26 observations (red points), calculated from the third week onwards, demonstrates that as the delay increases, the estimated indifference point in mango juice drops also increases. The mean and standard error is depicted in black. (C) Discounting curves. The plot illustrates estimated indifferent points obtained using POKE‐ADDICT within a single experimental session and fitted discounting curves. As the delay increases, estimated subjective value reduces, as expected. Each distinct color within the graph represents the performance of an individual rat. The subjective value is calculated by dividing 4 by the indifference point of the delay. We fitted the subjective value points with a hyperbolic function (Subjective value = 1/(1 + k × Delay)). Indeed, the data reflects the typical hyperbolic pattern found for delay discounting. The point obtained when the delay is equal to 0, while not obtained experimentally, is still useful for fitting the curve.

4. DISCUSSION

Investigating behavioral performance is a challenging task due to the difficulty in accurately quantifying the outcomes. Here we present a behavioral protocol to assess delay discounting, a psychological domain that can be quantitatively measured, ²⁸ and provide all details for the construction of an inexpensive behavior measurement apparatus. The examination of intertemporal choice performance not only enhances our understanding of the brain circuits involved but also assists the study of animal models of psychiatric disorders. Importantly, our apparatus is compatible with various in vivo procedures, including electrophysiological, surgical, and pharmacological interventions. By utilizing the modular operant test chamber and employing mango juice, we not only reduced costs and building complexity, but also achieved accurate and high‐resolution quantification of the reward. This enhances reliability when identifying indifference points. Additionally, the use of nose pokes instead of levers enables faster task learning, given the contingency of food reinforcement. ²⁹ We also observed that the distance sensor integrated into the 3D‐printed nose poke was highly reliable. It never activated randomly but only when the rat inserted its head into the hole. Although several papers have employed adaptive algorithms, ²¹ , ²⁴ the algorithm we propose exhibits high precision in determining the indifference points. We decided to switch between nose pokes after each block. This adds complexity, as the rat may not easily adjust to the changes, but we did this to prevent the rat from favoring one port. Additionally, we chose not to include a phase with varying reward amounts with the same delay, as we wanted the rat to grasp the idea of a smaller immediate reward and a larger delayed one from the beginning. Nevertheless, future users can opt to include these phases.

Our paper also aims to inspire the creation of cost‐effective, adaptable experimental equipment. By utilizing Raspberry Pi, researchers can achieve a high level of customization, incorporating new tools, sensors, and tailored algorithms for different behavioral protocols. These systems are budget friendly, making competitive experiments accessible even to low‐budget laboratories. A limitation of our study is the small number of animals used for testing the apparatus. However, similar works have also employed a limited number of rodents. ²⁰ Indeed, by observing the behavior of tested rats (Figure 3B,C), we confirmed the well‐established finding that increasing reward delays decreases the subjective value of the reward. ³⁰

AUTHOR CONTRIBUTIONS

Andrea Stefano Moro, Mattia Ferro, and Jacopo Lamanna: conceptualized the experiments. Andrea Stefano Moro and Alessia Seccia: performed the experiments. Andrea Stefano Moro and Jacopo Lamanna: analyzed the data. Andrea Stefano Moro: writing–original draft preparation. Andrea Stefano Moro and Jacopo Lamanna: writing–review and editing. Mattia Ferro, Daniele Saccenti, Jacopo Lamanna, and Antonio Malgaroli: supervision. All authors have read and agreed to the published version of the manuscript.

FUNDING INFORMATION

The author(s) reported there is no funding associated with the work featured in this article.

CONFLICT OF INTEREST STATEMENT

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

ETHICS APPROVAL

The protocols were approved by the Animal Care and Use Committee of San Raffaele Scientific Institute, in accordance with the guidelines set forth by the Italian Ministry of Health (IACUC 905).

CONSENT FOR PUBLICATION

Not applicable.

Moro AS, Saccenti D, Seccia A, Ferro M, Malgaroli A, Lamanna J. Poke And Delayed Drink Intertemporal Choice Task (POKE‐ADDICT): An open‐source behavioral apparatus for intertemporal choice testing in rodents. Anim Models Exp Med. 2023;6:619‐626. doi: 10.1002/ame2.12366

DATA AVAILABILITY STATEMENT

The codes and 3D component designs are available at https://github.com/jacolamanna/POKE‐ADDICT. The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.