Motivational state, reward value, and Pavlovian cues differentially affect skilled forelimb grasping in rats

Abstract

Motor skills represent high-precision movements performed at optimal speed and accuracy. Such motor skills are learned with practice over time. Besides practice, effects of motivation have also been shown to influence speed and accuracy of movements, suggesting that fast movements are performed to maximize gained reward over time as noted in previous studies. In rodents, skilled motor performance has been successfully modeled with the skilled grasping task, in which animals use their forepaw to grasp for sugar pellet rewards through a narrow window. Using sugar pellets, the skilled grasping task is inherently tied to motivation processes. In the present study, we performed three experiments modulating animals’ motivation during skilled grasping by changing the motivational state, presenting different reward value ratios, and displaying Pavlovian stimuli. We found in all three studies that motivation affected the speed of skilled grasping movements, with the strongest effects seen due to motivational state and reward value. Furthermore, accuracy of the movement, measured in success rate, showed a strong dependence on motivational state as well. Pavlovian cues had only minor effects on skilled grasping, but results indicate an inverse Pavlovian-instrumental transfer effect on movement speed. These findings have broad implications considering the increasing use of skilled grasping in studies of motor system structure, function, and recovery after injuries.

A substantial part of the movement repertoire of humans and other higher order mammals consists of motor skills learned over time. Many motor skills in higher mammals are dependent on upper limb control and dexterity (Lemon 2008). Skilled grasping is the most widely used paradigm to study motor skill learning and performance in rodents (Whishaw and Pellis 1990; Buitrago et al. 2004; Hermer-Vazquez and Moshtagh 2009; Azim et al. 2014; Esposito et al. 2014). In this task, animals use their forepaw to reach through a small opening for a sugar pellet positioned on a shelf or pedestal. Even though the dexterity of rodents is limited when compared with primates (Lemon 2008), rodent skilled grasping shows strong similarities to human skilled reaching behavior (Whishaw et al. 1992; Sacrey et al. 2009; Klein et al. 2012a). Skilled grasping protocols require several days of training to reach plateau performance (Buitrago et al. 2004), and were shown to be accompanied by motor cortex plasticity—including increased spine turnover and dendritic branching (Greenough et al. 1985; Xu et al. 2009; Fu et al. 2012), long-term potentiation (LTP) (Rioult-Pedotti et al. 1998, 2000; Zemmar et al. 2014), and major shifts in cortical motor maps (Kleim et al. 1998, 2004). Protein synthesis has been shown to be required for skilled grasping learning and for neuronal plasticity in motor cortex (Kleim et al. 2003; Luft et al. 2004) and striatum (Wachter et al. 2010). By using sugar pellets as reward for successful performance, skilled grasping is hypothesized to be inherently tied to motivational processes.

Here, we investigated the effect of three different but linked parameters, known to affect motivation, on skilled grasping learning and performance: internal motivational state (drive, hunger), hedonic value of reward (taste, size), and incentive salience of reward-associated stimuli (Pavlovian cues, context).

Early theories of motivation emphasized the role of drive, an internal motivational state such as hunger, as a concept to explain increased eating behavior of animals under food deprivation (Hull 1943; Bolles 1962; Savory 1988). The food reward itself was initially thought to simply reduce motivational state and thus the increased activity (Hull 1943). However, it became apparent that specific motivational states (such as hunger or thirst) mostly led to rather specific behavioral actions as opposed to increasing general activity per se (Campbell et al. 1966). This observation suggested motivational state has a specific enhancing effect on the behavior that has been learned to lead to reward (Changizi et al. 2002). Furthermore, it has been shown that such a specific invigorating effect of motivational state is dependent on an animal's experience with the reward in the motivational state (incentive learning) (Balleine 1992). This invigorating effect of motivational state is clearly evident in increased effort and speed/rate of instrumental behavior in classical experiments (Teitelbaum 1966) and is well-known to depend on mesolimbic dopamine (Salamone et al. 1994, 2003; Salamone and Correa 2009; Baldo et al. 2013; Sternson et al. 2013).

The reward itself has further been shown to have a hedonic value (Berridge 1991), which can be independent of but is often modulated by motivational state (Cabanac 1971, 1979; Cabanac 1992; Yeomans and Mobini 2006). Such hedonic value of the reward is also known to exert an invigorating effect on instrumental behavior (Hodos 1961). Hedonic value of reward has been shown to mainly depend on opioid neurotransmission (Kelley et al. 2005; Woolley et al. 2006; Berridge et al. 2009).

Early discoveries that animals ran faster down a runway when they were anticipating a bigger reward (Crespi 1942) led to the additional concept of an invigorating effect of reward expectancy, i.e., incentive motivation (Hull 1952). Such invigorating effects on instrumental behavior were also shown to be induced by Pavlovian stimuli (Estes 1948), which was later formalized in Pavlovian-instrumental transfer (PIT) experiments (Rescorla and Solomon 1967; Lolordo 1971; Lovibond 1983; Holmes et al. 2010). The Pavlovian stimuli are thought to obtain an own incentive salience and value (Robinson et al. 2014), a process known to depend on striatal dopamine and opioid signaling (Berridge and Robinson 1998; Lex and Hauber 2008; Shiflett and Balleine 2011; Pecina and Berridge 2013).

Despite the seeming separation of the described processes, the concepts are strongly interconnected, as internal motivational states can modulate the hedonic value of rewards and the incentive salience of Pavlovian stimuli (Cabanac 1979, 1992; Berridge 1991; Epstein et al. 2003; Raynor and Epstein 2003; Yeomans and Mobini 2006; Polivy et al. 2008; Cameron et al. 2014). Furthermore, hedonic value of a reward is thought to constantly update incentive salience of Pavlovian stimuli over the course of training (Holland 2004). Motivation to perform an instrumental action, i.e., willingness to work for a reward, thus depends on all three parameters (Hodos 1961; Epstein et al. 2003; Raynor and Epstein 2003; Barbano and Cador 2005; Corbit et al. 2007).

A further distinction is often made between the “directing” (choosing an action that will lead to a reward) and the “activating” or “energizing” (how much vigor or effort is put into an action) effects of motivation on behavior (Salamone 1988; Salamone and Correa 2012; Bailey et al. 2015); the latter being strongly modulated by dopamine (Salamone et al. 2003; Salamone and Correa 2009; Panigrahi et al. 2015). It has been proposed that vigor is determined by the net reward to be gained in a specific behavioral task, and is modulated by tonic dopamine (Niv et al. 2007). Studying reaction time tasks and velocity of goal-directed saccadic eye movements, such general reward effects have been shown (Guitart-Masip et al. 2011; Haith et al. 2012). Studies in rodents further supported this concept and showed that net expected rewards, rather than immediate reward history, determine performance vigor (Wang et al. 2013). However, when the value of the upcoming reward was known beforehand, faster saccades were performed in high reward value trials compared with low reward value or no reward trials (Takikawa et al. 2002; Xu-Wilson et al. 2009; Shadmehr et al. 2010; Chen et al. 2013). Similar results were observed in an experiment studying reaction times of wrist movements (Opris et al. 2011). Such trial-by-trial variations of movement speed have been proposed to depend on phasic dopamine bursts instead (Ljungberg et al. 1992; Shadmehr 2010). Taken together, it is suggested that in situations where the obtainable reward value is high, fast movements are induced. Such modulation of speed increases the maximally obtained rewards per time unit and lead to most cost-efficient behavior (Niv et al. 2007; Shadmehr 2010; Haith et al. 2012).

A similar increase in movement speed over time is also characteristic for motor skill learning (Krakauer and Mazzoni 2011). Learning a new motor skill is generally slow and requires multiple training sessions to increase performance and improve the quality of the movement (Dayan and Cohen 2011). Typically, motor skill learning is accompanied by a shift in the speed–accuracy trade-off curve (Willingham 1998; Krakauer and Mazzoni 2011; Shmuelof and Krakauer 2011) which allows movements to occur faster with both increased accuracy (Fitts 1954; Reis et al. 2009; Dayan and Cohen 2011) and reduced variability (Shmuelof et al. 2012). These characteristics indicate an evolutionary role of motor skill learning in maximizing rewards through faster and more accurate goal-directed movements (Crossman 1958; Shadmehr 2010; Hikosaka et al. 2013). Some human studies have shown that rewards can have profound effects on motor skill learning (Johannsen 1962; Sugawara et al. 2012; Wulf et al. 2014) and performance vigor (Meyniel and Pessiglione 2014), which was even reflected in corticospinal excitability (Klein et al. 2012b). Variability in skilled arm movements has been shown to depend on recent success history, with an unsuccessful trial increasing the variability in the next trial, inducing some sort of exploration (Pekny et al. 2015). Additionally, a role of dopamine has also been suggested in motor skill learning in humans (Butefisch et al. 2002; Sawaki et al. 2002; Floel et al. 2005a; Palminteri et al. 2011; Lissek et al. 2014). And recent studies show an effect of motivation and dopamine on rehabilitation of motor function after a central nervous system injury (Floel et al. 2005b; Nishimura et al. 2011; Lohse et al. 2013).

Reports showing motivational effects on motor skill learning in animals are however, scarce. In mice, a study using the staircase pellet grasping task (Montoya et al. 1991) found an effect of reward flavor on grasping performance as measured by a number of successfully retrieved pellets, but did not measure learning rates (Baird et al. 2001). Recent experiments manipulating dopamine in motor cortex have shown effects on learning rates and cortex plasticity (Molina-Luna et al. 2009; Hosp et al. 2011; Hosp and Luft 2013; Rioult-Pedotti et al. 2015). Using enriched housing, which has been previously shown to strongly influence dopaminergic signaling in the prefrontal cortex (Zhu et al. 2005; Del Arco et al. 2007; Segovia et al. 2008) and nucleus accumbens (Segovia et al. 2010), we have previously found a highly increased learning rate in the skilled grasping compared with standard housed animals (Starkey et al. 2014). Furthermore, amphetamine administration has been found to boost recovery of skilled grasping after motor cortex stroke (Feeney et al. 1982; Adkins and Jones 2005; Gilmour et al. 2005; Ramic et al. 2006; Papadopoulos et al. 2009; Wolf et al. 2014). Finally two studies investigating cortical plasticity after skilled grasping learning have used control groups in which animals were prevented from obtaining sugar pellets, but performed targeted reaching movements. These studies found reduced spine turnover (Xu et al. 2009), synapse formation, and cortical map plasticity (Kleim et al. 2004) in unrewarded controls. Together, these findings indicate a role of motivational processes in motor skill learning and cortical plasticity in rodents.

Here, we performed three experiments specifically investigating effects of different aspects of motivation on skilled grasping learning and performance using success scores and detailed kinematic analysis of grasping movements measuring changes in speed and accuracy over time. The role of motivation was investigated using different motivational states (food restriction schedules), reward values (sugar pellet flavor preferences), and Pavlovian incentives (Pavlovian instrumental transfer).

Results

We used an instrumental chain of action version of the skilled grasping task (Buitrago et al. 2004), where animals needed to perform a nose-poke (distal response) followed by approach behavior toward the pellet receptacle (proximal response). This sequence was pretrained in the instrumental training phase (Fig. 1A). During skilled grasping learning, animals had to perform the nose-poke response to gain access to the sugar pellet and perform the new skilled motor action (skilled proximal response, i.e., grasping) (Fig. 1B). We measured response latencies for both actions and interpreted those as indicators for vigor or willingness to perform (i.e., motivation). An additional measure of motivation was taken in a progressive ratio test requiring incrementally increasing numbers of nose-pokes to open the sliding door. We measured the maximal number of nose-pokes animals were willing to do (breaking point) to gain access to the sugar pellet. Motor skill learning and performance was measured with success rate of first attempt sugar pellet retrieval and kinematic measurements of the paw (Fig. 1C; speed and variability of targeting).

Figure 1.

Behavioral setup and kinematic analysis. (A) Experimental setup during instrumental training. A nose-poke response led to a sugar pellet released into a receptacle inside the behavior box. (B) Experimental setup during skilled grasping training. A nose-poke response opened a sliding door in front of the window. A sugar pellet was positioned on a pedestal outside the box. Individual animals retrieved pellets by reaching through the window with its forepaw. (C) Scheme of detailed analysis of grasping kinematics. Lateral paw trajectories were averaged (black line) and used to calculate average speed of each movement and variability in targeting (SD, white line).

Experiment 1: hunger as a motivational state strongly affects performance of skilled grasping

In this study, we investigated whether motivational state induced by food restriction affects motor skill learning and performance using success rate and kinematic parameters (Fig. 1C; movement speed and variability in targeting) of the skilled grasping task. After completion of instrumental pretraining, animals were split into three groups receiving either 40%, 60%, or 80% of their previously determined ad libitum food intake for the skilled grasping training (Fig. 2A). After acquisition of the skilled grasping over nine sessions, a 30-min long last session was performed to measure the performance under satiating conditions, and thus investigate the effect of motivational state independently from learning. Measurements of instrumental response latencies, as well as breaking point in a progressive ratio test were used to rate the animal's motivational state at the end of the experiment (Fig. 2A).

Figure 2.

Effects of motivational state as induced by food restriction on skilled instrumental grasping. (A) Experimental design. For skilled grasping training, animals were split into three groups receiving either 40%, 60%, or 80% of ad libitum food levels. (B) Success score as shown by percentage success on first attempt in skilled grasping over training time (time: F_(7,147) = 8.79, P < 0.0001; group: F_(2,21) = 2.94, P = 0.07). The 40% and 60% food level groups showed a significant increase in performance over time between day 2 and days 5–9 (40%: max. P < 0.01, 60%: max. P < 0.05) and performed significantly better than the 80% group at the end of training (day 7 and 9, P < 0.05). (C) Number of nose-pokes required to open the door to the sugar pellet was successively increased in the progressive ratio test which showed strong differences in breaking points between groups (F_(2,21) = 7.00, P = 0.005). The 80% group had a significantly lower breaking point than the 40% (P < 0.01) and 60% (P < 0.05) groups. (D) Variability of paw trajectories in targeting the pellet was strongly reduced with training (time: F_(3,63) = 15.09, P < 0.0001) in all groups from days 2 to 8 (40%: P < 0.01, 60%: P < 0.05, 80%: P < 0.01) and in the 80% group from days 2 to 5 (P < 0.05). In the long last session, the 80% groups showed an increased variability compared with day 8 (P < 0.05), while the 40% still showed a reduced variability compared with the beginning of training (day 2) (P < 0.05). (E) Average speed of paw trajectories showed a strong increase with training (time: F_(3,63) = 21.55, P < 0.0001), which was strongest between days 2 and 8 for all groups (P < 0.01), but also significant between days 2 and 5 (60%: P < 0.05, 80%: P < 0.05). In the long last session, the 60% (P < 0.01) and the 80% (P < 0.05) groups showed significantly slower grasps than on day 8, while the 40% group still grasped faster than at the beginning of training (day 2) (P < 0.05). (F) Latency to complete a grasping attempt after a nose-poke response has triggered door opening for the first 10 and last 10 trials in the long last session. Grasping latencies were significantly different between groups and from beginning to end of the session (group: F_(2,21) = 6.10, P = 0.008; time: F_(1,21) = 12.67, P = 0.002). For the 60% and 80% groups, the grasping latency significantly increased from the first 10 to the last 10 trials of the session (P < 0.05), while the 40% group showed no increase and had smaller latencies for the last 10 grasps than the 80% group (P < 0.01). (G) Latency to initiate a new trial with a nose-poke response for the first 10 and last 10 trials in the long last session was not different between groups. (H) Grasping speed was reduced over time within the long last session (time: F_(1,21) = 21.71, P = 0.0001). Both the 40% (P < 0.05) and the 60% groups (P < 0.01) reduced their movement speed over the course of the session. (I) Success rate of pellet retrieval changed significantly over time in the long last session (time: F_(1,21) = 9.98, P = 0.005), with the 60% group showing a strong decrease in success from the first to the last 10 trials (P < 0.01). All data are shown as means ± SEM, n = 8 animals for all groups.

Grasping accuracy measured by pellet retrieval success rate showed a strong effect of training time (daily sessions), with the 40% and 60% food level groups showing a significant increase over time in a multiple comparison test between day 2 and days 5–9 (Fig. 2B; 40%: P < 0.01, 60%: P < 0.05). The 80% group only reached a success rate of 18.1 ± 4.1, showing a flat learning curve, while both the 40% and 60% group reached rates of ∼33% (40%: 33.1 ± 3.9, 60%: 33.4 ± 5.8). These results suggest that a high motivational state increased learning of the skilled motor task. To control for the motivational state of the animal, a progressive ratio test, requiring an incrementally increasing number of nose-pokes to give access to the pellet, was performed at the end of the experiment (Fig. 2C). The 80% group had a significantly lower breaking point than the 40% (P < 0.01) and 60% (P < 0.05) groups, suggesting that groups with the highest success rate in skilled grasping were also the most motivated to perform the instrumental task.

To further investigate effects of motivational state on skilled grasping, we analyzed additional parameters of motor learning and performance—i.e., variability and speed of the grasping movement of the animal. The variability of the targeting movement was strongly reduced over time, from day 2 to day 8, independent of motivational state (Fig. 2D). Similarly, the average speed of each grasp increased with training in all groups (from day 2 to day 8) (Fig. 2E). The lack of a group effect in these measurements suggests that successful retrieval of pellets, which was better mastered by 40% and 60% groups, requires additional skills such as accuracy besides consistent and fast movements. The high motivational state likely therefore had its main effect on improving such accuracy.

To investigate the direct effect of motivation on performance after learning, a 30-min long last session was performed which allowed satiation over time. This satiation effect is shown by the latency to complete a grasping attempt after a nose-poke response had opened the sliding door (Fig. 2F). For less food restricted groups, this grasping latency increased significantly from the first 10 to the last 10 trials of this long session (P < 0.05). Grasping latency of the 40% group did not change over time and was lower by the end of this session than that of the 80% group (P < 0.01). The latency to initiate a new trial with a nose-poke did not show such a strong effect of motivational state (Fig. 2G). There was no significant difference between first and last trials or between groups in the nose-poke response latency (Fig. 2G). This finding is in accordance with the literature, as it has been shown that shifts in motivational state reveal their effect mainly on the proximal action of an action chain in the absence of incentive learning (Corbit and Balleine 2003).

In this long session, variability in targeting movements was increased when compared with performance on day 8 for the 80% group (Fig. 2D; P < 0.05), indicating a direct effect of motivation on performance. A similar effect was also seen in the speed of movement, which was decreased for both the 60% (P < 0.01) and the 80% (P < 0.05) groups in the long session compared with day 8, while the 40% group kept on producing fast grasping movements (Fig. 2E). To directly compare the effect of change in motivational state on movement speed, the first 10 and last 10 grasps of the 30-min long session were also analyzed separately. Figure 2H shows a strong effect of time on grasping speed during the long session, and both 40% (P < 0.05) and 60% (P < 0.01) groups started the session with fast movements which slowed down by the end of the 30 min. In contrast, the 80% group did not show such a decrease, and performed slow movements throughout the session (Fig. 2H). To test whether learning of accurate pellet retrieval skill was showing a persistent higher level of performance when tested under satiating conditions, the success rates during the long session were also compared between the first and last 10 trials (Fig. 2I). Success rate, similar to speed of each motion, showed a strong effect of time, with the 60% group significantly dropping in success over the course of the session (P < 0.01). The 40% group did not show a significant drop in performance, and the 80% group remained at its low level of grasping success throughout the session (Fig. 2I). Taken together, these observations suggest a nonlinear relationship between motivational state (measured in grasping latency) and skilled movement parameters (speed and accuracy—i.e., success rate). At a high motivational state (40% group) satiation was not apparent, but speed of the skilled movement decreased over the course of the session, while success rate was unchanged. At an intermediate motivational state (60% group), satiation was strong, leading to a strong reduction in movement speed as well as accuracy over time. At a low motivational state (80% group), a strong satiation effect was detected, however, neither movement speed, nor success rate were further decreased by this motivational shift.

In summary, motivation was successfully modulated by food restriction as evident from a strong effect on performance of instrumental aspects of the skilled grasping task. Speed and movement variability changes over time were not affected by motivational state; however, movement accuracy was increased over time for highly motivated groups. Results from the longer last training session show that there was a strong nonlinear but direct effect of motivational state on speed and accuracy performance of skilled grasping, which potentially underlies this effect.

Experiment 2: reward value affects skilled grasping speed

Next we investigated the effect of different reward values on skilled grasping learning and performance using differently flavored sugar pellets as rewards for animals in a low motivational state. A flavor preference test using four different flavors was performed to determine individual flavor preferences. Of all 12 animals used, 11 preferred caramel-flavored pellets, and one animal banana-flavored pellets. All 12 animals disliked lemon-flavored pellets the most, which were thus used as a nonpreferred flavor. Subsequently, animals were trained in the instrumental task, where each nose-poke was followed by one of two different tone cues signaling delivery of a pellet of either preferred or nonpreferred flavor. To increase the range of reward value, a third cue was used to signal an aversive quinine-flavored pellet. In the next phase, animals were split into two groups and trained to grasp by presenting them mainly their preferred or mainly their nonpreferred reward in a random order. Additional quinine-flavored pellets were presented to all animals at a low probability to test for effects of aversive tastes (Fig. 3A). We first analyzed differences between the two groups to investigate whether a higher ratio of preferred rewards led to increased learning of skilled grasping. As in experiment 1, performance of skilled grasping was measured with the first attempt success rate, variability in targeting, and movement speed. We also performed a progressive ratio test at the end of the experiment to measure motivation with instrumental performance.

Figure 3.

Reward value effects on skilled grasping. (A) Experimental design. During instrumental training, tone cues signaled appearance of a preferred (black), nonpreferred (gray), or quinine-flavored (white) pellet. During skilled grasping training, the preferred group had a high probability of being presented with their preferred flavor and paired tone, whereas the nonpreferred group had a high probability of being presented with their nonpreferred flavor and paired tone. Quinine-flavored pellets were interspersed at a low probability for both groups. (B) Both groups showed an increase in success rate over time in the skilled grasping task (time: F_(7,70) = 4.63, P = 0.0003, group: F_(1,10) = 0.34, P = 0.57), which was significant for the preferred group when tested by multiple comparison analysis (day 2 versus 5 and 8: P < 0.01, day 2 versus 7 and 9: P < 0.05). (C) Progressive ratio test did not show a difference in nose-poke response breaking point between groups (P = 0.22). N = 6 rats for both groups. (D) Variability of targeting trajectory was reduced with training in both groups (time: F_(2,18) = 11.21, P = 0.0007) with a significant reduction from day 2 to 8 (preferred: P < 0.05, nonpreferred: P < 0.01). (E) Average grasping speed showed a significant effect of training (time: F_(2,18) = 3.65, P = 0.047), with a strong increase between days 2 and 8 for the preferred group (P < 0.05). (D,E) Preferred n = 6, nonpreferred n = 5. (F–H) Data averaged according to reward flavor type, n = 12 matched values for all types, except speed: n = 10, six preferred group, four nonpreferred group. (F) Latency to complete a grasping attempt after a nose-poke response showed a strong effect of reward type (F_(2,22) = 42.42, P < 0.0001). Over the course of grasping training, grasping latency was significantly slower for quinine-flavored pellets than preferred or nonpreferred ones (P < 0.01). (G) Increase in movement speed with training was significant and affected by reward type, showing a significant interaction (time: F_(2,18) = 5.54, P = 0.013; reward type: F_(2,18) = 8.23, P = 0.003; interaction: F_(4,36) = 2.80, P = 0.04). Grasping speed only significantly increased with training for preferred (P < 0.01) and nonpreferred pellets (P < 0.05), leading to significantly faster speeds by days 5 and 8 compared with quinine-flavored pellets (P < 0.01). (H) Successful pellet retrieval showed a strong increase with training and a significant effect of reward type (time: F_(2,22) = 12.91, P = 0.0002; reward type: F_(2,22) = 4.66, P = 0.021). Success rate for all reward types increased significantly from day 2 to 8 (P < 0.01), and from day 2 to 5 for nonpreferred flavored pellets (P < 0.01). All data are shown as means ± SEM.

Figure 3B shows success rates of both groups over all training days. There was a significant overall effect of time; however, using Bonferroni-corrected multiple comparison analysis over all sessions, only the preferred group showed a significant increase in performance compared with day 2 on days 5, 7, 8, and 9 (P < 0.01, P < 0.05). Nevertheless, both groups had similar success rates at the end of training (preferred flavor group: 40.4 ± 7.7%, nonpreferred flavor group: 35.8 ± 7.5%) and no difference between groups was found. The progressive ratio test, requiring incrementally increasing numbers of nose-pokes to open the door, did not show a significant difference in breaking point between groups (Fig. 3C). Additionally, no difference was found between groups in the average variability in targeting the sugar pellet, as measured by standard deviation of paw trajectories (Fig. 3D). The variability significantly decreased for both groups between days 2 and 8 of training, showing clear motor skill learning in both groups (preferred: P < 0.05, nonpreferred: P < 0.01). The average grasping speed also showed a strong overall training effect, significant between days 2 and 8 for the preferred flavor trained group (Fig. 3E; P < 0.05), but was not different between the two groups. Taken together, these findings suggest that there was no overall effect of reward value ratios on skill learning and overall motivation to perform the instrumental task.

To measure a specific local effect of reward value on both motivation and skilled grasping, animals from both groups were then averaged and trials analyzed according to pellet flavor (indicated by the tone cue). Over the course of grasping training, latency to perform a grasping attempt after nose-poke initiation was strongly affected by reward type (Fig. 3F). Quinine-flavored pellets showed a significantly increased grasping latency compared wih the other two reward types on all days measured (P < 0.01). This suggests a strong selective effect of aversive tastes on motivation to perform the skilled movement. Accordingly, the increase in actual skilled movement speed with training showed a strong effect of pellet flavor (Fig. 3G). Movement speed increased for both the preferred (P < 0.01) and nonpreferred (P < 0.05) pellet flavors, while grasps for quinine-flavored pellets showed no change over time and were significantly slower compared to preferred and nonpreferred grasps on days 5 and 8 (P < 0.01). Similarly, but less profound, the first attempt success rate averaged over both groups also showed a significant effect of both training and of reward type; however, there was no significant interaction of the two (Fig. 3H). For all pellet types, success rate significantly increased from day 2 to 8 (P < 0.01), while grasps for nonpreferred pellets showed an additional significant increase in success rate between days 2 and 5 (P < 0.01). It should be noted however, that there was no difference in success rate for different reward types at the end of training. These findings suggest a motivational effect of reward value that strongly influences training-dependent speed improvements while leaving accuracy almost unaffected.

The flavor preference test was repeated at the end of the experiment and showed some variations among groups. In the preferred group, one animal changed its least preferred flavor from lemon to banana, while in the nonpreferred group one animal changed its preferred flavor from caramel to lemon, indicating a change in preference toward the most abundant reward. Furthermore, in this group, three animals had changed their least preferred flavor from previous lemon to new banana, indicating a tolerance effect toward the nonpreferred flavor. For most animals (11 out of 12) however, caramel remained the preferred flavor. These changes in reward preference could explain the lack of effect of preferred and nonpreferred rewards on any of the measured parameters.

Experiment 3: inverse effect of Pavlovian incentive motivation on performance speed of a new skilled movement

We next investigated motivating effects of Pavlovian stimuli on learning and performance of skilled grasping movements in animals with moderate food restriction (60% of their ad libitum food supply). We addressed this question by use of a Pavlovian conditioning phase conducted before instrumental training tasks, where animals were exposed daily to two conditioned stimuli (tone and white noise) paired with delivery of sugar pellets into the pellet receptacle (CS+) or no pellet delivery (CS−). After 10 d of Pavlovian conditioning training, animals were trained to perform a nose-poke to receive a sugar pellet during an instrumental training phase. Subsequently, a reminder session of Pavlovian training was conducted, followed 24 h later by a Pavlovian conditioning test performed in extinction, with no rewards delivered, to test for successful conditioning and Pavlovian instrumental transfer (PIT) effect on the instrumental chain of actions (Fig. 5A below). Figure 4 shows the pellet receptacle approach behavior (Fig. 4A) and nose-poke responses (Fig. 4B) during the Pavlovian conditioning test. Approach behavior was significantly higher during CS+ display than during CS− presentations (P < 0.01) or no tone periods (P < 0.01) (Fig. 4A). Accordingly, nose-poke response behavior was also significantly different between no tone and tone periods. The number of nose-pokes was strongly increased during no tone periods compared to CS+ (P < 0.01) and CS− presentation (P < 0.01) (Fig. 4B). These results suggest a strong PIT effect on approach behavior (proximal action), in accordance with previous reports (Corbit and Balleine 2003), and an inverse effect on nose-poke response behavior (distal action), indicating competition between the two actions, as they were performed on opposite sides of the behavior box.

Figure 5.

Pavlovian incentive motivation does not affect learning of a skilled grasping task and inversely affects grasping speed. (A) Experimental design. All animals underwent Pavlovian conditioning training during which a CS+ tone was paired with sugar pellets automatically dispensed into the receptacle. A CS− tone was displayed during intervals with no sugar pellet dispensed. After instrumental training and a Pavlovian conditioning test, skilled grasping training was performed under CS+ or CS− display for two groups of animals, respectively. (B) The CS+ and the CS− groups both improved significantly over time in their grasping success (time: F_(7,70) = 21.63, P < 0.0001). Both groups showed a significant increase in grasping performance from days 2 and 3 to days 5–9 (CS+: P < 0.01, CS−: P < 0.01). Additionally, the CS− group already improved significantly from day 2 to 4 (P < 0.01), while the CS+ group improved strongly from day 4 to 8 (P < 0.05). There was no difference in success rate between groups (group: F_(1,10) = 0.52, P < 0.49). (C) Grasping response latency in the long last session was not different between groups (P = 0.09). (D) Variability of targeting paw trajectory was affected by training (time: F_(2,20) = 5.20, P = 0.015), with the CS− group showing a significant decrease in variability between day 2 and 5 (P < 0.05) and day 2 and 8 (P < 0.01). (E) Average speed of grasping movement increased significantly over time (F_(2,20) = 7.20, P = 0.004), with the strongest increase shown by the CS− group between day 2 and 5 (P < 0.01) and day 2 and 8 (P < 0.01). (F,G) Latency to perform a grasping attempt after nose-poke initiation of a trial was not affected by CS+ or CS− tone display, even early in the grasping training (days 2 and 5). (H–J) Grasping movement speed for both groups during tone and no tone display. (H) On day 2, grasping speed was significantly affected by tone display (tone: F_(1,10) = 7.55, P = 0.021) with the CS− group showing faster movements during tone display than no tone periods (P < 0.05). (I) On day 5, the CS− group grasped significantly faster, than the CS+ group (group: F_(1,10) = 10.37, P = 0.01) for both tone (P < 0.05) and no tone periods (P < 0.05). (J) On day 8, movement speed was affected by tone display (tone: F_(1,10) = 9.60, P = 0.011), but no significant differences were detected by multiple comparison. (K–M) Success rate for both groups for tone and no tone periods. First attempt success in pellet retrieval was not affected by the group or tone display period on any of the analyzed days. All data are shown as means ± SEM. All conditions and groups n = 6 rats.

Figure 4.

Pavlovian cues affect approach behavior of the instrumental chain, with an inverse effect on nose-poke response. (A) Approach behavior to pellet receptacle was strongly affected by tone display (F_(2,22) = 27.20, P < 0.0001). Animals showed significantly more approach behavior during CS+ presentations compared with CS− (P < 0.01) and no tone periods (P < 0.01). (B) Number of nose-poke responses was significantly affected by tone display (F_(2,22) = 29.95, P < 0.0001). Nose-poke behavior was strongly increased during no tone periods compared with CS+ (P < 0.01) and CS− presentation (P < 0.01). All data are shown as means + SEM, n = 12 animals.

Having observed this strong PIT effect of the CS+ on approaching the pellet receptacle, we next tested whether this effect is transferred to learning of a skilled grasping task. For this phase of the experiment, conditioned animals were split into two groups and presented with either the CS+ or the CS− tone with a duration of 30 sec and an interstimulus interval of 30–60 sec while acquiring the skilled grasping task. Skilled grasping performance over time was again measured in first attempt success rate, variability in targeting movements, and movement speed for both groups. At conclusion of the experiment, grasping latency was measured in a 30-min long last session to test for general transfer of the PIT effect to the grasping response (Fig. 5A). Over the course of grasping training, both groups improved significantly reaching a comparable final first attempt success rate (Fig. 5B; CS+: 30.8 ± 4.9%, CS−: 37.5 ± 6.4%). Bonferroni-corrected multiple comparison revealed significant differences in performance between the first two and the last three sessions for both groups (CS+: P < 0.01, CS−: P < 0.01), while the CS− group already showed significant improvements between day 2 and 4 of training (P < 0.01). However, Pavlovian stimulus presentation did not affect grasping success rate at any time during training. At the end of training, a 30-min long last session was performed and grasping response latency measured (Fig. 5C). The groups showed no difference in grasping latency indicating that the initially observed PIT effect on approach behavior was not transferred to the grasping task, or has been extinguished during training.

We next investigated the additional parameters of skilled motor performance, which showed similar results. Variability in targeting movements showed a strong overall effect of training, with the CS− group significantly decreasing variability between days 2 and 5 (P < 0.05) and days 2 and 8 (P < 0.01) (Fig. 5D). Similarly, average speed of grasping movements increased significantly over time, showing a strong increase again in the CS− group (Fig. 5E; P < 0.01). Neither parameter showed an effect of CS type.

Taken together, these findings strongly suggest that Pavlovian stimuli do not enhance general learning of a new motor skill. Conversely, the CS− group showed a tendency to perform better in all tested parameters. To investigate whether the PIT effect was initially there but got lost during skilled grasping training, we next measured grasping latency on days 2 and 5 separately for trials performed during tone or of tone display for both groups (Figs. 5F,G). Both groups performed the proximal action of the task with similar speed independent of tone display, again suggestive of a complete loss of PIT effect as early as day 2 of skilled grasping training. However, when analyzing speed of the grasping movement at different time points during training and for trials with or without tone display, we saw that on day 2 (Fig. 5H), there was a significant decrease in speed of grasping in the CS− group during no tone periods compared with tone periods (P < 0.05). Further, on day 5 (Fig. 5I), the CS− group grasped significantly faster, than the CS+ group for both tone and no tone periods (P < 0.05). On day 8, speed of the grasping movement also showed a small overall tone display effect (Fig. 5J). Taken together, these data suggest that even though grasping latency did not show a motivational PIT effect on instrumental performance, speed of the newly learned grasping movement was significantly affected by the display of CSs. Unexpectedly, these movements were faster with CS− display, indicating an inverse PIT effect. Such an effect was not observed when success rate was analyzed in the same manner (Fig. 5K–M). Accuracy of grasping movement measured in success rate was not affected by display of CSs, suggesting that Pavlovian incentive motivation differentially affects specific parameters of skilled movements.

In conclusion, these findings suggest that display of a CS− initially affects performance of skilled grasping, as found in movement speed, presenting an inverse PIT effect on a newly learned motor skill. This could be due to lack of a representation of “free reward” during CS− (and no tone) display. However, if proven consistent in future experiments, the overall effect seems to be small and selective for specific parameters of motor skill.

Discussion

Results of this study clearly show that performance of skilled grasping in adult rats is affected by different motivational processes. We found a strong effect of motivational state on speed and accuracy of skilled grasping. Reward value showed no group effect, but aversive pellet flavor led to reduced speed of skilled grasping. Pavlovian cues slightly affected skilled movement speed in an inverse PIT effect. A clear effect on motor skill learning was not observed but could be detected with future studies as discussed below.

In the first experiment, we trained animals in different motivational states using food restriction. Groups with higher motivational state showed significantly higher success rates with training than the low motivational state group. An increased breaking point in the progressive ratio test confirmed the effect of food restriction on motivation (Hodos 1961). This difference in motivation thus seemingly had a strong impact on the learning curve of success rates. However, by analyzing detailed kinematics of the grasping movement itself, we found no effect of motivational state on learning. Motor skill learning was accompanied by a decrease in variability between single grasping movements and an increase in speed for all groups. We then tested the direct effect of motivational state on performance in a long last session. Satiation over time induced strong changes in motivation in the 60% and 80% food supply groups, as evident from grasping latencies. Grasping latency was stronger affected by satiety than nose-poke latency which is in accordance with previous findings using instrumental chains (Corbit and Balleine 2003). When trained in a heterogeneous chain of action task, comprised of a distal and proximal response, experience with the reward in the new motivational state (Balleine et al. 1995; Changizi et al. 2002) is required to influence rate of the distal response (Balleine et al. 1995; Corbit and Balleine 2003), but not the proximal response.

Even though motivation seemed mostly affected in the 80% and 60% groups, the skilled grasping speed decreased only for the 40% and 60% groups. Surprisingly, the accuracy (success rate) of skilled grasping also decreased with satiety in the 60% group. These findings indicate that the observed increased performance with training was most likely due to performance effects of motivation. Interestingly, the 80% group, which showed the strongest satiation, showed no reduction in speed or accuracy, while the 40% group with least satiation showed a strong reduction in speed but not accuracy, suggesting a nonlinear effect of motivational state on different parameters of skilled motor performance. On the one hand, there seems to be a threshold below which speed or accuracy are not costly enough to be affected by motivational state, which was reached by the 80% group. On the other hand, we found that animals can be capable of performing highly accurate movements, but choose not to do so when motivation is reduced (only 60% group), indicating that there is a costliness to accuracy as well, which is lower than the one for speed.

This direct effect of motivation on speed and accuracy was not observed at the beginning of training, which could indicate a learning effect in the 60% and 80% groups (Wong et al. 2015). However, all animals are probably performing at maximal speed to not impact accuracy at this early stage of learning (speed–accuracy trade-off curve) (Willingham 1998; Dayan and Cohen 2011; Shmuelof and Krakauer 2011; Haith et al. 2012). This is also supported by data from the second experiment where we found no faster grasps for preferred/nonpreferred pellets than for quinine-flavored pellets at the beginning of the training even though motivation in nonquinine trials was significantly increased. However, the lack of fast grasps at the start of training could also be due to a lack in incentive learning (Balleine 1992), as animals from the 40% and 60% groups were only switched to these states shortly before skilled grasping training commenced. Supporting this hypothesis, we found no difference in grasping latency between the groups on day 2 (data not shown).

To determine whether motivational state has any real effect on learning of skilled grasping the low performing 80% group could be switched to 40% food supply, allowed incentive learning in a second purely instrumental phase of the experiment (Balleine 1992), and then tested for grasping skill. If this group shows an immediate switch to increased speed and accuracy in grasping, the effect is merely on performance.

Furthermore, there seemed to be an upper limit to the impact of motivational state on motor skill performance, as the 40% group did not show higher success scores or faster grasps than the 60% group, but merely a reduced effect of satiation (i.e., no satiation). We do not expect our food restriction regime to have had any adverse effects on the animal's well-being, since none of our animals lost more than 10% of their initial body weight. Food restriction of similar levels has been shown to decrease organ pathology and increase longevity in rats (Keenan et al. 2005; Rowland 2007). Additionally, food restriction has recently been suggested to affect learning and performance in memory and discrimination tasks (Makowiecki et al. 2012; Kuhla et al. 2013; Talhati et al. 2014). On a more general level, direct effects of food restriction on brain plasticity have been reported (Murphy et al. 2014). Caloric restriction leads to changes in synapse number and structure (Mattson 2012), increased NMDA receptor expression (Yilmaz et al. 2011), and up-regulation of BDNF (Mattson and Wan 2005; Kishi and Sunagawa 2012). These findings could therefore reflect that the highly motivated animals had reached an upper speed/accuracy limit for skilled grasping.

In our second experiment, we investigated the effect of different reward values on motor skill learning and performance, using different ratios of preferred and nonpreferred pellet flavors in two different groups. Additionally, randomly interspersed aversively flavored pellets (quinine) were presented in both groups. We did not find a significant difference in skilled grasping learning or performance between groups. Both groups showed significant motor skill learning as shown by increased success rate, speed of movement, and reduced variability between grasps over time. When tested at the end of the training period, no difference was seen in nose-poke response breaking point in a progressive ratio test, indicating that the difference in preferred/nonpreferred reward ratio had no effect on motivation.

However, by pooling the two groups and separately analyzing movement kinematics according to reward types we found specific effects of pellet flavor on motivation and skilled grasping performance. By using associative cues displayed at time of nose-poke, we allowed animals to anticipate the pellet flavor and thus reward value they will grasp for in each trial. This predictive auditory cue was likely associated with different reward values, however, there was also an olfactory cue that the animals could detect, as they tend to sniff for the pellet before they initiate the grasping movement (Whishaw and Tomie 1989; Hermer-Vazquez et al. 2007). This olfactory information probably served as an additional predictive cue for different pellet flavors. The latency to complete a grasp after a nose-poke response was significantly increased in trials for quinine-flavored pellets (aversive) throughout the training period. This effect on motivation was also directly seen in grasping movement speed. Speed of skilled grasping movement increased with training, but only in grasps for nonquinine-flavored pellets, while grasps for quinine-flavored pellets showed no such increase. These findings provide strong evidence for an effect of reward value on motor skill performance. Similar performance effects of reward have been shown in saccade velocity and wrist movement reaction times (Takikawa et al. 2002; Xu-Wilson et al. 2009; Opris et al. 2011; Chen et al. 2013). Accuracy of the movement, measured in success rate, also showed a significant effect of reward value, but aversive quinine-flavored pellets were accurately grasped by the end of training as well, indicating that grasps for quinine-flavored pellets were not invigorated but were still performed accurately. This could suggest that cost of accurate movements is lower than the cost of high speed. This is in line with findings from experiment 1 where we showed a reduction in grasping speed even in the 40% group (which showed almost no satiation), while accuracy was not significantly changed.

Using preferred and nonpreferred pellet flavors as reward, we did not observe a group effect of reward value ratio as suggested by previous reports (Niv et al. 2007; Guitart-Masip et al. 2011). This might have been due to the equal caloric value of the two pellet flavors, or the adaptation of preferences due to exposure (Touzani et al. 2010), as seen in the shift in preference in 4 out of 6 animals in the nonpreferred flavor trained group. Those animals showed increased tolerance for the nonpreferred flavor and even a preference switch in one animal.

Future experiments could use ratio variations between quinine- and preferred-flavors, and might detect an effect on vigor by net reward. A potential effect on motor learning could then be tested by exposing the low performance group to high value rewards and measure immediate improvements.

We aimed to further modulate motivation of moderately food-restricted animals by use of Pavlovian cues in our third experiment. Pavlovian conditioning was performed with a CS+ and a CS− tone cue before animals were trained in the instrumental nose-poke task. A Pavlovian conditioning test under extinction revealed a strong PIT effect of the CS+, leading to increased approach behavior toward the pellet receptacle, the proximal action of the learned instrumental task, in line with previous findings (Corbit and Balleine 2003). It should be mentioned that in our behavior box the nose-poke was on the opposite side of the location of the pellet receptacle (O'Connor et al. 2010), thus creating a competition between actions, which becomes evident in the reduction of nose-poke responses seen during CS+ and CS− display. While responding during CS+ episodes was significantly lower than during CS− episodes when tested in a paired t-test (P < 0.05). A further explanation for increased nose-poke response during no tone periods could be that animals were put into an “instrumental training” state. No tone was played during instrumental training, which could have created a context to be associated with instrumental responding, thus triggering more nose-pokes.

The strong PIT effect on approach behavior under extinction was not transferred to learning the new motor skill, nor was it detectable in the grasping latency at the end of training. Both groups, trained either under CS+ or CS− display, showed strong learning of the skilled grasping task, measured by accuracy, speed, and variability of movement over time, but no overall difference between the groups was found. A transfer of the PIT effect to grasping latency was also not detected early in skilled grasping training. However, on day 2, grasping latencies might have also been prolonged because animals were learning that the task requirements had changed from the instrumental to the skilled grasping training. Measurements of speed of grasping movements hint toward a small inverse PIT effect early in training: on day 2, animals showed an increased grasping speed during CS− tone display compared to no tone periods and general grasping speed was significantly higher in the CS− group than in the CS+ group on day 5. We hypothesize that the CS+ induced an outcome representation of “a free reward delivery” that made animals of the CS+ group perform more approach behavior, which represented the associated action (Dickinson et al. 1996), but inhibited the active grasping movement, since this represented a new action, which was not previously associated with the reward (Overmier and Lawry 1979; Corbit and Balleine 2003). Conversely, the CS− display was likely associated with the outcome of “no free reward,” allowing the animal to better learn a new association between the grasping action and pellet retrieval, leading to faster grasping speed. Measuring the accuracy of the performance (with success rates) showed only a tendency of CS+ or CS− tone display, indicating that the Pavlovian stimuli had a stronger effect on movement speed than on quality of movement, which is in line with an invigorating effect of PIT that has also been described in humans (Talmi et al. 2008). We therefore suggest that Pavlovian cues do not have their classic effects on performance of a newly learned motor skill. Future experiments could further explore these findings using both CS+ and CS− cues in one group of animals to test for differential effects of the two cues.

Investigating effects of motivation on learning and performance of skilled grasping using a pellet grasping task bares certain caveats. The nature of the task dictates that the reward delivery is directly dependent on performance (i.e., successful retrieval of the sugar pellet) and changes over time with learning of the movement. In our specific paradigms, this might have affected motivational state of the animals in a more uncontrolled manner. The reward value ratios, even though set out specifically, were also influenced by the animals’ success rate. For the PIT study, associations between the CS+/CS− and the pellet reward was likely shifted during skilled grasping training, as rewards were obtained according to grasping performance and independent of tone display. This might explain the stronger effects of CS− display early in training, which had disappeared by the end of the experiment.

In conclusion, we successfully showed that in rodent skilled grasping, the key parameters that define motor skill learning, speed and accuracy, were strongly affected by motivational processes, mainly those involved with motivational state and reward value. We found that movement speed and success was directly modulated by the hunger state of the animal in a nonlinear way, and that grasping speed was strongly influenced by aversive pellet taste. Furthermore, we found an unexpected effect of CS− Pavlovian cues on movement speed. Our data provides evidence that skilled grasping speed and accuracy have different costs and are differentially affected by specific motivational processes. Considering the increasing use of the skilled grasping task in neuroscience, these findings have broad implications. Additional to the traditional experiments investigating motor skill learning (Kleim et al. 2004; Xu et al. 2009; Hosp et al. 2011) and recovery after central nervous system injuries (Conner et al. 2005; Alaverdashvili and Whishaw 2008; Krajacic et al. 2010; Starkey et al. 2012; Wahl et al. 2014; Garcia-Alias et al. 2015), more recent studies have made use of skilled grasping to evaluate the role of unexplored motor systems in skill learning and performance (Azim et al. 2014; Esposito et al. 2014; Alstermark and Pettersson 2014). Considering this gain of importance of skilled grasping in studies of systems neuroscience, it is paramount to be aware of and control for potential effects of motivation when designing the animal experiments and analyzing grasping learning and performance (Wong et al. 2015).

Materials and Methods

Subjects

A total of 48 female Long-Evans rats (Janvier Labs, France) were used in this study. At the start of the experiment, all animals were 10–12 wk old and weighed 190–230 g. Animals were housed in groups of three or four per cage and were kept on a 12 h light–dark cycle, with behavioral training and testing taking place in the second half of the light phase. Animals were food restricted to either 80%, 60%, or 40% of their previously determined ad libitum food intake, depending on the training phase and experiment. A restricted number of food pellets was provided after each training or testing session. The body weight of all animals was monitored daily and maintained over 90% of the initial weight. Water was available in the home cage at all times. All experimental procedures were conducted in accordance with the Veterinary Office of the canton of Zurich, Switzerland.

Training and testing apparatus

All behavioral training sessions were performed in a custom-made behavioral setup comprising an instrumental grasping box, a pellet dispenser, loudspeakers, and a video recording system.

The instrumental grasping box (15 × 40 × 30 cm) was made of Plexiglas with a grid floor of metal rungs (5 mm in diameter, 8 mm spacing). A custom-made nose-poke sensor was mounted in the middle of one of the 15 cm wide walls. A narrow window was cut into the Plexiglas (1 cm wide, 5.5 cm high, starting 3 cm from the rungs) on the opposite wall. A motorized Plexiglas door covered the window from the outside. In instrumental training sessions, the door was kept open and inactive. A pellet dispenser (Med Associates) connected to a custom-made pellet receptacle that extended through the window was positioned outside of the box. Activation of the nose-poke sensor led to the release of a sugar pellet (45 mg Dustless Precision Pellets, BioServ), which the rat could directly eat from the pellet receptacle. During skilled grasping training sessions, the door was closed and a pedestal (3 cm high, 5 mm diameter) was positioned outside of the box in front of the window. A sugar pellet was placed on the pedestal by the experimenter. Activation of the nose-poke opened the door, allowing the animal to reach through the window and grasp the pellet with its forelimb (Buitrago et al. 2004).

Two loudspeakers were attached to the roof of the instrumental grasping box. Depending on the experiment, different sound cues at 90 dB were used as conditioned stimuli (CS), i.e., a tone of either 1, 4, 8 kHz, or white noise (Syka et al. 1996).

All skilled grasping training sessions and behavioral tests were video recorded at 200 frames per second using an industrial high-speed video camera (Teledyne Dalsa) and a LabView-based (National Instruments) software interface. A mirror was positioned at a 45 degrees angle outside the instrumental grasping box to allow the simultaneous recording of front and side views of the grasping response and approach behavior.

Behavioral training phases

In all experiments, animals went through two specific training phases, first learning an instrumental nose-poke response and then the skilled forelimb grasping. A pretraining phase was performed to familiarize the animals with the setup and the sugar pellets. The pretraining phase consisted of at least 3 d during which the animals were handled and exposed to the instrumental grasping box in groups of four on the first 2 d and groups of two on the third day for 10 min each. During this time, sugar pellets were delivered automatically by the pellet dispenser into the pellet receptacle and could be eaten directly. Additionally, sugar pellets were placed into the home cages of all animals once in a satiated and once in a food deprived state to allow instrumental incentive learning. Finally, ad libitum food intake was measured for each cage over 3 d to determine the average food intake. In the second phase, all animals were placed singly into the instrumental grasping box for a maximum of 15 min each day. In these sessions, they learned to perform a nose-poke to receive a sugar pellet, which was delivered into the pellet receptacle on the opposite side of the box by the pellet dispenser (Fig. 1A). All animals were trained until they reached a criterion of 20 nose-pokes and pellet consumptions within 10 min on two consecutive days. In the last training phase, the skilled grasping training, animals learned to use their forelimb to reach and grasp for a sugar pellet, which was placed on the pedestal outside of the window in the wall. At the beginning of each trial, the door was closed, covering the window. Activation of the nose-poke sensor led to the opening of the door allowing the animal to reach for the pellet. After a grasp was performed by the animal, the door was closed by the experimenter pressing a button (Fig. 1B). On the first day of this phase, the pedestal was placed 12 mm away from the window and was positioned in the middle of the window. During this session, the paw preference of each animal was determined and the pedestal was positioned at the left or right edge of the window for right or left handed rats, respectively, on the following days. This ensured that only grasping movements with the preferred paw were successful. On all but the first day, the pedestal was positioned further away (15 mm) from the window to increase the difficulty of the task. The number of trials performed by each animal was held constant between groups for each experiment. Each experiment started with 20 trials on the first day and the number of trials per day was increased by steps of 5–10 once all animals completed the session within 10 min. The maximum number of trials per session was kept at 40 and the number of sessions for all experiments was nine, with one session per day.

Motivation paradigms

Experiment 1: motivational state

In this experiment, animals were trained under different levels of food supply (see Fig. 2A for experimental design). Instrumental training was performed with all animals receiving 80% of ad libitum diet. For skilled grasping training, animals were divided into three groups: 80% (n = 8), 60% (n = 8), or 40% (n = 8) food delivery of the original ad libitum amount.

Experiment 2: reward value

In this experiment, animals were trained using differently flavored sugar pellets, depending on their preference (see Fig. 3A for experimental design). All animals were kept under 80% ad libitum food supply. After the preference had been determined by the flavor preference test, animals were further trained in the instrumental training for an additional week. During this training, each nose-poke response was followed by a short tone (0.5 sec), signaling delivery of the preferred (1 kHz) or nonpreferred (4 kHz) pellet flavor. In total, 108 tone–reward pairings were performed in a counterbalanced fashion by all animals. During the last 3 d, quinine (300 mM) aromatized sugar pellets paired with an 8 kHz tone (probability of 0.14) were added. For the subsequent skilled grasping training, animals were divided into a preferred flavor (n = 6) and a nonpreferred flavor (n = 6) group. During each session, animals of the preferred flavor group were presented with the preferred reward with a probability of 0.72 and the nonpreferred reward with probability of 0.18, whereas animals of the nonpreferred flavor group were presented with the opposite odds. Additionally, all animals faced a probability of 0.09 to grasp for a quinine-flavored pellet. The progressive ratio test at the end of the experiment was performed with either the preferred or nonpreferred pellets only, depending on the group.

Experiment 3: Pavlovian incentive motivation

In this experiment, animals were trained under display of different Pavlovian cues (see Fig. 5A for experimental design). Animals were food restricted to 60% of ad libitum diet during the entire behavioral training period. Prior to the instrumental training phase, animals received 10 d of Pavlovian conditioning training. In these sessions, a 1 kHz tone or white noise was paired either with the delivery of five sugar pellets into the pellet receptacle (CS+) or no reward (CS−) (counterbalanced between animals). The two tones were presented in random order with a 0.5 probability. Each stimulus was 45 sec long with an interstimulus interval of 45–75 sec. Animals received 10 tone presentations on the first day and eight on days 2–9. After the Pavlovian conditioning, all animals were trained in the instrumental task for a total of 7 d during which no tone was presented. Afterward, a reminder session for the Pavlovian conditioning with eight tone presentations was performed. Additionally, a Pavlovian conditioning test was performed under extinction. For the skilled grasping training, animals were then divided into two groups, receiving either only CS+ or CS− tones with the same durations as in the Pavlovian conditioning test (CS+ n = 6, CS− n = 6). Additionally, the progressive ratio test was performed with the same stimulus animals received during skilled grasping training.

Behavioral tests

In experiment 2 (reward value), a flavor preference test was performed after the first 5 d of instrumental training. Sugar pellets were flavored with either of four natural food aromas (lemon, caramel, banana, violet flower [purchased from a local drug store]), by mixing pellets with the aroma in a Petri dish. To test for flavor preference, food restricted animals were put singly in a standard cage. Four trays filled with either of the differently flavored sugar pellets were placed next to each other in the cage. After 1 h, food intake of each pellet flavor was measured by weight. The food preference test was repeated two times in order to determine the most preferred and least preferred sugar pellet flavor for each animal. The test was repeated at the end of the experiment, to check for changes in preference due to training.

In Experiment 3 (Pavlovian incentive motivation), a Pavlovian conditioning test was performed under extinction 1 d after the Pavlovian reminder session. To increase the number of tone presentations in this test, tones had a duration of 30 sec and an interstimulus interval of 30–60 sec. Approach behavior to the pellet receptacle was recorded by high speed video recording and expressed as ratio of video frames that the animal spent at the receptacle divided by the total amount of frames. Additionally, the number of nose-pokes performed by each animal was recorded. All data were normalized to no tone and tone periods.

To assess motivation in the instrumental skilled grasping task, a progressive ratio test was performed in all experiments. Animals performed the skilled grasping task while the number of nose-poke responses required to open the door increased progressively from one trial to the next (Hodos 1961). The progressive ratio test was completed when an animal did not perform a nose-poke for 1 min, or when 40 trials were completed. The number of required nose-pokes in the trial during which animals stopped performing was taken as the breaking point value.

Data recording and analysis

To measure motivation of animals to perform different aspects of the skilled grasping task, time-stamps of nose-pokes and door closing after the grasp (manually triggered by the experimenter) were recorded automatically with a LabView-based program (National Instruments). These time-stamps were used to calculate the nose-poke response latency (measured as time between door closure and successive nose-poke) and grasping latency (measured as time between nose-poke and successive door closure).

The skilled grasping performance of each animal was scored by the experimenter during each session counting the first attempt successes in retrieving and eating the pellet. An attempt was counted if the animal tried to reach through the opened door, extending its paw through the window. A success was counted if only one attempt was needed for pellet retrieval. However, animals were allowed to reach out more than once to obtain the pellet. Learning curves were calculated from the first attempt successes divided by the number of total trials.

All sessions were recorded with a high-speed video camera at 200 frames per second (Teledyne Dalsa). Grasping kinematics were analyzed from high-speed videos on days 2, 5, and 8, (and in the last long training session for experiment 1) using the MTrackJ plugin of ImageJ (National Institutes of Health). The paw of the animal (lateral metacarpophalangeal joint) was marked and manually tracked frame by frame in a lateral view from the lift-off of the paw to the flexion of the digits at the end of the grasp. In each session, several grasps were tracked for each animal and split between first and last attempts of the session, reward type, or tone and no tone presentations for experiments 1, 2, and 3, respectively. The grasping trajectories were then averaged for each session and grasp type, and different parameters extracted using Matlab (Mathworks). Main learning effects and differences were found in the average speed and the variability in targeting (Fig. 1C). The speed was averaged over the whole trajectory to obtain a mean speed value for each grasp. To determine the variability in targeting, all trajectories were extrapolated to comprise 200 points and the mean trajectory and standard deviation (SD) calculated. The last 66 points of the trajectory were extracted and the SD averaged to measure the variability in targeting of the pellet. The variability was only compared between the same numbers of grasps for all animals within one experiment. In Experiment 1, the first and last seven attempts per session were analyzed on days 2, 5, and 8. However, on day 2, of one animal of the 40% group, only seven grasps were available. In the last training session, the first 10 and last 10 attempts were analyzed. In Experiment 2, grasps were split according to the presented pellet flavor. For each session, all the low probability pellets and the same number of high probability pellets were analyzed for each group, resulting in 3–8 attempts per animal and flavor. Quinine attempts were fewer, due to their low probability and reluctance of the animals to grasp, resulting in 0–4 attempts. In Experiment 3, grasps were split into attempts occurring during tone display and attempts occurring during no tone periods. Seven tone and seven no tone grasps were analyzed for each animal on days 5 and 8. On day 2, only 1–7 no tone, and 6–7 tone grasps were available and tracked.

Statistical analysis

All statistical analysis was performed with Prism 6.0 (GraphPad Software Inc.). Two-way analysis of variance (ANOVA) with repeated measures, followed by multiple comparisons with Bonferroni correction was used to detect differences between groups and within groups over time. One-way ANOVA followed by multiple comparison with Bonferroni correction was used to detect differences between more than two groups or conditions, using repeated-measures where implied. The Student's t-test was used to analyze differences between two groups. The threshold for statistical significance was set at *=P ≤ 0.05. Smaller P-values were represented as **=P ≤ 0.01. All data are reported as mean (M) ± standard error of the mean (SEM). Single dots in bar graphs represent individual animals.