Independent neural coding of reward and movement by pedunculopontine tegmental nucleus neurons in freely navigating rats

Abstract

Phasic firing of dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SN) is likely crucial for reward processing that guides learning. One of the key structures implicated in the regulation of this DA burst firing is the pedunculopontine tegmental nucleus (PPTg) which projects to both VTA and SN. Different literatures suggest that PPTg serves as a sensory-gating area for DA cells or it regulates voluntary movement. This study recorded PPTg single unit activity as rats perform a spatial navigation task to examine the potential for both reward and movement contributions. PPTg cells showed significant changes in firing relative to reward acquisition, the velocity of movement across the maze, and turning behaviors of the rats. Reward, but not movement, correlates were impacted by changes in context, and neither correlate type was affected by reward manipulations (e.g. changing the expected location of a reward). This suggests that the PPTg conjunctively codes both reward and behavioral information, and that the reward information is processed in a context-dependent manner. The distinct anatomical distribution of reward and movement cells emphasizes different models of synaptic control by PPTg of DA burst firing in VTA and SN. Relevant to both VTA and SN learning systems, however, PPTg appears to serve as a sensory gating mechanism to facilitate reinforcement learning, while at the same time provides reinforcement-based guidance of ongoing goal-directed behaviors.

Introduction

The dopaminergic system plays an essential neuromodulatory role in functions such as motivation, sleep/wake cycles, motor control, and cognition. Dopamine (DA) cells exhibit two modes of discharge, tonic and phasic (burst firing). Understanding the regulation of burst firing by DA cells is a first step toward understanding how DA modulates afferent neural circuits that underlie adaptive behavior (Zweifel et al., 2009). Of several nuclei that project to ventral tegmental area (VTA) and substantia nigra (SN) DA cells, the pedunculopontine tegmental nucleus (PPTg) has received special attention in this regard since PPTg seems to regulate the conditional responses of DA neurons (Lokwan et al., 1999; Pan & Hyland, 2005; Maskos, 2008; Mena-Segovia et al., 2008). Also, PPTg stimulation results in short-latency burst firing responses by DA cells (Scarnati et al., 1984; Lokwan et al., 1999; Floresco et al., 2003) and it evokes DA release in VTA efferent structures such as nucleus accumbens (NA) (Forster & Blaha, 2003). PPTg provides cholinergic (Woolf, 1991) and glutamatergic input to VTA and SN (Beninato & Spencer, 1987; Futami et al., 1995; Sesack et al., 2003). When DA cells are devoid of NMDA glutamate input, they show reduced burst firing, and PPTg stimulation no longer results in DA release in NA (Zweifel et al., 2009). Also, reversible inactivation of PPTg reduces burst firing by DA neurons as rats perform a well-learned classical conditioning task (Pan & Hyland, 2005).

Although it is clear that PPTg contributes to the burst firing of DA cells, the significance of this influence is not clear. Consideration of sensory afferents to the PPTg (Redgrave et al., 1987; Reese et al., 1995) along with the established role of DA in reinforcement-based operant learning (Schultz, 1998) suggests that PPTg may facilitate the processing of (or attention to) learned conditioned stimuli via a sensory-gating mechanism (Kobayashi & Isa, 2002; Winn, 2006). Indeed, PPTg neurons exhibit phasic responses to auditory and visual sensory stimuli that predict reward with a shorter latency (5-10 ms) than DA cells (Pan & Hyland, 2005).

The PPTg may, in addition, serve a more complex function. For example, context-dependent responses of PPTg neurons have been described in cats performing a motor conditioning task (Dormont et al., 1998). Also the VTA DA system is considered an essential part of a VTA-hippocampal loop that enables the context analysis of the hippocampus (Smith & Mizumori, 2006) to incorporate new associations (Mizumori et al., 2004; Lisman & Grace, 2005; Mizumori et al., 2009). Here, we sought to identify the nature of the information passed from PPTg to DA cells by investigating PPTg neural responses during performance of a task that is a) known to rely on intact hippocampal processing, and b) known to generate burst firing by VTA neurons in a context-dependent fashion (Puryear et al., 2010). We expected to observe PPTg spatial context-dependent neural responses that are related to reinforcement aspects of the task.

Materials and methods

Subjects

Six male Long-Evans rats (4-6 months old; Simonson Labs, Gilroy, CA, USA) were housed individually in Plexiglas cages in a temperature and humidity-controlled environment. All animals were maintained on a 12 h light/dark cycle (lights on at 7:00 am) and all behavioral experiments were conducted during the light phase of the cycle. Each rat was allowed access to water ad libitum and food-deprived to 80% of its free-feeding weight. All animal care and use was conducted according to the University of Washington's Institutional Animal Care and Use Committee guidelines.

Apparatus

A detailed description of the behavioral testing apparatus can be found in previous reports (Gill & Mizumori, 2006; Martig & Mizumori, 2010; Puryear et al., 2010). Briefly, an elevated eight-arm maze (79 cm from the floor) was used throughout the experiments. Black Plexiglas arms (58 × 5.5 cm each) radiated from a circular central platform (19.5 cm dia). The segment of each arm closest to the central platform could be raised or lowered by remote control to allow access to rewards located at the arm ends. The maze was surrounded by black curtains that were decorated with several distinctive visual cues. The data acquisition computer was located in an adjacent room.

Behavioral training

Detailed procedures were provided in a previous study (Puryear et al., 2010). Briefly, each rat was first familiarized with the testing environment by allowing it to freely forage for chocolate milk available in food cups at the ends of the arms. The rat started a differential reward, win-shift memory task once it consistently ran down all arms and consumed the rewards for the full training session. In each trial, food cups were baited with either a large (5 drops) or small (1 drop) amount of chocolate milk on alternating arms (e.g., large rewards on even-numbered arms and small rewards on odd-numbered arms). At the start of a trial, an animal was placed on the center platform with no access to rewards. Four of the eight arms (that contained in total two large and two small reward arms) were randomly selected, then presented individually and sequentially during the study phase. While the rat consumed the reward on the fourth arm, the test phase began by making all eight maze arms accessible. Rats learned to collect the remaining rewards in the four arms that were not presented during the study phase. Re-entries into previously visited arms were classified as errors. The trial ended by lowering all arms once the animal returned to the center area after obtaining the eighth reward. During an intertrial interval (ITI) of 2 min, all food cups were baited again. The locations of differentially rewarded arms were held constant for each rat throughout training but counterbalanced across rats. Once a rat performed 10 trials within one hour for three consecutive days, it was prepared for the surgical implantation of recording electrodes.

Surgery

Recording tetrodes were constructed from 20 μm lacquer-coated tungsten wires and were inserted into a custom-built microdrive that was made as described in previous studies (Gill & Mizumori, 2006; Puryear et al., 2010). Tetrode tips were gold plated to an impedance of 0.2-0.4 MΩ, tested at 1 KHz. Each rat was deeply anesthetized under isoflurane (5% mix with oxygen at a flow rate of 1L/min) in an induction chamber. The animal was placed in a stereotaxic instrument (David Kopf Instruments, Tujunga, CA, USA) and anesthesia was maintained by isoflurane (1~3%) that was delivered via a nosecone throughout surgery. The skull was exposed and adjusted to place bregma and lambda on the same horizontal plane. After small burr holes were drilled into the skull, a microdrive containing two tetrodes was implanted just dorsal to the PPTg of each hemisphere (AP: 7.2 and 7.8 mm posterior to bregma, 1.8 mm lateral to midline, 6.0 mm ventral from dura). A reference electrode was placed near the corpus callosum. The animals were allowed to recover for at least 7 days, during which they were handled daily.

Postsurgical procedures

Rats were retrained in the spatial memory task until the average number of errors was less than one per trial. Meanwhile, the electrodes were lowered into the PPTg. The neural activity was monitored using the Cheetah data acquisition system (Neuralynx, Bozeman, MT, USA). Waveforms were amplified 1,000 to 7,000 times and filtered between 0.6 and 6 KHz. If no clear unit signals were present, tetrodes were lowered in 21.8 μm increments, up to 175 μm per day. Once isolated and stable units were encountered, the context-dependent activity of the PPTg neurons was determined with a within-subjects design. Each recording session consisted of two blocks of five trials. Rats were tested with the familiar extramaze cues and reward locations in the first block (baseline trials). Then, the second block of five trials occurred after one of the following manipulations: 1) the maze room lights were extinguished (darkness), which has shown to be an effective context manipulation that alters spatial representations in the hippocampus (Gill & Mizumori, 2006; Puryear et al., 2006), 2) the locations of the large and small rewards were switched (reward reversal), or 3) two rewards (one large and one small reward) were omitted from pseudo-randomly selected arms (reward omission). 4) A fourth condition served to control for the passage of time and the number of trials (control). In this case a second block of baseline trials was evaluated. These manipulations allowed us to investigate whether PPTg neuronal activity was dependent on two aspects of reward expectancy (reward location and reward probability) and spatial context (visuospatial information). Each condition was pseudorandomly chosen for any given day of testing such that the same manipulation did not occur for two sessions in a row.

Data analysis

Behavioral performance

The average number of errors in each block of 5 trials was compared to examine whether the contextual manipulations affected animals’ performance on the spatial memory task. In addition, the proportion of times that a large reward arm was selected was calculated for each of the first four choices during the test phase. It was expected that rats would collect the large rewards before the small rewards, since they were food restricted (Puryear et al., 2010). The preference for maze arms that contained the large rewards was compared between blocks to investigate the impact of the experimental manipulations on preferences for rewards of known magnitudes.

Cell firing characteristics

PPTg single unit activity was isolated from other units and background activity using Offline Sorter (Plexon Inc., Dallas, TX, USA). Further analyses of sorted units were performed with Matlab (MathWorks Inc., Natick, MA, USA). The following basic electrophysiological properties were calculated during ITIs (approximately 18 min) to determine spontaneous firing patterns that were not reflective of the rats’ performance of behavioral task: average firing rate, spike duration, a skewness measure [(X-μ)³/(n-1)σ³] of the interspike interval (ISI) distribution, and the coefficient of variation (CV) of the ISIs. The spike duration was measured from the initiation and termination of an average spike trace. The CV of a cell was calculated by dividing the standard deviation the ISI distribution by the mean of all ISIs.

Reward-related activity

To evaluate the reward-related activity of the PPTg neurons, an event marker was automatically entered into the data during recording by metal ‘lick-detectors’ (custom designed by Neuralynx Inc.) that were connected to the food cups: the rat's first contact with the chocolate milk reward triggered an electrical pulse that was transmitted to the computer. Peri-event time histograms (PETHs; 50 ms bins; ± 2.5 sec) were constructed separately across blocks relative to the onset of all reward acquisitions. A cell was considered to have a reward-excited response if it passed the following three criteria in one of the blocks: a) the highest firing rate was observed within the 1000 ms epoch after reward acquisition, b) the average firing rate of the epoch was greater than 150 % of its average firing rate for a given block, and c) the same excited response to rewards was not observed during errors (i.e., there was no neural response when rats ceased forward movement at the arm ends in the absence of rewards). A reward-inhibited response was categorized with the same criteria except that the lowest firing rate in the PETH of a block was located within 300 ms epoch and its average firing rate was smaller than 80 % of the average firing rate of the block.

The impact of altered reward expectancy and contextual information were determined by comparing the reward-related activity of PPTg cells across block 1 (baseline) and block 2 (manipulation). The average firing rate 1000 ms after the acquisition of the rewards was first calculated and then expressed as a percent change relative to the cell's average firing rate for each block. These values of excited and inhibited responses were then normalized relative to each maximum value observed for both two blocks. In this way we could directly compare the magnitude of reward responses for block 1 vs. block 2. A visual summary of the results are displayed as scatter plots of the normalized reward-related activity for each block (Fig. 4A-D). It was expected that the magnitude of the reward response would be similar across blocks and distributed near the 45 degree diagonal line in the scatter plots if reward-related activity of the PPTg neurons did not change across blocks. Cells represented by dots below the diagonal line showed reduced reward responsiveness during block 2, while cells represented by dots above the diagonal line showed greater reward responsiveness during block 2. To quantify the change in magnitude of the reward response, a reward activity change index (RACI) was calculated; this was a measure of the distance of each value to the 45 degree diagonal line in the scatter plot using the following formula, where x₁ and x₂ were the normalized reward responses in blocks 1 and 2, respectively:

$$\mathrm{RACI}=\frac{\sqrt{2{(x_1-x_2)}^2}}{2}$$

Figure 4.

Alteration of reward-related activity after the manipulations of context and reward expectancy. A-D, Peri-event time histograms (bin width = 50 ms, T₀ = reward acquisition) of representative neural responses to rewards before (block 1) and after (block 2) various manipulations. The red line in each histogram shows the change in velocity during outbound movement. Scatter plots depict each neuron's normalized reward activity between blocks. In control sessions, the reward activity was consistent across blocks (A). When darkness was imposed, reward responses changed in magnitude and onset time relative to reward acquisition (B). Here, the reward cells began to fire as the rat approached the location of expected rewards. The reward responses of PPTg neurons did not change significantly after reward locations were switched (C), or when rewards were unexpectedly omitted (D). No change in activity was exhibited in response to the absence of the reward in randomly omitted arms. E, Reward activity change index (RACI) for all reward-responsive cells tested in each manipulation. The asterisk shows that the magnitude of reward responses was significantly altered during the darkness session relative to the other sessions (p < 0.05).

Movement-related activity

During the recording session, the animals’ position was video-recorded and tracked with the use of anterior and posterior infrared light-emitting diodes on the microdrive headstage.

VELOCITY

The positions of the animals on the maze were sampled at 30 Hz and their instantaneous velocity was measured by dividing the distance between position points by the video sampling rate. A Pearson's r linear correlation between velocity and firing rate during outbound (2500 ms to 200 ms before the reward acquisition) and turn/inbound movement (2500 ms after the onset of the turn response) was calculated for each cell, with a velocity range of 1 to 40 cm/sec. To exclude contamination with reward-related activity, time points between 200 ms prior to and 2500 ms after the reward acquisition were not included in this analysis. For cells with a significant correlation between firing rate and velocity, we further tested whether the movement correlation was affected by the manipulations of reward expectancy and contextual information. Since an r value relating velocity and firing rate was determined for each block, it was possible to calculate an r value change index (RVCI), analogous to the RACI value, which revealed the change in the velocity and firing rate correlation between blocks. The RVCI was calculated for each manipulation by replacing the x₁ and x₂ values in the RACI equation with r₁ and r₂, respectively.

TURNING BEHAVIOR

The onset times of turn/inbound movement were manually inserted into the data stream offline when animals started making a 180-degree turn at the end of maze arms after reward consumption to return to the center of the maze. PETHs represented neural activity in the 2500 ms period around the onset of turns (50 ms bin). Two criteria were used to determine whether a cell exhibited a significant excited response to turn behavior: a) the highest firing rate in the PETH of a certain block was observed within 700 ms epoch after the onset of turns, and b) the average firing rate of the epoch was greater than 150 % of the average firing rate of the block. Similarly, turn-inhibited PPTg cells were also classified when the lowest firing rate was located within the same epoch and its average firing rate was less than 80 % of the average firing rate of the block. In order to further determine the effects of the reward and contextual manipulations on turn activity, the average firing rate of excited and inhibited turn responses for700 ms after the onset of turns was measured and normalized to each maximum value observed for both trial blocks. Then, a turn activity change index (TACI) was calculated using the same formula for the RACI and compared across the manipulations.

Spatial distribution of cell firing

We also determined the spatial distribution of firing rates across the surface of the radial maze. These spatial distributions were visualized by plotting first the position occupied by the animals within an X-Y coordinate system. While the animal was in a 6.72 cm radius from the starting position on the maze, the mean firing rate was calculated and a position dot was plotted on a graphic output (Fig. 5A-D, right). When the animal moved outside of this radius, another position dot was generated, and this continued throughout the session. Superimposed upon these position dots were circles whose radii were proportional to the local firing rates. The vector on the circle indicates the direction of head movement as the rats passed through that location of firing.

Figure 5.

Velocity-correlated activity. A-B, Two examples of neurons whose firing correlated with velocity. Peri-event time histograms (bin width = 50 ms) show that the firing rate of a PPTg cell was increased as the recorded rat moved faster during outbound (left, T₀ = reward acquisition) and turn/inbound movement (middle, T₀ = turn onset) (A). The red line in each histogram shows the velocity of movement. The other cell increased its activity as the rat decreased velocity (B). The right column shows the spatial distribution of cell firing. Vectors indicate the direction of travel, and radius size is proportional to the firing rate in that particular area of the maze. For example, the firing-rate circles occurred at all visited locations except for where velocity was decreased at the end of all arms (A). C, Population summary of the proportion of velocity-correlated PPTg neurons. D, r value change index (RVCI) for velocity-correlated cells recorded in each manipulation. No significant result was found among manipulations (p = 0.98).

Histology

After the completion of all recording sessions, rats were deeply anesthetized under 5% isoflurane. The final position of each tetrode was marked by passing a 25 μA current through each recording wire for 20 sec. Then, the animals were given an overdose of sodium pentobarbital and transcardially perfused with 0.9% saline and a 10% formaldehyde solution. Each brain was stored in a 10% formalin-30% sucrose solution at 4°C for 72 h. The brains were frozen, and then cut in coronal sections (40 μm) on a freezing microtome. The sections were then mounted on gelatin-coated slides and stained with cresyl violet. The tetrode tracks and lesion sites were examined under light microscopy. Only cells verified to be recorded in PPTg were included in the data analysis.

Results

Behavioral performance

To examine the effects of the contextual manipulations on spatial memory performance, rats were tested in an eight-arm radial maze with large or small rewards on alternating arms. The number of errors, i.e. re-entries into an arm visited previously within a trial, was compared before and after each experimental manipulation (Fig. 1A). The well-trained rats showed a small number of errors in the first block of each condition and they maintained similar levels of performance in the control and reward omission sessions. Interestingly, similar to previous reports from this laboratory (Puryear et al., 2010), the switch of large and small reward locations resulted in significantly fewer errors in the second block. However, the animals made more errors under the darkness condition relative to the first block. Paired t tests showed significant differences in the darkness (t₍₁₄₎ = 5.19, p < 0.001) and reward reversal conditions (t₍₁₅₎ = 3.29, p < 0.01). No differences were found in the control (t₍₁₇₎ = 1.89, p = 0.08) and reward omission sessions (t₍₁₆₎ = 0.08, p = 0.9).

Figure 1.

Behavioral performance. A, The average number of errors in the spatial memory task. Significant differences in the number of errors between blocks were found in the darkness and reward reversal sessions. *p < 0.01, **p < 0.001. B, The probability of choosing large-rewarded arms during the first four choices of the test phases. Rats preferentially selected maze arms associated with large rewards, except during darkness testing and when the small and large reward locations were reversed. All graphs show mean ± SEM.

To investigate whether rats discriminated the locations of large and small rewards, the rats’ preference for large reward arms was determined by computing the proportion of times rats chose large rewarded arms during the first four arm choices of the test phase (block 2; Fig. 1B). During control sessions, rats displayed strong preferences for large rewards, as indicated by negative correlations between the proportion of large-rewarded arm choices and successive choices, in both blocks (block 1; r = -0.79, p < 0.001, block 2; r = -0.77, p < 0.001). However, such a preference for selecting large reward arms was affected by the alteration of contextual information. In the darkness condition, the animals failed to differentiate large and small rewarded arms, compared with performance in the first block of baseline trials (block 1; r = -0.76, p < 0.001, block 2; r = -0.23, p > 0.07). When the locations of large and small rewards were switched, the rats still preferentially chose the arms that previously contained large rewards without adjusting their behavior to the changes in the reward locations. This was demonstrated by a significant negative correlation in the first block and a significant positive correlation in the second block (block 1; r = -0.69, p < 0.001, block 2; r = 0.66, p < 0.001). This suggests that the behavioral choices were guided by the expectation of finding a large reward, and this expectation persisted across block 2. When a large and a small reward were randomly omitted within a trial, the rats maintained the preference for large rewards (block 1; r = -0.68, p < 0.001, block 2; r = -0.64, p < 0.001), indicating that the choice preference was not directed by reward-generated odor or visual cues.

Reward-related responses

A total of 105 neurons were recorded from 6 rats. Figure 2 shows the locations of recording sites in the PPTg and an example of clusters recorded from a tetrode. The firing rates of PPTg neurons ranged from 0.31 to 65.4 spikes/sec (average: 11.53 ± 1.49 spikes/sec, mean ± SEM), similar to what has been reported previously in freely behaving rats (Pan & Hyland, 2005). Of these cells, 47 cells (44.8%) exhibited significantly altered activity upon reward acquisition (Fig. 3A and B). A large proportion of these cells (83%, 39/47) was excited upon reward acquisition, while 17% of cells (8/47) showed inhibited responses (Fig. 3C). An independent t test showed that the average firing rate of reward-excited neurons was significantly lower than that of reward-inhibited neurons (t₍₄₅₎ = 89.87, p < 0.001). Depicted in Figure 3A and B are two examples of an excited and inhibited response after the acquisition of both large and small rewards. Both reward-excited and -inhibited neurons did not display reward-associated activity during error choices even though the approaching velocities depicted in red lines in the histograms showed similar patterns (Fig. 3A and B, right). In addition, the firing rates of all reward-responsive neurons were not correlated with velocity (p values > 0.05), ruling out the possibility that the reward activity resulted from animals’ movement. These results indicated that the excited and inhibited neural activity relied upon actual reward encounters.

Figure 2.

Single-unit recording in PPTg. A, Locations of recording sites (red dots) in PPTg (light blue). Each dot may represent the location of more than one neuron. B, Illustration of the signals from three simultaneously recorded PPTg cells. The distribution of spike heights on channels 3 and 4 of a recording tetrode are shown on a 2D cluster-cutting space. Analog traces show signals from each tetrode wire for three cells. Scale bar: horizontal, 1 ms; vertical, 0.2 mV.

Figure 3.

Peri-event histogram examples of reward-responsive PPTg neurons. A, An example of a neuron showing excited firing to large (left) and small (middle) rewards upon their acquisition (T₀, bin width = 50 ms). The same cell exhibited no change in firing during errors (right). The red line in each histogram indicates the change in velocity during outbound movement. B, An example of a neuron showing inhibited responses to rewards. C, Population summary of the proportion of reward-responsive PPTg neurons. D, Population histogram of reward-excited neurons. The shaded regions indicate SEM. The black bars at the bottom indicate a significant difference in responses between large and small rewards at a given time point (Wilcoxon test, p < 0.05). E, Population histogram of reward-inhibited neurons.

The population-averaged activity revealed that reward-inhibited cells showed short-lasting inhibition upon reward acquisition, whereas reward-excited neurons exhibited relative long-lasting activation (Fig. 3D and E). The duration of the excited response to large rewards was significantly longer than that of the response to small rewards (Fig. 3D, Wilcoxon signed rank test; p < 0.05), indicating that the excited reward activity was proportional to the amount of rewards that rats encountered. In particular, the excited response to the small rewards decreased when rats started moving 1 s after the reward consumption, which is reflected in the velocity change depicted by the red dotted line (Fig, 3D). However, there was no difference in the response duration of reward-inhibited cells between large and small rewards.

To determine whether reward-related activity was affected by contextual information and reward expectancy, the firing rates of all reward-related neurons were further analyzed across blocks by computing an index of change in reward activity, RACI (see methods). A larger RACI value indicates a greater change in the strength of neural responses to rewards (large and small rewards combined). Figure 4 shows reward activity before and after each manipulation. For control sessions, similar levels of responses to both large and small rewards were maintained across blocks (Fig. 4A). There was no significant change in RACI values during reward reversal and reward omission sessions (when reward location and probability were manipulated in block 2; Fig. 4C and D). Interestingly, none of the cells displayed reward prediction error-related activity when the amount of reward increased or decreased. In contrast, the darkness manipulation resulted in altered reward activity. In the example of Figure 4B, a PPTg cell is shown with elevated activity before reward acquisition in block 2 that was not observed during block 1, and its excited response during reward consumption was also greater in block 2. A one-way ANOVA showed a significant main effect of context conditions (F_(3,45) = 5.7, p = 0.002), and a post hoc Tukey's test revealed that RACI values were significantly higher for darkness sessions as compared to the other conditions (p values < 0.05). RACI values for reward reversal and reward omission sessions were not different from control sessions (p = 0.98 and p = 0.91, respectively).

Movement-related responses

VELOCITY

It was found that 24.8% (26/105) of the total number of the PPTg cells had a significant positive or negative correlation between the animal's movement velocity on the maze and the neural firing rate (Fig. 5A and B). No reward-related neuron showed a significant correlation between firing rate and velocity. Most of movement-related cells (80.8%, 21/26) had a positive correlation with velocity, while the other five neurons (19.2%) were negatively correlated (Fig. 5C). An independent t test showed no difference in average firing rate between velocity-positive and -negative neurons (t₍₂₄₎ = 1.04, p = 0.31). Figure 5A and B shows that different types of velocity-correlated cells co-exist in the PPTg. Although both cells were positively correlated with movement, the cell shown in Figure 5A exhibited greater firing rates as the velocity (in red) increased, whereas the firing rates of the other cell in Figure 5B increased when the velocity decreased. To determine whether velocity correlates were affected by the context and reward manipulation, RVCI values were calculated for each manipulation (Fig. 5D). A one-way ANOVA revealed no significant difference (F_(3,22) = 0.05, p = 0.98). Thus, it appears that the manipulation of reward locations, magnitude, and context did not affect the velocity correlate of PPTg neurons.

TURNING BEHAVIOR

Another type of movement-related neural response was observed as rats made 180-degree turns to return to the center of the maze after consuming a reward at the arm ends. Overall, 30/105 (28.6%) PPTg cells showed turn-related responses (Fig. 6A). Of these cells, 21 cells (70%) were excited during the turn responses, while a relative small proportion of these cells (30%, 9/30) displayed inhibited turn-responses (Fig. 6E). An independent t test showed no difference in average firing rate between turn-excited and -inhibited neurons (t₍₂₈₎ = 0.09, p = 0.93). In addition, among all turn-responsive neurons, 10 cells (33.3%) were velocity-correlated neurons as categorized previously, while a different set of 14 turn neurons (46.7%, 14/30) were reward-responsive (Fig. 6B and C). This suggests that individual PPTg neurons can encode two different types of information simultaneously (e.g., velocity/turns and reward/turn). The right panel of Figure 6A and B shows the spatial distribution of firing for turn-responsive cells. An excited response to turn movement is shown by the concentration of large circles when animals made turns at the arm ends. The effect of the context and reward manipulations on turn-related activity was also evaluated with TACI values (Fig. 6D). A one-way ANOVA showed no significant difference (F_(3,26) = 0.9, p = 0.45), suggesting that egocentric turn-related activity was not affected by the experimental manipulations. As shown in Figure 6B, when a PPTg cell responding to both rewards and turn movement was tested in the darkness session, only reward-related, but not turn-related, activity was altered in the second block. These results suggested that reward and turn responsiveness of PPTg neurons are independently regulated.

Figure 6.

Turn-related activity. A-B, Two examples of turn-related PPTg neurons. Peri-event time histograms (bin width = 50 ms) show a PPTg neuron responding to turn movement (middle, T₀ = turn onset), but not to rewards (left, T₀ = reward acquisition) (A). The red line in each histogram illustrates movement velocity. The spatial plots in the right column show increased activity during turn behaviors, and this did not discriminate specific arms or rewards. The other PPTg neuron exhibited dual-encoding of reward and turn response (B). When the same cell was tested in the darkness session, the reward-related activity was reduced, However, the turn-related activity was not changed by the contextual manipulation, C, Population summary of the proportion of turn-related PPTg neurons. D, Turn activity change index (TACI) for turn-related cells recorded in each manipulation. No significant result was found (p = 0.57). E, Population histograms of turn-excited (upper) and turn-inhibited cells (lower).

Firing characteristics of PPTg neurons

It was of interest to investigate whether the three types of behaviorally correlated PPTg neurons (i.e., reward, velocity, and turn) had distinct electrophysiological characteristics. Therefore, firing properties during the ITI were compared (e.g., average firing rate, spike duration, ISI skew, and CV; Table 1) across categories of behavior correlated neurons. The reward responsive group was further subdivided for analysis into reward-excited and -inhibited neuron groups since their firing rates significantly differed from each other as previously described. A one-way ANOVA showed significant differences across groups in terms of the average firing rate (F_(3,99) = 13.45, p < 0.001), spike duration (F_(3,99) = 5.91, p = 0.001), ISI skew (F_(3,99) = 7.95, p < 0.001), and CV (F_(3,99) = 7.39, p < 0.001). A post hoc analysis (Bonferroni's procedure) demonstrated that the average firing rates of both reward-excited and turn-responsive neurons were significantly lower than those of reward-inhibited and velocity-correlated neurons (p values < 0.01). No differences were found in the other paired comparisons (p values > 0.39). For the spike duration, significant difference was found only between reward-excited and velocity-correlated neurons (p < 0.001) but not the others (p values > 0.24). This indicates that reward-excited cells exhibited wider spike duration than velocity-correlated cells. In the two different measures of ISI distributions, velocity-correlated neurons showed higher ISI skew and CV than reward-excited neurons. In addition, velocity-correlated neurons also showed higher ISI skew than turn-responsive neurons (p = 0.01) and greater CV than reward-inhibited neurons (p = 0.003). No differences were found in the other paired comparisons (p values > 0.07). These results indicate that the ISIs of velocity-correlated cells tended to be positively skewed and dispersed from its mean, compared to the other types of PPTg neurons. The PPTg appears to be comprised of heterogeneous functional and physiological types of neurons.

Table 1.

Basal firing properties of four types of PPTg neurons

Characteristic	Reward-excited	Reward-inhibited	Velocity-correlated	Turn-responsive
Firing rate (spikes/s)	4.24 ± 0.71	24.85 ± 7.23*†	16.52 ± 3.35*†	4.68 ± 1.11
Spike duration (ms)	1.80 ± 0.03	1.68 ± 0.06	1.61 ± 0.04*	1.72 ± 0.03
ISI skew	6.36 ± 0.55	8.04 ± 3.54	15.44 ± 2.32*†	9.13 ± 0.98
Coefficient of variation	2.22 ± 0.11	1.60 ± 0.77	3.62 ± 0.34*#	2.99 ± 0.27

Spontaneous activity was analyzed during intertrial intervals. All values show mean ± SEM.

^*significant difference (p < 0.01) from reward-excited neurons

^†significant difference (p < 0.05) from turn-responsive neurons

^#significant difference (p < 0.01) from reward-inhibited neurons.

Anatomical Distribution of PPTg correlated cells

The previous literature suggests that the anterior (aPPTg) and posterior (pPPTg) portions of PPTg have different anatomical connections with midbrain dopaminergic neurons (Oakman et al., 1995; Joel & Weiner, 2000) as well as different effects on behavior (Alderson et al., 2006; 2008; Wilson et al., 2009). Thus, we further analyzed whether the distributions of reward- and movement-related neurons of PPTg varied across the anterior-posterior axis (from -7.44 to -8.04 mm posterior to bregma) (Fig. 8). Chi-square tests showed a significant difference in the distribution of the cell types in aPPTg (from -7.44 to -7.56 mm; ; χ²₍₃₎ = 19.22, p < 0.001). A further post hoc analysis with a Bonferroni-adjusted alpha demonstrated that reward-excited cells were more densely distributed in the aPPTg than reward-inhibited cells (χ²₍₁₎ = 15.69, p < 0.001) and velocity-correlated cells (χ²₍₁₎ = 7, p = 0.008). Turn-responsive cells also significantly outnumbered reward-inhibited cells in the aPPTg (χ²₍₁₎ = 10.89, p = 0.001). No difference was found in distribution between reward-excited and turn-responsive cells (χ²₍₁₎ = 0.68, p = 0.41). In the further analysis, there was no significant difference in the middle (-7.8 mm; ; χ²₍₃₎ = 5.63, p = 0.13) and posterior portions of the PPTg (from -7.92 to -8.04 mm; ; χ²₍₃₎ = 3.47, p = 0.32).

Discussion

In the current study, we examined PPTg neural activity as rats solved a hippocampal-dependent spatial working memory task that involved retrieving rewards of different magnitudes from known locations. 45% of the PPTg neurons recorded in the study were either excited or inhibited upon reward acquisition. A separate population of PPTg neurons exhibited firing rate correlations with the velocity of movement. There were also a small number of cells that encoded both rewards and a specific type of egocentric movement (i.e. turning behavior). The context-dependency of PPTg reward responses was tested by observing the impact of changes in visuospatial and reward information. Visuospatial, but not reward, manipulations significantly altered PPTg reward-related activity. Movement-related responses, however, were not affected by either type of manipulations. These results suggest that PPTg neurons conjunctively encode both reward and behavioral response information, and that the reward information is processed in a context-dependent manner.

REWARD-RELATED NEURAL ACTIVITY

PPTg codes the presence of rewards

Most PPTg cells recorded here displayed changes in firing rates upon encounter with either large or small rewards, in agreement with the previous literature (Kobayashi et al., 2002; Okada et al., 2009). Excited neural activity to large rewards tended to last longer than responses to small rewards (Fig. 3A and D). 17% of reward-related cells showed inhibited responses to (presumably) initial sensory inputs related to reward encounter, but the period of inhibition was relatively short compared to the duration of reward-excited activity. The PPTg response to unexpected reward omission was consistent with the observation of reward-triggered neural responses since no change in neural activity was observed when a reward was unexpectedly absent. That is, there was no evidence for reward prediction error coding by PPTg neurons.

PPTg reward responses are not related to movements associated with reward consumption

Our finding that a portion of PPTg neurons was correlated with animals’ movement suggests that reward-related activity of PPTg cells resulted from the execution of particular behaviors. For example, cessation of forward movement or orofacial movements associated with reward consumption may have caused PPTg cells to fire to rewards rather than sensory and/or reward qualities (Allen & Winn, 1995). However, some neurons displayed higher activity to small rewards than to large rewards (Fig. 4A) or other cells exhibited elevated excitation in the darkness session before animals acquired rewards (Fig. 4B). Moreover, all reward-responsive cells were not correlated with animals’ velocity. These results suggest that cessation of forward movement or orofacial acts were not the primary determinant of the reward-related responses of PPTg cells.

PPTg reward responses appear sensory based

It has been suggested that PPTg cells differentially respond to the same stimuli depending upon the sensory context in operant conditioning (Dormont et al., 1998; Kobayashi et al., 2002). This finding was extended here to a navigation-based task. We found that the context manipulation often produced a qualitatively different response to reward whereas the reward manipulation did not. In the latter case, reward-related neurons mirrored the different magnitudes of rewards encountered regardless of their locations (Fig. 4C). This suggests that the reward correlate of PPTg cells is sensory based (Pan & Hyland, 2005). Such an hypothesis is also consistent with our finding of altered reward activity in the darkness sessions: darkness-induced changes were likely due to the lack of visuospatial sensory inputs.

Comparison of PPTg and VTA reward-related neural activity

The reward-related properties of PPTg cells are strikingly different from reward-related activity of VTA DA cells in several aspects. First, the latter exhibit short-lasting burst firing (for about 300 ms) after the acquisition of rewards when tested with the same behavioral task in our laboratory (Puryear et al., 2010). In contrast, PPTg cells showed excited activity for at least 1000 ms (Fig. 3D). Second, VTA reward responses reflect the expectations for particular reward magnitudes while PPTg responses reflected sensory-based reward consumption. Third, unlike PPTg cells, VTA DA neurons showed only excited responses to rewards, and some of DA cells that discriminated large and small rewards exhibited selective phasic firing only to the large rewards. PPTg cells were either excited or inhibited in response to rewards, and the reward-excited cells discriminated between large and small rewards in terms of the duration of excitation instead of the existence of phasic activity. Finally, unlike what has been reported for VTA DA neurons, we found no evidence of prediction error coding by PPTg neurons. Thus, the lateral habenula may be the primary determinant of DA error prediction codes (Matsumoto & Hikosaka, 2007).

These differences in reward responsiveness clearly demonstrate that even though VTA DA burst firing is driven by direct PPTg input (Floresco et al., 2003; Zweifel et al., 2009), PPTg is not the only input to VTA that determines reward-responsiveness of DA cells. However, since both PPTg and VTA reward responses were context-sensitive (Puryear et al., 2010), both structures are likely part of a large limbic-striatal network that processes the context-dependent coding of rewards during goal-directed navigation. This network may not encode reward information properly in the absence of PPTg, a condition which results in behavioral deficits in a foraging task (Keating & Winn, 2002).

MOVEMENT-RELATED NEURAL ACTIVITY

PPTg codes general movement state

The PPTg has long been implicated in locomotion and motor control, based on the anatomical connections that PPTg receives inputs from the globus pallidus, substantia nigra, ventral pallidum, and subthalamic nucleus (Moriizumi & Hattori, 1992; Semba & Fibiger, 1992; Groenewegen et al., 1993; Mena-Segovia et al., 2004; Winn, 2006), and its descending outputs to the spinal cord (Rye et al., 1988; Skinner et al., 1990). Although spontaneous locomotor behavior is not affected by bilateral PPTg lesions (Swerdlow & Koob, 1987; Inglis et al., 1994; Keating & Winn, 2002), PPTg lesions produce locomotor changes after the injection of various drugs (Steiniger & Kretschmer, 2004). Furthermore, electrical stimulation of PPTg induces locomotion (Garcia-Rill et al., 1987). In the present study, 24.8% of PPTg cells showed a significant correlation between firing rate and velocity. Context and reward manipulations did not affect these velocity correlations, suggesting that while PPTg processes locomotor information, this correlate does not seem to be related to the spatial context, including reward.

PPTg codes specific behavioral acts

Other cells showed significant changes in firing during acute turns at the ends of maze arms. Also, a subset of reward-related neurons displayed a dual code for both turns and rewards (Fig. 6B). In line with these results, PPTg neurons in primates were activated in respond to both rewards and saccadic eye movement (Kobayashi et al., 2002). Regulation of reward and turn responsiveness appears to come from independent sources since a single experimental manipulation (e.g. darkness condition) changed the reward, but not turn, responses. Current studies are directed toward identifying these different sources of input. Nevertheless it is noteworthy that dual reward and movement codes exist in the PPTg of unconstrained, navigating animals, and this may have relevance to the motor and cognitive deficits in Parkinson's patients (Pahapill & Lozano, 2000; Winn, 2006)

ANATOMICAL DISTRIBUTION OF PPTg REPRESENTATONS

PPTg is characterized by an uneven distribution of distinct populations of cholinergic, glutamatergic, and GABAergic cells (Bevan & Bolam, 1995; Mena-Segovia et al., 2009; Wang & Morales, 2009) and differential projections of its anterior and posterior subdivisions (Oakman et al., 1995). Specifically, pPPTg has more cholinergic and glutamatergic cells relative to aPPTg and projects mostly to VTA, while aPPTg contains proportionately greater GABAergic cells and projects to SN. Four types of functionally categorized PPTg neurons in this study were placed across the anterior and posterior axis of the PPTg, suggesting that DA cells in both VTA and SN can access reward- and movement-related representations processed in the PPTg. However, a subregion-specific bias was found in the proportional distribution of the cell types in aPPTg. In particular, more reward-excited and turn-responsive neurons were recorded than velocity-related and reward-inhibted neurons in aPPTg, a region that projects to SN (Mena-Segovia et al., 2009). Since such a strong disproportional representation was not found in pPPTg mainly projecting to VTA, SN may receive stronger reward and turn signals from PPTg.

Previous studies suggested that cholinergic neurons have broad-spike duration and low firing rate (less than 5 Hz), whereas non-cholinergic neurons show brief-spike duration and higher firing rate (Takakusaki et al., 1997; Koyama et al., 1998). In the comparison of spontaneous firing characteristics of four PPTg cell types (Table 1), reward-excited and turn-responsive neurons exhibited relatively long spike duration and low firing rate, similar to the electrophysiological characteristics of cholinergic neurons. Velocity-related and reward-inhibited neurons, on the other hand, displayed non-cholinergic properties such as short spike duration, higher firing rate. These results further suggest that cholinergic projections from PPTg to midbrain DA cells may contain excitation-based reward information and specific behavioral responses (i.e., turning behavior). However, further studies are required to investigate the relationship between neurotransmitter-specific neuronal types and their behavioral functions.

The PPTg is reciprocally connected with the basal ganglia (Moriizumi & Hattori, 1992; Semba & Fibiger, 1992; Groenewegen et al., 1993) and it relays sensory inputs to midbrain DA neurons (Kobayashi et al., 2002; Pan & Hyland, 2005). In parallel with those anatomical connections, the current study showed that PPTg cells encode reward-related sensory information in a context-dependent manner and movement correlates in a context-independent manner. Further, a small number of PPTg cells simultaneously responded to both reward and a specific type of movement. These findings are consistent with a clinical literature indicating the likely relevance of PPTg pathology to the behavioral manifestations of Parkinson's disease (Pahapill & Lozano, 2000; Winn, 2006). This is a link that is certainly worth further study. It is possible that a loss of the ascending efferents of PPTg to midbrain DA neurons disrupts reinforcement coding in the brain, while loss of the descending connections to the spinal cord may contribute to motor disability in the same patients (Pahapill & Lozano, 2000).

Figure 7.

Distributions of reward- and movement-related PPTg cells along the anterior-posterior axis from -7.44 mm to -8.04 mm to bregma. The four types of behaviorally correlated PPTg cells were distributed throughout all PPTg areas recorded in the current study. However, proportionally more reward-excited cells were found in the anterior part of the PPTg than velocity-correlated and reward-inhibited cells (p values < 0.01). More turn-responsive cells were also recorded in the same subregion than reward-inhibited cell (p = 0.001). These indicate that SN may receive proportionately greater reward excitation and turn signals from the PPTg than does the VTA.